MultiCo

Stały URI dla kolekcjihttps://hdl.handle.net/11321/945

The MultiCo multimodal corpus is one of the outcomes of the project "Digital Research Infrastructure for the Humanities and Arts Studies DARIAH-PL." This project was funded by POIR 4.2 of the European Regional Development Fund from 2021 to 2023 and was carried out by a consortium of academic institutions across Poland with Adam Mickiewicz University, Poznan as a member of the consortium. The MultiCo multimodal corpus was developed at the Faculty of Modern Languages of Adam Mickiewicz University in Poznań. The motivation behind creating the corpus stems from contemporary research on interpersonal communication. The studies confirm that in order to understand and model the multifaceted process of communication, it's essential to study and describe not only speech but also other components of communication, such as gestures, facial expressions, and body posture. The MultiCo corpus was designed to support and facilitate this type of research approach. The corpus contains over 15 hours of recordings and consists of three sections: - Monologs representing persuasion in parliamentary speeches and motivational talks (TEDex), - Dialogs based on task-oriented activities recorded in a lab setting, - Multilogs illustrating discussions with multiple participants, exemplified by conversations on current sports events (TVP Sport 4-4-2). The monolog and multilog sections are based on materials available in public media or archives, while the dialog section includes task-oriented dialogs originally designed and recorded specifically for this resource.
Szukaj

Przeglądaj

Ostatnie zgłoszenia

Teraz wyświetlane 1 - 1 z 1
  • Miniatura
    Item type: Pozycja ,
    MultiCo-Hub: a corpus of multimodal enrichments with motion-trajectory annotation
    (Adam Mickiewicz University, Poznan, 2025) Klessa, Katarzyna; Karpiński, Maciej; Jarmołowicz-Nowikow, Ewa; Sawicka-Stępińska, Brygida; Klessa, Wojciech
    MultiCo-Hub is a multimodal dataset including 11 zipped subsets (henceforth: sessions) of time-aligned audio, video and motion-capture–derived BVH data, together with multi-layered Annotation Pro files (ANTx) extended with automatically extracted motion-trajectory layers. The dataset includes a dedicated training session demonstrating body movement (TESM_001). The video and audio files included in remaining 10 sessions are derived from the MultiCo corpus (http://hdl.handle.net/11321/942). The original MultiCo sessions were enriched by means of: - full audio, video and BVH streams synchronization to enhance precise multimodal analysis; - motion-capture (BVH) data normalization, conversion, and integration directly into the annotation files as layers describing trajectories of selected body parts (positions, speeds, gesture-space coordinates). Furthermore, for each session, the corpus provides a composite multi-view video file showing all four camera angles simultaneously. This makes the dataset easier to inspect and substantially more accessible for users working on standard-performance computers. MultiCo-Hub offers a compact, ready-to-use resource for research and education in the areas of speech–gesture coordination, gesture space, temporal properties of movement, communicative alignment of interlocutors, and multimodal interaction. Export to common formats (TextGrid, EAF, CSV, etc.) is supported via Annotation Pro, facilitating downstream statistical analysis, visualization and interoperability. The MultiCo-Hub set also served as input for developing a set of R and C# applications and scripts that support the analysis and visualization of gesture space, temporal movement properties, and communicative alignment in dialogue.