BAS
Bavarian Archive for Speech Signals
Multi-modal Corpora

Gleiche Seite in deutsch

This page was last updated 2014-07-10

Aside from the audio signal multi-modal coprora contain additional channels (modalities) of communication, eg. video signal of the face, the body, etc.

• SmartKom
• SmartWeb SVC
• German Sign Language Corpus SIGNUM
• Bielefeld Speech and Gesture Alignment Corpus SaGA

SmartKom (Revalidation Report)

• SmartKom Public - SKP 2.2
  96 Speakers - 172 Recording Sessions - Scenario: Public - Domains: 'Cinema', 'Fax' (primary domains) and 'Restaurant', 'Telephone', 'Email' (secondary domains)
• SmartKom Mobil - SKM 1.2
  73 Speakers - 146 Recording Sessions - Scenario: Mobil - Domains: 'Tourism', 'Navigation' (primary domains) and 'Telephony', 'Searching for a parking lot' (secondary domains)
• SmartKom Home - SKH 1.1
  65 Speakers - 130 Recording Sessions - Scenario: Home - Domains: 'TV', 'VCR' (primary domains) and 'Scheduling', 'Jukebox' (secondary domains)
• SmartKom Audio - SKAUDIO 1.1
  Special edition containing all audio channel recording of the SmartKom corpora - 224 speakers - 448 sessions - Scenarios: Public, Home, Mobil

Example of a SmartKom Public Recording (AVI, 320MB)
ww406_pxg; SK Public; g-track: face, lateral, system output + SIVIT, SIVIT (infrared); domain: hotel, fax, phone, restaurant, navigation.
Transkript w406_px.trl
Recording Protocol w406_px.rpr
Speaker Protocol AJK.spr

Download G track videos of all SmartKom recordings

Selected technical documents:

• TechDok-NR-26 : Interlabeler-Agreement in SmartKom Multi-Modal Annotations.

References of corpus production:

• Beringer N, Kartal U, Louka K, Schiel F, Türk U (2002): PROMISE: A Procedure for Multimodal Interactive System Evaluation. Proceedings of the Workshop 'Multimodal Resources and Multimodal Systems Evaluation' 2002, Las Palmas, Gran Canaria, Spain, pp. 77-80.
• Schiel F, Steininger S, Türk U (2002):The SmartKom Multimodal Corpus at BAS. Proceedings of the 3rd Language Resources & Evaluation Conference (LREC) 2002, Las Palmas, Gran Canaria, Spain, pp. 200-206.
• Schiel F, Steininger S, Beringer N, Türk U, Rabold S (2002): Integration of multi-modal data and annotations into a simple extendable form: the extension of the BAS Partitur Format. Proceedings of the Workshop 'Multimodal Resources and Multimodal Systems Evaluation' 2002, Las Palmas, Gran Canaria, Spain, pp. 39-44.
• Steininger S, Schiel F, Dioubina O, Rabold S (2002): Development of User-State Conventions for the Multimodal Corpus in SmartKom. In: Proceedings of the Workshop 'Multimodal Resources and Multimodal Systems Evaluation' 2002, Las Palmas, Gran Canaria, Spain, pp 33-37.
• Steininger S, Schiel F, Glesner A (2002): User-State Labeling Procedures for the Multi-modal Data Collection of SmartKom. Proceedings of the 3rd Language Resources & Evaluation Conference (LREC) 2002, Las Palmas, Gran Canaria, Spain, pp. 371-377.

References of Corpus Users:

• Vogt Th, Andre E (2005): Comparing featu re sets for acted and spontaneous speech in view of automatic emotion recognition. Proceedings of ICME 2005.
• Vogt Th, Andre E (2006): Improving automatic emotion recognition from Speech via gender differentiation. Proceedings of LREC 2006.

• SmartWeb Video Corpus - SVC
  2218 human-machine queries in a human-human-machine situation using a smart phone, video face-capture of asking person - 99 speakers - natural environment, 2 mics (collar, Bluetooth), UMTS + high quality channel - transliteration (Verbmobil compatible), manual turn segmentation, BAS Partitur Format

  Video examples

  Recording i097.mpg Male age 32, indoor, Blue tooth Headset UMTS
  Transcript i097.trl
  Recording Protocol i097.rpr
  Speaker Protocol AJAW.spr

  Recording i100.mpg Femail age 25 with glasse s, indoor, Bluetooth Headset UMTS
  Transcript i100.trl
  Recording Protocol i100.rpr
  Speaker Protocol APDW.spr

Sign Language

• German Sign Language Corpus - SIGNUM
  55h video of basic German sign language performed by 25 signers.

  Example of a SIGNUM video recording (Flash, 1MB)

Gestures

• Bielefeld Speech and Gesture Alignment Corpus (SaGA)
  1h Video Recordings + Gesture Annotation/Transcription of 6 Speakers.
• SmartKom