ISIS course

In this course students learn empirical methods used in speech research.

Requirements

Students complete six exercise sheets in groups during the seminar meetings.

Prerequisites

You are not required to have programming skills. However, you should be interested in learning Python. You are also required to run experiments on your own computer or use cloud services.

Format

• Introduction to the seminar and student introductions
• Introduction to Python and Jupyter notebook
• Basic principles and concepts
• On the basis of an analysis of “Berlin Emodb,” students complete the following tasks:
  • Mean value and scatter plots, confidence intervals
  • t-tests for a sample, for independent samples
  • t-tests for dependent samples, correlation
  • Regression and diagnosis
  • Variance analysis

ISIS course

Lecture in the master’s programs Language and Communication, Media Studies and German as a Foreign Language

Module allocation: MA-SK 1, MED 1, DaFF 1

Meeting day and time: Tuesdays, 14-16:00

Requirements: Written exam

ISIS course

Module allocation: MA-SK 3b, MA-MED 7/3

Meeting day and time: Wednesdays, 10-12:00

Requirements: Group presentations (approx. 60 min) and a written report (3-5 pages per author)

Proposed topics:

• Deepfake voices and adversarial attacks
• Age in the voiceAttractive voices
• Eckart/Laver: People and their voices
• Vocal expression in animals
• Music and emotions
• Voice and personality
• Vocal expression of emotions: categories versus dimensions
• The expression of valence in the voiceSpeech in the brain
• Articulatory synthesis and the VocalTractLab
• Forensics: recognizing uncooperative speakers
• Pathological voices: neurodegenerative disorders
• Pathological voices: lisping, stuttering, aphasia
• Speeches by politicians: Merkel, Scharping, Lafontaine…

ISIS course

Module allocation: MA-SK 5b, MA-MED 7/4

Meeting day and time: Tuesdays, 16-18:00

Requirements: Presentation and completed exercises on machine speech classification (each worth 50%)

The seminar focuses on machine speech processing and is composed of a lecture, exercises, and presentations:

• Lectures by the instructor
• Practical exercises using your own laptop
• Student presentations on sub-topics

The exercises are completed in Python using nkululeko.

Students also complete a worksheet for each exercise.

Familiarity with Python is not required. You will be taught the necessary skills. However, you must be willing to install software on your computer. Alternatively, cloud services can be used.

You can prepare for the course by working through this tutorial.

Exam

Multiple choice exam

Content

• Semiotics
• Communication models
• Non-verbal communication
• Personality in voices
• Emotions in voices
• Phonation
• Articulation
• Phonology
• Intonation
• Hearing
• Human computer interfaces
• Machine learning

Literature

Sendlmeier, W. F., 2019. Sprechwirkungsforschung – Grundlagen und Anwendungen mündlicher Kommunikation.3., korrigierte Auflage. Logos Verlag: Berlin.

ISIS

Moses

Analysis of scientific articles on speech communication

• Literature, generally English journal papers

Exam

20-minute presentation on a (sub)topic from the course

Format

• Student introductions, introduction to literature
• Intro talk: Emotional speech synthesis
• Presentations
  • Topics/ literature are provided
    • Students can select their topic or
    • suggest another topic
  • Two presentations with discussion per session

ISIS

Moses

Students give a speech and analyze this as well as learn machine classification.

Requirements

Oral presentation as well as annotation, programming and written analysis of the speech (approx. 5 pages)

Prerequisites

You are not required to have programming skills. However, you should be interested in learning Python. You are also required to run experiments on your own computer or use cloud services.

You can prepare for the course by working through this tutorial.

Format

• Introduction to the seminar and student introductions
• Introduction to and installation of Jupyter notebook
• Introduction to Python
• Introduction to openSmile
• Talk on emotional speaking styles

Tasks

• 5-minute talk on a topic of your choice
• Prepare an audio file for analysis: wav, 16kHz, 16bit, PCM
• Analyze one talk by another student:
• Segment the data and annotate by
  • Category: Nervousness on a 10-point Likert scale
  • Possible tool: Speechalyzer
• Import the segmented data into a pandas table
• Write/apply an analysis script
• Conduct analyses, e.g.:
  • pitch
  • intensity
  • voice quality
  • hnr
  • speechrate
  • pauses
• Present analyses

Submitted coursework

Jupyter notebook with comments

ISIS

Moses

Studierende suchen sich eine*n Sprecher*in aus und analysieren sie*ihn in Referat

Leistung

Referat/Präsentation zuzüglich Annotation, Programmierung und schriftlicher Ausarbeitung (Analyse der Rede, etwa 5 Seiten)

Voraussetzung

Programmierkenntnisse werden nicht vorausgesetzt, aber die Bereitschaft zum Erlernen von Python und das Aufsetzen eigener Experimente auf dem eigenen Rechner oder cloudbasiert wird erwartet.

Zur Vorbereitung kann gerne dieses Tutorial durchgeführt werden.

Durchführung

• Vorstellung der Teilnehmer*innen und des Seminars
• Einführung in Jupyter notebook
• Einführung in Python
• Einführung in openSmile
• Talk: Emotionale Sprechstile

Aufgabe

• Sprachdaten einer*s Zielsprecherin*s auswählen
• Audiodatei zur Analyse aufbereiten: wav, 16kHz, 16bit, PCM
• Sprechstile identifizieren, Kategorien definieren
• Daten segmentieren und annotieren
  • Mögliches tool: Speechalyzer
• Segmentierte Daten in pandas Tabelle importieren
• Analyseskripte schreiben / anwenden
• Analysen durchführen:
  • pitch
  • intensity
  • voice quality
  • hnr
  • speechrate
  • pauses
• Analysen darstellen

Abgabe

Jupyter notebook mit Anmerkungen

Students configure a dialog system from individual components.

Requirements

Oral presentation as well as programming and a written report (code + explanation, approx. 5 pages)

Prerequisites

You are not required to have programming skills. However, you should be interested in learning Python. You are also required to run experiments on your own computer or using cloud services.

You can prepare for the course by working through this tutorial.

Format

• Introduction to the seminar and student introductions
• Introduction to Jupyter notebook
• Introduction to Python
• Introduction to openSmile
• Talk on dialog systems

Tasks

Programming your own dialog system from the following modules:

• Dialog manager
• Input
• Output
• Semantic parsing
• Emotional analysis
• Data integration
• Output formatting

Submitted coursework

Jupyter notebook with comments

This lecture series held by the Chair of Speech Communication takes a closer look at spoken language, human-machine communication, and artificial intelligence. Invited speakers include researchers and leading experts from industry from Germany, England, and Switzerland who will share their know-how with students and other guests.

This course is held online. Start: 17:00 sharp!

Program

Moses