Quality and Usability

Dr.-Ing. Stefan Hillmann

Research Field Usability (evaluation) of spoken and multimodal dialogue systems.

Research Topics - dialog management and machine learning - automatic usability prediction - multimodal human computer interaction - modelling and simulation of user behaviour

Research Projects Since December 2018: Optimization of conversational user interface using reinforcement learning. Since April 2013: Employment in the project Universal Home Control Interface@Connected Usability. The project includes research on methods for the usability evaluation of multimodal and adaptive services in smart home environments. Stefan’s main task is the conception and implementation of the dialogue management for an adaptive spoken dialogue system.

January 2010 until July 2011: Employment in the project DFG project User Model. This project was focused on research on the automatic usability evaluation of spoken dialogue systems using methods of sociological technology studies.

Background He did his diploma in computer studies (Diplom-Informatiker) in 2006. Afterwards, he worked in a promotional program for innovative start-ups (hosted at Freie Universität Berlin). From year 2007 until end of 2009 he was employed as software developer at Novedia AG. Since 2010, he is an PhD student at the Quality and Usability Lab in the position of a research assistant. 

Contact E-Mail: stefan.hillmann@tu-berlin.de (encryption: public gpg key)

Teaching activities
Coordination MOOC Communication Acoustics

  • WS 2017/18, WS 2018/19, WS 2019/20, WS 2020/21 (with Tanja Kojic): edX courses Fundamentals of Communication Acoustics and Applications in Communication Acoustics
  • WS 2016/17 (with Christoph Hohnerlein): edX course Communication Acoustics

Seminar Quality and Usability

  • Modellbasierte Usability-Evaluierung: SS 2012 (with Klaus-Peter Engelbrecht), SS 2013 (with Klaus-Peter Engelbrecht, Matthias Schulz), SS 2014 (with Klaus-Peter Engelbrecht, Marc Halbrügge, Matthias Schulz), SS 2015 (with Klaus-Peter Engelbrecht, Marc Halbrügge)
  • Sprachdialogsysteme und Dialogsteuerung: WS 2012/2013, WS 2013/2014, WS 2014/2015 (all with Benjamin Weiss)
  • Neuronale Netze in der Sprachtechnologie: SS 2016 (with Benjamin Weiss, Marc Halbrügge)

Exercise Multimodal Interaction

  • SS 2012 (with Stefan Schaffer, Julia Seebode), SS 2013 (with Stefan Schaffer, Julia Seebode), SS 2014 (with Maija Poikela, Patrick Ehrenbrink), SS 2015 (with Robert Walter, Patrick Ehrenbrink), SS 2016 (with Patrick Ehrenbrink), SS 2017 (with Patrick Ehrenbrink)

Study projects

  • Smart Picture Frame (face recognition): WS 2014/2015
  • Put-That-There (parallel multimodal interaction): WS 2016/2017
  • Visualisierung von Richtungshören und binauralem Hören: SS 2017
  • TU Walking (Fußgängnernavigation an der TU Berlin): SS 2017, WS 2017/18
  • BC Stories (virtual reality environment, bcstories.ca)

Publikationen

2023

Nehring, Jan; Berk, René Marcel; Hillmann, Stefan
Context-Aware Module Selection in Modular Dialog Systems
In Mitkov, Ruslan and Angelova, Galia, Editor, Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, Page 785–791
In Mitkov, Ruslan and Angelova, Galia, Editor
Publisher: INCOMA Ltd., Shoumen, Bulgaria, Varna, Bulgaria
September 2023
Aicher, Annalena; Hillmann, Stefan; Möller, Sebastian; Minker, Wolfgang; Ultes, Stefan
Exploring the mobile Usability of Argumentative Dialogue Systems for Opinion Building
Social Computing and Social Media, Page 127–143
Publisher: Springer, Copenhagen, Denmark
July 2023
ISBN
978-3-031-35921-7
Görzig, Philine; Nehring, Jan; Hillmann, Stefan; Möller, Sebastian
A Comparison of Module Selection Strategies for Modular Dialog Systems
In Christoph Draxler, Editor, Studientexte zur Sprachkommunikation: Elektronische Sprachsignalverarbeitung 2023, Page 40–47
In Christoph Draxler, Editor
Publisher: TUDpress, Dresden
March 2023
ISBN
978-3-95908-303-4
Hillmann, Stefan; Görzig, Philine; Möller, Sebastian
Automatic Generation of Website-Based Multi-Turn Question-Answering Dialog Systems
, Elektronische Sprachsignalverarbeitung 2023Volume105, Page 48–55
Publisher: TUDpress, Dresden
March 2023
ISBN
9787-3-95908-303-4
Aicher, Annalena; Hillmann, Stefan; Feustel, Isabel; Michael, Thilo; Möller, Sebastian; Minker, Wolfgang
Factors in Crowdsourcing for Evaluation of Complex Dialogue Systems
Proceedings of the 13th International Workshop on Spoken Dialogue Systems Technology (IWSDS '23), Page 1–14
Publisher: Springer, Los Angeles, USA
February 2023
Siegert, Ingo; Hillmann, Stefan; Kowol, Philine T.; Busch, Matthias; Nehring, Jan; Klinge, Xenia
Die Chatbot-Challenge – Spielend mit KI von der Idee zum Dialogsystem
INFORMATIK 2023 - Designing Futures: Zukünfte gestalten
Page 377–380
Publisher: Gesellschaft für Informatik e.V., Bonn
2023
377–380
ISBN
978-3-88579-731-9
[en] Aicher, Annalena; Hillmann, Stefan; Michael, Thilo; Möller, Sebastian; Minker, Wolfgang; Ultes, Stefan
Evaluating a Spoken Argumentative Dialogue System
In Coman, Adela and Vasilache, Simona, Editor, Social Computing and Social MediaVolume14026, Page 425–434
In Coman, Adela and Vasilache, Simona, Editor
Publisher: Springer, Copenhagen, Denmark
2023
ISBN
978-3-031-35926-2 978-3-031-35927-9
Rebensburg, Mika; Hillmann, Stefan; Feldhus, Nils
Automatic User Experience Evaluation of Goal-Oriented Dialogs Using Pre-Trained Language Models
, Elektronische Sprachsignalverarbeitung 2023Volume105fromStudientexte zur Sprachkommunikation, Page 24–31
Publisher: TUDpress, Dresden
2023
ISBN
9787-3-95908-303-4

2022

Fernau, Daniel; Hillmann, Stefan; Feldhus, Nils; Polzehl, Tim; Möller, Sebastian
Towards Personality-Aware Chatbots
Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and DialoguefromSigDial, Page 135–145
Publisher: ACL, Stroudsburg PA
September 2022
ISBN
978-1-955917-66-7
Fernau, Daniel; Hillmann, Stefan; Feldhus, Nils; Polzehl, Tim
Towards Automated Dialog Personalization using MBTI Personality Indicators
Interspeech 2022fromInterspeech, Page 1968–1972
Publisher: ISCA, Incheon, Korea
September 2022
ISBN
2958-1796
Vergari, Maurizio; Kojic, Tanja; Koleva, Katerina Georgieva; Hillmann, Stefan; Möller, Sebastian
Towards Improvement of UX Using Gamification for Public Artistic and Historical Artifacts in AR
Virtual, Augmented and Mixed Reality: Applications in Education, Aviation and IndustryfromHCII, Page 145–152
Publisher: Spinger, Cham
June 2022
ISBN
978-3-031-06015-1
Möller, Sebastian; Hillmann, Stefan; Michale, Thilo; Nehring, Jan; Polzehl, Tim
Upcoming new ITU-T recommendation on the evaluation of text-based chatbots
33. Konferenz Elektronische SprachsignalverarbeitungfromESSV, Page 97–104
Publisher: TUDpress, Dresden
March 2022
ISBN
978-3-95908-548-9
Schreiber, Katja; Hillmann, Stefan
Vergleichende Evaluation von zwei Ansätzen für ein Question-Answering System
33. Konferenz Elektronische SprachsignalverarbeitungfromESSV, Page 154–162
Publisher: TUDpress, Dresden
March 2022
ISBN
978-3-95908-548-9

2021

Hillmann, Stefan; Möller, Sebastian; Michael, Thilo
Towards Speech-Based Interactive Post Hoc Explanations in Explainable AI
Workshop Establishing An InterDisciplinary pErspective on Speech-BAsed TEchnology
Publisher: Magdeburg, DE (virtually)
September 2021

2020

Jansson, C. -.; Thottappillil, R.; Hillmann, Stefan; Möller, Sebastian; Hari, K. V. S.; Sundaresan, R.
Experiments in Creating Online Course Content for Signal Processing Education
ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Page 9220–9223
May 2020
Hillmannn, Stefan; Himmelsbach, Tilo; Weiss, Benjamin
Vergleichende Untersuchung von Reinforcement Learning Verfahren beim Training eines Dialog Managers
Elektronische Sprachsignalverarbeitung 2020fromStudientexte zur Sprachkommunikation, Page 249–256
Publisher: TUDpress, Dresden
March 2020
ISBN
978-3-95908-193-1
Weiss, Benjamin; Hillmannn, Stefan; Michael, Thilo; Himmelsbach, Tilo
Zur Automatischen Schätzung von Kostenfunktionen aus Dialogen
Elektronische Sprachsignalverarbeitung 2020fromStudientexte zur Sprachkommunikation, Page 236–240
Publisher: TUDpress, Dresden
March 2020
ISBN
978-3-95908-193-1

2019

Hillmann, Stefan; Engelbrecht, Klaus-Peter; Weiss, Benjamin
Semi-Automatische Generierung Und Reinforcement Learning Basiertes Training Eines Dialogmanagers
Elektronische Sprachsignalverarbeitung 2019, Page 34–41
Publisher: Dresden, Germany
March 2019

2018

Hillmann, Stefan; Weiss, Benjamin; Michael, Thilo; Möller, Sebastian
Simulation von Nutzerverhalten bei der Interaktion mit Sprachdialogsystemen mittels Künstlicher Neuronaler Netzwerke
In Berton, A. and Haiber, U. and Minker, W., Editor, Elektronische Sprachsignalverarbeitung 2018Volume90fromStudientexte zur Sprachkommunikation, Page 151–158
In Berton, A. and Haiber, U. and Minker, W., Editor
Publisher: TUDpress, Dresden
March 2018
ISBN
978-3-95908-128-3
Weiss, Benjamin; Michael, Thilo; Hillmann, Stefan
Kontinuierliche Schätzung von Sprechgeschwindigkeit mit einem Rekurrenten Neuronalen Netzwerk
In Berton, A. and Haiber, U. and Minker, W., Editor, 29 Konferenz Elektronische Sprachsignalverarbeitung (ESSV), UlmfromStudientexte zur Sprachkommunikation, Page 186–191
In Berton, A. and Haiber, U. and Minker, W., Editor
Publisher: TUDpress, Dresden
2018

2017

Weiss, Benjamin; Wechsung, Ina; Hillmann, Stefan; Möller, Sebastian
Multimodal HCI: Exploratory Studies on Effects of First Impression and Single Modality Ratings in Retrospective Evaluation
Journal on Multimodal User Interfaces, 11 (2) :115–131
June 2017
Fullpaper
Publisher: Springer International Publishing
ISSN: 1783-8738
Möller, Sebastian; Ahrens, Jens; Altinsoy, Ercan; Buchschmid, Martin; Fels, Janina; Hillmann, Stefan; Hohnerlein, Christoph; Müller, Gerhard; Seeber, Bernhard; Vorländer, Michael; Weinzierl, Stefan; Knoth, Sebastian; Barodte, Wolfram
A Cross-University Massive Open Online Course on Communication Acoustics
, Acoustics, Proc. of the 173rd Meeting of the Acoustical Society of AmericaVolume8th Forum Acusticum, Page abstract,poster
Publisher: Acoustical Society of America, USA-NY
June 2017
Ehrenbrink, Patrick; Hillmann, Stefan
Comparing Priming Effects of Visual and Textual Task Representations - Texts can Influence Users' Utterances
International Workshop on Spoken Dialogue Systems, Page 1–10
Publisher: Springer International Publishing, Cham, Switzerland
2017
Michael, Thilo; Hillmann, Stefan; Weiss, Benjamin
ALEX: An Artificial Conversational Agent for Students at the TU Berlin
28th Konferenz Elektronische Sprachsignalverarbeitung (ESSV), Page 238–245
Publisher: TUDpress, Dresden
2017
ISBN
978-3-95908-094-1
Hillmann, Stefan
Simulation-Based Usability Evaluation of Spoken and Multimodal Dialogue Systems
from T-Labs Series in Telecommunication Services
Publisher: Springer, Heiderlberg
2017
full
ISBN
978-3-319-62517-1