Quality and Usability

Dr. -Ing. Babak Naderi

Research Interests:

  • Subjective quality assessment
  • Speech Quality Assessment in Crowdsourcing
  • Motivation, Workload, and Performance in Crowdsourcing
  • Statistical Modeling, field data and applied statistics
  • Speech Enhancement
  • Text Complexity and Simplification

Biography:

Babak Naderi has obtain his Dr.-Ing degree (PhD) on the basis of his thesis with a title of Motivation of Workers on Microtask Crowdsourcing Platforms in September 2017. Babak has Master's degree in Geodesy and Geoinformation Science form the Technical University Berlin with a thesis on "Monte Carlo Localization for Pedestrian Indoor Navigation Using a Map Aided Movement Model". He has also a Bachelor's degree in Software Engineering.

Since August 2012, Babak Naderi is working as a research scientist at the Quality and Usability Lab of  TU-Berlin.

2013-2015 Babak was awarded with an BMBF funded Education program for future IT and Development Leadership involving Bosch, Datev, Deutsche Telekom AG, Holtzbrinck, SAP, Scheer Group, Siemens, and Software AG  amongst highly ranked academic institution (Softwarecampus). He was taking part by leading CrowdMAQA project.

Within dissertation, Babak studies the motivation of crowdworkers in details. He has developed the Crowdwork Motivation Scale for measuring general motivation based on the Self-Determination Theory of Motivation. The scale has been validated within several studies. In addition, he has studied factors influencing the motivation, and influence of different motivation type on the quality of outcomes. Models for predicting task selection strategy of workers are developed, including models for automatically predicting expected workload associated to a task from its design, task acceptance and performance. 

Beside others research activities, Babak is actively working on the standardization of methods for speech quality assessment in crowdsourcing environment in the P.CROWD work program of Study Group 12 in ITU-T Standardization Sector.

Reviewed for WWW, CHI, ICASSP, CSCW, MMSys, PQS, HCOMP, ICWE, QoMEX, International Journal of Human-Computer Studies, Computer Networks, Behaviour & Information Technology, Quality and User Experience.

 

Selected talks:

  • "Motivation of Crowd Workers, does it matter?",Schloss Dagstuhl, Evaluation in the Crowd: Crowdsourcing and Human-Centred Experiments, November 2015.
  • "Motivation and Quality Assessment in Online Paid Crowdsourcing Micro-task Platforms",Schloss Dagstuhl, Crowdsourcing: From Theory to Practice and Long-Term Perspectives, September 2013.

 

Office Hours: On Appointment

 

Adresse:

Quality and Usability Lab

Technische Universität Berlin
Ernst-Reuter-Platz 7
D-10587 Berlin

babak.naderi[at]tu-berlin.de

Publikationen

2022

Mohtaj, Salar; Naderi, Babak; Möller, Sebastian
Overview of the GermEval 2022 Shared Task on Text Complexity Assessment of German Text
Proceedings of the GermEval 2022 Workshop on Text Complexity Assessment of German Text, GermEval@KONVENS 2022, Potsdam, Germany, September 12, 2022fromLecture Notes in Computer Science, Page 1–9
Publisher: Association for Computational Linguistics, Stroudsburg, USA
September 2022

2021

Naderi, Babak; Möller, Sebastian
Crowdsourcing approach for Subjective Evaluation of Noise suppression Algorithms
from ITU-T Contribution
CH-Geneva
May 2021
Naderi, Babak; Möller, Sebastian
Draft update for Recommendation P.808 on Subjective evaluation of speech quality with a crowdsourcing approach
from ITU-T Contribution
CH-Geneva
May 2021
Naderi, Babak; Mittag, Gabriel; Jiménez, Rafael Zequeira; Möller, Sebastian
Effect of Language Proficiency on Subjective Evaluation of Noise Suppression Algorithms
from ITU-T Contribution
CH-Geneva
May 2021
Naderi, Babak; Möller, Sebastian
Evaluation of the Comparison Category Rating Test Method for Speech Quality Assessment in Crowdsourcing
from ITU-T Contribution
CH-Geneva
May 2021
Naderi, Babak; Möller, Sebastian; Reimes, Jan
On the necessity of updating the recommended signal distortion simulating speech enhancement degradations in ITU-T Rec. P.835
from ITU-T Contribution
CH-Geneva
May 2021
Naderi, Babak; Mittag, Gabriel; Jiménez, Rafael Zequeira; Möller, Sebastian
Effect of Language Proficiency on Subjective Evaluation of Noise Suppression Algorithms
ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)fromICASSP, Page 4140–4144
Publisher: IEEE
2021
Rallabandi, Sai Sirisha; Naderi, Babak; Möller, Sebastian
Identifying the vocal cues of likeability, friendliness and skilfulness in synthetic speech
Proceedings of Speech Synthesis Workshop (SSW11)from11th ISCA Speech Synthesis Workshop, Page 1–6
2021
Jiménez, Rafael Zequeira; Naderi, Babak; Möller, Sebastian
Influence of Language Differences in Crowdsourcing Speech Quality Assessment Studies
13th International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX, Page 43–48
Publisher: IEEE
2021
Hoßfeld, Tobias; Seufert, Michael; Naderi, Babak
On Inter-Rater Reliability for Crowdsourced QoE
13th International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX, Page 37–42
Publisher: IEEE
2021
Rallabandi, Sai Sirisha; Bharadwaj, Abhinav; Naderi, Babak; Möller, Sebastian
Perception of social speaker characteristics in synthetic speech
Proceedings of Interspeech 2021 – Speech everywhere!fromInterspeech, Page 2771–2775
2021
Naderi, Babak; Möller, Sebastian; Cutler, Ross
Speech Quality Assessment in Crowdsourcing: Comparison Category Rating Method
13th International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX, Page 31–36
Publisher: IEEE
2021
Mittag, Gabriel; Zadtootaghaj, Saman; Michael, Thilo; Naderi, Babak; Möller, Sebastian
Bias-Aware Loss for Training Image and Speech Quality Prediction Models from Multiple Datasets
13th International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX, Page 97–102
Publisher: IEEE
2021

2020

Naderi, Babak; Jiménez, Rafael Zequeira; Hirth, Matthias; Möller, Sebastian; Metzger, Florian; Hoßfeld, Tobias
Towards speech quality assessment using a crowdsourcing approach: evaluation of standardized methods
Quality and User Experience, 6 (1) :2
November 2020
Fullpaper
Publisher: Springer
ISSN: 2366-0139
Naderi, Babak; Cutler, Ross
An Open Source Implementation of ITU-T Recommendation P.808 with Validation
Proc. Interspeech 2020fromInterspeech, Page 2862–2866
Publisher: ISCA, ISCA
October 2020
Sabet, Saeed Shafiee; Schmidt, Steven; Zadtootaghaj, Saman; Naderi, Babak; Griwodz, Carsten; Möller, Sebastian
A Latency Compensation Technique Based on Game Characteristics to Mitigate the Influence of Delay on Cloud Gaming Quality of Experience
Proceedings of the 11th ACM Multimedia Systems ConferencefromMMSys ’20, Page 15–25
Publisher: Association for Computing Machinery, New York, NY, USA
June 2020
ISBN
9781450368452
Naderi, Babak; Möller, Sebastian
Application of Just-Noticeable Difference in Quality as Environment Suitability Test for Crowdsourcing Speech Quality Assessment Task
12th International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX, Page 1–6
Publisher: IEEE
May 2020
ISBN
978-1-7281-5965-2
Schmidt, Steven; Naderi, Babak; Sabet, Saeed Shafiee; Zadtootaghaj, Saman; Carsten ; Möller, Sebastian
Assessing Interactive Gaming Quality of Experience using a Crowdsourcing Approach
2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX ’20, Page 1–6
Publisher: IEEE
May 2020
ISBN
978-1-7281-5965-2
Naderi, Babak; Hoßfeld, Tobias; Hirth, Matthias; Metzger, Florian; Möller, Sebastian; Zequeira Jiménez, Rafael
Impact of the Number of Votes on the Reliability and Validity of Subjective Speech Quality Assessment in the Crowdsourcing Approach
12th International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX, Page 1–6
Publisher: IEEE
May 2020
ISBN
978-1-7281-5965-2
Perkis, Andrew; Timmerer, Christian; Baraković, Sabina; Husić, Jasmina Baraković; Bech, Søren; Bosse, Sebastian; Botev, Jean; Brunnström, Kjell; Cruz, Luis; De Moor, Katrien; Saibanti, Andrea de Polo; Durnez, Wouter; Egger-Lampl, Sebastian; Engelke, Ulrich; Falk, Tiago H.; Hameed, Asim; Hines, Andrew; Kojic, Tanja; Kukolj, Dragan; Liotou, Eirini; Milovanovic, Dragorad; Möller, Sebastian; Murray, Niall; Naderi, Babak; Pereira, Manuela; Perry, Stuart; Pinheiro, Antonio; Pinilla, Andres; Raake, Alexander; Agrawal, Sarvesh Rajesh; Reiter, Ulrich; Rodrigues, Rafael; Schatz, Raimund; Schelkens, Peter; Schmidt, Steven; Sabet, Saeed Shafiee; Singla, Ashutosh; Skorin-Kapov, Lea; Suznjevic, Mirko; Uhrig, Stefan; Vlahović, Sara; Voigt-Antons, Jan-Niklas; Zadtootaghaj, Saman
QUALINET White Paper on Definitions of Immersive Media Experience (IMEx)
arXiv preprint arXiv:2007.07032 :1–15
May 2020
online
Publisher: European Network on Quality of Experience in Multimedia Systems and Services, 14th QUALINET meeting
Naderi, Babak; Möller, Sebastian
Transformation of Mean Opinion Scores to Avoid Misleading of Ranked based Statistical Techniques
12th International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX, Page 1–3
Publisher: IEEE
May 2020
ISBN
978-1-7281-5965-2
Schmidt, Steven; Shafiee Sabet, Saeed; Naderi, Babak; Zadtootaghaj, Saman; Griwodz, Carsten; Möller, Sebastian
Evaluation of Interactive Test Paradigm for P.CROWDG Recommendation
from ITU-T Contribution
ITU, CH-Geneva
April 2020

2019

Naderi, Babak; Schmidt, Steven; Zadtootaghaj, Saman; Möller, Sebastian
Draft text for P.CROWDG Recommendation Subjective Evaluation of Gaming Quality with a Crowdsourcing Approach
from ITU-T Contribution
ITU, CH-Geneva
November 2019
Mohtaj, Salar; Naderi, Babak; Ensikat, Kaspar; Möller, Sebastian
A Dataset for Subjective Assessment of German Text Complexity
Dialog for Good (DiGo), Workshop on Speech and Language Technology Serving Society, Stockholm, 10 September, 2019., Page 1–6
September 2019
Naderi, Babak; Mohtaj, Salar; Karan, Karan; Möller, Sebastian
Automated Text Readability Assessment for German Language: A Quality of Experience Approach
11th International Conference on Quality of Multimedia Experience (QoMEX 2019)fromQoMEX
Publisher: IEEE
June 2019