Quality and Usability

Dr. -Ing. Babak Naderi

Research Interests:

  • Subjective quality assessment
  • Speech Quality Assessment in Crowdsourcing
  • Motivation, Workload, and Performance in Crowdsourcing
  • Statistical Modeling, field data and applied statistics
  • Speech Enhancement
  • Text Complexity and Simplification

Biography:

Babak Naderi has obtain his Dr.-Ing degree (PhD) on the basis of his thesis with a title of Motivation of Workers on Microtask Crowdsourcing Platforms in September 2017. Babak has Master's degree in Geodesy and Geoinformation Science form the Technical University Berlin with a thesis on "Monte Carlo Localization for Pedestrian Indoor Navigation Using a Map Aided Movement Model". He has also a Bachelor's degree in Software Engineering.

Since August 2012, Babak Naderi is working as a research scientist at the Quality and Usability Lab of  TU-Berlin.

2013-2015 Babak was awarded with an BMBF funded Education program for future IT and Development Leadership involving Bosch, Datev, Deutsche Telekom AG, Holtzbrinck, SAP, Scheer Group, Siemens, and Software AG  amongst highly ranked academic institution (Softwarecampus). He was taking part by leading CrowdMAQA project.

Within dissertation, Babak studies the motivation of crowdworkers in details. He has developed the Crowdwork Motivation Scale for measuring general motivation based on the Self-Determination Theory of Motivation. The scale has been validated within several studies. In addition, he has studied factors influencing the motivation, and influence of different motivation type on the quality of outcomes. Models for predicting task selection strategy of workers are developed, including models for automatically predicting expected workload associated to a task from its design, task acceptance and performance. 

Beside others research activities, Babak is actively working on the standardization of methods for speech quality assessment in crowdsourcing environment in the P.CROWD work program of Study Group 12 in ITU-T Standardization Sector.

Reviewed for WWW, CHI, ICASSP, CSCW, MMSys, PQS, HCOMP, ICWE, QoMEX, International Journal of Human-Computer Studies, Computer Networks, Behaviour & Information Technology, Quality and User Experience.

 

Selected talks:

  • "Motivation of Crowd Workers, does it matter?",Schloss Dagstuhl, Evaluation in the Crowd: Crowdsourcing and Human-Centred Experiments, November 2015.
  • "Motivation and Quality Assessment in Online Paid Crowdsourcing Micro-task Platforms",Schloss Dagstuhl, Crowdsourcing: From Theory to Practice and Long-Term Perspectives, September 2013.

 

Office Hours: On Appointment

 

Adresse:

Quality and Usability Lab

Technische Universität Berlin
Ernst-Reuter-Platz 7
D-10587 Berlin

babak.naderi[at]tu-berlin.de

Publikationen

2021

Mittag, Gabriel; Zadtootaghaj, Saman; Michael, Thilo; Naderi, Babak; Möller, Sebastian
Bias-Aware Loss for Training Image and Speech Quality Prediction Models from Multiple Datasets
13th International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX, Page 97–102
Publisher: IEEE
2021
Naderi, Babak; Möller, Sebastian
Draft update for Recommendation P.808 on Subjective evaluation of speech quality with a crowdsourcing approach
from ITU-T Contribution
CH-Geneva
2021
Naderi, Babak; Mittag, Gabriel; Jiménez, Rafael Zequeira; Möller, Sebastian
Effect of Language Proficiency on Subjective Evaluation of Noise Suppression Algorithms
ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)fromICASSP, Page 4140–4144
Publisher: IEEE
2021
Naderi, Babak; Mittag, Gabriel; Jiménez, Rafael Zequeira; Möller, Sebastian
Effect of Language Proficiency on Subjective Evaluation of Noise Suppression Algorithms
from ITU-T Contribution
CH-Geneva
2021
Naderi, Babak; Möller, Sebastian
Evaluation of the Comparison Category Rating Test Method for Speech Quality Assessment in Crowdsourcing
from ITU-T Contribution
CH-Geneva
2021
Rallabandi, Sai Sirisha; Naderi, Babak; Möller, Sebastian
Identifying the vocal cues of likeability, friendliness and skilfulness in synthetic speech
Proceedings of Speech Synthesis Workshop (SSW11)from11th ISCA Speech Synthesis Workshop, Page 1–6
2021
Jiménez, Rafael Zequeira; Naderi, Babak; Möller, Sebastian
Influence of Language Differences in Crowdsourcing Speech Quality Assessment Studies
13th International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX, Page 43–48
Publisher: IEEE
2021
Hoßfeld, Tobias; Seufert, Michael; Naderi, Babak
On Inter-Rater Reliability for Crowdsourced QoE
13th International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX, Page 37–42
Publisher: IEEE
2021
Naderi, Babak; Möller, Sebastian; Reimes, Jan
On the necessity of updating the recommended signal distortion simulating speech enhancement degradations in ITU-T Rec. P.835
from ITU-T Contribution
CH-Geneva
2021
Rallabandi, Sai Sirisha; Bharadwaj, Abhinav; Naderi, Babak; Möller, Sebastian
Perception of social speaker characteristics in synthetic speech
Proceedings of Interspeech 2021 – Speech everywhere!fromInterspeech, Page 2771–2775
2021
Naderi, Babak; Möller, Sebastian; Cutler, Ross
Speech Quality Assessment in Crowdsourcing: Comparison Category Rating Method
13th International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX, Page 31–36
Publisher: IEEE
2021
Naderi, Babak; Möller, Sebastian
Crowdsourcing approach for Subjective Evaluation of Noise suppression Algorithms
from ITU-T Contribution
CH-Geneva
2021

2020

Naderi, Babak; Hoßfeld, Tobias; Hirth, Matthias; Metzger, Florian; Möller, Sebastian; Zequeira Jiménez, Rafael
Impact of the Number of Votes on the Reliability and Validity of Subjective Speech Quality Assessment in the Crowdsourcing Approach
12th International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX, Page 1–6
Publisher: IEEE
2020
ISBN
978-1-7281-5965-2
Naderi, Babak; Möller, Sebastian
Transformation of Mean Opinion Scores to Avoid Misleading of Ranked based Statistical Techniques
12th International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX, Page 1–3
Publisher: IEEE
2020
ISBN
978-1-7281-5965-2
Naderi, Babak; Jiménez, Rafael Zequeira; Hirth, Matthias; Möller, Sebastian; Metzger, Florian; Hoßfeld, Tobias
Towards speech quality assessment using a crowdsourcing approach: evaluation of standardized methods
Quality and User Experience, 6 (1) :2
2020
Fullpaper
Publisher: Springer
ISSN: 2366-0139
Perkis, Andrew; Timmerer, Christian; Baraković, Sabina; Husić, Jasmina Baraković; Bech, Søren; Bosse, Sebastian; Botev, Jean; Brunnström, Kjell; Cruz, Luis; De Moor, Katrien; Saibanti, Andrea de Polo; Durnez, Wouter; Egger-Lampl, Sebastian; Engelke, Ulrich; Falk, Tiago H.; Hameed, Asim; Hines, Andrew; Kojic, Tanja; Kukolj, Dragan; Liotou, Eirini; Milovanovic, Dragorad; Möller, Sebastian; Murray, Niall; Naderi, Babak; Pereira, Manuela; Perry, Stuart; Pinheiro, Antonio; Pinilla, Andres; Raake, Alexander; Agrawal, Sarvesh Rajesh; Reiter, Ulrich; Rodrigues, Rafael; Schatz, Raimund; Schelkens, Peter; Schmidt, Steven; Sabet, Saeed Shafiee; Singla, Ashutosh; Skorin-Kapov, Lea; Suznjevic, Mirko; Uhrig, Stefan; Vlahović, Sara; Voigt-Antons, Jan-Niklas; Zadtootaghaj, Saman
QUALINET White Paper on Definitions of Immersive Media Experience (IMEx)
arXiv preprint arXiv:2007.07032 :1–15
2020
online
Publisher: European Network on Quality of Experience in Multimedia Systems and Services, 14th QUALINET meeting
Sabet, Saeed Shafiee; Schmidt, Steven; Zadtootaghaj, Saman; Naderi, Babak; Griwodz, Carsten; Möller, Sebastian
A Latency Compensation Technique Based on Game Characteristics to Mitigate the Influence of Delay on Cloud Gaming Quality of Experience
Proceedings of the 11th ACM Multimedia Systems ConferencefromMMSys ’20, Page 15–25
Publisher: Association for Computing Machinery, New York, NY, USA
2020
ISBN
9781450368452
Schmidt, Steven; Shafiee Sabet, Saeed; Naderi, Babak; Zadtootaghaj, Saman; Griwodz, Carsten; Möller, Sebastian
Evaluation of Interactive Test Paradigm for P.CROWDG Recommendation
from ITU-T Contribution
ITU, CH-Geneva
2020
Schmidt, Steven; Naderi, Babak; Sabet, Saeed Shafiee; Zadtootaghaj, Saman; Carsten ; Möller, Sebastian
Assessing Interactive Gaming Quality of Experience using a Crowdsourcing Approach
2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX ’20, Page 1–6
Publisher: IEEE
2020
ISBN
978-1-7281-5965-2
Naderi, Babak; Möller, Sebastian
Application of Just-Noticeable Difference in Quality as Environment Suitability Test for Crowdsourcing Speech Quality Assessment Task
12th International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX, Page 1–6
Publisher: IEEE
2020
ISBN
978-1-7281-5965-2
Naderi, Babak; Cutler, Ross
An Open Source Implementation of ITU-T Recommendation P.808 with Validation
Proc. Interspeech 2020fromInterspeech, Page 2862–2866
Publisher: ISCA, ISCA
2020

2019

Naderi, Babak; Mohtaj, Salar; Ensikat, Kaspar; Möller, Sebastian
Subjective Assessment of Text Complexity: A Dataset for German Language
CoRR
2019
Zequeira Jiménez, Rafael; Llagostera, Anna; Naderi, Babak; Möller, Sebastian; Berger, Jens
Modeling Worker Performance Based on Intra-rater Reliability in Crowdsourcing : A Case Study of Speech Quality Assessment
2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX)fromQoMEX 2019, Page 1–6
Publisher: IEEE
2019
Zequeira Jiménez, Rafael; Llagostera, Anna; Naderi, Babak; Möller, Sebastian; Berger, Jens
Intra- and Inter-rater Agreement in a Subjective Speech Quality Assessment Task in Crowdsourcing
Companion Proceedings of The 2019 World Wide Web ConferencefromWWW '19, Page 1138–1143
Publisher: ACM, New York, NY, USA
2019
ISBN
978-1-4503-6675-5
Zequeira Jiménez, Rafael; Naderi, Babak; Möller, Sebastian
Evaluating Acoustic Features from Environmental Audio Recordings via Web. A Crowdsourcing Survey on Background Noise Characteristics
45. Deutsche Jahrestagung für Akustik (DAGA 2019), Page 1190–1193
Publisher: Deutsche Gesellschaft für Akustik DEGA e.V.
2019
ISBN
978-3-939296-14-0

Page 1 of 3