Video-conferencing via Skype or Google Hangouts has become one of the standard tools for communication in our daily lives. If we want the video-communication systems to react to network fluctuations and provide the best possible quality, it is necessary to understand what factors impact the users experience. In his PhD thesis "Personal Quality of Experience: Accurately modelling Quality of Experience for multi-party desktop video-conferencing based on systems, context and user factors" PhD student Marwin Schmitt of CWI’s <u>DIS group</u> explores how different aspects of user behavior, user state and context can improve modeling the Quality of Experience (QoE) of users.
Studies show that under the same technical conditions, the opinion of users is highly diverse. How we experience the quality of a multimedia system is highly personal. In turn, models which accurately predict the experience of users cannot only rely on the system parameters but must include user and context factors. Models relying on system factors alone often only explain around 30-40% of the variance in the user opinions. However, with a mixed effect model it was shown that the remaining variance is largely not random but can be attributed to systematic patterns of the individual users and the conversation group. This shows that the experience of an individual user can be accurately modeled.
Schmitt found out that the effect of presenting different video-qualities at the same time showed that a contrast effect for quality perception exists: lower video qualities appeared worse the more high-quality video streams were co-present and vice-versa high quality streams appeared better the more low-quality streams were co-present. Furthermore, the effect of delay in small group video conferencing was explored. The findings showed that the QoE of users was affected by their conversational role. Participants who took a more central and active role in the conversation, were stronger affected by the delay in the connection. Schmitt also investigated the relationship between user engagement and video quality. The results showed a clear correlation between engagement and QoE. Thus, users which are more involved in the ongoing task will notice quality degradations less.
During his thesis, Schmit has conducted many experiments, gathered data, observed conversations and discussed with participants about their experiences with video-conferencing. It has become clear that the fluctuating quality provided by over-the-top services, together with the complex system setups, cause troubles for users. As he says: “It is no surprise that the greeting in video-conferencing has moved from “Hello” to “Can you see me?”. In his thesis, he showed that user behavior and individual user factors contribute to a larger part of the QoE ratings than the system factors. The contribution of this thesis on quantifying and modeling them, will help operators to improve the quality of experience of their users. The impact of the thesis goes beyond scientific publications, with active contributions to future standard for multi-party tele-meetings from ITU (International Telecommunication Union) – ITU-T Study group 12 on Performance, QoS and QoE – and an open source testbed that allows researchers and practitioners to run subjective QoE for this type of setups.
CWI’s Distributed and Interactive Systems research group focuses on facilitating and improving the way people access media and communicate with others and with the environment. They address key problems for society and science, resulting from the dense connectivity of content, people, and devices. The group uses recognized scientific methods, following a full-stack, experimental, and human-centered approach.
More information
Everyone is welcome to attend the public defence of Marwin Schmitt of his thesis 'Personal Quality of Experience: Accurately modelling Quality of Experience for multi-party desktop video-conferencing based on systems, context and user factors'.
10 September 9.45-11.00 PM at <u>VU University Amsterdam</u>
promotor: prof. dr. D.C.A. Bulterman (CWI/VU)
co-promotors: dr. P.S. Cesar (CWI/TU Delft), dr. J.A. Redi (TU Delft)
Relevant Publications:
- M. Schmitt, D.C.A. Bulterman, and P. Cesar, "The Contrast Effect: QoE of Mixed Video-Qualities at the same Time," Springer Quality and User Experience, 3(1): article 7, 2018. https://link.springer.com/article/10.1007/s41233-018-0020-2
- M. Schmitt, J. Redi, D.C.A. Bulterman, and P. Cesar, "Towards individual QoE for multi-party video conferencing," IEEE Transactions on Multimedia (TMM), 20(7):1781-1795, 2018. https://ieeexplore.ieee.org/document/8119925
- M. Schmitt, S. Gunkel, P. Cesar, and D.C.A. Bulterman, "Asymmetric Delay in Video-Mediated Group Discussions," in Proceedings of the International Workshop on Quality of Multimedia Experience, (QoMEX2014), Singapore, Singapore, September 18-20, 2014. https://ieeexplore.ieee.org/document/6982280
Contributed ITU Recommendation:
Open Source:
QoE Testbed: Performing quality studies on video-conferencing systems
The QoE Testbed is a set of tools for conducting Quality of Experience (QoE) studies for video-conferencing. It can be used to produce standardized results from experiments as indicated by ITU Study Group 12 on Performance, QoS and QoE. For example, it can be used to evaluate adaptation techniques for palliating performance fluctuations on the internet.
