The work was described in an article authored by Rufael Mekuria, Kees Blom, and Pablo César from the Distributed and Interactive Systems (DIS) group at CWI. The award ceremony will take place during the IEEE International Symposium on Circuits and Systems (ISCAS) 2021, the flagship conference of the IEEE Circuits and Systems (CAS) Society.
The paper presents a generic and real-time dynamic point cloud codec for 3D immersive video. This codec is suitable for mixed reality applications in which 3D point clouds are acquired at a fast rate. As a result, a generic compression framework suitable for real-time 3D tele-immersion is developed. This framework has been optimized to run in real time on commodity hardware for both the encoder and decoder. Objective evaluation shows that a higher rate-distortion performance is achieved compared with available point cloud codecs. A subjective study in a state-of-the-art mixed reality system shows that introduced prediction distortions are negligible compared with the original reconstructed point clouds. In addition, it shows the benefit of reconstructed point cloud video as a representation in the 3D virtual world. The codec serves as a base reference software platform in JTC1/SC29/WG11 (MPEG) for the further development of standardized point-cloud compression solutions.
Remote medical consultation
This paper is a groundbreaking work that explores how new 3D media formats, like dynamic point clouds, could be used for real-time remote collaboration, or Social Virtual Reality. Social VR will result in highly immersive and multi-sensory communication systems, enabling remote medical consultation, cultural visits, or teleworking. Apart from the scientific impact, the work is available as open source (MP3DG) for integration in immersive and augmented communication applications. The DIS group has continued this research in a number of different projects such as VRTogether, with the creative industries, and MediaScape, with the cultural sector.
About IEEE CAS
The IEEE Circuits and Systems Society is the leading organization that promotes the advancement of the theory, analysis, design, tools, and implementation of circuits and systems. The field spans their theoretical foundations, applications, and architectures, as well as circuits and systems implementation of algorithms for signal and information processing. The Society brings engineers, researchers, scientists and others involved in circuits and systems applications access to the industry’s most essential technical information, networking opportunities, career development tools and many other exclusive benefits.
About the DIS group
CWI’s Distributed and Interactive Systems research group focuses on facilitating and improving the way people use interactive systems and how people communicate with each other. We combine data science with a strong human-centric, empirical approach to understand the experience of users. This enables us to design and develop next generation intelligent and empathic systems. We base our results on realistic testing grounds and data sets, and embrace areas such as ubiquitous computing, human-centered multimedia systems, and languages.
More information
- Link to paper: R. Mekuria, K. Blom, and P. Cesar Design, Implementation, and Evaluation of a Point Cloud Codec for Tele-Immersive Video. IEEE Transactions on Circuits and Systems for Video Technology, 27(4) , 2017.
- IEEE CAS
- Link to IEEE Transactions on Circuits and Systems for Video Technology
- DIS group
- Link to the thesis of Rufael Mekuria
- Link to open source software package open source (MP3DG)
