As one of the sponsors of WIAMIS 2012 (13th International Workshop on Image Analysis for Multimedia Interactive Services, 23rd-25th May 2012) at Dublin City University EMC2organised a special panel session on the theme, "What 3D Industry Needs from Research". The aim of the session was to consider and discuss some of the commercial needs and opportunities in the 3D media/ communication field, and the research challenges that must be overcome to enable new market opportunities to be exploited.
The Special Panel Session at WIAMIS 2012
The expert panellists were (left-right order in photo below): Benoit Michel from Belgium, Editor of Stereoscopy News; Maryline Clare-Charrier of 3DLIVE project, France Telecom R&D Orange Labs; Gavin Duffy, Director of RealSim Ltd., Galway, Ireland; and François Pitié of Sigmedia Group, Trinity College Dublin. The panel was chaired by George Whale of EMC2 (far right).
The four panellists
Each panellist presented a thought-provoking 10-minute talk on research challenges in 3D, and the four presentations were followed by lively audience Q&A.
François Pitié – Research for 3D movie post-production
François Pitié of Trinity College’s Sigmedia group gave a fascinating short talk on movie post-production, showing some of the many software techniques that can be applied to 2D and 3D film and video, e.g. to remove noise, flicker and camera shake, or to manage colour, focus and depth.
He described some of the visual effects tools he has been involved in developing for Green Parrot Pictures and for the Academy Award winning software suite Furnace from The Foundry. He also outlined his current work in 2D to 3D conversion for an Enterprise Ireland project of which he is principal investigator.
François identified some of the research challenges in this field, which include improving the speed and functionality of existing algorithms and the development of new diagnostic tools for post-production.
Maryline Clare-Charrier – Need of R&D for stereoscopic 3D
Maryline Clare-Charrier of Orange Labs R&D focused on quality aspects of stereoscopic 3D. At present, she said, quality falls short of market expectations, and the problems are both of equipment and of content production processes.
She stressed the importance of collaboration in research, and the need to work with partners spanning the whole chain from acquisition through processing, encoding, transmission and decoding to final adaptation for viewing. Such an integrated approach, she said, eliminates the need for simulation (which distances research from the real world) and makes research results more useful and relevant to industry than mere prototypes or demos.
Maryline described some of her R&D work at Orange Labs, especially in the context of 3DLIVE, a three-year French collaborative project which combined R&D with live 3D events.
She laid out the following principles for research in this area:
- combine R&D and the real world as often as possible;
- define a 3D perception model – stereoscopy is not just a matter of ‘visual quality’ but includes two new axes: visual comfort and depth perception;
- perform subjective tests to see how people react to these three axes, and what makes content fulfil quality criteria;
- create rules and tools to help stereographers and film directors make artistic decisions whilst respecting technical parameters and understanding how parameters interact;
- perform subjective screen tests.
She identified the following areas of research as of greatest potential value to industry:
- more work on fatigue, headaches and other discomforts;
- subjective testing of all new equipment, reporting to manufacturers and sharing results with the 3D community (via standardization, conferences, etc).
Gavin Duffy – What one small part of the 3D industry needs from the research community
Gavin Duffy began his presentation by reminding us just how broad the 3D industry is, covering such disparate areas as GIS (Geographic Information Systems), animation, 3D TV, engineering, medicine and architecture.
He noted that many of the technological advances have arisen from the film and games industries, as they continually strive to push the visual envelope for entertainment; consequently there is great potential for harnessing games technology and methodology to simulating and solving real world problems.
If academia is to exploit this opportunity, he argued, then traditional university departments where 3D is taught (film, media and gaming courses) must reach beyond their ‘silos’ to collaborate with other departments, especially in science and engineering.
Gavin described how his company, RealSim Ltd., has engaged with academia in R&D into 3D simulation of marine environments, use of game play analytics for resource management, and simulation and visualisation of interstellar forces in astronomy research.
Of special interest to RealSim is the creation of accurate and realistic 3D simulations of the world. Until recently, he said, GIS has been largely 2D, but the cumbersome architecture of GIS and CAD software does not allow for the manipulation of the massive, graphically rich datasets needed for a large, detailed, virtual world environment. The games industry, however, has developed smart algorithms and hardware (graphics card technology) to perform just this type of task. A game engine, said Gavin, can not only visualise the world as it is but, through physics engines, can simulate the world we want to live in, helping planners and decision makers to decide how the future should look.
There are technical problems to overcome in integrating games technology and data formats into existing GIS architecture, and this is where academia can come in. Students and researchers can harness the large array of software and hardware developed by the games industry to develop solutions for the 3D mapping industry and many others besides.
Benoit Michel – 2012: 3D Hits The Wall
In the last of the four expert presentations, Benoit Michel of Stereoscopy News considered a range of research challenges in the field of stereoscopic 3D cinema and television. He compared the situation now with what it could be in ten years’ time if concerted research effort is put into solving technical problems.
These include the set-up complexities of 3D camera rigs, incompatibilities among 3D file formats and display technologies, and the limitations of autostereoscopy for multiple viewer/ large screen situations.
Benoit argued that 3D movie creation/ distribution is currently too complex and expensive, that users are often frustrated and that the need for special glasses is a major obstacle to widespread uptake of 3D TV. He envisaged a simpler, less expensive state-of-the-art in 2022 with:
- triple-cam rigs able to acquire extreme resolution depth maps;
- automatic post-production correction of inter-ocular distance (IoD) and convergence;
- a universal 3D file format facilitating distribution and processing;
- good quality, inexpensive autostereoscopic displays up to two meters wide;
- TVs with interactive depth control.
Other difficult problems for researchers include 3D stereoscopic rendering of hair, smoke, clouds and semi-transparent edges, and the handling of mirrors, reflection and refraction.
In a short question and answer session following the presentations, several interesting issues were raised, among them the issue of how durable the present phase of 3D entertainment is likely to be, given its history of waxing and waning popularity. Panellists felt that the great advances in technology in recent years greatly encouraged uptake, especially in the home, which was seen as crucial for establishing 3D long-term.
Audience members cited negative factors such as the poor quality of some 3D content – for example, many automatic 2D to 3D movie conversions – as deterrents to public acceptance.
The problem of incompatible 3D technologies and data formats was discussed, and finally Gavin reiterated the need for academic researchers to engage in cross-disciplinary collaboration in order to address the range of difficult technical challenges facing different areas of 3D industry.
|-||WIAMIS 2012 programme: "What 3D Industry Needs From Research" (2)||-|
|-||WIAMIS 2012 intro: "What 3D Industry Needs From Research" (1)||-|
|François Pitié||Francois Pitié: "Research for 3D Movie Post-production" (2)||-|
|Maryline Clare-Charrier||Maryline Clare-Charrier: "Need of R&D for Stereoscopic 3D" (0)||-|
|Gavin Duffy||Gavin Duffy: "What 3D Industry Needs From Research" (0)||Gavin Duffy: notes (0)|
|Benoit Michel||Benoit Michel: "2012: 3D Hits the Wall" (1)||Benoit Michel: notes (0)|
François Pitié is a researcher at Trinity College Dublin. His research interests include digital cinema and digital film restoration. He has published over 30 papers on a variety of topics and has two patents. His work on colour transfer and deflicker are references in the field and have been implemented in a number of commercial applications including the Academy Award winning software suite Furnace from The Foundry. He has worked on many EU projects, including i3DPost, an FP7 project on multi-camera cinema production, and is currently principal investigator for an Enterprise Ireland project on 2D to 3D conversion. François has consulted for a number of companies, including Weta Digital, and recently founded PixelPuffin, a start-up specialising in the development of post-production tools.
Maryline Clare-Charrier graduated from INSA (Institut National des Sciences Appliquées) in 1989. She gained 10 years’ image coding experience at Canon Research, where she led research work linked to fractals. She then became heavily involved in the JPEG2000 standardization effort, as Head of the French Delegation and co-chair of the Transform-Quantization-Entropy Coding group. She subsequently joined Orange Labs R&D to work on advanced video coding, and was responsible for the 3D side of the Don Giovanni operation – the first live 3D transmission of an opera, in June 2009. Maryline led 3DLIVE, a consortium of 8 French industrial and academic partners aiming to develop high-quality live stereoscopic technologies, and will coordinate 4EVER, a new French collaboration whose main targets are ultra-high-definition transmission for home and cinema, and HD for all in every situation.
Gavin Duffy is a trained geophysicist, but seven years ago discovered the potential of gaming technology and methodology for solving real world problems. Having used traditional GIS software for interrogating 2D maps, he was impressed by the real-time rendering power of game engines in handling graphically rich, large-scale, real-world environments. In 2007 he set up RealSim, initially offering a 3D city planning and development solutions. Since then the company has expanded its 3D simulation services to other areas, producing simulations for marine, industrial, medical and historical environment applications. Game player analytics is another area of interest for Gavin as he believes it can provide insights into social and economic behaviour not possible with mathematical analysis, because as he says, “people are unpredictable and cannot be easily defined by a set of algorithms”.
Benoit Michel co-founded Neurones, one of the first 3D cartoon studios in Europe, in 1989. After that, he switched to R&D project management in telecommunications, human-machine interaction, digital cinema, and multimedia. He is currently at UCL (Université catholique de Louvain, Belgium), where he has managed the SIMILAR network of excellence on multimodal interfaces, the EDCINE project on enhanced digital cinema and the Belgian funded 3D Media research project focusing on stereoscopic and 3D imaging. He is now working on a medical imaging project called InVivo/ IGT improving the way 3D images are driving proton therapy machines for cancer treatment. Benoit is a member of the editorial board of ERCIM News, and is a consultant for the European Commission, the TWIST cluster of Walloon companies and various private companies. He is Editor of the free newsletter Stereoscopy News and co-founder of Eurospaceward, a European association promoting earth preservation through space activities.