Menu

Institute for IDEAS

Institute for Immersive Designs, Experiences, Applications, and Stories

Institute for IDEAS

Institute for Immersive Designs, Experiences, Applications, and Stories

Mission

The Institute for Immersive Designs, Experiences, Applications, and Stories (the Institute for IDEAS) is an interdisciplinary research center in the nations capital. The mission of the Institute is to lead interdisciplinary research focused on developing, applying, and understanding immersive technologies, including virtual, augmented, mixed, and extended reality. The Institute serves as an international hub for scientific progress, operating at the meeting point of a wide variety of research and creative fields. 

News

Research

Dr. Pietroszek's Talk at the MOVES Institute

Real-time Volumetric Telepresence

August 28, 2023

Professor Krzysztof Pietroszek traveled to Monterey,
California, to initiate collaboration between the
Institute for IDEAS and the Naval Postgraduate
School. At the end of his visit, he gave a talk at the
MOVES Institute on Real-time Volumetric
Telepresence: Development and Application in
Education and Training. Researchers approach
telepresence in two approaches: “Star Trek style,” where participants are fully present in a virtual environment, and “Star Wars style,” which involves holographic representation in the real environment. Dr.Pietroszek demonstrated the applications through previous volumetric systems.

The project “UniVResity” allows students to connect to a virtual classroom that mirrors real-time activities in a physical classroom. Using a VR application, remote students can join the class through a unique code provided by the teacher. Deep neural networks generate lip-sync and face-expression animation, non-verbal communication cues, gestures, and the teacher’s avatar movement using only the instructor’s voice as input.

The Mixed Reality Communication Telehealth System enhances access to procedural training and remote emergency assistance. It enables remote experts to guide local operators through medical procedures using real-time volumetric views captured by RGBD cameras. This augmented view helps experts provide spatial guidance using visual and verbal instructions. The system’s effectiveness was evaluated by comparing it with video communication for teaching ultrasound-guided procedures. The results highlight the system’s potential for improved visual communication compared to traditional video teleconferencing-based training.

Virtual Reality is a medium that can replace videoconferencing for remote class participation, medical procedures, and beyond. Soon, students can participate in face-to-face learning in person or as an avatar. Through ongoing research, development, and collaboration, we envision a future where our VR, AR, or Mixed-Reality telepresence systems become an integral tool in training and teaching.

Dr. Pietroszek presented on August 10, 2023

Talk at the MOVES Institute at Naval Postgraduate School

Research

Dr. Pietroszek at EduLearn 2023

Volumetric Communication in Medical Training

June 13, 2023

At EduLearn 2023, Professor Krzysztof Pietroszek
showcased his NSF-funded research on Volumetric
Communication in Medical Training. Over two years
of dedicated work, he and his team successfully
developed, evaluated, and published a cutting-edge
system that enables immersive and interactive
training for healthcare professionals. Furthermore, Pietroszek’s expertise was recognized as he served as the chair for the Virtual and Augmented Reality session, further highlighting the growing importance of these technologies in the field of education. 

Research

Mixed-Reality Research Presented at ITS2023

Allowing first responders to administer aid remotely

June 13, 2023

Dr. Pietroszek and Manuel Rebol recently presented
research at the 2023 Intelligent Tutoring Systems
Conference (ITS2023). Their paper, “
CPR Emergency
Assistance Through Mixed Reality Communication
,”
offers a unique solution to improve remote assistance
in the medical field. The system utilizes RGBD
cameras to capture a three-dimensional view of the local scene, including the patient, the first responder, and the surroundings. The volumetric capture is then augmented onto the remote expert’s view, allowing them to provide spatial guidance using visual and verbal instructions.

By combining visual and verbal instructions in real-time, the system enables remote experts to guide first responders during life-saving procedures such as CPR. The research study confirmed the system’s effectiveness, highlighting the advantages of mixed reality over videoconferencing-based assistance. This innovative technology can improve CPR outcomes and significantly redefine emergency medical communication.

The success of this research study opens doors to further advancements in mixed reality communication systems, allowing experts to provide real-time guidance and support remotely, even in high-stress scenarios. As the technology continues to evolve, it could be integrated into various healthcare settings, emergency response training programs, and even everyday situations where immediate CPR assistance is required.

Research

Immortalizing Wanda Rutkiewicz

Using Three Scenes for Virtual and Augmented Reality

May 19, 2023

To honor the indomitable spirit of Polish mountaineer
Wanda Rutkiewicz, the Institute for IDEAS recreated
pivotal moments from Rutkiewicz’s historical ascent
of Mount Everest as holographic recordings. Three
virtual and augmented reality scenes bring Wanda’s
awe-inspiring journey to life, transporting viewers
into the world of one of history’s most iconic female climbers.

One of the scenes depicts Wanda Rutkiewicz fearlessly traversing an ice-covered pass on her Everest expedition. The hologram showcases Wanda’s careful steps across treacherous terrain, immersing viewers in her challenges and the breathtaking beauty surrounding her. Another scene catapults viewers into the heart-stopping moment when Wanda fearlessly scaled the icy precipice. 

A crucial aspect of recreating Wanda Rutkiewicz’s climb lies in capturing the human experience amidst extreme conditions. The Institute for IDEAS encapsulates this by creating a scene depicting Wanda enjoying a meal in her tent before settling down for sleep. This intimate portrayal highlights the significance of sustenance and rest in the demanding world of high-altitude mountaineering. Viewers are transported into the tent, sharing a moment of respite with Wanda as they witness the importance of self-care and preparation for the next leg of her journey.

Through these holographic scenes, Wanda’s incredible feats become accessible to a global audience, fostering a deeper understanding of her courage, perseverance, and trailblazing spirit. These holograms exemplify the Institute’s commitment to pushing the boundaries of immersive technologies, enabling individuals to connect with and be inspired by the remarkable achievements of those who came before us.

Research

Transporting Dance to the Volumetric Stage

Rose Xinran Qi's Pioneering Journey into Holographic Dance Performances

May 20, 2023

The Institute for IDEAS collaborated with the
exceptionally talented dancer and choreographer
Rose Xinran Qi. This groundbreaking partnership
involved capturing and rendering holographic dance
performances with our state-of-the-art volumetric
studio. The result was an awe-inspiring visual
experience, where viewers could witness her dance as if she were physically present before them. As a result, Rose Xinran Qi’s visit to the studio marked a significant milestone in exploring the intersection between dance and holographic technology.

The collaboration between Rose Xinran Qi and presents us IDEAS has provided invaluable insights into holographic dance performances’ possibilities and potential applications. This research endeavor aims to revolutionize the way dance is experienced, not only by audiences but also by the dancers themselves.

The Institute for IDEAS intends to utilize the data and knowledge gained from this collaboration to explore various applications of holographic dance performances in augmented and virtual reality. This technology opens up new avenues for dance education, where aspiring dancers can learn from the holographic projections of esteemed performers. Moreover, choreographers can experiment and create stunning visual spectacles by blending physical and holographic dancers. 

Research

Musical Artists Record Holograms At IDEAS

September 1, 2022

The Institute hosted Maryland-based musical duo Dunia & Aram, composed of ska-punk performer Dunia Best and AU professor Aram Sinnreich. Dunia & Aram performed their arrangement of Billie Holiday’s heart wrenching classic “Strange Fruit”, a song from their new album, Bedfellows. The recordings will be used to create a 360° music video featuring multiple holograms of Dunia and Aram performing all the instruments and vocals for the song.

Research

Sharing History & Heritage Through a New Medium

The American University of Sharjah project bringing the past to life

August 3, 2022

The Institute for IDEAS’ cutting-edge volumetric
capture studio recently played a central role in
the development of a groundbreaking
documentary research project at the American
University of Sharjah. In collaboration with
Professor Suheil Dahdal, the Institute produced
several holograms for use in the documentary
experience/research initiative Emirati Stories AR Documentary.
The Institute for IDEAS’ cutting-edge volumetric capture studio recently played a central role in the development of a groundbreaking documentary research project at the American University of Sharjah. In collaboration with Professor Suheil Dahdal, the Institute produced several holograms for use in the documentary experience/research initiative Emirati Stories AR Documentary.

Research

Space Ocean Library At UFVA

Sharing Immersive Experiences With New Audiences

August 18, 2022

Last month, Becky Lake and Meagan Couture attended the University Film & Video Association Conference to represent their project Space Ocean Library, a collaboration with American University students and the Institute for IDEAS. Users tested the experience and provided feedback for future development.

Research

Dr. Pietroszek Delivers Talk At TAI

Introducing students to volumetric storytelling worldwide

July 25, 2022

On the 20th of June, Institute for IDEAS founder
Dr. Krzysztof Pietroszek hosted a meeting at the
prestigious TAI School of the Arts in Madrid,
Spain. This discussion, titled “Narrative Uses
Of Volumetric Capture: Storytelling And The
Artificial Reality Continuum”
, explored the
potential uses of volumetric capture through
cutting edge technology such as the Institute’s Tetavi capture studio
On the 20th of June, Institute for IDEAS founder Dr. Krzysztof Pietroszek hosted a meeting at the prestigious TAI School of the Arts in Madrid, Spain. This discussion, titled “Narrative Uses Of Volumetric Capture: Storytelling And The Artificial Reality Continuum”, explored the potential uses of volumetric capture through cutting edge technology such as the Institute’s Tetavi capture studio
Volumetric capture brings real world performers and objects seamlessly into the virtual space, using photogrammetry. In his talk, Dr. Pietroszek addressed the ramifications of this technology for filmmaking and other arts, as well as the many avenues for expansion within this medium, like haptic sensory experiences. Volumetric capture is an innovation in digital media, and the Institute for IDEAS is honored to collaborate with TAI in pioneering this new field of the arts.
Volumetric capture brings real world performers and objects seamlessly into the virtual space, using photogrammetry. In his talk, Dr. Pietroszek addressed the ramifications of this technology for filmmaking and other arts, as well as the many avenues for expansion within this medium, like haptic sensory experiences. Volumetric capture is an innovation in digital media, and the Institute for IDEAS is honored to collaborate with TAI in pioneering this new field of the arts.

Research

The Institute's Volumetric Capture Studio

Creating holograms and immersive experiences at AU

July 12, 2022

Based out of a high-tech studio space, the Institute is developing immersive volumetric content for VR, AR, and other applications. The core of this initiative is the Institute’s volumetric capture system, purchased with a $1 million grant from the National Science Foundation. TDesigned by international tech company
Tetavi, the volumetric capture studio smoothly transforms recorded performances into high-quality holograms. Tetavi’s groundbreaking algorithmic process delivers a detailed, animated model of the subject for deployment across a number of platforms.

Research

Dr. Pietroszek Presents a Keynote at Cannes

Dr. Pietroszek’s keynote describes a new and revolutionary form of storytelling.

July 14, 2021

Once a domain of science fiction, holographic
filmmaking is becoming the next technological
revolution. At the 2021 Cannes Film Festival, Dr.
Pietroszek voices the capabilities and
complexities of the evolving medium. He shares
how storytellers are now using volumetric
capture to recreate objects in virtual spaces.
Once recorded, these holograms can then be placed in virtual or real environments, creating an experience Dr. Pietroszek compares it to Star Trek’s Holodeck.
Once a domain of science fiction, holographic filmmaking is becoming the next technological revolution. At the 2021 Cannes Film Festival, Dr. Pietroszek voices the capabilities and complexities of the evolving medium. He shares how storytellers are now using volumetric capture to recreate objects in virtual spaces. Once recorded, these holograms can then be placed in virtual or real environments, creating an experience Dr. Pietroszek compares it to Star Trek’s Holodeck.
Actors, props, sets, and even entire structures can now be captured as photorealistic 3D models, shown as holograms; This process is known as photogrammetry. Photogrammetry allows the extraction of 3D data from photos and drawings. Once captured, audience members can view these elements through head-mounted glasses, known as a cave system. This cave system allows for an immersive experience where one can enter and interact with the virtual world. Some bring audience members into the set, others bring props and set elements into the real world. Like computer coding, the language and grammar of volumetric filmmaking can be very complex.
While the technology is ready for new stories and hniques, our understanding of the language and grammar of this medium is very limited. Through the Cannon keynote, Dr. Pietroszek discusses how filmmakers can learn and use volumetric filmmaking. Using low-cost volumetric capture technologies, Dr.Pietroszek invites filmmakers to experiment with volumetric capture. “Holographic filmmaking is a new and revolutionary form of storytelling that will not replace traditional cinema, but rather, it will become yet another” Dr. Pietroszek.
Dr. Pietroszek presented July 9, 2021

Cannes XR with Tabletop Stories

Scroll to Top

Institute for Ideas