Augmented Materiality Laboratory
PROJECTS
Radiant Soma emphasizes the ephemeral nature of dance by reinterpreting mocap data as living trails of laser light trapped on the surface of phosphorescent rocks.
More details: https://augmentedmaterialitylab.org/2021/11/22/soma/
Combining Tibetan Buddhist meditation practices and virtual reality technology (with neurofeedback) as a therapeutic intervention in palliative care.
An investigation of the potential of using Augmented Reality (AR)-based narratives to treat PTSD.
More details: https://augmentedmaterialitylab.org/2021/08/31/ar-therapies-for-ptsd/
This project explores how the human brain and body respond to Augmented Reality experiences and how these findings can be used to develop novel medical therapies.
A modular multi-camera system for remote teaching providing students with multi-perspective AI-curated content.
More details: https://augmentedmaterialitylab.org/2021/08/01/reverse-panopticon/
An interactive laser-based projection system capable of sensing touch at a distance without the use of external cameras.
More details: https://augmentedmaterialitylab.org/2021/08/31/laser-sensing-display/
Integrating and synchronising 3D stereoscopic laser-based vector graphics with conventional stereoscopic video projection systems.
More details: https://augmentedmaterialitylab.org/2021/08/18/stereoscopic-laser-graphics/
An interactive installation exploring the relationship between shape, geometry, resonance and energy.
More details: https://augmentedmaterialitylab.org/2022/06/03/cymatic-ground-2022/
An interactive installation inviting the public to discuss around a table. As soon as a word is uttered, it begins a life of its own over the surface, desperate to join semantically close friends.
More details: https://augmentedmaterialitylab.org/2021/08/31/viva-voce/
People
The Laboratory for Interventions in Speculative Finance (LISF)
People
Studio for Narrative Spaces
PROJECTS
Applying digital (VR) and physical environments for human collaborative purposes, enhancing human-machine collaborative work, or inducing emotional responses or rehabilitating human mental illness. This has resulted in work looking at how human-robot interactions change based on seating arrangement, mixed reality approaches to active game play, VR spaces for systematic desensitization therapy approaches in speech therapy, comparison of digital and physical interventions in augmented spaces. On-going projects include Configuration of Space, with Dr. Zhicong Lu (CityU CS) on the way digital spaces like Ohyay uses configurations to change human-human collaboration and drives changes in perception; Virtual Diagnostics, with Dr. Julian Lai (CityU Psyc) on the use of VR games to assess potential fear of intimacy and isolation mental disorders in youth populations; Empathy for Refugee in VR, on active navigation in navigating refugee environments for fostering empathy; Weaved Wearable Sensors for VR, using tangible wearable components for detecting stress and anxiety in the context of VR rehabilitation.
Key publications:
- LC R, Friedman N, Zamfirescu-Pereira JD, and Ju W. (2020) “Agents of Spatial Influence: Designing incidental interactions with arrangements and gestures.” HRI '20 Workshop: The Forgotten HRI: Incidental encounters with robots in public spaces. In 2020 ACM IEEE International Conference on Human-Robot Interaction.
- Liu Y, Si Y, LC R, Harteveld C. (2021) “cARd: Mixed Reality Approach for a Total Immersive Analog Game Experience.” In: Arai K., Kapoor S., Bhatia R. (eds) Proceedings of the Future Technologies Conference (FTC) , Vol. 2. Advances in Intelligent Systems and Computing, vol 1289.
- LC R and Fukuoka Y. (2019) “Machine Learning and Therapeutic Strategies in VR.” ARTECH 2019: Proceedings of the 9th International Conference on Digital and Interactive Arts. Braga, Portugal: 42, 1-6.
- Coutu Y, Chang Y, Zhang W, Sengun S, and LC R. (2020) “Immersiveness and usability in VR: a comparative study of Monstrum and Fruit Ninja.” In Bostan: Game User Experience and Player-Centered Design. International Series on Computer Entertainment, Springer.
- LC R, and Monir F. (2020) “A Case for Play: Immersive Storytelling of Rohingya Refugee Experience.” Media-N Journal of the New Media Caucus. Issue on NEoN Digital Arts Re@ct Social Change Art Technology. Dundee, UK. Online.
Key personnel:
- RAY LC, Lina Zhang, Luoying Lin, Zhicong Lu (collab), Julian Lai (collab).
Study machine movement gestures to affect human perception and behavior, using both physical and virtual testing strategies. This has resulted in work investigating perception of robot-robot dynamics during game play, signaling robot personality through appearances like clothing, using robotics to aid self-identification psychologically, investigating multi-robot signalling with observers, prototyping strategies using video and VR for human-robot interaction. On-going projects include Social Rejection in Robot-Robot-Human Interaction, using video and robot arms to study how group dynamics affect interpretation of being rejected by robots; AI as Active Writer, with Dr. Toby Li (Notre Dame) on human perception of collaborative expressive machine writing agent on a digital platform; Presentation of Self in Machine Life on human-machine dance performance through virtual and physical telepresence and expressive gestures of a robot arm; Select on moral responses of digital game characters; Connected Gaze on audience-machine collaboration using interactive gestures in telepresence spaces for performing arts. We would also love to collaborate with Dr. Alvaro Cassinelli on the machine emotional gestures projects.
Key publications:
- LC R, Benayoun M, Lindborg PM, Xu HS, Chan HC, Yip KM, Zhang TY. (2021) “Power Chess: Robot-to-robot nonverbal emotional expression applied to competitive play.” ACM ARTECH 2021: Proceedings of the 10th International Conference on Digital and Interactive Arts. Alveiro, Portugal: 13-15 October. ACM, NYC.
- Friedman N, Love K, LC R, Sabin JE, Hoffman G, Ju W. (2021) “What Robots Need From Clothing.” In ACM Designing Interactive Systems Conference (DIS’21). June 28-July 2.
- LC R, Alcibar A, Baez A, and Torossian S. (2020) “Machine Gaze: Self-Identification Through Play With a computer Vision-Based Projection and Robotics System.” Frontiers in Robotics and AI: Human-Robot Interaction.
- LC R. (2021) “Now You See Me, Now You Don’t: Revealing personality and narratives from playful interactions with machines being watched.” Proceedings of the 15th International Conference on Tangible, Embedded, and Embodied Interaction (TEI’21).
- Zamfirescu-Pereira JD, Sirkin D, Goedicke D, LC R, Friedman N, Mandel I, Martelaro N, Ju W. (2021) “Fake It to Make It: Exploratory Prototyping in HRI.” Companion Proceedings of the 2021 ACM IEEE International Conference on Human-Robot Interaction (HRI’21).
Key personnel:
- RAY LC, Hongshen Xu, Zijing Lillian Song, Toby Li (collab).
The way we interact with more ubiquitous robots in the future will involve human-like emotional interactions. For example, what happens if we interrupt a robot at work, and the robot stares at us? Our work investigates how groups of social robots in the environment react in human-robot collaborative contexts. In one example recent work for HRI, we investigated the cohesiveness of robot groups, termed entitativity, and how it affects humans' willingness to engage with the group and alter their perception of threat and cooperativity. To see how group composition affects how people perceive negative social intent from robots, we showed subjects videos of ways that humans are socially rejected by robots under high and low entitativity conditions. The results reveal that when robotic groups are have less entitativity (cohesiveness), the sense of rejection is greater, showing that humans experience increased anxiety over being rejected by more diverse groups of agents. We seek to understand how the social consequences of robot group dynamics are perceived, and hence avoid unanticipated conflict and negative perception of robot responses.
For more information, please visit https://recfro.github.io/pubs/
People
XRLab
People
Computational Media Lab
PROJECTS
This project targets to improve the viewing experience of 360-degree stereoscopic VR content by exploiting the human vision system (HVS) and the constant field of view characteristic in VR. We rely on our binocular vision system to fuse the two images displayed by the VR headset to create a stereoscopic mental picture. However, many people have slightly unequal refractive power between their eyes, and this might result in a sub-optimal stereo experience. Therefore, we propose an asymmetric approach which applies different degrees of image detail enhancement to the left and right eye images as an improvement measure. As modern VR headset allows the user to freely change the viewing angle but the field of view is kept constant at about 120 degrees; this means only a limited portion of the 360-degree image is visible to the user at any moment. To further refine the VR viewing experience, we propose to make the detail enhancement process view-dependent such that it dynamically adapts to the visible image content.
The GPU image detail enhancement algorithm used in this project is introduced by Mike wong in 2017 (published in GPU Zen: Advanced Rendering Techniques)
An asymmetrically enhanced 360-degree stereoscopic image sample (Original photo from the insta360 image samples archive)
People
Game Lab
People
Neuro-Design Lab
PROJECTS
POWER CHESS
Jul 2021 - Posted in Interactive Art, Robotics by moben
Power Chess presents two robots playing a game of chess unlike any other. Maurice Benayoun and his collaborators invited two robots (robotic arms, to be precise) to compete in a game of chess. MoBen chose the game as a metaphor for society. This is the meaning behind the piece: making societal phenomena visible, particularly power relations, hence the name of the piece: Power Chess.
At each new Focal Powered by Naim store location, Power Chess will be presented and experienced through different products, including the latest Focal Celestee and Clear Mg headphones, powered by the Naim Uniti Atom Headphone Edition, plus a range of classic, active and integrated Focal loudspeakers. You will experience all the emotions and actions of the chess game as the robotic competitors fight it out. For MoBen, works of art are not just objects to be observed but become “subjects capable of complex behaviors.
The Brain Factory project aims to give shape to human abstractions through BCI (Brain-Computer Interaction) through materialisation process called “reification,” in which thoughts are converted into objects by 3D Printing.
 |
 |
VALUE OF VALUES AT MOCA TAIPEI
Oct 2019 - Posted in Exhibition, Transactional Art by moben
DIGITAL ART FESTIVAL 2019, RETRANSENS.
MUSEUM OF CONTEMPORARY ART TAIPEI,
Curator: Loh Li-Chen
Co-curator: Hsiang Wen Chen
The largest exhibition of Value of Values, with 3 Neuro-design stations. Value of Values is now on Matic Blockchain after the Collector Period on Test-Net. It is now possible to trade, buy and sell Human Values
DÏALOG, THE FIRST ALIENS' ENCOUNTER
Jun 2021 - Posted in Installations, Interactive Art by moben
by Refik Anadol and Maurice Benayoun
DïaloG is an interactive urban media art installation exploring the themes of alterity, strangeness, and immigration, in a performative way. The work presents two pieces that are at the same time artworks and aliens. They are confronted to a new environment where they don’t belong to… yet. They will have to adapt their language, build common knowledge, integrating all new artifacts and natural phenomena that constitute now their environment. This includes the other living beings moving around them, and, eventually, they will have to understand each other. While we use the concept of language very broadly to include speech, performance, gesture, utterance, and even data, we focus on strangeness from an ontological perspective, trying to mark a terrain of possibilities for the intersection of interpersonal and digital experiences in the urban sphere.
HK ESCAPE VIEWS, PANOPTICAL MEMORIES
Sep 2020 - Posted in Installations, Interactive Art by moben
Hong Kong Escape Views, Panoptical Memories are inspired by prior projects like Art Impact, 2000, at the Pompidou Centre in Paris, where people painted their gaze watching in VR a contemporary Art Exhibition, together with environments of the daily life like supermarkets, slaughter house, a quarry, creating, in the Duchamp’s sense of the terms, a competition of retinal seduction. The supermarket won so many times when compared to art installations… Then So.So.So., Somebody, Somewhere, Some Time, 2002, a narrative version of the concept presented for the first time at the ZKM, Karlsruhe, in Future Cinema exhibition curated by Jeffrey Shaw and Peter Weibel. The story was written in real time through the experience of watching situations. Later, 2005, Cosmopolis, Overwriting the City, presented across China, (Shanghai, Beijing, Chongqing, Chengdu), 12 VR telescopes offered to the public to mix 12 cities around the world, keeping the best of each in terms or retinal seduction.
Stemming on the concept of creating a dynamic palimpsest based the experience of watching, Escape Views, Panoptical Memories, show first an unexpected vision of Hong Kong, places of seclusion, attempts of isolation, sometimes unfruitful, that fits so well the idea of escaping the pandemics and, at the same time, the political evolution of the City.
ASTRILAB
Nov 2017 - Posted in Exhibition, Installations, Interactive Art, Museography, Works by moben
ASTRILAB is a unique platform for experiment, creation and communication.
Astrilab scenic apparatus is constituted by 3 different elements, all playing with each other. The functions of presenting Astri activities, representing Astri dynamism, and offering a playful way to experience innovation in action, are performed by these 3 Astrilab components:
The Hypercube: a large scale cubic display made of 12 500 LED Balls. Hypercube presents volumes and data in space as a low resolution full depth information cloud in light and motion.
The Gateway: it is a 4K multitouch screen were detailed information is elegantly presented. Touch and touchless interactions are possible to get into all Astri research field strata.
The Brain Cloud: at the same time an innovative artistic installation by Maurice Benayoun and Tobias Klein, a world premiere, and a prototype for neuro-design technology. Brain Cloud is made of a massage chair, an EEG headband, an LCD monitor and, thanks to its connection to the Gateway and the Hypercube, it displays the 3D models resulting from the interaction as dynamic animated forms on the Hypercube and a full database of the brain production + brain waves monitoring on the Gateway.
VALUE OF VALUES, BAUHAUS 100, MACERATA
Jul 2019 - Posted in Exhibition, Transactional Art by moben
Value of Values, VoV, Bauhaus 100 exhibition,
Palazzo Buonaccorsi, Macerata, Italy
Artists: Maurice Benayoun, Tobias Klein, Nicolas Mendoza
Composer: Jean-Baptiste Barrière
Curator: Maria Grazia Mattei, MEET
July, 18 – Novembre, 3, 2019
WHEN TRANSACTIONAL AESTHETICS MEET NEURO-DESIGN ON THE BLOCKCHAIN
Speculative Speculation on Values
In VoV, the resulting shapes are brought into the context of an “ethical realism”, an objective representation of individual and collective hierarchy of values. While Brain Factory deals with a wide range of human abstractions, VoV only gives shape to those that could be considered as human values. The neuro-designed shapes, produced by individual visitors, become named and numbered digital 3D models: FREEDOM 0001, FREEDOM 0002… FREEDOM 000X. Each numbered item is an artwork registered on the Blockchain.
At the end of the process, the Brain Worker, the shaper of the value, becomes the owner of the Blockchain VoV and the owner of the 3D model, the “idea” of the shape. VoVs are convertible into Ethereum, a common cryptocurrency.. The owner can sell or barter the VoV, or even freely use the shape to produce artifacts, artworks, or goods.
VALUE OF VALUES, AT ISEA2019 GWANGJU
Jun 2019 - Posted in Exhibition, Transactional Art by moben
ISEA, International Symposium on Electronic Arts is the major academic event on Media Art, research and creation. Value of Values is an invited artwork curated by Art Center Nabi for the Lux Aeterna Exhibition.
"Value of Values (VoV) is a blockchain-based art project. It aims to find out the real, economic value of human values through EEG (Electroencephalography, and biofeedback). VoV is an extension to the acclaimed Brain Factory project. In both, exhibition visitors (aka Brain workers) give – straight from their brain waves – a three- dimensional shape to abstract concepts, like FREEDOM, PEACE, MONEY, LOVE, POWER.
ISEA, International Symposium on Electronic Arts is the major academic event on Media Art, research and creation. Value of Values is an invited artwork curated by Art Center Nabi for the Lux Aeterna Exhibition.
People
View-dependent Asymmetric Detail Enhancement for 360-degree Stereoscopic VR Content
This project targets to improve the viewing experience of 360-degree stereoscopic VR content by exploiting the human vision system (HVS) and the constant field of view characteristic in VR. We rely on our binocular vision system to fuse the two images displayed by the VR headset to create a stereoscopic mental picture. However, many people have slightly unequal refractive power between their eyes, and this might result in a sub-optimal stereo experience. Therefore, we propose an asymmetric approach which applies different degrees of image detail enhancement to the left and right eye images as an improvement measure. As modern VR headset allows the user to freely change the viewing angle but the field of view is kept constant at about 120 degrees; this means only a limited portion of the 360-degree image is visible to the user at any moment. To further refine the VR viewing experience, we propose to make the detail enhancement process view-dependent such that it dynamically adapts to the visible image content.
The GPU image detail enhancement algorithm used in this project is introduced by Mike wong in 2017 (published in GPU Zen: Advanced Rendering Techniques)
An asymmetrically enhanced 360-degree stereoscopic image sample (Original photo from the insta360 image samples archive)
Gestures and Tools for Human-Machine Collaboration
Study machine movement gestures to affect human perception and behavior, using both physical and virtual testing strategies. This has resulted in work investigating perception of robot-robot dynamics during game play, signaling robot personality through appearances like clothing, using robotics to aid self-identification psychologically, investigating multi-robot signalling with observers, prototyping strategies using video and VR for human-robot interaction. On-going projects include Social Rejection in Robot-Robot-Human Interaction, using video and robot arms to study how group dynamics affect interpretation of being rejected by robots; AI as Active Writer, with Dr. Toby Li (Notre Dame) on human perception of collaborative expressive machine writing agent on a digital platform; Presentation of Self in Machine Life on human-machine dance performance through virtual and physical telepresence and expressive gestures of a robot arm; Select on moral responses of digital game characters; Connected Gaze on audience-machine collaboration using interactive gestures in telepresence spaces for performing arts. We would also love to collaborate with Dr. Alvaro Cassinelli on the machine emotional gestures projects.
Key publications:
- LC R, Benayoun M, Lindborg PM, Xu HS, Chan HC, Yip KM, Zhang TY. (2021) “Power Chess: Robot-to-robot nonverbal emotional expression applied to competitive play.” ACM ARTECH 2021: Proceedings of the 10th International Conference on Digital and Interactive Arts. Alveiro, Portugal: 13-15 October. ACM, NYC.
- Friedman N, Love K, LC R, Sabin JE, Hoffman G, Ju W. (2021) “What Robots Need From Clothing.” In ACM Designing Interactive Systems Conference (DIS’21). June 28-July 2.
- LC R, Alcibar A, Baez A, and Torossian S. (2020) “Machine Gaze: Self-Identification Through Play With a computer Vision-Based Projection and Robotics System.” Frontiers in Robotics and AI: Human-Robot Interaction.
- LC R. (2021) “Now You See Me, Now You Don’t: Revealing personality and narratives from playful interactions with machines being watched.” Proceedings of the 15th International Conference on Tangible, Embedded, and Embodied Interaction (TEI’21).
- Zamfirescu-Pereira JD, Sirkin D, Goedicke D, LC R, Friedman N, Mandel I, Martelaro N, Ju W. (2021) “Fake It to Make It: Exploratory Prototyping in HRI.” Companion Proceedings of the 2021 ACM IEEE International Conference on Human-Robot Interaction (HRI’21).
Key personnel:
- RAY LC, Hongshen Xu, Zijing Lillian Song, Toby Li (collab).
Virtual and Augmented Environments in Collaborative Experience
Applying digital (VR) and physical environments for human collaborative purposes, enhancing human-machine collaborative work, or inducing emotional responses or rehabilitating human mental illness. This has resulted in work looking at how human-robot interactions change based on seating arrangement, mixed reality approaches to active game play, VR spaces for systematic desensitization therapy approaches in speech therapy, comparison of digital and physical interventions in augmented spaces. On-going projects include Configuration of Space, with Dr. Zhicong Lu (CityU CS) on the way digital spaces like Ohyay uses configurations to change human-human collaboration and drives changes in perception; Virtual Diagnostics, with Dr. Julian Lai (CityU Psyc) on the use of VR games to assess potential fear of intimacy and isolation mental disorders in youth populations; Empathy for Refugee in VR, on active navigation in navigating refugee environments for fostering empathy; Weaved Wearable Sensors for VR, using tangible wearable components for detecting stress and anxiety in the context of VR rehabilitation.
Key publications:
- LC R, Friedman N, Zamfirescu-Pereira JD, and Ju W. (2020) “Agents of Spatial Influence: Designing incidental interactions with arrangements and gestures.” HRI '20 Workshop: The Forgotten HRI: Incidental encounters with robots in public spaces. In 2020 ACM IEEE International Conference on Human-Robot Interaction.
- Liu Y, Si Y, LC R, Harteveld C. (2021) “cARd: Mixed Reality Approach for a Total Immersive Analog Game Experience.” In: Arai K., Kapoor S., Bhatia R. (eds) Proceedings of the Future Technologies Conference (FTC) , Vol. 2. Advances in Intelligent Systems and Computing, vol 1289.
- LC R and Fukuoka Y. (2019) “Machine Learning and Therapeutic Strategies in VR.” ARTECH 2019: Proceedings of the 9th International Conference on Digital and Interactive Arts. Braga, Portugal: 42, 1-6.
- Coutu Y, Chang Y, Zhang W, Sengun S, and LC R. (2020) “Immersiveness and usability in VR: a comparative study of Monstrum and Fruit Ninja.” In Bostan: Game User Experience and Player-Centered Design. International Series on Computer Entertainment, Springer.
- LC R, and Monir F. (2020) “A Case for Play: Immersive Storytelling of Rohingya Refugee Experience.” Media-N Journal of the New Media Caucus. Issue on NEoN Digital Arts Re@ct Social Change Art Technology. Dundee, UK. Online.
Key personnel:
- RAY LC, Lina Zhang, Luoying Lin, Zhicong Lu (collab), Julian Lai (collab).
