The Laboratory on Machine Learning for Art + Design
The Laboratory on Machine Learning for Art + Design (MLAD) is an international research initiative to critically explore applications of new machine learning techniques to the contexts of art and design.
Exploring the semiotics and aesthetics of Machine Learning through the creation of interactive and generative artworks, the MLAD lab investigates the possible convergence of new Machine Learning techniques with traditional art and design contexts. The MLAD represents a unique synthesis of technological innovation, theoretical research, artistic creation, and critical awareness that leads to the creation of a diverse range of tools, publications, and artworks.
The Laboratory on Machine Learning for Art + Design (MLAD) was initiated as part of the Center of Applied Computing and Interactive Media, supported by the School of Creative Media at City University Hong Kong.
Objectives
- To critically explore the aesthetics of Machine Learning through the creation of interactive and generative artworks
- To contribute to the understanding and transparency of Machine Learning processes (to 'open the black box').
- To artistically explore machine learning architectures that actively reflect on their social, cultural and political implications.
- To develop novel ML applications that better facilitate understanding of the relationships between technology and society.
- To disseminate art/research results in exhibitions, journals and conferences, as well as on a publicly accessible website.
PROJECTS
ThermAirGlove (Kening Zhu) is a pneumatic glove that provides thermal feedback to support the haptic experience of grabbing objects of different temperatures and materials in VR. The system consists of a glove with five inflatable airbags on the fingers and the palm, two temperature chambers (one hot and one cold), and the closed-loop pneumatic thermal control system. User studies on VR experience showed that using TAGlove in immersive VR could significantly improve users’ experience of presence compared to the situations without any temperature or material simulation.
Dialogic (Daniel Howe) is a system designed to help writers easily create interactive scripts with generative elements. It enables writers to craft compelling dialog that responds organically both to user prompts and to events in the environment. The system supports arbitrarily complex interactions between multiple actors, but makes no assumptions about how text is displayed, or about how users will choose their responses. Dialogic was awarded 'Exceptional Paper' and nominated for 'Best Paper' at The International Conference on the Foundations of Digital Games (FDG2020).
Spectre (Daniel Howe) Spectre is an AI-driven interactive installation that reveals the secrets of the digital influence industry as users pray to Dataism and the Gods of Silicon Valley. With the help of 'deep-fake' celebrities, created via machine learning, including Mark Zuckerberg, Marcel Duchamp, Marina Abramović, and Freddie Mercury, Spectre tells a cautionary tale of technology, democracy and society, curated by algorithms and powered by visitors' data. Winner of the 2019 Alternate Realities Commission.
Big Dada (Daniel Howe) features AI-generated 'deep-fake' character studies of Marcel Duchamp, Marina Abramović, Mark Zuckerberg, Kim Kardashian, Morgan Freeman and Freddy Mercury. Big Dada was released on social media in June 2019 and quickly went viral, leading to global press coverage and confused responses from Facebook and Instagram regarding their policies on computational propaganda.
Eyes-free Smartwatches (Kening Zhu) expands the interaction space of touch-screen devices (e.g., smartphones and smartwatches) via bezel gestures. While existing works have focused on bezel-initiated swipe on square screens, we investigate the usability of BIS on round smartwatches via six different circular bezel layouts. We evaluated the user performance of BIS on these layouts in an eyes-free situation and found that the performance of BIS is highly orientation dependent, and varies significantly among users. We then compare the performance of personal and general ML models, and find that personal models significantly improve the accuracy for a range of layouts. Lastly, we discuss the potential applications enabled by the BIS.
TipText (Kening Zhu) investigates new text entry techniques using micro thumb-tip gestures, specifically using a miniature QWERTY keyboard residing invisibly on the first segment of the index finger. Text entry can be carried out using the thumb-tip to tap the tip of the index finger. The keyboard layout is optimized for eyes-free input by utilizing a spatial model reflecting the users' natural spatial awareness of key locations on the index finger. Our user evaluation showed that participants achieved an average text entry speed of 11.9 WPM and were able to type as fast as 13.3 WPM towards the end of the experiment. Winner: Best Paper Award, ACM UIST 2019
People
