This is a collaborative project with Vince and Yves. It is an interactive voice/sound installation about a future AI therapy program - Jane.

Idea: An interactive storytelling installation inspired by a scene from the movie 2001: A Space Odyssey, where the astronaut Dave shut down the AI HAL 9000 that controls the spacecraft. " In the film, HAL's central core is depicted as a crawlspace full of brightly lit computer modules mounted in arrays from which they can be inserted or removed." In the scene, Dave was removing the modules to terminate the HAL, as each module was being pulled out the AI started to deteriorate. Its voice was getting slower and deeper and it was begging Dave to stop as it was dying. And it finally sang a song "Daisy", which was the first ever song sung by a computer in human history.

Concept option 1: As I just finished my Code of Music midterm project, a live audio-visual generator, I am thinking about keeping refining it, not just sound wise, but also interface wise. Because as I was playing with my own web-based instrument, I was pretty happy with the overall sound and visual reactivity and complexity, but I was still struggling with making my live composition more expressive. Having the whole interface living in the browser indicates certain limitations. I was only able to control the audio and visual with three sliders on the screen, a mouse drag function, and a key pressed function.  I was having trouble controlling multiple parameters at the same time because there is only one mouse. So I thought about why not make it a physical interface, not necessarily physical buttons and sliders like a MIDI controller, which has already been made probably thousands of times in other ITP projects. I wanted to create something that make people "dance", either some sort of games similar to DDR with visuals indicating what actions users should take to match with the sound in order to get points, or playing with the idea of "body as cursor", to have different user actions as input for different parameters, so as to generate sound and manipulate visuals accordingly, to have human body act as an instrument.

Midterm is during Halloween, so my partner Shivani had an idea of a Spooky photo booth with some sensor (TBD) that triggers webcam to take spooky distorted pictures. We started brainstorming by asking ourselves what kind of activities we would want our users to engage at the photo booth without giving explicit instructions or without them knowing what they need to do in order to create surprises and fun. I told Shivani about my experience playing with the mood ring the other day. As I put on the ring the color of the stone on the ring changes depending on my finger temperature and each color represents a mood. So I thought of implementing some wearables that output some individualized messages, that could be fortune or curses, depending on the attributes of the person wearing. Shivani instantly thought about the sorting hat from Harry Potter. That was a moment of excitement when we realized we can have a fancy fortune-telling witch hat as a prop for triggering the camera at the photo booth.

For application, I previewed two-way serial communication, and took one of my animation sketches from ICM class and have it controlled by Arduino.

In my original p5 sketch, I have two sliders one controlling the scaling speed and the other controlling the rotating speed of the squares, and also a button changing the color of the squares randomly. Each controlled by mouse dragging or clicking. So I thought it might be a good sketch to apply to serial communication as I can replace mouse interaction with potentiometers and button on my Arduino.

To apply all the topics we covered in the past two weeks, I came up with a 4/4 servo metronome which applies the concepts of digital output, analog input and analog output(PWM), tone() and servo.

One of my favorite quotes from the chapter Design Meets Disability by Graham Pullin, is "it is technology as a means to an end, not an end in itself ". "Design depends largely on constraints." - Charles Eames's disability inspired design also catalyzed a wider design culture. Constraints arise from both user needs and desires and from technical feasibility and business viability. A good design indicates a healthy balance between problem solving by recognizing the constraints and exploring freedoms by challenging the constraints. There are a lot of tensions between design for fashion and design for disabilities. The positive image of disability is achieved without discretion(invisibility) in the field of eyewear while design of hearing aids indicates priority of invisibility, and its functionality is in conflict with its miniaturization. Other wearable devices for disability such as prosthetics could support more positive image of disability, through emulating the approach of eyewear.

As I commute everyday by NYC subway, I have observed as well as personally experienced the frustrations from millions of subway riders swiping metro card at the turnstiles. According to MTA's answer to how to use metro card on the subway, "With the MetroCard name facing toward you, quickly swipe your MetroCard through the turnstile in one smooth move. Walk through when the turnstile screen says "GO." It sounds fairly intuitive and simple, get the direction correct, swipe and go, which aligns with my assumption of how to use the metro card in order to get into the station.

Application: analog input from temperature sensor to indicate whether water is cool enough to drink.   As shown in the videos below, when the temperature sensor is attached to a cup of water at 30 degree Celsius, green LED is lit, indicating ok to drink; when sensor gets close to a cup of hot water >57 degree celsius (I picked the value from a study which claims the optimal drinking temperature of hot drink is 57), red LED is lit, indicating it's too hot to drink. Of course the temperature threshold here is arbitrary, I can adjust the value to whatever I want.

Chapter 1 The Psychopathology of Everyday Things of Design of Everyday Things has provided me some very great insights on principles of physical design. Author Norman has illustrated his ideas with many real life good and bad examples like telephones, cars, and watches which I can closely relate in my everyday life. I understand that how we work/use/operate things should be reflected by the design of the things themselves without any needs for words or symbols.One of the most important principles of design is visibility. Natural design gives natural visual cues to convey how to correctly work with things. A successful design should have visible instruction or cues with feedback of actions, and there should be enough feedback for different features. There should also be a close relationship between what you want to do and what seems possible when mapping things. Natural mapping between a control and its function comes from physical analogies and cultural standards, rather than arbitrary decisions. There should also be constraints to limit the possibilities to avoid confusion. The psychology of materials/affordance of things(purpose) also matters: different materials is used for different things, which also direct people to interact with things differently.

Lab Electronics

I used the ToolSmart Digital Multimeter, a bluetooth enabled auto-range multimeter.

Hands really are amazing "tools" we all inherently own since the beginning of the humanity. It is amazing how a great variety of intricate tasks we can accomplish with our bare hands as we grasp, touch, pick, thump and performing all kinds of movements. Bret Victor's “A Brief Rant on the Future of Interaction Design” is looking at today's technology innovation from a different perspective. Rather than emphasizing on the human demand(need) or functionality improvement(technology) or fancy interfaces, he inspired creators/designers to think more about untapped human capabilities like what other things our body parts, not limited to hands, can do instead of simply touching and dragging on the "Pictures on the Glass". He calls attention to our bodies' tactile abilities to feel and sense things, which can be easily ignored when we orient our technologies around glassy visual displays. He plead to not bypass some human capabilities we take for granted. As the starting point and the center for any design is human. I was interested in how he discusses the brain interface in response to others' comments. Technology should not sacrifice/decrease our bodies' capabilities, rather it should be adapted to fit our bodies and to expand our capabilities. The future of interaction design is calling for more extensive human research than ever.