ideas - sound

Sound as input

This is something you mentioned earlier. Extending human actions by simulating the echo of actions from one place to another. Would be limited to actions that involve sound, e.g. talking, snapping your fingers etc. This could be very nice if it has a very short release time.

Verbal -> non verbal communication. Speech is transformed into abstract shapes / vibrations. Non-verbal form of communication. Like the two cups on a string sound is converted into vibration. Present an expanded way of communication aided by visuals. The pitch and tone of your voice is analysed. Each sound is presented in its wave format, but eace spline will connect to the one before, creating a generative landscape through sound.

Buoy / wave height data is basically a little slice / plane with a displacement modifier on it. Could be fun to try and use this data and apply it to an "urban slice". Either directly to create a surreal feeling or use the height data and map it to something else, such as time etc. The purpose of this would be to create a connection between two wildly different settings, busy urban setting ≠ ocean.

Alternatively go to a location and record audio. Then extract the 3D data of this location from a 3D map. Then use the sound to displace and distort the digital copy. Sound is vibrations that travels through the air. With tiny refractions. As such this project increases the impact / magnitude of these sounds. It presents us just a more extreme version of what already happens. Schlieren photography.

Sound as output

Create sound from motion. This would be about using realtime data via. a live stream. E.g. there are tons a traffic cams around Seoul which could be used. However it would be better to have a stream that involves more people and less cars. For this it would be about capturing the urban soundscapes. However this would be very similar to the wind project by Damian Stewart. To differenitate the project maybe i should try to analyse the input differently e.g. use the motion itself to create sounds, instead of pixel contrast.

Summed up: Use motion to draw paths, then convert these paths into waveforms / notes.

Water - LED experiment. This one has the same problem as the media art project about creating waves. To get around this, maybe i could make a small stand for the LED light and place it somewhere along the Cheonggyecheon and record data myself. Then use this data to do the experiment. However it wouldn't be real time data anymore. This project would be about capturing a specific moment for people to experience without being there. In a similar way to how photographs gives us a glimpse into the past. This project is about experiencing the past via an audio - visual installation / more immersive.

Alternatively I could try to use buoy / wave height data from a distant location and then use this data to generate sound, but i think this has been done many times before.

I was thinking about how to create forms related to heat. One way would be to go in the direction of a mirage / refraction of light. Or the human thermal boundary layer.

Problem with this model is that it may be too flimsy with just paper. E.g. if add support inside, it will interfere with the rear projection.

Experiments - Body heat

I don't think this turned out that good as it is not really that clear that it's about transferring / extending your body heat. Seems more like a proximity project. In hindsight i should've projected into the corner and cropped the video in a way that it feels like it fills the entire room (more similar to the second part of the setup video). It doesn't work that well on the pillar. However it was a nice experience to try and set it up. I thought i had thought of everything before projecting, but i should've made it easier to fill the entire surface with one color.

Ended up using the metal frame for the sensor as i think it shows the interaction more clearly. Also the sensor's range is not that far unfortunately.

This was such a quick and dirty setup with acrylic plates i found laying around and tape to hold it all together. With more time i would've laser cut a plate that fit the metal frame and sensor better. Ideally it would be better if i could set it up so that the projection was in one room and the sensor in another. To make it more clear that it is supposed to connect locations together.

This project also lacks feedback on the initial interaction. E.g. when you put your hand on the plate you don't get any feedback about what is happening. It should have another projection near the plate that is either similar to the projection on the other side of the pillar or a view that tells you your body temperature / the temperature that the sensor registers.

Experiment - bilocation

Translating position in space into paths. This experiment explores the idea of bilocation, being present in multiple locations at once. I chose to track the dog in this clip because it has the most interesting movement. It pokes its head around more, creating a more interesting pattern. However I think it would look pretty nice if all the movement in this clip was turned into paths and shown in real time / layer the laser cutter recordings on top of each other.

Idea proposals

So far I've mostly thought about connecting different locations together in a very "object" oriented way. I think i should do a few experiments that are more tied to connecting people across locations.

Optical Flow / difference keying. With an empty location as input data, then use movement / difference from the original input image to create distortion. People act as energy, distorting the texture around the room. E.g. like paddling in water. People as a brush.

Room size theremin?

Translating presence. With an unwrapped building / room people can lean into each room / building to experience a distant location. I think this would be more sensory data such as smells or sounds. Alternatively maybe the opposite could be done where actions in the original location is translated onto the unwrapped location. Maybe like lights?

Space within space. Projection mapping on model to reveal a bigger space contained within a smaller one. I was thinking this could be done via. either a laser pointer and a webcamera where you track the red dot in real time with a mask layered on top to reveal another space behind. Or it could be done via gyroscope data where your phone acts as a pointer. This would be less accurate but it would make it easier for people to interact with the model.

Volumetric / 3D map. Crosssections of space. Presence translated into Z - depth like the short movie Hedgehod in the fog. Layer 4 - 5 transparent lcd screens then use peoples position to place them on each screen. Another way would be to have one transparent screen and use a beam splitter to mirror the screen. However with this way there wouldn't be any delay between the screens. I was thinking if it would be possible to somehow incorporate the technique from NFT class for the thesis project. To keep it completely digital i could try to see how well it would translate currently. For an experiment maybe it would be best to do only one "slice".

Window into another location. Use screen with accelerometer to explore a distant location. I think with this, the window would either look directly into another space, so when you tilt the screen the space stays upright (like when you turn your head) or it would affect the space within, when you tilt the screen, objects will follow. This could be completely digital, or it could be a physical installation.

Wifi sniffing. Monitoring all the traffic on an unsercured wifi network. Either show each user connected or the most frequented websites. Visualize or compare this traffic to other locations. Each blob can represent a website visited. For each unique IP adress that is visited the blob grows bigger. Digital version of the previous experiment. I am not sure how legal this is however.

Experiment - morphing object

This project is about connecting locations through a physical, network - like object. Given different locations with proximity sensors and webcameras the object will deform when people approach it. Each location corresponds to a point being pulled. This a take on how we're currently always connected through mobile data / internet. Never at one point in our daily lives are we ever truly disconnected from the world around us. The project takes this idea and visualises it.

The nylon experiment works pretty well. With this one i wrapped polystyrene balls with nylon stocking. This one is pretty close to the original sketch, however unfortunately the motor is not strong enough the pull the fabric that much. Also the backside of the "blob" looks pretty bad, you can see the "pin" on the left side sticking out.

What i like about this one is the ambiguity of it. The black fabric masks a lot of the detail which makes it look like just a black blob. Also at times the silhouette reminds me of heads, like it's morphing between different "faces" / pareidolia. However i wish it was more "floaty", not as rigid as it currently is. This could be solved with more motors and more continuous movement.

The submerged condom experiment did not work that well. I tried three times, with different techniques, but i can't make it work properly.

The first time i tried a double wrapped condom with theaded polystyrene balls inside. The problem with this one was that i didn't secure it on the bottom and the condom was floating on the surface of the tank. The second attempt i tried the same just with threads keeping it in place on the bottom as well. However the effects doesn't work that well. The distortion doesn't work that well also the ink keeps leaking into the tank / while hanging it up. As a result the project has a limited timespan.

The third attempt, i tried to use pieces of tape instead of polystyrene balls as i thought maybe they would "seal" the holes when the thread is kept in place. This is not the case, ink is still leaking.

The silicon experiment worked ok. It's super flexible however the motor is still not strong enough to pull it properly. Also the silicon ended up being way too thick. The test run i did just on the side as i had way too much leftover turned out better than the one in the mold. Material looks very nice however.

I could have put way more tension on the material to make it more three dimensional. Right now its more relaxed / "wavey".