title

Showing posts with label experiment. Show all posts
Showing posts with label experiment. Show all posts

Tuesday, March 1, 2022

Experiment - merge / traffic networks

The goal of this project was to explore how to merge or fuse together real time video streams. It's about exhausting a scene down to colors and movement and repurpose this into an abstract representation. When info has been stripped away, the scenes can be merged together. First two networks are both a direct representation of the flow of traffic at different locations. The third data network is an attempt at trying to merge and fuse these two together.

It's a proof of concept of trying to analyse and gather info from live streams. To develop this further i would've done this with a camera where visitors can see themselves / their surroundings being analysed and taken apart and "merged" with other locations. As this is only a proof of concept I chose to analyse an already available live stream with a steady, continuous flow of movement. The live streams are taken from spatic.go.kr

This experiment grabs two random street cams around Seoul. Traffic from each camera is analysed and the color of each car registered is extracted and displayed as a small part of an abstract network. Traffic cameras are passive and running 24 - 7. As a result it will give different results based on what time it is. The third network represents the merge of the different scenes.

Analysis of scene -> breaking down each scene -> Merge of two scenes.

In the first two "networks" the x, y - position are chosen at random, the z - position is decided by the saturation of the color. In the third network x is position is controlled by brightness, z position by saturation and y is randomly chosen.

Issues
Originally I wanted to use traffic cameras / live streams from different parts of the world, e.g. Norway and Korea, but unfortunately I couldn't find any high quality streams from Norway. As a result it uses different traffic cams around Seoul instead.

Unfortunately the tracker is not that accurate. It's using contour / difference tracking via. Open CV where it looks for pixels that differ from a set background image. As a result it will also pick up light differences such as the headlights of the car. I realised halfway through the project that a more effective / accurate approach would be to use a machine learning model with TensorFlow trained on specific scenes. If I had to redo this project I would've gone with this approach instead / Google's "Teachable Machine" project.

Another issue is how there is not a limit to the amount of vertices. Ideally it would be time based, where every x seconds a vertex is deleted from the network. This way the network would be more representative of the current traffic as for example when there is a red light, most of the vertices would be deleted before the light becomes green.

Top left - original live stream, top right - grey scale copy to threshold, bottom left - background image, bottom right - threshold that is used for tracking.

Sunday, January 30, 2022

Experiment - sound landscape

Generative landscape based on sound frequency. Similar to a Spectrogram that shows how the spectrum of frequencies varies over time. The idea of this project was to explore audio based visuals. This could be an installation that shows a collection of these "scenes". Each visitor will generate their own audio signature and it will be displayed as part of a collection. To make the results more varied sentiment analysis is used to change the colors based on your mood, volume controls color intensity etc.

The connection is about feeling part of a community. People can make their own and compare and contrast with others' models. e.g. when other's models looks similar to yours in color, intensity etc. you can guess that they answered the question in a similar way.

The effect is shown here with music instead of speech as its an effective way to demonstrate how it works.

This project started initially as a tool that would "loft" together audio FFT waves, which would result in something that would resemble a landscape. In the final experiment i chose to present it as a circle to mimic more traditional audio formats such as record players. It also makes it more "model" - like in the way that it's not just a bar of "mountains".

This is how it initially worked. Just tried to recreate the sketch. Every x seconds a copy is generated and animated in the z - direction. In the final project it's using a data texture on a flat plane instead, which is basically a texture made up of raw data. E.g. the FFT size controls the resolution of the texture / how many columns (low FFT results in more voxel - style displacement) and the height controls how long it will be / how many rows (how long the animation is). By offsetting each row of data every time the audio analyser updates, the displacement will be animated in the z - direction.

To analyse the sentiment / polarity of your speech it has to go through a few loopholes. There are APIs out there that can do this based on the tone of your voice etc. but as these cost money i tried to go about it another way. The AFINN dictionary is a set of words where each word has a corresponding score ranging from -5 to 5. By first using the web speech API, your speech is transcribed to text. Then each word from this transcription is checked in the AFINN dictionary where a score is calculated. This is by no means an accurate approach to analysing your emotions (only gives a sense of positivity / negativity), but for shorter intervals of text it gives ok results.

Of course having the polarity value shown is redundant as the color change is more than enough. Random Kanye interview for demonstration purposes, shows transition between colors.

Also while making this, I realized just how awkward it is to do something sound reactive as it so easy to resort to just primal sounds instead of just talking naturally. As a result participants should be given a simple question to answer, such as "How is your day?".

However, in hindsight i think a much better sound project could be about doing - position -> echo.

Or using sound to perform 3D reconstruction, e.g. make a super low fidelity 3D scanner / point clouds with sonar sensors.

Thursday, December 16, 2021

Experiment - Intimacy without proximity.

Thermal energy / heat is transferred from one location to another by particles bouncing into each other. All matter contains energy. The more energy that is present, the hotter the object will be. With this project I wanted to explore how to extend this process. Where you can be present but distant at the same time / be close to someone without being physically present. This project illustrates physical presence through a heat reactive model. When the sculpture lights up you know that someone is present.

This project builds on the previous body heat experiment and the bilocation - laser cutter experiment. With this experiment the user receives feedback on their actions which results in a more satisfying interaction. They can see the sensor analysing the footage and also see the model light up through a live web camera setup. This project is also more passive in the sense that you don't need to approach the sensor that directly. Just passing by is enough to make the model light up.

In hindsight i wish that i tried to explore the way i presented the feedback a little more. E.g. maybe it would be better to have another projection with the sensor data and the web camera rather than keeping it on a small screen. This way it could be more immersive and I would have more space to play around with.

Another way to do this project would be to have two models in separate locations, where your current presence is added on top of the distant presence. E.g. other people's presence is represented with white light while your local presence is represented by another color. That way your presence can be overlapped at the same time. That way it really gives more of a sense of being "together" in the same location. An alternative way for interactive telecommunication.

With this type of project, I think i should spend the next few weeks deciding on what type of visuals i should use e.g. projection mapping on model / physical computing, sound etc. as well as if it should be two identical models / objects or one big object and one portable etc.

Friday, November 19, 2021

Experiments - Body heat

I don't think this turned out that good as it is not really that clear that it's about transferring / extending your body heat. Seems more like a proximity project. In hindsight i should've projected into the corner and cropped the video in a way that it feels like it fills the entire room (more similar to the second part of the setup video). It doesn't work that well on the pillar. However it was a nice experience to try and set it up. I thought i had thought of everything before projecting, but i should've made it easier to fill the entire surface with one color.

Ended up using the metal frame for the sensor as i think it shows the interaction more clearly. Also the sensor's range is not that far unfortunately.

This was such a quick and dirty setup with acrylic plates i found laying around and tape to hold it all together. With more time i would've laser cut a plate that fit the metal frame and sensor better. Ideally it would be better if i could set it up so that the projection was in one room and the sensor in another. To make it more clear that it is supposed to connect locations together.

This project also lacks feedback on the initial interaction. E.g. when you put your hand on the plate you don't get any feedback about what is happening. It should have another projection near the plate that is either similar to the projection on the other side of the pillar or a view that tells you your body temperature / the temperature that the sensor registers.

Friday, November 12, 2021

Experiment - bilocation

Translating position in space into paths. This experiment explores the idea of bilocation, being present in multiple locations at once. I chose to track the dog in this clip because it has the most interesting movement. It pokes its head around more, creating a more interesting pattern. However I think it would look pretty nice if all the movement in this clip was turned into paths and shown in real time / layer the laser cutter recordings on top of each other.

Friday, November 5, 2021

Experiment - morphing object

This project is about connecting locations through a physical, network - like object. Given different locations with proximity sensors and webcameras the object will deform when people approach it. Each location corresponds to a point being pulled. This a take on how we're currently always connected through mobile data / internet. Never at one point in our daily lives are we ever truly disconnected from the world around us. The project takes this idea and visualises it.

The nylon experiment works pretty well. With this one i wrapped polystyrene balls with nylon stocking. This one is pretty close to the original sketch, however unfortunately the motor is not strong enough the pull the fabric that much. Also the backside of the "blob" looks pretty bad, you can see the "pin" on the left side sticking out.

What i like about this one is the ambiguity of it. The black fabric masks a lot of the detail which makes it look like just a black blob. Also at times the silhouette reminds me of heads, like it's morphing between different "faces" / pareidolia. However i wish it was more "floaty", not as rigid as it currently is. This could be solved with more motors and more continuous movement.



The submerged condom experiment did not work that well. I tried three times, with different techniques, but i can't make it work properly.

The first time i tried a double wrapped condom with theaded polystyrene balls inside. The problem with this one was that i didn't secure it on the bottom and the condom was floating on the surface of the tank. The second attempt i tried the same just with threads keeping it in place on the bottom as well. However the effects doesn't work that well. The distortion doesn't work that well also the ink keeps leaking into the tank / while hanging it up. As a result the project has a limited timespan.

The third attempt, i tried to use pieces of tape instead of polystyrene balls as i thought maybe they would "seal" the holes when the thread is kept in place. This is not the case, ink is still leaking.

The silicon experiment worked ok. It's super flexible however the motor is still not strong enough to pull it properly. Also the silicon ended up being way too thick. The test run i did just on the side as i had way too much leftover turned out better than the one in the mold. Material looks very nice however.

I could have put way more tension on the material to make it more three dimensional. Right now its more relaxed / "wavey".

Friday, October 29, 2021

Experiment - morphing object

This project has been a failure so far. I've tried a few different variations with different materials, none of them really working. Ideally i would use motors to move the magnets / pull the strings.

Just condom filled with water and magnet inside burst immediately. Same with balloon and magnet. Wrapping multiple balloons inside each other solved this problem but then i couldn't fill it with water anymore as it was "too strong".

When i put the magnet inside a polystyrene ball and double wrapped the condom it worked ok. It's pretty durable but no interesting shapes occur. Idk if maybe this is because i didn't hang it tight enough. As you can see in the video it gives in pretty fast and i can move it around. Also the magnet needs to be pretty close for it to work. Ideally, i wouldn't have to move the magnet inside the box at all as it "breaks" the feeling a bit.

I also tried to fill a condom with only polystyrene balls and threading a couple of the balls to then pull from the outside. In this case the condom needs to be hanged very tightly as you have to pull the string pretty hard for it to displace properly. Also not super durable, the condom broke pretty fast in this case.

Maybe I could make a tesselation surface instead and use strings to pull it?

Maybe this could be one of the sound experiments. Using a LED strip above water, then use ripples to generate sound. Alternatively the Media Art project from last semester could be turned into an audio project where the height of each ball controls sound.