Inside/Outside awareness (day 101)

I created another test of the lungs breathing.  The breathing pattern is the calm breathing that I had worked out with the egg.  The videos are the same two videos of the pelicans and garden.  One video plays on the lungs and the other on the trachea and bronchi.  The lungs fade out at the top of the in breath and fade back in at the bottom of the out breath.  The video slows down at the in breath and goes to normal speed at the out breath.


I was trying to emulate a feeling I have when I meditate of the out breath being a little anxious with only a slight pause at the top, and the out breath feeling more relaxed with an extended pause before the next in breath.  In that pause there is lots of space and a feeling of being more aware of the outside.  This is why I ended up with the fade in at the bottom of the out breath.  I initially had tried it the other way (fade in at in breath) but it somehow didn't work. 


The next step is to get the video to slowly come in to the trachea/bronchi as the breath comes in.  I have to write a small shade to do this.  I'm hoping to have some time tomorrow.


Here is a video.  Unfortunately it's only 2 minutes long.  It's really  nice to keep breathing with the lungs.  Very calming.  Next time I'll make the video loop.  It's too late right now to rectify that little oversight.



fade out lung test from Maria Lantin on Vimeo.


A song for this post.

Feel the lungs (day 59)

Today I had lunch with Joy James and we talked about the Breath I/O project.  She asked me some interesting questions about the type of affect we would like the piece to have.  Why do what we are doing?  How will people feel the experience of the lungs and the video and the sound and the interface?  As she was talking and asking questions I could feel my mind going from the how to the why.  I am so often preoccupied with the technical details of doing something.   It felt nice to speak of the audience, to picture them in the room,  to imagine the sounds they might hear.  At first I pictured only one person entering a large dark room (the mocap studio actually) where the sound was louder than the picture.  The sounds were enveloping, breath like, rhythmic and calm.  There are several places to sit down near the ground and in front of each seat there is an object, each one different.  The lungs are in a chorus formation breathing in sync, the video interchanges between them.  The person picks up an object and holds it.  One of the lungs gradually takes on more importance visually and the video is more consistent and clear.  The sound of the video is heard over the breath.  The singled out lungs have more personality and are not so in sync with the rest of the chorus which has faded to the background.  The lungs are reacting to the video that is playing within them, sometimes sighing, sometimes coughing, sometimes fast breathing, sometimes deep breathing.   The object vibrates in a association with the lungs.   When more objects are picked up more lungs approach and start to interact with each other and the objects. 

The feeling of the environment to be calm and conducive to a reflection on the bittersweet nature of life,  the life cycles, the exchanges we have with people, the constant give and take of life.  The preciousness and sadness of being human.  The joy of movement and breath, of health.

Thanks for the talk Joy.

A song for this post.

Hello lungs (day 57)

I took some time today to render the lung model to see if I could get the transparency to work correctly and maybe play around with the video mapping.  I got as far as getting the transparency to work and then ran into a resolution problem.  To my chagrin, Touch Designer will only render to a maximum horizontal resolution of 1280...unless I pay $599 for a commercial license (per machine).  I've asked if they have educational pricing and we'll see.  It's strange that I hadn't notice this before when I rendered to the screen in the lab.  But looking back, it does explain the slightly squished look of the lungs.   Even at the slightly lower resolution, the lungs look pretty good.  I'm really starting to get fond of the shape of the lungs.  They're almost friendly and sweet.  Here is a pic (which is down-res'd):

The grey scale give it a medical X-ray look.  I wanted to keep it neutral until the video is added to see if it has a fluoroscopy look.  We'll see.  Trent is currently working on the texture coordinates.

I also spent some time learning Field which I can run now that I have my Macbook Pro.  It feels like learning a new language even if the component elements (Java, Python, OpenGL) are familiar.  It feels to me like a tool I could seriously love after spending some hours with it.   It reminds me of the old Unix days with lots of command line and immediate feedback.  It also makes me a bit worried that I would write something in Field and a few months later remember nothing of the mental model that led me to a particular design (i.e. more than one way to shoot yourself in the foot).   But this minor worry is not going to stop me.   I'll just trust that I knew what I was doing at the time I wrote the program which, believe it or not, is hard to do.  The belief that I am smarter in the present, so smart in fact that I know better than the previous self who was immersed in the code, is always lurking.

A song for this post.

Stereo ahoy! (day 35)

This week Leila and Thea were here from Montreal to work on the Breath I/O project.  It was worth the stress of carving time from a busy schedule to work on the project in a more sustained way.  The main thing we are trying to work out is a workflow for the output of the Sony HDR-TG1 cameras we are using to shoot stereoscopic footage.   We had great hopes for the little Sony's and I think they will work out in the end but we've hit many walls along the way.  The TG1 is ideal for us because of its small size.  We can mount two TG1s beside each other on a Slik twin camera mount and shoot stereo at varying inter-ocular distances.   Miles built us a remote that controls both cameras so we can start/stop and zoom them in tandem.  Another ideal aspect of the TG1 is that they record surround sound.   With such a small microphone we wondered how good the sound would be.  It turns out to be quite good!   Overall we were pleased with the output of the camera but we started to hit some snags when it came time to edit the video.  The footage is in AVCHD format and the sound is Dolby Surround.   Here is what we know so far:

• The Picture Motion Browser software that comes with the camera will export the video to mpg2 or wmv with surround sound but not full resolution (it downgrades to 720x480)
• Final Cut Pro downgrades the audio to stereo
• Adobe Premiere can import the mts with surround sound and full res, but cannot output surround sound.  We did find a plug-in that may help but it's $295 and at this point we haven't given up on a cheaper solution.  CS4 says it comes with a trial version of said plug-in but we don't seem to have it.
• Stereoscopic Player does not play the surround sound (this is just a minor irritant since eventually we'll be playing the stereoscopic footage in a virtual environment)
• Interlacing is an issue.  The TG1 records at 1080 60i and needs to be deinterlaced to 1080 30p.  

So we currently don't have a workflow that preserves both resolution and surround sound.  The belief that this it must be possible keeps us searching.

Despite these setbacks we had some nice stereoscopic results with the footage that Leila shot of her nephews in track and field, and hockey.  The twin camera mount needs a level so some of the footage  had some vertical disparity but we were able to fix that in post-processing.  We tested the stereo footage on the old lung prototype and it looked interesting.  It kind of looked like the lungs were transparent.  Not exactly what we were looking for but perhaps with a little bit of a bumpy surface on the lungs, they won't look so mirror-like or transparent.

Another highlight was Trent's new model of the lungs.  They look great!  With any luck we'll be working with these on Monday (Leila's last day).

Miles was also around working with different sounds.  Making soundtracks on the fly for the silent videos we were playing.   We talked about different ways of teasing out the deeper resonance of someone voice in real-time.  He showed us an effect in super-collider which may be the start of what we're looking for.

A song for this post.

Curious noise (day 20)

Today Miles and I talked about what an environment of mixed soundtracks might sound like.    In Breath I/O the videos and their associated soundtracks will 'swirl' around in the environment surrounding the lungs, mixing with each other.  When there is no video playing on the lungs it makes sense for the sound to be noisy with a slight foreground of interest, a hint of some of the soundtracks.   As more videos are playing, the noise retreats more and more to the background but is still present.  There is a seamless blend between when the soundtrack is fully audible and when it goes back as part of the noise.   All of this is in five channel surround sound.   One of the things we talked about is that the noise would be algorithmically composed of the granules from each soundtrack.  Much like the video pixels are being used to shade the atmosphere surrounding the lungs.  I like that parallel a lot.  Miles had some great examples of composers who work with noise and subtle foregrounds of interest.

One of the aspects of the atmosphere that we still need to develop is whether it will be influenced by some kind of action.   We've even talked about the possibility of the virtual affecting the real by being able to generate air currents in the installation space.   Personally I think it would be ok if the environment was like weather, unpredictable and interesting.  Something the lungs would be subject to.

A composition for this post (courtesy of Miles). 
More of Rosy Parlane can be found on myspace.