Mobile Interactions Video Updates

More about The Fish technology.

More about The Fish design.

More about The Butterfly design and technology.

From REplay on Vimeo.

Windmill & Mobile Interaction Progress

Myself and Siobhán working on the windmill interaction.

Windmill Interaction from REplay on Vimeo.

Siobhán & Fodhla brainstorming and testing interaction designs

Fodhla, Niall, Alannagh & Siobhán, (Annie's behind the camera)

Design and technology for the butterfly interaction on the mobile.

Pendulum Design from REplay on Vimeo.

Fodhla & Siobhán

Detecting a Microphone with ActionScript 3.0

For our deadline yesterday I used nested if statements to play and stop animations, and to manipulate their properties (the animations on screen are converted to movie clip symbols). For the final hand up code the movie clips will be removed from the stage rather than hidden from view to try to reduce the odds of animations crashing Adobe Flash during use.

It took several attempts to get this code working, and I am happy with the result.

screen grab from Annie's home computer

Interaction Walk Through
Alternatively you can just watch the video below.   

NOTE: Adobe Flash records the microphone input on a volume of 1 to 100 (100 being maximum volume). To allow for natural ambiance in our exhibition room the interaction is programmed to ignore sounds below 15.

(we may review the figures below pending a microphone test run in the exhibition room)

1. user speaks at a volume above 15: blue bird animation plays

2. user speaks between 25 and 60: butterfly animation plays

3. user speaks between 65 and 100: water and fish animations play

4. blue bird and butterfly no longer play. user speaks above 20: the fish plays (there is a limit of 5 plays to encourage the user to move along to the next interaction. Getting a bit technical... this is also a fail safe in case the user gets bored; we need the water and fish animations to stop in order to keep the total number of running animations on screen at any one time under 6.)

5. when user speaks above 20 for the 6^th time: butterfly and fish are removed from screen, water becomes still.

See this in action for yourself...

Coding the Microphone Interaction from REplay on Vimeo.

screen grabs of the output trace text 

click to enlarge

Adobe Flash CS4 ActionScript 3.0 Code
I'm afraid I'm too busy at the moment to colour code and indent the script, sorry.

import flash.events.ActivityEvent;
import flash.events.StatusEvent;
import flash.media.Microphone;

var count:Number = 0;
var fishCounter:Number = count;
var repeatFish:Number = fishCounter;

var deviceArray:Array = Microphone.names;
trace("Available sound input devices:");

fish_mc.visible = false;
fishFun_mc.visible = false;
blueBirdAnimation_mc.visible = false;
redButterflyAnimation_mc.visible = false;
water_mc.visible = false;

fish_mc.stop();
fishFun_mc.stop();
blueBirdAnimation_mc.stop();
redButterflyAnimation_mc.stop();
water_mc.stop();

for (var i:int = 0; i < deviceArray.length; i++) {
trace(" " + deviceArray[i]);
}

var mic:Microphone = Microphone.getMicrophone();

mic.gain = 60;
mic.rate = 11;
mic.setUseEchoSuppression(true);
mic.setLoopBack(true);
mic.setSilenceLevel(5, 1000);
mic.addEventListener(ActivityEvent.ACTIVITY, this.onMicActivity);
mic.addEventListener(StatusEvent.STATUS, this.onMicStatus);


function onMicActivity(event:ActivityEvent):void {

if (fishCounter < 1) {
if (mic.activityLevel > 10 && mic.activityLevel <= 100) {
blueBirdAnimation_mc.visible = true;
blueBirdAnimation_mc.play();
trace("1. playing Blue Bird");
}

if (mic.activityLevel > 25 && mic.activityLevel < 60) {
redButterflyAnimation_mc.visible = true;
redButterflyAnimation_mc.play();
trace("2. playing Red Butterfly");
}

if (mic.activityLevel > 65 && mic.activityLevel <= 100) {
water_mc.visible = true;
water_mc.play();
fish_mc.visible = true;
fish_mc.play();
count++;
fishCounter = count;
trace("3. playing Fish");
}
}


if (fishCounter > 0 && fishCounter <= 5) {
if (mic.activityLevel > 20 && mic.activityLevel < 100) {
fish_mc.visible = false; fish_mc.stop();
fishFun_mc.visible = true;
fishFun_mc.play();
repeatFish++; fishCounter = repeatFish;
trace ("4. playing repeat fish, #" + fishCounter + " of 5");
}
}


if (fishCounter > 5) {
redButterflyAnimation_mc.visible = false;
fishFun_mc.visible = false;
water_mc.stop();
trace("Microphone interaction complete.");
}

}

The bare bones of this code came from Adobe help files on detecting microphone activity.

Annie.

Keyboard Hack Working

Heres a video demonstrating its all working!

Keyboard hack button from REplay on Vimeo.

Testing 1, 2, 3...

A test to see if the animations will run smoothly without crashing Flash

The good news... 16 animations running together did not cause a crash.

Bad news... as you can see, the animations are jumpy.

---

The rest of this blog post is for team members.

Dear Alannagh & Fodhla,

Please take a look at the images below to see what is missing from the master file.

Annie x

for Alannagh

for Fodhla

Crochet Flowers Video Clip

Flowers from REplay on Vimeo.

Tweet

Animation

Here is an animation of how the project is looking so far...

Fodhla.

Tweet

Sea Meets Sky

This is a mock-up demo of a mobile that will hang in our exhibition space. The actual mobile will be a wooden structure measuring approx. 1½ x 2½ feet.

The mobile will have three interactive elements hanging among strings of yarn. They will be in the shape of a butterfly, a star and a fish ...I have yet to make them... and they will work as a pendulum; the swinging arm will complete the circuit in our technical design. To make the elements sturdy the shapes will be cut from plastic canvas. Plastic canvas is perfect material for this because it won't curl like knitted shapes do, and a flat surface is crucial to the pendulum design.

To see an example of cross-stitch on plastic canvas click here.

The butterfly, star and fish elements will allow the user to control our project as an alternative to using a keyboard and mouse. We will make them quite decorative and beautiful with crochet designs in sparkly muted coloured yarns.

Sky Meets Sea from rePLAY on Vimeo.

Water droplets, waves, and clouds; sky meets sea.

the mobile mock-up

luxurious popcorn wool for the mobile

Annie.

Tweet

Bloomin' Lovely

The first flowers bloom in REplay Forest...

My family lent a helping hand tonight.

my auntie knitting with Tivoli Waves yarn

Mam crocheting a flower in chunky yarn

Well done ladies, 15 flowers made tonight :)

Take a closer look at tonights progress...

Flower for Set Design from rePLAY on Vimeo.

The dimensions of the exhibition space will determine the amount of flowers needed to decorate the grass bank, I am aiming for one-hundred.

Annie.

Tweet

Arduino - Look Who's Talking

Connecting the Arduino to Flash from scratch was a success :)

First of all, we downloaded and installed the Arduino software and a USB-to-serial driver. The Arduino IDE expects a serial connection to the Arduino. Since our Duemilanove ATmega328 connects to a computer via USB, we need to install a FTDI driver so the Arduino IDE will find it.

1. Code to run on the Arduino ✔

Standard Firmata sketch uploaded in the Arduino IDE so we never have to upload code to Arduino again. Just as well since we don't know Java. Now we will use Flash with ActionScript 3 to send commands that will get the Arduino to do what we need, and visa versa.

'Firmata is a generic protocol for communicating with microcontrollers from software on a host computer.' firmata.org

2. Serial to Socket Server ✔

Serproxy configured correctly to connect Flash and the Arduino board using Arduino2Flash. After editing the serproxy.cfg file with our baud number and serial device name it worked straight away, and displayed a command prompt.

3. Flash Socket Library ✔

Next we need to add a software library to Flash that understands the Firmata protocol so that Flash will communicate with the Arduino board. For this we used the as3glue.

4. Flash Code ✔

The video below features an example that came with the glue library. Our own code is a work in progress.

So, it works! Check it out...

Look Who's Talking from rePLAY on Vimeo.

Many thanks to Jean-Philippe Cote for his step-by-step tutorial.

Annie.

Tweet

Arduino Test #1

Did a test run with the Arduino board and software, everything is in working order :)

Below is a basic blinking led light sketch built into the Arduino open project software.

Next I will learn about the physical interaction capabilities between Arduino and Adobe Flash.  I am working on this tutorial: Interfacing Flash and Arduino from scratch, and learning about Flash and the as3glue library.

Currently reading: Open Softwear for ideas on working with the Arduino board and soft circuits.

Annie.


Tweet

Abstract Inspiration

Links to existing projects that came up during Interactions class today:

• Flash projects by artists, designers and hackers within the openFrameworks community
• A blog for inspiring things: Digital Tools
• the HYPE framework, a creative coding framework built on top of ActionScript 3.0

Go look at the animation on the HYPE framework and then come back to me. Click here.

There are no images used in the animation, it is entirely a coded graphic. You will notice if you refresh the page that it starts as a blank canvas every time. That is pretty cool. This shows that programming for multimedia can be as creative and expressive as any other design medium; for example graphic design.  

Just the other day I was looking at the screensaver on my iMac ...and with rePLAY in mind... I was thinking how onscreen movement through colour and light could represent physical activity between a user and our onscreen elements.

Some short clips of the screensaver

1) Slow, defined shapes. Movement becoming more regular.
possibly user has just started to play

2) colour explosion
user really getting into our interactions

3) movement reverts to slower, formed shapes
maybe user has stopped playing

Audio added to this mix could explode the design into a whole other dimension.

rePlay started out on an abstract path, however, it has grown into more defined shapes and objects as we cement our idea, skills and resources.

Annie.

Tweet