Detecting a Microphone with ActionScript 3.0

For our deadline yesterday I used nested if statements to play and stop animations, and to manipulate their properties (the animations on screen are converted to movie clip symbols). For the final hand up code the movie clips will be removed from the stage rather than hidden from view to try to reduce the odds of animations crashing Adobe Flash during use.

It took several attempts to get this code working, and I am happy with the result.

screen grab from Annie's home computer

Interaction Walk Through
Alternatively you can just watch the video below.   

NOTE: Adobe Flash records the microphone input on a volume of 1 to 100 (100 being maximum volume). To allow for natural ambiance in our exhibition room the interaction is programmed to ignore sounds below 15.

(we may review the figures below pending a microphone test run in the exhibition room)

1. user speaks at a volume above 15: blue bird animation plays

2. user speaks between 25 and 60: butterfly animation plays

3. user speaks between 65 and 100: water and fish animations play

4. blue bird and butterfly no longer play. user speaks above 20: the fish plays (there is a limit of 5 plays to encourage the user to move along to the next interaction. Getting a bit technical... this is also a fail safe in case the user gets bored; we need the water and fish animations to stop in order to keep the total number of running animations on screen at any one time under 6.)

5. when user speaks above 20 for the 6^th time: butterfly and fish are removed from screen, water becomes still.

See this in action for yourself...

Coding the Microphone Interaction from REplay on Vimeo.

screen grabs of the output trace text 

click to enlarge

Adobe Flash CS4 ActionScript 3.0 Code
I'm afraid I'm too busy at the moment to colour code and indent the script, sorry.

import flash.events.ActivityEvent;
import flash.events.StatusEvent;
import flash.media.Microphone;

var count:Number = 0;
var fishCounter:Number = count;
var repeatFish:Number = fishCounter;

var deviceArray:Array = Microphone.names;
trace("Available sound input devices:");

fish_mc.visible = false;
fishFun_mc.visible = false;
blueBirdAnimation_mc.visible = false;
redButterflyAnimation_mc.visible = false;
water_mc.visible = false;

fish_mc.stop();
fishFun_mc.stop();
blueBirdAnimation_mc.stop();
redButterflyAnimation_mc.stop();
water_mc.stop();

for (var i:int = 0; i < deviceArray.length; i++) {
trace(" " + deviceArray[i]);
}

var mic:Microphone = Microphone.getMicrophone();

mic.gain = 60;
mic.rate = 11;
mic.setUseEchoSuppression(true);
mic.setLoopBack(true);
mic.setSilenceLevel(5, 1000);
mic.addEventListener(ActivityEvent.ACTIVITY, this.onMicActivity);
mic.addEventListener(StatusEvent.STATUS, this.onMicStatus);


function onMicActivity(event:ActivityEvent):void {

if (fishCounter < 1) {
if (mic.activityLevel > 10 && mic.activityLevel <= 100) {
blueBirdAnimation_mc.visible = true;
blueBirdAnimation_mc.play();
trace("1. playing Blue Bird");
}

if (mic.activityLevel > 25 && mic.activityLevel < 60) {
redButterflyAnimation_mc.visible = true;
redButterflyAnimation_mc.play();
trace("2. playing Red Butterfly");
}

if (mic.activityLevel > 65 && mic.activityLevel <= 100) {
water_mc.visible = true;
water_mc.play();
fish_mc.visible = true;
fish_mc.play();
count++;
fishCounter = count;
trace("3. playing Fish");
}
}


if (fishCounter > 0 && fishCounter <= 5) {
if (mic.activityLevel > 20 && mic.activityLevel < 100) {
fish_mc.visible = false; fish_mc.stop();
fishFun_mc.visible = true;
fishFun_mc.play();
repeatFish++; fishCounter = repeatFish;
trace ("4. playing repeat fish, #" + fishCounter + " of 5");
}
}


if (fishCounter > 5) {
redButterflyAnimation_mc.visible = false;
fishFun_mc.visible = false;
water_mc.stop();
trace("Microphone interaction complete.");
}

}

The bare bones of this code came from Adobe help files on detecting microphone activity.

Annie.

Testing 1, 2, 3...

A test to see if the animations will run smoothly without crashing Flash

The good news... 16 animations running together did not cause a crash.

Bad news... as you can see, the animations are jumpy.

---

The rest of this blog post is for team members.

Dear Alannagh & Fodhla,

Please take a look at the images below to see what is missing from the master file.

Annie x

for Alannagh

for Fodhla

Arduino - Look Who's Talking

Connecting the Arduino to Flash from scratch was a success :)

First of all, we downloaded and installed the Arduino software and a USB-to-serial driver. The Arduino IDE expects a serial connection to the Arduino. Since our Duemilanove ATmega328 connects to a computer via USB, we need to install a FTDI driver so the Arduino IDE will find it.

1. Code to run on the Arduino ✔

Standard Firmata sketch uploaded in the Arduino IDE so we never have to upload code to Arduino again. Just as well since we don't know Java. Now we will use Flash with ActionScript 3 to send commands that will get the Arduino to do what we need, and visa versa.

'Firmata is a generic protocol for communicating with microcontrollers from software on a host computer.' firmata.org

2. Serial to Socket Server ✔

Serproxy configured correctly to connect Flash and the Arduino board using Arduino2Flash. After editing the serproxy.cfg file with our baud number and serial device name it worked straight away, and displayed a command prompt.

3. Flash Socket Library ✔

Next we need to add a software library to Flash that understands the Firmata protocol so that Flash will communicate with the Arduino board. For this we used the as3glue.

4. Flash Code ✔

The video below features an example that came with the glue library. Our own code is a work in progress.

So, it works! Check it out...

Look Who's Talking from rePLAY on Vimeo.

Many thanks to Jean-Philippe Cote for his step-by-step tutorial.

Annie.

Tweet

Arduino Test #1

Did a test run with the Arduino board and software, everything is in working order :)

Below is a basic blinking led light sketch built into the Arduino open project software.

Next I will learn about the physical interaction capabilities between Arduino and Adobe Flash.  I am working on this tutorial: Interfacing Flash and Arduino from scratch, and learning about Flash and the as3glue library.

Currently reading: Open Softwear for ideas on working with the Arduino board and soft circuits.

Annie.


Tweet

Arduino

Our Arduino board and starter kit arrived a few days ago. 

First step is to download the Arduino Software.

The model of Arduino board used for rePLAY is the Duemilanove ATmega 328 microcontroller. It is a circuit board programmed via USB. I chose to use an Arduino over a keyboard hack because I feel it is well suited to the physical interface of the rePLAY project. It will provide a sturdy framework to the hardware.

Annie.

Tweet

Programming Objective

rePLAY is using the Adobe Flash platform to display a series of interactive animations, that collectively form a forest scene. The scene will be projected onto a wall in an exhibition space. The level of activity in the scene at any one time depends on the end-user. Several elements in the exhibition space will allow a user to control and launch animations. The animations, made in Adobe After Effects, will be imported to Flash and executed via ActionScript 3.0 when triggered by a physical sensor. Animations will sometimes overlap or run simultaneously.

Coding will involve if statements to make decisions based on user activity, for loops to manipulate the animations and audio, timer class code to delay animations and to run them at timed intervals, microphone activity detection and more.

Annie.

Tweet

ActionScript Microphone Activity

We followed this online tutorial today to interact with a microphone using ActionScript 2.0 and it is working fine.

Hiccup: iMac will not detect sound input from a microphone unless the mic is connected via USB. We tried using a Sennheiser mic, with a ¼" to 3.5mm jack converter, connected to the iMac's audio-out port.

The solution offered in relevant online forums is the iMic. In the meantime, for test purposes, we will use my Wii Band Hero microphone.

For this project we are using ActionScript 3.0 in Flash CS4. As a programming foundation to detect microphone activity, we will use source code from Adobe ActionScript 3.0 help files.

Annie.

Tweet