Well, I haven’t been posting here because I’ve been putting all of my free time and energy into my marriage, my house, and my grad school.  My, my my… Seems a little self-centered when I put it that way (and maybe it is).

My wife is helping me clean up my office.  I might be a bit of a pack-rat especially when it comes to things that might be useful in the future (like my friend Kevin who has a collection of disassembled printer gears, rods, and other things).  Basically this means that I keep all papers (never know when you might need to refer back to that information!), all outdated technology (why throw away a perfectly good coax cable?), and anything else that might cost something to replace (I know I have 50 different stacks of half-used post-it’s… but they’re useful).  However, with my beautiful wife’s loving encouragement, I’m throwing away old papers, parting with old technology, and organizing everything I’m going to keep.  I will admit, my office is getting neater.  And that’s a good thing.  It’s more functional this way, though I’m sure (if I wanted to – not that I have) I could argue that my office was functional before the cleaning as well.

On the grad school side of things, I’m updating the blog regularly now that I’ve started to build my actual project.  My project is building a game using a certain exploratory methodology.  You can read more about it if you want to.  But basically, I have to document every day I work on the project so I have it all archived when I write my thesis paper (if you have to spend 50 pages writing about a single focused topic, it had better be something you like and you had better document all of your sources).  Some people might question why I chose to create a game for my thesis when all my background and focus is in interactive media design and development.  The short answer to that question is basically that I see an important language emerging from this current / the next generation which is heavily based on gaming.  I think there are some important issues around gaming (doing good, reward systems, critical thinking) and I think gaming and education relate to each other on multiple levels (beyond the obvious).  I am one of those who believe all games teach (thank you Raph Koster) and as a teacher, I need to study this form of teaching.  Additionally, I’m using my interactive skills (project management, Flash / AS3 Coding, design, art) in producing this game, so in that way it’s directly applicable.  My hope is that this process will make me a better, more relevant teacher in years to come.  All that said, I also enjoy games immensely and I see game design as an artform.

So that’s my update for this quarter… ha!

On a side note, Josephine Leong and SuAnne Fu (both at SCAD) are really excellent teachers.  I’ve had a couple teachers in grad school that I haven’t really understood (which makes me more empathetic to my own students) but those two are really amazing.  I respect them very much.

I launched a new generative art project.  This one is vastly less involved than some of my others, but I still like it.  Check it out.

So I thought I might share something I learned today about Arrays in Flash AS3. In most languages I’ve used, it’s fairly simple to clone an array. For instance:

In Javascript:

1. var x = new Array(‘Bob’,'Tom’,'Joe’);
2. var y = x;
3. y[1] = “Stan”;
4. // x contains Bob, Tom, Joe
5. // y contains Bob, Stan, Joe

In PHP:

1. $x = array(‘Bob’,'Tom’,'Joe’);
2. $y = $x;
3. $y[1] = “Stan”;
4. // $x contains Bob, Tom, Joe
5. // $y contains Bob, Stan, Joe

However in Flash you would get:

1. var x:Array = new Array(‘Bob’,'Tom’,'Joe’);
2. var y:Array = x;
3. y[1] = “Stan”;
4. // x contains Bob, Stan, Joe
5. // y contains Bob, Stan, Joe

What? Why? Because Flash automatically creates a reference when you assign arrays to each other, it does not clone the arrays like the other languages. You can do something like that in PHP with the &= assignment operator, but it’s not the default. Why would Flash not do the same as PHP? How does it make any sense that when I assign one thing a value that the two values should then be tied together automatically?

There is a solution. The Flash help system actually talks about this problem here. So the new code would look like:

1. var x:Array = new Array(‘Bob’,'Tom’,'Joe’);
2. var y:Array = x.concat();
3. y[1] = “Stan”;
4. // x contains Bob, Tom, Joe
5. // y contains Bob, Stan, Joe

Basically, whenever you use the concat method it automatically returns a new array (rather than a reference), which solves the problem. But, if you have a multidimensional array, this method does not work. Concat does not touch the arrays within the arrays – it only converts the initial values in the base array. Fortunately Flash’s help system recommends the following code (which works):

1. import flash.utils.ByteArray;
2. function clone(source:Object):*
3. {
4. var myBA:ByteArray = new ByteArray();
5. myBA.writeObject(source);
6. myBA.position = 0;
7. return(myBA.readObject());
8. }
9. var x:Array = new Array(‘Bob’,'Tom’,'Joe’);
10. var y:Array = clone(x);
11. y[1] = “Stan”;
12. // x contains Bob, Tom, Joe
13. // y contains Bob, Stan, Joe

So I’m glad to have a solution, but it seems awfully weird to have to go to so much trouble…

Previously: Playing a Note of a Certain Frequency

So now that we have a pitch, how do we know how long a sound will be?  One second of audio generated in Flash requires 44,100 pieces of sample data.  So if we’re writing 8192 samples each time we call our function, we would have to call our function at least six times.  However, if we call our function six times, we end up with 49,152 samples – which is more than one second of audio.  So we have a problem.

Some of you might be saying, well why don’t we call our function 6 times, but drop the number of samples we write each time to 7350 – that way we have the perfect number of samples to make 44,100.  Your thinking is correct for this example, but there is at least one problem with that scenario.  What if you want to produce a quarter-second sound?  A half-second sound would work just fine, we would call our function three times instead of six.  But it is impossible to call a function one and one half times.  So what can we do?

My suggestion is to keep the sample size at 8192 and clip the audio where you need it.  If we know one second is 44,100 samples, we also know 11,025 is a quarter second (44,100 / 4 = 11,025).  In audio, a sound with an amplitude or volume set at 0 is silent.  So to play a quarter-second sound I would call my function twice which will produce 16,384 samples.  I only want to play 11,025 samples of real audio though, so I’ll let the sine wave write the first 11,025 samples, then I’ll fill the rest of the sample data with 0s.  Those 0s will be represented as silence and I’ll get a note that is only a quarter second long.

For example, consider the code:

1. var mySoundLength:int = 11025;
2. var mySound:Sound = new Sound();
3. function sineWaveGenerator(event:SampleDataEvent):void {
4. var frequency:Number = 440;
5. for ( var c:int=0; c<8192; c++ ) {
6. if(c+event.position < mySoundLength) {
7. event.data.writeFloat( Math.sin( Number(c+event.position) * (frequency * 2 * Math.PI) / 44100) * 0.25 );
8. event.data.writeFloat( Math.sin( Number(c+event.position) * (frequency * 2 * Math.PI) / 44100) * 0.25 );
9. } else {
10. event.data.writeFloat(0);
11. event.data.writeFloat(0);
12. }
13. }
14. }
15. mySound.addEventListener(SampleDataEvent.SAMPLE_DATA,sineWaveGenerator);
16. mySound.play();

In line 1 we set the length of the sound we want to play.  In line 6 we’re checking to make sure our current position within our audio is less than our sound length.  If it is, we’re writing our audio as normal.  If we’re past the length of the note we want to play, write 0 volume to our SampleDataEvent.

Next up: Playing a Note Without the Hideous Popping or Clipping

Previously: Overview of Generative Audio in Flash CS4

To play a normal sine wave of a specific note (or frequency) is pretty straight forward, but it’s different from what Adobe would tell you. Building on Adobe’s version, take a look at this:

1. var mySound:Sound = new Sound();
2. function sineWaveGenerator(event:SampleDataEvent):void {
3. var frequency:Number = 440;
4. for ( var c:int=0; c<8192; c++ ) {
5. event.data.writeFloat( Math.sin( Number(c+event.position) * (frequency * 2 * Math.PI) / 44100) * 0.25 );
6. event.data.writeFloat( Math.sin( Number(c+event.position) * (frequency * 2 * Math.PI) / 44100) * 0.25 );
7. }
8. }
9. mySound.addEventListener(SampleDataEvent.SAMPLE_DATA,sineWaveGenerator);
10. mySound.play();

In line 3 we are now setting our frequency or the note we want to play. 440 hz is middle A. On lines 5 and 6 we are taking that frequency and multiplying it by 2 PI. We then take the result, multiply that by our current position in the sample then divide all of that by 44100 – which is our samples per second. The rest you should recognize from the previous example.

So to play any note, you would just change the frequency to what you’re looking for. Wikipedia has a good quick reference of piano key frequencies (http://en.wikipedia.org/wiki/Piano_key_frequencies). Middle C for instance, is 261.626 hz – so that’s what you would put in your frequency variable.

Next up: Playing a Note of a Certain Length

Adobe Flash CS4 (Flash player 10) introduced the ability to generate audio in real time.  The ability, however, is a little confusing and there has not been much good information released on the subject.  There are the basics like “this is how you produce a tone” or “this is how you link the tone to your mouse x/y position”, but not much has been written on how to produce actual notes, on key, with a given length -so I’m hoping to break that down a bit.

Audio generation in Flash currently centers on the SampleDataEvent.  If you want to play a sound generated dynamically, Adobe’s help suggests something like the following:

1. var mySound:Sound = new Sound();
2. function sineWaveGenerator(event:SampleDataEvent):void {
3.   for ( var c:int=0; c<8192; c++ ) {
4.     event.data.writeFloat(Math.sin((Number(c+event.position)/Math.PI/2))*0.25);
5.     event.data.writeFloat(Math.sin((Number(c+event.position)/Math.PI/2))*0.25);
6.   }
7. }
8. mySound.addEventListener(SampleDataEvent.SAMPLE_DATA,sineWaveGenerator);
9. mySound.play();

Line by line, here’s what’s going on:

1. Create a new sound object and store it in a variable of the data type Sound with the name mySound.

2. Create a function named sineWaveGenerator. The function will take one argument, the SampleDataEvent that will be stored in a variable named event.  The function will not return any values so we declare it’s return type as void.

3. Set up a for loop.  Initialize a variable named c of the type integer with the value of 0.  Each time we loop we’re going to make sure c is less than 8192 (which means we’ll loop a total of 8192 times since we’re starting at 0 and going up to 8191).  Each time we’re finished with our loop we need to increment c which will add 1 to whatever the value of c is at the time.  The reason we loop for 8192, is that’s the maximum amount of data that can be written to a sound at one time.

4. Take our SampleDataEvent, get the data, and write a floating point decimal to that data.  The data we’re going to write  over time will be a sine wave.  The sine wave will be generated by taking the position in our audio sample and adding our variable c to that position, then calculating the sine.  Multiplying by .25 reduces the volume or amplitude of our audio sample to ¼ what it was.  If we don’t do this, Flash will produce a very loud tone.  All of this in line 4 is done for the right channel.

5. Do the same as line 4, except for the left channel.

8. Take our sound object, stored in the variable mySound and add an event listener to it.  Listen for a SampleDataEvent.SAMPLE_DATA and when we hear that event, run sineWaveGenerator once.  The event will be triggered when we try to play our sound – each time we need more data to play (if our sound keeps playing), we dip back into our function and get more audio data.

9. Tell the sound object, stored in the variable mySound to play.  When it tries to play there will be no sound data.  It will then trigger our event listener, which will call our function, which will write data to our sound object, which will then be played until it runs out of data, then the process will be repeated.

Adobe’s version above works, but it’s limited and they don’t give you a good explanation of what’s happening.  Even with the explanation I’ve given above, you would not know how to create a specific note, or how to tell the sound to last a certain amount of time.

Next up: Playing a Note of a Certain Frequency

Audizer: Image to Sound Translation

I’ve completed my first final project for grad school.  Audizer: Image to Sound Translation centers around extracting qualities and colors from an image and creating a song based on that data.  The project is available here: http://audizer.deptof.com.  Check it out.

The project makes extensive use of the SampleDataEvent in Flash’s Actionscript 3.  I had some problems with it but eventually got it working.  I’ll try to find the time to create a post centering around the AS3 code behind the project as there’s not much data to be found concerning generative audio in Flash.

Theo Jansen rocks.  I bought his book The Great Pretender to read partly for pleasure and partly for a class I’m in.  I’m writing a paper on Jansen and 2 other sculpture artists.  It’s very interesting reading.  He has used genetic algorithms, among other things, in developing his “beach animals“.  I love it.  I’ve done some experimentation with genetic algorithms myself, though I have yet to use them as effectively as he has.

On a reading break today I checked Facebook (sigh) and AID alumni Chris Griffith posted a note about “one of the best Flash games he’s ever played,” which caught my attention.  It’s Record Tripping by the Bell Brothers.  Go check it out.  Very interesting stuff.

I’m still banging away in grad school.  I’ll be taking 15 hours next semester.  Fortunately two of those classes are programming classes and *should* be a cake walk.  We shall see.

A generative art experiment with Genetic Algorithms and Color Transformations

As I increase in my understanding of number systems the world of programming opens up more and more.  I just realized (tada!) that the max number of color variations in our traditional web system is 16777215.  16777215 is equivilent to #ffffff.  So when you give Flash a color like 0×000000, it’s really just converting it to 0.  Alternatively, if you give Flash #ffffff, it’s converting the color to 16777215.  So if I want a random color in Flash it’s as simple as Math.round( Math.random() * 16777215 ); and if I want to transform a color over time, I only need to increment that variable my color is stored in.  This is a really simple realization that most people already understand, but for me it’s opening up a whole new world.

As for binary, I’m using a lot of it.  I’m encoding paths for genetic mutations, neural networks rely on binary, etc.  I keep finding new ways to use binary – ways that surprise me, but that make sense.  I hope my students in Programming Logic listen when I talk about Binary and Hex numbering systems.

Well, I got everything sent off to apply to grad school (SCAD) this week.  Now I’m waiting for some kind of word back.  Though I doubt they’ve received the materials yet as I sent them Wednesday.  Just nice to have everything out the door.

In the process of writing and thinking about my process and purpose I have decided to start carrying a notebook again.  And, mind you, not a computer but a real live notebook with a pen and everything.  I think I was at my most creative when I did, and I need to return to that mentality.

In other news, I’ve been working with genetic algorithms for the first time.  It’s a combination of binary, Flash’s drawing api, evolution, etc for the sake of creating something beautiful.  Right now it’s totally separate from the artificial neural network development I’ve been doing the past couple years, but there my eventually be a convergence.

I’ve added a link to the right for the portfolio I sent in to grad school.