ricardocabellov5.5

As Quasimondo rightly pointed out history keeps repeating. Specially on the web. Over the last decade we've been repeating what off-line graphic programmers have done two decades ago. Even worst, with Javascript we're repeating what Flash was doing half a decade ago.

In the case of Flash, the reason of why history repeated is because Flash is software rendering. Meaning that it's the CPU the one processing all these 2D/3D effects. Meanwhile, all these off-line graphic programmers had been enjoying from hardware rendering where the graphic card is outperforming the CPU in many levels.

In order to avoid history repeat itself we should know history on the first place. Watching what the people have done in the Demoscene, Processing, etc worlds helps. However, it's very tempting to just remake something for the sake of showing that it's possible. And I personally don't see anything wrong with it as it's good for learning. But we should focus on creating new things.

An example of this was back on the early days of Papervision3D. People were complaining that the engine didn't have lights or shadows. Amiga computers neither had enough power to efficiently render 3D scenes with lights and shadows but they managed to fake it by prerendering the shadows in the texture. That was the reason I did the pink ball experiment. I knew the trick because I knew this Amiga demo from 2006.



Now that hardware rendering is getting closer to be default (no plugins) on the web with WebGL, the possibilities are going to increment a lot. But we don't want to have a rotating cube on the screen, we should learn from what all the off-line graphic programmers have been doing with these possibilities and continue from there.

Here it's a selection of hardware accelerated demos from the last decade. I've sorted them (more or less) in chronologic order.

Remember, all these videos run in realtime on their original form. I'm aware that presenting them here in video form devaluates the work but it's just easier to watch.































Oh my... mix all these tricks and aesthetics with interactive experiences, audio visualisation, data visualisation, ... and that's a nice looking future for the web! :D

This Christmas, visiting the family was a bit harder and made me think why we were still in London. I've been in UK for about 7 years already. The experience has been invaluable. Learning the language, meeting a lot of people, work opportunities and a long etcetera. I'm extremely happy I decided to have a part of my life here.

Some months ago, before leaving Hi-ReS!, I had to think what the plan was going to be afterwards. At that point, seemed logical to think that staying in London in case I had to meet someone in town for a freelance job was a good idea. But London was starting to get claustrophobic so we though about moving to a town "close" to London (ie. Bath). At the same time I was getting offers to move to United States which were very tempting, but somehow UK was still winning for being closer to Spain.

Now that I was already freelancing I found myself that there was no real need to stay in London. All the projects I worked on where done from home. That home could be anywhere, so why not moving back to Barcelona instead of to Bath?

We have a lot of stuff in London, so moving is going to be a slow process. Hopefully in April/May we'll be done. There are many people I'll be missing but I have the feeling I'll be visiting quite often :)

Seems like 2010 is going to be a benchmark for us :)

Yesterday night I had the idea of retrieving all my twitter follower avatars and create an image out of it. Not an amazing concept but it was playful enough. I wanted to go down into quadtrees and play with the composition behaviour a bit but that would quite a bit of time and brain energy... I may go into it later.

Long time not sharing the sources in a .zip file! ;) Nothing too interesting tho.

Anyway... Happy holidays and a great 2010!

For a quick project I was asked to revisit my old blobs effect. The client (which, again, sells flavoured liquids) wanted a xmas greetings e-card, so the idea was to play with the liquid that would form specific shapes.

Animating was a matter of using eaze and its beziers, but making it look a bit realistic took most of the research time.

Here it's how the old version of the effect worked:

As I'm sure you had guessed already it's all based from a bunch of circles.



Then we apply a BlurFilter to the Sprite that contains the circles.



And then by copying it to a BitmapData we apply a threshold and a colorTransform.



That's it. Simple enough for a quick experiment...

This i's how the new version works:

What we had so far happened to be a bit too CPU intensive for what it was. If we were going to make it even more complex we had to optimise. We had to remove the BlurFilter step somehow.

Looking at how the Sprite wth the BlurFilter applied looked like, I though that maybe I could recreate a very similar look using pre-blurred circles.


That way, instead of doing a blur pass to a big Sprite, it would be just a copyPixels using this bitmap as source with the positions where the original circles are. The result looks like this:



Which, for the human eyes is very different, but for the computer is very similar. (I could play with brightness/contrast to make it more similar for the human eyes).

Another step for this one was to apply a colorTransform to the working bitmap instead of filling it with white every frame. By doing that we would win "trails" which would improve the liquid effect, and, at the same time doing a kind of brightness/contrast for us.



At this point we apply the threshold again.



This is pretty much like the old effect, but much faster (no fullscreen BlurFilter). Now I could make the effect a bit nicer emulating reflections. I tried many ways but in the end this 2 step process was the fastest and better looking.

First step is to apply a BevelFilter to the output of the threshold.



Doing a BevelFilter is quite slow, but I can't think of a better way.

The fact that we're using reds here is not because the final color is going to be red (which it wasn't for the client). This BevelFilter uses 0xff0000 for hightlights, 0x000000 for shadows and the threshold was 0x800000. That's basically a grayscale but using the red channel. And you may wonder why.

Well, there is this (fast) method called paletteMap that does a very interesting thing. You can map a colour channel to an array of colours. In fact, you can map all of the channels to arrays of colours but I don't know what I could do with that (yet). Lets keep it simple for now.


This is the strip of colours I'm using. Just to make it clear, what paletteMap will do is to replace all the reds with the colours in that strip... as in, 0x000000 will use the colour[0], 0x010000 will use colour[1], 0xff0000 will use colour[255]... and so on... we end up getting this:



Surprised? No? Well... I was. :)

Over the past few weeks I've been busy working on the Starbucks Love Project website for Tool and BBDO.

I had the pleasure to work with a great team: Aaron Koblin, Hello Enjoy, Medios y Proyectos and We Are Mammoth.

During the development of my bits on the site I faced some little challenges that I though I would share so other people could consider my solutions as options for similar challenges.

The website had many release dates. This is something I though it was going to be hard to handle, but the guys at WAM managed it smoothly. However, the project had basically 2 phases.

Phase I - Love Gallery & Drawing Tool

My responsibilities for the first phase were to build the love gallery and the drawing tool.



The idea behind the love gallery was to collect love-related drawings submitted by the people, very similar to Aaron Koblin's The Sheep Market. Opposite to The Sheep Market, in this case there was no limit to the amount of submissions; that was already quite a challenge to start with.

The visualisation part wasn't decided at the start, interfaces like Google Maps (tiling) were considered, but in the end the guys at BBDO decided that it had to be similar to Cooliris. If supporting infinite amount of items wasn't challenging enough now we had to emulate what a some guys have done with OpenGL (hardware renderer) but with Flash (software renderer).

Instead of using Papervision3D or Away3D I decided to use my own engine as it was easier for me to tweak and remove computations that weren't needed. You can't have many 3d planes (specially not infinite) so the way it works is just by having 40x10 3d planes that offset the x positions depending on the camera position. Using their x,y positions I can figure out what's supposed to be their ID and then load the data from the nicely setup VO by the WAM guys. This sounds straight forward, but the code ends up being quite messy.

Loading lots of little images from the server was also another challenge. You spend more time asking the server for an image than receiving the image. To make this a bit more smooth we composed big images (Big Ass Images as the Americans liked to call them) with hundreds of little images.



So with just one request to the server we would receive lots of images inside an image. However, we did some bad decisions here. The dimensions of the big images are way too big (2800x2850) that it takes too much load time and affects the experience. And the one that pisses me off more is that I didn't realised in time (thanks anyway Mike!) that having those big images as .png or even .gif would have not only make them look better but also even have smaller filesize. Look at the images, look horrible! Oh well... next time.

Next was interaction. The people had to be able to select items on the 3d wall. That means projecting 2D points to 3D coordinates. That's something I didn't have on my engine, neither it was something for my mind to easily understand, so I took at look at how Papervision3D and others did it, and when I managed to understand and implement it, it wasn't working 100% well, there were some angle computations that had some delays and time was running out. So I went to the good old Color ID route.

What? Color ID? Yes, Color ID. What we're after here is to know what's 3D object is under the mouse. So in every frame we render the scene twice. Once using a big plane that covers the whole wall textured with a bitmap that has big pixels (that match each plane) and that has a unique color. Mapping the blue component with the X and the green component with the y... we will have... top left plane will have the color 0,0,0, second from the top 0,1,0, fist plane on the second column will have the color id 0,0,1 and so on.



This is how the first pass of the render looks. The user never sees this because as soon as it's done we copy it to a bitmap and then draw the real scene on top. If you look close you can see that the right of the wall is getting blue. That's because the Color ID is reaching 40. So having this in a bitmapData is just a matter of doing a getPixel(mouseX, mouseY) and we easily get the ID by using the coords get get and apply the camera offset. Then we know which plane we should animate/select. No complicated 3D to 2D formulas, just one getPixel per frame. Fast.

(Actually, after writing this I realise it could have been simpler by using the Plane ID as color instead of mapping the blur component to X and green component to Y... live and learn).

There was yet another little challenge. This was more a design challenge than dev, but somehow nobody managed to come up with a scrollbar that could scale to infinite amount of items. Once we started to have 15k drawings, moving the scrollbar one pixel would move the whole wall 100+ items which would mean a lot of movement and specially a lot of bandwidth because the user reached an area were more items needed to be loaded. The solution got implemented at the end of Phase II after I found this article.



The drawing tool was an easier task. Specially because for another project I had a drawing tool started and many of the challenges were already sorted. Explaining all the tricks used on the tool would require an article by itself, so maybe I'll talk about it as soon as I finish the other project.

Phase II - Map & Video Wall

For the second phase the site was going to collect videos sent from many countries where people would sing to the 'All you need is Love' track. (I wonder what's with this song, I'm hearing it in every AD campaign this Christmas, maybe it's an AD trend). My tasks for this one was to make a Map and another infinite wall (this time 2D).



Here I had a good idea. We needed to place arrows where countries are. An easy way would have been doing a little tool so whoever could place them. But if we had to change the shape of the map (which we did), we would have had to remap the arrows. Instead we used longitude/latitude values for each country (which are easy to find out on the net).

Usually there are 2 kind of world maps. One that has the poles distorted (Equirectangular projection) and one that hasn't (Mercator Projection, like Google Maps). The first one is good when mapping a sphere, but please stop using it on 2D interfaces.

Only thing to find out was the formulas to translate latitude/longitude to X,Y. The formula for the Equirectangular one is easy:

private function getPoint(lat : Number, lon : Number, mapwidth : uint, mapheight : uint) : Point
{
	return new Point(((lon+180) / 360) * mapwidth, ((90-lat) / 180) * mapheight);
}

The Mercator one took a bit more to find out, but just for the sake of having it here easy to reuse in AS3:

private function getMercatorPoint(lat : Number, lon : Number, mapwidth : uint, mapheight : uint) : Point
{
	return new Point(((lon+180) / 360) * mapwidth, ((90-GudermannianInv(lat)) / 180) * mapheight);
}

private function GudermannianInv(lat : Number) : Number
{
	var sign : Number = Math.sin(lat) > 0 ? 1 : -1;
	var sin : Number = Math.sin(lat * 0.0174532925 * sign);
	return sign * (Math.log((1 + sin) / (1 - sin)) / 2) * 28.6478898;
}

Having this sorted out, as long as the designers didn't distort the map the mapping of arrows would be always easy task.

At this point I needed to add animations. Coincidentally, eaze was just being released so I used this as a excuse to give it a try. Verdict, perfect.



Then it was yet again going back to optimising a wall that would have infinite entries. However, I didn't stress it too much this time because there were 200 walls instead of just one, so the amount of items would spread along these making it less CPU intensive.

Used again eaze to animate everything and again proven to be great.

All in all, it was quite a big project done in a very short time and I think we were lucky that we all had quite a bit of experience, otherwise we wouldn't had been able to pull this one off.

I couldn't finish this extremely-long post without sending some props to the WAM guys for their DoneDone. tool which I hope they make free for public projects at some point. It proved very efficient. Think of it like the Basecamp of Issue trackers.

It all started with this animated gif I found lurking on the internets some months ago...



I saved it for the day I was inspired enough to make the effect with Javascript. That day was yesterday...



I know I know, this isn't really branching, it's just a disgusting worms kind of effect, that was something I got while playing with it. The code is simple, a bunch of particles and you generate a random value that apply to the direction of each particle (aka random walk). To get branching now you need to randomly generate more particles on the position of these particles. Here it's the result:



I posted the effects over twitter and minutes later @thespite emailed me a modified version of this last iteration.



That was interesting! I didn't know <canvas> had a method for blitting. It gave an effect as if it was algae continuously growing while the camera was moving back. To enhance the effect I changed the path drawing to thicker circles.



That looked nice but was starting to be too visually complex (and cpu intensive). I like the 2D version better and then I wondered how could it work if I wired some of the values to the sound amplitude of a tune and then using <audio> again. Here it's the result.



I spent 10x more time looking for a track that suited the effect than doing the effect itself. In the end I found a nice track at the always-interesting enoughrecords netlabel.

And... that is all for today... as usual, with Javascript, the source code is one right click and one left click away. Have fun!

PS: It was nice to see that most of the effects worked on my Android phone. I guess they also work on iPhone? :D

Skip to the steps.

The iBook G4 was not pleasing me anymore. Because not many Linux distros support PowerPC anymore I was stuck with MacOS (which, remember, I'm not a fan of). So it was time for a replacement.

After looking up some netbooks I decided to go the Asus EeePC route. The 1101HA to be precise. I found it quite cheap for the features it has (Decent resolution, 160gb, 10hours battery... all for £256). Bought it from Amazon and in a couple of days it was on my desk already.



Unfortunately it comes with a bloated Windows XP. I don't mind Window XP, but I don't like having Microsoft Live, Office Student Edition and all the ASUS crap installed. So first thing you end up doing is uninstall all that. Unfortunately again, the recovery mode comes with all that crap. Then I uninstalled too many things and had to learn how to reinstall the system from a USB key.

Anyway, the biggest disappointment was the graphics chip (GMA 500). The hardware side of it seems to be good, but the software side (drivers) seem to be poor. Basically, the chip supports OpenGL 2.0, but the driver doesn't. And you're not able to play HD videos. All because shitty drivers. Lame.

Not much you can do on Windows but wait until they release proper drivers (I won't even go into details of why they are not doing it). So next thing was installing Ubuntu on the side of Windows XP.

I installed Ubuntu 9.10. The process was as simple as it always is. However, once installed, the resolution wasn't 1366x768 as it was supposed to be. It was 1024x768 instead and no option to change it. So I assumed a couple of tweaks needed to be done. Here they are:

Steps

1. Install poulsbo graphics driver. Just use the PPA script from here.
2. Replace NetworkManager with WICD. With NetworkManager, the wireless stops working randomly. WICD can be easily installed from Synaptic Package Manager.

And that's it. The result is a very robust system. With 2D (HD videos play smoothly) and 3D Acceleration (OpenGL works). Funny that, for first time, Linux performs better than the OS that came with the device. Things are changing!

Thanks to HIDIHO! blog I found out about a new Tween library developed by Philippe Elsass.

Yes, yet another Tween library. My favourite at the moment is BetweenAS3 and I though no library could change that. But this one brings a fresh syntax.

eaze(target).to(duration, { x:dx, y:dy })
    .onUpdate(handler, param1, etc)
    .onComplete(handler, param1, param2, etc);

I really missed the (not so old) Zeh Fernando's Tweener days. When everything was simple and clear, without twists and clubs... BetweenAS3 gave me back these days, but it was good to read that Phillipe shared my thoughts.

Now, lets end the drama and start to have fun again!
http://code.google.com/p/eaze-tween/

As always, you just need a bit of practice with a language to start using it in nice ways. Now that I had that little 3D engine working and with the <audio> around, it was time to produce an idea I always had in mine. A 3D interpretation of a waveform.

I'm sure the first thing you would think after checking this experiment would be... What? I didn't know I could analyse the sound signal with the <audio> tag?! ... Well, you can't, if you check the source code (*hint* right click -> view source) you'll see an array of numbers (deltas in fact). These are all the sound level values of the waveform at 30 fps.

I got these values using the library BASS for linux. Unfortunately, my C skills aren't so good (example) and I can't seem to control how to get the values exactly at the fps I want without getting desyncronisations. The first part of the visualisation is spot on, but by the end things aren't that impressive. I'll keep researching on this and update a new array of values whenever I crack it.

Dean McNamee showed me the way. Forget C and BASS. All you need is python, the tune in .wav and 23 lines of magic.

import math
import struct
import wave
import sys

w = wave.open(sys.argv[1], 'rb')
# We assume 44.1k @ 16-bit, can test with getframerate() and getsampwidth().
sum = 0
value = 0;
delta = 0;
amps = [ ]

for i in xrange(0, w.getnframes()):
	# Assume stereo, mix the channels.
	data = struct.unpack('<hh', w.readframes(1))
	sum += (data[0]*data[0] + data[1]*data[1]) / 2
	# 44100 / 30 = 1470
	if (i != 0 and (i % 1470) == 0):
		value = int(math.sqrt(sum / 1470.0) / 10)
		amps.append(value - delta)
		delta = value
		sum = 0

print amps

I've updated the experiment with the new values; now is perfectly in sync all the time. Which makes you appreciate the little sounds at the end much more. Thanks Dean!

As you'll see on the source code, the rest is very simple. Create a couple of cubes, place them one in from on each other, and modify their scaleY depending on the waveform they are related to in that step.

The end result is quite interesting I think and I hope doing more like these. Hopefully more interactive next time.

The factor that turned this from just a nice experiment to an awesome was the fact that the eedl guys allowed me to use their (great) tunes for my experimentation.

Thanks once again!

Seems like I'm still hooked to Javascript. For the last few months, on my spare time I've been toying more and more with it, creating little pieces that will serve as personal benchmarks for browser performance improvements.

For the next few links I recommend using a WebKit based browser otherwise your browser may crash.

The first idea I wanted to try was creating a canvas using checkboxes as pixels. Then display animations with it. Similar to textmode renderers.



I posted the link over twitter and minutes later Aaron was sending me a drawing done with the checkboxes by Valdean Klump. On firefox, the experiment didn't worked correctly and it just created a grid where anyone could pixelate in. Because this, I created drawing tool out of it. I though it would be fun if I could also generate links on-the-fly so people could share their checkbox based drawings easily. This is how it ended up:



(Press Shift for the eraser tool)

Some days after, Joa Ebert ported a Strange Attractor code to Silverlight to compare performance with Flash. I knew that Javascript <canvas> was going to be way slower than these but I was curious how much.



I believe the code can get some performance optimisations for the platform but my interests were to compare exactly the same code. 7 F/S seemed a good start.

At this point I felt it was about time to go back to toying with 3D worlds. Many people were working on engines using <canvas> as the renderer. But I though I could try using <svg> as renderer instead.

Differences between <canvas> and <svg> are pretty much the same as BitmapData and Graphics. With <canvas>, performance decreases exponentially by the size of the viewport. It's easier to fill the pixels of a triangle at 10x10 than 1000x1000. With <svg> the bottle neck is the amount of vectors being drawn. Because they don't get rastered on a bitmap the nodes stay in memory and exponentially gets slower. However, viewport size isn't a much of a problem here, which ticks my fullscreen action requirement.

How do you draw with <svg>?

Good question. A <svg> is basically a XML. Every frame you fill it with nodes for each polygon with their properties. Then, on the next frame, you delete all the nodes and repeat the process. Sounds crazy, but it works. Here is the proof.



As you've probably ovserved, a benefit of <svg> that you can disable antialiasing (which you can't with <canvas>). This thing alone makes <svg> outperform <canvas>.

Now, don't get the wrong impression. At the moment I don't even know if texturing is possible with <svg>, I haven't investigated that yet. Something tells me that, if possible, it'll go horribly slow.

While working on all these I came up with some things I wanted to do that flat colors were good enough. One of these was a QR code in 3D. Now that I own an Android phone I keep seeing more and more QR codes. The shape of a QR code seemed like a little city and extruding it was something that excited my mind (yes, I'm weird).



I've learn quite a bit just for doing this piece. Inkscape for vectorising, Blender for extruding and colorising, Python for exporting from Blender and refactored my engine once again. Performance wise, its like going back to the Actionscript 2 days. But, hopefully, WebGL will arrive soon and I'll be able to play with more polygons. Until then, I can create a bunch of things just with these.

Feel free to study the sources for each piece if you are interested. If you find something wrong or that can be improved, please, let me know.