ricardo cabello

Or so they say... (javascript demo)

Sun, 25 Jul 2010 16:45:00 +0000

This weekend Euskal took place and for quite a while I was considering doing a demo for it, but it wasn't until Friday night that I finally got free time for doing it (and the deadline was Saturday evening...).

Now that three.js was starting to get stable and also thanks to some sequencing code I had done some months ago I had no excuses to get working on it. So, from Friday midnight until Saturday afternoon, I managed to get this:

That's just a video of the demo, watch the actual demo.

Quite amazing how quickly you can get stuff done with javascript once you have the basic libs ready ;)

I need to check what's wrong with Opera, for some reason is not cleaning the screen. Apart from that it should work in all browsers (albeit quite slow in some, specially MacOS ones). Oh, I think it wouldn't work on Safari either, as it seems to be the only modern browser that doesn't support .ogg files.

In case anyone is interested, I've also shared the source code.

EDIT: Found the problem with Opera. Turns out context.clearRect() doesn't work if the context has been transformed with negative values. I'll let them fix their bug ;)

Multiuser Sketchpad

Wed, 07 Jul 2010 17:22:00 +0000

Here it's a project I made over the past weekend. I warn you, it's kinda weird. (Chrome 4+, Safari 5+, Firefox 4+ only)

You'll find some more iterations on my twitpic.

Now, lets give some context...

Nerd talk

Over the past few weeks I've seen people over twitter talking about this node.js thing. Because I've been quite into Javascript this year last week Jaakko was asking if I had played with node.js yet. I still didn't know what it was but he was kind enough to explain.

So node.js is like Apache but super optimised and lightweight, the coolest part is that you code your server in Javascript. Besides a small problem I had to deal with it was really easy to compile and install on my Ubuntu system.

Then there was this other hype thing, and that's websockets which basically is an open connection between the client and the server. You can send data to the server via websocket without having to send additional headers for each request.

So Jaakko persuaded me to try doing a node.js + websockets + harmony experiment. I was going to spend the weekend doing the WebGL renderer for three.js (which I did yesterday), and my mind was in another place, but somehow he convinced me ("fucking do it now" may have been the key ;D).

Ok, there are a bunch of node.js libraries to handle websockets. At first I was using socket.io but it seemed to use a lot of cpu. After a looking at the code a bit we find out that they JSON all the data that gets sent all the time. That can be useful but in this case it was quite a bit of overhead (and I think they're working to fix that). So I ended up using miksago's node-websocket-server which broadcasts the data to all the connected users as it got sent.

At this point is when you just need to sit down and get the thing working. Once it was working locally I got a slice in slicehost and in a matter of minutes the experiment was live. Considering that it costs $20 for 150gb of BW and that I received €60 from flattr last month I though it was worth spending that money on this project. (EDIT: At this rate I suspect I'll end up spending the full €60 on this...)

The first version of the code was pretty much brute force, every user was sending their X,Y position every time they moved their mouse. Yes, that's *a lot* of data per second. There has been 50gb of data transfered in just 4 days :P However, yesterday night I finally got a bit of time and improved a bit the data that gets transferred, and not only it consumes less bandwidth, the server's CPU is much happier (btw, thanks sole for the top tip :D).

Monitoring interactions

Leaving the technicalities aside, this experiment is very different from everything I've done until now. I've never seen how the people interact with my experiments. The feedback I get from them are emails, comments on the blog or tweets. Which basically reduces to a couple of sentences defining how the experience was. Which is good, but it's impossible to get a clear impression of the experience of the user.

With this one, I can see the people interacting. I can see people experimenting and writing scripts and doing unexpected things with the system (which were fun, but please, don't). I was even able to see people trying XSS exploits in realtime.

Considering how easy it was to put this together I won't be surprised if in a year or two, most of the websites use websockets to monitor how exactly their users navigate the site and improve the UX in realtime. And this is just an exmaple, imagine the possibilities!

Human behaviour

As expected, people like to draw male reproduction organs all over the place, and an annoying one was people scribbling on top of nice drawings. It was so annoying that I even had to tweak the brush algorithm to discourage such actions. The faster you move the cursor the less opacity the pencil has. Stupid solutions for stupid problems.

However, when humans want to collaborate the experiment becomes super fun. If you manage to get in a time that there isn't much people (rooms next?) and there is a collaborative person it can be great. For instance, the screenshot at the beginning of this post, I did some of the drawings on it in collaboration with a stranger, First I did a square with eyes and mouth, then he did another one on the site as if they were two squares that were about to collide and then he started doing the animal bits on the new one which I also followed by doing a joke to a classic internet image. It's also great when 4/5 users get organised and draw something together. 3-in-a-row games are also great :D

Last but not least, it's amazing to see, not only how one user interacts with the tool, but how they interact with the other people. That creates many games, they even create their own social rules for the place. And if that wasn't enough, I find it amazing to see how the cursors behave like "humans". If you put your cursor on top of another cursor, that other cursor is likely to get away. Like when someone you don't know get too close to you. The most interesting case was "Talentina", which I suspect is a little girl. She seemed happy and was drawing cute things like a giraffes and so on, but then someone will arrive and draw a penis on it. She would move the cursor away, like looking from the distance, and move away like in a slow/sad pace...

And lets not forget the fact that the history doesn't get saved, if you refresh you'll get a blank canvas. That's also a very strange behaviour to get used to, but people seem to be turning it into a feature :D

Unexpectedly fascinating.

EDIT: Running MacOS? Need a new screensaver? Cédric Raud has done something for you.

http://hxt.fr/vrac/Multiuser%20Sketchpad.screensaver.1.1.zip

Also... from "ops" in the comments, a recording of a session at 4000x :O

stats.js

Wed, 07 Jul 2010 14:18:00 +0000

Some of you may have realised already that I've been using a Javascript FPS counter on some of my experiments.

Originally it had exactly the same look as the Actionscript version but, as far as I know, you can't query the system to get the amount of memory the application is using from Javascript. So 2 of the 4 common values of the Actionscript version were empty. That made me reconsider the design and tried to improve it a bit. And after using it for quite a while I think it's ready.

As you can see it has two modes, FPS and MS. Just click on the widget to switch between them. I may change MS to use greens instead of cyan to match the Actionscript version a bit more. Thoughts?

Unfortunately this time the code is not as simple as the Actionscript version, but I think it's still simple enough.

var stats = new Stats();
parentElement.appendChild(stats.domElement);

setInterval(function () {

    stats.update();

}, 1000/60);

Find the source code, compressed file and more examples at the project page on github.

Javascript plane deformations

Mon, 14 Jun 2010 03:56:00 +0000

It's not the first time I mess with plane deformations, I also did some Pixel Bender experiments with them some years ago.

The principle of this technique is to apply a formula to every pixel of the output bitmap. If you know about shaders of Pixel Bender you know how this works.

Depending of the formula you're going to deform a source texture creating different forms. Javascript seems to be quite fast doing Math computations, the bottleneck, as usual, is drawing the pixels to the bitmap. So I thought it was worth porting the code and the formulas in order to get a base for it and have yet another experiment to test the new browser versions with.

Once again, using shaders we'll be able to do this kind of effects with much better resolution and speeds as soon as WebGL gets enabled on browsers by default.

Until then, feel free to take a look at the code and mess with the formulas.

three.js r8

Mon, 14 Jun 2010 03:41:00 +0000

Since I made three.js public I've been refactoring a bit some parts and experimenting a bit with potential performance improvement tricks. I won't go much into details, but maybe this snapshot will give you some idea of what I tried.

So that's nice and all that, but I was starting to get tired of flat colors so I decided it was time to try to implement textures and uv mapping. I took a look other engines to see how they were implementing these and the one that seemed the simplest and fastest was the one in pre3d which was done by Thatcher Ulrich. That also needed a bit of refactoring because I was storing the UV data on each triangle. Instead, it's better to store the UV on the geometry level so you can have different UV mappings (also called channels) for the same geometry.

Once I got that working it was time to do a bunch of demos and see how the browsers were able to handle it. Coincidently, Apple published their infamous HTML5 some days before.

From all the demos I got mostly interested on the VR one:

I know, I was also excited when I saw that message. Anyway, I managed to try the demos in Safari and this demo is pretty much a panorama. 6 planes textures and the camera inside. For that they used the CSS transforms which are only available in Safari at the moment. So I wondered if I could do that same effect using three.js and the new feature.

Turns out you don't need Safari to do a panorama. It's not as fast (not hardware accelerated), but hey, it works. I wasn't too happy about these textures and I looked for better ones for a proper demo. This is how it ended up:

That's better. Certainly slower as it's fullscreen but it has potential.

Next thing to try was using <video> as a texture for a plane. Adding a fake reflection on the floor for extra niceness.

Although it may seem much more advanced it's not. We just need to draw the <video> element to the texture applied in every frame.

Last, but not least interesting, was a Terrain Generator demo. The code to generate the height map and texture is not too different from the code I wrote some years ago while trying to do a raycaster. Only this time I needed to generate the texture instead and let the engine do the rendering part. After a couple of iterations [1, 2], this is the result:

I know, is so slow it's pointless, but I'm having fun doing all this anyway. I hope browsers will keep improving their implementation of <canvas> and maybe all this will have some potential. However, I have to say that I think a lot of things can be done with this. People have done amazing things with even more limitations.

Ah... so tempting... :P Anyway, I better start implementing a WebGL renderer to the engine.

Flattr

Tue, 08 Jun 2010 05:59:00 +0000

As you may have seen, I've recently added a few Flattr buttons to some of my experiments.

Flattr? What? Here here:

I really hope it to succeed. It's like the old (and not so old) days, when the artists were funded by the people that enjoyed their work.

You may wonder, what was wrong with Paypal donations? Well... Paypal it's a bit like having to go to the bank every time you wanted to donate a coin.

Here it's my profile over there.

Harmony ports

Thu, 03 Jun 2010 22:40:00 +0000

Seems like Harmony has had a fairly amount of success since it was released. Nowadays, you just need to search on Facebook for fan pages created by others about any or your works to know if it was successful or not. In this case I found 2! :)

So I can't be happier about it. With the release I also decided to share the source code hoping that people would play with it, create more brushes and in the end improve the application.

Instead, people started porting (or encapsulating) the code on mobile platforms (iPhoneOS / Android) which I also found great. Here it's a list of all the ports I'm aware of:

Paid

AbtMagicSand Classic by Ultie, Inc. (iPhone OS)
ASKetch Pad by AKERN Software (iPhone OS)
Harmonious. by The Angry Robot Zombie Factory (iPhone OS)
LiveSketch by Yan Rabovik (iPhone OS)
LiveSketch HD by Yan Rabovik (iPhone OS)
OmniSketch by Richard Taylor (iPhone OS)
Touch Sketch by Luksor (iPhone OS)
Really Big Pictures HD by Zeno Machine (iPhone OS)
Qvik Sketch Pro by Qvik (iPhone OS)

Free

Draw(er) by Dansl (Android)
MiniTones by Banana Coding (iPhone OS)
Qvik Sketch by Qvik (iPhone OS)
Sketcher by wargoth (Android)

If I have forgotten any app, feel free to share it on the comments.

EDIT: I've modified the post to focus on the part of listing all the adaptations and removed the part about people behaviour using the open source code. Joa made a very good point in the comments: "Why should an app not be open source if it is completely based on other open source ideas?". I'll take that in consideration from now on.

Water and Parallax

Thu, 03 Jun 2010 05:11:00 +0000

This one is a really old effect that I always wanted to do. It has two part, first the 2d water effect, and then the parallax effect.

Water Effect

It all starts (once again) from this page. On there you'll see a "magic" trick to simulate 2d water. I've implemented that code in Actionscript and even as a Pixel Bender Shader but unfortunately never got something interesting out of it. Some months ago I revisited it again and made a Javascript version which has been on my site for a while.

At first it was a bit boring in black and white, but with a bit of experimenting I found out this color offset that made it look more interesting.

Parallax Effect

While working on the water effect I somehow remembered HeartQuake by Iguana. A MSDOS demo from 1994. One of the effects in that demo consisted in a 2'5D liquid surface.

(You can see the effect on the 5:32 mark)

I've seen that demo many times, but only this time I had enough knowledge to start guessing how each effect was made. The effect looked like the water effect but used as elevation map, maybe using voxels for creating the 2,5D illusion. I tried the voxels path but it was too slow for Javascript, so I forgot about it and left it there.

This week, for some reason, I ended up at Petri Leskinen's blog once again. On this visit I realised he shared a very interesting snippet. It seemed like a very fast technique to simulate 2'5D terrains. I quickly tried it out to see how fast it performed in Javascript and the results seemed pretty good (18ms for creating the perlin texture, 17ms for the elevated one).

So from there it was a matter a putting everything together and playing a bit with the code until I found an interesting level of interaction. The text idea seemed to work quite well, it had very mermerising motions so I went forward with it also as a bit of homage to Iguana's demo.

There are still a bunch of oldschool effects I want to try using <canvas> before I jump to WebGL shaders. As always, feel free to abuse of the beauty of Javascript: Right click > View Source :)

JavaScript: The Good Parts

Tue, 27 Apr 2010 19:19:00 +0000

Today neave was sharing this interesting talk over twitter. I wish I would have seen it before... now I'll need to go back and fix most of the JavaScript I've written :S

three.js

Sun, 25 Apr 2010 10:22:00 +0000

I think it was in 2004 when I started investigating, learning the maths and testing all this stuff of projecting 3d points into a 2d plane... Two years later, in 2006, I had a torus knot with flat shading with a light and a camera. Then Papervision3D came into the scene and I got distracted ;)

Two years later, in 2008, I reworked my code with the things I liked from Papervision3D but making it simpler to my mind. At that point I was calling the engine threedoob and I did a bunch of experiments with it. Later I ported the code to Javascript and also did a bunch of experiments with that version too.

Now I think I've reached a point in which I'm not ashamed of sharing the code and because doesn't seem to be much resources for rendering 3D with HTML5/Javascript I thought other people would find the javascript port useful.

So I've created a repository for it at github. It's still very early days, but feel free to take a look at it, use it, fork it, ...

Moving to github

Sun, 11 Apr 2010 17:40:00 +0000

I've asked many times to Sole about the benefits of using git and every time I thought it wasn't worth the learning time. I was happy with svn, but for Harmony I thought about giving github a try as Google Code didn't seem to make collaborations and forking too easy, plus github worked better with the way I was using it (different projects in the same place).

I guess Google Code works well for big projects, but in my case I do small projects/snippets, and creating a new repository/page for each one was a bit annoying as they also check if a project with the same name exist on sourceforge, and if it does you need to email the owner for permission first (although my project wouldn't be hosted on sourceforge but on Google Code).

Anyway, here it's the link for my github page:

http://github.com/mrdoob/

So far I'm liking it, and I've started moving my projects there. First one to move was stats.js, but expect the stats.as, videoplayer and so on to be moved soon. I've found github a bit hard at the beginning (ssh, keys and so on) but once that part is done is quite simple. Oh, and it's great to see the .svn folders go away!

Wacom Javascript API

Thu, 01 Apr 2010 23:45:00 +0000

Ever tried to make an online drawing application and got frustrated because you couldn't access the Wacom tablet pressure? I know the feeling :)

Although I'm a bit surprised of how little publicity this thing had (none?) seems like Wacom has had this ready for quite a while (Windows/MacOS only) and with the latest drivers it should all work smoothly.

If you have a Wacom tablet, take a look at these demos. To learn how to integrate it into your application, here it's the info but I found easier just to take a look at the sources of this demo. If you're a Flash/Flex developer, external interface is your friend, at least until Adobe implements it within the player.

Of course, I've implemented it already in Harmony ;)

Making of The Johnny Cash Project

Wed, 31 Mar 2010 02:03:00 +0000

And you may be wondering, where does all this thing about drawing tools come from? Good question!

Although I've always had interest on developing a drawing application, it all started when Aaron Koblin asked me if I would be up for developing a drawing tool for a Johnny Cash homage project he was planning with Chris Milk. I think this was about 6 months ago, even before the Startbucks Love Project (where an alpha version of the drawing tool was used).

Before passing away in 2003, Johnny Cash recorded one last album which was never released, until now. The idea was to build a platform where everyone would be able to draw a frame for the musicvideo for one of the tunes.

The website (front-end and back-end) was built by @radical.media so my work on it gets simplified to the repainter and painter system.

Repainter

One of the requirements for the drawing tool was to record how the frame was being drawn so the visitor on the site could see a timelapse. For example, here is a drawing I did:

There is small aspect ratio bug on that one, but you get the idea...

Doing this took a bit of time, but luckily I had learnt a few tricks some weeks before after trying to port the Colors! Java Repainter applet to AS3.

Creating a file format turns out to be quite simple. Just a matter of deciding a few tags (size, color, opacity, position...) and deciding how many bits you need for each value. For instance, I have STROKE_START which has the id 4, we do bytearray.writeByte(4) and then we save the positions of the mouse, but because these values can be bigger than a byte (256) we need to use int bytearray.writeInt(x) and bytearray.writeInt(y). We do the same for the rest of data and at the end we use bytearray.compress() which is pretty much a zip. The end result is a very simple and optimised file format, faster to parse than a xml and much smaller.

Having a hexadecimal editor is a must here (I used GHex). Here is a screenshot of how the uncompressed file format looks like:

If you know about file formats, I'm sure you already know what's going on, otherwise there you have a quick challenge for lunch time ;)

Painter

Having the file format sorted out I "just" had to develop the drawing tool itself. This may sound an easy task but, starting with the premultiplied alpha nightmare (more info) it has a bunch of little challenges... undo/redo, custom brushes, zoom, panning, brush outline... it just takes its time.

Using the original frame of the video as onion skin we can get nice results in a few minutes.

Now I can't wait to see how the project evolves and hopefully seeing the video without missing frames soon!

Harmony

Mon, 08 Mar 2010 02:48:00 +0000

This is the result of all the weekend playing with various drawing algorithms and <canvas>.

I'm sure you prefer playing with the thing than watching a video tho... Here here!

The whole thing is quite modular so I can keep adding more brush styles whenever I get inspired. During the process I found out that, for some reason (apparently lack of hardware acceleration), Firefox and Opera do not support context.globalCompositeOperation = 'darker'. This was on the HTML5 spec before but got removed. Just so you know what I'm talking about, this is like the "multiply" blending in Photoshop. Webkit does support it tho. I hope they put it back on the specs and all browsers support it.

I went to #processing.js to ask about this and while there sephr, apart from answring this same question, suggested making the app work on Webkit mobile (that means Android and iPhone) and also pointed me to the right direction. I didn't know this was possible at all but it was quite an easy task :) I haven't implemented multitouch support yet tho... that changes a bit the way my system work but I'll put it on the list.

Take a look at the code, play with it and if you come up with a nice brush you think should be added to the default pack, please, send it over!

EDIT: Thanks to Elke for creating a Facebook Fan Page. There is also a Facebook Group :D

EDIT 2: Yet another Facebook group. RRR, with a theme everyday and really talented artists.

César Alierta y las anotaciones de Youtube

Thu, 11 Feb 2010 00:07:00 +0000

Ayer, en el perfil de Marc de Facebook, me tope con esta perla.

Sole, al verlo esta mañana, comentaba que echaba de menos poder poner comentarios en tiempos especificos, como en soundcloud.

Dicho y hecho!

Seguramente el video terminara con demasiadas anotaciones pero por ahora el experimento ha valido la pena :D