[DDM] Node JS and AIM Client

It wasn’t easy. It was frustrating. But at least I can see the result. It wasn’t bad, well it’s working but I’m not easily satisfied with the result.

Okay, these past two days I’ve been working the client side of my project. At first I want to do it with just simple HTML and Javascript. But it turns out I can’t send OSC message directly from web browser. After hours searching about this problem, most of the solutions that work involving Node JS as a bridge server. I never use Node JS before but because this is the only (easy) way to solve my problem then I have no choice.

Learn Node JS wasn’t easy but was’t that hard too, well at least for me :p. Using Express web framework, I manage to create both client (i.e. the thing runs using web browser) and the bridge server all entirely in Node JS application. The client itself just basic interface for the application setting. Image below is the screenshot of the client

AIM Client

I also manage to integrate this with my Pure Data patch and of course it works. I still need to make the client response better and also need to clean up my patch. It still a long way to go but I’ll manage ^_^.

Btw, maybe you notice that I mention “AIM” in this blog title and you’re right, it is my project name, Angklung In Motion. Basically Angklung In Motion means Angklung using Leap Motion :D.

Advertisement

[DDM] Lucky

Today I found something that can be very helpful for my project. In my last blog post I wrote that I need to find a velocity of some point. Luckily after I read the documentation and read the developer forum, I found out that the library already provide finger velocity with this attribute I don’t need to calculate it my self and just use it. Well it doesn’t mean I don’t need calculate another thing, it just make my life easier :).

With Leap Motion data plotter plugin I can see that I have to find some threshold for this velocity so I can somehow use it to determine the amplitude of the sound output whether it faint, normal, or loud.

[DDM] Hello Leap Motion

For my project I will use Leap Motion controller as my motion sensor. For this purpose I have to learn about Leap Motion itself and of course how to code or create application using it. Leap Motion SDK available for a lot of programming languages. For my project I choose LeapJS, Leap Motion javascript library. Why I choose javascript because it can run in almost all modern browser so it will be easy for client.

I have to choose which hand and fingers that I will use, what information do I need, what data I can get from Leap Motion, etc. After some time poking around its documentation, I found some useful information on how to use it for my need. Image below is what Leap Motion see from its perspective.

Hand image from Leap Motion

I make this hand form because this form is the closest one with the actual thing when you hold Angklung. From this image you can see that there are three fingers that I can use as a reference. These fingers are the most visible from leap motion point of view and after some trial that I do, I choose thumb as my reference. Why thumb? It was because pinky finger can be mistaken as other fingers and sometime index finger not really visible when I shake.

I can’t stop here, I have to choose which point/position that I will use in thumb. According below image from Leap Motion documentation there are 5 (five) position that I can use and another 3 (three) bone type that I can use too (because I can get its center position).

I want to use “btipPosition” but I think it can be unclear. I want to get the closest point to “btipPosition” and the closest point is the center point of Distal Phalanges bone. So I decided to use the center point of Distal Phalanges.

To get this point/position is not that difficult in code, the challenge now is how to get the movement velocity of this point/position. Using this velocity then I can define some shaking parameter for my project.

[DDM] Between Leap Motion’s Range

Before I use Leap Motion as my project’s motion sensor I have to know Leap Motion’s sensor range. This range will determine how Leap Motion will be used in my project. After I read some article about it, I found out that Leap Motion have 150 degree angle of wide and 120 degree angle of deep [1].

Sensor range [1]

And with these information we can get roughly 24 inch wide and 13 inch height [2].

Area calculation [2]

With around 20 inch deep we can use 5 inch (12 cm) for each Angklung so we will get 4 Angklung notes in each controller (Leap Motion controller). I think 5 inch is sufficient for user to interact with.

[1] http://blog.leapmotion.com/hardware-to-software-how-does-the-leap-motion-controller-work

[2] http://leapgamer.com/blog/25/6_tips_for_effectively_using_the_leap_motion_controller

[DDM] Why Leap Motion

For my project I need a decent motion sensor that can detect not just my hand but also my fingers. Aside from Leap Motion, I already “played” with two motion sensors.

Intel Real Sense – I used this device only for competition. It’s a sensor camera from Intel that can detect hand, gesture, face, etc.

Microsoft Kinect – I used this device for my past DMSP project. As you may know, Kinect commonly used for a game in combination with Microsoft XBox. It can detect not just hand but also your full body.

I choose Leap Motion not just because it’s small and compact (so it can address my portability issue) but Leap Motion focus on hand and fingers. I think it will give better performance for my project.