As I work with Flex Builder 3 I downloaded and configured Air 2 with Flex SDK 3.4 accordingly to Adobe’s manual and everything worked and compiled nicely as well on Mac OS X as on Windows XP (yes, I tried both).

I tried to use UDP datagram sockets within my app (in order to get a direct TUIO connection via OSC/UDP), which are supported by Adobe Air beginning with Air 2. However, whenever I tried to start my compiled Air app in Flex Builder with the adl from Adobe Air 2 I got this error message:
VerifyError: Error #1014: Class flash.net::DatagramSocket could not be found. (with the additional stack trace).

I got rid of that message under Windows XP by following the hint by Adobe that the namespace in the app descriptor from Air has to be changed to Adobe Air 2 beta 2’s:

1

<application xmlns="http://ns.adobe.com/air/application/2.0beta2">

Now, my UDP datagram sockets work nicely under Windows XP. However, if I configure everthing accordingly under Mac OS X matters get worse. The app crashes and I get a native error stacktrace from the adl player.

In my opinion the implementation of the adl under Mac OS X is buggy, which is quite normal for a beta release. Thus, I filed an error message at Adobe’s website.

However, if you try to do the same and you get things working on Mac OS X it would be nice if you dropped me a comment.

By the way: I tried to use the new Touch functionality in Adobe Air 2 under Windows 7 with a HP Touchsmart and it works quite well (except there are no gestures supported as Windows 7 proposes them). Maybe, I find the time to write a follow-up article on this…

Update: It seems like the problem is not connected with Mac OS X but with my version of Flex Builder 3. I moved ahead to Flash Builder 4 beta 2 and now all the Adobe Air 2 beta 2 functionality works.

After being able to compile the Touchlib I quickly grew curious about what all those blobconfig parameters in the Touchlib’s config.xml REALLY do.

First of all, some parameters are really useful and it is a shame that the functionality is (nearly) nowhere documented. Even the pros in the nuigroup seem not to know for which purpose they are being used.

If someone wants to know why I am using the Touchlib and not tBeta aka CCV that is because CCV keeps crashing on my computers and I do not have Visual Studio 2008, which is necessary to compile CCV on Windows. Additionally, I had a student adding fiducial support to the Touchlib (by adding the reacTIVision code). Thus, I currently stick to the Touchlib.

Disclaimer: The code version of the Touchlib I used is the one that my student tweaked while adding reacTIVision fiducial support. You can find the Touchlib’s source code at the Touchlib code repository in Google code. However, many users are constantly submitting their code changes to this repository. E.g. the newest file of CBlobTracker, which does the actual tracking of blobs in the camera’s image, does not include the setup function anymore in which certain values of config.xml have been set for the tracker. Thus, those values are not being used in the code version anymore. However, they should still be used in the binary version that you can download here. I will use this older version of CBlobTracker to explain some parameter’s functionalities of config.xml. However, I think that in the Touchlib’s binary version those parameters are being used.

blobconfig
Touchlib searches in subsequent captured frames from the camera for the same blobs. I.e. if a blob in the previous frame has been found at position (x=250, y=300) and in the current frame a blob has been found at position (x=251,y=299) Touchlib interprets this as the same blob. However, tracking is not that simple that Touchlib could search for a blob in the direct surrounding but for each blob of a previous frame all blobs in the current frame are being compared for their distance. Touchlib thinks that the blob in the current frame with the shortest distance is the same blob (It is not that simple but let’s stick to this for now).

1. Attribute minDimension (very handy!):
This is the minimum size that a blob must have. If you e.g. have lots of small white dots in the RectifyFilter that are smaller than minDimension they must not be tracked. (see CBlobTracker line 209). Default value of minDimension is 2. I use a minDimension of 5. However, the appropriate minDimension depends on your camera’s resolution and your table’s size.

2. Attribute maxDimension (very handy!):
This is the maximum size that a blob can have. (see CBlobTracker line 209). Default is 250, which is way too big. I use a maxDimension of 20. However, the appropriate maxDimension depends on your camera’s resolution and your table’s size. This value is extremely handy if you want to prevent that a big object (like a cup or a sheet of paper) that is being put onto the touch surface is recognized as a touch.

3. Attribute distanceThreshold:
With distanceThreshold you can configure with which distance another blob in a consecutive frames might still be interpreted as the same blob. E.g. if distanceThreshold is big the distance between the two blobs can be big and vice versa. (see CBlobTracker lines 755 ff.). I currently do not see the point of this parameter.

4. Attribute minDisplacementThreshold (does nothing so far):
Should do basically the same as distanceThreshold but inverted: minDisplacementThreshold configures how far a blob must have been moved in order that an update is being sent. However, it is being set in CBlobTracker in line 51 but nowhere used in the code.

5. Attribute ghostFrames:
I do not know exactly what ghostFrames does. However, I guess that it regulates the amount of frames in which a blob is not being found in the current frame (for the count of ghostFrames frames) but not deleted as well out of the list of blobs. Thus, if a ghost frame is being sent by the camera (maybe an empty image because of capturing problems) not all blobs are being deleted but the ghost frame can be ignored. However, as I said I do not know the exact functionality. For more detail see CBlobTracker lines 717 – 732.

As already mentioned in this post, we presented Vispol at the booth of the state of Hessen/Germany at the fair “Moderner Staat” in Berlin on 24. and 25. November 2009. Vispol has been shown on the round multi-touch and TUI table from Sensory Minds.

Here are a few photos from the fair. Please apologize the poor quality of the photos as my camera did not see the sense in making proper use of its auto-focus functionality.

I love those nail tattoos of this young Vispol user:

My software Vispol will be shown at the booth of the state of Hessen/Germany at the fair “Moderner Staat” in Berlin on 24. and 25. November 2009.

It will be shown on the new really cool round [!] multi-touch and TUI table from sensory minds.

I will be there on both days.

Recently I had the problem that my Flex code, which used the ResourceManager, would not find the resource bundle, which contained all resources for my application. However, the application compiled but it would always crash whenever the first time a resource should be loaded. I checked my compiler arguments a 1000 times but everything was correct and did work well in another Flex application. The problem was that my resource bundle was not present in the ResourceManager, it just did not load it.

My application did not have any UI elements just a visualization UIComponent that I wrote myself. Thus, I did not load any resources from my MXML. A some point I added a UI element (a Button) and tried to load a resource directly from that Button in order to see if it worked there:

1

<mx:Button label="@Resource(bundle='myResources', key='OBJECT')" />

And it worked fine in that Button and suddenly in the whole application as well. So it seemed to me that Flex refused to load the resource bundle if it was not referenced from the MXML. As I did not want to use an UI element in my application I looked for a workaround and stumbled upon this:

1
2
3

<mx:Metadata>
        [ResourceBundle("myResources")]
</mx:Metadata>

This piece of MXML code just needs to be added in the top of your Application or WindowedApplication in order to trigger the loading of the resource bundle.

In terms of the NUI research project Squidy at the Konstanz University the TUIO overlay SurfaceToTuio has been developed for Microsoft Surface. Download the SurfaceToTuio component from the sourceforge project site (or via direct download).

SurfaceToTuio is available as source code only (there is no executable). However, you can just upload the SurfaceToTuio folder to your Surface, doubleclick the Visual Studio project and compile it in the Surface’s Visual Studio right away.

Finally, a decent software interface for a decent hardware.

Some time back, I’ve bought some infrared blocking foil for the Virttable from the German foil store Ifoha. Infrared blocking foil is an approach for the blocking of ambient infrared light (e.g. from a light bulb or daylight) ONLY in a FTIR multi-touch setup as it blocks light that comes from above the table. I.e. in DI your hands reflect light that would be blocked by an IR blocking foil.


Figure 2: The Virttable’s display with all IR blocking foil samples (without projection). Note the identifier labels from Ifoha.

I ordered samples from Ifoha from all IR blocking foils they have in stock. I put all samples on top of the table (figure 2 and 3). Onto the samples I each put a finger and a fiducial. Afterwards I made a screenshot of what the cameras see (figure 4).

Figure 3: The Virttable’s display with all IR blocking foil samples (with projection).

For the Virttable I used the foil with the id 1002300, which blocks 83% of the ambient infrared light.

Figure 4: What the camera sees on the Virttable’s display with all IR blocking foil samples on top.

If you want to know something specific ask in the comments.

UPDATE: The setup of the surface with the blocking foil is as following:

___ Projection Foil (e.g. Gerriets Optitrans) ___
___ IR blocking foil (e.g. Ifoha 1002300) _______
___ Silicone (e.g. SortaClear) __________________
___ Acrylic Glass _______________________________


Figure 1: 3D scatter plot of Fisher’s dataset assorted with the k-means algorithm

In VotkA users can interact with a three-dimensional cluster analysis visualization via a Tangible User Interface (TUI), the iPhone app OSCRemote or both in parallel. The TUI is provided by a small box that is technically based on Seth Sandler’s cheap multi-touch pad (you can see the box in the youtube video at the end of this article). On top of the box fiducials from reacTIVision are being tracked. Whenever a fiducial is being recognized a TUIO message with the fiducial’s id, position and rotation will be sent to VotkA.

In terms of the cluster analysis the k-means algorithm has been implemented. The underlying multivariate data that is being visualized is Fisher’s iris dataset. In this dataset different aspects of iris blossoms are being stored (e.g. petal length/width, sepal length/width). Within the visualization this data is being dispersed in a cube according to different aspects (e.g. on the x-axis petal length, on the y axis sepal width and on the y-axis sepal height).

Figure 2: Scatter plot projected on the dimenions sepal length / petal length

The implemented visualization methods 3D scatter plot (see figure 1), scatter plot matrix / Grand Tour have been slightly adjusted to visualize Fisher’s dataset. The interaction with the Grand Tour visualization method is being carried out in the TUI with a real cube, which incorporates the projection of the three-dimensional data on the dimensional layers in the graph. By tilting the cube in the application the appropriate projection side will be displayed on the screen (see figure 2 and video).

When interacting via fiducials an additional sound feedback mechanism has been provided by converting TUIO messages in VotkA to midi signals that are being sent to a software sequencer. Midi channels are mapped to fiducial ids and midi signals are produced according to position and rotation of the fiducials. Actually it should not have been necessary to convert TUIO messages in VotkA to midi because reacTIVision supports both outputs, midi and TUIO. However, only one output channel at a time can be handled by reacTIVision.

Video of interaction with VotkA. Please note that colors are being poorly displayed and that midi sound output has been disabled in this video:

Employed technologies:

• Java
• Processing for VotkA’s visualization part and for TUIO message processing
• reacTIVision for fiducial tracking
• mmj-Library for midi production in VotkA
• k-Means for cluster analysis
• Fisher’s Iris Dataset
• OSCRemote App as iPhone TUIO Client (for the production of the OSC messages)

Very interesting: Surface tracks shapes with its vision system. There are five cameras in the Surface. The system has a very high resolution (dice can be tracked). Surface uses high-power leds with a mirror system.

via blogs.msdn.com

Pictures of the actual LED and the light scattering mechanism that Surface uses can be found here.

It seems like Surface uses Osram SFH 4740 high power LEDs. Available in Germany here.

Most of this information has been found in that thread in the nuigroup.

Thanks, Stu.