Multi-touch in HCI: Bill Buxton

Bill Buxton is the absolute expert for touch-related human computer interaction research. Bill Buxton’s impressive amount of publications has started around 30 years ago and have never ceased since then. Among other things he is a Principal Researcher at Microsoft and participates in the Microsoft Surface research.

I especially like his research work about two-handed input as it tells a lot about how two-handed input could be employed in multi-touch setups. It is simple to make applications that use two hands but it is difficult not to overstrain the user. Thus, I endorse his theory about compound tasks that apply natural tasks to each hand: E.g. when you are right-handed and you are writing a text on a sheet of paper you will use your left hand to hold the paper in place and move the paper in a way that your right hand is only writing in one region. Hence, the left hand performs gradual tasks while the right hand is doing the precise and high-frequent work. With other words, for two-handed input in multi-touch you could choose tasks (for right-handed people) that give the left hand a helper function and give the right hand a fine-tune task.

However, what I actually wanted to do in this post was to set a link to Bill Buxton’s multi-touch page.

Google Maps Multi-touch Actionscript Code [Update_3]

Google Maps multi-touch image

After I published my youtube video on the Google Maps multi-touch app I have had some demands for the source code of this Google Maps multi-touch Flash application. Hence, I have decided to make the source code of it publicly available on Google Code. You will find the sources on the mapstouch Google Code download page.

To be able to use it you need to generate your own Google Maps Api key. Use this explanation on how to tweak the code and choose the appropriate URL for your API key in order to prevent the red DEBUG strings (you can still see those strings on the screen in the image on the top of this post):

Put your own Google Maps API key in the variable map.key in the class GoogleMapsPureAS. Use the qualified name of your computer in the URL e.g.

http://johannes.local/googleMapsMultitouch

and not

http://localhost/googleMapsMultitouch

to get rid of the red DEBUG strings on the screen. Additionally, opening the SWF file in the Flash Player is not possible if you don’t want to see the DEBUG string. Open it in your browser under the appropriate URL where your Google Maps API key has been registered for.

If you have got more questions just ask them in the comments.

Update: You need the Google Maps for Flash API that you can download here. You have to include the swc file from the folder lib of the uncompressed sdk.zip file either in your Flash GUI or put it into the libs folder of your Flex Builder project.

Update 2: Please beware that executing Google Maps on a local computer or in the intranet seems to be not committed according to Google’s license agreements (see comments).

Update 3: I’ve posted another (even more detailed) explanation how to remove the DEBUG strings in the forum of the nuigroup.. Additionally, i’ve updated the Downloads on the mapstouch Google code page.

Google Maps Multi-touch App

I’ve built a small Google Maps multi-touch app with Flex and Flash over the last days:

It uses the Google Maps Flash API and the TUIO Actionscript classes and the Touchlib from the Nuigroup.

The red stuff on the screen means that i am using the debug mode of Google Maps for Flash. That is because I execute the Flash file on my local computer. The problem is: For TUIO I need to connect to a local socket. Actually I don’t know how to access a local socket server with a Flash app that opens up in my browser on a remote site (and if this was possible it would scare the hell out of me). Seems like I need to live with the red stuff…

Maybe someone has wondered why I only use the view with satellite pictures. It is also possible to have the maps view but by the time I produced that video the maps servers were offline for a day.

Update: Google Maps Multi-touch Application Actionscript Code

A Multi-Touch Photo Browsing Application (with Source Code)

The Touchlib offers lots of Flash related possibilities to build your own multi-touch applications. There are even some demo applications included in the package in the code version of the Touchlib.

Thus I took a demo photo application from them and replaced their Flickr images with mine and exchanged the whole mechanism for dragging and scaling (and left out rotating) for my logic (resp. niqui merret‘s logic).

It’s just a me too photo app but nonetheless here we go:

Update: By common request I have decided to publish the source code: FlickrPhotoApp.zip. I put this code under the Creative Commons Attribution 2.5 License (this means that you can use the code literally for everything but must mention my name).

Embed Illustrator Vector Graphics In Actionscript 3

There’s a really cool article about the embedding of resources in Actionscript 3 in Keith Peters bit-101 blog.

If you’re using his technique you can get rid of that awkward Flash GUI and just develop Flash applications with Eclipse in combination with the Flex Builder plugin by Adobe (currently you can obtain a free license for the Flex Builder Plugin as a student) or with the FDT plugin by Powerflasher.

In the beginning you need the Flash GUI to put all the graphical stuff you need in the library of one Flash file each as a MovieClip symbol. Afterwards you export the swf, put it in you main source folder and follow Keith’s instructions.

I use this to embed Vector graphics I get in an Illustrator format from a Designer to embed it as a Vector graphics in Actionscript 3.

TimeTube Article in the G.I.T. Laboratory Journal

Our article about the project TimeTube we’ve been developing since last year has been published by the G.I.T Laboratory Journal Europe in the current Laboratory Journal issue 7-8/2008.

Snapshot from TimeTube Article
TimeTube article

Feel free to download the article TimeTube – Assist Scientists to Enhance Laboratory Scheduling in PDF format.

My 2nd (so far unfinished) multi-touch table

(Update: The follow-up article can be found here.)

Fortunately I’ve got some money from the Computer Science faculty of the Wiesbaden University of Applied Sciences to build another (better) multi-touch table. I decided to try Tim Roth’s DSI technology. By using this technology it should be possible to use objects on the table that can be traced by a camera just like in the reactable project.

I made a photoset of this DSI multi-touch table on flickr:

DSI multi-touch table from FH Wiesbaden

Continue reading “My 2nd (so far unfinished) multi-touch table”

Flash Actionscript 3.0 drawString equivalent [Update]

I’m coming from the Java world where you could ‘just’ draw Strings. However, in Flash it’s not as easy to actually draw a String on the screen.

For that purpose I use following code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

textSprite = new Sprite();
 
//automatically shows a hand as mouse cursor when the mouse hovers 
//over the text field
textSprite.buttonMode = true;
 
addChild(textSprite);
var labelType : TextField = new TextField();
labelType.text = "Text in a line.";
labelType.autoSize = TextFieldAutoSize.LEFT;
labelType.wordWrap = true;
 
//copy TextField into a bitmap
var typeTextBitmap : BitmapData = new BitmapData(labelType.width, 
                                labelType.height,true,0x00000000);
typeTextBitmap.draw(labelType);
 
//calculate center of TextField
var typeTextTranslationX:Number = -0.5*labelType.width;
var typeTextTranslationY:Number = -0.5*labelType.height;
 
//create Matrix which moves the TextField to the center
var matrix:Matrix = new Matrix();
matrix.translate(typeTextTranslationX, typeTextTranslationY);
 
//draw invisible rect in the background that will serve as click area
textSprite.graphics.beginFill(0xfff0000,0);
textSprite.graphics.drawRect(-0.5*labelType.width,-0.5*labelType.height, 
                                labelType.width, labelType.height);
textSprite.graphics.endFill();
 
//actually draw the text on the stage (with no-repeat and anti-aliasing)
textSprite.graphics.beginBitmapFill(typeTextBitmap,matrix,false,true);
textSprite.graphics.drawRect(typeTextTranslationX, typeTextTranslationY, 
                                labelType.width, labelType.height);
textSprite.graphics.endFill();
//...

Update: I’ve enhanced the drawString code with a mouse listener by putting another sprite on top of it by drawing an invisible rectangle behind the text in the same Sprite. You can see/download the complete example here.