See full paper in HTML version. (Actually there should be a PDF version of this paper, but the link is 404)

Recently, I had the problem that the TFT display module of an 2008 17″ Intel iMac refused to work properly: The whole image was inverted and the right half stayed completely grey. I thought that it would be a nice solution to use the iMac as a headless computer by attaching an external display and switching off the integrated display module. However, I found no software solution to switch off the integrated display and no further information on the Internet if this was possible.

Thus, I decided to open up the iMac and remove the plug of the internal display module from the mainboard and to attach an external display to try out if this works. Fortunately, there is a very nice iMac disassembly manual by ifixit.com that I followed up to step 9. To remove the display cable from the mainboard is pretty straightforward: You have to remove 8 screws and unplug 3 cables. Subsequently, you reassemble everything and leave the display module cable unplugged (this is what you see in the picture above).

And: IT WORKS!!! You can use the iMac with detached display module cable and an external display smoothly without any problems. The iMac than just works like, for instance, a Mac Mini.

I really like Donald Norman’s essay »Natural User Interfaces Are Not Natural« that he wrote for the ACM interactions magazine. In this essay Norman shares his view on the role of natural user interfaces (for him represented, e.g., by »speech, gesture, or the tapping of the body’s electrical signals for “thought control,”«) in terms of interaction with future computer systems.

I have got the impression that marketing guys love to pronounce the end of mouse and keyboard interfaces that will in their opinion (or at least in their words) become obsolete as their place will be absorbed by natural user interfaces that are based on gestural interaction provided by systems like the Kinect or by multi-touch screens. I do not share this opinion. As it comes to me, (nearly) every interaction approach has its right to exist and usually is best for certain types of tasks. I doubt, for instance, that there soon will be a better way to enter text than with a standard keyboard based on haptic keys.

So, Norman provides one of the few pieces of literature that keeps cool about the natural user interface hype and gives profound arguments where he sees their potentials and pitfalls. I strongly recommend to read the whole essay. In the following, I cite some strong points his essay makes.

It is also unlikely that complex systems could be controlled solely by body gestures because the subtleties of action are too complex to be handled by actions–it is as if our spoken language consisted solely of verbs. We need ways of specifying scope, range, temporal order, and conditional dependencies. As a result, most complex systems for gesture also provide switches, hand-held devices, gloves, spoken command languages, or even good old-fashioned keyboards to add more specificity and precision to the commands.
…
Gesture and touch-based systems are already so well accepted that I continually see people making gestures to systems that do not understand them: tapping the screens of non-touch-sensitive displays, pinching and expanding the fingers or sliding the finger across the screen on systems that do not support these actions, and for that matter, waving hands in front of sinks that use old-fashioned handles, not infrared sensors, to dispense water.
…
All new technologies have their proper place. All new technologies will take a while for us to figure out the best manner of interaction as well as the standardization that removes one source of potential confusion. None of these systems is inherently more natural than the others. The mouse and keyboard are not natural. Speech utterances will have to be learned and gestures carefully developed and standardized through time. The standards don’t have to be the best of all possibilities. The keyboard has standardized upon variations of qwerty and azerty throughout the world even though neither is optimal–standards are more important than optimization.

Are natural user interfaces natural? No. But they will be useful.

Donald Norman – »Natural User Interfaces Are Not Natural«

If another object changes the transformation matrix of a DisplayObject, the DisplayObject has no means to recognize that it has been transformed, as it is transformed automatically. E.g., there is no kind of TRANSFORMATION_MATRIX_CHANGED event that is dispatched on the DisplayObject. If your DisplayObject still should react on a change of its transformation matrix, a workaround is necessary, which is described in the following approach. I show a short solution at first and will explain the actual problem and solution approach in more detail afterwards.

DISCLAIMER: I found no solution where the change of the transformation matrix can be recognized without changing the object that actually does the matrix transformation. If you know or find one, be so kind to post it in the comments.

Short Solution:

Your DisplayObject called UpdateDetectionSprite:

1
2
3
4
5
6
7

public class UpdateDetectionSprite extends Sprite{
public override function set transform(value:Transform):void{
	super.transform = value;
	this.updated = true;
}
//…
}

Object that does the matrix transformation called ActionObject:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

public class ActionObject{
private function performTransformation():void{
	var transformedSprite:UpdateDetectionSprite = new UpdateDetectionSprite();
	var transform:Transform = transformedSprite.transform; 
	var matrix:Matrix = transform.matrix;
 
	//do something with matrix, e.g., move object by translation
	matrix.translate(10,20);
 
	//assign matrix to Transform object
	transform.matrix = matrix;
	//assign changed transform object to your DisplayObject. your overriden 
	//setter will be called, causing the update flag to be set.
	transformedSprite.transform = transform;
}
}

Actual Problem:
I have a Sprite (called UpdateDetectionSprite) in which I want to be informed, when the object has been transformed (moved, rotated or scaled). To establish that, one part of the approach is to override the appropriate setters for x, y, width, height, scaleX, scaleY and rotation in UpdateDetectionSprite. However, if the UpdateDetectionSprite is modified when its transformation matrix is changed, the mentioned setters will NOT be called, so UpdateDetectionSprite will not be informed on a matrix change, it will automatically transformed. My first idea was to override the setter for the matrix in order to react on the assignment of matrix to UpdateDetectionSprite, but the matrix itself actually is not a property of DisplayObject but a property of DisplayObject’s transform property. So I overrode the transform setter in UpdateDetectionSprite. In order to inform UpdateDetectionSprite that its matrix has been changed, you additionally have to set the transform property in ActionObject that does the transformation (see listing 2). That approach works as well with UpdateDetectionSprite as with each other DisplayObject.

Complete UpdateDetectionSprite:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

public class UpdateDetectionSprite extends Sprite{
 
private var _updated:Boolean;
 
public override function set transform(value:Transform):void{
	this.updated = true;
	super.transform = value;
}
public override function set x(x:Number):void{
	if(this.x != x){
		this.updated = true;
	}
	super.x = x;	
}
public override function set y(y:Number):void{
	if(this.y != y){
		this.updated = true;
	}
	super.y = y;	
}
public override function set rotation(rotation:Number):void{
	if(this.rotation != rotation){
		this.updated = true;
	}
	super.rotation = rotation;	
}
public override function set width(width:Number):void{
	if(this.width != width){
		this.updated = true;
	}
	super.width = width;	
}
public override function set height(height:Number):void{
	if(this.height != height){
		this.updated = true;
	}
	super.height = height;	
}
public override function set scaleX(scaleX:Number):void{
	if(this.scaleX != scaleX){
		this.updated = true;
	}
	super.scaleX = scaleX;	
}
public override function set scaleY(scaleY:Number):void{
	if(this.scaleY != scaleY){
		this.updated = true;
	}
	super.scaleY = scaleY;	
}
public function set updated(updated:Boolean):void{
	if(updated){
		this.addEventListener(Event.EXIT_FRAME, updatedSprite);
	}
	this._updated = updated;
}
protected function updatedSprite(event:Event):void{
	//do something
	//...
	//...
	updated = false;
	this.removeEventListener(Event.EXIT_FRAME, updatedSprite);
}
}

For testing purposes, I additionally tried to write a custom Transform object called UpdateDetectionTransform that inherits from Transform, assign it to UpdateDetectionSprite’s transformation property and override the matrix setter in UpdateDetectionTransform. I do not know why, but it is not possible to assign a custom Transform object to the transform property. This code

1
2

this.transform = new UpdateDetectionTransform(this);
trace(this.transform);

will print ‘[object Transform]‘ to the console. This means that the assignment of UpdateDetectionTransform to this.transform does not work. I do not know why. If you know why, post it in the comments please.

Alternatively to my outlined approach above, you could dispatch a custom event in ActionObject on UpdateDetectionTransform.

I have been looking for an application on the Internet that allows to record TUIO streams. However, I did not find a sufficient solution for this problem. Thus, I decided to write my own application called ThrongOSCDeck that can accomplish this task. It builds upon my Throng application that can be used to multiplex TUIO bundles from different TUIO providers.

Although initially intended for TUIO only, ThrongOSCDeck can be used for any kind of OSC stream. After recording an OSC stream, it can be played back or saved to file. I have designed the file format in a way that it is human-readable and editable.

Check out Throng’s Google code page for more information.

Update: I eventually found another OSC recorder via tuio.org called OSCRecordTools. It is written in Python.

Throng comes in two flavors:

I. Throng provides…

1.) … a Tuio proxy (or bridge) that is necessary for applications that may only access network sources on the local machine (like Adobe Flash). If other devices (like iPhones) want to access such an application via Tuio over the network, a proxy will be necessary.

2.) … multiplexing of Tuio alive messages: If Tuio messages from multiple sources access one client application (or as it is called in OSC: a server application), all sources use alive messages that only contain their alive ids. If the client application does not support different source providers (which is the case for many tuio libraries), each Tuio source will remove the alive objects and cursors of each other Tuio source causing, e.g., touches to flicker.

3.) … individualization of the session ids of each source: if different Tuio message sources use the same session id range, the results can be … awkward. Hence, Throng individualizes them.

4.) … one consistent Tuio Fseq message order for all arriving packets.

5.) … adding of a Tuio source message to each Tuio package or, if such a message already exists, modification of this message after the “@” to contain source_ip:source_port to allow for discerning of sources in the client application.

II. Throng Custom provides an application in which the x and y values of Tuio set messages of different sources can be cropped, shrunk and moved in order to allow for, e.g., a two camera – two tracker system on two computers for one interactive surface.

Throng is hosted (with full code available) on Google Code.

Throng is based on Flosc by Ben Chun the illposed OSC classes and the TUIO Java implementation by Martin Kaltenbrunner.

Eclipse with SVN (e.g. Subclipse plugin) looks in three places for a proxy server configuration:

The first place (NOT for SVN internet connections):
-> Windows -> Preferences -> General -> Network Connections
The proxy servers that are shown here are…
… added by default by eclipse (Proxy entries: HTTP, HTTPS, SOCKS; Proxy bypass: localhost, 127.0.0.1).

The second place is the Internet Explorer. If you go to Extras->Internet Options->Connections->LAN Options->Proxy Server in Internet Explorer, you can add a proxy server manually. Eclipse/Flash Builder also adds these manual proxy settings to its own connections.

The rest of the connections in Eclipse/Flash Builder are added manually by the user.

The third place (for SVN internet connections):
SVN stores its proxy in a configuration file. If you look for it in Windows XP here is a good manual. If you look for it under Windows 7 (or Vista?) go to C:\Users\MyUserId\AppData\Roaming\Subversion\servers. http-proxy-host and http-proxy-port control the proxy used by SVN (e.g. subclipse in eclipse).

Read more at Engadget and Microsoft.

This approach worked for my Huawei Fonic Surfstick (with model number E1550):

1. Choose “Network” in System Preferences
2. Choose “HUAWEIMobile-Modem”
3. Choose “More Options…”
4. In tab “Modem” choose “Null Modem” from Manufacturer dropdown menu
5. In tab “PPP” check “Connect via Terminal”
6. Press “OK”
7. Press “Apply Settings”
8. Press “Connect”
9. Enter “atz” and press return in appearing terminal window (you will not see the characters appearing)
10. Enter “atz^u2diag=0″ and press return
11. Press “Cancel”
12. Reset HUAWEI modem settings (uncheck “Connect via Terminal” and choose “Apple Modem” from manufacturer dropdown menu)

Happy no Mobile Partner time!