Multi-touch technology allows users to use their hands to manipulate digital information. We have observed that mainstream software frameworks do not offer support to deal with the complexity of these new devices. Current multi-touch frameworks only provide a narrow range of hardcoded functionality. Therefore the development of new multi-touch gestures and integrating them with other gestures is notoriously hard. The main goal of this framework is to provide developers with adequate software engineering abstractions to close the gap between the evolution in the multi-touch technology and software detection mechanisms.
Current frameworks force the programmer into an event driven programming model where the programmer has to register and compose event handlers manually. This results in an application where the control flow of the application is driven by external events and no longer by the sequential structure of the program. Reuse, composition and understanding are hampered when using such frameworks.
In this work we propose a solution based on research conducted by the Complex Event Processing domain. We advocate the use of a rule language which allows programmers to express gestures in a declarative way. The advantage of such an approach is that the programmer no longer needs to be concerned about how to derive gestures but only about describing the gesture. We present a first step in that direction in the form of a domain-specific language supporting spatio-temporal operators.
Complex gestures which are extremely hard to be implemented in traditional approaches can be expressed in one or multiple rules which are easy to understand. The use of a rule language has the benefit that the developed gestures are reusable and ease to compose. Further a strong connection to application-level entities allows developers to activate and deactivate gestures depending on their graphical context.
Keywords: Gesture Recognition, Touch Events, Multi-Touch, Declarative Language, Describing a sequence of events