Point and Control
NOTE: If you find this component useful or want to comment leave a short message on the discussion page of this component
It removes the need for browsing lists of devices in favour of spatial selection gestures, thus requires motor abilities and spatial cognition to do so.
Particularly we expect a lower digital literacy need for using the system and also expect (in combination e.g. also with speech output) applications benefits for users with visual impairment or users who do not prefer small screens on mobile devices in general.
The platform support is currently limited by the dependency on the Kinect sensor. The current implementation is based on the official Kinect for Windows SDK by Microsoft, which is only available for Windows 7 and above. The complete SDK is not needed for execution, but the redistributable files and drivers are currently not included in the repository. The code is written in C# tested with .Net 4.0 and above. The configuration and location of the devices in a 3D room layout is stored in a XML file which is read out during the initialization. Changes in the program are written back to the file. HTTP is used for communication between client and server and the remote control user interfaces are designed using HTML.
A client on the smartphone of the user is required to interact with the system. The current implementation uses a native Android application with an integrated WebView (a simple HTML renderer) to display the remote control user interface. The application manages the connection to the server and provides the UI for registration and device selection.
The code of the system is released under MIT License as stated in the repository.
Status, Known Issues & Planned Work
The system works reliably with the first version of the Kinect sensor. Currently the device repository, device selection and control is integrated in a single component.
- Test performance of Kinect One sensor
- Separate repository, selection and control into individual components, communicating via REST/JSON
- Structured API with documentation
- Migrate Android native app to web app
A video demonstrating the system can be found here: http://www.youtube.com/watch?v=RqlCYBIUMos
A paper abstract with more information on the system can be found here: http://www.teco.edu/~berning/papers/ubicomp2013_video.pdf
The code is hosted at github: https://github.com/teco-kit/PointAndControl This is a public repository, so everybody can contribute. If you find any bugs, you can also open a new issue.