05/07/2017

Use a XBox Controller to control your Angular2 App

TLDR;

For repeated multiple choice data entry you might want to consider alternative forms of input, like e.g. a gamepad.
Gamepad support is provided by XInput on Windows.
SendInput() allows sending of Keyboard events to foreground applications
Combine both in a simple application
Add Hotkey support to your Angular2 app
Now you are able to control your app with an XBox Controller

Who would want that?

A customer had an interesting idea - he asked if there were alternative user input devices available that could be used for data entry for an Angular2 Intranet Application that I was working on.

His reasoning was that the WebApp was used for data entry - and lots of it! After login, the application basically consists of a main loop that would present it's user with visual and textual information and would ask the user to take a decision based on it. As the possible input is limited and speed for data entry is paramount, hotkeys support was introduced. I've wrote my take on hotkey support here. But additionally the customer was worried about the amount of stress that an employee could suffer from repeated usage.

The customer asked for ergonomic input devices that could be used in addition to a keyboard. The idea was to allow the users to freely switch between different devices if he felt uncomfortable using one.

Why the XBox Controller?

An ergonomic input device in widespread use that is also within budget are Gamepads. They are basically made for varying methods of input and allow extended usage with minimum strain.

Additionally I knew that the WebApp was used as an intranet application and the customer was using Windows Machines, which do have native support for XBox Controllers and on which additional software could be installed.

Support for gamepads and joysticks on Windows is provided via XInput or the now deprecated DirectInput API - both of them are a part of DirectX.

The XBox Controller works with XInput and DirectInput and most of the third party Gamepads provide a hardware switch for changing between XInput and DirectInput. This comes in handy if your game or app supports only one API, but not the other.

Getting access to the XBox Controller

So the first obstacle was to get access to the XBox Controller, as you cannot get access to DirectX (and specifically XInput) from an Angular Application directly. For this, a native Windows Application was necessary.

I opted for a C# Wpf Application and chose SharpDX (http://sharpdx.org), which is a thin wrapper for the C++ DirectX Api.

Polling the Controller

XInput comes with a poll based API, which is perfect for it's natural audience, that is games. They usually poll the state of the controllers inside of their main loop and do immediate mode rendering - it all fits together.

Example of polling the Keystroke:

var controller = new Controller(UserIndex.One);
Keystroke keystroke;
var result = controller.GetKeystroke(DeviceQueryType.Gamepad, out keystroke);
if (result.Success)
    ...

or the more concise :

var controller = new Controller(UserIndex.One);
if (controller.GetKeystroke(DeviceQueryType.Gamepad, out Keystroke keystroke).Success)
    ...

From Polling to Pushing

In this case however, I am creating a normal Windows Desktop Application using retained mode drawing and I would prefer a push based interface for the gamepad events. That's why I've chosen to wrap the poll based API inside of my application with push based events.

For exposing the push events to the rest of the application, you can use c# events or plain callbacks using delegates, but I've chosen Reactive Extensions as I greatly prefer the interface.

So my interface looks like this:

public interface IXInputService
{
    bool IsListening { get; set; }
    IObservable<Keystroke> Keystrokes { get; }
    IObservable<ControllerConnected> Connected { get; }
}

For implementing the continuous polling there are also a couple of options, for example a timer, a delayed task, a thread pool thread or the Application Idle event.

I've chosen to spin up a dedicated Thread, as the thread is both long running and the timing between each Poll request is fixed.

To test it out, I created a small Wpf prototype app (XInput2Key), that is able to read out the state of the gamepad's buttons and shows if they are currently pressed.

XInput2Key

Interfacing with Angular2

Now I was able to read out the gamepad input and the only thing that remains is to interface with the angular2 application. As I mentioned previously, the application already had hotkey support, so the obvious choice was to just send keyboard input to the angular application.

This has the additional advantage that the applications are very loosely coupled and you can use one without the other. This allows using the XInput application to work with any other application as well, as long as the target app has somekind of hotkey support.

As I already took a dependency on the host operating system being windows (for the DirectX support), I took a quick look at the native Windows Api and chose the SendInput() function. This function allows sending of keystrokes and mouse events to the foreground application, which in my case would be a browser with the loaded angular2 application.

To access the native C WinApi Method, I needed [DllImport] declarations for SendInput() which I luckily found on PInvoke.net.

Thanks to both SharpDX and PInvoke.Net I was quickly able to throw together a prototype application that:

reads in a config file to for mapping the gamepad buttons -> keyboard keys
listens for gamepad button presses using XInput
maps them to keystrokes
sends the keystrokes to the foreground application using SendInput()

Sample App

I've pushed the resulting app to github (https://github.com/8/XInput2Key) incase you want to try it yourself.

If you check the Checkbox 'Is Emulating Keys', then the mapped keyboard input is sent to the foreground window. If you open up an instance of your favourite text editor, you can try it out.

XInput2Key

The only caveat is that due to security constraints of SendInput() it cannot send input to applications running with administrator rights, if the app isn't started with administrator rights itself.

Setting up Angular2 to deal with Hotkeys

For hotkey support in angular I am basically using the javascript library mousetrap with an angular2 wrapper. If you're interested in how I've done that, you can check out my blog post about using Hotkeys in Angular2.

References

Last updated 05/07/2017 15:26:16

blog comments powered by Disqus

Questions?

Ask Martin