Using Automation

Most voice commands control an application via its user interface (UI), by simulating keystrokes and mouse clicks. UI commands are very useful but can be fragile, for example if they require a waiting period of just the right length or clicking particular screen coordinates.

For some applications there's a better way—using an automation interface. For example, MS Office and MS Visual Studio support rich automation interfaces. By writing a Vocola extension which invokes automation methods you can make it easy to create voice commands which control the application more reliably and expressively.

For an example, see the VisualStudio sample extension in ExtensionSamples\VisualStudio in the Vocola installation folder. Among other capabilities it lets you refer to a visible line by its line number's last two digits, and implements Emacs-style word navigation (much better than Visual Studio's built-in navigation, in the author's opinion). The sample commands for Visual Studio include commands which call these extension methods. (To see them, start Visual Studio and say "Open Sample Commands").

Circumventing User Account Control (UAC) restrictions

The Vocola API includes a helper method to work around an annoying barrier to using automation interfaces. Windows "User Account Control" (UAC) prevents Vocola from connecting to a running application because they have different security levels.

One solution is to just disable UAC so your extension can connect to an automation server directly. But if you prefer to keep UAC enabled you can instead use the Vocola API method GetAutomationObject to connect. It creates a separate process with a "normal" security level and marshals function calls through that. See the VisualStudio extension for an example. It connects to a running instance of Visual Studio this way:

The dte2 object may then be used to invoke any Visual Studio automation interface method.

Using GetAutomationObject works well, though it appears to have one drawback—if an automation call returns an object rather than a simple type you can't pass that object as an argument in another call. But you can accomplish a lot without hitting this restriction. Or you can disable UAC.

Heavy use of the sample Visual Studio extension has revealed an intermittent issue—after a few days of voice interaction the extension may sometimes hang for about 90 seconds, during which time both Vocola and Visual Studio are unresponsive. If you wait it out, everything is usually OK.