Most voice commands control an application via its user interface (UI), by simulating keystrokes and mouse clicks.
UI commands are very useful but can be fragile, for example if they require a waiting period of just
the right length or clicking particular screen coordinates.
For some applications there's a better way—using an automation interface. For
example, MS Office and MS Visual Studio support rich
automation interfaces. By writing a Vocola extension which invokes automation methods you can make it easy to
create voice commands which control the application more reliably and expressively.
For an example, see the VisualStudio sample extension
in ExtensionSamples\VisualStudio in the Vocola installation folder. Among
other capabilities it lets you refer to a visible line by its line number's last two digits, and implements
Emacs-style word navigation (much better than Visual Studio's built-in navigation, in the author's opinion).
The sample commands for Visual Studio include commands which call these
extension methods. (To see them, start Visual Studio and say "Open Sample Commands").
Circumventing User Account Control (UAC) restrictions
The Vocola API includes a helper method to work around an annoying barrier to using automation interfaces.
Windows "User Account Control" (UAC) prevents Vocola from connecting to a running application because they
have different security levels.
One solution is to just disable UAC so your extension can connect to an automation server directly. But if you
prefer to keep UAC enabled you can instead use the Vocola API
method GetAutomationObject to connect. It creates a separate process with a "normal"
security level and marshals function calls through that. See the VisualStudio
extension for an example. It connects to a running instance of Visual Studio this way:
dte2 = (DTE2)VocolaApi.GetAutomationObject("VisualStudio.DTE");
The dte2 object may then be used to invoke
any Visual Studio automation interface
Using GetAutomationObject works well, though it appears to have one drawback—if
an automation call returns an object rather than a simple type you can't pass that object as an argument in
another call. But you can accomplish a lot without hitting this restriction. Or you can disable UAC.
Heavy use of the sample Visual Studio extension has revealed an intermittent issue—after a few days of
voice interaction the extension may sometimes hang for about 90 seconds, during which time both Vocola and
Visual Studio are unresponsive. If you wait it out, everything is usually OK.