Integrating Cortana with the Universal Windows Platform

Introduction

Three mobile platforms, iOS, Android, and Windows, has been improving their digital assistants (“Siri” on iOS platform, “Google Now” on Android, and “Cortana” on Windows) very frequently. All these digital assistants offer search device data and Cloud-based information. Cortana’s voice command features integrated into Windows 10 enable developers to give users convenient ways to access data. Cortana offers integration into Bing, Office 365, and local data sources. With a wide range of features, Cortana helps users organize and manage their data easily. Cortana also exposes an interface to developers that permits applications to integrate into the Cortana search experience. In this article, I’ll walk through you Cortana integration with the Universal Windows Platform (UWP) travel app.

Walkthrough

Step 1

Create a new Windows Universal Project Opening Visual Studio 2015/2017 -> New Project -> Visual C# ->Windows -> Universal -> Blank App.

Figure 1: New Windows Universal App Project

Figure 2: New Windows Universal App Project created

Step 2

Next, we have to add a Windows Runtime Component as a new project to the existing solution. This project will act as a background AppService for UWP app and activates whenever the user asks anything of Cortana.

To add, right-click the existing solution -> Add new project -> Add Windows Runtime Component.

Figure 3: New Windows Runtime Component Project

Figure 4: New Windows Runtime Component Project created

Step 3

Next, we have to store the voice commands. Cortana listens to an XML file (Voice Command Definition/VCD file).

Add a Voice Command Definition (VCD) file in your UWP app that needs to be installed on app launch. Do this from CortanaUniversalApp Project -> Add ->New Item ->XML File.

Figure 5: Add new XML VCD file

Copy and paste the following XML code into your VCD file. A voice command is a single utterance with a specific intent. A VCD file defines one or more voice commands, each with a unique intent.

<?xml version="1.0" encoding="utf-8"?>
<VoiceCommands
      >
   <CommandSet xml_lang="en-us" Name="CortanaVSDFile">
         <AppName>Cortena Voice Assistant</AppName>
         <CommandPrefix>
            Test Command USA Travel
         </CommandPrefix>
         <Example> Show trip to California </Example>
      <Command Name="showTripToDestination">
         <Example> show trip to California </Example>
         <ListenFor> show trip to {destination} </ListenFor>
         <Feedback>
            Showing trip to {destination}
         </Feedback>
         <Navigate/>
      </Command>

      <PhraseList Label="destination">
         <Item> California </Item>
         <Item> Dallas </Item>
         <Item> New York </Item>
      </PhraseList>

      <PhraseTopic Label="newDestination" Scenario="Search">
         <Subject>City/State</Subject>
      </PhraseTopic>
   </CommandSet>
   <CommandSet xml_lang="en-in" Name="examplevcd">
      <AppName>Voice Assistant</AppName>
         <CommandPrefix>
            Test Command USA Travel
         </CommandPrefix>
         <Example> Show trip to California </Example>
      <Command Name="showTripToDestination">
         <Example> show trip to California </Example>
         <ListenFor> show trip to {destination} </ListenFor>
         <Feedback> Showing trip to {destination} </Feedback>
         <Navigate/>
      </Command>

      <PhraseList Label="destination">
         <Item> California </Item>
         <Item> Dallas </Item>
         <Item> New York </Item>
      </PhraseList>

      <PhraseTopic Label="newDestination" Scenario="Search">
         <Subject>City/State</Subject>
      </PhraseTopic>
   </CommandSet>
</VoiceCommands>

Following are the attributes of Cortana’s VCD file:

• AppName: Cortana will listen to the user only with this app name as a prefix of the sentence.
• Example: Windows shows examples for each individual command.
• ListenFor: These are the words Cortana listens for.
• Feedback: Text shows to the user, while working on creating a user response.
• Navigate: This is the XAML page that Cortana navigates to when it parses what you’ve said.
• PhraseTopic: You can add multiple phrases inside this PhraseTopic for better response and understanding of the user.

Step 4

To install the preceding written VSD, add the following async function and call that from the OnLaunched() event of app.xaml.cs file. It will activate the app and call InstallCommandDefinitionsFromStorageFileAsync to register the commands that the system should listen for.

public async Task InstallVCD()
{
   try
   {
      StorageFile CortanaVSDFileStorage = await
         Package.Current.InstalledLocation
         .GetFileAsync("CortanaVSDFile.xml");
      await Windows.ApplicationModel.VoiceCommands
         .VoiceCommandDefinitionManager.InstallCommandDefinitions
         FromStorageFileAsync(CortanaVSDFileStorage);
   }
   catch (Exception ex)
   {
      throw new Exception(ex.ToString());
   }
}

Step 5

Next, you have to add the reference of the runtime component into the CortanaUniversalApp project.

Choose CortanaUniversalApp Project -> References ->. Right-click ->Add Reference -> Project -> Solution -> Check CortanaUWPDemoBgTask ->. Click OK.

Step 6

Adding Cortana voice commands to a Universal Windows Platform (UWP) app requires a few lines of code to be added to the OnLaunch event, overriding the OnActivated method in the App.xaml.cs file, and overriding the OnNavigatedTo method in any other XAML views that will be opened by the commands.

protected override void OnLaunched(LaunchActivatedEventArgs e)
{

   #if DEBUG
      if (System.Diagnostics.Debugger.IsAttached)
      {
         this.DebugSettings.EnableFrameRateCounter = true;
      }
   #endif

   Frame rootFrame = Window.Current.Content as Frame;

   // Do not repeat app initialization when the Window already
   // has content, just ensure that the window is active
   if (rootFrame == null)
   {
      // Create a Frame to act as the navigation context and
      // navigate to the first page
      rootFrame = new Frame();

      rootFrame.NavigationFailed += OnNavigationFailed;

      if (e.PreviousExecutionState ==
         ApplicationExecutionState.Terminated)
      {
         // TODO: Load state from previously suspended application
      }

      // Place the frame in the current Window
      Window.Current.Content = rootFrame;
   }

   if (rootFrame.Content == null)
   {
      // When the navigation stack isn't restored navigate to the
      // first page, configuring the new page by passing required
      // information as a navigation parameter.
      rootFrame.Navigate(typeof(MainPage), e.Arguments);
   }
   // Ensure the current window is active
   Window.Current.Activate();
   InstallVCD();
}

Step 7

Next, we need to add BackgroundTask into the Windows Runtime component project. This background task will be called on every voice command executed by the user.

Step 8

Lastly, add the following code to the package.appmanifest file and execute the app.

<Extensions>
   <uap:Extension Category="windows.appService"
      EntryPoint="CortanaUniversalApp.CortanaBackgroundTask">
      <uap:AppService Name="CortanaBackgroundTask" />
   </uap:Extension>
   <uap:Extension Category="windows.personalAssistantLaunch" />
</Extensions>

Summary

I believe you learned how to implement Cortana’s basic voice commands using VCD files. Cortana’s experience can be customized to limit the need for users to continually open and close applications. Also, Cortana allows you to drive a richer engagement with your apps. That’s all for today! Keep reading.