Migrating your Watson Assistant workspace to WebHooks

In the last post we examined the differences between webhooks and the old way (sometimes called web actions) that IBM Watson assistant called REST functions through the use of IBM Cloud Functions. In this post we will look at a simple example of migrating from one to the other.

There is a tutorial on DeveloperWorks which guides you through all of the steps to connect your Watson Assistant skill to the Wikipedia API using the old style mechanism. In this blog post we will assume that you have already gone through the steps in the original tutorial and we will describe how you can convert your workspace to use the newly released webhooks feature.

The first step in the migration is that  you need to make sure that the Cloud Function you created to lookup Wikipedia call able to be called externally.

If the function definition is associated with a resource group, the security model will make it hard to be called, so you need to ensure that your function definition it is associated with a Cloud Foundry space. You can tell if your selected namespace is IAM based or Cloud Foundry based because of the drop-down selector at the top of the page will say (CF-Based) e.g.:

If your function is defined in a IAM resource group, the easiest way to move it is to create a new function (with the exact same code) in a Cloud Foundry space i.e. switch to the new space with the drop-down and then create the function as described in the original tutorial.

After you have saved the Action you should try it out by changing the object_of_interest to different things you might be interested in looking up and then see what the function returns.

Since WebHooks doesn't interact directly with Cloud Functions as such, you will need to ensure that your action is turned into a WebAction which can be called by any REST client. To do this, click on the EndPoints link in the left margin. This will give you the option to make your action invokable by a REST URL.

Once  you do this, you will see a curl command which can be used to invoke the web action. Initially the screen only shows API-KEY rather than the actual API key assigned to you. Click on the eye icon on the right to display the fully correct curl command.

If you have curl installed you can copy this command and execute it in your command line window, However, you will get an error because you haven't given any input parameters. To solve this add more command line options to specify the object of interest and the fact that the data you are supplying in in JSON format. e.g.:

curl -u 3a686c56-12fc-4bd9-8a08-55317fec468d:CE4A88p1qGYV69dF43iVENNn3Ok6DHdtlYz7tlrCh0yG7aNRvUgzcHHBNJxi15z9 --header "Content-Type: application/json"  --data "{\"object_of_interest\": \"love\"}" -X POST https://us-east.functions.cloud.ibm.com/api/v1/namespaces/brian_odonovan_bod-space/actions/Assistant-Functions/Wikipedia-Lookup?blocking=true

If you prefer using another tool like POSTMAN, you can easily convert this command to suit. The one thing you need to be aware of is the fact that the authorisation you supply with the -u parameter to the curl command consists of two parts - the part before the colon is effectively a username and the part after the colon is the password.

Once you have verified that you WebAction is working correctly, you next need to change the Watson Assistant skill to use webhooks when calling Wikipedia. You do this by clicking on the options tab when editing the dialog and then selecting the Webhooks option on the left (it should be selected by default) and then entering details of the URL you want to call, what credentials to use and any other headers you want to pass to the function.

You can use the URL and credentials from the curl command that you got from the web actions page described above. You might be slightly worried that there is a single URL assigned to a  skill because in some cases where you might need to access services from different sites. There are ways of getting around this limitation, but I won't describe them here since our use case doesn't need to connect to multiple services. The next blog post in this series will describe in detail how you can connect to multiple REST services from a single WA workspace..

The Dialog node which implements the interface to Wikipedia through Webhooks will be significantly different from the old one, therefore I suggest that you either delete or disable the old node. For example you could rename the node to Old Wikipedia and disable it my changing the match condition to false as illustrated below.

Now you have to create a new Dialog for calling the webhook. You should call the node Wikipedia or something similar and make sure it is activated whenever the #tell_me_about intent is detected. Next click on the 'customize' icon and this will give you an option to turn on Webhooks for this node.

As soon as you close the customize dialog you will see that you see additional UI elements which parameters you would like to pass to the REST call and what context variable you would like to use to store the response.

You will also see that the node has been converted into a multi condition response node and it pre-configures two output slots for what to say when your REST call succeeded (i.e. when the context variable has been set) or when the variable wasn't set (which probably indicates a network error or something similar).

You can use the same responses as in the in the original tutorial since the format of the response won't have changed. You can now test your application and see that it behaves more or less as before.

There are two things that you should note about the way that Watson Assistant Webhooks work:

1. We specified that you add a parameter named object_of interest and set its value to the contents of the @object_of_interest entity (lets assume that you asked "what is love" so the value will be "love" ).
   
   Normally when people say that they are adding parameters to a POST call they mean that they are adding a header with the value "object_of interest: love". However, this is not what Watson Assistant does. Instead it sends a JSON body with each of the parameter values e.g. {"object_of_interest": "love"}.
   
   This is actually a better thing to do, but make sure you don't get confused by the terminology in the documentation.
2. Watson Assistant tells you that it stores the response from the REST call in the context variable you specify, but this is not exactly what it does. While the Webhooks functionality is not totally tied to the Cloud Functions, it does make certain assumptions based upon the way Cloud Functions operate.
   
   The response from a call to a Cloud Function will contain lots of information about the call other than just the response from the REST service called. It look something like: 

{

   "activationId": "xxx",
   "end": <time_stamp>,
   "start": <time_stamp>,
   "response": {
      "result": { ...},
      "status": "success",
      "success": true
   },

   ...
}

When you IBM Cloud Function returns, the data returned by the REST service is contained in the response.result part of the JSON structure retuned. If you are not using Cloud Functions, make sure you follow this convention because Watson Assistant will be expecting it. Similarly, you should also set the response.sucsess variable to the value true because otherwise Watson Assistant will assume that the call has failed.

What is the benefit of the new webhooks feature in Watson Assistant

When building an AI chatbot it is impossible to incorporate all knowledge directly in your skill. As a result developers often find themselves wanting to call external functions to answer certain queries. IBM has responded to this requirement by supporting the calling of cloud functions from within a Dialog node in Watson Assistant.

While developers have found this useful, they have also complained that it is inflexible and not so easy to use. To answer these complaints,  IBM has recently released a new feature called webhooks. This feature was available in limited Beta for several months, but has just been released generally so now is a good time to look at it.

This table summarises the differences between the two mechanisms:

Aspect	webhooks	old way
URL Flexibility	With webhooks you can call any arbitrary URL. This means that you are not tied to using IBM Cloud Functions as an intermediate layer, Of course Watson Assistant will always make a POST call to your URL and supply the parameters in JSON. If the REST API you want to call does not accept this, you will need some mechanism to transform the call. However, you are free to choose any transformation tool that you want.	With the old mechanism, you could only call a Cloud Function which is defined in the same environment as the Watson Assistant instance which is doing the calling. This was quite restrictive and although it worked OK with Cloud Foundry based authentication, it was not really compatible with the new IAM style resource group authentication currently used in the IBM cloud.
UI Assistance	There is a UI to guide you in defining the authorisation and other parameters for your call to a webhook. This makes it quite user friendly.	The way that you specified that a REST call should be made is by editing the JSON response from a node to include an action parameter. Apart from the documentation there was no assistance to developer to define this correctly.

Now that we have compared the two mechanisms, our next blog post will look at a simple example of migrating from one to the other.