Visualizing Chatbot Quality with Swarm Plot

When you create a chatbot you frequently want to see where it is going wrong so that you can fix problems. When you look at the logs or run tests you get results of the form

Question, Correct Intent, Returned Intent, Confidence

Can I update my account settings?,Update,Check,0.332

I have a demo dataset here you can use to follow along with the code. Swarm csv

Usually the confusion matrix of which intentions are mixed up with each other can be shown with a heatmap. But an interesting visualisation for this type of data is a swarm plot using the Python seaborn library. There is a nice guide to the seaborn visualization library here

# Pandas for managing datasets
import pandas as pd
# Matplotlib for additional customization
from matplotlib import pyplot as plt
%matplotlib inline
# Seaborn for plotting and styling
import seaborn as sns
#read in the csv
df = pd.read_csv('swarm.csv', index_col=0, encoding='mac_roman')
df.columns = ['Intent', 'Expected','Confidence']
#draw the swarm chart
plt.figure(figsize=(10,6))
swarm_plot = sns.swarmplot(y='Confidence',
                           x='Expected', 
                           hue='Intent', 
                           data=df)
plt.legend(bbox_to_anchor=(1, 1), loc=2,title='Got')
plt.title('Swarm Report')

The graph shows you which intentions are being mixed up and the confidence that your chatbot has in its answers.