Artificial Intelligence is becoming ubiquitous and nowadays anyone can get their hands on natural language processing technologies. One example of an application of natural language processing is a chatbot that provides customer support or augments call centers by supplying computer generated responses to customer questions.
Building a chatbot that provides customer support on a website is technologically quite feasible. However, comparing chatbot development with typical software application development, there are major differences. The key aspect of chatbot development is natural language understanding for which we can’t provide completely detailed specifications up front. Natural language means that customers may pose questions in many different forms, not all of which can be planned ahead.
How do we begin to develop a chatbot? Below I have listed the main components and addressed some challenges to be overcome when building chatbots.
Define the purpose of the chatbot
Chatbots come in different types, depending on which target group of users they address and which business problem they are meant to solve. The most common type of chatbot in business environments is an assistant whose purpose is strictly limited to a chosen scope of service. Some examples of assistants are customer service with respect to products that customers buy in an online shop, or making dinner reservations in a restaurant. The scope of such assistant chatbots is limited and focused on a specific business area. Chatbots don’t understand human language and are unable to chat about anything other than what they have been taught.
Prepare the knowledge base
Since chatbots don’t possess human intelligence, we can’t expect that they would be able to answer arbitrary questions. Depending on the purpose of the chatbot we must prepare answers to questions that we expect from the users. The answers must be available to the chatbot in a structured form, for example with indexed keywords, key concepts, topics, search criteria or any other classification that will allow the chatbot to find the most relevant answer based on a question that was asked by a user.
Design the dialog
A chatbot will not be able to answer questions about the weather or about sporting events or anything that is asked outside of a predefined dialog. For example, if we are developing a chatbot for customer support related to purchases in an online store, we must develop dialogs that include questions about finding items (search, recommendations), shipping (shipping methods, shipping costs), returns (instructions on how to return an unwanted item), payment (payment methods, gift cards) and so on. We break down each group of questions into sub-questions and design the dialog flow. Then we provide answers that the chatbot will use to respond to the questions. We must also decide how the chatbot responds when it doesn’t understand the question as well as how to respond to greetings, praise and criticism.
Design the user experience
Before we start development, we should understand the intended user experience. One of the aspects of user experience is graphical design, which should be clear to read and easy to navigate and should preferably also match the website where the chatbot is deployed. Another aspect is the personality that the chatbot personalizes. Although we are talking about a robot – and we must ensure that users are aware that they are talking to a machine and not a human so that there aren’t any misunderstandings – we should still decide which tone to use when the chatbot addresses the users, for example we may decide that the chatbot is strictly professional or childishly playful or may even try to be humorous.
Develop the chatbot
When we have everything ready, including the definition of the purpose of the chatbot, scripted dialogs and a database with answers, we may begin actual development. Depending on whether we decide to use a framework for chatbot development or whether we want to code from scratch, development may be more or less complex. Additional complexity arises from adding API calls, for example when we want to source answers to questions from a database or when we want to look up details about a customer that the chatbot is serving in an ERP system. Answers may be provided as simple text, or we may decide to add multimedia, pick lists or menus.
Test the chatbot
Testing a chatbot differs from testing typical software applications, where we define test cases, run the test cases and when all tests pass we declare the application as complete. Because users interact with the chatbot in natural language, we can’t anticipate all possible questions that will be asked by the users after the chatbot is in production. Users will ask questions using their own jargon, various grammatical structures and may also make typos. The best that we can do during testing is to test as many possible dialog flows and question variants as we can think of, but we have to accept that thorough testing of the chatbot will be a continuous process after the chatbot is in production.
Monitor and enhance the chatbot
When the chatbot is in production we can start monitoring how well it is providing service by answering questions. To do that, the chatbot must log all questions and answers so that we may periodically analyze them. Based on questions that the chatbot was not able to answer, we enhance the chatbot by improving understanding of questions, updating the dialogs and providing additional answers. We should also check answers to questions that the chatbot provided to ensure that they were the correct answers. Monitoring the chatbot must be a constant activity while the chatbot is in production, which means that chatbot development never completely ends but is a process of continuous improvement.