One of the challenges we are faced with when building a chatbot is that questions the users ask the bot is highly unstructured. There are many ways different ways to ask for the same thing. Users can ask an extremely broad range of questions that require a diverse range of strategies to answer. The first step in building an helpful chatbot is to understand the user question well and putting it into a structured format that maps to entities that exist in the knowledge base. This can be done by doing intent classification and entity recognition.
Intent Classification
It is often not ideal to have a single retrieval method and prompt to answer all questions. Instead, we can have different strategies for different types of questions. For instance, some intents like summarize, translate or “thanks” do not need a search in the knowledge base. The intent is also used for routing the query and to select or combine prompt templates.
The idea with query routing shown in the figure, is to use one or more different strategies depending on the intent.
Query routing is the process of determining the best strategy to use for a given query. The strategy can be a search in the knowledge base, a function call, a template or a combination of these.
Extracting Named Entities
When the user is asking a question, the question needs to map as parameters into a structured query for search or for querying a knowledge graph. A common approach is to rely on the LLM to extract the entities when generating a query or doing function calling. However, experimentation quickly shows that this is not a robust solution. A better approach is to extract and map the entities before generating the query. This can be done in the following way:
Step 1: The sentence is parsed and named entities are extracted. Named entities are real-world objects that can be denoted with a proper name. Examples of named entities are persons, locations, organizations, dates, times, percentages, currencies, and more.
Step 2: The values of the named entities might not match the class instances in the knowledge base. For this reason, a semantic layer is used to map the named entities into a class instances. This can be done by doing a vector search finding the most similar class instances related to the named entities extracted.
The named entity classes are carefully selected to match the classes that can be used for filtering in a search query or for a graph query to extract information from a knowledge graph. An ontology can be used to define the classes and the relationships between the classes, this can be stored in RDF or owl format. An ontology is a formal representation of knowledge as a set of concepts within a domain, and the relationships between those concepts. The benefit is that it increases the ability to share and reuse knowledge, and it makes it easier to combine information from various sources.
Now we have a structured understanding of the user question that maps to the knowledge base. Next is to prepare the advanced search query and retrieve the relevant documents. This will be covered in a future post.
How to implement intent classification and entity recognition
The two features mentioned above can be implemented in several ways:
- Using LLM function calling. Each tools is an intent, and the entities is extracted as the parameters.
- Using an LLM is to contstruct a prompt with all the choices as text presented for the LLM and it is asked to make a choice.
- Using a SaaS application like Azure Conversational Language Understanding (CLU) can be used to train a model for intent and named entity recognition.
- Train your own model using a transformer based model like BERT or RoBERTa.
Remember there are other ML tools than just LLMs. I think this is often forgotten these days, because of the hype around LLMs. We selected Azure CLU for our project. Having a specialized model is easier to control and debug. If it is not performing, we know we can improve it improving data quality or by adding new examples to the training data.