# Building a Chatbot That Answers From Your Own Documents 2024-09-03 · https://ibraverse.ca/tech/rag-montreal-pools/ > A RAG chatbot over Montréal’s indoor pool schedules: one text file per pool, embeddings with LlamaIndex, a ChainLit chat and a link to the source. I built a chatbot that uses a Large Language Model (LLM) to answer questions about pool availability in Montréal. Allowing an LLM to operate unchecked and make decisions can be risky. For example, Air Canada had to pay compensation because of a misleading response from an LLM: [Washington Post article](https://www.washingtonpost.com/travel/2024/02/18/air-canada-airline-chatbot-ruling/). ## The problem I started swimming a couple of weeks ago, and fortunately, Montréal has many indoor swimming pools. The city's website lists all the indoor pools along with their availability: [Indoor swimming pools](https://montreal.ca/lieux?mtl_content.lieux.available_activities.code=ACT0&mtl_content.lieux.installation.code=PISI). The downside is that you can't search for an available pool at a specific time, as each pool has its own schedule, which varies significantly. Some pools have their schedules embedded as a table on the website, like this one: [Bain Émard](https://montreal.ca/en/places/bain-emard). However, for other pools, you have to leave the city website and visit a [Facebook page](https://www.facebook.com/photo?fbid=410463438816703&set=a.107090795820637) to see the pool's schedule: [Complexe Sportif de Saint-Laurent](https://montreal.ca/en/places/pools-complexe-sportif-de-saint-laurent). Since I didn't have a fixed time to go to the pool, I found myself constantly navigating through pool websites to find the next available slot. ## Large language models I decided to use an LLM like ChatGPT to address this problem. LLMs excel at understanding complex text, such as pool schedules, and grasping concepts related to time and space, like pool opening and closing times, or pools nearest to a particular address. One challenge is that pool schedules change frequently, and recent schedules weren't included in the LLMs' training data. Here are three possible solutions: - **Fine-tune the model using recent data.** Fine-tuning involves updating the model's weights by training it with a new dataset. This is effective if you have a large, "static" dataset that was not available during the initial training of the LLM. By *static*, I mean data that doesn't change over time. However, this approach can be costly and resource-intensive. - **Provide the entire database alongside the user's query.** It works well if the dataset is relatively small. Current LLMs support a large context window, but if the dataset is too large, this approach becomes impractical because the LLM can only process a limited portion of the prompt and will only consider the most recent part. - **Use retrieval-augmented generation.** RAG integrates LLMs with specific knowledge databases, without the need for retraining the model. This approach is cost-effective and helps keep the LLM's output relevant and accurate. RAG works by first searching through a large database to find relevant information related to a query. Then, it uses this information to generate a response. ![Diagram of a basic RAG pipeline, from documents through indexing and retrieval to the language model](https://ibraverse.ca/tech/rag-montreal-pools/01-rag-pipeline-diagram_hu_1267427c7cce6617.webp) *A basic RAG pipeline. Source: [DeepLearning.AI — Building and Evaluating Advanced RAG](https://learn.deeplearning.ai/courses/building-evaluating-advanced-rag/lesson/2/advanced-rag-pipeline).* ## The design and implementation The pipeline has two halves that run at different times. Everything on the left happens once, offline, when the schedules are collected. Everything on the right happens per question: OFFLINE · ONCE Montréal pool pages HTML tables · Facebook One text file per pool schedule · address · closures Embedding model text → vectors Vector store LlamaIndex PER QUESTION User question ChainLit chat UI Same embedding model semantic search Prompt = context + question retrieved schedules LLM answer + link to the source ### Prepare the dataset We need to collect data on all the indoor pools. I tried to automate this process as much as possible, but there is room for improvement to handle more complex websites. Using some Python packages, I created a text file with the schedule of each pool: ![Directory listing of one text file per Montréal indoor pool](https://ibraverse.ca/tech/rag-montreal-pools/02-pools-dataset_hu_f9bce9f5214c0844.webp) *Montréal indoor pools dataset: one file per pool.* To achieve this, we need to extract all tables and relevant information from each pool's HTML page, such as the pool's address and any exceptional closures. There are tools that can help automate this process, such as LlamaIndex's web readers. See [data connectors](https://docs.llamaindex.ai/en/stable/module_guides/loading/connector/) and the [web reader package](https://llamahub.ai/l/readers/llama-index-readers-web). Each schedule looks something like this: ![Extracted pool schedule as plain text, with days, times and activity types](https://ibraverse.ca/tech/rag-montreal-pools/03-pool-schedule-example_hu_8740f1196404cdc8.webp) *Example of a pool schedule after extraction.* ### Create the embeddings To find relevant information in the database, we can use various search methods, such as semantic search or keyword search. Semantic search finds data based on the intent and contextual meaning of a query, rather than an exact match on query words. So, once the dataset of text files is ready, we need to represent each word or phrase with a vector in a high-dimensional vector space, where words with similar meanings will be close to each other. I used [LlamaIndex](https://www.llamaindex.ai/) to handle data preprocessing. ![Diagram of text being converted into vectors in a high-dimensional embedding space](https://ibraverse.ca/tech/rag-montreal-pools/04-embeddings_hu_9fb15dd984a0123d.webp) *Text becomes vectors.* To create an embedding, we need an embedding model, which is an LLM used to convert text into vectors. It's important to use the same model for both the user query and the database embeddings. ### Prepare the prompt With the embeddings ready, we need to obtain the user query, create an embedding for it, search the database for relevant documents, and prepare the prompt for the LLM. Tools like [ChainLit](https://docs.chainlit.io/get-started/overview) can be used to create the chatbot, handling server creation, message history, themes, parsing, and more. ## Demo Here's the chatbot running: ![The chatbot answering a question about which pool is open next, and linking to the pool's page on the city website](https://ibraverse.ca/tech/rag-montreal-pools/05-chatbot-demo_hu_3fce3af1516ac2fd.webp) *ChatGPT used in a RAG system to answer questions about Montréal pools.* ## The lesson - As a software developer, I always aim to find exact solutions that are deterministic and proven through mathematics, physics, and science. Working with LLMs made me uncomfortable because I couldn't fully trust them, even though the responses were accurate in my case. As far as I know, there's no guarantee that the model will consistently perform well. This is why in my demo I provide the URL to the pool website as a source. - Due to the AI boom, there are countless tools available, with many attempting to achieve similar objectives. Standards and interfaces are not yet well defined, and code that works today may become obsolete in a few months. - The whole thing took an afternoon, with an LLM as a coding partner, to build an app that answers questions with an LLM. - I was occasionally banned from accessing pool websites due to intensive data collection for the database. My apologies to Ville de Montréal for the load. :) ## Sources - Washington Post — [Air Canada must honor refund policy invented by its chatbot](https://www.washingtonpost.com/travel/2024/02/18/air-canada-airline-chatbot-ruling/) - Ville de Montréal — [indoor swimming pools](https://montreal.ca/lieux?mtl_content.lieux.available_activities.code=ACT0&mtl_content.lieux.installation.code=PISI) - [LlamaIndex](https://www.llamaindex.ai/) — [data connectors](https://docs.llamaindex.ai/en/stable/module_guides/loading/connector/) and the [web reader](https://llamahub.ai/l/readers/llama-index-readers-web) (the original `WebPageDemo` URL has since 404'd) - [ChainLit](https://docs.chainlit.io/get-started/overview) - DeepLearning.AI — [Building and Evaluating Advanced RAG](https://learn.deeplearning.ai/courses/building-evaluating-advanced-rag/lesson/2/advanced-rag-pipeline)