Build an LLM RAG Chatbot with LangChain

In today's digital age, chatbots have become essential tools for businesses to interact with customers and provide information efficiently and personalized. Large Language Models (LLMs) offer exceptional capabilities for generating human-quality text but can struggle with tasks requiring in-depth knowledge retrieval. This is where Retrieval-Augmented Generation (RAG) comes in, bridging the gap between information retrieval and generative text.

This blog delves into building an LLM RAG chatbot using LangChain. We'll explore the core concepts step-by-step and equip you with the knowledge to construct your own intelligent chatbot.

Steps to Build an LLM RAG Chatbot with LangChain

Here's a breakdown of the steps involved in building your LLM RAG chatbot: 

Step 1: Get Familiar with LangChain

LangChain is a library of abstractions for Python and JavaScript that can be used to build chatbots that use RAG. First, use your code editor to create a new Python project and virtual environment in this step.

• Firstly, ensure that Python 3.10 or later is installed in your system.
• Now, activate your virtual environment and install the following libraries:

(venv) $ python -m pip install langchain==0.1.0 openai==1.7.2 langchain-openai==0.0.2 
langchain-community==0.0.12 langchainhub==0.1.14 

• Install python-dotenv to manage environment variables:

(venv) $ python -m pip install python-dotenv 

• Next, Add the following folders and files: 

./ 
│ 
├── data/ 
│   └── reviews.csv 
│ 
├── langchain_intro/ 
│   ├── chatbot.py 
│   ├── create_retriever.py 
│   └── tools.py 
│ 
└── .env 

Now, let's build a chatbot with LangChain!

LangChain also provides several features that can help develop chatbots, including:

Chat Models:

LangChain provides several different chat models that can be used to build chatbots. These chat models include:

• Retrieval models: Retrieval models are used to retrieve information from external sources. For example, a retrieval model could retrieve information from a database or a knowledge base.
• Generative models: Generative models are used to generate text. For example, a generative model could create a response to a user query.
• Reinforcement learning models: Reinforcement learning models are used to learn how to improve the performance of a chatbot over time.

Getting started with chat models in LangChain is straightforward. To instantiate an OpenAI chat model, navigate to langchain_intro and add the following code to chatbot.py: 

import dotenv 

from langchain_openai import ChatOpenAI 
  
dotenv.load_dotenv()  

chat_model = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) 

• Prompt Templates:

LangChain provides several different prompt templates that can be used to generate text. These prompt templates can improve the quality of the text generated by the chatbot. For example, a prompt template could be used to specify the style of text that should be generated (e.g., formal or informal). 

For Example: - To build a chatbot that answers questions about patient experiences from their reviews. Here’s what a prompt template might look like for this: 

>>> from langchain.prompts import ChatPromptTemplate 


>>> review_template_str = """Your job is to use patient 

... reviews to answer questions about their experience at a hospital. 

... Use the following context to answer questions. Be as detailed 

... as possible, but don't make up any information that's not 

... from the context. If you don't know an answer, say you don't know. 

... 

... {context} 

... 

... {question} 

... """

>>> review_template = ChatPromptTemplate.from_template(review_template_str)

>>> context = "I had a great stay!" 

>>> question = "Did anyone have a positive experience?"
  

>>> review_template.format(context=context, question=question) 

"Human: Your job is to use patient\nreviews to answer questions about 

their experience at a hospital.\nUse the following context to 

answer questions. Be as detailed\nas possible, but don't make 

up any information that's not\nfrom the context. If you don't 

know an answer, say you don't know.\n\nI had a great 

stay!\n\nDid anyone have a positive experience?\n" 

• Chains and LangChain Expression Language (LCEL):

LangChain allows you to define chains of chat models. These chains can perform complex tasks, such as retrieving information from a database and then using that information to generate a response. LCEL is a language that can be used to define these chains.

To see how this works, look at how you’d create a chain with a chat model and prompt template:

 import dotenv 

from langchain_openai import ChatOpenAI 

from langchain.prompts import ( 

    PromptTemplate, 

    SystemMessagePromptTemplate, 

    HumanMessagePromptTemplate, 

    ChatPromptTemplate, 

)

dotenv.load_dotenv()

review_template_str = """Your job is to use patient 

reviews to answer questions about their experience at 

a hospital. Use the following context to answer questions. 

Be as detailed as possible, but don't make up any information 

that's not from the context. If you don't know an answer, say 

you don't know. 


{context} 

"""

review_system_prompt = SystemMessagePromptTemplate( 

    prompt=PromptTemplate( 

        input_variables=["context"], 

        template=review_template_str, 

    ) 

)

review_human_prompt = HumanMessagePromptTemplate( 

    prompt=PromptTemplate( 

        input_variables=["question"], 

        template="{question}", 

    ) 

) 

messages = [review_system_prompt, review_human_prompt]

review_prompt_template = ChatPromptTemplate( 

    input_variables=["context", "question"], 

    messages=messages, 

) 


chat_model = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)

review_chain = review_prompt_template | chat_model 

• Retrieval Objects:

LangChain provides retrieval objects that can retrieve information from external sources. These objects can also connect LangChain to various data sources, such as databases, knowledge bases, and APIs. There are two main types of retrieval objects in LangChain:

• Vector Retrieval Objects: Vector retrieval objects retrieve information from databases that store data in vector form. For example, a vector retrieval object could retrieve information from a database that stores product embeddings.
• Cypher Retrieval Objects: Cypher retrieval objects retrieve information from graph databases. Graph databases store data in the form of nodes and edges. Nodes represent entities, and edges represent relationships between entities. Cypher is a query language that can be used to query graph databases.

In this step, now install ChromaDB with the following command: 

(venv) $ python -m pip install chromadb==0.4.22 

• Next, use the following code to create a ChromaDB vector database with patient reviews: 

 import dotenv 

from langchain.document_loaders.csv_loader import CSVLoader 

from langchain_community.vectorstores import Chroma 

from langchain_openai import OpenAIEmbeddings 


REVIEWS_CSV_PATH = "data/reviews.csv" 

REVIEWS_CHROMA_PATH = "chroma_data"

dotenv.load_dotenv()

loader = CSVLoader(file_path=REVIEWS_CSV_PATH, source_column="review") 

reviews = loader.load() 

reviews_vector_db = Chroma.from_documents( 

    reviews, OpenAIEmbeddings(), persist_directory=REVIEWS_CHROMA_PATH 

) 

• Now, open a terminal and run the following command from the project directory: 

(venv) $ python langchain_intro/create_retriever.py 

• Afterwards you can start performing semantic search over the review embeddings:
• Now, add your reviews retriever to review_chain so that relevant reviews are passed to the prompt as context. 

Agents:

LangChain agents are chatbots built using LangChain. They consist of one or more chat models chained together using LCEL. The chat models in a LangChain agent can be of any type, including retrieval, generative, and reinforcement learning models.

Step 2: Understand the Business Requirements and Data

Before you start building a chatbot, it is important to understand the business requirements and data. This involves the following steps:

• Understand the Problem and Requirements:

The first step is understanding the problem you are trying to solve with your chatbot. What are the chatbot's goals? What tasks do you want it to perform?

• Explore the Available Data:

Once you understand the problem and requirements, you must explore the available data. What data do you have that can be used to train the chatbot? Is the data in a format that can be used by LangChain?

• Design the Chatbot:

Once you have explored the available data, you can start to design the chatbot. This involves defining the conversation flow of the chatbot and the types of prompts and responses that will be used.

Step 3: Set Up a Neo4j Graph Database

Graph databases store data in the form of nodes and edges. Nodes represent entities, and edges represent relationships between entities. For example, a node might represent a customer, and an edge might represent a customer's purchase.

Graph databases are a good choice for storing data naturally represented as a graph. For example, customer, social network, and product data can be naturally represented as graphs. In this step, we will set up a Neo4j graph database to store the data for your chatbot.

Create a Neo4j Account and AuraDB Instance 

Neo4j is a popular graph database platform. To start with Neo4j, you must create a Neo4j account and create an AuraDB instance. AuraDB is Neo4j's managed graph database service.

Here are the steps on how to create a Neo4j account and AuraDB instance:

• Go to the Neo4j website and create a free account. 
• Once you have created an account, click the "AuraDB" tab. 
• Click on the "Create free cluster" button. 
• Follow the on-screen instructions to create a free AuraDB instance. 

Upload your data to Neo4j 

Once you have designed your graph database schema, you can upload your data to Neo4j. There are several ways to upload data to Neo4j, including the Neo4j browser, the Neo4j command line tool, and the Neo4j REST API. Here are the steps on how to upload data to Neo4j using the Neo4j browser: 

• Open the Neo4j browser and connect to your AuraDB instance. 
• Click on the "Create" button in the top left-hand corner of the screen. 
• Select "Cypher" from the drop-down menu. 
• Paste your Cypher queries to create nodes and edges in your graph database. 
• Click the "Run”. 

Once you have uploaded your data to Neo4j, you can query the graph database with Cipher queries. Cypher is a query language that is designed explicitly for graph databases.

Step 4: Build a Graph RAG Chatbot in LangChain 

Now that you have a Neo4j graph database, you can build a graph RAG chatbot in LangChain. Here are the steps involved: 

Create a Neo4j Vector Chain: 

A Neo4j vector chain is a retrieval object in LangChain that can retrieve information from a Neo4j graph database. To create a Neo4j vector chain, you must specify the Cypher query that will be used to retrieve information from the graph database. 

You can use Neo4jVector to create review embeddings and the retriever needed for your chain. Here’s the code to create the reviews chain: 

import os 

from langchain.vectorstores.neo4j_vector import Neo4jVector 

from langchain_openai import OpenAIEmbeddings 

from langchain.chains import RetrievalQA 

from langchain_openai import ChatOpenAI 

from langchain.prompts import ( 

    PromptTemplate, 

    SystemMessagePromptTemplate, 

    HumanMessagePromptTemplate, 

    ChatPromptTemplate, 

)

HOSPITAL_QA_MODEL = os.getenv("HOSPITAL_QA_MODEL")  

neo4j_vector_index = Neo4jVector.from_existing_graph( 

    embedding=OpenAIEmbeddings(), 

    url=os.getenv("NEO4J_URI"), 

    username=os.getenv("NEO4J_USERNAME"), 

    password=os.getenv("NEO4J_PASSWORD"), 

    index_name="reviews", 

    node_label="Review", 

    text_node_properties=[ 

        "physician_name", 

        "patient_name", 

        "text", 

        "hospital_name", 

    ], 

    embedding_node_property="embedding", 

) 


review_template = """Your job is to use patient 

reviews to answer questions about their experience at a hospital. Use 

the following context to answer questions. Be as detailed as possible, but 

don't make up any information that's not from the context. If you don't know 

an answer, say you don't know. 

{context} 

"""
  

review_system_prompt = SystemMessagePromptTemplate( 

    prompt=PromptTemplate(input_variables=["context"], template=review_template) 

) 


review_human_prompt = HumanMessagePromptTemplate( 

    prompt=PromptTemplate(input_variables=["question"], template="{question}") 

) 

messages = [review_system_prompt, review_human_prompt] 

  

review_prompt = ChatPromptTemplate( 

    input_variables=["context", "question"], messages=messages 

)  

reviews_vector_chain = RetrievalQA.from_chain_type( 

    llm=ChatOpenAI(model=HOSPITAL_QA_MODEL, temperature=0), 

    chain_type="stuff", 

    retriever=neo4j_vector_index.as_retriever(k=12), 

) 

reviews_vector_chain.combine_documents_chain.llm_chain.prompt = review_prompt 

Now, you’re ready to try out your new reviews chain. So, now navigate to the root directory of your project, start a Python interpreter. 

• Create a Neo4j Cypher Chain:  

A Neo4j Cypher chain is another type of retrieval object in LangChain that can retrieve information from a Neo4j graph database. However, unlike Neo4j vector chains, Neo4j Cypher chains return the raw results of the Cypher query rather than embeddings.

For creating your Cypher generation chain, import dependencies and instantiate a Neo4jGraph: 

import os 

from langchain_community.graphs import Neo4jGraph 

from langchain.chains import GraphCypherQAChain 

from langchain_openai import ChatOpenAI 

from langchain.prompts import PromptTemplate 

  

HOSPITAL_QA_MODEL = os.getenv("HOSPITAL_QA_MODEL") 

HOSPITAL_CYPHER_MODEL = os.getenv("HOSPITAL_CYPHER_MODEL") 

  

graph = Neo4jGraph( 

    url=os.getenv("NEO4J_URI"), 

    username=os.getenv("NEO4J_USERNAME"), 

    password=os.getenv("NEO4J_PASSWORD"), 

) 

  

graph.refresh_schema() 

• Create Wait Time Functions:  

Wait time functions can be used to control the timing of the chatbot's responses. This can be useful for simulating human conversation or making the chatbot appear more natural.

• Create the Chatbot Agent:

The chatbot agent is the core of the LangChain chatbot. It consists of one or more chat models chained together using LCEL. The chat models in the chatbot agent can be of any type; we will likely use a retrieval model (e.g., Neo4j vector chain or Neo4j Cypher chain) to retrieve information from the Neo4j graph database and a generative model to generate a response to the user query.

In this step, start by loading your agent’s dependencies, reading in the agent model name from an environment variable, and loading a prompt template from LangChain Hub: 

 import os 

from langchain_openai import ChatOpenAI 

from langchain.agents import ( 

    create_openai_functions_agent, 

    Tool, 

    AgentExecutor, 

) 

from langchain import hub 

from chains.hospital_review_chain import reviews_vector_chain 

from chains.hospital_cypher_chain import hospital_cypher_chain 

from tools.wait_times import ( 

    get_current_wait_times, 

    get_most_available_hospital, 

) 

HOSPITAL_AGENT_MODEL = os.getenv("HOSPITAL_AGENT_MODEL")

hospital_agent_prompt = hub.pull("hwchase17/openai-functions-agent")

• Next, you define a list of tools your agent can use and instantiate you agent: 

# ... 

  

chat_model = ChatOpenAI( 

    model=HOSPITAL_AGENT_MODEL, 

    temperature=0, 

) 

  

hospital_rag_agent = create_openai_functions_agent( 

    llm=chat_model, 

    prompt=hospital_agent_prompt, 

    tools=tools, 

) 

  

hospital_rag_agent_executor = AgentExecutor( 

    agent=hospital_rag_agent, 

    tools=tools, 

    return_intermediate_steps=True, 

    verbose=True, 

) 

Now, deploy your system agent with FastAPI and Streamlit. This makes your agent accessible to anyone who calls the API endpoint or interacts with the Streamlit UI. 

Step 5: Deploy the LangChain Agent

Once you have built your LangChain chatbot agent, you can deploy it to production. Here are a few ways to deploy a LangChain chatbot agent: 

• Serve the Agent With FastAPI: FastAPI is a web framework for building APIs in Python. You can use FastAPI to create an API that exposes your LangChain chatbot agent. FastAPI does this with Pydantic: 

 from pydantic import BaseModel 

  

class HospitalQueryInput(BaseModel): 

    text: str 

  

class HospitalQueryOutput(BaseModel): 

    input: str 

    output: str 

    intermediate_steps: list[str] 

• Create a Chat UI With Streamlit: Streamlit is a Python library for building web applications. You can use It to create a chat UI for your LangChain chatbot agent. Here are the dependencies for the Streamlit UI: 

[project] 

name = "chatbot_frontend" 

version = "0.1" 

dependencies = [ 

   "requests==2.31.0", 

   "streamlit==1.29.0" 

]  

[project.optional-dependencies] 

dev = ["black", "flake8"] 

• Now, create an entrypoint file to run the UI: 

#!/bin/bash 

  

# Run any setup steps or pre-processing tasks here 

echo "Starting hospital chatbot frontend..." 

  

# Run the ETL script 

streamlit run main.py 

• And finally, the Docker file to create an image for the UI: 

FROM python:3.11-slim 

  

WORKDIR /app 

  

COPY ./src/ /app 

  

COPY ./pyproject.toml /code/pyproject.toml 

RUN pip install /code/.

CMD ["sh", "entrypoint.sh"] 

• Orchestrate the Project With Docker Compose: Docker Compose is a tool for defining and running multi-container applications. You can use It to orchestrate your LangChain chatbot deployment, including the FastAPI server and the Streamlit chat UI. 

This last step is to build and run your project with docker-compose. To complete your docker-compose.yml file, add a chatbot_frontend service. Your final docker-compose.yml file look like this: 

 version: '3'

services: 

  hospital_neo4j_etl: 

    build: 

      context: ./hospital_neo4j_etl 

    env_file: 

      - .env 

  

  chatbot_api: 

    build: 

      context: ./chatbot_api 

    env_file: 

      - .env 

    depends_on: 

      - hospital_neo4j_etl 

    ports: 

      - "8000:8000" 

  

  chatbot_frontend: 

    build: 

      context: ./chatbot_frontend 

    env_file: 

      - .env 

    depends_on: 

      - chatbot_api 

    ports: 

      - "8501:8501" 

• Finally, open a terminal and run: 

$ docker-compose up --build 

Once everything builds and runs smoothly, open your web browser and head to the local host. This is the magic portal where you can interact with your chatbot creation. 

Take it for a spin! Ask it questions, explore its strengths and weaknesses, and see how it performs on the example questions provided. This is your chance to identify areas for improvement. Can you refine its responses with better prompts or data feeding? The possibilities are exciting – let's see how well you can train your chatbot! 

Why Build an LLM RAG Chatbot With LangChain?

There are several reasons why you should build an LLM RAG chatbot with LangChain. Here are a few of the benefits:

• More Natural Conversation:

RAG chatbots can generate more natural conversations than traditional LLMs. This is because RAG chatbots can access and process information from external sources, which allows them to provide more comprehensive and informative responses to user queries.

• Reduced Hallucination:

Large language models are notorious for sometimes making things up or hallucinating when they lack context. RAG steps in like a fact-checker, using external knowledge to ground the LLM's responses in reality. This makes your chatbot more trustworthy and reliable.

• Deeper Understanding:

Think of RAG as giving your chatbot a superpower—the ability to truly understand your questions. By referencing external information, the chatbot can grasp the nuances and complexities of your query, leading to more insightful and helpful responses.

• Personalized Interactions:

Imagine a chatbot that remembers your past conversations and tailors its responses accordingly. With RAG, this becomes a reality. The chatbot can access information specific to you, creating a more personalized and engaging interaction. 

• Improved Performance:

LLMs can sometimes need help to answer questions that require them to access and process information from external sources. RAG can help improve LLMs' performance by allowing them to do so.

• Learning and Evolving:

The beauty of RAG is that it's constantly learning. As users interact with the chatbot, the system gathers data on successful retrieval and generation techniques. Over time, the chatbot becomes more adept at understanding your needs and delivering exceptional experiences.

• Cost-Effective Solution:

Building a chatbot from scratch can be expensive and time-consuming. RAG offers a cost-effective alternative by leveraging the power of existing LLMs and knowledge sources. This allows you to create a sophisticated chatbot without breaking the bank.

Here are some additional benefits of using LangChain to build an LLM RAG chatbot: 

• Ease of Use: LangChain provides a high-level abstraction that makes it easy to build chatbots. This is because LangChain takes care of many low-level details, such as connecting to external data sources and managing the conversation flow. 
• Flexibility: LangChain is a flexible platform for building chatbots. It supports various chat models, retrieval objects, and agents.
• Scalability: LangChain is a scalable platform for building chatbots that can handle a large number of users. It can be deployed on various platforms, including cloud platforms.
• Wider Range of Applications: RAG chatbots can be used for broader applications than traditional LLMs. They can be used for customer service, technical support, education, and healthcare.

Future of LLM RAG Chatbots

The future of communication is poised for a dramatic shift, driven by the innovative connection of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) technology. This powerful combination creates a new generation of LLM RAG chatbots that promise to revolutionize how we interact with machines. 

Here are five key ways LLM RAG chatbots are likely to shape the world of tomorrow:

• Revolutionizing Customer Service: 

Imagine a future where customer service interactions are seamless and frustration-free. LLM RAG chatbots hold the key. By combining the power of natural language understanding with real-time information retrieval, these chatbots can answer complex questions, troubleshoot issues efficiently, and even personalize recommendations. This translates to happier customers, reduced wait times, and a significant boost in customer satisfaction.

• Unlocking Personalized Learning:

Education is on the cusp of a transformation, with LLM RAG chatbots acting as intelligent tutors. These AI companions can tailor learning experiences to individual student needs. Imagine a chatbot that explains complex concepts that resonate with your learning style, provides relevant practice problems based on your strengths and weaknesses, and offers real-time feedback to keep you on track. This personalized approach to education has the potential to unlock a new era of effective and engaging learning.

• Bridging the Language Gap:

In a globalized world, language barriers can be a hindrance. LLM RAG chatbots can bridge this gap by offering real-time translation and interpretation. Imagine traveling to a new country and having a chatbot seamlessly translate conversations, navigate cultural nuances, and even help you order local delicacies! This technology has the potential to foster greater cultural understanding and break down communication barriers on a global scale.

• Empowering Creative Professionals:

The future of creativity is collaborative. LLM RAG chatbots can empower writers, artists, and designers by acting as intelligent brainstorming partners. Imagine a chatbot that can generate creative ideas, provide factual information to enrich your work, and even help you overcome writer's block. This technology can streamline the creative process, spark new ideas, and push the boundaries of artistic expression.

• Shaping the Future of Work:

The rise of automation doesn't have to mean job displacement. LLM RAG chatbots can augment human capabilities in the workplace. Imagine a chatbot that can automate routine tasks, freeing your time for strategic thinking and problem-solving. It can also be a virtual assistant, scheduling meetings, summarizing documents, and keeping you organized. With these AI companions, we can work smarter, not harder, and unlock new productivity levels.

These are just a few examples of the exciting future that awaits LLM RAG chatbots. As technology continues to evolve, we can expect even more groundbreaking advancements to reshape how we interact with machines and access information. Struggling to turn your LLM RAG chatbot vision into reality due to lack of Python expertise? Look no further! VLink's vetted Python developers are here to bridge the gap. We'll help you leverage the power of Python to craft the intelligent, interactive chatbot you've been dreaming of. Let's get started!

Hire Python Developers from VLink

VLink  connect you with a pool of talented and experienced Python developers who can bring your chatbot vision to life. VLink's vetting process ensures you get matched with developers who have the specific skills and experience required for your project.

Here are some of the benefits of hiring Python developers through VLink:

• Access to a Global Talent Pool: VLink connects you with a vast network of Python developers worldwide, allowing you to find the perfect fit for your project, regardless of location.
• Rigorous Screening Process: VLink evaluates developers based on their skills, experience, and cultural fit, ensuring you get top-tier talent for your project.
• Streamlined Project Management: VLink handles administrative tasks like payroll and taxes, freeing you to focus on the strategic aspects of your chatbot development.

If you're looking for a reliable and efficient way to acquire the Python expertise needed to build your LLM RAG chatbot, Contact Us Now!

That’s it from our side in this blog. By following the steps mentioned above and leveraging LangChain's power, you can contribute to the exciting future of chatbots and revolutionize how users interact with information. So, don't wait to build an LLM RAG chatbot with LangChain, which empowers you to create an intelligent and informative conversational interface.

Conclusion

LLM RAG chatbots are a powerful new tool for building more informative, engaging, and versatile chatbots than traditional LLMs. With the ongoing advancements in LLM technology and LangChain's capabilities, the future of chatbot development looks incredibly promising.

We can expect to see even more sophisticated and natural-sounding chatbots emerge, capable of handling complex conversations and providing valuable assistance across various domains. Feel free to experiment and get creative – the possibilities are truly limitless!