[行业应用] How to Use Your Own Knowledge Base/Chat History to Build an AI Chatbot Customer Service [Code Sharing]

When it comes to customer service chatbots, many people immediately think of fine-tuning large models, and then after getting started they find that it cannot achieve the expected chat experience. This is a misunderstanding caused by a lack of understanding of the linear workflow of LLM tasks and of the overall AI workflow. When handling LLM tasks, we need to consider multiple implementation methods. This is the sequence: start with prompt engineering > RAG > then fine-tuning. Think of these methods as levers for solving problems; you need to adjust how the levers work together to handle complex and personalized application requirements. These are the main capabilities included in LLMs, so our job is to optimize these capabilities.

Carrying Out a Linear LLM Workflow

A typical LLM task starts with prompt engineering in the lower-left corner, where we test, learn, and evaluate to obtain a baseline. Once we have baseline examples and have evaluated why they are incorrect, we can take one of the following actions:



The main issues in the workflow include

1. RAG optimization:
1) The model lacks contextual knowledge (because it is not in its training set);
2) The knowledge base is outdated or incorrect;
3) Proprietary information/knowledge is required, which is the most critical factor for ensuring accuracy.

2. LLM optimization:
1) The model produces inconsistent results and incorrect formats;
2) The tone or speaking style is incorrect;
3) Reasoning is not consistently followed. This is the most critical factor for ensuring accuracy.

This process actually becomes a series of optimization steps: we evaluate, form hypotheses about how to optimize, apply them, and re-evaluate the next step.

Example



In this example, we perform the following operations:
[Prompt] Start with the prompt, input the prompt, and then evaluate its performance.
[Prompt engineering] Add sample references, which should improve the consistency of results.
[RAG] Add a retrieval step so that a small number of samples can be dynamically introduced based on the question; this improves performance by ensuring relevant context for each input.
[Fine-tuning the large model] Prepare a dataset of more than 50 examples and fine-tune the model to improve style consistency.
[RAG] Adjust retrieval and add a fact-checking step to find errors, thereby achieving higher accuracy.
[RAG] Retrain the fine-tuned model using new training examples that include enhanced RAG inputs.

This is a typical optimization process for complex, personalized business problems. It helps us decide whether more relevant context is needed, or whether more consistent behavior in the model is needed.
To use a somewhat imprecise analogy, RAG is like a knowledge base, fine-tuning is like the mouth, and prompt engineering is like the command. Although the analogy is not quite appropriate, it is easy to understand.

A Practical Example of Building a Customer Service Chatbot

Let us ensure that answers to questions meet expectations by using a combination of prompt engineering, RAG, and fine-tuning to implement the underlying technical logic of a bot that can be easily built in any industry.
I am sharing the code I have written; you can copy it directly.

import openai
import pandas as pd
from sentence_transformers import SentenceTransformer, util
import torch
import re

# 设置API密钥
openai.api_key = "your_api_key_here"

# 初始化SentenceTransformer模型
embedder = SentenceTransformer('paraphrase-MiniLM-L6-v2')

# 使用新的UTF-8编码的CSV文件路径
knowledge_base_path = 'D:\\360MoveData\\Users\\Administrator\\Desktop\\数据集\\knowledge_base_fixed_utf8.csv'

# 读取CSV文件
def read_csv_with_encodings(file_path):
    df = pd.read_csv(file_path, encoding='utf-8')
    print(f"Successfully read the file with encoding: utf-8")
    return df

# 读取并打印CSV文件内容
knowledge_base = read_csv_with_encodings(knowledge_base_path)
knowledge_base['question_processed'] = knowledge_base['question'].apply(lambda x: re.sub(r'\W+', '', x.lower()))
knowledge_base_embeddings = embedder.encode(knowledge_base['question_processed'].tolist(), convert_to_tensor=True)

def preprocess_text(text):
    return re.sub(r'\W+', '', text.lower())

def retrieve_relevant_context(query):
    query_embedding = embedder.encode(preprocess_text(query), convert_to_tensor=True)
    scores = util.pytorch_cos_sim(query_embedding, knowledge_base_embeddings)
    best_match_index = torch.argmax(scores).item()  # 确保转换为整数
    best_match_score = scores[0, best_match_index].item()  # 获取最高分数
    return knowledge_base.iloc[best_match_index], best_match_score

def query_gpt_with_context(query, context, model="gpt-3.5-turbo"):
    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": query},
        {"role": "assistant", "content": context}
    ]
    response = openai.ChatCompletion.create(
        model=model,
        messages=messages
    )
    return response.choices[0]['message']['content'].strip()

# 示例对话
def chat():
    while True:
        user_query = input("You: ")
        if user_query.lower() in ['exit', 'quit']:
            break
        context_row, best_match_score = retrieve_relevant_context(user_query)

        if best_match_score > 0.7:  # 设置阈值为0.7
            # 知识库中找到了匹配的答案
            platform_name = "平台名称"
            context = context_row['answer'].replace("{platform}", platform_name)
            response = context
        else:
            # 知识库中没有找到匹配的答案，使用微调模型生成回答
            response = query_gpt_with_context(user_query, "", model="ft:gpt-3.5-turbo-0125:personal:lianai:9aKg8G6U")
        
        print("Bot:", response)

if __name__ == "__main__":
    chat()