技术应用 Development principles of AI customer service chatbots, and the RAG implementation methods and specific code for the OpenAI ChatGPT API

When it comes to customer service chatbots, many people immediately think of fine-tuning large models, and then after trying it they find that it cannot achieve the expected chat experience. This is a misconception, caused by a lack of understanding of the linear workflow of LLM tasks and of the overall AI workflow. When handling LLM tasks, we need to consider multiple implementation methods, and this is the order:start with prompt engineering > RAG > then fine-tuning. Think of these methods as levers for solving problems; you need to adjust how the levers work together to handle complex and personalized application requirements. These are the main capabilities included in LLMs, so our job is to optimize these capabilities.

Develop an LLM linear workflow

A typical LLM task starts from prompt engineering in the lower-left corner, where we test, learn, and evaluate to obtain a baseline. Once we have baseline examples and have assessed why they are incorrect, we can take one of these actions:



The main problems in the workflow include

1. RAG optimization:
1) The model lacks contextual knowledge (because it is not in its training set);
2) The knowledge base is outdated or incorrect;
3) Proprietary information/knowledge is required, which is the most critical factor in ensuring accuracy.

2. LLM optimization:
1) The model produces inconsistent results and incorrect formats;
2) The tone or speaking style is incorrect;
3) Reasoning is not followed consistently. This is the most critical factor in ensuring accuracy.

This process actually becomes a series of optimization steps: we evaluate, form a hypothesis about how to optimize, apply it, and re-evaluate the next step


Example illustration



In this example, we perform the following operations:
[Prompt] Start with the prompt, input the prompt, and then evaluate its performance
[Prompt engineering] Add sample references, which should improve consistency of results
[RAG] Add a retrieval step so that a small number of samples can be dynamically introduced based on the question; this improves performance by ensuring relevant context for each input
[Fine-tune the large model] Prepare a dataset of more than 50 examples and fine-tune the model to improvestyle consistency
[RAG] Adjust retrieval and add a fact-checking step to detect errors, thereby achieving higher accuracy
[RAG] Retrain the fine-tuned model using new training examples that include enhanced RAG inputs

This is a typical optimization process for complex, personalized business problems; it helps us decide whether more relevant context is needed, or whether more consistent behavior is needed in the model.
Here is a somewhat inexact analogy: RAG is like a knowledge base, fine-tuning is like the mouth, and prompt engineering is like a command. Although it is not quite accurate, it is easy to understand.

A practical example of building a customer service chatbot

Let us ensure that answers meet expectations by using a combination of prompt engineering and RAG fine-tuning to implement the underlying technical logic of a robot that can be easily built in any industry.
I am sharing the code I have written; you can copy it directly.

import openai
import pandas as pd
from sentence_transformers import SentenceTransformer, util
import torch
import re

# Set API key
openai.api_key = "your_api_key_here"

# Initialize SentenceTransformer model
embedder = SentenceTransformer('paraphrase-MiniLM-L6-v2')

# Use the new UTF-8 encoded CSV file path
knowledge_base_path = 'D:\\360MoveData\\Users\\Administrator\\Desktop\\数据集\\knowledge_base_fixed_utf8.csv'

# Read CSV file
def read_csv_with_encodings(file_path):
    df = pd.read_csv(file_path, encoding='utf-8')
    print(f"Successfully read the file with encoding: utf-8")
    return df

# Read and print CSV file contents
knowledge_base = read_csv_with_encodings(knowledge_base_path)
knowledge_base['question_processed'] = knowledge_base['question'].apply(lambda x: re.sub(r'\W+', '', x.lower()))
knowledge_base_embeddings = embedder.encode(knowledge_base['question_processed'].tolist(), convert_to_tensor=True)

def preprocess_text(text):
    return re.sub(r'\W+', '', text.lower())

def retrieve_relevant_context(query):
    query_embedding = embedder.encode(preprocess_text(query), convert_to_tensor=True)
    scores = util.pytorch_cos_sim(query_embedding, knowledge_base_embeddings)
    best_match_index = torch.argmax(scores).item()  # Ensure conversion to integer
    best_match_score = scores[0, best_match_index].item()  # Get highest score
    return knowledge_base.iloc[best_match_index], best_match_score

def query_gpt_with_context(query, context, model="gpt-3.5-turbo"):
    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": query},
        {"role": "assistant", "content": context}

    response = openai.ChatCompletion.create(
        model=model,
        messages=messages
    )
    return response.choices[0]['message']['content'].strip()

# Example conversation
def chat():
    while True:
        user_query = input("You: ")
        if user_query.lower() in ['exit', 'quit']:
            break
        context_row, best_match_score = retrieve_relevant_context(user_query)

        if best_match_score > 0.7:  # Set threshold to 0.7
            # A matching answer was found in the knowledge base
            platform_name = "平台名称"
            context = context_row['answer'].replace("{platform}", platform_name)
            response = context
        else:
            # No matching answer was found in the knowledge base; use the fine-tuned model to generate a response
            response = query_gpt_with_context(user_query, "", model="ft:gpt-3.5-turbo-0125:personal:lianai:9aKg8G6U")
        
        print("Bot:", response)

if __name__ == "__main__":
    chat()