使用 Redis、LangChain 和 OpenAI 构建电子商务聊天机器人

pip install langchain==0.0.123
pip install openai==0.27.2
pip install redis==4.5.3
pip install numpy
pip install pandas
pip install gdown

gdown --id 1tHWB6u3yQCuAgOYc-DxtZ8Mru3uV5_lj

import pandas as pd

MAX_TEXT_LENGTH=1000  # Maximum num of text characters to use

def auto_truncate(val):

    """Truncate the given text."""

    return val[:MAX_TEXT_LENGTH]

# Load Product data and truncate long text fields

all_prods_df = pd.read_csv("product_data.csv", converters={

    'bullet_point': auto_truncate,

    'item_keywords': auto_truncate,

    'item_name': auto_truncate

})

# Replace empty strings with None and drop

all_prods_df['item_keywords'].replace('', None, inplace=True)

all_prods_df.dropna(subset=['item_keywords'], inplace=True)

# Reset pandas dataframe index

all_prods_df.reset_index(drop=True, inplace=True)

# Num products to use (subset)
NUMBER_PRODUCTS = 2500  

# Get the first 2500 products
product_metadata = ( 
    all_prods_df
     .head(NUMBER_PRODUCTS)
     .to_dict(orient='index')
)

# Check one of the products
product_metadata[0]

{'item_id': 'B07T2JY31Y',
 'marketplace': 'Amazon',
 'country': 'IN',
 'main_image_id': '71vX7qIEAIL',
 'domain_name': 'amazon.in',
 'bullet_point': '3D Printed Hard Back Case Mobile Cover for Sony Xperia Z1 L39H Easy to put & take off with perfect cutouts for volume buttons, audio & charging ports. Stylish design and appearance, express your unique personality. Extreme precision design allows easy access to all buttons and ports while featuring raised bezel to life screen and camera off flat surface. Slim Hard Back Cover No Warranty',
 'item_keywords': 'mobile cover back cover mobile case phone case mobile panel phone panel LG mobile case LG phone cover LG back case hard case 3D printed mobile cover mobile cover back cover mobile case phone case mobile panel phone panel Sony Xperia mobile case Sony Xperia phone cover Sony Xperia back case hard case 3D printed mobile cover mobile cover back cover mobile case phone case mobile panel phone panel Sony Xperia mobile case Sony Xperia phone cover Sony Xperia back case hard case 3D printed mobile cover mobile cove',
 'material': 'Wood',
 'brand': 'Amazon Brand - Solimo',
 'color': 'others',
 'item_name': 'Amazon Brand - Solimo Designer Leaf on Wood 3D Printed Hard Back Case Mobile Cover for Sony Xperia Z1 L39H',
 'model_name': 'Sony Xperia Z1 L39H',
 'model_number': 'gz8056-SL40528',
 'product_type': 'CELLULAR_PHONE_CASE'}

import os

from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores.redis import Redis as RedisVectorStore

# set your openAI api key as an environment variable
os.environ['OPENAI_API_KEY'] = "YOUR OPENAI API KEY"

# data that will be embedded and converted to vectors
texts = [
    v['item_name'] for k, v in product_metadata.items()
]

# product metadata that we'll store along our vectors
metadatas = list(product_metadata.values())

# we will use OpenAI as our embeddings provider
embedding = OpenAIEmbeddings()

# name of the Redis search index to create
index_name = "products"

# assumes you have a redis stack server running on local host
redis_url = "redis://localhost:6379"

# create and load redis with documents
vectorstore = RedisVectorStore.from_texts(
    texts=texts,
    metadatas=metadatas,
    embedding=embedding,
    index_name=index_name,
    redis_url=redis_url
)

from langchain.callbacks.base import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.chains import (
    ConversationalRetrievalChain,
    LLMChain
)
from langchain.chains.question_answering import load_qa_chain
from langchain.llms import OpenAI
from langchain.prompts.prompt import PromptTemplate

template = """Given the following chat history and a follow up question, rephrase the follow up input question to be a standalone question.
Or end the conversation if it seems like it's done.
Chat History:\"""
{chat_history}
\"""
Follow Up Input: \"""
{question}
\"""
Standalone question:"""

condense_question_prompt = PromptTemplate.from_template(template)

template = """You are a friendly, conversational retail shopping assistant. Use the following context including product names, descriptions, and keywords to show the shopper whats available, help find what they want, and answer any questions.

It's ok if you don't know the answer.
Context:\"""

{context}
\"""
Question:\"
\"""

Helpful Answer:"""

qa_prompt= PromptTemplate.from_template(template)

# define two LLM models from OpenAI
llm = OpenAI(temperature=0)

streaming_llm = OpenAI(
    streaming=True,
    callback_manager=CallbackManager([
        StreamingStdOutCallbackHandler()
    ]),
    verbose=True,
    max_tokens=150,
    temperature=0.2
)

# use the LLM Chain to create a question creation chain
question_generator = LLMChain(
    llm=llm,
    prompt=condense_question_prompt
)

# use the streaming LLM to create a question answering chain
doc_chain = load_qa_chain(
    llm=streaming_llm,
    chain_type="stuff",
    prompt=qa_prompt
)

chatbot = ConversationalRetrievalChain(
    retriever=vectorstore.as_retriever(),
    combine_docs_chain=doc_chain,
    question_generator=question_generator
)

# create a chat history buffer
chat_history = []
# gather user input for the first question to kick off the bot
question = input("Hi! What are you looking for today?")

# keep the bot running in a loop to simulate a conversation
while True:
    result = chatbot(
        {"question": question, "chat_history": chat_history}
    )
    print("\n")
    chat_history.append((result["question"], result["answer"]))
    question = input()

Hi! What are you looking for today?

>> gold-plated earrings

Hi there! I'm happy to help you find the perfect earrings. Do you have a preference for yellow gold plated sterling silver or platinum or gold-plated sterling silver?

>> My preference is the yellow gold plated sterling silver

Hi there! Are you looking for yellow gold-plated sterling silver earrings with Swarovski Zirconia or Topaz gemstones? We have a few options that might fit the bill. We have yellow gold-plated sterling silver Swarovski Zirconia fancy green stud earrings, yellow gold-plated sterling silver honey topaz stud earrings made with Swarovski Topaz gemstones, and yellow gold-plated sterling silver antique drop earrings set.

import json
from langchain.schema import BaseRetriever
from langchain.vectorstores import VectorStore
from langchain.schema import Document
from pydantic import BaseModel

class RedisProductRetriever(BaseRetriever, BaseModel):
    vectorstore: VectorStore

    class Config:
        arbitrary_types_allowed = True

    def combine_metadata(self, doc) -> str:
        metadata = doc.metadata
        return (
           "Item Name: " + metadata["item_name"] + ". " +
           "Item Description: " + metadata["bullet_point"] + ". " +
           "Item Keywords: " + metadata["item_keywords"] + "."
        )

    def get_relevant_documents(self, query):
        docs = []
        for doc in self.vectorstore.similarity_search(query):
            content = self.combine_metadata(doc)
            docs.append(Document(
                page_content=content,
                metadata=doc.metadata
            ))

        return docs

redis_product_retriever = RedisProductRetriever(vectorstore=vectorstore)

chatbot = ConversationalRetrievalChain(
    retriever=redis_product_retriever,
    combine_docs_chain=doc_chain,
    question_generator=question_generator
)

Hi! What are you looking for today?

>>> fancy footwear for going out

Hi there! We have a few great options for women's shoes and sandals. We have the Amazon Brand - The Fix Women's Giana Open Toe Bootie with Pearl Buckle, bright white leather, 9.5 B US, Flavia Women's Beige Fashion Sandals-7 UK (39 EU) (8 US) (FL/236/BEG), Flavia Women's Blue Fashion Sandals-8 UK (40 EU) (9 US) (FL/211/BLU), and The Fix Women's Faris Flat Slide Sandal with Pearls. All of these shoes feature a variety of styles and colors to choose from. Let me know if you have any questions about any of these items!

>>> These are nice. However, I am looking for men's shoes. Can you help me?

Hi there! We have a great selection of men's formal shoes available. We have Amazon Brand - Symbol Men's Formal Shoes, Amazon Brand - Symbol Men's Leather Formal Shoes, and more. All of our formal shoes are made from high quality materials and feature a variety of closure types, toe styles, and heel types. They also come with a manufacturer's warranty and care instructions to ensure they last. Let me know if you have any questions or need help finding the perfect pair of shoes for you!

>>>Can you show me some more men's options?

Hi there! I'm here to help you find the perfect item for you. We have a few options available for men's formal shoes. We have the Men's Stainless Steel Link Bracelet, the Amazon Brand - Arthur Harvey Men's Leather Formal Shoes, and the Amazon Brand - Symbol Men's Formal Derby shoes. All of these items feature a variety of features such as leather material, lace-up closure, pointed toe, block heel, and more. If you have any questions about any of these items, please let me know. I'm happy to help!

>>> Ok this looks great, thanks!