Building Enterprise Intelligent Operations Knowledge Base with RAG

1. The Pain of Operations Knowledge Management

In large organizations, operations knowledge is often scattered across Wikis, Runbooks, Slack records, and incident reports. On-call engineers must navigate multiple systems to locate critical information. RAG technology offers a new approach to solving this problem.

2. System Architecture

We built a complete RAG pipeline based on the LangChain framework: documents are parsed, split, and stored in Chroma vector database; when an engineer asks a question, the system first retrieves the most relevant document fragments, then generates an answer via an LLM.

3. Key Technologies and Implementation

We chose text-embedding-3-small as the embedding model, with a chunk size of 512 tokens and 64 token overlap. The retrieval uses a hybrid strategy: vector similarity + BM25 keyword weighting.

LangChain RAG Pipeline

python
from langchain_community.vectorstores import Chroma
from langchain_community.embeddings import OpenAIEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chains import RetrievalQA
from langchain_openai import ChatOpenAI

text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=512,
    chunk_overlap=64,
    separators=["\n\n", "\n", ".", " ", ""]
)
documents = text_splitter.split_documents(raw_docs)

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectordb = Chroma.from_documents(
    documents=documents,
    embedding=embeddings,
    persist_directory="./chroma_db"
)

qa_chain = RetrievalQA.from_chain_type(
    llm=ChatOpenAI(model="gpt-4o", temperature=0),
    retriever=vectordb.as_retriever(search_kwargs={"k": 5}),
    return_source_documents=True
)

result = qa_chain.invoke({"query": "How to recover Redis data after a master-slave switchover?"})
print(result["result"])

Service Configuration

yaml
version: '3.8'
services:
  chroma:
    image: chromadb/chroma:latest
    ports:
      - "8000:8000"
    volumes:
      - ./chroma_data:/chroma/chroma
    environment:
      - IS_PERSISTENT=true
      - PERSIST_DIRECTORY=/chroma/chroma
    deploy:
      resources:
        limits:
          memory: 2G

  rag-api:
    build: .
    ports:
      - "8080:8080"
    depends_on:
      - chroma
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - CHROMA_HOST=chroma
      - CHROMA_PORT=8000

4. Summary

After deploying the RAG knowledge base, the average fault localization time for on-call engineers decreased from 45 minutes to 12 minutes. Future plans include adding multi-turn conversation context understanding and proactive recommendation capabilities.