# rag

**Repository Path**: daxiang2008/rag

## Basic Information

- **Project Name**: rag
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-06-30
- **Last Updated**: 2026-06-30

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Vector Knowledge Base System

## Project Overview

This project is a local vector knowledge base system built with Python, with two core functions:
1. Provide vector support for natural language development systems (such as RAG, semantic search)
2. Serve as an independent knowledge base for other users

## System Architecture

### Overall Architecture Diagram
```
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  Frontend Layer │    │   API Service   │    │ Vector KB Core  │
│                 │    │      Layer      │    │                 │
│ Vue3 + Element  │───▶│ Flask REST API  │───▶│ Vector Database │
│     UI          │    │    (Python)     │    │    (FAISS)      │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                             │                        │
                             │                        │
                    ┌─────────────────┐    ┌─────────────────┐
                    │  Data Processing│    │  Index Management│
                    │      Layer      │    │      Layer      │
                    │                 │    │                 │
                    │ Doc Parsing     │    │ Vector Index Ops│
                    │ Text Segmentation│   │ Search Optimize │
                    │ Feature Extraction│  │ Cache Mechanism │
                    └─────────────────┘    └─────────────────┘
```

## Technology Stack

### Backend Tech Stack
- **Web Framework**: Flask (Python)
- **Vector Database**: FAISS (for efficient similarity search)
- **Text Embedding Model**: sentence-transformers/all-MiniLM-L6-v2
- **Data Storage**: PostgreSQL (metadata and vector storage)
- **Dependency Management**: pip + requirements.txt
- **Virtual Environment**: Conda

### Frontend Tech Stack
- **Framework**: Vue3
- **UI Library**: Element Plus
- **Build Tool**: Vite
- **Package Manager**: npm

## Project Structure

```
vector-knowledge-base/
├── backend/                 # Backend code
│   ├── app/
│   │   ├── config/          # Configuration files
│   │   ├── models/          # Data models
│   │   ├── controllers/     # Controllers
│   │   ├── services/        # Business logic layer
│   │   ├── utils/           # Utility functions
│   │   ├── database.py      # Database connection
│   │   └── main.py          # Application entry
│   ├── requirements.txt     # Python dependencies
│   ├── run.py               # Start script
│   └── .env                 # Environment variables
├── frontend/                # Frontend code
│   ├── src/
│   │   ├── components/      # Components
│   │   ├── views/           # Page views
│   │   ├── assets/          # Static assets
│   │   ├── router/          # Router configuration
│   │   └── main.js          # Frontend entry
│   ├── package.json         # Frontend dependencies
│   └── vite.config.js       # Build configuration
├── README.md                # Project documentation
└── optimization_plan.md     # Optimization plan
```

## Functional Modules

### 1. Knowledge Base Management Module
- Document upload and parsing (PDF, TXT, DOCX, etc.)
- Text segmentation and vectorization
- Vector index building and management
- Metadata storage (document info, tags, etc.)
- Document version control (enhanced feature)

### 2. Search Service Module
- Vector similarity search
- Keyword hybrid search
- Result sorting and filtering
- Search history records
- Multi-dimensional search optimization (enhanced feature)

### 3. API Interface Module
- Document management API (CRUD)
- Search API (text search, similarity search)
- Index management API
- System configuration API
- User permission management API (enhanced feature)

### 4. Frontend Display Module
- Document upload interface
- Search result display
- Knowledge base statistics
- Operation log viewing
- Multi-language support (enhanced feature)

## Installation and Deployment

### Development Environment Requirements
- Python 3.9+
- Node.js v16+
- Miniconda (for virtual environment management)
- pip package manager
- npm package manager

### Environment Setup Steps
1. **Create Conda Virtual Environment**:
   ```bash
   conda create -n vector-kb python=3.9
   conda activate vector-kb
   ```

2. **Install Backend Dependencies**:
   ```bash
   cd backend
   pip install -r requirements.txt
   ```

3. **Install Frontend Dependencies**:
   ```bash
   cd frontend
   npm install
   ```

4. **Configure Environment Variables**:
   Create a `.env` file in the project root directory, configure parameters with reference to the example

### Running Steps
1. **Start Backend Service**:
   ```bash
   cd backend
   python run.py
   ```

2. **Start Frontend Service** (in a new terminal):
   ```bash
   cd frontend
   npm run dev
   ```

### System Configuration

#### Backend Environment Variables Example (`.env` file)
```
# Database configuration
DATABASE_URL=postgresql://admin:password@localhost:5432/knowledge_base

# Service configuration
SECRET_KEY=your-secret-key-here
HOST=0.0.0.0
PORT=5000

# Vector search configuration
EMBEDDING_MODEL=sentence-transformers/all-MiniLM-L6-v2

# File upload configuration
UPLOAD_FOLDER=./uploads
MAX_CONTENT_LENGTH=16777216
```

## Enhanced Features

### 1. Version Control Support
- Support management of different versions of the same document
- Provide version comparison and rollback functions

### 2. Search Optimization
- Multi-dimensional search result sorting and filtering
- Support boolean retrieval and advanced search syntax
- Search history records and user preference saving

### 3. User Permission Management
- User role management and access control
- Authentication and authorization system
- Operation log tracking

### 4. Multi-language Support
- Interface multi-language switching
- Cross-language document processing capability

## API Documentation

### Document Management API
- `GET /api/documents/` - Get document list  
- `POST /api/documents/` - Create new document
- `GET /api/documents/{id}` - Get specified document
- `PUT /api/documents/{id}` - Update document
- `DELETE /api/documents/{id}` - Delete document

### Search API
- `POST /api/search/` - Document search
- `GET /api/search/health` - Health check

## Development Plan and Deployment

The project is built with the following features:

1. **Local Deployment**: All data is stored locally, ensuring security
2. **Vector Search**: Uses advanced semantic similarity search algorithms
3. **Multi-format Support**: Supports parsing and management of multiple document formats
4. **API Friendly**: Provides complete RESTful API interfaces
5. **Easy Extension**: Modular design for easy feature expansion

### Future Optimization Directions

1. Add user permission management module
2. Integrate more advanced search algorithms (e.g., Elasticsearch)
3. Support more types of document parsing
4. Enhance frontend UI interaction experience
5. Add data import/export functionality
6. Implement document version control
7. Support multi-language and internationalization