Skip to content

QP: My Personal AI System

QP Logo

QP (Quick Prompt) is my personal AI system that combines local language models with custom pipelines for an integrated AI experience. It's designed to provide powerful AI assistance while keeping data private and under my control.

System Overview

QP is built on several core components:

  1. OpenWebUI - A powerful, customizable web interface for interacting with AI models
  2. Ollama - Local LLM server for running models on my personal hardware
  3. Custom Pipelines - Integration points for connecting AI to my knowledge and tools
  4. Vector Databases - Weaviate and Chroma for knowledge retrieval
  5. Knowledge Management - Integration with Obsidian via Captain's Log

Key Features

Local-First Architecture

QP runs entirely on my local infrastructure, which means:

  • Complete privacy - my data never leaves my network
  • No ongoing API costs for interactions
  • Full control over model selection and parameters
  • Ability to operate offline

The system leverages Weaviate for efficient semantic search:

  • High-performance vector database optimized for AI operations
  • Real-time data processing with low latency
  • Native AI model integration for embeddings
  • Advanced filtering and hybrid search capabilities

Custom Pipelines

I've developed several custom pipelines that extend QP's capabilities:

  • Captain's Log Pipeline - Connects AI models directly to my Obsidian vault
  • Vector Search Pipeline - Enables semantic search across my knowledge base
  • Image Generation Pipeline - Creates images using Flux and other models
  • Code Analysis Pipeline - Helps with code understanding and development

Model Flexibility

QP supports multiple models:

  • Large models like Claude and GPT-4 (via API when needed)
  • Local models like Llama 3, Mistral, and Stable Diffusion
  • Specialized models for specific tasks
  • Unified access through LiteLLM API proxy

Technical Implementation

Hardware Requirements

QP runs on:

  • NVIDIA RTX 4090 GPU
  • 128GB System RAM
  • High-speed SSD storage with JuiceFS

Storage Architecture

  • JuiceFS provides a unified storage layer with:
  • Data sharing across services
  • Cache acceleration
  • Cost-effective storage management
  • High availability

Software Architecture

graph TD
    User[User] --> WebUI[OpenWebUI]
    WebUI --> LiteLLM[LiteLLM API]
    LiteLLM --> Ollama[Ollama LLM Server]
    LiteLLM --> ExternalAPI[External APIs]
    WebUI --> Pipelines[Custom Pipelines]
    Pipelines --> Weaviate[Weaviate Vector DB]
    Pipelines --> CaptainsLog[Captain's Log]
    CaptainsLog --> Obsidian[Obsidian Vault]
    Pipelines --> Tools[System Tools]
    Ollama --> Models[(Local Models)]
    Weaviate --> Knowledge[(Knowledge Base)]
    WebUI --> Windmill[Windmill Automation]

Integration with Abyss

QP is a core component of my Abyss infrastructure, integrating with:

  • Media services for content analysis
  • Documentation for knowledge management
  • Automation systems for task execution

Recent Developments

Captain's Log Pipeline

My newest addition to QP is the Captain's Log Pipeline, which enables bidirectional communication between AI models and my Obsidian knowledge vault. This allows the AI to:

  • Read notes and knowledge directly from my vault
  • Create new notes based on conversations
  • Append to existing notes for ongoing projects
  • Organize information according to my established structures

Windmill Automation Integration

I've recently added Windmill for workflow automation:

  • Visual workflow editor for creating custom automations
  • Script execution for complex tasks
  • Scheduled tasks for routine operations
  • Integration with the rest of the QP ecosystem

Future Directions

I'm continuously improving QP with new features:

  • Enhanced RAG capabilities with multi-vector retrieval
  • Fine-tuning models on personal data
  • Additional specialized tools and pipelines
  • Improved cross-platform access

Resources

For those interested in building similar systems:

Feel free to contact me if you're interested in discussing AI systems or have questions about my approach!