Skip to content

Abyss: Integrated Services Platform

Abyss Logo

Abyss is my comprehensive self-hosted infrastructure platform that integrates media management, AI services, knowledge systems, and storage solutions in a unified ecosystem.

System Overview

Abyss is designed as a modular, containerized platform that provides:

  1. Media Services - Content acquisition, organization, and streaming
  2. AI Infrastructure - LLM servers, vector databases, and custom pipelines
  3. Knowledge Management - Documentation, note-taking, and information organization
  4. Infrastructure Services - Monitoring, backups, and system management

Core Technology Stack

Storage Solutions

  • JuiceFS - Distributed POSIX-compliant file system
  • Local SSDs/HDDs - Tiered storage approach
  • Network Storage - For backups and archives

Databases

  • PostgreSQL - Primary relational database
  • Weaviate - High-performance vector database
  • Chroma - Document embeddings and retrieval

AI Components

  • OpenWebUI - Web interface for AI model interaction
  • Ollama - Local LLM server
  • LiteLLM - Unified API for LLM providers

Media Services (PlexAbyss)

  • Plex - Media server for movies, TV shows, music, and photos
  • Sonarr/Radarr - Automated content management
  • Prowlarr - Indexer management
  • Overseerr - Request management and discovery interface

Knowledge Services

  • Captain's Log - Custom Obsidian integration with API access
  • MkDocs - Documentation system
  • Obsidian - Personal knowledge management
  • Radicale - Calendar and contacts server

Automation

  • n8n - Workflow automation tool
  • Windmill - Workflow automation platform

Infrastructure Services

  • Portainer - Container management
  • Grafana/Prometheus - Monitoring and alerting
  • Restic - Backup system
  • Traefik - Reverse proxy and SSL termination

Technical Implementation

Deployment Architecture

Abyss is deployed using Docker Compose with multiple compose files for different service groups:

  • plexabyss.yml - Media services
  • augment.yml - AI services
  • sara.yml - Storage and infrastructure services
  • docs.yml - Documentation and knowledge services

Captain's Log Integration

A custom service I developed that connects OpenWebUI with my Obsidian knowledge base:

  • Secure API for reading/writing to Obsidian vault
  • Function calling integration with AI models
  • Template system for daily logs
  • Path validation for security

Recent Developments

Recent additions to Abyss include:

  • Captains Log Pipeline - Connecting AI models to my Obsidian vault
  • Hoarder Integration - Enhanced media acquisition and organization
  • Multi-GPU Support - Utilizing multiple GPUs for different AI workloads
  • Improved Vector Databases - Migration to Weaviate for better performance

Future Directions

I'm continuously evolving Abyss with:

  • Enhanced RAG pipelines for knowledge retrieval
  • Improved media organization with AI tagging
  • Better integration between services
  • Hardware upgrades for increased processing power

Technologies Used

  • Languages: Python, JavaScript, Bash
  • Infrastructure: Docker, Docker Compose
  • Networking: Traefik, Caddy
  • Monitoring: Prometheus, Grafana, Loki

Resources

For those interested in building similar systems:

If you're interested in discussing self-hosted infrastructure or have questions about my approach, feel free to contact me!