Lake of Tears Logo Lake of Tears Open Source
Star on GitHub
Open Source & Free • Self-Hosted Databricks Alternative • v0.2.0 Helm Ready

All your data tools in one unified lakehouse. Without the enterprise price tag.

Lake of Tears is an open-source, self-hosted datalakehouse engineered for Kubernetes and Docker. Combines S3-compatible storage, DuckDB SQL execution, automated Airflow pipelines, JupyterLab notebooks, Superset BI, and Gemini AI search behind a single entry point.

Quick Start (Helm / Docker) Explore Architecture GitHub Repository
lake-of-tears — unified entry point (port 80 / Ingress)
Single URL: http://lake.local
# Deploy production datalakehouse on Kubernetes with Helm:
$ helm install lake-of-tears ./deploy/helm/lake-of-tears -f values-prod.yaml -f values-secret.yaml
# Or deploy on Docker Compose with one command:
$ cp .env.example .env && docker compose up -d
✨ Instant access: Lake UI shell, DuckDB SQL Editor, JupyterLab, Apache Superset, Airflow DAGs, and MinIO S3 storage.

Built For Scale & Simplicity

Everything Databricks offers, without the lock-in

Lake of Tears brings together the best open-source analytical tools into a cohesive, beautifully integrated platform.

Unified Web Shell

Databricks-inspired navigation with Notebooks, SQL Editor, Dashboards, Ingestion, and AI Query in a single sidebar. Proxied through a single nginx entry point—no port hopping.

Jupyter · Superset · Airflow embedded

DuckDB Analytical SQL

Run blazingly fast SQL directly over Hive-partitioned Parquet files on S3. In-process analytical processing with zero Spark server overhead, cold-start latency, or warehouse compute costs.

Hive Partitioned · S3-Native · Zero-Server

Native Vector Search & RAG

No expensive separate vector database (no Pinecone or Qdrant needed). Embeddings stored directly in Parquet with DuckDB VSS (HNSW indexing) and Google Gemini for grounded Q&A.

768-dim Vectors · DuckDB VSS · Gemini LLM

Enterprise SSO & RBAC

Dedicated FastAPI auth backend with httpOnly JWT cookies. Includes single sign-on via Google, GitHub, Microsoft Azure AD, and generic OIDC (Okta, Keycloak, Auth0) plus role management.

OIDC · OAuth2 · Auto-Admin · RBAC

Automated AI Analytics

Daily automated data summarization and scikit-learn Isolation Forest anomaly detection. Outliers are explained in plain English by Gemini with root-cause insights.

Isolation Forest · Daily Summaries · Auto-Alerts

Kubernetes & Cloud Native

Complete Helm v0.2.0 chart with Ingress routing, PersistentVolumeClaims (NFS, Longhorn, Ceph, local-path), decoupled production secrets, and automated migrations.

Helm 3 · Ingress Nginx · PVC · 100% On-Prem

Compare & Save

Databricks vs Lake of Tears

Why pay six figures in SaaS DBU fees when you can host your own modern datalakehouse on your infrastructure?

Feature Databricks Lake of Tears 💧
License & Cost Proprietary SaaS (DBU fees + cloud compute markups) 100% Free & Open Source (MIT)
Deployment Target Vendor cloud only (AWS, Azure, GCP) Any Kubernetes cluster, bare metal, or Docker Compose
SQL Query Engine Heavyweight Spark Clusters (minutes cold start, high RAM) In-process DuckDB (instant cold start, low RAM, reads S3)
Vector Search Engine Separate Databricks Vector Search managed tier Native Parquet vectors + DuckDB VSS (HNSW indexing)
AI / LLM Integration Databricks Mosaic AI (metered per token / compute) Google Gemini (embeddings, RAG, summaries, anomaly triage)
Workflow Orchestration Databricks Workflows (proprietary YAML/UI) Industry standard Apache Airflow (embedded in UI)
Interactive Notebooks Proprietary Databricks workspace notebooks Standard JupyterLab (embedded in UI shell)
BI & Dashboards Databricks SQL Dashboards (limited visualization types) Apache Superset (enterprise-grade charts, SQL Lab)
Data Privacy & Control Telemetry and data control planes in vendor cloud 100% Self-Hosted — your data never leaves your network

Clean Architecture

How Lake of Tears Works

One ingress controller, one unified web shell, zero complex cluster dependencies.

Ingress Gateway

nginx (port 80) or Kubernetes Ingress

Single entry point for all UI and APIs

Lake UI Shell & Auth Backend

FastAPI
  • ✓ Home Dashboard & Storage Browser
  • ✓ DuckDB In-Browser SQL Editor
  • ✓ Embedded JupyterLab Notebooks
  • ✓ Embedded Apache Superset BI
  • ✓ Embedded Apache Airflow DAGs
  • ✓ AI Query (RAG) & Anomaly Detection

Storage & AI Engine

MinIO S3 + DuckDB + Gemini

• Parquet Hive partitions
• DuckDB VSS vector index
• Gemini 768-dim embeddings

Batteries Included

Built-in Ingestion Pipelines

Pre-built Airflow DAGs pull operational data into raw Parquet files automatically.

Stripe Payments & MRR

Hourly

Extracts charges, refunds, subscriptions, and churn events directly into partitioned Parquet.

s3://datalake/raw/stripe_charges/

Shopify Orders & Items

Every 6h

Syncs e-commerce order history, line items, revenue discounts, and inventory fulfillment metrics.

s3://datalake/raw/shopify_orders/

HubSpot CRM & Pipeline

Daily

Captures sales deals, pipeline stages, contact lifecycle stages, and win-rate analysis.

s3://datalake/raw/hubspot_deals/

PostgreSQL Databases

Hourly

Runs configurable incremental snapshots of production tables without placing read locks.

s3://datalake/raw/postgres/

Open-Meteo Weather

Daily (Free)

Hourly temperature and precipitation forecasts used as join dimensions for demand correlation.

s3://datalake/raw/weather/

HomeLab Telemetry

Real-Time

TrueNAS pool/disk health, Jellyfin streaming watch metrics, and personal voice exports.

s3://datalake/raw/truenas/

Get Started in Minutes

Deploy to Kubernetes or Docker

Choose between production Kubernetes Helm deployment or single-node Docker Compose.

1. Clone the repository

git clone https://github.com/Lake-of-Tears/lake-of-tears.git
cd lake-of-tears

2. Prepare your secret and production values

Create your values-secret.yaml outside Git with passwords and API keys:

minio:
  rootUser: "minio"
  rootPassword: "YOUR_MINIO_ROOT_PASSWORD"
backend:
  auth:
    secretKey: "YOUR_RANDOM_JWT_SECRET"
gemini:
  apiKey: "YOUR_GOOGLE_AI_STUDIO_API_KEY"

3. Install with Helm

helm install lake-of-tears ./deploy/helm/lake-of-tears \
  --namespace lake-of-tears \
  --create-namespace \
  --values deploy/helm/values-prod.yaml \
  --values /path/to/values-secret.yaml \
  --wait --timeout 10m