Local inference, production API
PrivateGPT is the Open-source API layer that turns local models into production AI applications.
57K+
GitHub Stars
7.5k+
Derived Projects
5k+
Discord members
LOCAL AI APPLICATION LAYER
PrivateGPT is the API layer for building Private AI applications
Running a model locally is only the first step. To build useful AI applications you need a set of higher-level building blocks.
PrivateGPT provides them as an open-source API following the Claude API model, so you can build private AI products without rebuilding the same backend primitives from scratch, and without depending on cloud APIs.
WHAT PRIVATEGPT GIVES YOU
Production-ready building blocks for local AI apps
Production-ready from day one.
PrivateGPT ships a built-in workbench UI for testing and demos, but the API is the actual product.
Messages API
Standard messages API with streaming, async processing, and token counting.
File & artifact ingestion
Upload, process, and use documents and artifacts as context for private AI workflows.
Retrieval with citations
Build RAG experiences with source-grounded answers and citation support.
Workbench UI
Use the built-in /ui workbench for testing, demos, internal pilots, and inspecting API requests.
Structured data access
Work with databases, CSVs, and tabular data through the same private API layer.
Embeddings & orchestration
Coordinate embeddings, retrieval, tools, and model calls behind one local application API.
Custom tools & MCP
Built-in secure AI workflows, integrations, and agents either running on top of local MCPs or connected to online providers you trust (e.g. n8n)
WORKS WITH YOUR LOCAL STACK
Integrate PrivateGPT with the tools developers already know
Any tool that works with a local OpenAI-compatible provider will also work with PrivateGPT.
Claude-compatible workflows
Use PrivateGPT as the private backend for Claude-style local workflows and API patterns.
Developer tools
Connect coding assistants such as Claude Code, OpenCode, VS Code, Cline, and other local AI tools.
Automation platforms
Power private AI workflows through tools like n8n and internal automation systems.
Local model servers
Run your models with Ollama, llama.cpp, vLLM, LocalAI, or your preferred OpenAI-compatible inference server.
TWO PROJECTS, ONE TEAM
PrivateGPT vs Zylon
PrivateGPT is built by the team at Zylon.
PrivateGPT is the open-source application API layer, while Zylon is the end-to-end AI Infrastructure orchestrating the hardware and software layers into a complete production platform for regulated organizations.
The Open Source API Layer
Use PrivateGPT if you want the open-source local AI application layer and developer API.
The Full Enterprise AI Infrastructure
Use Zylon if you need the full enterprise AI infrastructure around it deployment, governance, operations, user management, integrations, auditability, and support.
Enterprise Ready
Curious why Zylon is the go-to Private AI Infrastructure for the Enterprise?
USED AT
Followed by tech giants
Proven outcomes shared by industry leaders and innovators.
TESTIMONIALS
Backed Up By the Community
Proven outcomes shared by industry leaders and innovators.
Qdrant
Vector Search Engine
Qdrant X PrivateGPT!
With PrivateGPT, you can build context-aware AI apps based on your documents using Large Language Models (LLMs), even without an Internet connection. 100% private. Your data never leaves your environment.
Sudip Chakrabarti
Partner at Decibel.vc
Enterprise users are concerned about sending sensitive data to a 3rd-party API service. PrivateGPT shows how it is possible to ensure your data never leaves your environment.
Jerry Liu
CEO at Llamaindex
“I highly recommend PrivateGPT if you're looking for a local packaged RAG setup that works well out of the box, with both an API interface and a UI.”
Harrison Chase
CEO at LangChain
PrivateGPT is sick! I've always had people asking me if it was feasible to use LangChainAI with open source models, and my answer was "at the moment, not really..." but PrivateGPT did it! This is a huge step forward.
Network Chuck
Tech influencer
Run your own AI (but private) https://youtu.be/WxYC9-hBM_g
LlamaIndex
Data Framework for LLM apps
Check out how PrivateGPT lets you easily spin up a RAG pipeline as a prod-ready API in a fully local manner.
Works out of the box, directly tackles enterprise needs around privacy.
Query it similar to the OpenAI API, or use as a Gradio app!
Matthew Berman
AI & Open Source expert
PrivateGPT was the first project to enable "chat with your docs." They are back with TONS of updates and are now completely local (open-source). Chat with csv, pdf, txt, html, docx, pptx, md, and so much more! Here's a full tutorial and review: https://t.co/NGFwo86qZR