Who this is for
- Data that cannot ride a public API — regulated, contractual, or just yours
- Steady token volume where a box beats metered spend
- A team that will actually operate a workstation or small GPU cluster
Who this is not for
- Frontier-only workloads that have to be the latest closed model
- Nobody on staff who can own the machine after handover
- Air-gapped FedRAMP / ATO theater we will not pretend to sell
What we deliver
- Hardware assessment and GPU/VRAM sizing for your target models and concurrency
- Model selection and quantization strategy (GGUF, GPTQ, AWQ) balancing quality, speed, and memory
- Production serving stack: llama.cpp, vLLM, or Ollama — picked for the job, not the brochure
- RAG pipelines on your own vector store — documents stay on your network
- Security hardening, monitoring, and documentation your team can operate
How an engagement runs
- Assess. Workload, concurrency, data class, and who runs the box. We size VRAM before anyone shops for GPUs.
- Propose. Fixed-scope hardware + model + runtime. Honest about workstation vs small-cluster; we do not upsell a data center you do not need.
- Deploy. Stand up serving, RAG if it earns it, hardening, and a runbook. You keep the keys.
Stack we actually use
- llama.cpp
- vLLM
- Ollama
- GGUF / GPTQ / AWQ
- local embeddings + RAG
Tested on our fleet
Dated lab notes — not a client logo wall.
- How much GPU do you actually need to self-host an LLM?
- Self-hosted vs API: the real cost breakdown
- Self-hosting an LLM without it phoning home
Related services
Common questions
Do I really need my own GPUs to run AI privately?
If your data is sensitive, regulated, or contractually required to stay in-house, yes. Modern quantized models run well on prosumer and workstation GPUs, so the barrier is far lower than most teams expect. We size hardware to your actual workload so you don't overbuy.
How much does an on-premises LLM deployment cost?
A capable single-workstation deployment starts around the cost of one high-VRAM GPU. Multi-user production systems scale from there. We start every engagement with a requirements assessment and a fixed-scope proposal.
Which models can be self-hosted?
Open-weight models including Qwen, Llama, Mistral, DeepSeek, Gemma, and many others — covering chat, code, vision, embedding, and speech workloads at a range of sizes.
llama.cpp vs vLLM vs Ollama — which do you deploy?
llama.cpp for single-box and mixed-GPU work we actually run. vLLM when you need high-throughput OpenAI-compatible serving. Ollama when the team needs a fast internal on-ramp. We pick from those three; we do not invent a fourth runtime to sound complete.
Can this run on a workstation, or do we need a data center?
A serious first deployment is often one workstation. We move to a small multi-GPU box only when concurrency or context length demands it. On-prem does not have to mean a rack.
What do we own at the end?
The hardware, the weights, the configs, the docs, and the eval notes. No platform royalty. A maintenance plan is optional.
Will you help us buy GPUs?
We size and specify. We do not resell hardware.