The short answer
There are two broad ways to use a large language model in your organization:
- Use an AI service. You send prompts to a provider, such as OpenAI, Anthropic, Google or Microsoft, directly or through your cloud account, and pay per use. Business terms exclude your data from model training, and several providers offer zero data retention for eligible customers, usually by approval. Through your cloud account, the models run in the cloud provider's environment: Microsoft, for example, states that prompts sent to the models it sells through Azure are not available to OpenAI or other model providers (Microsoft Learn).
- Host an open-weight model yourself. You run a model such as Llama, Qwen, DeepSeek, GLM, Mistral or gpt-oss on GPU servers you control, in your own cloud account or on your own hardware. No outside AI provider sees your prompts.
Choose the service when its terms satisfy your policies, your usage is light or uneven, or you need the most capable model available. Choose self-hosting when information must not leave your environment, when usage is steady and high enough to keep a server busy, or when you need control over the model and its updates. Many organizations end up with both, routing each request by how sensitive it is.
Side by side
| AI service (business terms) | Self-hosted open-weight model | |
|---|---|---|
| Where prompts go | To the provider, processed under its business terms | Nowhere outside your environment |
| Training on your data | Excluded by business terms | Not possible: the model runs only for you |
| Retention | Provider's policy, often a limited period for abuse monitoring; zero data retention where offered and approved | Your choice, including none |
| Answer quality | The most capable models available | Strong for many business tasks, behind the leading commercial models on the hardest ones |
| How you pay | Per use, so cost follows volume | Per server hour, whether the server is busy or idle |
| Time to start | Hours | Days to weeks, including sizing and security |
| Effort to run | Low: the provider runs the model | Yours or your IT partner's: patching, monitoring, capacity, updates |
| Model changes | The provider updates and retires models on its schedule | You decide when to change, and test first |
| Evidence for auditors | The provider's terms and reports | Your own architecture, network rules and logs |
How to compare the cost
The two options are billed in different shapes, so compare them on the same measure: cost per month at your expected volume, in US dollars.
- AI service: requests per month × tokens per request × the provider's rate. It starts near zero and grows with use.
- Self-hosted: GPU server hours × the hourly rate, plus storage, networking, monitoring and the people time to run it. It is roughly flat, whether the server answers ten questions or ten thousand.
The point where self-hosting becomes cheaper depends on how busy you can keep the server. A server that runs around the clock for a team that uses it during business hours pays for many idle hours. Ways to tilt the numbers:
- Right-size the model. A smaller model that meets your accuracy bar needs less GPU memory and costs less to run.
- Schedule capacity. Start and stop servers with working hours, or run batch work overnight on the same hardware.
- Share the platform. One private endpoint can serve several applications, so the fixed cost is spread.
- Count the people time. Patching, monitoring and testing new models are real costs, whether your team does them or a partner does.
Uncontrolled use costs money either way. OWASP lists "unbounded consumption" among the top risks for AI applications, so set usage limits and alerts on both options (OWASP). Our guide to what an AI project costs covers the rest of the budget.
Rather talk it through? If you are not sure whether your volumes justify a private server, we can model both options on your expected use. Talk to a Promatics specialist
Quality: test it, do not assume it
Benchmarks change monthly and rarely match your work. The only result that matters is how a model answers your questions, from your documents, in your languages. A fair comparison:
- Build an evaluation set of real questions with the answers you expect.
- Run it on two or three open-weight candidates and one commercial model, if policy allows test data to go to a provider.
- Compare accuracy, response time and cost per answer.
- Decide with the numbers, and keep the evaluation set to re-test when new models are released.
For answering from documents, summarizing, drafting and extracting data, a well-chosen open-weight model often meets the bar. Independent trackers report that the best open-weight models have trailed the leading closed models by about four months on average in 2026 (Epoch AI), and the gap is wider on the hardest reasoning and coding tasks, which is one reason many organizations use both.
Using both: route by sensitivity
A hybrid design keeps the benefits of each:
- Confidential information goes to the private model. Client files, health records, financial data and internal plans never leave your environment.
- Everything else can use a service. Public information, marketing drafts and general research can use a commercial model with business terms.
- One gateway decides. Applications call a single internal endpoint that routes each request by data classification, user or application, and logs the decision.
Because self-hosted inference servers such as vLLM offer an OpenAI-compatible API (vLLM), the same application code can usually talk to either option, which keeps you free to change your mind. Test each model anyway: chat formats and tool calling differ between models.
Decide in ten questions
- What information will the AI see, and does any policy, contract or regulator forbid sending it to an outside provider?
- Would a provider's business terms, with zero data retention, satisfy your privacy lead and your clients?
- Must the data stay in the US or fall under a HIPAA business associate agreement, and which services and GPU regions qualify?
- How many requests per month do you expect now, and in a year?
- Is usage steady through the day or concentrated in bursts?
- Which accuracy level is good enough, measured on your own questions?
- How fast must answers be at peak use?
- Who will patch, monitor and update a self-hosted platform?
- How will you show auditors and clients where AI data goes?
- Could a hybrid design give you most of the benefit for less effort?
When to bring in help
If your data is not sensitive and usage is modest, start with a business AI service and good settings; you can move later. Bring in help when confidential or personal information is involved, when clients or regulators ask where AI data goes, when volumes are growing fast, or when you are considering buying GPU servers.
Privacy law applies either way. State privacy laws and sector rules such as HIPAA and the GLBA apply to personal information used in an AI system wherever it runs (see AI laws in the US). For protected health information, you need a business associate agreement with whichever provider stores or processes it: the AI service, or the cloud provider that hosts your own model, even if the data is encrypted (HHS). This is general information, not legal advice.
Our private AI and self-hosted models service tests both options on your own questions, designs the hosting, and deploys and runs the platform if self-hosting wins. If a service with strong terms is the better answer, we will say so and help you set it up through our AI strategy and readiness work.
Sources and further reading
Product capabilities and guidance change. These are the primary sources this article relies on, checked on the review date above.
- Online serving (OpenAI-compatible server), vLLM documentation
- Open models lag state-of-the-art closed models by 4 months, Epoch AI
- Data, privacy and security for Microsoft Foundry models, Microsoft Learn
- LLM10:2025 Unbounded Consumption, OWASP Gen AI Security Project
- Guidance on HIPAA and cloud computing, U.S. Department of Health and Human Services
This article is general information, not legal, accounting or security advice for your specific situation. Examples are hypothetical unless stated otherwise.