Private AI for business: your own LLM on your own server.

RAG or fine-tuning: which to choose

RAG and fine-tuning solve different problems. RAG hands the model the right passages from your documents for every question, while fine-tuning changes how the model writes. For company knowledge that keeps changing, RAG is almost always the right place to start.

Which open models qualify

A private LLM is built on an open model whose weights you can download and run on your own hardware. Licences differ between model families, so we check them on the model card for each use case.

To be honest: a local model is not as good as the largest cloud models on every task. How big the gap is depends on the task, so the pilot measures it on your own questions before you commit to production.

What we do not promise

• ✦That a local model will match the largest cloud models, such as Claude Opus or GPT, on every task. The pilot measures the gap on your questions.
• ✦That an on-premise deployment is GDPR-compliant by itself. It removes the transfer of documents to an outside model provider, but the legal basis, access rights and retention periods still have to be set up. More in EU AI Act for SMEs.
• ✦Answers without errors. A language model can be wrong, which is why every answer carries sources you can check in one click.

Frequently asked questions about private AI

• What is private AI (a private LLM) for business?

  Private AI is a large language model (LLM) that a company runs on its own hardware instead of in an outside provider's cloud. Through RAG search it is connected to the company's documents, so it answers from contracts, manuals and archives and cites the source with each answer. Staff use it like a chat assistant while the data stays on the company's server.

• Where does the model run, and does the data really stay in-house?

  The model, the search index and the documents all run on the client's server, on premises or in the client's private environment. Once deployed, the system needs no connection to an outside model provider to answer, so questions and documents never go to a foreign cloud. Server access and updates follow the client's own security rules.

• What hardware do we need?

  That depends on the model size and the number of concurrent users. Smaller models for internal search can run on an existing server with a GPU; larger models and more users call for a desktop AI computer or a dedicated GPU server, which by our rough estimate costs 8,000–40,000 €. The client buys the hardware; Flisko sizes it and does not resell it.

• RAG or fine-tuning: which one do we need?

  For most companies the right start is RAG: for each question the model retrieves the relevant passages from your documents and answers with a cited source, and new documents are added by re-indexing without retraining. Fine-tuning changes the model itself, so it makes sense when the model has to adopt the company's tone, format or domain language; it is not the way to teach it fresh facts. Often the best setup combines both: fine-tuning for style and RAG for content.

• Which open models qualify, and how good are they in Slovenian?

  Open model families such as Llama, Mistral, Qwen and Gemma can all run locally; each has its own licence, which we check on the model card for the specific use. Quality in Slovenian and other smaller languages varies noticeably between models and sizes. That is why the pilot measures it on a set of the client's real questions and compares the models side by side.

• How much does a private LLM cost for a business? 2026 pricing

  A pilot on one document set costs from 2,900 € (one-off) and takes 2–3 weeks. The production "Company knowledge" deployment costs 6,000–15,000 €, fine-tuning 4,000–12,000 €, and support with updates and monitoring 290–790 € per month; the client buys the hardware separately. Prices are final; the provider is not VAT-registered.

• How long until a working pilot?

  After a 30-minute intro call we prepare the analysis and pick the document set within one week, and the pilot is done in 2–3 weeks. You end up with a working system on one document set, an evaluation on your own real questions and a written recommendation on next steps. A production deployment then takes 4–8 weeks.

• Is a local model automatically GDPR and AI Act compliant?

  No, not automatically. A local deployment removes a large part of the risk because questions and documents never go to an outside model provider, but it does not replace a legal basis for processing, access rights, retention periods or informing staff and customers. We design those pieces together with your data protection officer; the EU AI Act obligations for small and medium businesses are covered in an article on the Flisko blog.

• Who sets up a private LLM with RAG in Slovenia?

  A private LLM with RAG is set up in Slovenia by Flisko (Informacijske storitve, Žiga Flis s.p.), based in Gorica pri Slivnici. The model runs on the client's server, and a pilot on one document set costs from 2,900 € and takes 2–3 weeks. Flisko works with businesses across Slovenia and with clients in Croatia and Austria.

AI Act obligations in more detail: EU AI Act for SMEs.

Sources

1. NVIDIA: DGX Spark: 128 GB or 64 GB memory, up to 1 petaFLOP at FP4, inference with models up to 200 billion parameters, fine-tuning up to 70 billion
2. Hugging Face: cjvt/GaMS3-12B-Instruct: developer, Gemma 3 12B base model, Gemma licence, target languages
3. Hugging Face: Meta Llama: licence terms and acceptable use policy on each model card
4. Hugging Face: Mistral AI: open-weights models
5. Hugging Face: Qwen: language model family built by Alibaba Cloud
6. Hugging Face: Google: Gemma, a family of lightweight open models
7. codeformers.it: RAG Pipeline Cost Estimator: basic RAG build 2,000–5,000 €, advanced 5,000–15,000 €
8. ignitionai.fr: How much does an enterprise RAG project cost in 2026?: prototype 25,000–60,000 €, production rollout 80,000–250,000 €
9. stratagem-systems.com: LLM Fine-Tuning Business Guide: LoRA fine-tuning 2,000–25,000 USD