We hear this question in almost every first conversation, and it's a fair one. Our answer is usually: you probably don't. Building your own makes sense above all when you need more than a plain chat.
We hear this question in almost every first conversation, phrased one way or another. "We have ChatGPT." "We already have Copilot." "Why would we build anything of our own?" It's exactly the right question.
The short answer: in most cases, you shouldn't build anything. Building your own tailored software solution on top of the well-known AI models makes sense above all when you need more than a plain chat.
First things first: ChatGPT is a product, not the model
One distinction up front, because it becomes decisive later. ChatGPT is not "the model." ChatGPT is a product made by OpenAI, and the language model behind it is only one part. Memory across conversations, uploading and searching files, reading images and PDFs, searching the web, and so on: all of that is built around the model. It's the same with Claude from Anthropic. The model itself can do just one thing: handle language, very well.
In the paid Business and Enterprise tiers, the vendors assure you that data is encrypted and not used to train new models. Whether to trust that is something every company has to decide for itself.
When you shouldn't build anything yourself
Basically, it's everything where you've already noticed, working with ChatGPT and the like, how powerful these tools are: everything that is essentially language and doesn't require sensitive data. Texts in the company's tone, presentations, summaries, research, translations, first drafts of almost anything. A company recently asked us whether it needed something of its own for marketing, for copy, social media posts, and documents in its own design. Our answer was no. With the right templates and instructions, what the vendors already offer is enough.
That holds as long as it's internal work that a single person triggers and keeps an eye on: someone asks, checks the answer, and uses it. Where that stops being enough is covered further down.
One thing always holds, though: training your team to work with AI properly delivers more than any innovation project. These systems can talk extremely convincing nonsense. Why that is, we've described elsewhere. Those who have learned to handle it get more out of a subscription than most people get out of a custom system.
When developing your own LLM-based system can make sense
The first case: your company knowledge is unique and valuable. Picture a machine builder that has been making special-purpose machines for thirty years. Its service tickets and engineering notes hold the reasons its machines run and the competition's don't. Loading that knowledge unfiltered into an American model is something to think hard about. It's a large part of what makes the company what it is. Not to mention data protection.
The solution in many cases: open-source models running on your own infrastructure or at a hosting provider in Austria. Open-source models typically trail the best commercial models by only about four months, and for most tasks that's enough. A middle ground is more protected cloud environments with an EU data region, such as Bedrock from Amazon Web Services (AWS), where the big commercial models run too, without data leaving the EU. How we set that up is on our page on GDPR-compliant AI.
The second case: you want to build a product yourself. As soon as AI is meant to become part of your own offering, an app, a platform, or a service for your customers, you have to build it yourself. Here we regularly hear: "But ChatGPT can do that." Yes, ChatGPT, the product. The model you get through OpenAI's API can't. Everything ChatGPT can do and the model can't has to be built separately.
The third case: you want to connect many existing systems, and hallucinations have to be kept to a minimum. A business owner recently wanted to know where sales and operations currently stood, so he could react faster. They already had Copilot, and it didn't work. Not because Copilot is bad, but because out of the box it only sees what's in Microsoft 365.
For ERP, CRM, and industry software there are connectors that have to be set up and authorized, and even then what you get is a search across documents, not a system you'd want to rely on without question. Anyone who wants to know where sales stand needs numbers that are correct, not a summary. As soon as several systems have to work together and the answer has to be exact, it's worth considering whether to build your own system around the existing AI models.
The fourth case: you want to map a clearly defined workflow. A process with several steps across several systems that runs company-wide by the same rules, without anyone triggering each step. You can set something like that up in ChatGPT or Copilot, but only with effort and little control. As a system of your own, it runs the way you defined it, and you can see what happens at every step.
One thing applies in every case: first check whether what you need already exists. Established products exist for many problems. If one of them solves yours, you don't have to build anything.
What you get when you build your own
This is where it gets interesting, because most people picture a custom system as a chat with the company logo on it. That's the most obvious case, and the one most similar to ChatGPT. It isn't the most exciting one.
What you actually get is a system, built on the most advanced models, that is at home in your domain because it works with your data: connected to your databases, your documents, your processes. And a system like that can work in the background.
At one end are systems that process expertise and deliver value that way. For example: a utility gives its customers an app that uses consumption data, tariff, and weather to explain why this month's bill was higher, and what the customer can change. The numbers come from the billing system, the explanation from the model, and customer service stops getting those calls. A plant operator has fifteen years of maintenance reports and sensor data. The system recognizes which fault patterns precede a failure and tells the maintenance crew which plant to look at this week.
At the other end is the automation of everyday work, which above all saves time. For example: a wholesaler has two thousand supplier contracts, each worded differently. The system reads them all, flags every liability and termination clause that deviates from the company's own standard, and hands purchasing a list nobody could have produced before. Incoming invoices are matched against the purchase order and only shown to a person when there's a discrepancy. Customer emails are assigned to the right case in the CRM and given a suggested reply. Product descriptions are generated in five languages by the same rules, thousands of times, at consistent quality.
But that only works if the system was built properly. More on that in our Insight on hallucinations and on our pages on document AI and AI development.
Building doesn't mean building a model
Nobody should build a language model from scratch. You buy one, from OpenAI or Anthropic, or you use one of the freely available open-source LLMs. Gartner expects most companies that train their own models to give up on it: too expensive, too complex, too much technical debt. What gets built is the system around it: data access, permissions, tests, workflows. These topics are fairly new. So it pays to work with teams that have already delivered systems like this.
What can very well make sense is to further train an open-source model on your own data. Experts call that fine-tuning. It's suitable when a model is supposed to adopt a company's style, technical vocabulary, or conventions, such as your own code base or the terms and abbreviations that only exist in your company. Facts don't get into the model that way. For facts, connecting the model to your own data remains the method of choice. Fine-tuning is a complement, not a replacement for the system around it.
Sources
- OpenAI: Business data privacy, security, and compliance
- Gartner: Hype Cycle for Generative AI 2026 (May 2026)
- Epoch AI: Open models lag state-of-the-art closed models by 4 months (May 2026)