Firms Plan to Use More Open‑Weight AI Models to Cap AI Bills

Technician in a server room adjusting a GPU rack, teal lighting and a monitor showing cost metrics.

What the data shows

The latest reports from two independent sources highlight a striking cost trend. Yahoo Finance headline dated 12 November 2026 reports that Duolingo’s use of open‑weight AI models cuts some of its costs to less than a cent. A Capgemini study released on 12 November 2026 found that nearly 6 in 10 senior executives at large enterprises plan to increase their use of cheaper open‑weight models in the coming year to cap AI bills.

Both findings underline a common theme: organisations plan to use more open‑weight models, which can be downloaded and run on machines you own, an option not available with closed‑weight models. The Capgemini study reports that open‑weight tokens are, on average, 87 % cheaper than closed‑weight alternatives. While the Duolingo figure is a single company’s internal metric, it shows what one company reports, and readers should measure their own cost per request.

Why it matters to your organisation

For operations, IT and compliance leaders in small and mid‑size firms, these numbers translate into a tangible budgetary advantage. Cloud‑based AI services often charge per token or per inference, making costs difficult to predict and scale. When a model can be run entirely on premises, the variable component of spend drops to the cost of hardware, electricity and maintenance.

The Capgemini study indicates that cost containment is a key factor for many executives, with nearly 6 in 10 planning to increase use of open‑weight models to cap AI bills. Organisations that can demonstrate lower per‑inference costs are more likely to secure executive approval for broader AI initiatives, especially in regulated sectors where auditability and data residency are critical.

Plays from YouTube. Nothing is loaded from YouTube until you press play.

Cost drivers of open‑weight models

Open‑weight models are characterised by several key cost drivers that differ from cloud‑based services:

  1. Hardware utilisation – A single inference typically requires a GPU or specialised accelerator. The cost is spread over thousands of inferences, reducing the per‑inference expense.
  2. Energy consumption – Running inference locally incurs electricity costs, but modern GPUs can deliver high throughput per watt, keeping energy costs low.
  3. Model licensing – Open‑weight models are released under a range of licences; some are permissive, others restrict commercial use or large deployments, so each licence should be checked.
  4. Maintenance overhead – While you must manage updates and security patches, the absence of a cloud vendor’s billing cycle removes a layer of cost complexity.

Practical steps to evaluate

Before committing to an open‑weight solution, consider the following checklist:

  • Hardware inventory – Verify that existing GPUs or accelerators meet the minimum compute requirements for the chosen model. If not, estimate the cost of an upgrade.
  • Inference throughput – Measure how many inferences per second a single GPU can deliver for your workload. This metric will inform whether a single machine suffices or a cluster is needed.
  • Energy cost modelling – Calculate the kilowatt‑hour cost of running the GPU at full load for the expected inference volume.
  • Security posture – Ensure that the model can run behind existing firewalls and that audit logs can capture inference events for compliance reporting.
  • Vendor support – Evaluate the community and documentation around the model to gauge the effort required for troubleshooting.

Aligning with data governance

Running AI locally offers inherent advantages for data governance. All data remains within your network, a location you control. Additionally, audit logs can be written to protected, append‑only storage, giving auditors a reliable record.

When deploying open‑weight models, integrate role‑based access controls to limit who can trigger inference. Combine this with encryption at rest for any model checkpoints stored on disk. These controls help maintain the integrity of the AI pipeline without incurring additional cloud‑based monitoring costs.

Integrating private RAG on your own hardware

Retrieval‑augmented generation (RAG) is a common use case for organisations that need to answer queries over proprietary documents. By hosting both the embedding model and the vector store locally, you eliminate the need to transmit sensitive documents to external services.

Open‑weight embedding models can be paired with vector databases such as Milvus or Weaviate, both of which can run on the same hardware that hosts your inference GPU. This architecture keeps the entire pipeline – from embedding to generation – on premises, allowing data to remain within the organisation’s network.

Performance tuning is straightforward: you can adjust the vector dimensionality, index type and query batch size to optimise latency without incurring additional cloud costs.

Next steps for your AI strategy

  1. Pilot a cost‑analysis – Run a small‑scale inference test on your existing GPU to measure your own cost per request on your own workload and compare it with what you currently pay.
  2. Define governance policies – Document how model weights, data, and logs will be managed, and map these policies to your organisation’s compliance framework.
  3. Scale incrementally – Start with a single use case, such as automated customer support or internal knowledge search, before expanding to enterprise‑wide deployments.
  4. Monitor and optimise – Continuously track inference latency, GPU utilisation and energy consumption to identify optimisation opportunities.

By following this structured approach, your organisation can take advantage of the cost savings highlighted by recent industry data while maintaining control over data and compliance.

Sources

Start a project

Tell us what you want AI to do. We will tell you what it takes to run it on your own machines.