Why AI Costs Up to 10x More Than Planned

Most finance leaders think of AI as a software investment. You evaluate it, approve the business case, and the budget line gets set. What follows is largely an operational matter.

That mental model is wrong. And it's costing organisations more than they expect.

AI has a fundamentally different cost structure than any enterprise software you've managed before. Traditional software investment is front-loaded: licence fees, implementation, integration, training. Once deployed, the marginal cost of an additional user or transaction is close to zero.

AI inverts this pattern entirely.

The Cost That Starts at Go-Live

With AI, the real cost begins after deployment. Every prompt generates tokens. Every agent workflow chains multiple inference calls. Every document retrieval, guardrail check, and multi-step reasoning cycle adds to a consumption bill that scales with usage, not with headcount.

NVIDIA's research puts this clearly: pre-training a model is essentially a one-time cost, but inference — every prompt generating tokens — creates ongoing expense that scales with usage. For finance organisations deploying AI rather than building models from scratch, inference dominates the cost structure.

The numbers compound quickly. Basic queries consume hundreds of tokens. Complex document analysis can exceed 250,000. Multi-agent workflows surpass one million per execution. And the reliability architecture required in finance — retrieval-augmented generation to ground outputs in source documents, guardrails to validate accuracy, multi-step verification — each adds further consumption. The very design choices that make AI trustworthy for financial decisions increase its cost profile.

Gartner shared at the 2024 CFO & Finance Executive Conference that AI cost estimates are often off by 500 to 1,000 percent. Initial business cases capture development costs and systematically underestimate everything that follows.

Three Layers, Three Cost Blind Spots

The Three-Layer AI Architecture for Finance — Foundation, Intelligence, and Flexibility working together as a complementary stack — describes not only how AI creates value but also where costs accumulate, often invisibly.

Foundation Layer

This covers AI embedded in core systems: ERP integration, automated workflows, systematic data infrastructure. It runs whether anyone actively uses the AI or not. Data preparation, vector embeddings, and knowledge base maintenance create a perpetual consumption baseline. As document repositories grow without active management, the same queries retrieve more and more context over time, silently inflating costs while potentially degrading output quality. Practitioners call this context rot. In mature implementations this layer can represent 20 to 40 percent of total AI costs — a figure that rarely appears in business cases.

Intelligence Layer

This covers purpose-built agents that draw from multiple systems for complex analysis. This is where design decisions create the largest and most durable cost multipliers. Two agent workflows producing identical business outcomes can have cost profiles differing by a factor of 100, depending on how they are architected. A "chatty" design requiring five or six model interactions to complete what a streamlined design handles in one creates a 500% consumption premium that runs every time the workflow executes. Implementation consultants measured on speed-to-deploy rather than efficiency-to-operate produce fast go-lives with cost structures nobody has reviewed.

Flexibility Layer

This covers prompt engineering, ad hoc analysis, and conversational interfaces for executive dashboards and one-off queries. This is the most visible layer and paradoxically the one most organisations over-optimise, sometimes driving users toward workarounds that inflate costs in the Intelligence and Foundation layers instead.

Each layer has a different owner. Foundation costs sit with data engineering. Intelligence with IT or business process teams. Flexibility with end users, HR, or training functions. Without someone accountable for total consumption across all three, each group optimises locally while aggregate spend grows. The architecture that was designed for strategic coherence fragments into three separate cost centres with no unified view.

Hidden Structural Risks

Beyond architecture, AI consumption models carry structural risks that traditional software budgeting doesn't prepare finance for.

Decentralisation of spend authority. In per-seat licensing, IT controlled procurement. In consumption models, every user with AI access generates costs, and every automated workflow consumes resources. Hundreds of users and dozens of processes collectively create spend patterns that are genuinely difficult to predict from individual actions.

Lag between consumption and discovery. Many platforms allow consumption beyond allocation with settlement at period end. Organisations can approach their annual budget without real-time warning. By the time the CFO sees the variance in a quarterly review, the bill has already been generated.

Jevons' Paradox at scale. Deloitte's 2025 tokenomics research documents the counterintuitive dynamic: token prices are falling fast, from roughly $0.04 per million tokens in 2025 toward an estimated $0.01 by 2030. Yet enterprise AI spending continues to rise. As efficiency improves, consumption expands faster than unit costs fall. The assumption that cheaper tokens mean lower bills is wrong. It means more usage.

The maintenance multiplier. QASource research quantifies what organisations discover too late: for smaller AI applications, annual maintenance runs 30 to 50 percent of original development cost. Enterprise-scale systems face 15 to 30 percent annually for monitoring, retraining cycles, and compliance updates. AI is never set and forget. The cost that wasn't in the business case is the one that recurs every year.

The Value Question HBR Raises

The Harvard Business Review article "Recalculating the Costs and Benefits of Gen AI" adds a dimension that pure cost analysis misses. Mark Mortensen argues that when evaluating AI adoption, output and efficiency are the obvious benefit categories. But the hidden costs aren't purely financial. They include loss of organisational learning, skill atrophy, reduced engagement with ideas, and erosion of the institutional knowledge that accumulates through doing work.

For finance functions specifically, this matters. The analyst who never learns to build a model because AI builds it. The controller who stops engaging deeply with variance analysis because summaries arrive automatically. The productivity metrics look strong. The capability curve quietly flattens.

Mortensen recommends what he calls an AI value audit: mapping not just what AI produces but what each task creates in terms of learning, relationship maintenance, and unique judgment before automating it. For finance leaders, this reframes the cost-benefit question. The question is not only whether AI saves money. It's whether AI adoption preserves the judgment capacity that finance functions are actually paid to exercise.

What Governance Needs to Look Like

FinOps for AI is not optional at scale. The FinOps Foundation has developed frameworks specifically for AI workloads: cost attribution by use case, anomaly detection, showback mechanisms, and consumption forecasting. Most finance AI initiatives don't need this at pilot stage. They need it before they scale. Building the instrumentation after a budget surprise is always more expensive than building it first.

Design decisions are financial decisions. Cost-aware design must be a first-class criterion in the Intelligence Layer, not an afterthought. Implementation consultants measured on speed-to-deploy, not efficiency-to-operate, produce fast go-lives that cost more to run. Finance leaders who accept "it works" as the success criterion in AI deployment are accepting a cost structure they haven't reviewed.

Ownership must be explicit before consumption begins. Who owns the AI budget when Flexibility Layer consumption is generated by HR-owned tools, Intelligence Layer workflows are built by IT, and Foundation Layer infrastructure is maintained by data engineering? Most organisations default to treating it as an IT line item. That creates misaligned incentives: the teams generating consumption are not accountable for the spend.

Forecasting requires different thinking. Traditional software budgets are essentially fixed. Consumption budgets require probabilistic forecasting: expected ranges, confidence intervals, contingency reserves. The muscle for this kind of variability doesn't exist in most finance teams yet. Building it is a prerequisite for governing AI costs at scale.

AI cost management is not a technology discipline. It's a financial governance discipline applied to a new kind of variable cost. The organisations that treat it as such from the start will convert AI investment into P&L impact. The ones that don't will spend years wondering why productivity gains never show up in the numbers.