How to Implement AI Successfully: seven lessons from the past

Hi, I'm Harry. If a friend forwarded this to you, welcome, and you can subscribe here to get it every week. If you enjoy it, I'd be grateful if you passed it on.

Observations and practical guidance from someone deploying AI inside companies every day.

Turns out the instructions are in a chocolate company's 1999 annual report.

The AI trade had a difficult week. Thursday's sell-off took 5.5% off the Nasdaq, while investors were questioning what Anthropic might be worth when it lists, and whether November was still realistic timing. After last week's look at the S-1, I wanted to spend some time on the part of the story we can actually influence: how companies put the technology to work.

That sent me back to the earlier waves of enterprise software. Some of the problems were surprisingly familiar.

Elsewhere, XPO's operating results offer something useful to measure, Figma is working on the cost of serving AI, and the builders are asking how much of an analyst's job an agent can really take on.

But first, seven lessons from the past that I think are relevant to the deployments we're doing today.

In today's Acquisition Intelligence:

From The Trenches

How to implement AI successfully: seven lessons from the past

Turns out the instructions are in a chocolate company's 1999 annual report.

Last week I wrote about the questions Anthropic's S-1 left me with. Chief among them was the question of “economic diffusion”, aka how quickly businesses could put AI to work. Of all the issues I had, the pace at which this was happening, compared with the growth Anthropic would need to hit, was what left me with the most consternation.

It certainly feels like I wasn't the only one this week. The AI market sell-off saw the Nasdaq fall 5.5% on Thursday, wiping out more than $2.5 trillion in market value. Friday brought a rebound, but the index still recorded its fourth consecutive weekly decline.

The fundraising news added to the uncertainty. Friday's reporting put OpenAI's initial fundraising close at roughly $11 billion, valuing it at “only” $1.25 trillion. That followed reports the previous week that it had cut the amount it wanted to raise from $30 billion to $20 billion.

Around the same time, we were getting reports that investors were questioning Anthropic's valuation and whether its November IPO would go ahead. The discussion had moved towards $1.5 trillion from the $2 trillion talked about earlier, with some investors now treating even $1.5 trillion as the high end. A difficult week, and a reason to come back to the question that bothered me most in the filing: how quickly can businesses actually absorb all this technology?

In periods of uncertainty, I find it helpful to look back before trying to work out what comes next. And whilst I wasn't really watching enterprise software budgets or paying that much attention at the time, I've been doing some reading on the digital transformation of the late 1990s and early 2000s to see what happened and what we can learn from it. What caught my attention was how often the problems sounded like conversations we're having at DealSage now.

“Digital transformation” is a term that's been used to describe a bunch of technological evolutions, whether it's the ERP and enterprise software wave of the late '90s, the dot-com or “new economy” era of the early 2000s, or the later move to cloud, mobile, social and analytics. To me, nothing has felt quite as all-encompassing as the AI transformation, but we've certainly gone through similar technology changes before.

The attraction and rationale are familiar. Better information leads to better decisions and a more productive business. But along the way there are lessons it seems we have to learn each and every time. The gap between impressive capabilities and impressive results is much larger than people think, and closing it requires more work than people expect. These are the seven points that stood out to me.

In 1987, Robert Solow observed that you could see the computer age everywhere except in the productivity statistics. Companies were spending on computers without an obvious improvement in the economy's productivity growth. The acceleration arrived around 1995, roughly eight years after his remark and more than a decade after the early-1980s arrival of the PC. There was no single starting gun, but the benefits took years to spread through the economy. Federal Reserve research describes that long lag.

What happened inside companies helps explain why. In their 2000 research, Erik Brynjolfsson and Lorin Hitt examined how returns depended on investments around the computers: redesigning processes, changing who could make decisions, and developing employees' skills. Installing the technology left that work unfinished. Their 2003 study of 527 large US firms found productivity contributions up to five times greater over five-to-seven-year periods than over one year. Whilst benefits were already visible in the first year; the larger gains took longer.

The later productivity J-curve research, by Brynjolfsson, Daniel Rock and Chad Syverson, explains how this investment can depress measured productivity before the benefits emerge. People spend time learning and reorganising while the business still has its normal work to do. The costs, unfortunately, arrive before the return.

Today, I see companies handing Claude, ChatGPT or Copilot to employees and expecting things to improve materially. It just doesn't work that way. You amplify what people were already doing and meet the same bottlenecks faster. Someone at the top has to decide how the organisation should change and give people the time and authority to do it.

With AI, I think the technology can embed and scale much faster. For a well-prepared deployment already in production, I'd expect meaningful gains over 6 to 12 months in many cases. AI amplifies and accelerates, so if you've got your direction dialed in, so too will the gains. I don't think we're looking at seven or eight years for those gains by any stretch, but the upfront work is equally important.

Michael Hammer’s 1990 line “Don’t automate, obliterate” was aimed at companies that digitised paper forms when the more pertinent question is whether the form should exist in the first place. New technology allows us to entirely rebuild processes, otherwise you just end up making unnecessary work more efficient.

Elon Musk is famous for making a similar point, detailed in his engineering approach: remove unnecessary parts or process steps, then simplify what remains before accelerating and automating it. Enterprises should be taking a similar approach here. Before asking how AI can do something faster, ask how much of it you can remove.

A good example today is using AI to generate Excel reports faster. Before doing that, I'd ask whether the report needs to exist at all. Often we're extracting data and arranging it in a spreadsheet so another human can understand what's happening. If they can ask the question directly against reliable source data, with the calculation and evidence available, some of that preparation can disappear.

There will still be reports we need for a board or an audit. But rebuilding every existing report with an AI button preserves a lot of work that may no longer serve a purpose. This is where I think the bigger opportunity sits: reconstruct the process around what is now possible, including removing steps that existed because of the limitations of the old technology.

Announcing AI deployment plans, and declaring them a success, can seem more important than the rollout itself these days. There's an obvious temptation to talk up AI use in the hope of a quick share-price pop. I tend to be skeptical of the companies making the most noise, because the announcement can get well ahead of the organisation's ability to deliver.

The earlier enterprise software wave shows how publicly that gap can unravel. Hershey's 1999 implementation difficulties are a useful example. Its 2000 annual report describes problems shipping orders after the final phase of its new business system, compounded by a shortage of loading-dock space. Recovery involved fixing the system and expanding distribution capacity. One new facility had 160 dock doors.

The software had to work within an operation capable of handling what came next. For AI, I'd want to see an agent running alongside the existing process, with its results checked, before giving it authority to change records or make commitments. That gives you a chance to find the awkward cases before the business depends on it.

My bet is on the teams getting on with the work in the background, recognising that it takes time and extending what they've proved works.

CRM was the classic cautionary tale: Bain's researchers wrote in 2002 that more than half of CRM initiatives failed to produce the anticipated results. Dirty data and salespeople not entering information could undermine the whole exercise. The software often worked but its goal as the system of record did not.

I'd argue this is even more relevant in the age of AI. How is your AI supposed to understand what's happening in your business if the data is inconsistent and nobody agrees what the numbers mean? How can you make better decisions if you can't trust what you're looking at?

We've seen this multiple times in our deployments, it is critical to do the upfront work to ensure there is data consistency and integrity otherwise you cannot make well informed decisions about how to improve your business. The reality is that in many organisations different people, teams, departments are all using different definitions for the same KPIs. How can a company move towards standardized goals if there’s no agreement on what that direction is?

This has become so important to us that is take centre stage on the DealSage website. “You don’t have an AI problem. You have a data problem.”. Giving a model access to everything won't resolve conflicting definitions or fill in the context that lives in someone's head. That work is part of the deployment.

The earlier enterprise software lesson was that access didn't guarantee use. Rigby's recommendations included changing jobs and incentives alongside the software. Counting seats provisioned tells you little about whether the work has changed. A tool outside the daily workflow gives people another task; one that captures information through work they're already doing has a much better chance of sticking.

I think we underestimate the inertia here. We've had Excel, PowerPoint and Word for decades. Cloud changed how software was delivered and made collaboration easier, but much of the everyday work still happens through familiar documents and spreadsheets. Now we're asking people to learn another way of interacting with technology while continuing to do their jobs.

We've learnt this ourselves. The aim is to make deployments fit the formats people already use, whether that's email or an existing system, or to replace something they already hate with something easier. If useful work can happen in the background without requiring another interface, even better.

That also means deciding where human involvement adds value. Some tasks need approval every time; others can run within agreed limits and bring exceptions to a person. Making everybody check every step can recreate the bottleneck we're trying to remove. Ease of use, including how little extra work the deployment asks of people, will determine whether it lasts.

Borders handed its online store to Amazon in 2001. Amazon became the seller of record and handled fulfilment and customer service. That agreement doesn't explain Borders' eventual fate by itself, but it put much of an emerging capability in the hands of a competitor.

At DealSage, we learnt early that software is moving towards much more customisation around exactly what a business needs. Signing up for the same product everybody else uses will only get you so far. The useful capability comes from how it understands your business and fits the decisions you need to make.

That doesn't mean building every component yourself. It means retaining the operating knowledge developed along the way, even when you hire help. Once we've agreed how a productivity metric works, that definition and the reason behind it should remain available to the business and whoever maintains the system next.

Customisation also creates maintenance work. The more you bury business logic inside one model's prompts, the harder it becomes to change that model. I'd keep the company's data and rules separately maintained and testable, so improvements in models, or cheaper alternatives as they become more interchangeable, can be adopted without rebuilding everything. Tailor the application closely to the company while keeping the model underneath replaceable.

The dot-com era laid enormous amounts of fibre ahead of demand. A 2003 Joint Economic Committee study described excess capacity and falling telecom prices that left companies struggling with their debts. The infrastructure could remain useful even when the people financing it lost money.

That's the parallel I find relevant today. If AI compute is being overbuilt, the winners may include the companies that exploit falling inference costs to rebuild their businesses, rather than those spending most heavily today. Chips have different economics from fibre, so I wouldn't assume the outcome is inevitable. But the distinction between useful technology and a good investment remains.

Cheaper models would be welcome in our work. We'd still have to resolve the data and organisational problems described above. The companies doing that preparation can benefit from better, cheaper technology as it arrives, even if the returns to the model providers disappoint. That's why I can be optimistic about deploying AI and still have questions about Anthropic's valuation.

Of course, AI is different from what's come before. I think we're looking at a step change in what technology can do, so whilst these lessons are useful, they don't mean the transformation will follow the same path.

The groundswell behind AI deployment feels unlike anything we've seen before. That leads many people to believe organisations will transform much faster this time: there are more companies offering help, more internal attention and more people working on it. I can see the argument, but I'm not sure all that activity translates into effective implementation.

First, I think a lot of companies are deploying AI incorrectly. Giving everyone access and hoping useful changes emerge leaves much of the organisational work undone. The mixed messages about usage and companies reconsidering deployments don't particularly surprise me in that context, although they won't all have the same explanation.

Second, doing this properly may require even more alignment than earlier technology changes. Cloud was a major shift, but plenty of employees carried on using familiar applications without needing to think about where they were hosted. AI can change who does the work, how decisions get made and who takes responsibility when something goes wrong. That reaches much further into the organisation.

Third, implementation requires an unusual combination of skills. The forward-deployed engineer title has become popular, but changing someone's title doesn't give them the ability to understand a business deeply enough to change how it operates. You need people embedded with the customer, able to work through the operational problems and then build something that solves them.

The demand for those people is becoming expensive. The WSJ reported elite FDE compensation of $586,000 to $1.45 million, including equity and bonuses; its companion piece, drawing on the same study, described demand for help getting pilots into production. More firms offering deployment services won't immediately create more people who can do that work well.

At DealSage, we've been making this pitch for a while now: you have to do the work up front to develop a top-down strategy. Ask yourself:

That takes time and effort. There are no shortcuts through the work of understanding your business and aligning the organisation. To me, the greatest trick that's been pulled is making you think AI is a shortcut through all of that. For many organisations, it isn't.

From The Builders

Rahul on AI diligence

I'd want to see how an agent handles conflicting evidence and justifies its assumptions. A plausible memo can hide weak analysis, so I'd need to trace its conclusions back to the records before relying on it.

Ethan Mollick on the two camps in AI transformation

If the AI team can buy software but can't change an approval or a performance measure, I wouldn't expect much transformation. That's why I want the people running the operation involved from the start.

Aaron Levie on agents and hours worked

I agree with the direction. Before putting the savings into a business plan, though, I'd measure how much work makes it through review and into use, including the hours spent checking and correcting it.

News Digest

The AI trade meets the ballot box

Thursday's 5.5% Nasdaq fall followed a midterm result that brought opposition to data-centre construction into sharper focus. In Texas, Gina Hinojosa beat Greg Abbott after campaigning for a moratorium on new data centres. Investors had another reason to question the pace of the AI buildout, just as the companies funding it were asking for more money.

The details:

What This Means: The buildout depends on access to capital and permission to build. This week raised questions about both, with political opposition adding another constraint to the timelines investors have been underwriting.

My Take: My concern is that boards use a week like this as a reason to postpone the work inside their own businesses. Slower construction could affect the cost and availability of the models we use, which is another reason to preserve the ability to change suppliers. But the reporting bottleneck or inconsistent data that existed before Thursday still needs fixing, and I'd want a commercially sensible deployment judged on its own merits. XPO's operating results below give us something more useful to examine than the day's share-price move.

Read more: Reuters, November 5

Anthropic's IPO meets a more sceptical buyer

Anthropic's roadshow was expected to start the week of November 9, with pricing the following week. Reuters reported on November 5 that potential investors were questioning whether a November listing remained achievable and pushing for a lower valuation. Some viewed $1.5 trillion as the high end, against earlier talk of $2 trillion. These were investor doubts, rather than an announced delay; Anthropic declined to comment.

The details:

What This Means: The valuation will test how much investors are willing to pay for rapid growth alongside enormous future obligations. A lower price or a delay would also influence expectations for other AI businesses seeking capital.

My Take: This comes back to last week's concern about economic diffusion. I can believe Claude will become much more useful and still question whether customers will adopt it fast enough to support the spending commitments. Google's backing of the lease also shows how much the financing depends on other companies standing behind those obligations. For a business choosing a supplier, I'd want the work we're building to survive a change in model or commercial terms; that's why we care about keeping the company's knowledge separate from the model underneath it.

Read more: Reuters, November 5

Other Interesting Things I've Read or Seen This Week

Investors can't agree whether AI job cuts are good news (Bloomberg, November 3). Companies are linking restructuring to AI, with very different reactions from investors. I'd want to see whether the remaining team can sustain the output and service levels before treating the headcount reduction as evidence of productivity.

OpenAI says its first half was “pretty rough” (WSJ, November 4). Greg Brockman described a recovery in coding and enterprise after a difficult first half. It's another reason I'd be reluctant to build a company's workflows around the assumption that today's leading model stays ahead.

Figma is using its own model to protect its margins (Reuters, November 4). Figma says its own design model is faster and cheaper to serve. I'd expect more companies to make these choices task by task as they work out how much capability they actually need and what they're paying for it.

XPO put AI planning across its trucking network (XPO Q3 call, October 30). Shipments grew nearly 6% with a smaller increase in headcount, while freight rehandles fell to their lowest in more than a decade. These are the measures I'd want in a deployment review, with enough detail to understand how much of the improvement AI actually contributed.

Shopping agents are coming, slowly. Uber reported just four agent-booked rides during Halloween week, while Amazon and Walmart are taking different approaches to outside agents. I'd prepare for customers arriving through an agent, but those usage figures would keep me cautious about the near-term revenue forecast.

The rules are catching up with AI that acts. The OCC's examiner guidance focuses attention on what an AI mistake could do; McDonald's is contesting AI pricing allegations. Both make me think about the authority we give a system, especially when its output can turn into a transaction or a price.

Two JPMorgan advisers left rather than sell their clients' AI stocks (WSJ, November 5). The advisers took their concentrated portfolios to a new firm before this week's sell-off. Believing in AI's long-term prospects wouldn't make me comfortable with nearly a fifth of a client's portfolio in one company.

Marvell is buying Lumentum for $115 billion (Light Reading, November 2). The all-stock deal isn't expected to close until the first half of 2028. After a week like this, I'd spend some time on what the buyer and seller are each assuming about demand by then.

Know someone who'd find this useful? Forward this email, or send them your share link. Most new readers come from a friend passing it on.

Acquisition Intelligence is a weekly newsletter on how mid-market companies put AI to work, for operators and the investors who back them.

If you're working through any of these problems, reply. I'd be interested to hear where you're getting stuck or what's working.

P.S. I'm Harry, co-founder of DealSage. We work with companies to understand the bottlenecks, get their data into a form they can trust and build AI into the work itself. If you're trying to work out where to start in your business or a portfolio company, reply and we can compare notes.