Xelix started life with classical machine learning at the core of its product: detecting invoice errors and duplicates. A large enterprise can process millions of invoices a year from thousands of suppliers, often on ERP systems set up decades ago. This process is highly manual: keying in invoice details, matching them to purchase orders, chasing statements, answering supplier queries. Manual work at that volume creates errors, and in accounts payable errors cost money: the same invoice paid twice, or a credit note that was never applied.

This made accounts payable (AP) a great domain for classical machine learning:

That said, AP is far from a solved problem. Every customer's ERP, suppliers and processes are a little different, the edge cases are endless, and at enterprise scale a model that is right 99% of the time can still get thousands of things wrong, which is what makes it such an interesting domain to work in.

From NLP to LLMs

Much of the work of an AP clerk is text-based, so NLP has long been a big part of what we do, from embeddings and clustering to fine-tuning small transformer language models. When LLMs became powerful, cheap and accessible enough to use in production, what was possible changed almost overnight. We used to need a separate model for each text-based task within our product, with its own labelled dataset. Now one model can read an email, classify it, pull out the relevant information, and draft a reply. This leap in capability, available to anyone with an API key, has spawned a whole new class of vertical AI companies, and lets us add value in ways that simply weren't possible before. However, while the models changed, the underlying domain expertise, thoughtfulness, and product sense needed to turn any kind of model into something that produces real value for customers are still as needed as ever.

Agents and the ladder of autonomy

But the advancement in capabilities didn’t stop there… next came agents, systems built using LLMs that can carry out almost any task when given the right tools. Suddenly, it feels like agents are everywhere: every webpage you visit, every service you use. Agents make it possible to climb much further up the ladder of automation. Bain Capital Ventures (BCV) offers a handy way to picture that ladder, borrowed from the levels used to describe self-driving cars: the "6 Levels of Autonomous Work”:

As BCV puts it, moving up the levels brings "higher ROI and easier implementation, but it will also require higher trust that the AI understands your use cases and has durable mechanisms to kick out of the autopilot to a (trained) human in emergency situations." Full automation now feels within reach for more and more tasks, given the tools we have and how fast they're evolving.

So, the scope of what we can build is now enormous - but so are customer expectations. Among AP teams we see a broad spectrum of attitudes to AI. Some teams remain relatively cautious - they want full observability into the decisions made by their AP platform’s intelligence, so they can check every step taken. Others increasingly use AI products in their everyday lives and so are very AI-native: they understand the difference between an RPA or rule-based workflow and a true agentic workflow, and they expect to be able to configure these workflows to their specific needs. Serving both ends of that spectrum makes flexibility and configurability a core requirement of any system we develop, and means always designing for different levels of autonomy.

What it means to be an AI engineer now

This shift towards building on top of LLMs is reflected in the rise of the “AI Engineer” as a role, as detailed in this essay from Latent Space, which describes a "once in a generation 'shift right' of applied AI": tasks that "used to take 5 years and a research team to accomplish in 2013, now just require API docs and a spare afternoon in 2023."

A few years ago our day-to-day questions were about whether we were underfitting or overfitting, or how balanced our training set was. Now, our discussions often revolve around prompt engineering, latency, cost and model evaluations. But the fun part of the role is that you get to do both, and a large part of the job is deciding which to use. Just because you can use an LLM for it (which you can do for pretty much anything) doesn’t mean you should. Between tree-based methods at one end and frontier LLMs at the other there's a whole spectrum: fine-tuning a smaller open-weight LLM, fine-tuning a compact encoder, or embeddings plus a simple classifier. A whole playground of options to explore!

Agents add a new question: how do we show the user what the agent is doing, and give them confidence as we increase the level of automation in our product? Part of the answer is a mix of observability and evals: giving users the chance to review an agent’s actions before they take effect, logging every step an agent takes, showing users why it did what it did, and testing every change against golden sets of real cases before it reaches production.

However, since LLMs are non-deterministic, there’s a non-zero chance they give incorrect answers in production, even if they pass similar cases in golden dataset tests. In some industries this might be an acceptable risk, but in finance there are strict rules and regulations, and mistakes can be very expensive. So the real question is which actions within the platform the agent can handle autonomously, and which actions always need a human in the loop. Those lines can move as confidence in each action grows and as models become increasingly capable.

This shift in engineering has also caused roles across the team to increasingly converge. Product decisions are now core to being an AI engineer shipping agents: the prompt you are designing and testing encodes the core behaviour of the product, so deciding what it should do and what a good answer looks like to an AP clerk is now an inherent part of your job. The pull works in both directions: the traditional technical vs product split is degraded, meaning that to survive in the brave new world of “product engineers”, product managers and designers may be expected to upskill technically and commit code, just as engineers may be expected to upskill on product management.

An exciting time to build

It’s an exciting time to be an AI engineer! New developments are arriving at a dizzying pace. Nothing is fixed or can be taken for granted; even parts of the platform that had stayed the same for years can be challenged overnight by a new release. (For example, TypeSafe AI's System One models return calibrated, structured decisions instead of text, which changes the playing field for classifiers that used to need their own labelled training data.) Some developments are counter-intuitive. The Jevons paradox says that when using a resource becomes more efficient, we tend to use more of it, not less. As tokens get cheaper, we don't spend less on AI; we find many more tasks that are suddenly worth automating. Others change how we work with models altogether. Beyond prompting, more and more can be done in weight space itself: fine-tuning small models for narrow tasks, swapping in LoRA adapters, even getting LLMs to write another network's weights directly!) All of which makes reading papers, release notes (and maybe even AI twitter) a key part of the job too.

Xelix is a great place to do this work. Despite growing quickly over the past year, we are still a startup with an incredibly fast-paced and entrepreneurial culture, where everyone is encouraged to suggest and try new ideas like these. At the same time, our products are relied on every day by AP teams at large enterprises around the world, so we get to apply these advancements to real, customer-facing problems, and get fast feedback on what we’ve built. If that sounds like your kind of problem, we're hiring: https://careers.xelix.com/jobs.