LLMs Aren’t The Only Game In Town

We’ve spent the last few years putting LLMs everywhere, but a new class of models released this week is a good reminder that they aren’t the only way to put intelligence into a product.

Jev from TypeSafe AI is designed for small, focused decisions rather than generating lots of text: which route should this take, should this be escalated, which model should handle it? It doesn't generate anything so it can't hallucinate. What it can do is process a lot of raw data and give probabilities on how to act. You just need a threshold to put into your app for when to take action. TypeSafe says responses can come back in tens to hundreds of milliseconds, with input pricing around $0.04 per million tokens, which makes this affordable to use in lots of different use cases.

So what does this mean for product teams?

Fortunately, the first wave of putting the sparkle icon into an app and saying it is now AI is coming to an end. Customers don't want a weaker model embedded when the core models can access your site anyway. What this offers is a much smoother UI experience, but it requires thinking about the workflows that your users are doing and redesigning them to take advantage of this option. One thing people are experimenting with is detecting traffic type in each request and routing agents versus humans to different areas / giving different responses. Or immediate categorization of large blocks of data. When a new technology like this comes out, there are always a range of ways to apply it. And the good news is that it means there is opportunity for product teams to experiment and find what works. Good luck.

This Week's Poll

Last week’s poll: We asked what you thought about the pace of AI. 73.3% wanted it to slow down, 13.3% felt the pace was about right, 13.3% were unsure and no one wanted it to accelerate.

This Week’s Updates

Enabling the Team

Parsing “Can” from “Should”: How Netflix’s Product Team Got Thousands of Creatives to Trust an AI Overhaul by First Round Review, featuring Mckenzie Lock
Netflix’s AI rollout started with where workflows were genuinely broken and what people should still own, using shadowing, structured debate and dissent to separate valuable automation from work where craft and human judgment needed to stay in control.

Everything Important Is Optional Until It Isn’t by Steve Huynh
Preventive work loses to urgent requests until something breaks. Trace recurring crises backwards and protect time for the work that prevents them.

Product

Barbell-Shaped Product Roles by Richard Mironov
Faster development increases the importance of discovery and commercialisation, shifting product work towards the decisions before and after building.

4 New Evals and 16 Experiment Variants to Fix 1 Customer Complaint by Teresa Torres
Fixing one customer complaint in an AI-generated opportunity solution tree required four evals and sixteen experiment variants, showing that reliable AI product quality comes from measuring specific failure modes, iterating against production data and combining prompts with orchestration, guardrails and deterministic checks.

Research

Building AI Evals: Who Gets to Decide What’s Good? by Nam Pham and Andrea Cavicchi
DoorDash shows why useful AI evaluations start with the right human answer key, distinguishing user preferences, domain expertise and firm boundaries.

Hiring Isn’t a Scaling Strategy: Turning Research Expertise into Self-Serve Infrastructure by Lucy Sutton
Two ResearchOps specialists supported 120 researchers by turning repeated requests into self-service infrastructure and measuring greater autonomy.

Design

Designing with AI by Buzz Usborne
Designing with AI works best when designers use it to explore interactions, generate working prototypes and pressure-test ideas earlier, while keeping human judgment in control of the problem, constraints and quality bar rather than treating generated output as the design itself.

Test Complex Interactions Earlier with AI Prototyping by Megan Chan
Use AI prototypes to test realistic behaviour earlier, while keeping research goals and human review ahead of the temptation to build a polished demo.

Dev

Nobody Needs a Human Router by James Stanier
Clear ownership, written updates and protected focus time let engineering leaders replace status routing with technical judgement and targeted help.

Trying the Software Factory Pattern by Will Larson
An early Imprint experiment connects agent work to goals, shared task state and escalation, treating coordination as part of the delivery system.

Seen an interesting article online? Share it with us, and we might feature it in our next issue!
Click here to share an article

Global Product Week Starts Next Week

Oxford and Lisbon join a week of local conversations

It is often easier to work through a difficult product question with people who are facing it too. Global Product Week starts next week, bringing local product, UX, design and engineering communities together from 5–8 October.

Oxford and Lisbon are the newest additions, joining events in New York, Amsterdam, Austin, Dublin, San Francisco and Boise, plus an online workshop. Whether you want to compare approaches or meet people building products nearby, there is plenty happening across the week.

Explore the events and find your local agenda. New York’s AI Evals Sandbox workshop is already full; the separate evening community event is listed below.

FREE COMMUNITY EVENTS

Video Of The Week

Never Done: Evolving UX Teams Who Earn Influence

Growing a UX team does not automatically give it more influence. The harder part is changing how the organisation works with that team, and showing where design makes a difference to the business.

In this UXDX talk, Ben Hewett from Allied Solutions shares how he helped grow UX from three people to 25, moving from an internal agency that took on projects to designers embedded with product teams. He covers the resistance and missteps along the way, and how the team made its contribution clearer through business outcomes.

Watch the full session for practical ways to earn trust, build advocates and evolve the way your UX team works:

Want to go into how careers and leadership are shifting as AI compresses the ladder? My ebook Managing Your Career In The Age Of AI explores how to build judgement, relationships and influence in a world that keeps trying to automate the surface of the work.