perpetual inspiration from this HCI classic 93' Donald Norman Book :
Donald A. Norman, Things that Make Us Smart: Defending Human Attributes in the Age of the Machine
Problem Representations
Let's play a game: the game of "15." The "pieces" for the game are the nine digits- 1, 2, 3, 4, 5, 6, 7, 8, 9. Each player takes a digit in turn. Once a digit is taken, it cannot be used by the other player. The first player to get three digits that sum to 15 wins. Here is a sample game: Player A takes 8. Player B takes 2. Then A takes 4, and B takes 3. A takes 5.
Question 1: Suppose you are now to step in and play for B. What move would you make?
Or instead, consider this:
Suppose the game of ticktacktoe has reached this state. Picture taken from the book.
Question 2: Suppose you are now to step in and play an O for B. What move would you make?
These are the same problem in two different representations. This third representation makes it clear: same game, sum up to 15 in a row of three:
The two problem isomorphs.
The point of this exercise is this: For a human the problem is easy to solve visually, while for a computer the problem is much easier solved arithmetically. The form of representation of the same task makes a dramatic difference in how easy it is to solve. And for whom.
How it relates to AI
Watch this principle be applied with extreme success across different benchmarks, tasks and mediums. Coding was the first domain that saw real hillclimbing breakthroughs for agentic capability, and it's quite a universal tool.
Python Universality
When Norman wrote this book in '93, computers could crunch through problems if you managed to convert them into a computable program. Today, agents solve a task if you manage to convert it into a coding problem.
Tufa Lab's key breakthrough on the ARC-AGI-3 benchmark: turn the visual game state into a Python coding problem the model can inspect, reason over, and solve.
Tufa Labs won the ARC-AGI3 milestone prize for their 'The Duck' approach of turning unseen games that the model has to solve into python coding tasks the model reasons over.
Claude learns to paint beautiful images by writing python code, drawing the image pixel by pixel. It doesn't match what a diffusion model or a VLM can do in terms of realistic and diverse image generation, but it does create better image output than anything available even just a year ago. Because the problem was turned into a coding problem.
Code is all you need
Much of Anthropic success over OpenAI is them betting on code as the one important task to tune and focus RL efforts on early, while OpenAI went much broader, launching models with better visual understanding and even the video model app sora. The result was that Claude models were just so much better at writing code, and that could be applied in broad domains from personal agents to 3d CAD drawing, web design and layouting, that they not only caught up but overtook OpenAI in enterprise adoption this year.
humans don't (read) code
The problem with the extremely code proficient agents of today is that their proficiency no longer matches the problem representations that humans are good at. We built interfaces to work through tasks with visual aids and metaphors that make them intelligible for us, and we are not that great at code.
So when our agents code a painting, it's not a photoshop document we can explore and edit, its a bunch of python code with a structure that completely evades our grasp. We can promtp and get an output, but all intermediate representations are increasingly incomprehensible for us. We now have a blackbox model that works over a blackbox project, spitting out a human-interpretable representation only at the very end: the painting, website, showreel.
Now if you want to productize this, you have to once again introduce a thin veil of Human-readable problem representation and interactivity. @ForgeCAD lets coding agents write complex CAD files (using code, you guessed it). But since the underlying model is now code, the CAD interaction that we used to have, modifying edges and surfaces etc, is missing and instead you have to introduce code-like controls: sliders, graphs, toggles.
closing thoughts
it's always worth it to read thinkers from the past, it refreshes your view of the current world: We are seeing the problem representation principle playing out so clearly and at immense scale, the real software eating the world. Not only can HCI teach you how to help humans use your tools to solve their problems, but AI agents too!
I do think co-operation, steerability and working alongside agents in a continuous flow state are really important, pressing and wholly unsolved issues. Imagine a swarm of astra or fable class model working in real time with you, enriching your understanding and aligning all the key decisions with you. Sure, it's cool to grind out theorem proofs with pure force, but putting that firepower behind the greatest minds excites me just as much.
so time to read some books!