The idea
The Adeept PiCar-B2 is four wheels with front-wheel steering, a pan/tilt camera and an ultrasonic distance sensor on a Raspberry Pi. Its stock software is a manual web joystick. This project replaces the human driver with a closed-loop agent. You hand the robot a goal in plain English ("find the red ball", "drive forward until you see a doorway, then stop") and it works out the rest.
The loop
- Perceive. Grab a camera frame plus the forward ultrasonic distance.
- Reason. The model looks at the image and decides the single best next move.
- Act. It calls exactly one tool:
drive,turn,lookorstop. - Repeat. Observe the new frame and re-plan, until the goal is met or judged unreachable.
One tool call per step is the constraint that makes the thing debuggable. The agent cannot plan a five-move sequence and execute it blind. Every action is followed by fresh perception, which is the only way the loop stays honest about a world that does not match its expectations.
Architecture
The design principle is one hardware abstraction, many interchangeable control interfaces. Every brain, whatever paradigm it works in, drives the robot through the same RobotBackend surface (drive, turn, look, stop, distance, frame), so adding a new interface never touches hardware code.
That surface has two implementations: a MockBackend with a synthetic camera that runs on a laptop with no hardware attached, and a PicarBackend driving real motors, servos and camera. Being able to develop and test the whole agent loop without the robot on the desk is what made the project tractable.
Where it goes next
The abstraction exists so the brain can be swapped. Two interfaces are on the roadmap and not yet built: a Vision-Language-Action policy mapping pixels and text straight to actions, and a world model doing action-conditioned prediction in the V-JEPA family. The interesting comparison is not which one wins, but what each paradigm needs from the same physical surface.