Open-source recipe teaches coding AI to paint watercolours
Original: Training a coding model to paint watercolours with TRL and OpenEnv
Source: huggingface.co ↗
Who: Posted on the Hugging Face blog by Sergio Paniego, an engineer at Hugging Face. He is building on and open-sourcing a viral project by Surya Narreddi, an artist and designer who first trained a model to paint watercolors.
What's new: A complete, publicly available recipe for training a coding model to paint watercolor-style images. The model doesn't generate a picture directly. Instead, it writes JavaScript code, about 150 lines, that uses a drawing library to paint the image brushstroke by brushstroke. The viral video showed single flowers; this open version expands to full compositions and publishes every piece of the pipeline.
How it works: The training uses , not the more common approach of memorizing fixed examples. The model starts from Qwen, a 35-billion-parameter language model, and is nudged to improve through a reward system with four parts. Two parts are simple checks: the code must actually run and produce a paintable image, and longer code is mildly encouraged. The other two are taste judges. One, HPSv3, an open source model, scores how much an average person would prefer the image. The other, Qwen3-VL, is called a pairwise judge because it compares the model's painting side-by-side against four hand-selected reference watercolors and awards a win percentage.
The numbers: The team ran three versions of the training, each weighing the two taste judges differently. One run relied entirely on the average-person judge; the other two mixed it with the style judge that compares against the curated pool. Both mixed runs learned successfully, which matters because relying on a tricky style judge could have caused the model to fail entirely. All runs happened on Hugging Face's cloud infrastructure, with a training script that takes one command to launch.
Why it matters: This project tackles "reinforcement learning over taste," where there is no right answer — only aesthetic preference. The novel shift is that beauty is defined not by a model trained on everyone's preferences, but by a small hand-curated pool of reference images chosen by one person. That moves the craft from tuning training parameters to curating the set that defines what is beautiful. The final code is readable and editable, making each brushstroke decision transparent rather than hidden inside a neural network.