AI models learn from imperfect vision-language teachers only when needed
Original: Selective Agent Guidance via Entropy: Learning Autonomous Policies from Imperfect VLM Teachers
Source: arxiv.org ↗
Who: Matteo Merler, Giovanni Bonetta, Davide Zago, Rossella Cancelliere, and Bernardo Magnini, researchers from Italian institutions, posted this paper on arXiv.
What's new: The paper introduces a way to train a small, fast AI agent to solve visual puzzles and navigate spaces, using a big, slow AI teacher only when the student gets confused. This is called SAGE (Selective Agent Guidance via Entropy). The key idea is that the teacher, a or VLM, does not need to be in charge all the time.
How it works: Normally, a smart but slow AI like a VLM would make every decision for an agent, which is costly and can repeat mistakes. SAGE flips this. A lightweight student AI, trained with or RL, tries to act on its own. It only calls the VLM teacher when it is uncertain, a state measured by entropy, which is a fancy way of saying "how unsure the student is." When the teacher gives advice, the student does not blindly copy it. Instead, it weighs that advice based on how helpful it actually turns out to be in the environment, a process the paper calls distilling guidance. This way, bad advice gets ignored, and good advice gets learned permanently.
The numbers: The paper does not headline a single big number but shows results across several sparse-reward visual and navigation tasks. SAGE beats unguided RL in several environments and even surpasses the VLM teacher in some cases. It achieves this by querying the teacher only for a fraction of training steps and zero times at deployment.
Why it matters: This suggests big, expensive AI models are best used as temporary coaches, not permanent drivers. The student learns the useful parts of the coach's knowledge, internalizes it, and then runs fast and free on its own. This could make AI agents much cheaper and more reliable in the real world.
Caveats: SAGE is not a magic bullet. It only helps when the teacher can show the student a path to a big reward that random exploration would miss. If the student can already succeed on its own, or if the teacher's advice leads nowhere, the extra complexity adds little value.