I am shifting my research focus from efficient computer vision toward Embodied AI and Reinforcement Learning. This post is a living document — a roadmap I will keep updating as I make progress, and the first entry in this research space.
Why Embodied AI?
Classical vision systems learn to see, but embodied agents must also learn to act in a physical world that is partial, noisy, and unforgiving. The challenge that excites me most is closing the loop between perception, reasoning, and action — so that a robot can generalize across tasks instead of being engineered for a single one.
The Core Stack I Am Learning
- Reinforcement Learning fundamentals — MDPs, policy gradients, value functions, and modern offline RL.
- Vision-Language-Action (VLA) models — how foundation models map language + vision directly to robot actions.
- World models & critics — learning dynamics models and value functions that provide dense credit for long-horizon tasks.
- Simulation & sim-to-real — building scalable, high-fidelity training environments and transferring to real robots.
A good critic does not just say "success or failure" at the end — it tells the agent which intermediate actions actually mattered.
Immediate Next Steps
- Finish a systematic review of recent VLA-RL methods (RECAP, ViVa, ROVE, WCM).
- Reproduce a small manipulation benchmark in simulation.
- Write reading notes on progress reward modeling and critic-based robot learning.
I will post updates here as I go — consider this the anchor of my research space. 🚀