Natural Programming Lab

Natural Programming Lab (NPL) is led by Prof. Yewen Pu at Nanyang Technological University. NPL studies how to bridge the gap between today’s agentic systems and true human collaborators. While modern LLMs excel at tasks with clear objectives and feedback, many real-world problems require alignment through interaction. Our work focuses on communication-driven intelligence—such as iterative repair, convention formation, and the development of shared mental models—and builds interactive systems that are both natural to instruct and reliable in execution.

Research Highlight

mrCAD: Multimodal Refinement of Computer-aided Designs

William P. McCarthy, Saujas Vaduguru, Karl D. D. Willis, Justin Matejka, Judith E. Fan, Daniel Fried, Yewen Pu

EMNLP 2025

Multimodal instructions for iteratively refining CADs (text, sketch, or both) across human games—benchmarking VLMs on refinement versus generation.

News

  • April 2026: Our paper (When Robots should say I don’t know) is selected as a CVPR highlight paper
  • April 2026: one paper (Bongards at the Boundary of Perception and Reasoning: Programs or Language) is accepted by CogSci