Physical AI perceives and acts in physical environments, using the outcomes of its actions as feedback for learning. Building on multimodal learning and computational science, MACS Lab aims to extend its research from models that interpret observations to models that interact with their surroundings.
Research directions
Multimodal perception
We explore how images, language, gaze, physiological signals, and sensor data can jointly describe people and their surroundings. Topics include understanding human intent and task context, and learning representations that connect perception with action planning.
Action learning and decision-making
Our research interests include imitation learning from demonstrations and reinforcement learning from environmental feedback. Vision–Language–Action (VLA) models offer a direction for interpreting task instructions and choosing actions appropriate to the situation.
Connecting simulation with the real world
We are interested in Sim-to-Real transfer: applying models trained in physics-based simulation to real environments. This includes accounting for sensor noise, changing conditions, and physical constraints, and evaluating the reliability of actions through uncertainty estimation and feedback.
Application directions
- Medicine and healthcare: assistive systems and human–robot interaction informed by human movement and physiological signals
- Robotics and autonomous systems: connecting perception, planning, and control in task-specific contexts
- Digital twins and simulation: learning and evaluation in virtual environments, with model refinement from real observations
These directions connect with our work in medical AI, multimodal learning, and computational science. Alongside model accuracy, we consider generalization to real environments, computational efficiency, and safe interaction as research questions.