--- Frank C. Wimberly 140 Calle Ojo Feliz, Santa Fe, NM 87505 505 670-9918 Santa Fe, NM
---------- Forwarded message --------- From: Diane Stidle <[email protected]> Date: Mon, Jul 29, 2024, 1:38 PM Subject: Thesis Proposal - July 30, 2024 - Ananye Agarwal - Pretrain in sim, adapt in real: A framework for sensorimotor intelligence To: [email protected] <[email protected]>, Pieter Abbeel < [email protected]>, <[email protected]> *Thesis Proposal* Date: July 30, 2024 Time: 2:00pm Place: NSH 1305* Speaker: Ananye Agarwal *Title: Pretrain in sim, adapt in real: A framework for sensorimotor intelligence* Abstract Machines today can write poetry, compose music, and create art but are incapable of mundane physical tasks like household chores or assembly – they lack simple sensorimotor skills. Internet data alone, while useful for high-level planning, might not be enough for learning how to affect the physical state of the world. In this proposal, I will talk about a route to acquiring these skills using large-scale learning from both simulation and real world data. Affecting world state involves changing the robot state (locomotion/navigation) or that of the environment (manipulation) in a desired way. First, I present a simple framework that allows learning adaptive, agile locomotion for a low-cost quadruped from large-scale data in simulation. Next, I apply this to dexterous manipulation for functional grasping. Capable general-purpose robots must not only manipulate and locomote, but do so together in a seamless fashion. I present a mobile manipulation system that navigates, manipulates objects, and chooses where to look, all learnt concurrently using a single, end-to-end neural network. Finally, for proposed work, I present roadblocks to scaling sensorimotor learning and an approach to leverage real-world data. *Thesis Committee:* Deepak Pathak (chair) Ruslan Salakhutdinov Pieter Abbeel (UC Berkeley) Jitendra Malik (UC Berkeley) Zoom link: https://cmu.zoom.us/j/95525240691?pwd=naKUBhWtQ2RXqv6ZRkLyeuzpuTVAHO.1 <https://www.google.com/url?q=https://cmu.zoom.us/j/95525240691?pwd%3DnaKUBhWtQ2RXqv6ZRkLyeuzpuTVAHO.1&sa=D&source=calendar&ust=1722700104671601&usg=AOvVaw2qo2z4jsqzXMnlvRURlPH_> *To get to NSH 1305, please take the elevator in front of the RI reception down to floor 1, and find the lecture room after exiting on the left. -- Diane Stidle PhD Program Manager Machine Learning Department Carnegie Mellon [email protected]
-. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. --- -.. . FRIAM Applied Complexity Group listserv Fridays 9a-12p Friday St. Johns Cafe / Thursdays 9a-12p Zoom https://bit.ly/virtualfriam to (un)subscribe http://redfish.com/mailman/listinfo/friam_redfish.com FRIAM-COMIC http://friam-comic.blogspot.com/ archives: 5/2017 thru present https://redfish.com/pipermail/friam_redfish.com/ 1/2003 thru 6/2021 http://friam.383.s1.nabble.com/
