Watch Notice Robot Maker Video Tutorials & Courses

Two-wheeled balance bots are a long-solved problem thanks to techniques like PID loops. In this series, we explore using reinforcement learning algorithms (RL) to have a robot learn to balance on its own. We will deploy the trained agent to a real ESP32-based robot platform, tackle the sim-to-real gap using post-processing and domain randomization, and eventually build a full remote-controlled balance bot. The same techniques used here are what researchers use to train state-of-the-art walking robots. So, while we are starting with a simpler project, know that these skills will transfer to modern robotics. Click here to register for Shawn’s webinar: https://event.on24.com/wcc/r/5384134/958EFFE494BF1C6A2AD94F9B093DC01B?partnerref=vid To follow along with the series, you will need the following hardware: ● M5Stack BALA2 Fire robot kit: https://www.digikey.com/short/7h2fb5hd All project files, including the FreeCAD model, MuJoCo XML descriptor, and Docker environment, are available in the following GitHub repository: https://github.com/ShawnHymel/reinforcement-learning-for-robotics/ In this first episode, we start by covering the major robot simulation platforms available today (including Gazebo, PyBullet, MuJoCo, Isaac Sim, and Genesis) and explain why MuJoCo is the right choice for this series. From there, we walk through the process of measuring the physical robot, building a simplified CAD model in FreeCAD, and exporting the rigid body meshes along with their mass and inertia properties using some custom Python scripts. You can read a full write-up of everything we cover in this video here: https://www.digikey.com/en/maker/projects/c918da3a6f914778a4df894d8fdc29c4 Once we have our meshes and physics data, we build the MJCF descriptor file that brings everything together in MuJoCo, including defining the robot's bodies, joints, actuators, and sensors. We finish the episode by loading the model into a Jupyter notebook, firing up the MuJoCo viewer, and attempting to balance the robot manually using keyboard controls. By the end, you'll have a working digital twin of the robot that we can use to train a reinforcement learning agent in the next episode. Related link: https://www.digikey.com/en/maker/tutorials/2026/reinforcement-learning-for-robotics-part-1-cad-to-mujoco-simulator Links: Khan Academy Statistics and Probability course: https://www.khanacademy.org/math/statistics-probability Coursera Deep Learning specialization: https://www.coursera.org/specializations/deep-learning Coursera Reinforcement Learning specialization: https://www.coursera.org/specializations/reinforcement-learning Introduction to Reinforcement Learning textbook by Sutton and Barto: http://incompleteideas.net/book/the-book-2nd.html Reinforcement Learning math blog post series: https://shawnhymel.com/3316/what-is-reinforcement-learning/ Introduction to Reinforcement Learning video: https://www.youtube.com/watch?v=3av8vozEczU Introduction to FreeCAD: https://www.youtube.com/watch?v=8VPYTTnqmfs Learn More: Maker.io - https://www.digikey.com/en/maker DigiKey’s Blog – TheCircuit https://www.digikey.com/en/blog Connect with DigiKey on Facebook https://www.facebook.com/digikey.electronics/ And follow us on X: https://x.com/digikey Product: K014-E M5Stack Technology Co., Ltd. | Robotics Kits | DigiKey K038-B M5Stack Technology Co., Ltd. | Robotics Kits | DigiKey Chapters: 0:00 Intro 2:00 Why use RL 3:19 Prerequisites 5:13 Overview of simulators 7:15 Measure robot parts 9:15 CAD to simulator process overview 10:24 Python scripts for FreeCAD 14:22 Robot model in FreeCAD 24:28 Measuring IMU location 25:02 URDF overview 27:32 MJCF overview 43:28 Docker image 46:56 Simulating the robot with MuJoCo in Jupyter Lab 55:21 Challenge: PID loop 55:48 Conclusion Key Highlights & Overview: Understanding <strong>Notice Robot Maker</strong> is essential for modern technical workflows and software skills. Explore curated video courses, expert walkthroughs, and detailed lesson notes today on <strong>TutorTube</strong>.