Sergei Gukov: AI tools for long-horizon, sparse-reward tasks

8 Просмотры

Спасибо! Поделитесь с друзьями!

Вам не понравилось видео. Спасибо за то что поделились своим мнением!

Издатель Jul 9, 2026

AI systems are getting better at coding, robotics, games, and mathematics, but many of the hardest scientific problems share two bottlenecks: sparse rewards and long horizons. Sergei Gukov explains why these challenges make tasks like fusion reactor design, autonomous experimentation, and research-level mathematics fundamentally different from ordinary benchmark problems.
Using examples from Ramsey theory, the algebraic Hirsch conjecture, reinforcement learning, Montezuma's Revenge, algorithm discovery, and the Andrews-Curtis conjecture, Sergei argues that mathematics is an ideal laboratory for building stronger AI reasoning systems. Because mathematical data can often be generated synthetically, it offers a way to study rare, high-value discoveries without relying on expensive or proprietary datasets.

Outline of this video:
00:00 Sparse rewards and long horizons in AI
03:10 Coding agents and domains where feedback is scarce
06:01 Robotics, AI labs, and real-world feedback loops
07:37 From quantum topology to Caltech's MathAI Lab
09:29 Can AI become a stronger mathematician?
12:24 Ramsey numbers and astronomical search spaces
14:34 The algebraic Hirsch conjecture as an AI case study
16:45 Reinforcement learning as a discovery framework
18:26 DQN, Atari, and Montezuma's Revenge
22:57 Go-Explore and lessons from failed exploration
25:10 Algorithm discovery and two levels of sparse reward
26:51 Andrews-Curtis conjecture and synthetic data gaps
29:27 Rubik's cube analogies for hard group theory search
33:19 Mathematics as a cheap laboratory for AI research