Catalogo dei Corsi di studio

Office hours / ricevimento: Wednesday, 3:30pm-5:30pm (contact me first, booking is required)

Academic Year 2023/2024

Courses

• Fall, Reinforcement Learning (6 CFU), M.Sc. in Artificial Intelligence and Robotics
  • Login to Google account on studenti.uniroma1.it and access Classroom (https://classroom.google.com or Classroom smartphone app), subscribing to the RL course using the code "kbrncgq".
  • Lessons will take place every:
    • Wednesday 12:00 - 14:00 (theory), Room 201, Building D, Viale Regina Elena 295
    • Friday 11:00 - 15:00 (theory + practical), Room 201, Building D, Viale Regina Elena 295

    All communications will occur through Google Classroom.

  • Description:
    Modern Artificial Intelligent systems are often required to make sequential decisions in an unknown and uncertain environment, by actively interacting with it. Reinforcement Learning is a general framework and powerful paradigm that captures such interactive learning setting and it is relevant to an enormous range of tasks, including robotics. In the latest years, it has also been used to develop agents that achieve super-human level performances on challenging tasks such as Go, computer games, and robotics manipulation.
    This graduate level course focuses on the foundations of Reinforcement Learning and provides a solid introduction to the field of reinforcement learning. Students will learn about the core challenges and approaches, through a combination of lectures, and written and coding assignments. Assignments will include the basics of reinforcement learning as well as deep reinforcement learning — that combines deep learning techniques with reinforcement learning. Students will be able to conduct research on RL related topics, if they want.
  • Prerequisites: This is a math-heavy course, with focus on algorithm design and analysis. For this reason, we require students to be comfortable about basics of calculus, probability and linear algebra. A previous Machine Learning background is preferred but not required.
    Since practicals and assignments consist of programming problems, we expect ALL students to be able to implement algorithmic ideas in code and to be proficient in Python programming. There is a tutorial here (http://cs231n.github.io/python-numpy-tutorial/) for those who are not familiar with Python. If you have a lot of programming experience but in a different language (e.g. C/C++/Matlab/Javascript) you will probably be fine.
  • Learning Outcomes: By the end of the class students should be able to:
    • Define the key features and challenges of reinforcement learning, with specific focus on the exploration vs exploitation challenge;
    • Given an application domain, define it formally as a RL problem (in terms of the state space, action space, dynamics and reward model), state what algorithm is best suited for addressing it and justify your answer;
    • Describe multiple criteria for analyzing RL algorithms and evaluate algorithms on these metrics: e.g. regret, sample complexity, computational complexity, empirical performance, convergence, etc;
    • Implement in code common RL algorithms;
    • Be able to understand research papers in the field and extend them with ideas of your own.
  • Grading: Details on Classroom

Academic Year 2022/2023

Courses

• Fall, Reinforcement Learning (6 CFU), M.Sc. in Artificial Intelligence and Robotics
  • Login to Google account on studenti.uniroma1.it and access Classroom (https://classroom.google.com or Classroom smartphone app), subscribing to the RL course using the code "kbrncgq".
  • Lessons will be recorded and will take place every:
    • Wednesday 11:00 - 15:00 (theory + practical), San Pietro in Vincoli, Room 29
    • Friday 12:00 - 14:00 (theory), San Pietro in Vincoli, Room 38
  • All communications will occur through Google Classroom.
  • Description:
    Modern Artificial Intelligent systems are often required to make sequential decisions in an unknown and uncertain environment, by actively interacting with it. Reinforcement Learning is a general framework and powerful paradigm that captures such interactive learning setting and it is relevant to an enormous range of tasks, including robotics. In the latest years, it has also been used to develop agents that achieve super-human level performances on challenging tasks such as Go, computer games, and robotics manipulation.
    This graduate level course focuses on the foundations of Reinforcement Learning and provides a solid introduction to the field of reinforcement learning. Students will learn about the core challenges and approaches, through a combination of lectures, and written and coding assignments. Assignments will include the basics of reinforcement learning as well as deep reinforcement learning — that combines deep learning techniques with reinforcement learning. Students will be able to conduct research on RL related topics, if they want.
  • Prerequisites: This is a math-heavy course, with focus on algorithm design and analysis. For this reason, we require students to be comfortable about basics of calculus, probability and linear algebra. A previous Machine Learning background is preferred but not required.
    Since practicals and assignments consist of programming problems, we expect ALL students to be able to implement algorithmic ideas in code and to be proficient in Python programming. There is a tutorial here (http://cs231n.github.io/python-numpy-tutorial/) for those who are not familiar with Python. If you have a lot of programming experience but in a different language (e.g. C/C++/Matlab/Javascript) you will probably be fine.
  • Learning Outcomes: By the end of the class students should be able to:
    • Define the key features and challenges of reinforcement learning, with specific focus on the exploration vs exploitation challenge;
    • Given an application domain, define it formally as a RL problem (in terms of the state space, action space, dynamics and reward model), state what algorithm is best suited for addressing it and justify your answer;
    • Describe multiple criteria for analyzing RL algorithms and evaluate algorithms on these metrics: e.g. regret, sample complexity, computational complexity, empirical performance, convergence, etc;
    • Implement in code common RL algorithms;
    • Be able to understand research papers in the field and extend them with ideas of your own.
  • Grading: Details on Classroom
• Spring, Seminars in Artificial Intelligence and Robotics, M.Sc. in Artificial Intelligence and Robotics

Academic Year 2021/2022

Courses

• Spring, Seminars in Artificial Intelligence and Robotics, M.Sc. in Artificial Intelligence and Robotics
  • Login to Google account on studenti.uniroma1.it and access Classroom (https://classroom.google.com or Classroom smartphone app), subscribing to the ML course using the code "4j2kuqe".
  • Lessons will take place every Friday at 11:00am, in Room A3 and on zoom, in hybrid format.
• Spring/Summer, Advanced Topics in Reinforcement Learning: From Theory to Practice, PhD in Engineering in Computer Science
  • Login to Google account on studenti.uniroma1.it and access Classroom (https://classroom.google.com or Classroom smartphone app), subscribing to the ML course using the code "tov5h5x".
  • Lessons will take place starting from June 2022 in hybrid format.

Academic Year 2020/2021

Courses

• Fall, Machine Learning (Reinforcement Learning Part), MSc in Artificial Intelligence and Robotics
  • Login to Google account on studenti.uniroma1.it and access Classroom (https://classroom.google.com or Classroom smartphone app), subscribing to the ML course using the code "kitgmrs".
  • Lessons will take place every Monday at 8:00am, starting from October 5th 2020, in blended mode (Room B2, via Ariosto and on Zoom https://uniroma1.zoom.us/j/82583993429), and they will be recorded and uploaded on YouTube on the teacher’s personal channel (https://www.youtube.com/channel/UCIxb9KSJ-M35ClQVl78uA9Q).
• Spring, Advanced Topics in Reinforcement Learning: From Theory to Practice, PhD in Engineering in Computer Science
  • Login to Google account on studenti.uniroma1.it and access Classroom (https://classroom.google.com or Classroom smartphone app), subscribing to the ML course using the code "biuuwcc".
  • Lessons will take place every Monday at 2:00pm, starting from April 26th 2021, remotely on zoom.

Academic Year 2019/2020

10/Mar/2020 - Important updates for the Seminars in Artificial Intelligence course:

Despite the coronavirus outbreak, classes will continue for the 2019/2020 semester in a remote mode. In order to attend, students must register to the course using the Google Classroom application. In order to register to the course, students should login to their Google account on studenti.uniroma1.it and access Classroom (https://classroom.google.com or Classroom smartphone app), subscribing to the Seminars in Artificial Intelligence course using the code "qe47hb7".

Remote classes will be organized according to the scheduled time-slots: Tuesday, 4pm-7pm. Links for the live class streaming on Youtube, as well as recordings and any other teaching material will be provided on the Classroom platform, and they will be accessible until the end of the semester.