Machine Learning Strategy and Intro to Reinforcement Learning


Stanford School of Engineering

Machine Learning Strategy and Intro to Reinforcement Learning


NOTE: This course is a continuation of XCS229i: Machine Learning. Though not strictly required, it is highly recommended to take XCS229i before enrolling in XCS229ii, as assignments assume knowledge of topics in the first course.

As machine learning models grow in sophistication, it is increasingly important for its practitioners to be comfortable navigating their many tuning parameters. Through video lectures and hands-on exercises, this course will equip you with the knowledge to get the most out of your data. You will learn the concepts and techniques you need to guide teams of ML practitioners. This course also introduces you to the field of Reinforcement Learning. You will learn to solve Markov decision processes with discrete state and action space and will be introduced to the basics of policy search.

In the last segment of the course, you will complete a machine learning project of your own (or with teammates), applying concepts from XCS229i and XCS229ii. You will have the opportunity to pursue a topic of your choosing, related to your professional or personal interests.

This course features classroom videos and assignments adapted from the CS229 graduate course delivered on-campus at Stanford. In order to make the content and workload more manageable for working professionals, the course has been split into two parts, XCS229i: Machine Learning I and XCS229ii: Machine Learning Strategy and Intro to Reinforcement Learning.

What you will learn

  • Reinforcement learning (Markov decision processes, including continuous and discrete state, finite/infinite horizon; value Iteration, policy Iteration, linear quadratic regularization, policy search)
  • Machine learning strategy (regularization, model selection and cross validation, empirical risk minimization, ML algorithm diagnostics, error analysis, ablative analysis)


  • Proficiency in Python: All class assignments will be in Python. If you have extensive programming experience in a different language (C/C++/MATLAB/Java/JavaScript) we recommend you familiarize yourself with Python basics before you begin your first course. Some assignments will require familiarity with basic Linux command line workflows.
  • College Calculus, Linear Algebra: You should be comfortable taking (multivariable) derivatives and understanding matrix/vector notation and operations. We strongly recommend you review this baseline problem set from the Fall 2018 graduate course upon which much of this course is based. You should be familiar with the topics covered before enrolling in XCS229i.
  • Basic Probability and Statistics: You should know the basics of probabilities, gaussian distributions, mean, and standard deviation.


This professional online course, based on the on-campus Stanford graduate course CS229, features:

  • Classroom lecture videos edited and segmented to focus on essential content
  • Coding assignments enhanced with added inline support and milestone code checks
  • Office hours and support from Stanford-affiliated Course Assistants
  • Cohort group connected via a vibrant Slack community, providing opportunities to network and collaborate with motivated learners from diverse locations and professional backgrounds

How is this different from the machine learning course on Coursera?

The Machine Learning MOOC offered on Coursera covers a few of the most commonly used machine learning techniques. XCS229ii will cover completely different topics than the MOOC and include an open-ended project. You may gain a better sense of comparison by examining the CS229 course syllabi linked in the Description Section above and the course lectures posted on YouTube.

Time Commitment

Expect to commit 8-12 hours/week for the duration of the 10-week program.


Andrew Ng
Adjunct Professor of Computer Science

Guest Lecturers

Anand Avati, CS229 Head TA


Upon completing this course, you will earn a Certificate of Achievement in Certificate of Achievement in Machine Learning Strategy and Intro to Reinforcement Learning from the Stanford Center for Professional Development.

You may also earn a Professional Certificate in Artificial Intelligence by completing three courses in the Artificial Intelligence Professional Program.

Continuing Education Units

By completing this course, you'll earn 10 Continuing Education Units (CEUs). CEUs cannot be applied toward any Stanford degree. CEU transferability is subject to the receiving institution’s policies.


Prior to enrolling in your first course in the AI Professional Program, you must complete a short application (15-20 minutes). The application allows you to share more about your interest in joining this cohort-based course, as well as verify that you meet the prerequisite requirements needed to make the most of the experience. If you have previously completed the application, you will not be prompted to do so again.


Contact us at 650-204-3984

Course Outline


001 Autumn 2020-21 Online

Enroll Now

Dates:October 12 - December 20, 2020
Delivery Option:
Online Course $1,595.00

This is a cohort-based program that will run from OCTOBER 12, 2020 - DECEMBER 20, 2020.

Online Program Materials 
Online program materials are available on the first day of the course cohort (October 12, 2020). A course syllabus and invitation to an optional Orientation/Q&A Webinar will be sent 10-14 days prior to the course start.

To successfully complete the program, participants will complete three assignments (mix of programming assignments and written questions) as well as an open-ended final project. 

Course Evaluation
Participants are required to complete the program evaluation.

This course may not currently be available to learners in some states and territories.