Microsoft Open Source Blog 
Autonomous driving has transcended far beyond being a crazy moonshot idea over the last half decade or so. It has quickly become one of the biggest technologies today that promises to shape our tomorrow, not very unlike when cars first came into existence. A big driver powering this change is the recent advances in software (Artificial Intelligence), hardware (GPUs, FPGAs etc.) and cloud computing, which have enabled ingest and processing of large amounts of data, making it possible for companies to push for levels 4 and 5 of autonomy. Achieving those levels of autonomy though, require training on hundreds of millions and sometimes hundreds of billions of miles worth of training data to demonstrate reliability, according to a report from RAND. This is where the ability to simulate real driving environments plays a key role and is hence the current focus of the cookbook.
Recipes for everyone
Tutorials in the cookbook are presented as Jupyter notebooks, making it very easy for you to download the instructions and get started without a lot of setup time. To help this further, wherever needed, tutorials come with their own datasets, helper scripts and binaries. While the tutorials leverage popular open source tools (like Keras, TensorFlow, etc.), as well as Microsoft open source and commercial technology (like AirSim, Azure virtual machines, Batch AI, CNTK, etc.), the primary focus is on the content and learning, enabling users to take what they learn here and apply it to their work using tools of their choice.
We aim to make the cookbook a valuable resource for anyone working in autonomous driving, be it research or industry, beginners or experts. For people who are just starting out, we have a tutorial on End-to-End Deep Learning for Autonomous Driving, which serves as a great introduction to the fundamentals of steering models, end-to-end deep learning techniques, and the AirSim simulation platform. For those looking to create scalable models on the cloud, we have a tutorial on Distributed Reinforcement Learning which uses Azure Batch to spin up virtual machines to run training on multiple instances of the simulator simultaneously. We will also be soon releasing a tutorial on Lane Detection using Deep Learning where instead of using popular computer vision-based techniques, which are inherently dependent on camera and image properties, we take data collected from simulations and use techniques like transfer learning to train deep learning models using very little real-world data.
Simulation meets reality
Simulation has become the backbone of the autonomous driving industry, providing a means to collect extensive amounts of data for model training as well as providing a safe testbed to crash-test these models. The closer our simulated scenarios are to reality, the closer we get to making autonomous cars a part of our lives. With that in mind, we are also unveiling a new simulation environment, inspired by the popular Hana Highway in Maui. The famous 64.4 miles long ‘Road to Hana’ is a winding path that goes through lush tropical rainforests, ocean front and grassy mountains with many waterfalls, beaches, tunnels and bridges along the way, all of which serve as the perfect backdrop for training autonomous driving algorithms. Our simulated environment is a condensed 1-mile version of this popular tourist destination and is available to use in the binaries included with the cookbook. We will also make it available on the main AirSim repository soon. This environment has an intermediate difficulty level which fits perfectly between the easy Landscape environment and the more advanced City environment currently available in AirSim. Our upcoming lane detection tutorial also uses this environment to train deep learning models.
Growing together for a better tomorrow
The autonomous driving industry is undergoing an interesting phase. We are living in a reality where Artificial Intelligence has created immense potential for a better tomorrow. This has resulted in a plethora of opportunities that did not exist before. There is a very high demand for skilled talent in this field which is unfortunately not being met. Therefore, it is a great opportunity for those of us already working in the field to come together and create collaborative learning platforms, for example the Eclipse OpenADx initiative, which Microsoft is an active contributor to, together with Bosch and several other stakeholders. A big motivation behind the Autonomous Driving Cookbook for us is to promote this spirit of collaboration and so, we welcome other experts in the area to contribute tutorials to the cookbook to make it a go-to resource for people who wish to enter and grow in this exciting field.
Any feedback or other resources you’d like to see? Let us know in the comments below.