The Future of AV Sensors - Enabling the Autonomous Vehicles of Tomorrow

Austin Russell, CEO of Luminar Technologies, sits down with Alexandr Wang, CEO of Scale AI, to discuss the future impact of LIDAR sensors on the autonomous vehicle industry. Austin shares what he saw that was lacking in existing AV sensors that led him to create Luminar Technologies and dives into the core requirements that they set out to fulfill with their own LIDAR sensors. He explores the optimal way to balance the capabilities of different sensor technologies. Together Austin and Alexandr go on to discuss the bottlenecks inherent in learning from data at the scale required to build safe autonomous vehicles, how sensor manufacturers and OEM companies should be partnering together, and the business models and market opportunities that show the most promise in the future. What does the choice of sensor technologies mean for downstream perception, prediction, and planning algorithms? What do we need from today's hardware or software to enable L4/L5 urban self-driving autonomy? What is the most critical strategic decision that CEOs of companies in this space should be thinking about? Join this discussion to hear how LIDAR sensors for autonomous vehicles are moving from small-scale experimentation to large-scale production and what impact they might have in the industry and for vehicle-owners everywhere.

Speaker 1 (00:22): Next up, we're excited to welcome Austin Russell. Austin Russell is an engineer, self-driving technology pioneer and the founder and CEO of Luminar Technologies. Austin developed the idea for Luminar in 2012 when he was just 17, while studying physics at Stanford University, and has since invented a new type of industry leading LIDAR that helped self-driving vehicles see and understand their surroundings.

Speaker 1 (00:49): Following Luminar's listing on NASDAQ, he became the world's youngest self-made billionaire and was recognized in Forbes' 30 under 30 lists. Austin is a Thiel Fellowship alum and resides in Palo Alto, California. Austin is joined by Scale CEO and founder, Alex Wang for a Fireside chat. Alex, take it away.

Alexandr Wang (01:17): Awesome. Thank you so much for sitting down with us and taking the time to chat. We're super excited to sit down and talk about Luminar.

Austin Russell (01:24): Yeah, no, thanks for taking a second, excited to be on here and great show you got on.

Alexandr Wang (01:30): Awesome. So, I wanted to start out just by talking about your journey. So Austin you've had a pretty incredible journey to running, now one of the largest public companies that's in the autonomous space, and it's been a pretty impressive growth path, not only in terms of technology, but also business success. Would you mind sharing for us at first, what was the journey to starting Luminar in the first place and how has that journey evolved over time?

Austin Russell (01:58): Yeah, I mean, it's definitely been quite the journey since the beginning for this, and I'm sure as you're familiar by your own success in this at earlier age. It really started out with just all this and the intersection between extreme curiosity, passion, and drive, and just the opportunity for what it can mean if we can build something that is meaningful to the holistic autonomous vehicle space.

Austin Russell (02:28): And just that there was a pretty clear opportunity within the LIDAR domain specifically in terms of building these new laser systems. I had been building different types of optimal electronic systems starting at a pretty young age, early on, and always wanted to find novel ways to apply these technologies.

Austin Russell (02:46): So, I really saw a pretty clear opportunity for the autonomous vehicle space in terms of what was needed, and saw that as a critical enabling system, if you were to be able to do it right, but it was going to require something built from the ground up, there was really no feasible way to do this by just using off the shelf parts and components. So that's why I had to be able to start from scratch and build the company to be able to do this.

Austin Russell (03:06): So making all of our own components was definitely not an easy journey, but with the laser, the receiver, the scanning mechanism, processing electronics, it all went into it. But I had to bring on a couple of 100 highly specialized engineers to be to execute against the vision and the strategy in terms of what we had laid forth.

Austin Russell (03:22): And from the perspective of also going into automotive, one of the key goals that I had with this was also to be able to have a product that you could actually ultimately see through, into series production vehicles, something that would have an opportunity to allow autonomy to get out of testing development mode and ultimately make its way into real consumer's hands.

Austin Russell (03:42): And I just saw the opportunity by building a cost effective industrialized product that could actually be scalable economically, in working directly with some of the major automakers. So, that was the vision that we had and just had been continuing to execute to.

Austin Russell (03:57): But it's been a really exciting to actually see all of this come to fruition, be realized and to go from ideas in our heads to a serious business that's tackling the problem and really is the first to be able to make autonomy start to happen in the real world on cars that people can buy.

Alexandr Wang (04:17): Yeah. One question that I have is obviously there's a number of companies and players focused on the LIDAR space, particularly also thinking about autonomy. What have you done over time... I think Luminar has created a pretty differentiated position throughout its lifetime as a company. How have you thought about that from a strategic and technical perspective of differentiating Luminar versus the other players in the space and taking a pretty unique approach?

Austin Russell (04:46): Yeah. So, I think it all comes down to whether you're leveraging existing approaches of building a system or doing something with a clean slate or blank slate approach at a first principal's perspective of how would you build it if you were able to eliminate the constraints of different elements of the design that already exist.

Austin Russell (05:08): And from a differentiation perspective, we're still the company that's truly done this and gone through this exercise. And what became clear is that while there's the 2000 different ways, combination of permutation of ways that you can build a LIDAR system, there's actually a lot of different approaches.

Austin Russell (05:25): There's only one path that we found that could actually meet all the incredibly stringent performance, requirements that are out there to be able to deliver on this. Well, at the same time also mini economics is scalability requirements. But even going back to the performance, I mean, I think there was also a general thought about how much performance do you really need? What problem are we truly trying to solve?

Austin Russell (05:48): If you asked 10 different tech or AV companies, or even OEMs, five years ago about what specs were needed to solve the problem, you'd probably get 10 different answers. And that's the thing, is that really just doing the analysis of what's really needed to be able to solve this, I think, a lot of people have thought that this autonomous vehicle problem was going to be a lot easier than it ultimately ended up to be.

Austin Russell (06:11): And you may not need an extreme level of performance to really understand everything going on, but it all comes down to all the really challenging edge cases, that long tail of all the different possible things that could be thrown out of vehicle which makes it so incredibly hard. Whereas historically, there's physics limitations on what you can build with these systems that the legacy LIDARS and other approaches out there generally only give you one to two seconds of reaction time.

Austin Russell (06:38): And oftentimes these are multi thousand dollars even tens of thousands of dollars to be able to build. So then the question is that, how do you get an extreme level of performance? Like a 10X improvement in multiple dimensions into a device that costs $1,000, rather than tens of thousands of dollars. And that's the really hard part, that's the part that has differentiated us, and really that's our whole claim to fame, is that we were able to solve that problem. And do so in a really unique way.

Austin Russell (07:02): It's allowed us to really have a... what? I mean, even for the fundamentals of the technology perspective, we have a greater IP portfolio than all these other major LIDAR players combined. We have a more commercial contracts with series of production than all these other guys combined. And also I think a greater market capitalization than most of the other guys combined.

Austin Russell (07:26): So that's really the realization and coming to fruition of what it means to have this technology and start to introduce it into market. Now, the reality is, is that it's less about a competitive landscape for this, it's not about how big of a slice of the pie are we going to have versus anyone else is going to have.

Austin Russell (07:48): The question is how big can we make the pie? How quickly can we see the market adoption of this, and how do we make this happen? And that's the golden question, that I think if you asked folks just a few years ago, in terms of when consumers could see this technology, I think people would've thought that for consumer vehicles, it would've been maybe a decade plus away, just by the way of having the technology at this level and at this cost.

Austin Russell (08:18): Bt that's what we're really transforming to actually happen. Now, sooner than later where you can actually buy a Luminar powered car, come end of next year, beginning of '23.

Alexandr Wang (08:28): Yeah. I have to ask the question. There's definitely some notable automobile companies or auto OEMs, Tesla, probably namely, who focus on camera only approaches or visual spectrum sensor only approaches to L2, L3 systems. How do you think OEMs or AV companies in general should be thinking about the trade offs during choosing different sensors, and what do you think about to solve the necessity of LIDAR and why that's an important component of the stack?

Austin Russell (09:01): Yeah. It's a good question. I think actually... here's the thing. When it comes to OEMs using camera based systems for assisted driving solutions, there's nothing wrong with that at all. In fact, it's great, it's actually been a major evolution and improvement upon what's been out there, which is, what a couple decades ago, people didn't really have any of this stuff, and maybe you've started out with just Radar systems, but they've made an impact.

Austin Russell (09:28): Folks like Mobileye, take for example, they have 80%, or there abouts, market share in that space, where they've done a great job of implementing these systems. The key distinction is understanding the limitations of these different sensing modalities and what it actually results in terms of vehicle function.

Austin Russell (09:45): So, the distinction is that camera systems and Radar systems are really good at seeing 95% of what's out there. They see most things, they can be able to recognize it, but they never really have the extreme level of competence, or even just the edge case detection ability to be able to enable you to see that long tail of different events that can happen that's critical for autonomous driving.

Austin Russell (10:09): Even if you were to say, "Hey, we only miss one out of every 20 people with your 95% accuracy." That's completely unacceptable to run over one out of every 20 people. Even if it was 99%, it's completely unacceptable. You need 10, nines worth of reliability to actually have a safe vehicle. And that's the distinction.

Austin Russell (10:33): So everyone recognizes for what it is. And I think every OEM... I mean, we're working with eight of the top 10 major OEMs that are out there to be able to do task and development for these series production scenarios, and many of which have actually already put the stake in the ground and planning for series production with us, which is a huge undertaking altogether between the 2022 through 2025 window.

Austin Russell (10:57): Well, yeah, it's specifically only Tesla and one specific individual at Tesla that's gone the extra step to say that you can try and do full self driving with these systems, which is just really not the case. Again, you can get most of the Radar, but there is a key distinction between having to constantly pay attention, ready to take over the wheel at any given moment, whenever it makes a mistake, versus actually having the driver out of the loop, actually being autonomous, hands off, eyes off, read a book, use your phone, work on your laptop, watch a movie, take a nap, et cetera.

Austin Russell (11:34): So, and that's the key distinct as well. So again, not to say that it doesn't have value. But that's the important part. The hard part is, is going to be the question, if you're selling a system like that for some period of time, and it's not possible, then how do you get yourself out of that position? But that's a different discussion.

Austin Russell (11:55): When it comes to camera systems, I mean, I think you also see something similar with Mobil eye with there system, obviously they have a certain level of capability with the cameras, but once they want to be able to take it to the next level and be able to deliver autonomy, that's where... by the way, if there was any company on the planet that was incentivized to use cameras and only cameras, it would be the camera company that has that 80% market share, would be Mobileye, but they partnered with us to be able to help make this happen from an autonomous perspective.

Austin Russell (12:23): In fact, I just got back from IAA, the automotive conference in Germany, where the CEO of Intel, which owns Mobileye, unveiled on stage their new vehicle that's in part powered by Luminar as well. So, it's been an exciting run. But the industry has a lot more head to develop, and I think one of those things is that it come... and we could talk about this in a bit here.

Austin Russell (12:53): But one of the most important things is focusing also not just on autonomy, but even the fundamental aspects of safety and how we can improve vehicle safety holistically, which does also require LIDAR systems in addition to the autonomous use case, which people have historically thought about LIDAR about.

Alexandr Wang (13:10): Yeah. Cool. One embedded question here is, there's obviously this... you'd mentioned, there's this trade off naturally between the cost of your sensor configuration and the quality of output or maybe how useful that it is for some of the downstream tasks. And even if there's an incredibly cost effective Luminar sensor, then you solve the decision, how many Luminar sensor to put on your vehicle, what angles to put them at, et cetera.

Alexandr Wang (13:37): So, how do you think this sensor choice and the choices you make in managing this trade off, how do you think you should factor those in, especially when you consider a lot of the downstream algorithmic requirements around localization, planning, prediction, and ultimately, like we mentioned the safety of the vehicles?

Austin Russell (13:53): Yeah. No, it is a good question and all this stuff. And I think I heard you mention before, garbage in, garbage out. When it comes down to this, it's the same stuff that applies to labeling is the same stuff that applies to sensing, and interpretation of that sensing and all the AI that goes into it. So, I would say it really just depends on the application.

Austin Russell (14:12): So, for example, for most of the consumer vehicle applications, we're talking about a one forward facing sensor configuration, to be able to enable highway autonomy use cases and also improve vehicle safety. Generally when you're talking about urban scenarios, you're talking about a four sensor configuration around the vehicle to be able to have a 360 degree overlap and redundancy, in that case, each sensor is 120 degree field of view. So it actually, you get a decent field coverage.

Austin Russell (14:40): But when it comes down to it, I would say that there's no question that the whole software side is going to be an absolutely critical part of all of this. I mean, the hardware and the LIDAR is just the sensing platform that enables this to be possible in the first place. But when it comes to the software that's needed to actually be able to do the interpretation of the data, the perception and the controls and planning decision making et cetera, that's that next frontier.

Austin Russell (15:14): And really, I mean, from when it comes to our perspective, we've actually evolved significantly beyond the LIDAR. I mean, that was the foundation because we knew that was the critical problem that was going to enable this industry to really be possible, to happen, to be onto production vehicles.

Austin Russell (15:28): So, we've been focusing just as much on software. I think we actually now have more software engineers than we do hardware engineers on the team, in terms of what we're developing and having great partnerships with something that we just announced with Zenseact, it's Volvo's subsidiary there where they're helping license their stack over to us, to be able to build this holistic software suite and solution.

Austin Russell (15:49): And I think that's something that a lot of companies start from the software side and then realize that, "Okay, well, we need to have hardware that can actually make this possible, that can enable this and hack something together to try and make it work." We approach it from a completely different opposite end of the pyramid of starting with the foundation there, and having this breakthrough technology, and now building those software layers on top.

Austin Russell (16:14): But when you have this quality of data and when you can have this integrated hardware software solution, it really makes all the difference. And that's really what OEMs have been starting to see and why we're already seeing now, not just our hardware, but a software being designed into these production vehicles as well. So, that's the future as much anything, and then continuing to be able to improve functionality via over the year updates.

Austin Russell (16:38): But in the software case, this applies to both, a really a couple of different areas. One is the highway autonomy category, the other is what we're calling proactive safety of improving general vehicle function and safety. And I can dive into that but I want to make sure that we've time to keep running through everything.

Alexandr Wang (16:56): Yeah, no. I mean, I think one of the natural, or one of the questions that many people in the industry ask themselves is, at this point, and I'm really curious to hear your views working with a number of players in the space. What are the biggest bottlenecks towards both widespread L2, L3 systems, as well as successful L4, L5 systems, what are the main technical bottlenecks that are blocking these systems today?

Alexandr Wang (17:24): And how do you think those requirements flow all the way downstream into what are the needs of the sensors? What are the requirements of the sensor suites for these vehicles of the future?

Austin Russell (17:37): Yep. So I would say that from a performance standpoint it's become more and more clear in terms of the level of capability that you need from a sensing perspective of, you really need to be able to see a full 250 meters out reliably for all types of objects, and even the hardest to see types of objects. And do so almost from a ground truth standpoint in terms of what's going on, but let's break it down.

Austin Russell (17:59): There's... let's talk about autonomy first, and that whole side of the house. So there's two distinct categories for autonomy, there's driver in the loop and driver out of the loop. Driver in the loop isn't really autonomous, but people call it that apparently. So, that's where it's the mobilized scenario, the Tesla autopilot scenario, all these different things, where your hands on the wheel, eyes on the road, constantly paying attention, ready to take over.

Austin Russell (18:26): The question is, I mean, doing that in multiple domains is actually not super difficult, the question is, is that how reliable will it be? And the answer is it has to be orders of magnitude are more reliable to be able to actually get the driver out of the loop, to be able to safely take your hands off the wheel, eyes off the road, where it's not going to be constantly running you off the road.

Austin Russell (18:48): And that's the distinction in terms of what you can enable with this sensing system and software. So, that's part of the focus. And a lot of people have been focused on trying to do this from an R&D perspective, for urban environments, you take a look at Waymo, Cruise, Argo, Aurora, Zoox, Motional, all these guys, the historical groove of players for the robotaxi AV world, for example.

Austin Russell (19:13): And the thing is that solving urban autonomy for those driver out of loop scenarios, the L4, L5 scenarios, is just really, really hard, not just from a hardware perspective, you need to have the right hardware to enable it, but the software, even with the Luminar level capability that you have on the vehicles is just... this is not an overnight problem, this is a decade long type journey that people are going to be seeing.

Austin Russell (19:46): And that's the thing, is just identifying all the different edge cases that can happen and being able to react successfully to them in these complex environments and scenarios, isn't something that you can just instantly solve with, I mean, borrowing, unless any of your speakers in this solves general AI in the next year or so. I think we won't be seeing that.

Austin Russell (20:13): So, that's going to be the key, is how can you tackle all these different edge cases, but more specifically, how do we make this as a product and how do we make this industry actually happen sooner than later, where it's not a decade or even decades away. And that's where you constrain the problem, you try and solve a more narrow use case, take for example, highway autonomy.

Austin Russell (20:33): And at the same time trying to go from the level of product majority from a hardware standpoint, says that you can be designed into production vehicles with auto grade hardware. And that's really why we've been the only company that's been developing a proper auto grade system, auto grade hardware, auto grade software, that actually goes into production vehicles and have these partnerships with consumer vehicle OEMs, to ultimately enable this.

Austin Russell (20:59): So, that's the significance. And I think what the focus has to be, if we want to see this happen in more a year or two from now, as opposed to 10 years from now. And it comes down to the software. I mean, obviously the hardware is critical, you have to have the right hardware, that has to be a given in the first place.

Austin Russell (21:19): And I think by the way, just for all practical purposes, I hope that people don't think that, I mean, when you have $100,000 roof rack full of sensing systems and a super computer in the trunk, that's going to be the thing that goes into production, let's not get ourselves, for most of these AV test vehicles out there, it has to be something that's production worthy.

Austin Russell (21:40): So, and that's where you can give credits to the companies like the Mobileye of this world, that actually have put in production of automotive systems. So I'd say that. And then when it comes to ADAS systems, I think the interesting part is that it's a little bit counterintuitive.

Austin Russell (22:01): The focus for ADAS, and everything about it has really all been trying to emulate the feeling of autonomy, without actually being autonomous. It's all about how do you get an L2 type lane keep assist, automated Cruise control system, auto Tesla, autopilot, all these other systems. And I mean, well, it is really interesting as a novelty feature, it's not really holier that you're getting real value out of that at the end of the day, if you still have to be constantly paying attention, ready to take over, and it can even lull you into a false sense of security.

Austin Russell (22:38): The thing that I think is most interesting for ADAS, and part of this is why we've been developing this software, and as far as we you know, we're the only ones actually developing this at all in the first place for production vehicles is something that can actually focus on accident prevention.

Austin Russell (22:55): And right now when you talk about autonomy and the path to autonomy, people always think about it from a scale going from L0 to L5. And people think, "Okay, well, L0 is a solved problem, L1 is solved problem, L2 is solved problem, and apply equal weight into all of these things and try and get to higher levels.

Austin Russell (23:16): The reality is that going back to the basics, like L0, it is not solved at all. I mean, people still get... I mean, 1.5 million people die on a road every year from vehicle accidents, and I think it's like 50 million serious injuries. You would think it would be easy to program your car to not allow you to hit the thing right in front of it, it turns out it's actually really hard, because to take controls over from the driver, you have to have an extreme level of competence if you're going to say, "I'm going to slam on the brakes."

Austin Russell (23:52): Because false positives for those things can be catastrophic in their own right if you're just randomly going full ABS on the freeway for no reason, for example. So that's why there's just such level of sensitivity. And that's the area that we've also been focusing on, and that is something that's unique, but also a driver for why auto makers have started to think about this, not just as a technology that can be an option on a high end vehicle to introduce a new platform, but start to be standard on new vehicles produced.

Austin Russell (24:24): And that's really the break through that's happened, is that this is something that isn't just a new tech that's on a car, this is something that it can have as much of an impact, or even potentially more than the advent of all these modern safety systems like seatbelt and airbags and everything on vehicles, you don't have your seatbelt option upgrade package, it's something that actually comes standard and rightfully so.

Austin Russell (24:53): So that's where the goal is, and what we're starting to do is now be able to go from vehicles that can reduce the severity of accidents with their existing ADAS systems, to now actually starting to prevent the accidents from happening in the first place. And that's what we can do when you have this level of confidence with the LIDAR data, and when you have the software to do it. So yeah, if that makes sense.

Alexandr Wang (25:17): Totally. Well, you bring up a really good point, Austin, which is around, "Hey, the technology that we've built for autonomy, or everywhere from the sensor technology to the perception technology, to the planning, prediction technology, all of this is, there's one world where it's useful as a high end option or within these high end vehicles.

Alexandr Wang (25:40): But it's also a lot of this technology is generally useful for making every single car safer. Which is a good segue into, I think, a topic that's on a lot of people's minds in the industry, which is what is the future of regulation? What is the future of policy? And so, Luminar devices are going to be shipped into production vehicles pretty soon, as you mentioned end of last year, early the year after, and so it's quickly approaching.

Alexandr Wang (26:06): I really love to get your thoughts on where are we today from a autonomy regulation standpoint, and where do you think this goes long term, especially as it's just in the hands of more and more consumers.

Austin Russell (26:17): Yep. No, it's a good question, and it can answer maybe a U.S focus, obviously, ultimately there's international expansion. But I think from the regular side is really interesting because there's a lot of misconceptions about the regulatory side. Historically people view politicians for new technologies, everything it's like, "Oh, they're way behind the curve. They're the limiting factor."

Austin Russell (26:42): People in tech are really good at throwing them out of the bus to be able to just say that's the limiting factor rather than their own tech, in terms of deploying these systems. But the reality is that, this is not actually a super tightly regulated industry in space, despite the perception or common belief that it is, it's actually much more akin to the wild west of everything, where there's very little regulation for a lot of areas than it is to something that's tightly regulated.

Austin Russell (27:09): And that's where there are regulations that are going to need to be introduced. Part of this confusion is drawn in by, like I said, by certain companies or the company that maybe you talked about, had alluded to earlier in the call about, implied that, "Okay, this is the regulatory side, is the limiting factor as opposed to the tech."

Austin Russell (27:32): But when it comes down to it, for example, and what surprises a lot of people in the vast majority of places in the U.S, there's actually nothing preventing you from deploying an autonomous vehicle on the streets and having it ready to go, and having the driver add the loop. There's no legal framing that preventing you from doing that. You can do this.

Austin Russell (27:52): Now, the answer is it actually should be regulated, there should be safety standard, there should be minimum threshold in terms of what you can deploy, and there should be all these things. The reason why people oftentimes think of this when it comes into play, is because again, it goes back to a lot of people think about robotaxi specifically, when you think about autonomous vehicles.

Austin Russell (28:08): What you do have to adhere to is the federal motor vehicle safety standards FMVSS and the playbook there, where the problem is actually, when you start ripping out the steering wheel and breaking systems and side mirrors and all that other stuff. And when you actually replace the driver altogether, that's where you start having issues.

Austin Russell (28:26): But having a driver that's in the car, where it can be autonomous for example, on highway scenarios and having just a much safer vehicle, there's generally no issue with that. Now obviously there's a lot of different constraints that come into play. And I'd say one of the biggest things is for example, that people really haven't thought a lot about is insurance.

Austin Russell (28:46): How do insurance? How do you handle this from an insurance standpoint? What about liability? What about all these other things? These are all the reasons why we've actually gone into even offering our own insurance product associated with our LIDAR and the sales as our software, so we can basically put our money where our mouth is.

Austin Russell (29:02): And that there's actually some significant savings that are realized when you have this system on your car. So, there's a lot of second order of effects that make it really interesting. But yeah, I would say that there's still a lot more work to do from a regulatory perspective where you want to see the level of refinement that's there. But I wouldn't say that's the top of the list in terms of what's needed for just a basic level of deployment, at least not in the U.S.

Austin Russell (29:30): So Alex, maybe a question for you. I'm interested. So, we were just talking about bottlenecks, in a lot of cases, regulatory isn't necessarily one of them, but we were talking about technology bottlenecks earlier. What do you see of some of the biggest bottlenecks coming from your angle when it comes to a data and labeling side?

Austin Russell (29:49): Because I mean, from a data standpoint of, how do you get this volume of data that's labeled, that's going to be required to be able to deploy autonomy at scale? And how do you see that whole space evolving to be able to tackle the holistic challenge and ultimately welcome our robot overlords?

Alexandr Wang (30:06): Well, I think it's a super interesting question, and we've talked about this a little bit, but the volume of data in particular, high definition of lighter data or general autonomy data that's going to be produced, is going to grow many orders of magnitude over the course of the next few years, because all of a sudden, vehicles like the Volvo vehicles with Luminar sensors on them are going to be shipped out in production, and that's going to create these huge volume of data that didn't exist before.

Alexandr Wang (30:37): So, there's this very exciting, I think, trend which is, as you've talked about before, the long tail is really where a lot of the work need to be done to make these systems very, very safe. Well, all of a sudden we're going to have the capability to collect data on that long tail, very efficiently, just through users using these vehicles on production.

Alexandr Wang (31:01): As I think from a data perspective or from a data annotation perspective, a lot of what we think about is, okay, what is the Moore's law, so to speak of this data look like? How do we get significantly more bang for our buck out of the data that is going to go through the system? Every year is a possible process to X the amount of data through traditional machine learning systems, for example.

Alexandr Wang (31:23): And so, that's something we're really excited about, something we think about a lot. The other thing is, and I think is equally exciting, is building machine learning systems and AI systems that are semantically more and more aware and capable and interesting.

Alexandr Wang (31:39): And so, perception has been obviously one of the first problems you need to solve if you want to build an autonomy system, it's pretty clear you need a car to see everything around it for it to make good decisions, but then it's like, "Okay, can these cars meaningfully predict what people around it are going to do??

Alexandr Wang (31:55): So, what is that driver going to do? What is that person going to do? What is that car going to do in the future? And then based on what I think everyone's going to do, what's the action I should take? And the really interesting thing is we've seen through other areas of machine learning, this incredible advancement in the capabilities of prediction.

Alexandr Wang (32:14): One of our other speakers, Ilya Sutskever from Open AI, and they've shown incredible results from these large language models in predicting text. And so we've seen that machine learning systems are intrinsically capable, actually pretty complex prediction tasks, and the application of those same techniques, or the application of these breakthroughs to the machine learning tasks of self-driving is something that I think we're starting to see, and we'll continue to see, is pretty exciting.

Alexandr Wang (32:42): And so, if you think about that in terms of the implications all the way back to the data, it's all about... again, one of the things that's just true about these machine learning systems that we've been developing, is that they're incredibly data hungry.

Alexandr Wang (32:55): And so, how do we, like I mentioned before, how do we just jack up the amount of data through widespread deployment of sensors that are constantly uploading some small percentage of the data that they're processing up to these cloud systems to then train better algorithms? That whole life cycle, I think is something that we're very early on in the stages of ramping. So I'm pretty excited about that.

Austin Russell (33:20): That's awesome.

Alexandr Wang (33:22): This takes me to the-

Austin Russell (33:23): A lot of good perspective.

Alexandr Wang (33:25): Yeah, I do. Well, it takes me to a super question for you, which is around the partnership model between tech players in autonomy and the traditional automobile OEMs. Which is that Luminar in particular, and we've talked about this before, you've taken a very OEM forward approach in autonomy. You partner very closely with the OEMs and you've had pretty successful partnerships with them already, and I expect you to continue to have successful partnerships with them in the future.

Alexandr Wang (33:56): How do you think the industry can use developing in terms of what the OEMs to do, versus what maybe tech or autonomy companies do in the long run? And how do you see that playing out over the the next decade? Lets say.

Austin Russell (34:10): Yep. No, it's interesting. And this is where having the right OEM relationships is one of the most important things you could possibly do as a company in this space. Because at the end of the day, I mean, unless you plan on becoming a car maker yourself. And even then, if you do, it turns out very hard to ramp to millions of vehicles, or tens of billions or hundreds of millions, ultimately as this plays out between all these different modes of transportation and across automakers.

Austin Russell (34:44): They're the gatekeepers to the industry. And OEMs and auto makers, just to be totally upfront, they're some of the most difficult companies on the planet to be able to work with. It's not an easy thing, the organizational structure lend a much greater focus towards everything from commodity management to getting that incremental competitive edge over all the other OEMs out there, working in the same playing field, it's an extremely competitive space.

Austin Russell (35:23): And oftentimes these companies can have tens or hundreds of thousands of people that you have to be able to have on board conceptually to actually be able to see this system through. They're also generally extremely risk averse, they don't like change, and the list goes on and on.

Austin Russell (35:43): That's not to say that it's not important, it's extremely important. And the role they play has obviously been monumental for transportation for the broader world. And it turns out it's really hard to run an automaker and be able to make sure that you can integrate new technologies while also not keeping your eye up the ball and making sure that you still have a profitable business to work with.

Austin Russell (36:04): So that's where really we've come into play, of you have to earn and win the trust of these players. Coming in as an outsider is very difficult as well, but everyone knows they have to be able to advance the technology on these vehicles to make all of this happen. So, while we do work with many tech companies as well, our strong bet is that the OEMs are going to see the ultimate path to realization of this market.

Austin Russell (36:35): And again, it's going to happen more so than a lot of the tech providers out there. And as much as I love Silicon valley and everything that it represent and everything around it, is that trying to be able to own everything holistically, it's not clear that's going to be a successful approach. So the other side of it is also, you can't really develop a technology system in a vacuum and then hand it off to an OEM.

Austin Russell (37:05): You really have to develop it from the ground up with auto grade standards and process in mind to see a technology through, into market. And this is where we've had to basically get some of the best people from automotive Tier 1, from automotive OEMs, from these other areas that have industrialized other types of technologies before, to be able to actually see its realization into marketing and into the industry.

Austin Russell (37:33): And that's one of those things where, I think it has to be a continued focus and you can never take your eye off the ball on, but we're fortunate enough that this was part of the strategy from the beginning to be able to work directly with OEMs to make this happen.

Austin Russell (37:47): And there's so many different considerations when the rubber hits the road on just every aspect of the vehicle. There's the obvious things like cost, there's obvious... but even just from the fundamentals of how do you integrate the technology into the vehicle? Is it capable of running on the same compute system that they have designed into the vehicle? What's the aerodynamic impacts that it has on the vehicle by integrating a technology into it.

Austin Russell (38:18): Even the little details that go into it, how do you clean the system? How do you do all of this? But I'd say the big picture stuff is auto grade process. And the other thing is just making sure that you can deliver and deliver on time. When you talk about... part of the reason why they're so paranoid and difficult to work with, it's not for no reason either.

Austin Russell (38:42): If you end up... I mean, in tech people go through agile development processes all the time where there's some level of flexibility on things, and you're just in a continuous development stream, and yeah you have milestones, yeah you have timelines, yeah you have all this other stuff, but it's not the end of the world if things change around with this.

Austin Russell (38:59): It's much more waterfall oriented with OEMs where, if you miss a production date, I think in some cases the cost of stopping the line can be a million dollars a minute, for every minute you have that line down because of something related to you. So, it's basically like a non-starter for you to be... if you're being designed in, and this is a multi-billion dollar bet that they're making on.

Austin Russell (39:28): So I'd say that's part of the significance. It's not the easiest road that we went down on, it's probably the hardest road from a business standpoint and in customer engagement model and what we've done, but this is also where we're starting to see it pay off, we're at the tip of the iceberg in starting to see a payoff, and this is where we're going to see that exponential growth.

Austin Russell (39:52): When you take a look at number of autonomous vehicles that are out on the road, in terms of all these different approaches and everything right now, I mean, the most any company has, is in 100 of test vehicles. The question is, when is that going to get to millions?

Austin Russell (40:07): Well, now actually we can make that happen in just a handful of years from now, whether there's going to be hundreds of thousands of cars and hopefully millions of cars, because these are production vehicles that are rolling off the line that are powered by Luminar, and then we're actually making that happen.

Austin Russell (40:24): So that was all part of that strategy. And I would say, for these other types of autonomous vehicles that have been out there, I would be surprised if it got to thousands or even tens of thousands of truly driverless functional vehicles, even by the end of the decade. So it was a strategy that I think a lot of peoples would have thought was crazy, just a handful of years from now, but it's really already been paying off.

Alexandr Wang (40:52): Yeah. Totally. Hey, one of the... I think this is highly related to another question I have for you, which is, I think one of the big questions that has always been there for autonomy is, what are the business models that are ultimately going to prevail and ultimately be the most successful? Both Luminar and Scale, we've selected bismals that are more platform oriented in nature where we work with players in building the most of it is the autonomy stocks possible.

Alexandr Wang (41:21): I'm curious, how does that translate to one of the bigger questions of autonomy, which is what are the business models that are ultimately going to be most successful? Luminar and Scale are both platform oriented business models in helping other players build the best possible autonomy stacks. And so, I'm really curious to hear, given your view on the OEMs role, what business models do you think will prevail long-term and ultimately, as well, what are the applications that you think are going to be the most successful? Whether it be delivery, or trucking, or the safety functionality that we mentioned before. I'm really curious.

Austin Russell (41:57): Yep. So, I think from a business model standpoint, there's no question that the platform players are going to see huge value that goes into this. I think for end to end owners, if you're talking about all the way up to the vehicle level, that's going to be really hard, as a building a car company is already hard enough.

Austin Russell (42:16): So trying to build a tech stack on top of that, and in much less a hardware system solid to that, it's going to be a tough road. You have to start somewhere with the right tech and then build your way out. We started with the LIDAR and then we slowly we went to the software, and now we're going to some of the rest of the full stack approach.

Austin Russell (42:36): So you have to be able to start somewhere. But you have to be able to know and make your decisions on build versus buy, and partner versus, just a total ownership, you need to be really, really smart about those, and that's what makes all the difference at the end of the day. But I think that part of the same reason to why a more focused scope is going to really prevail.

Austin Russell (42:58): I think that consumer vehicles, as well as trucks are really going to see the majority of the realization of this industry and volume in over the course of the next decade. I think trucking has definitely... historically has been an overlooked opportunity and it's picked up a lot more recently with a lot of companies, but it's definitely still has a lot left to go, to be able to make that happen.

Austin Russell (43:19): But because of the highway focus of that, and I think there's definitely an opportunity to see it realized sooner than later. But then even within that, there's going to be a question of, do people just sell the trucks or do they actually become an operator themselves of these vehicles? And this stuff becomes really hard to do.

Austin Russell (43:34): So, I think that people are going to have to start small and then work their way up. These are already hard enough problems as it is. But from a customer perspective, it all comes back down to though, do you have a production worthy system? And do you have the partner to put it into production with?

Austin Russell (43:54): Because even if you have the best tech on the planet that you can make it work, if it's not actually going to make its way into the real world, then I'd also question what's the point of seeing it through? Now, there's some level of advantage, and I think, Alex, you have a great company with an advantage of being somewhat insulated from some of that as well, where you get to work with a lot of different key players in the space of the pickax, so to say in the gold rush.

Austin Russell (44:25): But I think there's definitely an element of that to our business model as well. Just for the case of OEMs, in many ways in the same dimension, is that you want to be a partner directly to these OEMs to be able to provide the technology, to provide the system, to make them competitive in this whole next generation of vehicles.

Austin Russell (44:52): But I think being acutely aware of what markets there are opportunities for, is going to be important. There's also obviously a lot of attention on other applications of these kinds of technologies, like LIDAR systems and other things. Can you use it for other industries? And the answer is absolutely yes. I do think there are some real opportunities there.

Austin Russell (45:11): I don't think they're nearly as big as autonomy, over the big picture here, probably by at least an order of magnitude, maybe two orders of magnitude, but there's still some real near-term revenue opportunities and some great business to be had in other adjacent markets.

Austin Russell (45:25): Take, for example, one of the major partnerships that we have is with Airbus, and specifically their up next division that's building the next generation flying platforms that they have, where it's the same autonomous system that you would have for cars, but being applied to helicopters, to planes, to everything for collision avoidance automation and just improving overall safety of these systems, really can make all the difference. And just more accessible generally.

Austin Russell (45:55): So that's where there's definitely some interesting application use outside of automotive as well. But at least our focus... the automotive part is the most challenging part of all. I think what we're going to see, like in the case for example, LIDAR players, we're going to see a lot more try and pivot out of automotive and into these adjacent markets, just because you know how hard it is to build an automotive product, and realistically aren't going to be able to get there.

Austin Russell (46:23): But with that, I think there will be some real opportunities for the adjacent space for that. But the thing with the automotive is that if you do it right, the economies of scale are just unprecedented in terms of what you can do for these breakthrough technologies, and nothing with this level of complexity like in the optics and photonics world has ever been manufactured at these kinds of volumes at scales and why we've had to basically build out our own supply chain as well for this.

Austin Russell (46:54): But there are certain elements of vertical integration that are super helpful. At certain levels and layers, take for example for the LIDAR itself, we actually, one recent bit of news is we just acquired our provider of indium gallium, arsenide chips, the InGaAs that detects that 15, 15 centimeter wailing, the light that we shoot out with our laser.

Austin Russell (47:15): This is a super specialized material, there's only a handful of people in the world that can build this kind of system, they all work at this company, OptoGration, that we ended up buying. And now we control our own supply chain, we control our own destiny with our own fab. There's a global chip shortage going on, but we get to be insulated from all of that, and at the same time can deliver product at the end of the day.

Austin Russell (47:38): So, there's smart things that you can do from both the technology level supply chain level and all of these other things, but it's a balancing act. So, I think making those bill by partner decisions is the most important thing from a strategy standpoint, as a leader, CEO of any company in this space can do.

Alexandr Wang (48:06): Yeah. Awesome. Where I wanted to close, I think one thing that's just super exciting is that the, again, as we go back to thinking about the Luminar sensors are going to be on Volvo vehicles, and it'll be the first larger scale production deployment of modern LIDAR technology, which is incredibly exciting.

Alexandr Wang (48:26): And so, and in particular, Volvo is notable because long ago there were the first company to install the three-point seatbelt, which at the time there's a lot of public debate around the safety feature and whether or not it was necessary. But since then it's been incredible with saving over a million lives.

Austin Russell (48:46): Right.

Alexandr Wang (48:46): And so, with Luminar being selected by Volvo GB, the lighter sensor for their next generation vehicles, it's an incredibly exciting, I think moment for not only LIDARS and autonomy, but also safety of automotive vehicles. And so, to take it all the way through to the consumer, how would you speak to a potential car buyer of a vehicle, let's say, who's considering buying a vehicle at the end of 2022?

Alexandr Wang (49:13): Why should the general public want to buy a vehicle with Luminar sensor, as opposed to one that maybe doesn't have a Luminar sensor, has other sensor technologies, and what does it mean in terms of safety, convenience for them and their families?

Austin Russell (49:28): Yup. So, no, it's a great question. I think this is the first time also where we're seeing this kind of technology actually reach the end consumer at the end of the day. Most of the time with automakers, it's white labeled systems and technologies, you take a look in a vehicle and you never really know who's providing what tech for what in the vast majority of the time.

Austin Russell (49:47): I think it's only in recent history with Android auto or Apple car play or these kinds of things that are being introduced where you actually start to see some level of branding of the technology. But when it comes down to it, I think we're at the same playing field here when it comes to autonomous systems. And the value proposition that's being provided when you have a car that's powered by Luminar is really just in two dimensions.

Austin Russell (50:10): Of course, the golden question is, what does it mean to the end consumer at the end of the day? Having this stuff is cool, but cool is novelty cool, doesn't provide a great business over decade that's going to be a multi hundred billion dollar company or more. So, what that means is there's really two dimensions that we're focused on what people are starting to be able to get over the coming few years here with vehicles being introduced with Luminar.

Austin Russell (50:44): One is safety. It's all about that. There's an opportunity to dramatically reduce the number of accidents out there by actually preventing the accident from happening in the first place. If it senses that you're going to get into an accident, it will start to take over the controls of the vehicles, steering wheel, breaking systems, everything. Much more reliably in consistently than you would have with a camera based system, which is on vehicle.

Austin Russell (51:08): So there's really never been anything like this that's been introduced into the market that's going to actually start preventing accidents. Most of these systems are really meant to try and reduce the severity of accidents, but that's where collision prevention is going to be key. And you have folks like Volvo going and stating, "We have a vision of zero collisions ultimately on vehicles."

Austin Russell (51:26): Basically a vision towards building the uncrushable car. And it's only going to get better and better with software updates over time, as you own the vehicle. The second aspect is time savings. What does autonomy mean for a consumer owned vehicle? It means that you can save time. And if you have a long commute, if you have whatever it may be, you're actually able to, whether you're working in the car or whether you're going to sleep, you can recover some of that time.

Austin Russell (51:57): And that's what makes a difference versus a monotonous driving task. So, again, the focus for that is really exit to exit on a highway scenarios, and that's what will be the initial operating domain for deployment, and then ultimately expand there on out. But those are really the two key value propositions.

Austin Russell (52:16): And I mean, even independent of the autonomy, there's a very clear business case and value proposition, even just for the safety side. That's why the safety aspect will be standard on all vehicles, for example, in the Volvo case. Like I said, it was a breakthrough outright in the industry when it was first included as an option, but to have it be standard, I don't think anybody saw that coming. So, that's something that provides a great case.

Austin Russell (52:38): But now you also have an upgrade option opportunity to get the autonomous capabilities and get time-saving capabilities on a vehicle with the same hardware. It's just a software flick of the switch. And that's part of the value proposition that can be had on these vehicles. So, yeah, couldn't be more excited about the future for this, and what's being introduced.

Austin Russell (52:59): I'm not sure people have really taken in the full impact of what all of this means. I mean, it's pretty incredible how far things have come along. And I think there's some level of disillusionment in autonomy of how long is this really going to take? Is this really going to happen? Are we going to see these vehicles out on our road? We've been in testing world for, I don't know how many years, and how many more years are we going to be stuck in this purgatory?

Austin Russell (53:23): And that's the whole goal that we're solving for, breakout of that loop. Let's make this happen, let's get real cars out on the road, that have this capability, and change the world. So walk the walk, not just talk the talk

Alexandr Wang (53:36): Super exciting. Awesome. Well, thank you so much again for sitting down with us and I'm super excited to see the Luminar sensors out on Volvo and other vehicles starting in 2022.

Austin Russell (53:47): Awesome. Thanks for having me, Alex. Great. Good to see you and best of luck with the rest of the conference.