As Toyota Research Institute’s CTO looks around corners, Dr. Kuffner and his $1B+ organization are busy building intelligent machines of the future. What does the world’s largest automaker see in the road ahead? How fast and how far are we now traveling? What’s the urban mobility future? As ride-sharing evolves, and emerging tech creates new space for innovation, can we speed up the transition to a more sustainable city? Considering the massive allocations of scarce urban land to parking, when and where can cities make their move into the new world of safer and cleaner mobility?


Download slides


Gordon Feller:

Life and death issues, whether it’s the climate resiliency challenge that new vehicles are going to have to help us address and overcome, or the life and death on the safety of the roadways. This is really why somebody like Toyota has stepped up to create a startup, a different kind of startup than these two startups. The Toyota Research Institute is based here in Palo Alto with three other offices around the world, led by an amazing team of people, like Dr. James Kuffner who’s coming up in a moment. Dr. Kuffner came out of the Stanford University, PhD Computer Science program, and is the father of something that almost none of us have ever heard of called Cloud Robotics, that I hope we’ll learn a little bit about in a few minutes. But really is helping as Chief Technology Officer at Toyota Research Institute to guide us into the world of both assisted technology, where the machine and the human being are engaged with each other and the fully autonomous future. So, thank you for joining us, Dr. Kuffner. The floor is yours. Here’s your device…

Dr. James Kuffner:

Well, it’s a great pleasure to be speaking with this audience this morning. I’d like to tell you a little bit about some of the work that we’re doing at the Toyota Research Institute. For those who don’t know about us, we were founded about nine months ago, and our CEO, Dr. Gill Pratt, was a DARPA program manager and also an MIT professor and our goal is really to partner with academic institutions in the field, as well as our industrial partners and the Toyota family of companies, and also with government to help bring in new technologies related to advanced safety and development.

Our key focus areas are, of course, automotive safety and autonomy, but we’re also looking at exploring new forms of mobility, and some of you may get to ride the i-Road outside but also in the robotics space. And other philosophy of our development is to leverage modern, state-of-the-art Cloud Computing and software tools to explore new forms of energy as well as battery technology and materials that could help transform the next generation of mobility solutions.

So a lot of Silicon Valley companies talk about Moonshot Research. I was part of Google for seven years, and the focus is really on trying to find technology leaps. And how to do that is still, I think, an unsolved problem, but we know some of the ingredients. And let’s just take, for example, a topic that of course myself and many of us at Toyota, care deeply about is the historical precedence of the car. So in 1885, the first gas-powered engines. Of course, they tended to explode and were not very safe and were very expensive, but over the next 20 to 30 years, there was incredible advancement in transmission designs and of course, manufacturing technology that really brought the cost down to the point where it was possible for people to personally own the vehicles. And of course, that absolutely changed the world. In fact, all of our cities are designed around cars. Many people have said that if aliens observed our planet from afar, they would say that it is populated by metal beings with four wheels and every once in awhile, some carbon-based prototypes would enter and exit these metal-based vehicles.

Of course, another historical precedent is the computer. In 1945, we had very expensive, kind of unreliable vacuum tube-based computers, but over the next 20 to 30 years, you saw rapid advances in hardware, transistor storage and displays. And then of course, in the early 80’s, personal computers became widely affordable and that absolutely transformed our society.

And then of course, there’s the transformative shift that all of us have lived through, which is the mobile phone. So back in 1983, if you had $4000.00, you could buy this pretty big brick that made occasionally good phone calls. But of course, over the next 20 to 30 years, incredible advances in wireless transmission speeds, reliability, battery technology, led to the development of smart phones. And of course smart phones have now overtaken PC Desktops worldwide. And in fact, what’s truly incredible, is that most likely the phone that you carry in your pocket, is more powerful than the most powerful super computer of 20 years ago.

So what is next? And we think that the confluence of the car, the computer, and connectivity, and data in the Cloud, is going to cause a huge shift. And in fact, Cloud-connected intelligent vehicles and robots, we just heard from the panel about the vehicle side. We also think intelligent robots, powered by machine-learning and A.I. breakthroughs that have been happening in the last decade are going to be a big thing. And so the Toyota Research Institute is really working hard to try and help explore this area.

So when we think about Moonshot Research and when we think about what are the ingredients for disruption? I think that it comes down to strong partnerships between academia, industry, and government, plus the presence of a critical mass of people and capital in order to realize it. So for example, the DARPA Grand Challenge as a program where for two decades, the DARPA invested in autonomy for vehicles and really kind of had mediocre results. But instead of just funding individual contracts, they sort of flipped it around and said we’re going to have a prize bounty, and we’re going to encourage anybody who thinks they have a great idea to come and enter a competition and try and build a car that could navigate successfully and autonomously through a desert track. And the first year it was offered, there was no winner. The entry from Carnegie Mellon, the university that I was teaching at the time, went the furthest, but it was only about seven miles, of course. But within twelve months, several cars would finish the race including CMU and Stanford’s car. And then two years later, they upped the bounty and had it handle urban environments. And then of course, there were many cars that finished; Carnegie Mellon was the winner. And that led to the Google self-driving car project. You could see me sitting in one of the Toyota Lexus that had been outfitted with Google software. And I helped sort of build the initial prototype. And it meant that at that time, that technology was proven enough that companies started to invest in it. And what has been the waterfall in the last seven years, is now there’s probably, this was in August, there’s probably many more now of companies that are working really hard now to bring autonomous vehicles. So this incredible activity in the space is truly exciting.

And so what does all this mean for our cities? And so, let’s speculate a little bit on how intelligent vehicle technology could impact the design of future cities. So let’s say you were in a world where you could have driverless cities. That means that most of the transportation will be on demand, mobility as a service. That could lead to dramatic reduction in overall traffic, noise, and pollution, as you have ride-share vehicles that can be, that are fleet-managed, that are zero-emissions, eco-friendly, and of course, the utilization of the car is so much higher that you will end up with a reduction of traffic and noise and pollution.

Also, as was mentioned before, if most of the vehicles are not parked all the time, but actually moving around, and utilization is higher, you should be able to convert a lot of land that’s currently dedicated to parking lots into useful residential and commercial areas. This is a problem that the Bay Area faces a lot in that there is no more space left to develop more housing; in San Francisco, of course, the desirable place to live. But what if you had cars that no longer needed to be parked on the curbside and they could park offsite?

And so, let’s think about today’s reality. So right now, when we just think about the parking, the average car is parked about 95% of the time, in the U.S. So only 5% utilization of the car. Worldwide, the estimate is that urban drivers spend an average of 20 minutes per trip actually looking for a parking spot. And this actually is in contrast or is a seemingly contradiction to the fact that the United States has over a billion parking spots for only 253 million cars. So there’s four times more parking spaces than there are vehicles. So how can we better utilize these under-utilized resources?

And another study showed in the County of Los Angeles, there’s over 200 square miles of land dedicated to parking. And there’s 18.6 million spaces for about 5 million cars. So again, it’s roughly 4x, 3.5x. And that’s 14% of all the land area in Los Angeles County.

So let’s think about what happens if an autonomous car can actually drop you off at work downtown and then drive away to an offsite parking garage. So that means that parking structures can be located away from urban centers. There’s no need to have a giant parking garage in very valuable real estate in the downtown area. That means you can also have more dense, efficient, packed parking that are managed robotically. That means that you don’t need parking structures that have stairs and elevators and wide alleyways for vehicle access. You can really, more efficiently, utilize the space.

And the system, if it is connected to all the cars, it understands sort of from a machine-learning point of view, modeling traffic flows and demand curves for a population that is mostly using on-demand mobility as a service systems, that means you can do dynamic load balancing the same way that internet routers load balance traffic and packets of information between different servers. You could actually dynamically load balance the traffic and vehicle flows according to the demand. And even proactively start dispatching cars from the remote garage in order to meet demand and have very low latency.

Let’s also think about what else parking lots can do if all of the vehicles are now going towards a robotically-managed, centrally-located location for parking, they could also double as charging stations. Then suddenly, if every time the vehicle is parking it automatically recharges itself or refuels itself, you can reclaim the land that is currently utilized for gas stations. People then potentially charge their cars at home or while the cars are parked. If they’re personally owned, they charge at home. If they’re fleet-managed vehicles, they charge themselves overnight in the parking structures. That also means that potentially, car services such as cleaning, repairs, safety-checks, and maintenance, can all be performed at the parking lot facilities in a central way and be much more efficient.

You could also imagine that there’s so many delivery vehicles currently on our roads today – FedEx, the mail, everyone is getting packages delivered directly to their home. But what if the parking lot was smart enough and became a logistics hub. And suddenly, when your on-demand ride-sharing car picks you up from work when you get off work downtown, loaded in the backseat of your car is all the packages you ordered, and you take it home.

So let’s think about urban center redesign. What would be the far future if all the parking lots and gas stations were able to be converted to green spaces, how would that change our cities? I think many of us would agree that would be wonderful. And in fact, a lot of the curbside parking spaces that are currently utilized because people want to have cars parked close to where they work, could be removed because a car drops you off and then it self-parks at a parking facility far away. That means all that space that’s dedicated to parking along the curbs could either add another lane of traffic or convert those lanes to nice bike lanes or expanded sidewalks. And so suddenly, we’ve not only improved the reclamation of retail, commercial, and residential space in our urban centers, but we’re also increasing throughput and encouraging more eco-friendly forms of transportation such as bicycles and pedestrians.

And then you can really think about the far future. It was mentioned that our cities have currently been designed around traditional cars. But intelligent vehicles will enable new forms of design. What if you moved all of the vehicle traffic underground? And suddenly all of the above-ground streets that are currently used for cars, can be converted to the Fusgangerzone. I love this tradition in Europe to have pedestrian-only zones in the central part of the city. It’s very quiet, there’s better air quality, and of course, it’s much safer. In the United States, the number one cause of traffic fatalities is lane departures – cars going off the road or crossing the yellow line into oncoming traffic. In Japan, it is pedestrian fatalities. And that’s because the dense urban centers in Japan and lots of the world, not just Asia and Europe, means that there’s very narrow sidewalks or non-existent sidewalks and there’s very heavy pedestrian and bicycle traffic in the urban centers. If suddenly all the cars were moved underground, then you have a much quieter, cleaner, safer urban center. And it doesn’t even have to be that you have to dig these giant tunnels. You could do what Disneyland did, which is Disneyland, of course as many of you know, has an incredible network of underground tunnels so that Mickey Mouse can go from one place to the other and be in the park at all different times. So they have incredible underground logistics network supporting the park but it was actually built on ground-level and they built the park on top of level one. That was because the water table was too low in Anaheim. We could absolutely do the same thing. And in fact, I heard that the city of Amsterdam is now moving all of its roadways to under its canals. So that means that they’re digging new roadways to go underneath the existing waterways in the city, and of course the construction then doesn’t disrupt the existing roadways, but all the cars will then be moved to these new underground roadways that run under the waterways. And then those existing roadways can be converted to pedestrian zones and bicycle zones; the Netherlands already being an incredible place if you’re a bicyclist.

So I think this incredible potential will obviously require the cooperation of governments, forward-thinking industry leaders, and of course, people in academia who were thinking hard about these problems and being able to do research about it. But I think if the society comes together and makes this a reality, we could dramatically improve the quality of life for millions of people around the world. As the urban areas are now becoming the place where people live and rural areas are actually losing population. So when we think about a lot of these efforts, there’s many ongoing broad initiatives. I have mentioned a couple here of organizations or private companies and governments that are trying to push things forward and trying to think about some of these issues. I also want to highlight what Toyota is doing about it so, some of you have already taken a ride, but if you haven’t, you can sign up at the front door to take a ride of a concept vehicle that Toyota is already doing test programs in Grenoble, France and also Toyota City in Japan. But it’s a very, very cool, zero-emissions electric vehicle with compact three wheel design. It has active-balancing and stability control, so very, very fun to drive. Strong and lightweight and it has batteries that average 50 kilometers on a single charge. It is also covered so that unlike a motorcycle where you get wet when it rains, you can stay dry, you can have climate control, and you don’t need a helmet. So please try it out. It’s really, really nice. One of the cool things about this vehicle is that you can fit four of these in a traditional vehicle parking space. So you can imagine the incredible density that could be achieved through a mobility program like this. There’s versions of the car that can support two passengers, but the one outside that you can test drive has the single passenger version. And the sad fact is that in California at any given time, 93% of the cars on the road have a single passenger in them. And so I think there’s a lot of opportunity to dramatically increase the existing roadway infrastructure to have many, many more vehicles. Effectively, you could have twice the number of lanes if all the vehicles were this size.

Toyota is also investing in true zero-emissions future for transportation. You can buy today, a Toyota Mirai, a hydrogen fuel-cell vehicle that was brought to market, and it has 312 EPA-rated miles per tank and it only takes five minutes to refuel. And wheel to well is one of the most environmentally-friendly solutions you can build. It essentially uses the hybrid synergy drive which, of course, became very popular with our Prius line and our hybrid models, but instead of the gas engine, it uses a hydrogen fuel cell stack and the only emissions is water vapor. In fact, it’s very cool, I’ve test driven these; they are wonderful cars. You can push the H2O button and water will come out of the tail pipe, so if you’re stuck in the desert maybe you can get some water out of your car. The incredible thing about it though is that the car itself, with the fuel cell stack and a tank of hydrogen, could power an average home for about a week. So if you, for some reason, are out of power, you could just hook up your home to your car and you have a hydrogen fuel cell stacker that powers your home for a week. That’s truly off the grid. Only takes five minutes to refuel; you can buy it today. There’s incredible tax incentives because it is so wonderful for the environment. Of course there’s lots of infrastructure that would need to be built to support this, but if a lot of future mobility solutions are moving towards centrally managed fleets, this actually is very synergistic. The parking garage would simply have a hydrogen fuel station that would then refuel all the cars. So try this out, if you haven’t test drove a Mirai, it’s really, really fun.

So when we think about Connected Cars and the future, we talked a lot about machine learning and data and I think what’s really the promise here in terms of software, if you think about big data and Deep Learning, a lot of the big tech companies – my former at Google has open-sourced TensorFlow, one of the state-of-the-art algorithms for machine learning – and many other open source solutions now exist for essentially extremely powerful machine learning algorithms like Caffe or Torch or Theano. And that means that anyone can use the state-of-the-art algorithms. What is valuable then, is the data. And so the data collected from these connected cars is going to be really, really powerful. Of course, it’s already demonstrated incredible advances in natural language and understanding and speech recognition on our mobile devices. And also we’ve now demonstrated through Imagenet and some of our collaborators at Stanford have built a project over several years to demonstrate object recognition and Semantic Labeling that is as good, or better, than a human.

So Connected Cars will gather novel data. They’ll be able to upload new exemplars to a training set and then in the Cloud, we can train these new models and then broadcast updates so that the entire fleet can download a much more reliable and proficient perception system. And that means that recognizing cars and pedestrians can be crowd sourced and then that data is going to enable a much more reliable next generation of these intelligent vehicles. So TRI, the Toyota Research Institute, is working hard on this. We just got started in January of this year and we now have over 150 employees, we’ve opened three offices, and we would love to hire anybody else who’s interested in working with us.

I want to close with just an example of how a Connected Car can maintain something that a lot of companies are investing in right now and that is maps of our road infrastructure. Some companies are spending billions of dollars to collect data about all of the roads that are in the world. Of course, as soon as you dispatch a car equipped with sensors and you collect that data, that data is immediately out of date and it is stale. And so maintaining the freshest of the data in the map is very challenging given the fact that every day, lane markings get repainted, there’s road construction, new roads get built. And so this is something that was demonstrated at this year’s CES, which is using the imagery from the cameras that are built into a lot of back up cameras and forward cameras on existing vehicles, and using a comparing the imagery collected of the road surface with a prior map that can be loaded on each car. And so, as the car is driving on the road, it can notice street signs; it can notice road signs; it can notice speed markings, lane markings. And then when it notices a difference between its prior map and what it just collected, it can then send that differential information up to the Cloud, where the Cloud can reassemble and stitch together dynamically, a map of the entire road driving surface. This is a truly scalable way of maintaining a fresh map of the world using the existing vehicles that are already out there. Toyota sells 10 million vehicles a year, each vehicle lasts about 10 years. That means that at any given time, there’s 100 million Toyota vehicles driving on the road. And we think that that should be the way that we can crowd source and gather information. But not only about the drivable surfaces and weather conditions and traffic patterns; this will enable a new era of really managing traffic and managing our cities on a much more scale that will be very productive and really, really exciting.

So with that, I’d like to just close with a couple of thoughts. So I am very excited about the next generation of autonomous vehicles that I think will enable all of us to rethink how our urban centers are designed to build. I also think that Cloud Computing, big data, machine learning, and the ubiquitous connectivity that is coming will dramatically advance not only the capability of our vehicle autonomy but also the safety and the access that comes along with it. People who traditionally have not had access to cars such as people who are visually impaired or blind can suddenly mobility, freedom, and transportation on demand and have privacy at the same time. And it’ll be truly wonderful. And of course, I think, in order to realize this, all of us in this room and many people who are passionate about this need to come together. It’ll require cooperation and partnerships between the government and the industry and our academic partners. We think that is key to having successful solutions. So thank you for your time. I’d be happy to take a couple of questions if there’s any. Thank you.


Good morning and thank you, I’m Gil Friend, Chief Sustainability Officer from Palo Alto. And we’re right with you on most of what you’ve talked about. I want to quibble with you about hydrogen later on. But for now, about autonomous vehicles, we were just awarded a million dollar grant from the Federal Mobility Sandbox, (unclear 26:42) coalition to pursue some of these ideas here in the Bay Area. And one thing that we really don’t understand well is autonomous vehicles. You suggest that it may reduce congestion. I think that’s a logical assumption, but there’s also people who assume that will actually increase congestion because it’ll be so frictionless. Just, you know, call a robotic car, hop into it, go somewhere; there may be more cars on the road all the time. How do you weigh those futures and how do you do the analysis in trying to figure out where we’re going with that?

Dr. Kuffner:

Yeah so, that’s a very, very good question. And there’s lots of different models that, depending on the different assumptions that are made, will show different results. But I think the fact is that if you really look at utilization, utilization of vehicles and utilization of seats, you can actually meet the needs of point-to-point transportation for many, many people with proper ride-sharing and carpooling of these on-demand services. That will dramatically reduce the number of cars. The fact is that most places, and Palo Alto’s also in that camp, that over 90% of the cars on the road during the peak commute times have only a single passenger in them. And I think just that alone, is going to reduce the number of vehicles during our peak traffic times. Better utilization, having full seats. A lot of people are going the same place, and they’re parking their vehicles in a downtown center. If we can get rid of all the curbside parking and we add more lanes, it’s going to dramatically increase traffic throughput. So I’m very optimistic that with the right set of assumptions, you can easily see that this is going to be a big win.


Hi there. (unclear 28:26). Very exciting possibilities that you presented. I do want to take, sort of, issue or question with one of them, which is that the best place for vehicles is going to be underground. I think this is kind of an idea we explored in the 1930’s and actually cities have been digging out from that idea and trying to create streets that work for people and really mix people together and kind of optimize that friction, optimize the places that people have to be together, and I wonder if that, if you think that, the right idea is to really put more vehicles on the road to increase lanes and increase capacity or to right size it so we can mix traffic together with the more vital public realm.

Dr. Kuffner:

That’s a very, very good question. For me, at least in my personal experience, walking down pedestrian zones, pedestrian only zones, or when your downtown area gets closed on the weekends and you can just walk around, it’s just so much more pleasant and it’s so much safer. And if, you know, it used to be that vehicles underground had a big problem, just like Boston’s Big Dig. They had to actually invest in this super expensive HVAC system to suck all the pollution that could congregate in the tunnels. But if we’re going to zero emissions vehicles, that is no longer a problem and if everything is electric, that means there is no longer a problem of smog accumulating underground. And instead, you can really have cars going point-to-point and dropping you off right beneath the place where you need to go and you take a stair or elevator up and then you’re above ground and it’s quiet and it’s safe and it’s pollution-free. I’m actually really (unclear 30:02) on it. Of course, people have tried it in the 70’s, but I think in many ways it was ahead of its time.

Alright, I guess I’m out of time. But thank you so much.


Share This