I think one of my biggest concerns is the ever-increasing complexity of the computer systems required in a self driving car. Seamless integration of hundreds of sensors and the raw computing power required to process the live streams of data while making split second adjustments.
Think of it this way; when was the last you purchased an operating system for your computer like Windows, or even just a single application running on your computer that didn't constantly download updates for security patches or bug fixes? Even the apps on your smart phones require constant updates because a released version of software is never perfect.
I recently attended NXP Technology days in Detroit a couple weeks ago. Super impressed with the new products and the capabilities but at the end of the day there is still a human writing the code that quite literally "drives" the hardware.
I think in some ways self driving cars will increase highway safety because it will provide for faster reaction times etc. but on the other side of the coin you have to keep in mind vehicle manufacturers hit the reset button every year to come out with a new upgraded model, not necessarily always taking the time to make sure its right the first time, just right now. If a vehicle manufacturer, any of the big ones, would take a model they have if production and extend the lifecycle by 1 year, their tooling costs would be fully recovered, they could concentrate on improvements to the current offering and sell a year's worth of production at increased margins, or even lower the initial retail price... Hey, there's a novel idea.
Oh well, no one has commented on this thread up till now so maybe I'm just pontificating into the wind but sometimes it feels good to rant.