A small part of the Apple iOS 12 announcement got me thinking: Live Listen.

I love spotting little pieces of new technology that can profoundly affect the evolution of human culture: I call them Seeds of Discontinuity. For example, I vividly remember seeing a flip phone in 1999 that had an embedded camera.  I had never before considered it possible for a camera to be part of a phone, but, at that moment, I knew the world of photography would change forever.  Of course, the capabilities of that phone’s camera were far less than what was then available from a standalone camera.  However, always having a camera in your pocket meant always being able to capture an image. This was extremely powerful.  Once photography crossed into the digital realm, attributes like resolution would ride Moore’s Law, improving exponentially.  All this was clear to me in that moment of insight, though the now-commonplace social networking aspects of photography were not apparent to me at that time. 

GPS (Global Positioning System) is another example of a complementary technology that has led to profound results. The GPS project was announced in 1973 for use by the U.S. military.  While it did not become fully operational until 1995, civilian use started in the 1980s.  The iPhone of 2008 iPhone (just ten years ago!) provided positioning information for the first time by triangulating cell towers.  It was the iPhone 4S (Oct. 2011) that first supported GPS.  I wrote a blog post in 2008 about GPS. contemplating some of the effects of GPS, but that was before its integration with the cell phone.  The integration of GPS and the smartphone led directly to companies like Uber and Waze.  Positional information is now prominent in our interactions on the internet, including interactions through social media.  For instance, our photos are now tagged with location data.   If we choose, we can let our location be known to our friends on social media.  We can find information about stores and restaurants that might be near us.  We can even find our devices if we lose them.

Spotting the Seeds of Discontinuity

My ability to spot the Seeds of Discontinuity and understand at least some of their potential impacts has not just been my hobby.  It is also largely how I made my living, especially when I co-founded Intel Capital along with Les Vadasz.  For reasons I still do not fully understand, I have been pretty good at this.  Frankly, I don’t know if I am still good at it, because it takes many years to learn whether your predictions were correct!

    

Expanding the range of human communication

Before the nineteenth-century inventions of the telephone and the telegraph, humans could only converse with each other within speaking distance.  Now, almost any human being can talk to any other human being in real time, no matter where they are located.  Even hundreds of people can communicate with each other in real time over audio, from anywhere on the planet.

In 1965, when I was just twenty years old, I lived in Paris.  I could call my mother in San Francisco, but it cost a small fortune, and I had to use a phone booth.  I also had to tell the operator if the call would be Person to Person or Station to Station.  Person to Person was more expensive. While Station to Station meant you would speak to anyone, the operator found on the other end of the call.  To save money, I worked out a protocol with my mom.  I would call Person to Person and ask for my mother, Marilyn Miller.  She would pick up and tell the operator that Marilyn Miller was not there and that I should call back in an hour.  That was the code telling me she was there and would answer the next call.  Then I would call her Station to Station.

Later, I developed a way to bypass payment.  In those days, an operator knew you had put in your money by listening to tones generated by the phone as you deposited your coins.   Different coins had different tones.  I bought one of the first portable tape recorders and recorded the sounds of the coins dropping.  So, when the operator said, “deposit 50 francs,” I played the recording of the tones adding up to 50 francs and had a pleasant conversation with my mom.

Today, with applications like Skype, FaceTime, WhatsApp, and WeChat, we can speak to one or more people for as long as we like, wherever they and we may be.  I believe that the next evolution will seem like continuous communication, but more about that later. A whole generation is growing up that has never known anything else.

Until very recently, two people could not see each other unless they were physically close together.  That, too, has changed.  While less prevalent than voice communication, real-time video communications between two or more people located anywhere on the planet are now common.  I routinely have video conferences with as many as ten people found on multiple continents.  While we all know where we are each situated personally, and while we may also be aware of where the others are located, the virtual location of the conference has no physical counterpart. 

At some point, we will be able to experience the sensation of touching others, no matter where they are.

My early experiences with computing and audio

In 1967, the year I learned to program computers, I had my first experience with computing and audio.  I was providing technical support to Dr. Joe Kamiya in his research on brainwave bio-feedback at the Langley Porter Institute at the School of Medicine, University of California San Francisco.  I worked closely with another programmer, Pete Harris, on several projects.  We used to program late at night when the computer, a PDP-7 from Digital Equipment Corp., was not busy being used for experiments.  We noticed that the console lights on the computer’s control panel would interfere with our transistor radio. Then, one of us got the idea to program the lights to generate sounds.  Next, we were able to get the computer to play songs.  (We later got the tape drives to dance, but that is another story!)  Since I was also a pianist, I took a continued interest in the use of computers in music. 

The end of time and space

In 2006, FastCompany Magazine asked “ten of their favorite brains” to describe the most important change that would affect business in the coming ten years     (2006–2016).  I was one of those “brains,”  and I was amongst excellent company, such as Malcolm Gladwell (author), Kevin Roberts (CEO of Saatchi and Saatchi), John Mackey (CEO of Whole Foods), and Ester Dyson (Host of PC Forums).  I identified the most important trend as “the end of time and space.”  It’s fun to see what everyone had to say since we are now past the predicted ten years and can see what has happened.  Indeed, as I predicted, it no longer matters where you are or even what time it is where you are, as we have moved more and more of our communication onto the internet. 

 

Back to Live Listen, the iOS feature that had me thinking

The iPhone has a capability Apple calls Live Listen, which allows specific hearing aids to “pair” with an iPhone and use the iPhone’s more powerful microphones.  So, instead of using the hearing aid’s microphone and its speaker, the hearing aid acts as earbuds alone, allowing the microphone to be placed at some distance from the listener.  At the very end of 2016, Apple released the AirPods, a set of Bluetooth earbuds.  I think they are a great product, and I use mine daily.  They seem very popular: by the end of 2018, worldwide sales are expected to exceed 50 million. AirPods can be connected to the iPhone, iPad, Macs, Apple Watches, and Apple TVs, or almost any Bluetooth device, although many advanced features work only with Apple devices.

Someone at Apple had the idea to make Live Listen available to anyone with AirPods. I started to think about the implications of decoupling the microphone from the speaker.

Conversation Peace

About eight or nine years ago, I had the idea to create a product called the “Conversation Peace.”  I often find myself in noisy restaurants.  It can be frustrating to go out for a meal with friends or business associates and try to have a conversation.  I have a decibel (loudness) meter as an app on my phone, and it’s shocking how loud some restaurants are.  My concept was a tabletop device that could make noise cancellation to create a “cone of silence.”  Of course, while creating such a product might have been possible, it would have been too large and too expensive to be practical.

Eventually, I realized that everyone at a restaurant table could use their phones and earphones to call into a conference line.  I never actually did this, but I have tried calling individual people at a table so we could converse.

I think if I were out at a restaurant with a group of friends that all had AirPods, I would certainly consider trying Live Listen to cope with the noise.  Apple will even have some presets audio filters available with Live Listen.  One of them is called “Restaurant.” 

Networking conversation

I then started thinking about all the capabilities that networking could add.  There is no reason to limit me to one microphone at one location. For instance, instead of using my iPhone at a restaurant, to listen to you, I could also connect to your iPhone’s microphone.  In fact, all the iPhones on the table could link to some audio mesh network, creating fantastic fidelity.  Just think about how this capability could be applied at a concert. 

But there is no reason the “microphone” must be physically near.  One could place a microphone in a child’s room: if the child cries for longer than some set period, the sound of crying could be mixed into one’s audio. I could have a conversation with you while my assistant, located halfway around the world, could “whisper in my ear.”

Of course, voice control, as we already have with Alexa and Siri, is an integral part of such applications. One powerful capability would be to couple the microphone to highly intelligent reminder applications.  So, if I said to you, “let’s meet on Saturday for lunch,” a reminder would be captured on my calendar. The microphone need not be in your ear to do this.

Augmented reality will take a lot longer

Perhaps I should mention augmented reality here.  I first became involved in augmented reality in 1999, as an advisor to a company called Microvision.  Driving my interest in augmented reality was my sense that smartphone screens would be too small for many critical mobile applications.  I also believed that voice could better handle input instead. I was right about the problem, but I was way off on the timing.  (I continue to be somewhat involved with the field, having worked with http://www.emacula.io/ for more than 15 years.)  Screens have very significant constraints due to physics and physiology that are difficult to overcome.  Audio has very different limitations and possibilities, and I believe we will see an explosion of audio applications that will alter the way humans communicate with each other and with the technology that serves them.

The future of audio communication

In 1999, when I first saw a phone that was also a camera, it would have been difficult for me to imagine that almost everyone would have a phone/camera in their pockets today.  But this has happened.  Right now, it’s hard to believe that nearly everyone will have something in their ear all the time, but I am willing to take that bet.  I just might not be around long enough to collect.