The vast majority of people in developed countries now carry a smartphone everywhere.

And while many of us are already well aware of privacy issues associated with smartphones, like their ability to track our movements or even take surreptitious photos, an increasing number of people are starting to worry that their smartphone is actually listening to everything they say.

There might not be much evidence for this but, it turns out, it isn’t far from the truth. Researchers worldwide have begun developing many types of powerful audio analysis AI algorithms that can extract a lot of information about us from sound alone.

While this technology is only just beginning to emerge in the real world, these growing capabilities – coupled with its 24/7 presence – could have serious implications for our personal privacy.

Instead of analyzing every word people say, much of the listening AI that has been developed can actually learn a staggering amount of personal information just from the sound of our speech alone.

It can determine everything from who you are and where you come from, your current location, your gender and age and what language you’re speaking – all just from the way your voice sounds when you speak.

If that isn’t creepy enough, other audio AI systems can detect if you’re lying, analyze your health and fitness level, your current emotional state, and whether or not you’re intoxicated.

There are even systems capable of detecting what you’re eating when you speak with your mouth full, plus a slew of research looking into diagnosing medical conditions from sound.

AI systems can also accurately interpret events from sound by listening to details like car crashes or gunshots, or environments from their background noise.

Other systems can identify a speakers’ attitude in a conversation, pick up unspoken messages or detect conflicts between speakers.

Another AI system developed last year can predict, just by listening to the tone a couple used when speaking to each other, whether or not they will stay together. These are all examples of current AI technology developed in research labs worldwide.

All of these technologies – no matter what they’re trying to learn about you – use machine learning. This involves training an algorithm with large amounts of data that has been labelled to indicate what information the data contains.

By processing thousands or millions of recordings, the algorithm gradually begins to infer which characteristics of the data – often just tiny fluctuations in the sound – are associated with which labels.

For example, a system used to detect your gender would record speech from your smartphone, and process it to extract “features” – a small set of distinct values that compactly represent a bigger speech recording.

Typically, features represent amplitude and frequency information in each successive 20 millisecond period of speech. The way that these fluctuate over time will be slightly different for male or female speech.

Machine learning systems will not only look at those features, but also how much, how often, and in which way the features change over time.

While the recording happens in the smartphone itself, clips are sent to internet servers which will extract features, compute their statistics, and handle the machine learning part.