Google has infused its so-called Smart Reply feature, which uses machine learning to suggest words and sentences you may want to type next, into various email products for the last several years. But with Android 11, those contextual nudges—including emojis and stickers—are built directly into Gboard, Google’s popular keyboard app. They can follow you everywhere you type. The real trick? Figuring out how to keep the AI that powers all of this from becoming a privacy nightmare.
First, some basics. Google has been adamant for years that Gboard doesn’t retain or send any data about your keystrokes. The only time the company knows what you’re typing on Gboard is when you use the app to submit a Google search or input other data to the company’s services that it would see from any keyboard. But offering reply recommendations has broader potential privacy implications, since the feature relies on real-time analysis of everything that’s going on in your mobile life to make useful suggestions.
“Within Gboard we want to be smart, we want to give you the right emoji prediction and the right text prediction,” says Xu Liu, Gboard’s director of engineering. “But we don’t want to log anything you type and there’s no text or content going to any server at all. So that’s a big challenge, but privacy is our number one engineering focus.”
To achieve that privacy, Google is running all of the necessary algorithms locally on your device. It doesn’t see your data, or send it anywhere. And there’s another thing: Google isn’t trusting the Gboard app itself to do any of that processing.
“It’s great to see advanced machine learning research work its way into practical use for strictly on-device applications,” says Kenn White, a security engineer and founder of the Open Crypto Audit Project.
Even with the precaution of keeping all the AI magic on the device, giving a keyboard app access to the content that feeds those calculations would be high risk. Malicious apps, for example, could try to attack the keyboard app to access data they shouldn’t be able to see. So the Gboard team had an idea: Why not box Gboard out of the equation entirely and have the Android operating system itself run the machine learning analyses to determine response recommendations? Android already runs all of your apps and services, meaning you’ve already entrusted it with your data. And any malware that’s sophisticated enough to take control of your smartphone’s operating system can ransack the whole thing anyway. Even in a worst-case scenario, the reasoning goes, letting Android oversee predictive replies doesn’t create an additional avenue for attack.
So when Gboard pops up three suggestions of what to type next in Android 11, you’re actually not looking at the Gboard app when you scan those options. Instead, you’re experiencing a sort of composite of Gboard and the Android platform itself.
“It’s a seamless experience, but we have two layers,” Google’s Liu says. “One is the keyboard layer and the other is the operating system layer, but it’s transparent.”
Gboard is the default keyboard on stock Android, but it’s also available on iOS. These new features aren’t available for iPhone and iPad owners, but because Android is open source, Google can offer the same predictive feature it’s using in Gboard for any other third-party keyboard to incorporate into its app. This way, alternative keyboards don’t have to do anything sneaky or try to work around Android’s permission limits for apps to offer predictive replies. And the whole system is powered by Google’s “federated learning” techniques, a way of building machine learning models off of data sets that come from all different sources and are never combined—like using data from everyone’s phones to refine prediction algorithms without ever moving the data off their devices.
