Google Glass Snoopers Can Steal Your Passcode With a Glance

The odds are you can’t make out the PIN of that guy with the sun glaring obliquely off his iPad’s screen across the coffee shop. But if he’s wearing Google Glass or a smartwatch, he probably can see yours.
GoogleGlassVSiPadedit
The UMass researchers testing PIN-spying with Google Glass.Cyber Forensics Laboratory at University of Massachusetts Lowell

The odds are you can’t make out the PIN of that guy with the sun glaring obliquely off his iPad’s screen across the coffee shop. But if he’s wearing Google Glass or a smartwatch, he probably can see yours.

Researchers at the University of Massachusetts Lowell found they could use video from wearables like Google Glass and the Samsung smartwatch to surreptitiously pick up four-digit PIN codes typed onto an iPad from almost 10 feet away—and from nearly 150 feet with a high-def camcorder. Their software, which used a custom-coded video recognition algorithm that tracks the shadows from finger taps, could spot the codes even when the video didn’t capture any images on the target devices’ displays.

“I think of this as a kind of alert about Google Glass, smartwatches, all these devices,” says Xinwen Fu, a computer science professor at UMass Lowell who plans to present the findings with his students at the Black Hat security conference in August. “If someone can take a video of you typing on the screen, you lose everything.”

Fu and his students tested a variety of video-enabled devices including Glass, an iPhone 5 camera, and a $72 Logitech webcam. They used Glass to spot a four-digit PIN from three meters away with 83 percent accuracy—and greater than 90 percent with some manual correction of errors. Webcam video revealed the code 92 percent of the time. And the iPhone’s sharper camera caught the code in every case. The researchers have tested the Samsung smartwatch just a few times, but it caught the target PIN about as often as Glass.

Other hackers have shown it’s possible to perform automated over-the-shoulder password stealing. But Fu notes that older video tools had to actually see the display, which often is impossible from a distance or from indirect angles. (See UMass’s PIN-capturing footage taken by Glass in the GIF below.) His team’s video recognition software can spot passcodes even when the screen is unreadable, based on its understanding of an iPad’s geometry and the position of the user’s fingers. It maps its image of the angled iPad onto a “reference” image of the device, then looks for the abrupt down and up movements of the dark crescents that represent the fingers’ shadows.

XinwenFu-GoogleGlass-vs-iPad-3metersv2Video: Cyber Forensics Laboratory at University of Massachusetts Lowell. GIF: WIRED Fu says the researchers didn’t test longer passwords. But after a quick, back-of-the-envelope estimate based on Glass’s recognition of individuals characters, he believes it could recognize an eight-character password on an iPad’s QWERTY keyboard around 78 percent of the time. And despite the iPhone’s superior abilities as a spying tool, Fu notes Glass’ eye-level position provides a better angle for undetectable passcode theft.

“Any camera works, but you can’t hold your iPhone over someone to do this,” says Fu. “Because Glass is on your head, it’s perfect for this kind of sneaky attack.”

Google, which has been on the defensive about Glass’s reputation for privacy intrusion, disagrees.

“Unfortunately, stealing passwords by watching people as they type them…is nothing new,” a Google spokesman wrote in a statement. “We designed Glass with privacy in mind. The fact that Glass is worn above the eyes and the screen lights up whenever it’s activated clearly signals it’s in use and makes it a fairly lousy surveillance device.”

The UMass researchers concede the problem doesn’t lie specifically with Glass; the problem is the passcodes. After all, a $700 Panasonic camcorder’s optical zoom was able to catch a PIN typed on a glare-obscured screen from 44 meters away, a surveillance tactic that would be impossible with Google’s headset. In that setup, the researchers used the camcorder to detect an iPad’s code every time they tried from a window four stories up and across the street from their target.

The researchers' diagram showing how they were able to reliable capture PINs with a camcorder from 44 meters away.The researchers’ diagram showing how they were able to reliably capture PINs with a camcorder from 44 meters away. Cyber Forensics Laboratory at University of Massachusetts Lowell To demonstrate a fix for that PIN privacy issue, the researchers have built an Android add-on that randomizes the layout of a phone or tablet’s lockscreen keyboard. They plan to release the software, dubbed Privacy Enhancing Keyboard or PEK, as an app in Google’s Play store and as an Android operating system update at the time of their Black Hat talk. “You can’t prevent people from taking videos,” says Fu. “But for the research community, we need to think about how we design our authentication in a better way.”

Of course, there’s also NSA whistleblower Edward Snowden’s method of protecting passwords against video surveillance: As a fugitive in Hong Kong, he typed them only while wearing “a large red hood over his head and laptop.” For the rest of us, one cautious hand over the screen might do the trick.