Nicholas Carlini*, Pratyush Mishra, Tavish Vaidya, Yuankai Zhang, Micah Sherr, Clay Shields, David Wagner, and Wenchao Zhou
Voice interfaces are becoming more ubiquitous and are
now the primary input method for many devices. We explore
in this paper how they can be attacked with hidden
voice commands that are unintelligible to human listeners
but which are interpreted as commands by devices.
We evaluate these attacks under two different threat
models. In the black-box model, an attacker uses the
speech recognition system as an opaque oracle. We show
that the adversary can produce difficult to understand
commands that are effective against existing systems in
the black-box model. Under the white-box model, the
attacker has full knowledge of the internals of the speech
recognition system and uses it to create attack commands
that we demonstrate through user testing are not understandable
We then evaluate several defenses, including notifying
the user when a voice command is accepted; a verbal
challenge-response protocol; and a machine learning approach
that can detect our attacks with 99.8% accuracy.
* authors listed alphabetically, students appearing first
Nicholas Carlini, Antonio Barresi, Mathias Payer, Thomas R. Gross and David Wagner
Control-Flow Integrity (CFI) is a defense which prevents control-flow hijacking attacks.
While recent research has shown that coarse-grained CFI does not stop attacks,
fine-grained CFI is believed to be secure.
We argue that assessing the effectiveness of practical CFI implementations is non-trivial
and that common evaluation metrics fail to do so. We then evaluate fully-precise static
CFI -- the most restrictive CFI policy that does not break functionality -- and reveal
limitations in its security. Using a generalization of non-control-data attacks which we call
Control-Flow Bending (CFB), we show how an attacker can leverage a memory corruption
vulnerability to achieve Turing-complete computation on memory using just calls to the
standard library. We use this attack technique to evaluate fully-precise static CFI on six
real binaries and show that in five out of six cases, powerful attacks are still possible. Our
results suggest that CFI may not be a reliable defense against memory corruption vulnerabilities.
We further evaluate shadow stacks in combination with CFI and find that their presence
for security is necessary: deploying shadow stacks removes arbitrary code execution
capabilities of attackers in three of six cases.
Return Oriented Programming (ROP) has become the exploitation technique of choice for modern memory-safety
vulnerability attacks. Recently, there have been multiple attempts at defenses to prevent ROP attacks. In
this paper, we introduce three new attack methods that break many existing ROP defenses. Then we show how
to break kBouncer and ROPecker, two recent low-overhead defenses that can be applied to legacy software on
existing hardware. We examine several recent ROP attacks seen in the wild and demonstrate that our techniques
successfully cloak them so they are not detected by these defenses. Our attacks apply to many CFI-based defenses
which we argue are weaker than previously thought. Future defenses will need to take our attacks into account.
USENIX Journal of Election Technology and Systems (JETS), Volume 1 Issue 1. Presented at EVT/WOTE 2013.
Eric Kim, Nicholas Carlini, Andrew Chang, George Yiu, Kai Wang, and David Wagner.
This paper studies how to provide support for ballot-level post-election audits. Informed by our work supporting pilots of
these audits in several California counties, we identify gaps in current technology in tools for this task: we need better ways
to count voted ballots (from scanned images) without access to scans of blank, unmarked ballots; and we need improvements
to existing techniques that help them scale better to large, complex elections. We show how to meet these needs and use our
system to successfully process ballots from 11 California counties, in support of the pilot audit program. Our new techniques
yield order-of-magnitude speedups compared to the previous system, and enable us to successfully process some elections
that would not have reasonably feasible without these techniques.
Kai Wang, Eric Kim, Nicholas Carlini, Ivan Motyashov, Daniel Nguyen, and David Wagner.
We present OpenCount: a system that tabulates scanned
ballots from an election by combining computer vision
algorithms with focused operator assistance. OpenCount
is designed to support risk-limiting audits and to be scalable to large elections, robust to conditions encountered
using typical scanner hardware, and general to a wide
class of ballot types--all without the need for integration with any vendor systems. To achieve these goals,
we introduce a novel operator-in-the-loop computer vision pipeline for automatically processing scanned ballots while allowing the operator to intervene in a simple,
intuitive manner. We evaluate our system on data collected from five risk-limiting audit pilots conducted in
California in 2011.
Nicholas Carlini, Adrienne Porter Felt, and David Wagner.
Vulnerabilities in browser extensions put users at risk by
providing a way for website and network attackers to
gain access to users’ private data and credentials. Extensions can also introduce vulnerabilities into the websites
that they modify. In 2009, Google Chrome introduced
a new extension platform with several features intended
to prevent and mitigate extension vulnerabilities: strong
isolation between websites and extensions, privilege separation within an extension, and an extension permission
system. We performed a security review of 100 Chrome
extensions and found 70 vulnerabilities across 40 extensions. Given these vulnerabilities, we evaluate how well
each of the security mechanisms defends against extension vulnerabilities. We find that the mechanisms mostly
succeed at preventing direct web attacks on extensions,
but new security mechanisms are needed to protect users
from network attacks on extensions, website metadata attacks on extensions, and vulnerabilities that extensions
add to websites. We propose and evaluate additional defenses, and we conclude that banning HTTP scripts and
inline scripts would prevent 47 of the 50 most severe vulnerabilities with only modest impact on developers.