# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong.
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong.
Okay, let's analyze each equation based on the specification that $f(x') \le 0$ if and only if $C(x') = t$. Let $x' = x+\delta$. The condition $C(x') = t$ means $t = \arg\max_i F(x')_i$, which is equivalent to $F(x')_t \ge F(x')_i$ for all $i \ne t$. This, in turn, is equivalent to $F(x')_t \ge \max_{i \ne t} F(x')_i$. Similarly, using the property that softmax preserves order ($a > b \implies \softmax(a) > \softmax(b)$ when considering components corresponding to $a$ and $b$), $C(x') = t$ is also equivalent to $Z(x')_t \ge Z(x')_i$ for all $i \ne t$, or $Z(x')_t \ge \max_{i \ne t} Z(x')_i$. 1. **$f_{1}(x') = -\loss_{F,t}(x') + 1$** * The cross-entropy loss for target class $t$ is $\loss_{F,t}(x') = -\log(F(x')_t)$. * So, $f_1(x') = -(-\log(F(x')_t)) + 1 = \log(F(x')_t) + 1$. * The condition $f_1(x') \le 0$ becomes $\log(F(x')_t) + 1 \le 0$, which means $\log(F(x')_t) \le -1$. * Exponentiating both sides gives $F(x')_t \le e^{-1} \approx 0.367$. * This condition is not equivalent to $C(x') = t$. For example, if $F(x') = (0.4, 0.3, 0.3)$ and $t=1$, then $C(x')=1$, but $F(x')_1 = 0.4 > e^{-1}$, so $f_1(x') > 0$. Conversely, if $F(x') = (0.1, 0.8, 0.1)$ and $t=1$, then $F(x')_1 = 0.1 \le e^{-1}$, so $f_1(x') \le 0$, but $C(x') = 2 \ne t$. * Therefore, $f_1$ is **incorrect**. 2. **$f_{2}(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+$** * Let $M = \max_{i \ne t} F(x')_i$. Then $f_2(x') = (M - F(x')_t)^+ = \max(M - F(x')_t, 0)$. * The condition $f_2(x') \le 0$ means $\max(M - F(x')_t, 0) \le 0$. Since the output of $\max(\cdot, 0)$ is always non-negative, this requires $\max(M - F(x')_t, 0) = 0$. * This occurs if and only if $M - F(x')_t \le 0$, which is equivalent to $M \le F(x')_t$. * This condition $\max_{i \ne t} F(x')_i \le F(x')_t$ is precisely the definition of $C(x') = t$. * Therefore, $f_2$ is **correct**. 3. **$f_{3}(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)$** * Let $M = \max_{i \ne t} F(x')_i$. Then $f_3(x') = \text{softplus}(M - F(x')_t) - \log(2) = \log(1 + e^{M - F(x')_t}) - \log(2)$. * The condition $f_3(x') \le 0$ means $\log(1 + e^{M - F(x')_t}) \le \log(2)$. * Since $\log$ is monotonically increasing, this is equivalent to $1 + e^{M - F(x')_t} \le 2$. * This simplifies to $e^{M - F(x')_t} \le 1$. * Since $e^x$ is monotonically increasing and $e^0 = 1$, this is equivalent to $M - F(x')_t \le 0$. * This is equivalent to $M \le F(x')_t$, which is the definition of $C(x') = t$. * Therefore, $f_3$ is **correct**. 4. **$f_{4}(x') = (0.5 - F(x')_t)^+$** * $f_4(x') = \max(0.5 - F(x')_t, 0)$. * The condition $f_4(x') \le 0$ means $\max(0.5 - F(x')_t, 0) = 0$. * This occurs if and only if $0.5 - F(x')_t \le 0$, which is equivalent to $F(x')_t \ge 0.5$. * If $F(x')_t \ge 0.5$, then since $\sum_i F(x')_i = 1$, we must have $F(x')_i \le 0.5$ for all $i \ne t$. Thus $F(x')_t \ge F(x')_i$ for all $i \ne t$, which means $C(x') = t$. So, $f_4(x') \le 0 \implies C(x') = t$. * However, the converse is not true. If $F(x') = (0.4, 0.3, 0.3)$ and $t=1$, then $C(x') = 1$, but $F(x')_1 = 0.4 < 0.5$, so $f_4(x') = (0.5 - 0.4)^+ = 0.1 > 0$. * Since the condition is not "if and only if", $f_4$ is **incorrect**. 5. **$f_{5}(x') = -\log(2 F(x')_t - 2)$** * The argument of the logarithm is $2 F(x')_t - 2 = 2(F(x')_t - 1)$. * Since $F(x')_t$ is a probability from softmax, $0 \le F(x')_t \le 1$. * Therefore, $F(x')_t - 1 \le 0$. The argument $2(F(x')_t - 1)$ is always less than or equal to zero. * The logarithm function $\log(x)$ is only defined for $x > 0$. * Thus, $f_5(x')$ is not well-defined over the real numbers for any possible output $F(x')_t$. * Therefore, $f_5$ is **incorrect**. 6. **$f_{6}(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+$** * Let $M_Z = \max_{i \ne t} Z(x')_i$. Then $f_6(x') = (M_Z - Z(x')_t)^+ = \max(M_Z - Z(x')_t, 0)$. * The condition $f_6(x') \le 0$ means $\max(M_Z - Z(x')_t, 0) = 0$. * This occurs if and only if $M_Z - Z(x')_t \le 0$, which is equivalent to $M_Z \le Z(x')_t$. * The condition $C(x') = t$ means $F(x')_t = \max_k F(x')_k$. Since $F(x') = \softmax(Z(x'))$ and the softmax function is order-preserving (i.e., $Z(x')_a > Z(x')_b \implies F(x')_a > F(x')_b$), the condition $C(x') = t$ is equivalent to $Z(x')_t = \max_k Z(x')_k$, which is equivalent to $Z(x')_t \ge \max_{i \ne t} Z(x')_i$, or $Z(x')_t \ge M_Z$. * Thus, $f_6(x') \le 0$ is equivalent to $C(x') = t$. * Therefore, $f_6$ is **correct**. 7. **$f_{7}(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2)$** * Let $M_Z = \max_{i \ne t} Z(x')_i$. Then $f_7(x') = \text{softplus}(M_Z - Z(x')_t) - \log(2) = \log(1 + e^{M_Z - Z(x')_t}) - \log(2)$. * The condition $f_7(x') \le 0$ means $\log(1 + e^{M_Z - Z(x')_t}) \le \log(2)$. * This is equivalent to $1 + e^{M_Z - Z(x')_t} \le 2$. * This simplifies to $e^{M_Z - Z(x')_t} \le 1$. * This is equivalent to $M_Z - Z(x')_t \le 0$, or $M_Z \le Z(x')_t$. * As established for $f_6$, this condition is equivalent to $C(x') = t$. * Therefore, $f_7$ is **correct**. The incorrect equations are $f_1$, $f_4$, and $f_5$.
1 | [1, 4, 5] |
Resulting in output: False