# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
Let's analyze each equation to see if it satisfies the condition $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. We'll use $x'$ to represent $x+\delta$ for brevity. 1. **fâ‚(x') = -loss_{F,t}(x') + 1:** Cross-entropy loss is minimized when the predicted probability for the target class is close to 1. However, `-loss + 1 <= 0` implies `loss >= 1`. A loss of 1 or greater doesn't guarantee that class `t` is the most likely. For example, if all classes have equal probability (1/m), the loss will be greater than 1 (for m > 2), but no class is preferred. **INCORRECT** 2. **fâ‚‚(x') = (max_{i ≠t}(F(x')áµ¢) - F(x')â‚œ)âº:** This equation checks if the probability of the target class `t` is greater than or equal to the probability of any other class. If `fâ‚‚(x') <= 0`, then `F(x')â‚œ >= max_{i ≠t}(F(x')áµ¢)`, which means `t` is the most likely class, so `C(x') = t`. Conversely, if `C(x') = t`, then `F(x')â‚œ` is maximal, meaning `fâ‚‚(x') <= 0`. **CORRECT** 3. **f₃(x') = softplus(max_{i ≠t}(F(x')áµ¢) - F(x')â‚œ) - log(2):** `softplus(x) - log(2) <= 0` implies `softplus(x) <= log(2)`, which means `log(1 + exp(x)) <= log(2)`, so `1 + exp(x) <= 2`, thus `exp(x) <= 1`, and `x <= 0`. This reduces to the same condition as `fâ‚‚`: `max_{i ≠t}(F(x')áµ¢) - F(x')â‚œ <= 0`. **CORRECT** 4. **fâ‚„(x') = (0.5 - F(x')â‚œ)âº:** This equation checks if the probability of the target class `t` is greater than or equal to 0.5. However, this doesn't guarantee that `t` is the most likely class. Another class could have a probability greater than 0.5 as well. **INCORRECT** 5. **fâ‚…(x') = -log(2F(x')â‚œ - 2):** This equation is problematic because `2F(x')â‚œ - 2` can be negative or zero, leading to undefined or infinite values for the logarithm. Even if we restrict to cases where `F(x')â‚œ > 1`, `-log(2F(x')â‚œ - 2) <= 0` implies `2F(x')â‚œ - 2 >= 1`, so `F(x')â‚œ >= 1.5`. Probabilities cannot be greater than 1. **INCORRECT** 6. **f₆(x') = (max_{i ≠t}(Z(x')áµ¢) - Z(x')â‚œ)âº:** This uses the logits `Z(x')` instead of the probabilities `F(x')`. Since the softmax function is monotonic, the order of the logits is preserved in the probabilities. Therefore, if `f₆(x') <= 0`, then `Z(x')â‚œ >= max_{i ≠t}(Z(x')áµ¢)`, which implies `F(x')â‚œ >= max_{i ≠t}(F(x')áµ¢)`, so `C(x') = t`. Conversely, if `C(x') = t`, then `F(x')â‚œ` is maximal, implying `Z(x')â‚œ` is maximal, so `f₆(x') <= 0`. **CORRECT** 7. **f₇(x') = softplus(max_{i ≠t}(Z(x')áµ¢) - Z(x')â‚œ) - log(2):** Similar to the relationship between `fâ‚‚` and `f₃`, this reduces to the same condition as `f₆` and is therefore correct. **CORRECT** Final Answer: The incorrect equations are 1, 4, and 5.
1 | [1, 4, 5] |
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
Let's analyze each equation to determine if it satisfies the condition $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Recall that $C(x') = \arg\max_i F(x')_i$. Let $x' = x + \delta$. 1. **fâ‚(x') = -loss_{F,t}(x') + 1:** Cross-entropy loss is minimized when the predicted probability for the true class is close to 1. If $C(x') = t$, the loss will be low, but not necessarily less than 1. Conversely, if the loss is less than 1, it doesn't guarantee that class $t$ has the highest probability. **INCORRECT** 2. **fâ‚‚(x') = (max_{i ≠t}(F(x')_i) - F(x')_t)âº:** If $C(x') = t$, then $F(x')_t$ is the maximum, making the expression inside the parenthesis negative or zero. Thus, $fâ‚‚(x') = 0$. If $fâ‚‚(x') = 0$, this means $\max_{i ≠t}(F(x')_i) \le F(x')_t$, so $C(x') = t$. **CORRECT** 3. **f₃(x') = softplus(max_{i ≠t}(F(x')_i) - F(x')_t) - log(2):** Similar to $f_2$, if $C(x') = t$, then $\max_{i ≠t}(F(x')_i) \le F(x')_t$. Thus, the argument to softplus is $\le 0$. Since softplus(0) = log(2), we have $f_3(x') \le 0$. Conversely, if $f_3(x') \le 0$, then $\text{softplus}(\max_{i ≠t}(F(x')_i) - F(x')_t) \le \log(2)$. Since softplus is monotonically increasing, this implies $\max_{i ≠t}(F(x')_i) - F(x')_t \le 0$, so $C(x') = t$. **CORRECT** 4. **fâ‚„(x') = (0.5 - F(x')_t)âº:** If $C(x') = t$, we don't necessarily have $F(x')_t \ge 0.5$. In a multi-class setting, $F(x')_t$ could be less than 0.5 and still be the maximum. Conversely, $F(x')_t \ge 0.5$ does not guarantee that it's the maximum. **INCORRECT** 5. **fâ‚…(x') = -log(2F(x')_t - 2):** This function isn't even well-defined when $F(x')_t \le 1$, which is always the case due to the softmax function. Further, even if we fix the domain issue (e.g., by adding 1 inside the logarithm), $f_5(x') \le 0$ doesn't imply $F(x')_t$ is the maximum. **INCORRECT** 6. **f₆(x') = (max_{i ≠t}(Z(x')_i) - Z(x')_t)âº:** Since softmax is monotonically increasing, the ordering of the logits $Z(x')$ is preserved in the probabilities $F(x')$. Therefore, $\arg\max_i Z(x')_i = \arg\max_i F(x')_i$. This means that if $C(x') = t$, then $Z(x')_t$ is the maximum, and $f_6(x') = 0$. Conversely, if $f_6(x') = 0$, then $Z(x')_t$ is the maximum, implying $C(x') = t$. **CORRECT** 7. **f₇(x') = softplus(max_{i ≠t}(Z(x')_i) - Z(x')_t) - log(2):** Using the same reasoning as for $f_3$ and $f_6$, this is also correct. **CORRECT** Final Answer: The incorrect equations are 1, 4, and 5.
1 | [1, 4, 5] |
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
Let's analyze each equation to see if it satisfies the condition $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. We'll use $x'$ to represent $x+\delta$ for brevity. 1. **$f_1(x') = -\loss_{F,t}(x') + 1$**: Cross-entropy loss is minimized when the predicted probability for the target class is close to 1. However, $-\loss_{F,t}(x') + 1 \le 0$ implies $\loss_{F,t}(x') \ge 1$. This doesn't guarantee that class $t$ is the most likely. The loss could be greater than 1 even if $t$ is the predicted class. **Incorrect.** 2. **$f_2(x') = (\max_{i \ne t}(F(x')_i) - F(x')_t)^+$**: This equation checks if the probability of the target class $t$ is greater than or equal to the probability of any other class. If $F(x')_t$ is the largest, then the difference will be non-positive, and the $\max$ with 0 will result in 0. If any other class has a higher probability, the difference will be positive. Thus, $f_2(x') \le 0$ if and only if $F(x')_t$ is maximal, which is equivalent to $C(x') = t$. **Correct.** 3. **$f_3(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t) - \log(2)$**: Similar to $f_2$, this equation checks if the probability of target class $t$ is greater than or equal to the probability of any other class. Softplus is a smooth approximation of the ReLU function. $\text{softplus}(x) \le \log(2)$ if and only if $x \le 0$. Thus $f_3(x') \le 0$ if and only if $\max_{i \ne t}(F(x')_i) - F(x')_t \le 0$. This is the same condition as $f_2$. **Correct.** 4. **$f_4(x') = (0.5 - F(x')_t)^+$**: This equation checks if the probability of the target class $t$ is greater than or equal to 0.5. However, the target class could have a probability less than 0.5 and still be the most likely class. **Incorrect.** 5. **$f_5(x') = -\log(2 F(x')_t - 2)$**: Since $0 \le F(x')_t \le 1$, the term $2F(x')_t - 2$ is always non-positive. The logarithm of a non-positive value is undefined in the real domain. Moreover, $f_5(x') \le 0$ would imply that $\log(2F(x')_t - 2)$ is non-negative, which is not possible. **Incorrect** 6. **$f_6(x') = (\max_{i \ne t}(Z(x')_i) - Z(x')_t)^+$**: This equation compares logits. Since the softmax function is monotonic, the order of logits corresponds to the order of probabilities. Thus, this equation is equivalent to $f_2$. **Correct.** 7. **$f_7(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t) - \log(2)$**: This is the same logic as $f_3$, but using logits instead of probabilities. Again, due to the monotonicity of softmax, this equation is correct. **Correct.** Final Answer: The incorrect equations are 1, 4, and 5.
1 | [1, 4, 5] |
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
Let's analyze each equation to determine if it satisfies the condition $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. We'll use $x'$ to represent $x+\delta$. 1. **$f_1(x') = -\loss_{F,t}(x') + 1$**: Cross-entropy loss is minimized when the predicted probability for the target class $t$ is close to 1. If $C(x') = t$, the loss will be low, but not necessarily less than 1. So, $C(x') = t$ does *not* imply $f_1(x') \le 0$. Conversely, if $f_1(x') \le 0$, it means $-\loss_{F,t}(x') \le -1$, or $\loss_{F,t}(x') \ge 1$. This doesn't guarantee that class $t$ has the highest probability. Therefore, $f_1(x') \le 0$ does *not* imply $C(x') = t$. **INCORRECT** 2. **$f_2(x') = (\max_{i \ne t}(F(x')_i) - F(x')_t)^+$**: If $C(x') = t$, then $F(x')_t$ is the largest probability, meaning $\max_{i \ne t}(F(x')_i) < F(x')_t$. This implies $f_2(x') = 0$. Conversely, if $f_2(x') \le 0$, then since it's the positive part, it must be 0, meaning $\max_{i \ne t}(F(x')_i) \le F(x')_t$. This means $t$ is the predicted class. **CORRECT** 3. **$f_3(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t) - \log(2)$**: If $C(x') = t$, then $\max_{i \ne t}(F(x')_i) < F(x')_t$. Let $d = \max_{i \ne t}(F(x')_i) - F(x')_t < 0$. Then, $\text{softplus}(d) < \text{softplus}(0) = \log(2)$. So $f_3(x') < 0$. Conversely, if $f_3(x') \le 0$, then $\text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t) \le \log(2)$. This implies $\max_{i \ne t}(F(x')_i) - F(x')_t \le 0$, so $t$ is the predicted class. **CORRECT** 4. **$f_4(x') = (0.5 - F(x')_t)^+$**: If $C(x') = t$, $F(x')_t$ could be greater than or less than 0.5. If $F(x')_t < 0.5$, then $f_4(x') > 0$, violating the condition. Conversely, if $f_4(x') \le 0$, then $F(x')_t \ge 0.5$. This doesn't guarantee that $t$ is the argmax. **INCORRECT** 5. **$f_5(x') = -\log(2F(x')_t - 2)$**: This function isn't even defined when $F(x')_t \le 1$, which is always true. **INCORRECT** 6. **$f_6(x') = (\max_{i \ne t}(Z(x')_i) - Z(x')_t)^+$**: Since softmax is monotonic, the order of logits $Z(x')$ is preserved in the probabilities $F(x')$. Therefore, this functions similar to $f_2(x')$. **CORRECT** 7. **$f_7(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t) - \log(2)$**: Similar to $f_3(x')$, this uses logits instead of probabilities, but due to the monotonic nature of softmax, it's still correct. **CORRECT** Final Answer: The incorrect equations are 1, 4, and 5.
1 | [1, 4, 5] |
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
Let's analyze each equation to determine if it satisfies the condition $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. We'll use $x'$ to represent $x + \delta$. 1. **$f_1(x') = -\loss_{F,t}(x') + 1$**: Cross-entropy loss is minimized when the predicted probability for the target class $t$ is close to 1. If $C(x') = t$, then $F(x')_t$ is the largest probability, and the loss will be relatively small, potentially making $f_1(x') \le 0$. However, $f_1(x') \le 0$ doesn't necessarily mean $F(x')_t$ is the *largest* probability. It just means the loss is sufficiently small, which could occur even if $F(x')_t$ is not the maximum. Thus, this equation is *incorrect*. 2. **$f_2(x') = (\max_{i \ne t}(F(x')_i) - F(x')_t)^+$**: This equation correctly captures the condition. If $F(x')_t$ is the maximum, then $\max_{i \ne t}(F(x')_i) - F(x')_t \le 0$, so $f_2(x') = 0$. Conversely, if $f_2(x') = 0$, then $\max_{i \ne t}(F(x')_i) \le F(x')_t$, meaning $F(x')_t$ is the maximum, so $C(x') = t$. Thus, this equation is *correct*. 3. **$f_3(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t) - \log(2)$**: Softplus is a monotonically increasing function. $\text{softplus}(x) \le \log(2)$ if and only if $x \le 0$. Therefore, $f_3(x') \le 0$ if and only if $\max_{i \ne t}(F(x')_i) - F(x')_t \le 0$, which is equivalent to $f_2(x') \le 0$. Thus, this equation is *correct*. 4. **$f_4(x') = (0.5 - F(x')_t)^+$**: $f_4(x') \le 0$ if and only if $F(x')_t \ge 0.5$. While $C(x') = t$ implies $F(x')_t$ is the largest probability, it doesn't guarantee $F(x')_t \ge 0.5$. Conversely, $F(x')_t \ge 0.5$ doesn't guarantee that it's the largest. Thus, this equation is *incorrect*. 5. **$f_5(x') = -\log(2F(x')_t - 2)$**: If $2F(x')_t - 2 \le 0$ then we have $F(x')_t \le 1$, which does not mean that $F(x')_t$ is the maximum. If $2F(x')_t - 2 > 0$ then we have $F(x')_t > 1$ which is not possible because $0 \le F(x')_t \le 1$. Thus, this equation is *incorrect*. 6. **$f_6(x') = (\max_{i \ne t}(Z(x')_i) - Z(x')_t)^+$**: This is analogous to $f_2$, but uses logits $Z$ instead of probabilities $F$. Since the softmax function is monotonically increasing, the order of the logits is preserved in the probabilities. Therefore, this equation is *correct*. 7. **$f_7(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t) - \log(2)$**: This is analogous to $f_3$ and is also *correct* for the same reasons. Final Answer: The incorrect equations are 1, 4, and 5.
1 | [1, 4, 5] |
Resulting in output: False