# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
Let's analyze each equation one by one to see if $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. 1. $f_{1}(x') = -\loss_{F,t}(x') + 1$ This equation does not satisfy the condition. Cross entropy loss $\loss_{F,t}(x')$ is a non-negative value. When $\loss_{F,t}(x') = 0$, $f_1(x') = 1 > 0$. So, we cannot guarantee that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. 2. $f_{2}(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+$ This equation satisfies the condition. If $C(x+\delta) = t$, it means that $F(x+\delta)_t > F(x+\delta)_i,\, \forall i \neq t$. Thus, $f_2(x+\delta) \le 0$. Conversely, if $f_2(x+\delta) \le 0$, then $F(x+\delta)_t \ge F(x+\delta)_i,\, \forall i \neq t$. Since the output values are probabilities, $F(x+\delta)_t > F(x+\delta)_i,\, \forall i \neq t$ and $C(x+\delta) = t$. 3. $f_{3}(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)$ This equation does not satisfy the condition. The softplus function is always positive for positive inputs and equal to 0 when its input is zero. Thus, $-\log(2) < f_3(x') \le 0$ when $C(x+\delta)=t$, which makes the condition not hold. 4. $f_{4}(x') = (0.5 - F(x')_t)^+$ This equation does not satisfy the condition. If $C(x+\delta) = t$, then $F(x+\delta)_t > 0.5$ since it's a probability distribution. So, $f_4(x+\delta) = 0$, which is correct. However, if $f_4(x+\delta) \le 0$, this only implies that $F(x+\delta)_t \ge 0.5$, but we cannot guarantee that it is the maximum among all output classes. 5. $f_{5}(x') = -\log(2 F(x')_t - 2)$ This equation does not satisfy the condition. If $C(x+\delta) = t$, then $F(x+\delta)_t > 0.5$. So, $-\log(2 F(x+\delta)_t - 2) > 0$, which does not satisfy the condition. 6. $f_{6}(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+$ This equation does not satisfy the condition. It is a similar condition as equation 2, but here we are dealing with logits ($Z(x')$) instead of output probabilities ($F(x')$). Ensuring $f_6(x+\delta) \le 0$ does not guarantee that $C(x+\delta) = t$ when the softmax function is applied to the logits. 7. $f_{7}(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2)$ This equation does not satisfy the condition. It is a similar condition as equation 3, but here we are dealing with logits ($Z(x')$) instead of output probabilities ($F(x')$). The softplus function in combination with $-\log(2)$ does not guarantee that $C(x+\delta) = t$ when the softmax function is applied to the logits. Conclusion: The only correct equation is $f_2$. So, the Python list containing the incorrect equations is:
1 | [1, 3, 4, 5, 6, 7] |
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
Let's analyze each equation one by one to see if they satisfy the condition $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. 1. $f_{1}(x') = -\loss_{F,t}(x') + 1$ This equation is incorrect. The cross-entropy loss $\loss_{F,t}(x')$ is non-negative, so $-\loss_{F,t}(x') + 1$ can never be less than or equal to zero. Therefore, the condition $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$ is not satisfied. 2. $f_{2}(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+$ This equation is correct. If $C(x+\delta) = t$, then $F(x+\delta)_t > F(x+\delta)_i$ for all $i \ne t$, so $\max_{i \ne t}( F(x+\delta)_i) - F(x+\delta)_t < 0$ and $f_{2}(x+\delta) = 0 \le 0$. Conversely, if $f_{2}(x+\delta) \le 0$, then $\max_{i \ne t}( F(x+\delta)_i) - F(x+\delta)_t \le 0$, so $F(x+\delta)_t \ge F(x+\delta)_i$ for all $i \ne t$ and $C(x+\delta) = t$. 3. $f_{3}(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)$ This equation is incorrect. If $C(x+\delta) = t$, then $F(x+\delta)_t > F(x+\delta)_i$ for all $i \ne t$, so $\max_{i \ne t}( F(x+\delta)_i) - F(x+\delta)_t < 0$ and $\text{softplus}(\max_{i \ne t}( F(x+\delta)_i) - F(x+\delta)_t) = 0$, but $-\log(2) > 0$, so $f_{3}(x+\delta) > 0$. Therefore, the condition $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$ is not satisfied. 4. $f_{4}(x') = (0.5 - F(x')_t)^+$ This equation is correct. If $C(x+\delta) = t$, then $F(x+\delta)_t > F(x+\delta)_i$ for all $i \ne t$, so $F(x+\delta)_t > 0.5$ and $f_{4}(x+\delta) = 0 \le 0$. Conversely, if $f_{4}(x+\delta) \le 0$, then $0.5 - F(x+\delta)_t \le 0$, so $F(x+\delta)_t \ge 0.5$ and $C(x+\delta) = t$. 5. $f_{5}(x') = -\log(2 F(x')_t - 2)$ This equation is incorrect. If $C(x+\delta) = t$, then $F(x+\delta)_t > F(x+\delta)_i$ for all $i \ne t$, so $F(x+\delta)_t > 0.5$ and $2 F(x+\delta)_t - 2 > -1$, but $-\log(-1)$ is undefined, so $f_{5}(x+\delta)$ is undefined. Therefore, the condition $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$ is not satisfied. 6. $f_{6}(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+$ This equation is incorrect. The output of the softmax function is used to determine the class, not the logits. Therefore, the condition $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$ is not satisfied. 7. $f_{7}(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2)$ This equation is incorrect for the same reason as $f_{6}(x')$. The output of the softmax function is used to determine the class, not the logits. Therefore, the condition $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$ is not satisfied. The python list of numbers for those that are wrong is [1, 3, 5, 6, 7].
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
Let's go through each equation one by one and check if $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. 1. $f_{1}(x') = -\loss_{F,t}(x') + 1$ This equation is incorrect. The cross-entropy loss $\loss_{F,t}(x')$ is non-negative and equals zero if and only if the model's prediction is perfectly correct. Therefore, $f_{1}(x') \le 0$ if and only if $\loss_{F,t}(x') \ge 1$, which does not guarantee that $C(x+\delta) = t$. 2. $f_{2}(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+$ This equation is correct. If $f_{2}(x') \le 0$, then $\max_{i \ne t}( F(x')_i) - F(x')_t \le 0$, which means $F(x')_t \ge \max_{i \ne t}( F(x')_i)$. Therefore, $C(x+\delta) = t$. 3. $f_{3}(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)$ This equation is incorrect. The softplus function is always positive, so $f_{3}(x') > -\log(2)$ for all $x'$. Therefore, $f_{3}(x') \le 0$ is never satisfied, regardless of the value of $C(x+\delta)$. 4. $f_{4}(x') = (0.5 - F(x')_t)^+$ This equation is incorrect. If $f_{4}(x') \le 0$, then $0.5 - F(x')_t \le 0$, which means $F(x')_t \ge 0.5$. This does not guarantee that $C(x+\delta) = t$, as there could be other classes with a probability greater than $0.5$. 5. $f_{5}(x') = -\log(2 F(x')_t - 2)$ This equation is incorrect. If $f_{5}(x') \le 0$, then $2 F(x')_t - 2 \le 1$, which means $F(x')_t \le 1.5$. This does not guarantee that $C(x+\delta) = t$, as there could be other classes with a probability greater than $F(x')_t$. 6. $f_{6}(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+$ This equation is incorrect. The output of the softmax function is not directly proportional to the logits, so $\max_{i \ne t}( Z(x')_i) - Z(x')_t \le 0$ does not guarantee that $C(x+\delta) = t$. 7. $f_{7}(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2)$ This equation is incorrect for the same reason as $f_{3}(x')$. The softplus function is always positive, so $f_{7}(x') > -\log(2)$ for all $x'$. Therefore, $f_{7}(x') \le 0$ is never satisfied, regardless of the value of $C(x+\delta)$. Therefore, the list of incorrect equations is [1, 3, 4, 5, 6, 7].
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
Let's analyze each equation one by one to determine if $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. 1. $f_{1}(x') = -\loss_{F,t}(x') + 1$ This equation is incorrect. Cross entropy loss $\loss_{F,t}(x')$ is non-negative and equals zero if and only if $F(x')$ is one-hot encoded with 1 at index $t$. Therefore, $-\loss_{F,t}(x') + 1 \le 0$ if and only if $\loss_{F,t}(x') \ge 1$. This does not guarantee that $C(x+\delta) = t$. 2. $f_{2}(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+$ This equation is correct. If $f_{2}(x') \le 0$, then $\max_{i \ne t}( F(x')_i) \le F(x')_t$, which implies that class $t$ has the highest probability, and so $C(x+\delta) = t$. Conversely, if $C(x+\delta) = t$, then $\max_{i \ne t}( F(x')_i) \le F(x')_t$, and so $f_{2}(x') \le 0$. 3. $f_{3}(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)$ This equation is incorrect. If $f_{3}(x') \le 0$, then $\text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t) \le \log(2)$. However, this does not guarantee that $\max_{i \ne t}(F(x')_i) \le F(x')_t$, and so it does not necessarily imply that $C(x+\delta) = t$. 4. $f_{4}(x') = (0.5 - F(x')_t)^+$ This equation is incorrect. If $f_{4}(x') \le 0$, then $F(x')_t \ge 0.5$. However, this does not guarantee that $F(x')_t$ is the highest probability, and so it does not necessarily imply that $C(x+\delta) = t$. 5. $f_{5}(x') = -\log(2 F(x')_t - 2)$ This equation is incorrect. If $f_{5}(x') \le 0$, then $\log(2 F(x')_t - 2) \ge 0$, which implies that $F(x')_t \ge 1$. However, the output of the softmax function is always less than or equal to 1, and so this is not possible. 6. $f_{6}(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+$ This equation is incorrect. The softmax function is a monotonically increasing function of its inputs. Therefore, if $\max_{i \ne t}( Z(x')_i) \le Z(x')_t$, then $\max_{i \ne t}( F(x')_i) \le F(x')_t$, and so $C(x+\delta) = t$. However, the converse is not necessarily true. If $C(x+\delta) = t$, it does not necessarily imply that $\max_{i \ne t}( Z(x')_i) \le Z(x')_t$. 7. $f_{7}(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2)$ This equation is incorrect for the same reasons as $f_{6}(x')$. Therefore, the list of incorrect equations is: [1, 3, 4, 5, 6, 7].
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
Let's analyze each equation to see if it meets the condition that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. 1. $f_{1}(x') = -\loss_{F,t}(x') + 1$: This equation seems incorrect because the cross entropy loss $\loss_{F,t}(x')$ is always non-negative, and subtracting it from 1 does not guarantee that $f(x+\delta) \le 0$ when $C(x+\delta) = t$. 2. $f_{2}(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+$: This equation is correct. When $C(x+\delta) = t$, $F(x')_t$ is the maximum value in the output vector, so the difference inside the parentheses is non-positive, and the $(e)^+$ operation ensures that $f(x+\delta) \le 0$. Conversely, if $C(x+\delta) \ne t$, the difference inside the parentheses is positive, and $f(x+\delta) > 0$. 3. $f_{3}(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)$: This equation is also correct. The softplus function ensures that the output is positive when the input is positive and close to zero when the input is negative. Since $\max_{i \ne t}(F(x')_i) - F(x')_t$ is negative when $C(x+\delta) = t$, $f(x+\delta)$ will be less than or equal to $-\log(2)$, which is less than 0. 4. $f_{4}(x') = (0.5 - F(x')_t)^+$: This equation is incorrect. When $C(x+\delta) = t$, $F(x')_t$ should be close to 1, but not necessarily greater than 0.5. Therefore, $f(x+\delta)$ can be greater than 0 even when $C(x+\delta) = t$. 5. $f_{5}(x') = -\log(2 F(x')_t - 2)$: This equation is incorrect. When $C(x+\delta) = t$, $F(x')_t$ should be close to 1, but not necessarily greater than 1.5. Therefore, the argument of the logarithm can be non-positive, resulting in an undefined or positive value for $f(x+\delta)$. 6. $f_{6}(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+$: This equation is incorrect. The logits $Z(x')$ do not necessarily follow the same order as the output probabilities $F(x')$, so the maximum logit among the incorrect classes does not necessarily correspond to the maximum probability among the incorrect classes. 7. $f_{7}(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2)$: This equation is also incorrect for the same reason as $f_{6}(x')$. Based on the analysis above, the python list of numbers for those that are wrong is: [1, 4, 5, 6, 7]
Resulting in output: False