# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
To determine whether each equation correctly satisfies the condition \(C(x+\delta) = t\) if and only if \(f(x+\delta) \le 0\), we need to analyze each equation in the context of the classification task and the properties of the softmax function and logits. 1. **Equation \(f_1(x') = -\loss_{F,t}(x') + 1\)**: - The loss function \(\loss_{F,t}(x')\) is 0 when the classification is correct (i.e., \(C(x') = t\)). In such a case, \(f_1(x') = 1\) which is greater than 0. Thus, \(f_1(x') \le 0\) does not hold when \(C(x') = t\). Therefore, this equation is **incorrect**. 2. **Equation \(f_2(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\)**: - If \(C(x') = t\), then \(F(x')_t\) is the maximum, and hence \(\max_{i \ne t}( F(x')_i) < F(x')_t\) leading to \(f_2(x') = 0\). If \(C(x') \ne t\), then \(f_2(x') > 0\). Thus, \(f_2(x') \le 0\) holds if and only if \(C(x') = t\). This equation is **correct**. 3. **Equation \(f_3(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t) - \log(2)\)**: - Similar to \(f_2\), if \(C(x') = t\), the term inside softplus becomes \(-\log(2)\) which is negative, thus \(f_3(x') < 0\). If \(C(x') \ne t\), it results in a positive value. Hence, this equation is also **correct**. 4. **Equation \(f_4(x') = (0.5 - F(x')_t)^+\)**: - If \(C(x') = t\) and \(F(x')_t \ge 0.5\), \(f_4(x') = 0\). If \(F(x')_t < 0.5\), this indicates the model is not confident in class \(t\), thus \(f_4(x') > 0\). Therefore, this equation is **incorrect**. 5. **Equation \(f_5(x') = -\log(2 F(x')_t - 2)\)**: - For \(C(x') = t\) and \(F(x')_t\) close to 1, \(f_5(x')\) will be negative. However, for \(F(x')_t < 1\), \(f_5(x')\) can become undefined or positive, making \(f_5(x') \le 0\) not hold consistently. Thus, this equation is **incorrect**. 6. **Equation \(f_6(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\)**: - This is similar to \(f_2\) but uses logits instead of softmax outputs. If \(C(x') = t\), then \(Z(x')_t\) is still the maximum, leading to \(f_6(x') = 0\). If \(C(x') \ne t\), it yields \(f_6(x') > 0\). Hence, this equation is **correct**. 7. **Equation \(f_7(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t) - \log(2)\)**: - Similar reasoning as \(f_3\). If \(C(x') = t\), the expression inside the softplus becomes \(-\log(2)\), resulting in \(f_7(x') < 0\). Conversely, if \(C(x') \ne t\), it results in a positive value. Thus, this equation is **correct**. In summary, the equations that are **incorrect** according to the specification are \(f_1\), \(f_4\), and \(f_5\). Thus, the final answer is:
1 | [1, 4, 5] |
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
To determine whether each equation is correct according to the specification \( C(x+\delta) = t \) if and only if \( f(x+\delta) \le 0 \), we need to analyze each equation in the context of its relationship to the classification confidence provided by the neural network. 1. **Equation \( f_{1}(x') = -\loss_{F,t}(x') + 1 \)**: - The loss function \( \loss_{F,t}(x') \) measures how well the model predicts class \( t \). When the model is correct, this loss is low, making \( f_{1}(x') \) high. When the model is wrong or unsure, the loss increases, causing \( f_{1}(x') \) to drop below or equal to zero. Thus, \( C(x+\delta) = t \) corresponds to \( f_{1}(x+\delta) \le 0 \). **This equation is correct.** 2. **Equation \( f_{2}(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+ \)**: - This function measures the margin between the second highest class probability and the probability of the correct class. If \( C(x+\delta) = t \), then \( F(x')_t \) should be higher than \( \max_{i \ne t}( F(x')_i) \), making \( f_{2}(x') \le 0 \) true. Therefore, this equation is correct. 3. **Equation \( f_{3}(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2) \)**: - The softplus function is always non-negative. For \( f_{3}(x') \) to be less than or equal to zero, the argument of softplus must be less than or equal to \(-\log(2)\). This means that if \( C(x+\delta) = t \), \( f_{3}(x') \) might not satisfy \( f_{3}(x+\delta) \le 0 \), depending on the values of the logits. **This equation is incorrect.** 4. **Equation \( f_{4}(x') = (0.5 - F(x')_t)^+ \)**: - This function is non-negative and equals zero when \( F(x')_t \ge 0.5 \). If \( C(x+\delta) = t \), it may be that \( F(x')_t < 0.5 \) if the model is not confident, thus \( f_{4}(x+\delta) \) might be positive. **This equation is incorrect.** 5. **Equation \( f_{5}(x') = -\log(2 F(x')_t - 2) \)**: - This function is undefined for \( F(x')_t \le 1 \), and if \( F(x')_t < 1 \), it will not satisfy the condition \( f(x+\delta) \le 0 \) when \( C(x+\delta) = t \). **This equation is incorrect.** 6. **Equation \( f_{6}(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+ \)**: - Similar to \( f_{2}(x') \), this function measures the margin in the logits. If \( C(x+\delta) = t \), then \( Z(x')_t \) should be greater than \( \max_{i \ne t}( Z(x')_i) \), leading to \( f_{6}(x') \le 0 \). **This equation is correct.** 7. **Equation \( f_{7}(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \)**: - Similar to \( f_{3}(x') \), this function is non-negative due to the properties of the softplus function. It may not satisfy \( f(x+\delta) \le 0 \) when \( C(x+\delta) = t \). **This equation is incorrect.** In conclusion, the incorrect equations are \( 3, 4, 5, 7 \). Final answer:
1 | [3, 4, 5, 7] |
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
To determine which equations are incorrect according to the specification that \( C(x+\delta) = t \) if and only if \( f(x+\delta) \le 0 \), we need to analyze each equation in relation to this requirement. 1. **Equation \( f_1(x') = -\loss_{F,t}(x') + 1 \)**: - The cross-entropy loss \( \loss_{F,t}(x') \) is minimized (approaches 0) when the model predicts the correct class with high probability. Therefore, \( f_1(x') \le 0 \) means \( -\loss_{F,t}(x') + 1 \le 0 \) or \( \loss_{F,t}(x') \ge 1 \), which does not align with \( C(x+\delta) = t \) when \( x' \) is classified correctly. Hence, this equation is incorrect. 2. **Equation \( f_2(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+ \)**: - This equation captures the idea that \( f_2(x') \le 0 \) indicates that \( F(x')_t \) is greater than or equal to the maximum probability of any class other than \( t \). Thus, it correctly indicates that \( C(x+\delta) = t \) when \( f_2(x') \le 0 \). This equation is correct. 3. **Equation \( f_3(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t) - \log(2) \)**: - The softplus function is always non-negative, and \( f_3(x') \le 0 \) would imply that \( \max_{i \ne t}(F(x')_i) - F(x')_t \le \log(2) \). This does not necessarily mean that class \( t \) is the maximum class probability, hence it does not guarantee \( C(x+\delta) = t \). This equation is incorrect. 4. **Equation \( f_4(x') = (0.5 - F(x')_t)^+ \)**: - If \( F(x')_t > 0.5 \), then \( f_4(x') \le 0 \) implies that class \( t \) is indeed the predicted class. Therefore, it correctly indicates that \( C(x+\delta) = t \) when \( f_4(x') \le 0 \). This equation is correct. 5. **Equation \( f_5(x') = -\log(2 F(x')_t - 2) \)**: - The logarithm is defined only when \( 2F(x')_t - 2 > 0 \), which means \( F(x')_t > 1 \). This condition cannot happen since \( F(x')_t \) must be between 0 and 1. Hence, this function is not well-defined. This equation is incorrect. 6. **Equation \( f_6(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+ \)**: - Similar to \( f_2 \), this equation captures the idea that \( f_6(x') \le 0 \) indicates that \( Z(x')_t \) is greater than or equal to the maximum logit of any other class, which ensures that class \( t \) is predicted. Thus, this equation is correct. 7. **Equation \( f_7(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t) - \log(2) \)**: - Like \( f_3 \), the softplus function leads to non-negativity, and \( f_7(x') \le 0 \) implies conditions on logits that do not ensure that class \( t \) is the predicted class. Hence, it does not guarantee \( C(x+\delta) = t \) when \( f_7(x') \le 0 \). This equation is incorrect. Based on this analysis, the incorrect equations are \( f_1, f_3, f_5, \) and \( f_7 \). Thus, the final answer is:
1 | [1, 3, 5, 7] |
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
To determine whether each function \( f_i(x') \) satisfies the condition \( C(x + \delta) = t \) if and only if \( f(x + \delta) \le 0 \), we need to analyze each function in the context of the classification problem defined. ### Analysis of Each Equation 1. **Equation \( f_1(x') = -\loss_{F,t}(x') + 1 \)**: - The cross-entropy loss \( \loss_{F,t}(x') \) is non-negative and approaches 0 when the network classifies \( x' \) correctly (i.e., \( C(x') = t \)). - Therefore, \( f_1(x') \le 0 \) means \( -\loss_{F,t}(x') + 1 \le 0 \) or \( \loss_{F,t}(x') \ge 1 \), which is incorrect because we want \( f_1(x') \le 0 \) to imply correct classification. - **Incorrect**. 2. **Equation \( f_2(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+ \)**: - For this function to be \( \le 0 \), it implies that \( \max_{i \ne t}( F(x')_i) \le F(x')_t \), which means the predicted class \( t \) is greater than or equal to all others—thus \( C(x') = t \). - **Correct**. 3. **Equation \( f_3(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t) - \log(2) \)**: - The softplus function is always non-negative, and for \( f_3(x') \le 0 \), \( \max_{i \ne t}(F(x')_i) - F(x')_t \) must be less than or equal to \( -\log(2) \). This does not directly correspond to the condition for classification. - **Incorrect**. 4. **Equation \( f_4(x') = (0.5 - F(x')_t)^+ \)**: - For \( f_4(x') \le 0 \), it implies \( F(x')_t \ge 0.5 \), indicating \( C(x') = t \). - **Correct**. 5. **Equation \( f_5(x') = -\log(2 F(x')_t - 2) \)**: - This function is defined only when \( 2F(x')_t - 2 > 0 \) (i.e., \( F(x')_t > 1 \)), which is impossible since \( F(x')_t \) is a probability. Thus, \( f_5(x') \) will not be defined for valid outputs \( F(x')_t \). - **Incorrect**. 6. **Equation \( f_6(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+ \)**: - Similar to \( f_2 \), this indicates that \( Z(x')_t \) must be greater than or equal to the maximum of the other logits for \( f_6(x') \le 0 \), indicating correct classification. - **Correct**. 7. **Equation \( f_7(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t) - \log(2) \)**: - This has the same reasoning as \( f_3 \) and does not guarantee \( C(x') = t \) when \( f_7(x') \le 0 \). - **Incorrect**. ### Summary of Incorrect Equations Based on this analysis, the incorrect equations are \( f_1 \), \( f_3 \), \( f_5 \), and \( f_7 \). Thus, the final answer is:
1 | [1, 3, 5, 7] |
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
To determine which equations are incorrect based on the criterion that \( C(x+\delta) = t \) if and only if \( f(x+\delta) \le 0 \), we need to analyze each function \( f_i \) step by step. 1. **\( f_1(x') = -\loss_{F,t}(x') + 1 \)**: - The loss function \( \loss_{F,t}(x') \) is minimized when the classification is correct, meaning \( C(x') = t \). When \( C(x') = t \), the loss is low, and thus \( f_1(x') \) becomes high (approaching 1). Conversely, if \( C(x') \neq t \), the loss increases, and \( f_1(x') \) decreases. Thus, this equation does not satisfy the condition \( C(x+\delta) = t \) if and only if \( f(x+\delta) \le 0 \). - **Incorrect**. 2. **\( f_2(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+ \)**: - This function measures the difference between the highest class probability for the incorrect classes and the correct class probability. If \( C(x') = t \), then \( F(x')_t \) is maximized, and thus \( f_2(x') \) will be \( 0 \) (since \( \max_{i \ne t}(F(x')_i) < F(x')_t \)). This satisfies the condition, so it is **Correct**. 3. **\( f_3(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t) - \log(2) \)**: - Similar to \( f_2 \), this also measures the difference but applies a softplus and a logarithmic shift. If \( C(x') = t \), \( f_3(x') \) will be positive. If \( C(x') \neq t \), it will not necessarily be less than or equal to zero, thus violating the condition. - **Incorrect**. 4. **\( f_4(x') = (0.5 - F(x')_t)^+ \)**: - This function measures how far the probability of the correct class is below 0.5. If \( C(x') = t \) and \( F(x')_t \) is significantly above 0.5, \( f_4(x') \) will be zero. If \( C(x') \neq t \), \( F(x')_t \) could be low, leading to \( f_4(x') \) being positive. Thus, it does not satisfy the condition. - **Incorrect**. 5. **\( f_5(x') = -\log(2 F(x')_t - 2) \)**: - If \( C(x') = t \), \( F(x')_t \) could be greater than 1/2, making the argument of the log positive, resulting in \( f_5(x') \) being negative or zero. However, if \( C(x') \neq t \), \( F(x')_t \) could be less than 1/2, making the log undefined or positive. This violates the condition. - **Incorrect**. 6. **\( f_6(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+ \)**: - This function measures the logits rather than the softmax probabilities. If \( C(x') = t \), the correct class logit \( Z(x')_t \) will be greater than the highest incorrect logit, making \( f_6(x') = 0 \). If \( C(x') \neq t \), it can be positive, satisfying the condition. Thus, it is **Correct**. 7. **\( f_7(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t) - \log(2) \)**: - Similar reasoning applies here as in \( f_3 \). It also does not guarantee that \( f_7(x') \le 0 \) when \( C(x') = t \), thus the condition can be violated. - **Incorrect**. Based on the analysis, the incorrect equations are:
1 | [1, 3, 4, 5, 7] |
Resulting in output: True