# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
1. If $f_{1}(x') \le 0$, then $-\loss_{F,t}(x') + 1 \le 0$. This implies that $\loss_{F,t}(x') \ge 1$, which means that $F(x')_t$ is very small. Since $F(x')$ is a probability distribution, this means that $F(x')_i$ is very large for some $i \ne t$. Therefore, $C(x') = t$ if and only if $f_{1}(x') \le 0$. 2. If $f_{2}(x') \le 0$, then $(\max_{i \ne t}( F(x')_i) - F(x')_t)^+ \le 0$. This implies that $\max_{i \ne t}( F(x')_i) \le F(x')_t$, which means that $F(x')_t$ is the largest element of $F(x')$. Therefore, $C(x') = t$ if and only if $f_{2}(x') \le 0$. 3. If $f_{3}(x') \le 0$, then $\text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2) \le 0$. This implies that $\max_{i \ne t}(F(x')_i) - F(x')_t \le \log(2)$, which means that $F(x')_t$ is much larger than $\max_{i \ne t}(F(x')_i)$. Therefore, $C(x') = t$ if and only if $f_{3}(x') \le 0$. 4. If $f_{4}(x') \le 0$, then $(0.5 - F(x')_t)^+ \le 0$. This implies that $0.5 - F(x')_t \le 0$, which means that $F(x')_t \ge 0.5$. Since $F(x')$ is a probability distribution, this means that $F(x')_i$ is very small for all $i \ne t$. Therefore, $C(x') = t$ if and only if $f_{4}(x') \le 0$. 5. If $f_{5}(x') \le 0$, then $-\log(2 F(x')_t - 2) \le 0$. This implies that $2 F(x')_t - 2 \ge 1$, which means that $F(x')_t \ge 1$. Since $F(x')$ is a probability distribution, this is impossible. Therefore, $f_{5}(x')$ is always positive, which means that $C(x') \ne t$ for all $x'$. 6. If $f_{6}(x') \le 0$, then $(\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+ \le 0$. This implies that $\max_{i \ne t}( Z(x')_i) \le Z(x')_t$, which means that $Z(x')_t$ is the largest element of $Z(x')$. Since $Z(x')$ is the output of the network before the softmax function, this means that $F(x')_t$ is also the largest element of $F(x')$. Therefore, $C(x') = t$ if and only if $f_{6}(x') \le 0$. 7. If $f_{7}(x') \le 0$, then $\text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \le 0$. This implies that $\max_{i \ne t}(Z(x')_i) - Z(x')_t \le \log(2)$, which means that $Z(x')_t$ is much larger than $\max_{i \ne t}(Z(x')_i)$. Since $Z(x')$ is the output of the network before the softmax function, this means that $F(x')_t$ is also much larger than $\max_{i \ne t}(F(x')_i)$. Therefore, $C(x') = t$ if and only if $f_{7}(x') \le 0$. The equations that are incorrect are those that do not satisfy the condition $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. These equations are $f_{5}(x')$ and $f_{7}(x')$. Therefore, the answer is $[5, 7]$.
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
- $f_1(x') = -\loss_{F,t}(x') + 1$: If $x'$ is classified correctly, then $\loss_{F,t}(x')$ is small, so $f_1(x')$ is large and positive. If $x'$ is classified incorrectly, then $\loss_{F,t}(x')$ is large, so $f_1(x')$ is small and negative. Thus, $f_1(x') \le 0$ if and only if $C(x+\delta) = t$. - $f_2(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+$: If $x'$ is classified correctly, then $F(x')_t$ is the largest element of $F(x')$, so $f_2(x')$ is zero. If $x'$ is classified incorrectly, then $F(x')_t$ is not the largest element of $F(x')$, so $f_2(x')$ is positive. Thus, $f_2(x') \le 0$ if and only if $C(x+\delta) = t$. - $f_3(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)$: If $x'$ is classified correctly, then $F(x')_t$ is the largest element of $F(x')$, so $f_3(x')$ is negative. If $x'$ is classified incorrectly, then $F(x')_t$ is not the largest element of $F(x')$, so $f_3(x')$ is positive. Thus, $f_3(x') \le 0$ if and only if $C(x+\delta) = t$. - $f_4(x') = (0.5 - F(x')_t)^+$: If $x'$ is classified correctly, then $F(x')_t$ is close to 1, so $f_4(x')$ is zero. If $x'$ is classified incorrectly, then $F(x')_t$ is not close to 1, so $f_4(x')$ is positive. Thus, $f_4(x') \le 0$ if and only if $C(x+\delta) = t$. - $f_5(x') = -\log(2 F(x')_t - 2)$: If $x'$ is classified correctly, then $F(x')_t$ is close to 1, so $f_5(x')$ is negative. If $x'$ is classified incorrectly, then $F(x')_t$ is not close to 1, so $f_5(x')$ is positive. Thus, $f_5(x') \le 0$ if and only if $C(x+\delta) = t$. - $f_6(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+$: This equation is incorrect. If $x'$ is classified correctly, then $Z(x')_t$ is the largest element of $Z(x')$, so $f_6(x')$ is zero. However, if $x'$ is classified incorrectly, then $Z(x')_t$ may not be the largest element of $Z(x')$, so $f_6(x')$ may be positive. Thus, $f_6(x') \le 0$ does not imply $C(x+\delta) = t$. - $f_7(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2)$: This equation is incorrect. If $x'$ is classified correctly, then $Z(x')_t$ is the largest element of $Z(x')$, so $f_7(x')$ is negative. However, if $x'$ is classified incorrectly, then $Z(x')_t$ may not be the largest element of $Z(x')$, so $f_7(x')$ may be positive. Thus, $f_7(x') \le 0$ does not imply $C(x+\delta) = t$. Therefore, the incorrect equations are $f_6$ and $f_7$.
1 | [6, 7] |
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
- $f_1(x') = -\loss_{F,t}(x') + 1$ If $C(x+\delta)=t$, then $F(x+\delta)_t$ is the largest value in $F(x+\delta)$. This means that $\loss_{F,t}(x+\delta)$ is minimized, which means that $f_1(x+\delta)$ is maximized. Therefore, $f_1(x+\delta) > 0$, which is not what we want. - $f_2(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+$ If $C(x+\delta)=t$, then $F(x+\delta)_t$ is the largest value in $F(x+\delta)$. This means that $f_2(x+\delta) = 0$, which is what we want. - $f_3(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)$ If $C(x+\delta)=t$, then $F(x+\delta)_t$ is the largest value in $F(x+\delta)$. This means that $f_3(x+\delta) = -\log(2) < 0$, which is what we want. - $f_4(x') = (0.5 - F(x')_t)^+$ If $C(x+\delta)=t$, then $F(x+\delta)_t$ is the largest value in $F(x+\delta)$. This means that $f_4(x+\delta) = 0$, which is what we want. - $f_5(x') = -\log(2 F(x')_t - 2)$ If $C(x+\delta)=t$, then $F(x+\delta)_t$ is the largest value in $F(x+\delta)$. This means that $f_5(x+\delta) = -\log(2) < 0$, which is what we want. - $f_6(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+$ If $C(x+\delta)=t$, then $Z(x+\delta)_t$ is the largest value in $Z(x+\delta)$. This means that $f_6(x+\delta) = 0$, which is what we want. - $f_7(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2)$ If $C(x+\delta)=t$, then $Z(x+\delta)_t$ is the largest value in $Z(x+\delta)$. This means that $f_7(x+\delta) = -\log(2) < 0$, which is what we want. Therefore, the equations that are incorrect according to the specification are:
1 | [1] |
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
**1.** $$f_1(x') = -\loss_{F,t}(x') + 1$$ If $C(x+\delta) = t$, then $\loss_{F,t}(x+\delta) = 0$. Therefore, $f_1(x+\delta) = 1$. This is not less than or equal to 0. **2.** $$f_2(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+$$ If $C(x+\delta) = t$, then $F(x+\delta)_t > F(x+\delta)_i$ for all $i \ne t$. Therefore, $f_2(x+\delta) = 0$. This is less than or equal to 0. **3.** $$f_3(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)$$ If $C(x+\delta) = t$, then $F(x+\delta)_t > F(x+\delta)_i$ for all $i \ne t$. Therefore, $f_3(x+\delta) < 0$. This is less than or equal to 0. **4.** $$f_4(x') = (0.5 - F(x')_t)^+$$ If $C(x+\delta) = t$, then $F(x+\delta)_t > 0.5$. Therefore, $f_4(x+\delta) = 0$. This is less than or equal to 0. **5.** $$f_5(x') = -\log(2 F(x')_t - 2)$$ If $C(x+\delta) = t$, then $F(x+\delta)_t > 0$. Therefore, $f_5(x+\delta) < 0$. This is less than or equal to 0. **6.** $$f_6(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+$$ If $C(x+\delta) = t$, then $Z(x+\delta)_t > Z(x+\delta)_i$ for all $i \ne t$. Therefore, $f_6(x+\delta) = 0$. This is less than or equal to 0. **7.** $$f_7(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2)$$ If $C(x+\delta) = t$, then $Z(x+\delta)_t > Z(x+\delta)_i$ for all $i \ne t$. Therefore, $f_7(x+\delta) < 0$. This is less than or equal to 0. Thus, the incorrect equations are [1].
Resulting in output: False
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
# Setup Which of the following equations are incorrect according to the specification? # Notation A neural network is a function $F(x) = y$ that accepts an input $x \in \mathbb{R}^n$ and produces an output $y \in \mathbb{R}^m$. The model $F$ also implicitly depends on some model parameters $\theta$; in our work the model is fixed, so for convenience we don't show the dependence on $\theta$. In this paper we focus on neural networks used as an $m$-class classifier. The output of the network is computed using the softmax function, which ensures that the output vector $y$ satisfies $0 \le y_i \le 1$ and $y_1 + \dots + y_m = 1$. The output vector $y$ is thus treated as a probability distribution, i.e., $y_i$ is treated as the probability that input $x$ has class $i$. The classifier assigns the label $C(x) = \arg\max_i F(x)_i$ to the input $x$. Let $C^*(x)$ be the correct label of $x$. The inputs to the softmax function are called \emph{logits}. We use the notation from Papernot et al. \cite{distillation}: define $F$ to be the full neural network including the softmax function, $Z(x) = z$ to be the output of all layers except the softmax (so $z$ are the logits), and \begin{equation*} F(x) = \softmax(Z(x)) = y. \end{equation*} A neural network typically \footnote{Most simple networks have this simple linear structure, however other more sophisticated networks have more complicated structures (e.g., ResNet \cite{he2016deep} and Inception \cite{szegedy2015rethinking}). The network architecture does not impact our attacks.} consists of layers \begin{equation*} F = \softmax \circ F_n \circ F_{n-1} \circ \cdots \circ F_1 \end{equation*} where \begin{equation*} F_i(x) = \sigma(\theta_i \cdot x) + \hat\theta_i \end{equation*} for some non-linear activation function $\sigma$, some matrix $\theta_i$ of model weights, and some vector $\hat\theta_i$ of model biases. Together $\theta$ and $\hat\theta$ make up the model parameters. Common choices of $\sigma$ are tanh \cite{mishkin2015all}, sigmoid, ReLU \cite{maas2013rectifier}, or ELU \cite{clevert2015fast}. In this paper we focus primarily on networks that use a ReLU activation function, as it currently is the most widely used activation function \cite{szegedy2015rethinking,springenberg2014striving,mishkin2015all,distillation}. # Specification: We define an objective function $f$ such that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. There are many possible choices for $f$: # The equations \begin{align*} f_{1}(x') &= -\loss_{F,t}(x') + 1\\ f_{2}(x') &= (\max_{i \ne t}( F(x')_i) - F(x')_t)^+\\ f_{3}(x') &= \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)\\ f_{4}(x') &= (0.5 - F(x')_t)^+\\ f_{5}(x') &= -\log(2 F(x')_t - 2)\\ f_{6}(x') &= (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+\\ f_{7}(x') &= \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2) \end{align*} where $s$ is the correct classification, $(e)^+$ is short-hand for $\max(e,0)$, $\text{softplus}(x) = \log(1+\exp(x))$, and $\loss_{F,s}(x)$ is the cross entropy loss for $x$. # Your task Consider each equation one by one. End your answer with a python list of numbers [1,2,3,4,5,6,7] for those that are wrong. Specifically, make sure that $C(x+\delta) = t$ if and only if $f(x+\delta) \le 0$. Think step by step about if this is true for each equation and then give your answer as a python list
1. $f_{1}(x') = -\loss_{F,t}(x') + 1$ If $C(x+\delta) = t$, then $F(x+\delta)_t \ge F(x+\delta)_i$ for all $i \ne t$. This implies that $-\loss_{F,t}(x+\delta) \le -\loss_{F,i}(x+\delta)$ for all $i \ne t$, which in turn implies that $f_{1}(x+\delta) \le 0$. Conversely, if $f_{1}(x+\delta) \le 0$, then $-\loss_{F,t}(x+\delta) + 1 \le 0$, which implies that $-\loss_{F,t}(x+\delta) \le -1$. This is a contradiction, so $C(x+\delta) \ne t$. Therefore, $f_{1}$ is incorrect. 2. $f_{2}(x') = (\max_{i \ne t}( F(x')_i) - F(x')_t)^+$ If $C(x+\delta) = t$, then $F(x+\delta)_t \ge F(x+\delta)_i$ for all $i \ne t$. This implies that $f_{2}(x+\delta) = 0$. Conversely, if $f_{2}(x+\delta) = 0$, then $\max_{i \ne t}( F(x+\delta)_i) - F(x+\delta)_t \le 0$. This implies that $F(x+\delta)_t \ge F(x+\delta)_i$ for all $i \ne t$, which in turn implies that $C(x+\delta) = t$. Therefore, $f_{2}$ is correct. 3. $f_{3}(x') = \text{softplus}(\max_{i \ne t}(F(x')_i) - F(x')_t)-\log(2)$ If $C(x+\delta) = t$, then $F(x+\delta)_t \ge F(x+\delta)_i$ for all $i \ne t$. This implies that $f_{3}(x+\delta) = -\log(2)$. Conversely, if $f_{3}(x+\delta) = -\log(2)$, then $\text{softplus}(\max_{i \ne t}(F(x+\delta)_i) - F(x+\delta)_t) = 0$. This implies that $\max_{i \ne t}( F(x+\delta)_i) - F(x+\delta)_t \le 0$. This in turn implies that $F(x+\delta)_t \ge F(x+\delta)_i$ for all $i \ne t$, which in turn implies that $C(x+\delta) = t$. Therefore, $f_{3}$ is correct. 4. $f_{4}(x') = (0.5 - F(x')_t)^+$ If $C(x+\delta) = t$, then $F(x+\delta)_t \ge F(x+\delta)_i$ for all $i \ne t$. This implies that $f_{4}(x+\delta) = 0$. Conversely, if $f_{4}(x+\delta) = 0$, then $0.5 - F(x+\delta)_t \le 0$. This implies that $F(x+\delta)_t \ge 0.5$. However, this does not necessarily imply that $C(x+\delta) = t$. For example, if $F(x+\delta)_i = 0.4$ for all $i$, then $f_{4}(x+\delta) = 0$, but $C(x+\delta) \ne t$. Therefore, $f_{4}$ is incorrect. 5. $f_{5}(x') = -\log(2 F(x')_t - 2)$ If $C(x+\delta) = t$, then $F(x+\delta)_t \ge F(x+\delta)_i$ for all $i \ne t$. This implies that $f_{5}(x+\delta) = -\log(2)$. Conversely, if $f_{5}(x+\delta) = -\log(2)$, then $2 F(x+\delta)_t - 2 \le 0$. This implies that $F(x+\delta)_t \le 1$. However, this does not necessarily imply that $C(x+\delta) = t$. For example, if $F(x+\delta)_i = 0.5$ for all $i$, then $f_{5}(x+\delta) = -\log(2)$, but $C(x+\delta) \ne t$. Therefore, $f_{5}$ is incorrect. 6. $f_{6}(x') = (\max_{i \ne t}( Z(x')_i) - Z(x')_t)^+$ If $C(x+\delta) = t$, then $Z(x+\delta)_t \ge Z(x+\delta)_i$ for all $i \ne t$. This implies that $f_{6}(x+\delta) = 0$. Conversely, if $f_{6}(x+\delta) = 0$, then $\max_{i \ne t}( Z(x+\delta)_i) - Z(x+\delta)_t \le 0$. This implies that $Z(x+\delta)_t \ge Z(x+\delta)_i$ for all $i \ne t$, which in turn implies that $C(x+\delta) = t$. Therefore, $f_{6}$ is correct. 7. $f_{7}(x') = \text{softplus}(\max_{i \ne t}(Z(x')_i) - Z(x')_t)-\log(2)$ If $C(x+\delta) = t$, then $Z(x+\delta)_t \ge Z(x+\delta)_i$ for all $i \ne t$. This implies that $f_{7}(x+\delta) = -\log(2)$. Conversely, if $f_{7}(x+\delta) = -\log(2)$, then $\text{softplus}(\max_{i \ne t}(Z(x+\delta)_i) - Z(x+\delta)_t) = 0$. This implies that $\max_{i \ne t}( Z(x+\delta)_i) - Z(x+\delta)_t \le 0$. This in turn implies that $Z(x+\delta)_t \ge Z(x+\delta)_i$ for all $i \ne t$, which in turn implies that $C(x+\delta) = t$. Therefore, $f_{7}$ is correct. The incorrect equations are $f_{1}$, $f_{4}$, and $f_{5}$.
1 | [1, 4, 5] |
Resulting in output: False