What is the hidden dimension of LLAMA-2 70b?
What is the hidden dimension of LLAMA-2 70b?
The hidden dimension (often referred to as `d_model` or embedding dimension) of the **Llama-2 70B** model is **8192**. This value represents the size of the vectors used internally by the model to represent tokens and contextual information within its transformer layers. It's a key parameter indicating the model's capacity and complexity.
Testing if the previous output contains the string 8,192
: False
Testing if the previous output contains the string 8192
: True