What is the hidden dimension of LLAMA-2 70b?
What is the hidden dimension of LLAMA-2 70b?
The hidden dimension (or hidden size) of LLAMA-2 70B is 8192. This parameter represents the dimensionality of the vectors used throughout the transformer architecture's layers and is one of the key architectural specifications that determines the model's capacity and computational requirements.
Testing if the previous output contains the string 8,192
: False
Testing if the previous output contains the string 8192
: True
What is the hidden dimension of LLAMA-2 70b?
What is the hidden dimension of LLAMA-2 70b?
The hidden dimension (also called model dimension or embedding dimension) of LLaMA-2 70B is 8192. This is the size of the vectors used throughout the model's transformer architecture for representing tokens and internal states, and it's one of the key architectural parameters that determines the model's capacity.
Testing if the previous output contains the string 8,192
: False
Testing if the previous output contains the string 8192
: True