What is the hidden dimension of LLAMA-2 70b?
What is the hidden dimension of LLAMA-2 70b?
The hidden dimension of LLaMA-2 70B refers to the size of the hidden layers in the model's architecture. For LLaMA-2 70B, the hidden dimension is **8192**. This means that each hidden layer in the transformer architecture has 8192 neurons or units. This large hidden dimension contributes to the model's capacity to process and generate complex language patterns.
Testing if the previous output contains the string 8,192
: False
Testing if the previous output contains the string 8192
: True