6 Comments
User's avatar
petitlegumechien's avatar

RWKV is the SOTA for non-Transformer architecture.

Expand full comment
Nathan Lambert's avatar

caw caw! Congrats!

Is v4 the same as the paper? What's v5?

Expand full comment
Eugene Cheah's avatar

The current paper is v4 : https://arxiv.org/abs/2305.13048

The paper for v5 is being written now, eta 1 month!

Expand full comment
Vaclav Kosar's avatar

The latest iteration of this model EagleX is here: https://substack.recursal.ai/p/eaglex-17t-soaring-past-llama-7b

Expand full comment
Andreas's avatar

> Trained on 1.1 Trillion Tokens across 100+ languages

is the dataset open publicly?

Expand full comment
Devansh's avatar

I love the work you guys are doing. Wanna Collab?

Expand full comment