6 Comments
Jan 29Liked by Eugene Cheah, RWKV

RWKV is the SOTA for non-Transformer architecture.

Expand full comment
Jan 29·edited Jan 29Liked by Eugene Cheah

caw caw! Congrats!

Is v4 the same as the paper? What's v5?

Expand full comment
author

The current paper is v4 : https://arxiv.org/abs/2305.13048

The paper for v5 is being written now, eta 1 month!

Expand full comment
Feb 1·edited Feb 1

> Trained on 1.1 Trillion Tokens across 100+ languages

is the dataset open publicly?

Expand full comment

I love the work you guys are doing. Wanna Collab?

Expand full comment