Skip to content

[DeepSpeed-Chat] Fix OOM issue in dataloader#841

Open
youkaichao wants to merge 1 commit into
deepspeedai:masterfrom
youkaichao:fix_oom
Open

[DeepSpeed-Chat] Fix OOM issue in dataloader#841
youkaichao wants to merge 1 commit into
deepspeedai:masterfrom
youkaichao:fix_oom

use online tokenizer to avoid oom

17c8243
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs