Skip to content

fix(electric-db-collection): preserve persisted rows on progressive resume#1493

Open
pkudinov wants to merge 3 commits into
TanStack:mainfrom
pkudinov:codex/fix-progressive-persisted-resume
Open

fix(electric-db-collection): preserve persisted rows on progressive resume#1493
pkudinov wants to merge 3 commits into
TanStack:mainfrom
pkudinov:codex/fix-progressive-persisted-resume

Conversation

@pkudinov
Copy link
Copy Markdown

@pkudinov pkudinov commented Apr 24, 2026

Summary

Fixes #1478.

Progressive Electric collections wrapped with SQLite persistence can hydrate rows from the persisted store, then wipe them when the resumed Electric stream sends an up-to-date control message without replaying snapshot rows.

This changes progressive startup so a valid persisted Electric resume point is treated as already past the initial atomic-swap phase. That preserves hydrated persisted rows while still allowing normal progressive initial sync to use the existing buffer/truncate/apply behavior.

Root cause

Progressive mode uses the first up-to-date message to finish initial sync by truncating the collection and replaying buffered stream rows. With persisted resume metadata, the stream starts from a saved handle/offset, so it may legitimately send up-to-date with no old rows. The collection had already hydrated those rows from persistence, but the initial atomic swap ran anyway and truncated them.

Changes

  • Initialize progressive hasReceivedUpToDate when using a compatible persisted Electric resume point.
  • Add a regression test that hydrates persisted rows, resumes with saved Electric metadata, receives up-to-date without snapshot rows, and verifies rows remain in memory and persistence.
  • Add a patch changeset for @tanstack/electric-db-collection.

Validation

pnpm --filter @tanstack/electric-db-collection test -- --coverage.enabled false
pnpm --filter @tanstack/electric-db-collection build
pnpm exec eslint packages/electric-db-collection/src/electric.ts packages/electric-db-collection/tests/electric.test.ts --max-warnings=0

I also verified this patch in a real Next.js app using browser OPFS SQLite persistence: after resetting local SQLite, loading a persisted progressive Electric collection, and refreshing/warm-loading twice, the collection stayed populated instead of dropping to zero rows.

Summary by CodeRabbit

  • Bug Fixes

    • Progressive sync no longer truncates persisted collection rows when resuming; previously saved data is preserved during resume.
  • Tests

    • Added integration coverage to ensure persisted rows remain visible through progressive resume and up-to-date transitions.
  • Chores

    • Published a patch release entry to bump the collection package reflecting this fix.

@pkudinov pkudinov marked this pull request as ready for review April 24, 2026 06:49
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 21, 2026

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1184a710-8c48-4b46-8f53-a7adedafbba7

📥 Commits

Reviewing files that changed from the base of the PR and between 9528f1d and eec58b0.

📒 Files selected for processing (1)
  • packages/electric-db-collection/tests/electric.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/electric-db-collection/tests/electric.test.ts

📝 Walkthrough

Walkthrough

Initialize progressive resume state to reuse persisted resumes and avoid the initial atomic-swap buffering, update test harness to support seeded persisted rows and synchronous loaders, and add an integration test that verifies hydrated persisted rows survive a progressive resume and the subsequent up-to-date message.

Changes

Progressive Persisted Resume

Layer / File(s) Summary
Progressive resume atomic-swap skip logic
packages/electric-db-collection/src/electric.ts, .changeset/progressive-persisted-resume.md
hasReceivedUpToDate is initialized using syncMode === 'progressive' && canUsePersistedResume so the initial buffering/atomic-swap path is skipped when resuming from a valid persisted offset; changeset added for patch release.
Test harness and progressive resume validation
packages/electric-db-collection/tests/electric.test.ts
createPersistedAdapter updated to accept seeded persisted rows, make loadSubset/loadCollectionMetadata synchronous-resolving, apply collectionMetadataMutations and mutations (including tx.truncate) to in-memory Maps, and a new test asserts persisted hydrated rows remain after progressive resume and after up-to-date.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Suggested reviewers

  • samwillis

Poem

I’m a rabbit in the syncy glen,
I guard hydrated rows again.
No more truncation when streams resume,
I tuck each row safe in my roomy bloom. 🐇✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main fix: preserving persisted rows during progressive Electric collection resume.
Description check ✅ Passed The description thoroughly covers changes, root cause, validation steps, and aligns with the template by indicating code testing and changeset generation.
Linked Issues check ✅ Passed All requirements from issue #1478 are met: persisted rows are preserved on progressive resume by initializing hasReceivedUpToDate based on persisted resume compatibility, with regression test and changeset included.
Out of Scope Changes check ✅ Passed All changes directly address the linked issue: test harness updates support the new regression test, electric.ts fix implements the core solution, and changeset documents the patch release.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Persisted collection with progressive sync mode wiped on reconnect

1 participant