interactive: scope-tree IR — structural scopes, region rendering, within-scope optimization#752
Merged
Merged
Conversation
A tree of scopes, each owning its IR: imports and exports are explicit edges (nothing crosses a boundary except through one), feedback variables are a first-class list, and items (operators and child scopes) are kept in dependency order so consumers read the structure rather than reconstruct it. The module doc carries the design: why a tree (positional Scope/EndScope boundaries forced every consumer to re-derive structure, and that reconstruction is where the explanation rewrite level errors lived), the two axes of a scope (structural region vs dynamic iteration coordinate), and the cross-scope name/identity model. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mirrors the source nesting: each `name: { .. }` becomes an owned child
scope; a scope imports its transitive free outer names (threaded through
every level on the path, so a grandchild reference works), exports the
fields its parent asks for via `scope::field`, and declares its `var`s
up front with binds closing the loops. Verified by structure tests on
reach (single scope, cross-scope ref), a two-level synthetic (transitive
import), and scc (depth two: outer containing fwd and bwd).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
render_tree walks the tree in item order: feedback vars are set up from the list (no scanning), each child scope opens a region_named (imports enter it, exports leave it structurally, then the dynamic coordinate pops via leave_dynamic), and binds close the loops. The dynamic PointStamp model is unchanged — regions add structure, not timestamp types. The Linear chain logic is factored into render_linear, shared with the flat renderer. The tree path is the default; FLAT=1 forces the flat path for A/B comparison, and --explain stays on the flat IR (the rewrite operates on it). DIAG=1 registers timely/DD logging and serves the diagnostics WebSocket on worker 0 (port 51371), which shows the per-scope regions. Verified: reach, scc (nested regions), stable, and kcore (applicative parser) match the flat renderer byte-for-byte at rounds 0 and 7. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Port of ddir_vec render_tree to the columnar backend, with the flat renderer Linear/Join/Reduce/Inspect bodies factored into shared fns so the two paths cannot drift. Tree is the default; FLAT=1 forces the flat path. Same corpus verification as ddir_vec, byte-for-byte on this backend. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`Scope::optimize` iterates three rewrites to a fixed point, recursing into child scopes: collapse an Arrange whose input is already an arrangement, fuse a Linear into its Linear input when it is the only consumer, and deduplicate structurally identical operators. All three are within-scope: a boundary is a semantic barrier, so nothing merges or moves across one. (The flat IR's dedup was position-blind and could merge across boundaries — sound only because the dynamic model has no structural nesting; here the structure makes the restriction automatic.) Use-counting reads the explicit refs; rewrites redirect refs and a final compaction drops dead items and remaps indices. lower_tree now arranges join/reduce inputs explicitly (as the flat lowering does), so identical arrangements are visible to dedup and shared at render — previously the tree renderer arranged per use. Both drivers call optimize and report op counts. Verified: corpus outputs match flat-optimized on both backends, and operator counts are identical like-for-like (reach 10 = 10, scc 31 = 31). Unit tests cover fusion, arrangement sharing across joins, arrange-of-reduce collapse, and recursion into nested scopes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the positional scope encoding (
Scope/EndScopemarkers in a flat node list) with a tree of scopes as the IR the backends actually run, and renders each source{ .. }as a real timely region.What
scope_ir: aProgramis its rootScope; a scope owns itsimports, first-class feedbackvars,items(operators and child scopes, in dependency order),binds, andexports. References resolve within their scope; nothing crosses a boundary except through an explicit import/export edge. The module doc carries the design rationale.lower_tree: AST → tree, mirroring the source nesting; a scope imports its transitive free outer names and exports the fields reached viascope::field. Join/reduce inputs are arranged explicitly.region_namedper scope — importsenterthe region, exportsleaveit, and the dynamicPointStampcoordinate rides inside (the dynamic model is unchanged; regions add structure, not timestamp types). Feedback variables come from thevarslist; items render in recorded order — no scanning, no topological re-analysis.Scope::optimize: within-scope linear fusion, structural dedup, and arrange-collapse to a fixed point, recursing into children. Notably within-scope by construction: the flat dedup was position-blind and could in principle merge across scope boundaries (sound only in the dynamic model); the tree makes the restriction automatic.Verification
fwd/bwdregions; stable; kcore via the applicative parser) matches the flat renderer byte-for-byte on both backends, at rounds 0 and 7, optimized on both sides.DIAG=1serves the diagnostics WebSocket; the per-scope regions are visible in the diagnostics UI.What this does not do
--explainstill runs on it (the explanation rewrite operates on flat programs), andFLAT=1forces the flat path for A/B comparison. Retirement comes with a follow-up PR that ports the explanation rewrite to the tree and deletes flat in the same change.Scopebecomes anIterative | Regionenum.🤖 Generated with Claude Code