interactive: explanation rewrite on the scope tree; retire the flat IR (re-land #753)#754
Merged
Merged
Conversation
#753) * explain_tree: clone-with-lifts on the scope tree First piece of the explanation rewrite's tree port: `clone_into` clones a program's scopes into an output scope, and every nested scope additionally lift_iters and exports each internal collection — so every op and feedback var in the subtree has a host-visible form (user-iter coords folded into the value, innermost first) at the embedding level, cascading one lift per enclosing scope exit. This is the flat rewrite's `host` map become structure: "a scope exports its lifted internals". The flat version's positional scope tracking, pending pile, leave depth-offset arithmetic, and the Leave-aliasing fix-up pass all dissolve; the embedding depth is no longer a parameter (the renderer derives depth structurally, so a clone needn't know where it will sit). Verified: structural tests (host coverage of every site on scc, one lift per level on the re-export chains, identity-clone shape), and behaviorally via CLONE_RT=1 in ddir_vec — the cloned program's outputs match the plain run byte-for-byte across the corpus (reach, scc, stable, kcore). The lift chains execute but are not yet consumed; their content gets exercised when the reverse rules build pair tables from them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * explain_tree: the explanation transform on the scope tree The full rewrite: output root { sources, query input, witness clone } plus an iterative `explain` scope { demand-set vars, forward clone on demanded rows, reverse-tracing ops, demand exports }. The reverse dataflow is flat inside the explain scope by design (iteration time folds into values, the `folded` layout), so the per-op lookup rules port nearly verbatim. What the tree changes is boundary bookkeeping: a reference is *resolved* through explicit import/export edges to the value site it names, and the ordinary shape- preserving lookup against that site's host form injects or strips chain coordinates as depths dictate — flat's special Leave rule and its two-length Side both dissolve, and flat's latent cross-scope pass-through mismatch can't arise (routing adapts depth uniformly). ddir_vec --explain now runs the tree path by default (FLAT=1 --explain for the flat rewrite); flat gains a matching demand_set debug inspect for A/B. Verified: demand-sets MATCH flat byte-for-byte on shape-valid queries across reach, scc on two graphs (3 queries each), and stable. An earlier apparent scc over-inclusion traced to a SHAPE-INVALID query (key/val not matching the aggregate export); on such junk queries the two paths produce different junk - a loud shape check on the query input is the right guard (future: the shapes pass). On valid queries the paths agree everywhere tested. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * explain_tree: reject shape-invalid queries loudly `export_shape` exposes the first export's (k, v); ddir_vec asserts the QUERY against it at seeding. A mismatched query addresses nothing and yields plausible-looking junk demand (the source of a false "regression" during the flat/tree comparison); now it fails with the expected shape spelled out. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * scope_ir: structural tree dump; dump_explain on the tree pipeline `Program::dump` prints the tree as indented structural text — per scope, imports and vars first, items in order with Subs nested, then binds and exports. dump_explain now shows lower_tree output and the explain_tree rewrite (the flat printer goes with the flat IR). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Retire the flat IR: the scope tree is the only program representation Deletes, now that explain_tree removed the last reader: - explain.rs (the flat rewrite, builder, clone, reverse rules) — its tree port is explain_tree; the arity transfer functions move to ir.rs. - The flat Lowering and lower() in lower.rs (lower_tree remains). - ir::Node / ir::Program and its impl (dump, depths, optimize, rewrite, validate_lift_iter). ir.rs is now the shared row/op vocabulary: LinearOp, RowLike, field/condition evaluation, arity transfer. - render_program and the FLAT=1 branches in both backends; the tree path (regions, optimizer, --explain via explain_tree) is the only path. Noted: the flat validate_lift_iter discipline check (LiftIter result must not be referenced in its own scope) has no tree equivalent yet; LiftIter is rewrite-emitted today, with only the applicative parser able to surface it. Verified after the sweep: 15 lib tests, corpus output counts unchanged on both backends, --explain demand sets unchanged (scc pairs query). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * explain_tree: module doc describes the full transform Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
frankmcsherry
force-pushed
the
scope-tree-ir
branch
from
817018c to
777b375
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#753 merged into its stacked base branch (
scope-tree-ir) rather thanmaster-next— its base never auto-retargeted because the branch wasn't deleted when #752 merged. This PR carries that already-reviewed content tomaster-next; the diff is identical to #753's (the #752 portion is tree-identical to what already landed as aa642f4).See #753 for the full description and verification.
🤖 Generated with Claude Code