Skip to content

Expose source_tables on the plugin Query message#4473

Open
Prateeks16 wants to merge 2 commits into
sqlc-dev:mainfrom
Prateeks16:query-source-tables
Open

Expose source_tables on the plugin Query message#4473
Prateeks16 wants to merge 2 commits into
sqlc-dev:mainfrom
Prateeks16:query-source-tables

Conversation

@Prateeks16
Copy link
Copy Markdown

What

Adds a source_tables field to the plugin Query message: the list of base tables a query reads from, deduplicated and sorted ascending.

Why

Plugins currently learn a query's tables only through Query.columns[].table, which covers only tables that contribute a column to the output. A table referenced solely in a JOIN, a subquery, or a common table expression body is invisible to plugins even though the query depends on it. Plugins that build reactive/refetch logic (for example keyed off SQLite's update_hook) need a query's full read set, not just its output columns.

What's included

  • source_tables field on the Query proto (and regenerated codegen.pb.go).
  • The compiler resolves the read set from the statement AST: every base table in a FROM, a JOIN, or a subquery in any clause, including CTE bodies. Common table expression names and the target relations of INSERT/UPDATE/DELETE/TRUNCATE are excluded.
  • A unit test (internal/compiler) covering the example below plus deduplication, write-target exclusion, and the no-tables case. The existing JSON codegen goldens are regenerated to include the new field.

For this query:

WITH filtered_accounts AS (
    SELECT account_id FROM accounts WHERE accounts.space_id = $1
    AND NOT EXISTS (SELECT 1 FROM account_tags t WHERE t.account_id = accounts.account_id)
)
SELECT acc.* FROM accounts acc
JOIN filtered_accounts fa ON acc.account_id = fa.account_id
LEFT JOIN transactions t ON t.debit_account_id = acc.account_id;

source_tables is ["account_tags", "accounts", "transactions"]: accounts appears once (read in the CTE body and the outer query), account_tags appears though it is read only inside a NOT EXISTS subquery, transactions appears though it projects no column, and filtered_accounts is excluded as a CTE name.

Closes #4434.

Plugins receive Query.columns[].table, which only covers tables that
contribute a column to a query's output. Tables referenced only in a
JOIN, a subquery, or a common table expression body are invisible to
plugins, even though the query depends on them (issue sqlc-dev#4434).

Add a source_tables field to the plugin Query message listing every base
table a query reads from, deduplicated and sorted. Common table
expression names and the target relations of INSERT, UPDATE, DELETE, and
TRUNCATE statements are excluded.
Copilot AI review requested due to automatic review settings June 7, 2026 10:55
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds “source table” dependency information to compiled queries and exposes it through the plugin/codegen JSON output.

Changes:

  • Extend the plugin Query schema with source_tables.
  • Compute source table names from the AST during compilation and pass them into the plugin shim.
  • Update end-to-end JSON fixtures and add unit tests for source table extraction.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
protos/plugin/codegen.proto Adds source_tables field to the plugin Query message.
internal/compiler/source_tables.go Introduces AST walker to collect/dedupe/sort read tables and exclude write targets/CTEs.
internal/compiler/source_tables_test.go Adds unit tests covering CTEs, aliases, INSERT read-vs-write, and no-table queries.
internal/compiler/query.go Adds SourceTables field to compiler Query model with doc comment.
internal/compiler/parse.go Populates Query.SourceTables during parse.
internal/cmd/shim.go Copies SourceTables into the plugin Query output.
internal/endtoend/testdata/**/codegen.json Updates golden JSON to include source_tables.
Files not reviewed (1)
  • internal/plugin/codegen.pb.go: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/compiler/source_tables.go
Comment thread internal/compiler/source_tables.go
Comment thread internal/compiler/source_tables.go
Comment thread protos/plugin/codegen.proto
- Report schema-qualified names when a schema is present, and deduplicate on
  the qualified form so same-named tables in different schemas stay distinct.
- Return an empty slice rather than nil so the field serializes as [].
- Document the field on the proto message and the compiler Query model.
@Prateeks16
Copy link
Copy Markdown
Author

Thanks for the review. Addressed in the follow-up commit:

  • Schema qualification: source_tables now reports schema-qualified names when a schema is present and deduplicates on the qualified form, so tables of the same name in different schemas (e.g. audit.accounts vs accounts) stay distinct. Unqualified references keep their bare name, matching the issue's expected output. Added a unit test for this.
  • nil vs empty slice: the function now returns an empty slice instead of nil, so the field serializes as [] rather than null. Updated the no-tables test case accordingly.
  • Field documentation: added a doc comment on the proto field (and the compiler model) describing what is included/excluded, the ordering, and the qualification rule.
  • ParseTableName error handling: ParseTableName is total for a *ast.RangeVar (parseRelation reads the relation name directly and cannot return an error for that node type), so a fallback there would be unreachable. I left it as a guard rather than adding dead defensive code, but happy to plumb an error upward instead if you'd prefer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: expose source_tables on Query proto for plugin access to full query table dependencies

2 participants