Skip to content

Improve metadata OOM diagnostics#18039

Open
Caideyipi wants to merge 1 commit into
masterfrom
improve-metadata-oom-diagnostics
Open

Improve metadata OOM diagnostics#18039
Caideyipi wants to merge 1 commit into
masterfrom
improve-metadata-oom-diagnostics

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

This PR improves metadata/query-analysis OOM diagnostics when result-set column expansion or schema fetch runs out of memory.

Changes:

  • Track matched, expanded, generated, and schema-fetch deserialized result columns in MPPQueryContext.
  • Enrich MemoryNotEnoughException during metadata analysis with column counts, SLIMIT/SOFFSET, ALIGN BY DEVICE guidance, memory shortage, and estimated exceeded columns.
  • Record column counts during expression expansion and schema fetch batch deserialization.
  • Add MPPQueryContextTest coverage for enriched result-column and schema-fetch OOM messages, plus the no-context fallback path.

Verification

  • mvn spotless:apply -pl iotdb-core/datanode
  • git diff --check
  • mvn "-Ddevelocity.off=true" test -pl iotdb-core/datanode -Dtest=MPPQueryContextTest currently fails before running tests during datanode main-source compilation because of pre-existing/unrelated missing symbols in AstBuilder, DataNodeMemoryConfig, TableConfigTaskVisitor, DatasetHeaderFactory, ShowCreate*Task, and DataRegionConsensusImpl.

@sonarqubecloud

Copy link
Copy Markdown

@codecov

codecov Bot commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 76.92308% with 45 lines in your changes missing coverage. Please review.
✅ Project coverage is 41.32%. Comparing base (7f41f06) to head (99fe166).

Files with missing lines Patch % Lines
...e/iotdb/db/queryengine/common/MPPQueryContext.java 78.28% 38 Missing ⚠️
...lan/analyze/schema/ClusterSchemaFetchExecutor.java 0.00% 6 Missing ⚠️
...eryengine/common/schematree/ClusterSchemaTree.java 75.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18039      +/-   ##
============================================
- Coverage     41.42%   41.32%   -0.11%     
  Complexity      318      318              
============================================
  Files          5282     5282              
  Lines        369246   369439     +193     
  Branches      47786    47808      +22     
============================================
- Hits         152978   152666     -312     
- Misses       216268   216773     +505     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant