Skip to content

fix(eval): preserve custom eval metadata#5908

Closed
he-yufeng wants to merge 2 commits into
google:mainfrom
he-yufeng:fix/eval-model-extra-metadata
Closed

fix(eval): preserve custom eval metadata#5908
he-yufeng wants to merge 2 commits into
google:mainfrom
he-yufeng:fix/eval-model-extra-metadata

Conversation

@he-yufeng
Copy link
Copy Markdown
Contributor

Summary

  • allow eval models to carry custom metadata fields instead of rejecting them as extras
  • cover both top-level EvalCase metadata and nested SessionInput metadata in serialization

Fixes #5906

To verify

  • PYTHONUTF8=1 uv run --with pytest pytest tests/unittests/evaluation/test_eval_case.py -q
  • PYTHONUTF8=1 uv run ruff check src/google/adk/evaluation/common.py tests/unittests/evaluation/test_eval_case.py
  • PYTHONUTF8=1 uv run pyright src/google/adk/evaluation/common.py
  • PYTHONUTF8=1 uv run python -m py_compile src/google/adk/evaluation/common.py tests/unittests/evaluation/test_eval_case.py
  • git diff --check

@adk-bot adk-bot added the eval [Component] This issue is related to evaluation label May 30, 2026
@rohityan rohityan self-assigned this Jun 5, 2026
@adk-bot
Copy link
Copy Markdown
Collaborator

adk-bot commented Jun 5, 2026

I have completed the analysis of Pull Request #5908 ("fix(eval): preserve custom eval metadata"), which resolves Issue #5906.

I verified the contributor's Google CLA signature and performed an architectural/style review against the current repository state:

  • Checked EvalBaseModel baseline which was indeed configured with extra="forbid".
  • Reviewed the PR's implementation of changing this configured value to extra="allow".
  • Assessed the newly added test test_eval_models_preserve_extra_metadata, finding that it adheres to ADK's testing standards.

Please review the comprehensive findings and detailed checklist in our generated report:

👉 pr_analysis_report.md

Key Decisions / Actions For You:

  • No blockers identified: The change is perfectly aligned, safe, and backwards-compatible.
  • Next Steps: You may approve and merge this pull request in the repository. Let me know if you would like

@rohityan
Copy link
Copy Markdown
Collaborator

rohityan commented Jun 5, 2026

/adk-pr-analyze

@adk-bot
Copy link
Copy Markdown
Collaborator

adk-bot commented Jun 5, 2026

I have completed the read-only triage and architectural review of Pull Request #5908 ("fix(eval): preserve custom eval metadata"), which resolves Issue #5906.

Please review the comprehensive findings and detailed checklist in our generated report:

👉 pr_analysis_report.md

Key Highlights & Decisions:

  • No architectural blockers found: The transition of EvalBaseModel's model_config from extra="forbid" to extra="allow" matches target Pydantic patterns perfectly and retains backwards compatibility.
  • Recommended Action: Approve (or Approve with Nits to request adding a behavioral docstring to the new test test_eval_models_preserve_extra_metadata in compliance with Rule 2 of the ADK Testing Style Guide).

@rohityan
Copy link
Copy Markdown
Collaborator

rohityan commented Jun 5, 2026

Hi @he-yufeng , Thank you for your contribution! We appreciate you taking the time to submit this pull request.

@rohityan rohityan added request clarification [Status] The maintainer need clarification or more information from the author and removed request clarification [Status] The maintainer need clarification or more information from the author labels Jun 5, 2026
@he-yufeng
Copy link
Copy Markdown
Contributor Author

Closing this older branch in favor of the cleaned replacement PR #5922 for the same issue and same fix. #5922 has been rebased onto current main and keeps the diff limited to the intended two files.

@he-yufeng he-yufeng closed this Jun 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

eval [Component] This issue is related to evaluation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

EvalBaseModel extra='forbid' rejects custom fields in session_input and eval cases

3 participants