chore(regression): remove legacy central regression framework#757
Open
lewisjared wants to merge 2 commits into
Open
chore(regression): remove legacy central regression framework#757lewisjared wants to merge 2 commits into
lewisjared wants to merge 2 commits into
Conversation
Replace the old central regression framework (the DiagnosticValidator / ExecutionRegression helpers and the diagnostic_validation, execution_regression and regression_data_dir fixtures) with the RFC 0005 native baselines managed by `ref test-cases`, leaving a single regression mechanism. - Delete the central tests/test-data/regression/ tree (~79 MB) - Drop the Framework-A test_diagnostics / test_build_results tests from the PMP and ESMValTool integration suites; the Framework-B test-case scaffolding (collect_test_case_params, test_validate_test_case_regression, test_run_test_cases) is retained and skips cleanly with no committed baselines - Add a slim solved_definition_factory fixture to replace the only non-regression use of the old validator, used by the PMP command-building unit tests - Relocate the PDO unit-test fixture into the climate-ref-pmp package - Update documentation and the pre-commit exclude list PMP and ESMValTool native baselines land in separate PRs.
Codecov Report✅ All modified and coverable lines are covered by tests.
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 11 files with indirect coverage changes 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Removes the legacy central regression-testing framework ("Framework A") in favour of the RFC 0005 native baselines ("Framework B"), so the repository has a single regression mechanism. The PMP and ESMValTool native baselines are landing in separate PRs and slot directly into the test-case scaffolding kept here.
Removed
tests/test-data/regression/baseline tree (1355 files, ~79 MB).DiagnosticValidatorandExecutionRegression, and thediagnostic_validation,execution_regression, andregression_data_dirfixtures fromclimate_ref.conftest_plugin.test_diagnostics/test_build_resultstests (and their module-level diagnostic lists) from the PMP and ESMValTool integration suites.Kept / adjusted
collect_test_case_params,test_validate_test_case_regression, andtest_run_test_casesin each provider. With no committed baselines these skip cleanly today; when the PMP and ESMValTool minted datasets land in their PRs they slot straight in.solved_definition_factoryfixture to replace the only non-regression use of the old validator (building a definition from the sample-data catalog), used by the PMP command-building unit tests.packages/climate-ref-pmp/tests/unit/test-data/pdo-example/.Documentation
adding_custom_diagnostics.mdto useref test-cases.execution_regressionsubsection fromtesting-diagnostics.md.--force-regencapture note indevelopment.md.tests/test-data/README.md..pre-commit-config.yaml.Verified locally: ruff, mypy (139 files), full test collection (2317 tests, 0 errors), and the targeted PMP unit tests (5 passed).
Checklist
Please confirm that this pull request has done the following:
changelog/