Skip to content

chore(dataflow/gemma): update dependencies and format code#14321

Open
XrossFox wants to merge 10 commits into
GoogleCloudPlatform:mainfrom
XrossFox:package-update-dataflow-gemma
Open

chore(dataflow/gemma): update dependencies and format code#14321
XrossFox wants to merge 10 commits into
GoogleCloudPlatform:mainfrom
XrossFox:package-update-dataflow-gemma

Conversation

@XrossFox

Copy link
Copy Markdown
Contributor

Description

  • Update tensorflow base image to 2.20.0-gpu and beam sdk to 3.11/2.74.0
  • Update apache_beam, keras, keras_nlp, and protobuf dependencies
  • Update test dependencies including google-cloud-aiplatform, storage, and pytest
  • Format custom_model_gemma.py and e2e_test.py
  • Update ignored python versions in noxfile_config.py

Fixes b/521868825

Checklist

Testing

  • I have tested this change on a live environment and verified it works as intended.

Compliance & Style


Post-Approval Actions

  • Please merge this PR for me once it is approved

@product-auto-label product-auto-label Bot added api: dataflow Issues related to the Dataflow API. samples Issues that are directly related to samples. labels Jun 16, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the Gemma Dataflow sample by changing the Python SDK base image from version 3.14 to 3.11, upgrading TensorFlow to 2.20.0, and updating Apache Beam to 2.74.0. It also upgrades several runtime and test dependencies and applies code formatting across multiple files. Feedback on the changes suggests updating noxfile_config.py to ignore Python versions 3.12, 3.13, and 3.14 to align with the target Python 3.11 environment and prevent redundant test runs.

Comment thread dataflow/gemma/noxfile_config.py Outdated
XrossFox added 2 commits June 22, 2026 22:35
 - Update tensorflow base image to 2.20.0-gpu and beam sdk to 3.11/2.74.0
 - Update apache_beam, keras, keras_nlp, and protobuf dependencies
 - Update test dependencies including google-cloud-aiplatform, storage, and pytest
 - Format custom_model_gemma.py and e2e_test.py
 - Update ignored python versions in noxfile_config.py
@XrossFox XrossFox force-pushed the package-update-dataflow-gemma branch from 24624a0 to 7534737 Compare June 23, 2026 21:47
@XrossFox

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the project to Python 3.11, bumps dependency versions in the Dockerfile and requirements files, and modifies the end-to-end test configuration. The reviewer identified critical issues where several specified Docker image and PyPI package versions do not exist, which will cause build and installation failures. Additionally, changing the GCS test bucket to a hardcoded personal bucket in the end-to-end tests will break the CI/CD pipeline.

Comment thread dataflow/gemma/Dockerfile
Comment thread dataflow/gemma/Dockerfile
Comment thread dataflow/gemma/requirements.txt
Comment thread dataflow/gemma/requirements-test.txt
Comment thread dataflow/gemma/e2e_test.py Outdated
@XrossFox

Copy link
Copy Markdown
Contributor Author

I have finally managed to get this sample running, it is working as expected:

image

(working on linting issues)

@XrossFox XrossFox marked this pull request as ready for review June 23, 2026 23:41
@XrossFox XrossFox requested review from a team as code owners June 23, 2026 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: dataflow Issues related to the Dataflow API. samples Issues that are directly related to samples.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant