feat: migrate smoke testing scripts to python

.github/workflows/base-std-fork-tests.yml

@@ -122,6 +122,12 @@ jobs:
           "$BASE_ANVIL_DIR/target/release/anvil" --version
           "$BASE_ANVIL_DIR/target/release/forge" --version
 
+      - name: Set up fork-test runner venv
+        shell: bash
+        # The runner is `python -m fork` (web3 only); reuse the smoke venv target with the
+        # runner's preinstalled python3 instead of the python3.13 default.
+        run: make smoke-setup PYTHON=python3
+
       - name: Run base-std fork tests
         id: fork_tests
         shell: bash
@@ -131,7 +137,7 @@ jobs:
           ANVIL_BIN="$BASE_ANVIL_DIR/target/release/anvil" \
             FORGE_BIN="$BASE_ANVIL_DIR/target/release/forge" \
             ANVIL_LOG="$RUNNER_TEMP/base-std-anvil.log" \
-            ./script/run-fork-tests.sh 2>&1 | tee "$RUNNER_TEMP/fork-test-output.txt"
+            make fork-tests 2>&1 | tee "$RUNNER_TEMP/fork-test-output.txt"
           echo "fork_tests_exit=${PIPESTATUS[0]}" >> "$GITHUB_OUTPUT"
         continue-on-error: true
 

FORK_TESTING.md

@@ -57,14 +57,21 @@ Clone two repos as siblings (base/base is fetched automatically by cargo):
 ```
 
 If your layout differs, set `ANVIL_BIN` and `FORGE_BIN` env vars to
-override the script's defaults.
+override the runner's defaults.
 
 Install Rust + the fast linker, plus stock foundry (for `cast`):
 
 ```bash
 curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal
 brew install lld  # macOS; Linux uses mold per base-anvil's .cargo/config.toml
-curl -L https://foundry.paradigm.xyz | bash && foundryup  # stock foundry, for `cast` (used by run-fork-tests.sh)
+curl -L https://foundry.paradigm.xyz | bash && foundryup  # stock foundry, for `cast` (the manual probe below)
+```
+
+The runner itself is Python ([`script/fork/`](script/fork/)), driven by `web3`.
+Create its venv once (shared with the smoke suite):
+
+```bash
+make smoke-setup
 ```
 
 Build the patched forge + anvil (~30 min first build, incremental after).
@@ -79,10 +86,10 @@ cargo build --release -p anvil -p forge
 
 ```bash
 cd ~/code/base-std
-./script/run-fork-tests.sh
+make fork-tests
 ```
 
-The script:
+The runner ([`script/fork/`](script/fork/)):
 
 1. Launches `anvil --base --base-activation-admin 0x9965507D...` on port 8546.
 2. Funds + impersonates the activation admin, sends `activate(bytes32)` for
@@ -91,10 +98,10 @@ The script:
    http://localhost:8546`.
 4. Tears down anvil.
 
-Forward any `forge test` flag through the script:
+Forward any `forge test` flag through `ARGS`:
 
 ```bash
-./script/run-fork-tests.sh -vvvv --match-test test_transfer_success_debitsSender
+make fork-tests ARGS="-vvvv --match-test test_transfer_success_debitsSender"
 ```
 
 ## Exercising the inactive-feature dispatch path
@@ -106,8 +113,8 @@ feature is active), set `SKIP_ACTIVATE` to a comma-separated list of feature
 names (or raw `0x` ids) to leave un-activated:
 
 ```bash
-SKIP_ACTIVATE=POLICY_REGISTRY ./script/run-fork-tests.sh \
-    --match-contract PolicyRegistryDispatchInactive
+SKIP_ACTIVATE=POLICY_REGISTRY make fork-tests \
+    ARGS="--match-contract PolicyRegistryDispatchInactive"
 ```
 
 The inactive-dispatch tests (`test/unit/PolicyRegistry/dispatch_inactive.t.sol`)
@@ -118,8 +125,8 @@ activation gate rather than masked by `FeatureNotActivated`) encodes a fix that
 isn't in the Rust impl yet, so it's gated behind `POLICY_DISPATCH_FIX`:
 
 ```bash
-SKIP_ACTIVATE=POLICY_REGISTRY POLICY_DISPATCH_FIX=true ./script/run-fork-tests.sh \
-    --match-contract PolicyRegistryDispatchInactive
+SKIP_ACTIVATE=POLICY_REGISTRY POLICY_DISPATCH_FIX=true make fork-tests \
+    ARGS="--match-contract PolicyRegistryDispatchInactive"
 ```
 
 Run it that way against a build that carries the dispatch-ordering fix (e.g. via
@@ -149,10 +156,12 @@ Skim the new commits for new precompile addresses, feature IDs, or ABI
 changes (`crates/common/precompiles/src/activation/storage.rs` for
 `FEATURE_*` constants).
 
-**Step 2: if new feature IDs were added**, append them to the
-`FEATURE_IDS` array in `script/run-fork-tests.sh`. The script must activate
-every gated feature before tests run; otherwise the feature's calls revert
-`FeatureNotActivated`.
+**Step 2: if new feature IDs were added**, add them to the derived feature set
+the runner activates: the canonical `FEATURE_*` keccak constants in
+[`script/smoke/config.py`](script/smoke/config.py) and the `FEATURES` table in
+[`script/fork/__main__.py`](script/fork/__main__.py) (which reuses those
+constants). The runner must activate every gated feature before tests run;
+otherwise the feature's calls revert `FeatureNotActivated`.
 
 **Step 3: if new precompiles were added** (a new `*Precompile::install`
 call appeared in `base/crates/common/precompiles/src/provider.rs`):
@@ -170,7 +179,7 @@ Then rerun `./script/bump-base.sh <ref>` to rebuild against the new pin.
 
 ```bash
 cd ~/code/base-std
-./script/run-fork-tests.sh
+make fork-tests
 ```
 
 **Step 5: triage the deltas.** Compare against the last run's failure
@@ -223,23 +232,24 @@ block when you're done iterating.
 ## Common failure modes & fixes
 
 **`anvil binary not found`** — run `cargo build --release -p anvil` in
-`base-anvil/`. Or `ANVIL_BIN=/abs/path ./script/run-fork-tests.sh`.
+`base-anvil/`. Or `ANVIL_BIN=/abs/path make fork-tests`.
 
 **`port 8546 is already in use`** — `pkill -f "base-anvil/target/.*/anvil"`
-or `PORT=8547 ./script/run-fork-tests.sh`.
+or `PORT=8547 make fork-tests`.
 
 **`anvil exited during startup`** — check `/tmp/anvil.log`. Usual cause:
 rust build is stale after a base/base change; rebuild.
 
 **Hundreds of `EvmError: Revert` with `gas: 0` in `setUp`** — either the
 `LIVE_PRECOMPILES` env var wasn't set (BaseTest etched the mocks over the
 precompile addresses) or `[profile.fork] base = true` is missing from
-`foundry.toml` (forge isn't installing the precompiles). The script sets
+`foundry.toml` (forge isn't installing the precompiles). The runner sets
 both, but if you're invoking forge directly, set both.
 
 **`FeatureNotActivated(bytes32)` revert payload** — a new gated feature
-landed in base/base. Add its ID to `FEATURE_IDS` in
-`script/run-fork-tests.sh`. The payload's 32-byte tail IS the feature ID
+landed in base/base. Add its ID to the derived feature set (see Step 2 above:
+`FEATURE_*` in `script/smoke/config.py` + the `FEATURES` table in
+`script/fork/__main__.py`). The payload's 32-byte tail IS the feature ID
 (grep `base/crates/common/precompiles/src/activation/storage.rs` for the
 matching `FEATURE_*` constant).
 
@@ -275,7 +285,7 @@ If this returns garbage / fails, the fork's build is broken or out of date.
 
 | Thing | Path | Notes |
 |---|---|---|
-| The test runner script | `script/run-fork-tests.sh` | bash; takes forge args through `$@` |
+| The test runner | `script/fork/` (`make fork-tests`) | Python + web3; forwards forge args through `ARGS` |
 | Forge profile config | `foundry.toml`, `[profile.fork]` | `base = true` enables Rust precompile dispatch |
 | Skip-etch logic | `test/lib/BaseTest.sol` | guarded by `LIVE_PRECOMPILES` env var |
 | Slot assertions | `test/unit/**/*.t.sol`, `test/lib/mocks/Mock*Storage.sol` | `vm.load`-based slot-layout assertions paired with surface tests |

Makefile

@@ -3,10 +3,12 @@ LOAD_ENV = pre=$$(export -p); set -a; [ -f .env ] && . ./.env; set +a; eval "$$p
 
 PYTHON ?= python3.13
 VENV = script/smoke/.venv
-# `smoke` is the package at script/smoke/, so its parent (script) is on the path.
+# `smoke` and `fork` are packages under script/, so script/ is on the path. Both share the one venv
+# (web3 is their only dependency); `make smoke-setup` provisions it.
 SMOKE_RUN = $(LOAD_ENV) PYTHONPATH=script $(VENV)/bin/python -m smoke
+FORK_RUN = $(LOAD_ENV) PYTHONPATH=script $(VENV)/bin/python -m fork
 
-.PHONY: build coverage smoke smoke-all smoke-factory smoke-asset smoke-stablecoin smoke-policy smoke-invariants smoke-setup
+.PHONY: build coverage smoke smoke-all smoke-factory smoke-asset smoke-stablecoin smoke-policy smoke-invariants smoke-setup fork-tests
 
 # Generate an lcov coverage report and open it in the browser.
 # Scoped to src/ and test/lib/mocks/ (excludes test runner files and the smoke probe helper).
@@ -22,6 +24,14 @@ smoke-setup:
 	$(VENV)/bin/python -m pip install --upgrade pip
 	$(VENV)/bin/python -m pip install -r script/smoke/requirements.txt
 
+# Run the unit suite against a local anvil with Base's Rust precompiles, cross-validating the Solidity
+# reference against the live Rust impl. Needs the patched anvil+forge from the base-anvil fork (see
+# script/fork/__main__.py for env vars). Forward forge args via ARGS, e.g.
+#   make fork-tests ARGS="-vvvv --match-test test_transfer_success_debitsSender"
+#   make fork-tests ARGS="--match-contract PolicyRegistryDispatchInactive" SKIP_ACTIVATE=POLICY_REGISTRY
+fork-tests:
+	@$(FORK_RUN) $(ARGS)
+
 # Compile the contracts.
 build:
 	forge build

script/fork/README.md

@@ -0,0 +1,63 @@
+# fork-test runner
+
+`python -m fork [forge test args...]` runs the base-std unit suite against a
+**local anvil that dispatches Base's Rust precompiles**, cross-validating the
+Solidity reference against the live Rust impl from `base/base`. It is the Python
+port of the former `script/run-fork-tests.sh`.
+
+Both binaries (anvil + forge) must come from the **base-anvil fork** of
+foundry-rs, which adds a `--base` flag that installs the B-20 precompile suite
+into the EVM. Stock foundry binaries will not work.
+
+## What it does
+
+1. Discovers the patched `anvil` + `forge` (env override or the base-anvil
+   `target/release` → `target/debug` default layout).
+2. Launches anvil on `$PORT` with `--base`, waits for the RPC to come up, and
+   tears it down on exit (pass, fail, or crash).
+3. Funds + impersonates the activation admin and calls
+   `ActivationRegistry.activate(bytes32)` for each gated feature.
+4. Runs `forge test --fork-url` under `FOUNDRY_PROFILE=fork` +
+   `LIVE_PRECOMPILES=true`, forwarding any extra args, and propagates forge's
+   exit code.
+
+The gated feature ids are **derived** (`keccak` of the canonical feature names)
+by importing them from the sibling [`smoke` package's `config.py`](../smoke/config.py),
+so there is a single source of truth shared with the smoketest — no hand-kept
+hex table to drift from the Solidity `ActivationRegistryFeatureList` / Rust
+`storage.rs`.
+
+## Running
+
+```bash
+make smoke-setup          # one-time: create the shared venv + install web3 (shared with `make smoke`)
+
+make fork-tests                                                   # whole suite
+make fork-tests ARGS="-vvvv --match-test test_transfer_success"  # scope + verbosity
+make fork-tests ARGS="--match-contract PolicyRegistryDispatchInactive" SKIP_ACTIVATE=POLICY_REGISTRY
+```
+
+Or directly (the Makefile just sources `.env` and sets `PYTHONPATH=script`):
+
+```bash
+PYTHONPATH=script script/smoke/.venv/bin/python -m fork --match-test test_transfer_success
+```
+
+## Environment
+
+| Var | Default | Meaning |
+| --- | --- | --- |
+| `ANVIL_BIN` | `../base-anvil/target/release/anvil` (→ `debug`) | patched anvil binary |
+| `FORGE_BIN` | `forge` next to `ANVIL_BIN` | patched forge binary |
+| `PORT` | `8546` | local RPC port for anvil |
+| `ACTIVATION_ADMIN` | `0x9965…A4dc` | address authorized to activate features |
+| `ANVIL_LOG` | `/tmp/anvil.log` | anvil stdout/stderr log path |
+| `SKIP_ACTIVATE` | _(none)_ | comma-separated feature names or `0x` ids to leave un-activated (exercises the inactive-feature dispatch path); matched case-insensitively |
+
+## Exit codes
+
+| Code | Meaning |
+| --- | --- |
+| `0` | all targeted tests pass |
+| `1` | at least one targeted test fails — the output **is** the cross-validation signal |
+| `2` | environment problem (missing binary, port in use, anvil failed to start, activation tx failed) |

script/fork/__init__.py

@@ -0,0 +1 @@
+"""Fork-test runner: drive the base-std unit suite against a local anvil that dispatches Base's Rust precompiles."""