Skip to content

Fix BNGsim bridge silent fallbacks: strict routing error (#109) + Perl-faithful list-arg grammar (#110)#111

Open
wshlavacek wants to merge 7 commits into
RuleWorld:mainfrom
wshlavacek:fix/bngsim-bridge-routing-issues
Open

Fix BNGsim bridge silent fallbacks: strict routing error (#109) + Perl-faithful list-arg grammar (#110)#111
wshlavacek wants to merge 7 commits into
RuleWorld:mainfrom
wshlavacek:fix/bngsim-bridge-routing-issues

Conversation

@wshlavacek

@wshlavacek wshlavacek commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Two related fixes to the optional BNGsim bridge, both addressing cases where a model silently ran on the legacy BNG2.pl subprocess when the caller expected BNGsim. One hardens the routing contract; the other removes the most common trigger.

Closes #109
Closes #110

#109simulator='bngsim' must error, not silently fall back

Under simulator='bngsim', the routing bridge silently routed to the legacy BNG2.pl subprocess whenever the Python parser (modelapi.bngmodel) couldn't inspect a model's actions — e.g. an unrecognized simulate argument that bngmodel rejects but BNG2.pl tolerates. That violates the simulator='bngsim' contract ("use BNGsim, or error"): the caller got legacy output with no exception and no warning. A parity sweep over 895 models hit this on 9 models.

_classify_bngl_actions_for_bngsim() returned ROUTE_SUBPROCESS for the un-inspectable case unconditionally, with no reference to simulator.

Fix (bionetgen/core/tools/bngsim_bridge.py), at the classify_bngsim_route call site where simulator is in scope:

  • simulator='bngsim' + un-inspectable actions → ROUTE_ERROR, propagating the underlying parse reason so the caller learns why it couldn't route to BNGsim. Both consumers (main.py, runner.run) already raise BNGSimError(route.reason) on ROUTE_ERROR.
  • simulator='auto' keeps the subprocess fallback but emits a one-time WARNING per file.
  • simulator='subprocess' is unchanged (returns before action inspection).

To surface the reason, the routing parse now returns (actions, reason) through the memoized loader (_load_bngl_routing_actions; _load_bngl_actions_for_routing kept as a thin list-only wrapper for its other callers). The unavailable-BNGsim case already returned ROUTE_ERROR and is untouched.

#110 — action grammar: accept e-notation and trailing commas in list args

ActionList.define_parser (bionetgen/core/utils/utils.py) was stricter than BNG2.pl/Perl for list-valued arguments. arg_type_list matched elements with pp.Word(pp.nums + ".") (digits and . only) inside a plain pp.delimitedList, so it rejected:

  • scientific notation, e.g. par_scan_vals=>[2.3e-10,5.1e-10]
  • a trailing comma, e.g. par_scan_vals=>[1,2,3,]

Both are valid Perl that BNG2.pl parses and runs. When a model used either (commonly par_scan_vals on parameter_scan), bngmodel raised BNGParseError — which is exactly what triggered the #109 silent-fallback class. Real models hit this: RuleHub's Mitra2019/15-igf1r fits (e-notation and trailing commas) and Salazar-Cavazos2019 CHO_EGFR best-fit (e-notation).

Fix: broaden arg_type_list to use arg_type_expr (which already spans e/E and +/-), allow an empty list, and tolerate one optional trailing comma:

arg_type_list = (
    "[" + pp.Optional(pp.delimitedList(quote_word ^ arg_type_expr)) + pp.Optional(",") + "]"
)

Still rejects genuinely-malformed lists — double commas ([1,,2]) and unclosed lists ([1,2,).

#109 follow-up — strict mode also errors on unrecognized actions

#109's first fix closed the parse-failure hole (actions can't be inspected → None). But classify_bngsim_route returned the _classify_bngl_actions_for_bngsim result verbatim, so under an explicit simulator='bngsim' request a BNGL with an action the router doesn't recognize still silently ran on the legacy stack (its ROUTE_SUBPROCESS was never promoted to ROUTE_ERROR).

BNGsim can run such a model via the BNG2.pl backend hook — the router just can't confirm an unknown action is safe to delegate — so strict mode now surfaces ROUTE_ERROR with the reason instead of degrading silently. PLA and BNGsim-unsupported methods are genuine BNGsim incapabilities, so they stay on the subprocess route in every mode; auto keeps the legacy fallback for unrecognized actions too. simulator is threaded into _classify_bngl_actions_for_bngsim (default 'auto', back-compatible).

#109 follow-up — unknown simulation methods error instead of silent legacy

The router treated every BNGsim-unsupported method the same (ROUTE_SUBPROCESS), so a malformed/unknown method — a typo like oed, or quadratic — silently ran on the legacy stack in both auto and bngsim. BNG2.pl's own simulate() validates the method against its $METHODS hash (cvode/ssa/pla/psa/nf, plus the ode alias) and rejects anything else ("Simulation method '...' is not a valid option"), so this is faithful, not stricter than BNG2.pl.

Now split (_unsupported_method_route):

  • A valid BNG method this BNGsim build can't run — pla, or nf without an NFsim build — stays on the subprocess route (genuine/build incapability), in both modes.
  • A method outside the valid set errors in both modes, surfacing it.

_KNOWN_BNGL_METHODS = {ode, cvode, ssa, psa, pla, nf, rm} is taken verbatim from BNG2.pl's $METHODS keys (+ ode alias, + rm RuleMonkey extension), so valid methods like the cvode alias keep their existing legacy routing rather than newly erroring.

Note the deliberate asymmetry with actions: BNG2.pl supports many actions outside the router's curated list (setVolume, setOption, readSBML, quit, …), so auto keeps the legacy fallback for unrecognized actions (only bngsim errors). Methods are a closed, BNG2.pl-validated set; actions are open.

#109 follow-up — rm/RuleMonkey awareness; build incapabilities error under strict mode

method=>"rm" routes to BNGsim's RuleMonkey (via the rmnf rewrite + backend hook) only when RuleMonkey is installed — but the router marked rm supported unconditionally (_method_supported_by_bngsim_for_routing never checked RuleMonkey, unlike nf/NFsim), and classify_bngsim_route took no RuleMonkey arg. So an rm model on a BNGsim without RuleMonkey still classified as ROUTE_BNGL_BNGSIM, then run_bngl_with_bngsim (which hardcodes simulator="auto") silently fell back to legacy — invisibly even under simulator='bngsim'.

Now the classifier is RuleMonkey-aware (bngsim_has_rulemonkey threaded through), so the decision is made up front where strict mode is enforced, not in the simulator-blind runner. nf-without-NFsim and rm-without-RuleMonkey are treated uniformly as build incapabilities (_BNGSIM_OPTIONAL_COMPONENT_METHODS): simulator='bngsim' errors (surfacing "install NFsim/RuleMonkey"), auto falls back. This matches the existing direct BNG-XML nf behavior. pla stays subprocess in both modes (a categorical incapability, not a missing component); the cvode alias keeps its legacy route.

Why together

#110 makes these models parseable; #109 ensures that when something still isn't inspectable, bngsim errors instead of silently degrading. Shipping them together closes the silent-fallback gap end to end.

Tests

…spect actions

Under simulator='bngsim', the routing bridge silently fell back to the
legacy BNG2.pl subprocess engine whenever the Python parser
(modelapi.bngmodel) could not inspect a BNGL's actions — e.g. a typo'd or
unknown simulate argument (atoll, abstol, par_scan_vals) that bngmodel
rejects but BNG2.pl tolerates. This violated the simulator='bngsim'
contract ("use BNGsim, or error"): the caller got legacy output with no
exception and no warning. A parity sweep over 895 models hit this on 9
models.

_classify_bngl_actions_for_bngsim() returned ROUTE_SUBPROCESS for the
un-inspectable case unconditionally, with no reference to simulator.

Fix, at the classify_bngsim_route call site where simulator is in scope:
- simulator='bngsim' + un-inspectable actions -> ROUTE_ERROR, propagating
  the underlying parse reason so the caller learns why it couldn't route
  to BNGsim. Both consumers (main.py, runner.run) already raise
  BNGSimError(route.reason) on ROUTE_ERROR.
- simulator='auto' keeps the subprocess fallback but emits a one-time
  WARNING per file (deduped across the ~4 routing queries per run).
- simulator='subprocess' is unchanged (returns before action inspection).

To surface the reason, the routing parse now returns (actions, reason):
_parse_bngl_actions_for_routing and the memoized loader (renamed
_load_bngl_routing_actions; _load_bngl_actions_for_routing kept as a thin
list-only wrapper for its other callers) carry the parse message. The
unavailable-BNGsim case already returned ROUTE_ERROR and is untouched.

Adds tests covering the bngsim error, the auto one-time-warning fallback,
the subprocess no-inspect path, reason propagation through the cache, and
an end-to-end repro of the atoll-typo model from the issue.
…n args

ActionList.define_parser's list-valued argument grammar was stricter than
BNG2.pl/Perl: arg_type_list matched elements with `pp.Word(pp.nums + ".")`
(digits and '.' only) inside a plain `pp.delimitedList`, so it rejected

  - scientific notation, e.g. par_scan_vals=>[2.3e-10,5.1e-10]
  - a trailing comma, e.g. par_scan_vals=>[1,2,3,]

Both are valid Perl that BNG2.pl parses and runs. When a model used either
(commonly par_scan_vals on parameter_scan), modelapi.bngmodel raised
BNGParseError, so under simulator='bngsim' the bridge couldn't inspect the
actions and silently fell back to the legacy subprocess (the RuleWorld#109 class) —
the model never ran on bngsim. Real models hit this: RuleHub's
Mitra2019/15-igf1r fits and Salazar-Cavazos2019 CHO_EGFR best-fit.

Broaden arg_type_list to use arg_type_expr (which already spans e/E and
+/-), allow an empty list, and tolerate one optional trailing comma:

  arg_type_list = "[" + Optional(delimitedList(quote_word ^ arg_type_expr))
                      + Optional(",") + "]"

Still rejects genuinely-malformed lists (double commas `[1,,2]`, unclosed
`[1,2,`). Adds parametrized accept/reject tests over the issue's matrix and
the affected real-model forms.
…legacy

Follow-up to the RuleWorld#109 contract. classify_bngsim_route returned the
_classify_bngl_actions_for_bngsim result verbatim, so under an explicit
simulator='bngsim' request a BNGL with an action the router doesn't
recognize still silently ran on the legacy BNG2.pl stack (ROUTE_SUBPROCESS
was never promoted to ROUTE_ERROR). RuleWorld#109 only closed the parse-failure
(actions==None) hole; this closes the unrecognized-action hole.

BNGsim can run such a model via the BNG2.pl backend hook — the router just
can't confirm an unknown action is safe to delegate — so strict mode now
surfaces ROUTE_ERROR with the reason instead of degrading silently. PLA and
BNGsim-unsupported methods are genuine BNGsim incapabilities, so they stay
on the subprocess route in every mode (auto and bngsim alike); auto keeps
the legacy fallback for unrecognized actions too.

Threads `simulator` into _classify_bngl_actions_for_bngsim (default 'auto',
back-compatible) and promotes only the unrecognized-action branch. Adds
tests: bngsim unknown-action -> error, auto unknown-action -> subprocess,
and bngsim PLA / unsupported-method -> subprocess.
The router lumped "method BNGsim can't run" together as ROUTE_SUBPROCESS,
so a malformed/unknown method (a typo like 'oed', or 'quadratic') silently
ran on the legacy stack in auto AND bngsim modes. BNG2.pl's simulate()
validates the method against its $METHODS hash (cvode/ssa/pla/psa/nf, plus
the 'ode' alias) and itself rejects anything else with "Simulation method
'...' is not a valid option" — so routing an unknown method to legacy just
defers to a less clear BNG2.pl error.

Split the unsupported-method handling (_unsupported_method_route):
- A method in the valid BNG universe that this BNGsim can't run — pla, or nf
  without an NFsim build — stays on the subprocess route (genuine/build
  incapability), in both modes.
- A method outside the valid set is malformed and now errors in both auto
  and bngsim, surfacing it instead of silently degrading.

_KNOWN_BNGL_METHODS = {ode, cvode, ssa, psa, pla, nf, rm} is taken verbatim
from BNG2.pl's $METHODS keys (+ ode alias, + rm RuleMonkey extension), so
valid methods like the cvode alias keep their existing legacy routing rather
than newly erroring.

Unrecognized *actions* are deliberately NOT treated this way: BNG2.pl
supports many actions outside the router's curated list (setVolume,
setOption, readSBML, quit, ...), so auto keeps the legacy fallback for them
(only bngsim errors, per the prior commit). Methods are a closed,
BNG2.pl-validated set; actions are open.

Adds parametrized tests: unknown method -> error in both modes; cvode alias
and nf-without-nfsim -> subprocess (regression guards, no spurious errors).
method=>"rm" routes to BNGsim's RuleMonkey (via the rm->nf rewrite + backend
hook) only when RuleMonkey is installed. But the router marked rm supported
UNCONDITIONALLY — _method_supported_by_bngsim_for_routing never checked
RuleMonkey, unlike nf/NFsim — and classify_bngsim_route took no rulemonkey
arg. So an rm model on a BNGsim without RuleMonkey still classified as
ROUTE_BNGL_BNGSIM, then run_bngl_with_bngsim (which hardcodes
simulator="auto") silently fell back to legacy — invisibly even under
simulator='bngsim'. Same silent-downgrade class as RuleWorld#109.

Make the classifier RuleMonkey-aware (thread bngsim_has_rulemonkey through
classify_bngsim_route -> _classify_bngl_actions_for_bngsim ->
_method_supported_by_bngsim_for_routing) so the decision is made up front
where strict mode is enforced, not in the simulator-blind runner.

Treat nf-without-NFsim and rm-without-RuleMonkey uniformly as *build*
incapabilities (_BNGSIM_OPTIONAL_COMPONENT_METHODS): BNGsim could run them
with the component installed, so simulator='bngsim' errors (surfacing
"install NFsim/RuleMonkey") while auto falls back to legacy. This matches
the existing direct BNG-XML nf behavior (classify already errored there
under bngsim).

NOTE: this revises the BNGL nf-without-NFsim case shipped in the previous
commit (was subprocess in both modes) so nf and rm are consistent and the
strict-bngsim silent gap is closed for both. pla stays subprocess in both
modes (a categorical incapability, not a missing component); the cvode alias
keeps its legacy route. Updates tests accordingly and adds rm coverage.
test_atomic_supported_bngl_methods_use_bngsim forced bngsim_available=True
but let bngsim_has_nfsim/bngsim_has_rulemonkey default to the ambient module
values, which are False in any environment without a feature-complete BNGsim
(all CI runners). After "rm" routing was made RuleMonkey-aware, the rm case
correctly routes to subprocess there, so the test's hard-coded bngl-bngsim
expectation failed on all 15 matrix jobs.

Force bngsim_has_nfsim=True and bngsim_has_rulemonkey=True in the test so the
"supported methods route to BNGsim" assertion is environment-independent,
matching how it already forces bngsim_available. Verified by rerunning the
routing + method-defaults suites with both component flags monkeypatched to
False (the CI condition): 93 passed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant