Skip to content

ADR-060: Bounded Handler Execution Defaults

Status

Accepted Date: 2026-08-25

Context

Security-audit finding F-DP5 (CWE-400, threat-model scenario S7, risk 12): command handlers default to timeout=None (unbounded), so one hung handler stalls its entity's FIFO dispatch worker indefinitely — no log line, no error publish, no availability change. The device-context command path (ctx.on_command / sub-entity handlers) has no timeout mechanism at all, and the @app.periodic loop likewise invokes its handler unbounded, silently freezing the background task. This is the same permanent-wedge class as the production incident that motivated the ADR-024 amendment, which already added a per-invocation timeout backstop for @app.telemetry (auto-default interval × 1.0, None disables).

The asymmetry is now a liability: telemetry invocations are bounded by default while command and periodic invocations are not, even though commands occupy the entity worker that also serializes state updates, and a wedged periodic loop produces a silent zombie task. A hung MQTT-facing handler additionally keeps the broker connection's command path busy, which an attacker can compound by repeatedly issuing commands once they discover a slow handler (amplified by the fact that command payloads are attacker-chosen within the inbound size cap).

Commands have no natural period to derive a bound from (unlike telemetry's interval), so the design must pick an explicit constant, and because a bounded default changes existing behavior (legitimately slow handlers will now be cancelled), the change needs a documented opt-out that matches the mental model users already learned from the ADR-024 amendment.

Decision

Extend the ADR-024 three-state timeout backstop to all remaining handler surfaces. Commands get an explicit bounded default of 30 seconds; periodic tasks inherit the telemetry rule (interval × 1.0); device-context command handlers get the same bounded default with a per-handler override. All three surfaces share one semantics table: omitted → bounded default, explicit float → used as-is, None → explicitly unbounded (opt-out), callable spec → resolved against Settings at bootstrap.

# Command: bounded by default (30 s), override or disable explicitly
@app.command("reboot", timeout=120.0)   # slow bus operation
async def reboot(cmd: Command) -> dict[str, object]: ...

@app.command("fast-toggle")             # omitted -> 30 s backstop
async def fast_toggle(cmd: Command) -> dict[str, object]: ...

@app.command("legacy-longrun", timeout=None)   # explicit opt-out
async def legacy_longrun(cmd: Command) -> dict[str, object]: ...

# Periodic: auto-default interval x 1.0 (telemetry rule, ADR-024)
@app.periodic("cache-refresh", interval=60)
async def refresh(cache: CachePort) -> None: ...          # bounded at 60 s

# Device/sub-entity context handlers: same 30 s default
@ctx.on_command("calibrate", timeout=90.0)
async def calibrate(sub_topic: str | None, payload: str) -> None: ...

Decision Drivers

  • F-DP5/S7 (risk 12): one hung handler must not stall an entity FIFO worker or freeze a periodic task without any observable signal
  • Consistency with the ADR-024 amendment: one semantics table (unset/explicit/None/callable) across telemetry, command, periodic, and device-context surfaces
  • TimeoutError is already mapped to error_type 'timeout' in the error taxonomy and flows through publish_error_safely, so cancellation needs no new error machinery
  • Behavior change must be reversible per registration (timeout=None) because the framework cannot know which handlers are legitimately slow
  • No new global Settings namespace: the post-F-TP1 posture is to avoid growing reserved environment names

Considered Options

Option 1: Status quo (explicit opt-in only)

Keep timeout=None defaults everywhere and only document that app authors should pass timeout= themselves.

  • Advantages: Zero migration risk for existing apps; No new constants or resolution code
  • Disadvantages: Leaves the risk-12 finding untreated; silent freezes remain the default experience; Every app must remember to configure each of the four surfaces independently; Audit finding stays open indefinitely

Option 2: Bounded defaults, three-state spec everywhere (chosen) (chosen)

Commands default to a 30-second backstop; periodic tasks auto-default to interval x 1.0 exactly like telemetry; device-context handlers get the 30-second default with a per-handler override. Omitted/explicit/None/callable semantics identical across all surfaces, resolved at bootstrap like intervals (ADR-020 pattern).

  • Advantages: Closes F-DP5 across every execution surface, not just @app.command; One semantics table to learn; reuses ADR-020 bootstrap resolution and the ADR-011/012 error-publishing and health machinery unchanged; Per-registration opt-out preserves legitimate long-running handlers; TimeoutError cancellation publishes structured error_type 'timeout' events — observable, not silent
  • Disadvantages: Behavior change: previously-unbounded handlers now get cancelled at 30 s (or one interval) unless migrated; Adds resolution code paths and tests for three additional surfaces; The 30 s constant is arbitrary and may need tuning per ecosystem

Option 3: Global settings namespace knob

Add an execution (or similar) section to Settings, e.g. EXECUTION__COMMAND_TIMEOUT, read as the global default for all handler timeouts, with decorator arguments overriding it.

  • Advantages: Single deployment-level control point for operators; No repeated per-registration boilerplate for homogeneous apps
  • Disadvantages: Grows the reserved environment namespace right after F-TP1 documented how collision-prone those are; Two-layer precedence (env then decorator) complicates reasoning about effective values and introspection; Precedent: retry/backoff and the telemetry timeout were both scoped per-registration; a global knob would be a second, competing configuration philosophy

Decision Matrix

Criterion Status quo (explicit opt-in only) Bounded defaults, three-state spec everywhere (chosen) Global settings namespace knob
Closes F-DP5 across surfaces 1 5 3
Consistency with ADR-024/ADR-020 patterns 3 5 2
Migration risk contained 5 4 3
Environment/config surface discipline 5 5 2
Observability of hangs 1 5 4

Scale: 1 (poor) to 5 (excellent)

Consequences

Positive

  • A hung command handler can no longer stall its entity's FIFO worker beyond 30 seconds; the worker logs, publishes a structured timeout error, and continues with queued commands
  • Periodic loops self-heal: a wedged cycle is cancelled at one interval and the loop continues with the next tick instead of becoming a silent zombie task
  • Device-context and sub-entity handlers get the same protection as app-level commands, closing the last unguarded execution path from F-DP5
  • Cancellation is observable end-to-end: TimeoutError maps to error_type 'timeout' (existing taxonomy), flows through ADR-011 redaction/deduplication and ADR-012 health reporting
  • Users migrate with a single keyword per registration (timeout=None) — the same escape hatch the ADR-024 amendment established

Negative

  • Breaking-ish behavior change (0.x minor): apps with handlers legitimately exceeding 30 s (or one periodic interval) will see cancellations until they set timeout= explicitly
  • asyncio.timeout cancels inside the coroutine and awaits cleanup, so actual elapsed time may slightly exceed the nominal bound (standard asyncio semantics)
  • Three more surfaces carry resolution logic and tests, increasing maintenance area alongside telemetry's existing timeout machinery
  • Reactor dispatch after a successful command remains outside the timeout window by design; a hung reactor is still only guarded by its own registration rules

2026-08-25