Documentation for ResumableJobMixin and resumable tasks

[amoghrajesh]

---

Was generative AI tooling used to co-author this PR?

• Yes - claude sonnet 4.6

closes: #67706

---

What?

Operators that submit work to external systems such as Spark, BigQuery, EMR, Kubernetes share a common failure mode: the worker holds its slot for the full polling duration, and if the worker crashes the task retries from scratch, often submitting a duplicate job. Airflow 3.3 now introduces ResumableJobMixin to solve the crash-and-duplicate problem, and the task state store enables the broader checkpoint-and-resume pattern. Neither had documentation, and there was no guidance on when to reach for these over deferrable operators or async tasks.

Current behaviour

The only comparison page (task-sdk/docs/deferred-vs-async-operators.rst) covered deferrable and async operators but made no mention of resumable tasks. ResumableJobMixin had no user-facing documentation at all. Users experiencing repeated job submission failures had no clear path to a solution.

Proposed change

• Adds airflow-core/docs/core-concepts/resumable-tasks.rst - a decision guide covering all three patterns (deferrable, resumable, async) with trade-off descriptions, a three-way comparison table, and a general checkpoint example using task_store directly.
• Adds task-sdk/docs/resumable-job-mixin.rst- a reference page for ResumableJobMixin covering the interface, method descriptions with inline examples, the retry flow, the pre-submit crash window limitation, and the external_id_key renaming warning.
• Updates task-sdk/docs/deferred-vs-async-operators.rst with a versionchanged:: 3.3.0 note pointing to both new pages.

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[eladkal]

This breaks the current structure the
.. versionadded:: 3.2.0
Is related to the pargraph below the new additon

[jroachgolf84]

Suggested change

	* Writing a custom deferrable operator requires implementing a separate ``Trigger`` class in
	* Writing a custom deferrable operator requires implementing a ``Trigger`` class in

[jroachgolf84]

Suggested change

	This pattern works without any additional work, just plain old context. The state store is just
	This pattern works without any additional work, relying only on ``context``. The state store is just

[jroachgolf84]

Suggested change

	the external job identifier to the task state store before polling begins. If the worker is restarted
	the external job identifier to task state store before polling begins. If the worker is restarted

[dabla]

Or: requires implementing a dedicated Trigger class...