An AI assistant can create tickets, issue refunds, or change accounts on a user's behalf. A refund request times out, and the system cannot tell whether it completed; retrying may issue a second refund. How would you check the outcome, control retries, and ensure actions stay within the user's authorization?

Question-specific analysis · reviewed

This question has an edited analysis. Use it as a prompt for evidence, not as an automatic score.

Objective problem to resolve or clarify

When an external write times out with an unknown outcome, prevent duplicate refunds or unauthorized actions and define safe states for verification, pausing, and recovery.

What it examines

Assess whether you can separate an AI assistant's decisions from real actions and handle unknown outcomes, retries, and permission changes safely.

Signals worth noticing

Separates information lookup from actions that change data or charge money.
Gives each action a unique request ID and explains how to detect a duplicate, check its status, or ask a person before repeating it.
Sets confirmation requirements based on permission and potential harm.
Explains how to recover or pause after a timeout, partial completion, or unknown result.
Distinguishes retrying the same network request from the assistant deciding to take the action again.
Explains how changed permissions stop actions that have not run and how completed steps are handled.

Context worth asking for

Can the partner accept an idempotency key, expose request status, or return a reconcilable transaction ID?
How is user authorization bound to scope, amount, target, and expiry?
Who pauses, informs the user, and decides when to resume while status is unknown?
If a partner cannot detect duplicate requests and its status service is unavailable, how would you pause, verify, and resume the task?
How would you tell the user that a task is still in progress without claiming it has completed?
What goal, baseline, constraints, and responsibility boundary applied?
What options were available, and what evidence or dependencies shaped the choice?
Who was affected, what changed, and over what time window?
How was the result verified, and what alternative explanation or downside was considered?

What else the answer may imply

Separating a model's new decision from a network retry of the same request may show understanding of the boundary between model intent and external side effects. Alternative interpretations to check: A dependency may offer neither idempotency nor status lookup, making exactly-once execution impossible to guarantee. Authorization may change between queueing and execution, invalidating earlier approval.

Comparisons to consider

Automatic retry and status reconciliation

Compare idempotent retry, status lookup, human confirmation, and a paused queue by duplicate risk and delay.

Authorization check timing

Compare checks at request creation, queueing, and execution, including what happens after revocation.

Decision after context

Model side-effecting work as explicit states; retry only when request identity, current authorization, and execution status allow it. Otherwise preserve the unknown state and escalate verification.

Examine the trade-offs

Benefits created

Can the design prevent duplicate side effects while keeping approved work recoverable?
Can users distinguish in progress, confirmed complete, and unknown outcome?

Costs and risks

What ongoing costs come from state ledgers, reconciliation, human review, and secure storage?
How do pauses and extra confirmations affect completion time, user experience, and operations load?

Assessment guide

Use these as evidence anchors for this capability, not as an automatic score or an overall judgment of a person. Calibrate expectations to the role, level, and scope. A score of 2 is the usual baseline; excellent evidence in one or two dimensions is meaningful, while exceptional evidence is rare.

1 · Not passing

Core concepts or basic facts are unclear; the reasoning or account of events cannot be followed. For experience questions, the person cannot distinguish their own actions and responsibility. Confirmed deliberate fabrication is not passing; inconsistency or missing detail alone is a reason to verify, not proof of dishonesty.

Treats a timeout as failure and retries directly, or ignores authorization scope and unknown outcomes.

2 · Passing

Explains the basic concepts and approach clearly and gives credible evidence of having done the work or being able to reason through it.

Explains a basic safe path using idempotency, status lookup, or pausing for verification.

3 · Excellent

Gives a structured account, reflects clearly on what worked and what did not, weighs benefits against costs or risks, and offers an insight or practice above the ordinary expectation for the role and level.

Compares state handling, authorization timing, and recovery, including partial success and dependency failure.

4 · Exceptional

Shows an outcome or practice at an exceptional, field-leading level for the relevant scope. The impact is measurable, independently supportable, and credibly attributable to the person’s decisions and work.

Verifies side-effect control, recovery, and authorization boundaries at real scale with attributable evidence of systemic improvement.

Evidence that could make a contribution visible

Duplicate-action rate, unknown-state backlog, reconciliation gaps, authorization revocation latency, and escalation volume.

Question context and related views

RoleAI engineering · Backend · General software engineering · Frontend · Mobile

Responsibility scopeA project or team · Multiple teams

Career trackIndividual contributor · Team manager · Senior manager

Select questions for the responsibility you actually hold or seek. These labels are not universal stage requirements. Stage expectations

Same question, different stage expectations

The question stays the same. Choose a target to compare responsibility, reasoning, and evidence. This changes the reading framework, not the question’s difficulty or your score.

Loading stage model…

Target roleTarget responsibility

These are rule-based reading prompts, not separately reviewed stage-specific answers or a verdict that this question applies to every stage. Use your actual role and authority.

View the full role and stage model →

Compare expectations across stages

This table compares responsibility lenses, not mandatory question coverage or a ranking of people.

Stage	Responsibility	Result evidence

Want deeper feedback? Generate a prompt with this question’s evidence framework, then choose an AI tool yourself.

AI analysis prompt

# Interview Compass response analysis
Analyze the candidate response using only evidence provided. Do not invent facts.

## Question and review status
Question: An AI assistant can create tickets, issue refunds, or change accounts on a user's behalf. A refund request times out, and the system cannot tell whether it completed; retrying may issue a second refund. How would you check the outcome, control retries, and ensure actions stay within the user's authorization?
Analysis review status: Question-specific analysis · reviewed. This question has an edited analysis. Use it as a prompt for evidence, not as an automatic score.
Objective problem: When an external write times out with an unknown outcome, prevent duplicate refunds or unauthorized actions and define safe states for verification, pausing, and recovery.
Assessment intent: Assess whether you can separate an AI assistant's decisions from real actions and handle unknown outcomes, retries, and permission changes safely.
Possible inference (a hypothesis to verify): Separating a model's new decision from a network retry of the same request may show understanding of the boundary between model intent and external side effects. Alternative interpretations to check: A dependency may offer neither idempotency nor status lookup, making exactly-once execution impossible to guarantee. Authorization may change between queueing and execution, invalidating earlier approval.

## Evidence framework
Observable signals:
- Separates information lookup from actions that change data or charge money.
- Gives each action a unique request ID and explains how to detect a duplicate, check its status, or ask a person before repeating it.
- Sets confirmation requirements based on permission and potential harm.
- Explains how to recover or pause after a timeout, partial completion, or unknown result.
- Distinguishes retrying the same network request from the assistant deciding to take the action again.
- Explains how changed permissions stop actions that have not run and how completed steps are handled.
Context worth asking for:
- Can the partner accept an idempotency key, expose request status, or return a reconcilable transaction ID?
- How is user authorization bound to scope, amount, target, and expiry?
- Who pauses, informs the user, and decides when to resume while status is unknown?
- If a partner cannot detect duplicate requests and its status service is unavailable, how would you pause, verify, and resume the task?
- How would you tell the user that a task is still in progress without claiming it has completed?
- What goal, baseline, constraints, and responsibility boundary applied?
- What options were available, and what evidence or dependencies shaped the choice?
- Who was affected, what changed, and over what time window?
- How was the result verified, and what alternative explanation or downside was considered?
Comparison dimensions:
- Automatic retry and status reconciliation: Compare idempotent retry, status lookup, human confirmation, and a paused queue by duplicate risk and delay.
- Authorization check timing: Compare checks at request creation, queueing, and execution, including what happens after revocation.
Decision after context: Model side-effecting work as explicit states; retry only when request identity, current authorization, and execution status allow it. Otherwise preserve the unknown state and escalate verification.
Benefits to examine:
- Can the design prevent duplicate side effects while keeping approved work recoverable?
- Can users distinguish in progress, confirmed complete, and unknown outcome?
Costs and risks to examine:
- What ongoing costs come from state ledgers, reconciliation, human review, and secure storage?
- How do pauses and extra confirmations affect completion time, user experience, and operations load?

## 1–4 evidence anchors
- 1 · Not passing: Core concepts or basic facts are unclear; the reasoning or account of events cannot be followed. For experience questions, the person cannot distinguish their own actions and responsibility. Confirmed deliberate fabrication is not passing; inconsistency or missing detail alone is a reason to verify, not proof of dishonesty. Question-specific reference: Treats a timeout as failure and retries directly, or ignores authorization scope and unknown outcomes.
- 2 · Passing: Explains the basic concepts and approach clearly and gives credible evidence of having done the work or being able to reason through it. Question-specific reference: Explains a basic safe path using idempotency, status lookup, or pausing for verification.
- 3 · Excellent: Gives a structured account, reflects clearly on what worked and what did not, weighs benefits against costs or risks, and offers an insight or practice above the ordinary expectation for the role and level. Question-specific reference: Compares state handling, authorization timing, and recovery, including partial success and dependency failure.
- 4 · Exceptional: Shows an outcome or practice at an exceptional, field-leading level for the relevant scope. The impact is measurable, independently supportable, and credibly attributable to the person’s decisions and work. Question-specific reference: Verifies side-effect control, recovery, and authorization boundaries at real scale with attributable evidence of systemic improvement.

## Analysis instructions
1. Separate explicit evidence, missing context, and hypotheses. Cite exact words or facts from the response.
2. List the most important follow-up questions first. If there is not enough evidence, state “insufficient evidence; do not score.”
3. Assess each relevant signal against the 1–4 anchors only when supported; explain evidence and uncertainty.
4. Discuss comparison choices, causal reasoning, trade-offs, and the candidate’s actual participation where relevant. Identify strengths, unsupported conclusions, and one concrete next practice.
5. Do not provide a sample answer, rewrite the response as a script, diagnose personality, infer dishonesty from inconsistency alone, combine scores into an overall score, or recommend hiring/rejection.

If the response is not provided, ask the candidate to paste their response first; do not analyze or score the question alone.
Do not write a sample answer.