fix(harness): extend startup retry to push events; add Turns=0 driver-handoff diagnostics

[Copilot]

The exit-code-2 startup retry (MAX_SCHEDULED_EXIT2_RETRIES=1) only fired for schedule-triggered runs. Push-triggered agentic workflows (e.g. Tidy on push/main) that hit a transient Copilot API startup failure exited immediately with Turns=0 — no retry, deterministic red.

Changes

copilot_harness.cjs — extend retry eligibility to push

• Introduces isStartupRetryEligible = isScheduledRun || GITHUB_EVENT_NAME === "push", replacing the isScheduledRun guard on the exit-code-2 retry block and on emitInfrastructureIncomplete
• Retry log message now includes the trigger label (scheduled vs push) and the driver-handoff Turns=0 marker for traceability

// Before
if (isScheduledRun && result.exitCode === 2 && !result.hasOutput && ...)

// After
const isStartupRetryEligible = isScheduledRun || process.env.GITHUB_EVENT_NAME === "push";
if (isStartupRetryEligible && result.exitCode === 2 && !result.hasOutput && ...)

• Adds a driver-handoff-turns0 diagnostic log whenever any attempt produces no output, making the signature scannable across all event types

copilot_sdk_session.cjs — surface turn count at session end

• Adds assistantTurnCount counter incremented on each assistant.message event
• Appends assistantTurns=N to all session completed: log lines; Turns=0 sessions are now directly visible without cross-referencing JSONL

session completed: hasOutput=false assistantTurns=0 durationMs=1234

copilot_harness.test.cjs — update and extend retry tests

• Renames the isScheduledRun test parameter to isStartupRetryEligible
• Adds tests: push-event startup retry fires once; non-eligible triggers (e.g. pull_request) do not retry

• Fixes [deep-report] Isolate the deterministic Tidy x2 failure on push/main (Turns=0 driver exit) #47349

---

Generated by 👨‍🍳 PR Sous Chef · gpt54 6.54 AIC · ⌖ 7.98 AIC · ⊞ 7K · ◷
_{Comment /souschef to run again}

[Copilot]

Pull request overview

Extends transient Copilot startup recovery to push-triggered workflows and adds Turns=0 diagnostics.

Changes:

• Enables one fresh retry for push startup failures.
• Logs driver-handoff and SDK assistant-turn diagnostics.
• Adds retry-policy tests.

Show a summary per file

File	Description
actions/setup/js/copilot_harness.cjs	Extends retry eligibility and diagnostics.
actions/setup/js/copilot_harness.test.cjs	Adds push and non-eligible retry tests.
actions/setup/js/copilot_sdk_session.cjs	Logs assistant turn counts.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 3/3 changed files
• Comments generated: 3
• Review effort level: Medium

[github-actions]

✅ PR Code Quality Reviewer completed the code quality review.

[github-actions]

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

[github-actions]

✅ Test Quality Sentinel completed test quality analysis.

[github-actions]

✅ Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR #47384 does not have the 'implementation' label and has 0 new lines of code in business logic directories (threshold: 100).

[github-actions]

Review

The fix is directionally correct — extending startup retry eligibility to push events addresses the Turns=0 deterministic failure on push-triggered workflows. Three blocking issues identified by prior inline comments remain unaddressed:

1. assistantTurnCount not incremented in sendAndWait fallback (copilot_sdk_session.cjs) — sessions completing via that branch log assistantTurns=0 despite having output, making the diagnostic actively misleading.

2. Push-event retry test doesn't exercise GITHUB_EVENT_NAME (copilot_harness.test.cjs) — the test passes a precomputed true flag; the production guard process.env.GITHUB_EVENT_NAME === "push" is never covered.

3. emitInfrastructureIncomplete can fire on successful retry (copilot_harness.cjs) — the guard fires whenever lastExitCode === 2 && scheduledExit2RetryAttempted, even if the retry succeeded and a later attempt failed for an unrelated reason.

Please address the three issues flagged in the inline comments before merging.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · sonnet46 18.6 AIC · ⌖ 7.35 AIC · ⊞ 5K

[github-actions]

Skills-Based Review

Applied /diagnosing-bugs and /tdd — requesting changes on issues already flagged inline by Copilot.

Key issues:

1. assistantTurnCount not incremented in fallback paths (sendAndWait, idle-timeout, post-completion watchdog) — sessions that produced output will log Turns=0, defeating the diagnostic.
2. Push-event test passes a precomputed bool, so it never exercises the GITHUB_EVENT_NAME === 'push' expression itself.
3. emitInfrastructureIncomplete fires based on final exit code + retry flag alone — a retry that succeeds then later errors still triggers the false-positive message.

Positive highlights:

• Clean, minimal diff scoped to the retry block
• driver-handoff-turns0 diagnostic log is a useful traceability addition
• Non-eligible trigger guard test is solid

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 19.4 AIC · ⌖ 4.51 AIC · ⊞ 6.7K
_{Comment /matt to run again}

[github-actions]

Review: non-blocking observations

The core change is correct: extending isStartupRetryEligible to push events is sound, and the driver-handoff Turns=0 diagnostics are a clear improvement. Two medium issues worth fixing before this pattern is reused:

Issues found

1. assistantTurnCount counts streaming chunks, not turns (copilot_sdk_session.cjs:148) — the counter increments on every if (content) fragment, not once per assistant.message event. This inflates assistantTurns=N and makes Turns=0 triage ambiguous. Move the increment outside the content guard.

2. triggerLabel is duplicated and will silently lie for future triggers (copilot_harness.cjs:~1374, ~1406) — isScheduledRun ? "scheduled" : "push" appears twice and produces "push" for any non-scheduled eligible trigger that might be added later. Compute it once as process.env.GITHUB_EVENT_NAME ?? "unknown" at the top of main().

Existing review comments (by @Copilot) on sendAndWait fallback missing turn-count and the emitInfrastructureIncomplete false-positive are also valid and worth addressing.

🔎 Code quality review by PR Code Quality Reviewer · sonnet46 32.3 AIC · ⌖ 4.81 AIC · ⊞ 5.7K
_{Comment /review to run again}

Comments that could not be inline-anchored

actions/setup/js/copilot_sdk_session.cjs:148

assistantTurnCount over-counts on multi-chunk turns: incrementing inside the if (content) block counts streaming fragments, not assistant turns, inflating assistantTurns=N in the log.

<details>
<summary>💡 Suggested fix</summary>

Move the increment outside the content guard so it fires once per assistant.message event:

case "assistant.message": {
  const content = event.message?.content?.text ?? "";
  if (content) {
    hasOutput = true;
    output += content;
  }
  assista…

</details>

<details><summary>actions/setup/js/copilot_harness.cjs:1374</summary>

**`triggerLabel` will silently say &quot;push&quot; for any future non-push eligible trigger**: the expression `isScheduledRun ? &quot;scheduled&quot; : &quot;push&quot;` is also duplicated at ~line 1406, so both sites must be updated if `isStartupRetryEligible` ever covers a third event type.

&lt;details&gt;
&lt;summary&gt;💡 Suggested fix&lt;/summary&gt;

Compute it once from the actual event name rather than deriving from `isScheduledRun`:

```js
const triggerLabel = process.env.GITHUB_EVENT_NAME ?? &quot;unknown&quot;;

Or, if a display-frien…

[github-actions]

🧪 Test Quality Sentinel Report

✅ Test Quality Score: 82/100 — Excellent

Analyzed 2 test(s): 2 design, 0 implementation, 0 violation(s).

📊 Metrics (2 tests)

Metric	Value
Analyzed	2 (Go: 0, JS: 2)
✅ Design	2 (100%)
⚠️ Implementation	0 (0%)
Edge/error coverage	2 (100%)
Duplicate clusters	0
Inflation	No (test:prod ≈ 0.5:1)
🚨 Violations	0

Test	File	Classification	Issues
retries once for push startup interruption with exit code 2 and no output	copilot_harness.test.cjs	design_test	⚠️ Uses local shouldRetry helper (re-implements logic) rather than calling shouldRetryFailedExecution directly
does not retry on exit code 2 with no output for non-eligible triggers	copilot_harness.test.cjs	design_test	None

⚠️ Flagged Tests (1)

retries once for push startup interruption with exit code 2 and no output (copilot_harness.test.cjs) — design_test, high_value. The test uses an inline shouldRetry helper that re-implements retry-eligibility logic locally rather than calling the production shouldRetryFailedExecution. This means the test verifies the local model, not the actual production function. Consider adding a companion test that calls shouldRetryFailedExecution directly with push-event inputs.

Verdict

✅ Passed. 0% implementation tests (threshold: 30%). New push-event startup retry tests are behavioral and well-targeted. No violations found.

References:

• §29955300551

🧪 Test quality analysis by Test Quality Sentinel · sonnet46 48.4 AIC · ⌖ 6.74 AIC · ⊞ 7K · ◷
_{Comment /review to run again}

[github-actions]

✅ Test Quality Sentinel: 82/100. 0% implementation tests (threshold: 30%). New push-event startup retry tests are behavioral and well-targeted. No violations found.

[pelikhan]

@copilot run pr-finisher skill

[Copilot]

@copilot run pr-finisher skill

All three review comments addressed in commit fix: address review comments — turn counter fallback, push-event wiring tests, terminal hasOutput guard:

1. sendAndWait fallback turn counter (copilot_sdk_session.cjs:440): Added assistantTurnCount++ when the fallback captures content, so sessions that reach the fallback no longer log hasOutput=true assistantTurns=0.

2. Event-to-policy wiring tests (copilot_harness.test.cjs): Extracted computeStartupRetryEligible(eventName) as an exported function from the harness and replaced the precomputed-flag push test with one that calls computeStartupRetryEligible("push") directly. Added a full computeStartupRetryEligible wiring describe block covering schedule, push, pull_request, workflow_dispatch, and undefined — removing the production eligibility clause would now break these tests.

3. Terminal hasOutput guard (copilot_harness.cjs:1419): Added lastHasOutput tracking (updated alongside lastExitCode on each attempt). The emitInfrastructureIncomplete condition now requires !lastHasOutput, so a retry that recovers and produces output but later exits code 2 no longer emits the false Turns=0 verdict. New tests cover all four cases (terminal no-output emits, terminal has-output suppresses, no retry suppresses, non-eligible event suppresses).

[gh-aw-bot]

@copilot please run the pr-finisher skill, review the latest open feedback and comments on this PR, address any unresolved items, and rerun checks once the branch is up to date.

Generated by 👨‍🍳 PR Sous Chef · gpt54 6.54 AIC · ⌖ 7.98 AIC · ⊞ 7K · ◷
_{Comment /souschef to run again}