feat(ai-gemini): stream structured outputs natively#971
Open
Missing-Identity wants to merge 2 commits into
Open
feat(ai-gemini): stream structured outputs natively#971Missing-Identity wants to merge 2 commits into
Missing-Identity wants to merge 2 commits into
Conversation
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
🚧 Files skipped from review as they are similar to previous changes (3)
📝 WalkthroughWalkthroughNative structured-output streaming was added to the Gemini text and experimental Gemini Interactions adapters. Both accumulate JSON deltas, emit parsed completion events before terminal events, handle stream failures, and add tests plus E2E feature coverage. ChangesGemini structured-output streaming
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Caller
participant GeminiAdapter
participant GeminiAPI
participant Consumer
Caller->>GeminiAdapter: structuredOutputStream(options)
GeminiAdapter->>GeminiAPI: Request JSON schema with streaming enabled
GeminiAPI-->>GeminiAdapter: JSON text deltas
GeminiAdapter-->>Consumer: TEXT_MESSAGE_CONTENT chunks
GeminiAdapter-->>Consumer: structured-output.complete
GeminiAdapter-->>Consumer: RUN_FINISHED
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/ai-gemini/src/adapters/text.ts`:
- Around line 229-283: The structuredOutputStream method uses inconsistent run
IDs between stream processing and structured-stream errors. Resolve a single
runId, using the existing override-or-generated fallback behavior, create an
options object containing it, and pass that same object to processStreamChunks
and every structuredStreamError call so all emitted events share the resolved
ID.
In `@packages/ai-gemini/tests/gemini-adapter.test.ts`:
- Line 3: Reorder the named imports from `@tanstack/ai` alphabetically to satisfy
ESLint sort-imports in packages/ai-gemini/tests/gemini-adapter.test.ts:3-3 and
packages/ai-gemini/tests/text-interactions-adapter.test.ts:3-3; update only the
import member ordering.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: c313a8d4-d80c-4e8d-a85c-1fc5ab15f7ca
📒 Files selected for processing (6)
.changeset/tidy-gemini-streams.mdpackages/ai-gemini/src/adapters/text.tspackages/ai-gemini/src/experimental/text-interactions/adapter.tspackages/ai-gemini/tests/gemini-adapter.test.tspackages/ai-gemini/tests/text-interactions-adapter.test.tstesting/e2e/src/lib/feature-support.ts
Comment on lines
+229
to
+283
| const { chatOptions, outputSchema } = options | ||
| const mappedOptions = this.mapCommonOptionsToGemini(chatOptions) | ||
|
|
||
| try { | ||
| chatOptions.logger.request( | ||
| `activity=structuredOutputStream provider=gemini model=${this.model} messages=${chatOptions.messages.length}`, | ||
| { provider: 'gemini', model: this.model }, | ||
| ) | ||
| const result = await this.client.models.generateContentStream({ | ||
| ...mappedOptions, | ||
| config: { | ||
| ...mappedOptions.config, | ||
| responseMimeType: 'application/json', | ||
| responseSchema: outputSchema, | ||
| }, | ||
| }) | ||
|
|
||
| let rawText = '' | ||
| let finished: | ||
| | Extract<StreamChunk, { type: typeof EventType.RUN_FINISHED }> | ||
| | undefined | ||
| let failed = false | ||
| for await (const chunk of this.processStreamChunks( | ||
| result, | ||
| chatOptions, | ||
| chatOptions.logger, | ||
| )) { | ||
| if (chunk.type === EventType.TEXT_MESSAGE_CONTENT) { | ||
| rawText += chunk.delta | ||
| } | ||
| if (chunk.type === EventType.RUN_ERROR) failed = true | ||
| if (chunk.type === EventType.RUN_FINISHED) { | ||
| finished = chunk | ||
| } else { | ||
| yield chunk | ||
| } | ||
| } | ||
|
|
||
| if (failed) return | ||
| if (!finished) { | ||
| yield structuredStreamError( | ||
| chatOptions, | ||
| 'Gemini structured-output stream ended without a terminal event', | ||
| 'truncated-stream', | ||
| ) | ||
| return | ||
| } | ||
| if (!rawText) { | ||
| yield structuredStreamError( | ||
| chatOptions, | ||
| 'Gemini structured-output stream contained no content', | ||
| 'empty-response', | ||
| ) | ||
| return | ||
| } |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate relevant symbols and inspect the surrounding code
rg -n "structuredOutputStream|processStreamChunks|structuredStreamError|generateId\\(" packages/ai-gemini/src/adapters/text.ts
echo
echo "--- structuredOutputStream / nearby lines ---"
sed -n '220,320p' packages/ai-gemini/src/adapters/text.ts
echo
echo "--- processStreamChunks / nearby lines ---"
sed -n '330,430p' packages/ai-gemini/src/adapters/text.ts
echo
echo "--- structuredStreamError / nearby lines ---"
sed -n '1040,1085p' packages/ai-gemini/src/adapters/text.tsRepository: TanStack/ai
Length of output: 8862
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the interaction adapter for the pattern mentioned in the comment
rg -n "runId\\s*=|structuredStreamError\\(|processStreamChunks\\(" packages/ai-*/src/adapters/**/*.ts
echo
echo "--- interaction adapter runId handling ---"
sed -n '1,220p' packages/ai-interactions/src/adapters/text.tsRepository: TanStack/ai
Length of output: 224
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find the TextOptions definition and all structuredOutputStream call sites
rg -n "interface TextOptions|type TextOptions|runId\\??:" packages -g '*.ts'
echo
echo "--- structuredOutputStream call sites ---"
rg -n "structuredOutputStream\\(" packages -g '*.ts'
echo
echo "--- Gemini text adapter type definitions around runId ---"
rg -n "runId" packages/ai-gemini/src/adapters/text.ts packages -g '*.ts' | head -n 80Repository: TanStack/ai
Length of output: 50367
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the call site that builds structuredCallOptions and invokes structuredOutputStream
sed -n '1990,2055p' packages/ai/src/activities/chat/index.ts
echo
echo "--- runId plumbing around the chat activity ---"
sed -n '640,720p' packages/ai/src/activities/chat/index.ts
echo
echo "--- structured call option types ---"
sed -n '850,1010p' packages/ai/src/types.tsRepository: TanStack/ai
Length of output: 13293
Use a single resolved runId for structured-output errors. processStreamChunks(...) falls back to generateId(this.name), but structuredStreamError(...) still reads options.runId. The chat activity passes runId: this.runIdOverride, so when the caller omits it the RUN_ERROR events end up with runId: undefined and can’t be correlated. Resolve runId once in structuredOutputStream and pass that same options object to processStreamChunks(...) and each structuredStreamError(...) call.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai-gemini/src/adapters/text.ts` around lines 229 - 283, The
structuredOutputStream method uses inconsistent run IDs between stream
processing and structured-stream errors. Resolve a single runId, using the
existing override-or-generated fallback behavior, create an options object
containing it, and pass that same object to processStreamChunks and every
structuredStreamError call so all emitted events share the resolved ID.
| import { describe, it, expect, beforeEach, vi } from 'vitest' | ||
| import { z } from 'zod' | ||
| import { chat, summarize } from '@tanstack/ai' | ||
| import { chat, EventType, summarize } from '@tanstack/ai' |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
ESLint sort-imports error on the @tanstack/ai import in both test files. Adding EventType to the named import left the members out of alphabetical order, which the rule reports as an error and fails lint CI. Reorder the named members in each import to satisfy sort-imports.
packages/ai-gemini/tests/gemini-adapter.test.ts#L3-L3: reorder members inimport { chat, EventType, summarize } from '@tanstack/ai'.packages/ai-gemini/tests/text-interactions-adapter.test.ts#L3-L3: reorder members inimport { chat, EventType } from '@tanstack/ai'.
🧰 Tools
🪛 ESLint
[error] 3-3: Member 'EventType' of the import declaration should be sorted alphabetically.
(sort-imports)
📍 Affects 2 files
packages/ai-gemini/tests/gemini-adapter.test.ts#L3-L3(this comment)packages/ai-gemini/tests/text-interactions-adapter.test.ts#L3-L3
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai-gemini/tests/gemini-adapter.test.ts` at line 3, Reorder the named
imports from `@tanstack/ai` alphabetically to satisfy ESLint sort-imports in
packages/ai-gemini/tests/gemini-adapter.test.ts:3-3 and
packages/ai-gemini/tests/text-interactions-adapter.test.ts:3-3; update only the
import member ordering.
Source: Linters/SAST tools
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
structuredOutputStreamsupport to the standard Gemini text adapterstructured-output.completeCloses #570.
Validation
@tanstack/ai-geminitest suite: 239 passed@tanstack/ai-geminiTypeScript check: passed@tanstack/ai-geminiESLint: no errors; seven pre-existing warnings@tanstack/ai-geminiproduction build: passedgit diff --check: passedLimitations
The hosted environment could not download Playwright Chromium, so browser E2E execution was not claimed. The existing structured-output-stream E2E matrix was updated to include Gemini.
Summary by CodeRabbit
structured-output.completecustom event with the parsed JSON result (plus the original text) before the final completion event.