[
https://issues.apache.org/jira/browse/CAMEL-24671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Federico Mariani updated CAMEL-24671:
-------------------------------------
Description:
Follow-up of CAMEL-24670, which covers the Responses API. The items below exist
in openai-java 4.55 (the version Camel uses) and in the OpenAI API spec, but
not in camel-openai.
* *PDF and audio input on chat-completion*: the API accepts {{file}} and
{{input_audio}} content parts, but {{OpenAIProducer}} only sends text and
images. A PDF body is rejected, or silently dropped when {{userMessage}} is
set, the same bug CAMEL-24670 fixes for {{responses}}.
* *Speaker-labelled transcription*: a {{diarized_json}} response falls through
to {{response.toString()}}, and {{diarized_json}} is missing from the
{{audioResponseFormat}} values. {{chunkingStrategy}} (required for
{{gpt-4o-transcribe-diarize}} input over 30 seconds), {{knownSpeakerNames}} /
{{knownSpeakerReferences}}, {{keywords}}, {{languages}} and {{include}}
(logprobs) are not exposed.
* *Audio upload filename*: a {{byte[]}} or {{InputStream}} body without
{{CamelFileNameOnly}} is uploaded as {{audio}}, with no extension or content
type, while the spec asks for both.
* *Error metadata and observability*: the docs say {{CamelAiErrorCategory}} and
{{CamelAiRetryAfterMillis}} are set when any OpenAI producer call fails, but
audio-transcription, audio-translation, audio-speech, image-generation,
image-edit and tool-execution never call {{GenAiErrorSupport}} and record no
GenAI observation ({{GenAiOperationName}} has no audio or image value).
_Claude Code on behalf of Federico Mariani (Croway)_
was:
Follow-up of CAMEL-24670, which covers the Responses API. The items below exist
in openai-java 4.55 (the version Camel uses) and in the OpenAI API spec, but
not in camel-openai.
*Fixes*
* *PDF and audio input on chat-completion*: the API accepts {{file}} and
{{input_audio}} content parts, but {{OpenAIProducer}} only sends text and
images. A PDF body is rejected, or silently dropped when {{userMessage}} is
set, the same bug CAMEL-24670 fixes for {{responses}}.
* *Speaker-labelled transcription*: a {{diarized_json}} response falls through
to {{response.toString()}}, and {{diarized_json}} is missing from the
{{audioResponseFormat}} values. {{chunkingStrategy}} (required for
{{gpt-4o-transcribe-diarize}} input over 30 seconds), {{knownSpeakerNames}} /
{{knownSpeakerReferences}}, {{keywords}}, {{languages}} and {{include}}
(logprobs) are not exposed.
* *Audio upload filename*: a {{byte[]}} or {{InputStream}} body without
{{CamelFileNameOnly}} is uploaded as {{audio}}, with no extension or content
type, while the spec asks for both.
* *Error metadata and observability*: the docs say {{CamelAiErrorCategory}} and
{{CamelAiRetryAfterMillis}} are set when any OpenAI producer call fails, but
audio-transcription, audio-translation, audio-speech, image-generation,
image-edit and tool-execution never call {{GenAiErrorSupport}} and record no
GenAI observation ({{GenAiOperationName}} has no audio or image value).
*Improvements*
* *Image moderation*: {{omni-moderation-latest}} accepts {{image_url}} inputs,
which would let a route screen uploaded images. The operation only sends text.
* *Webhook verification*: {{webhooks().unwrap()}} and {{verifySignature()}}
verify and parse OpenAI webhook events ({{response.completed}},
{{batch.completed}}, fine-tuning, eval, safety alerts). A helper usable from a
{{platform-http}} route would make background responses and batches
event-driven instead of polled.
* *Batch API*: batches, with the Files API for the JSONL input and output, run
{{/v1/responses}}, {{/v1/chat/completions}}, {{/v1/embeddings}},
{{/v1/moderations}} and image requests offline at a lower price, a natural fit
for file-based integration flows.
_Claude Code on behalf of Federico Mariani (Croway)_
> camel-openai: close the remaining gaps outside the Responses API
> ----------------------------------------------------------------
>
> Key: CAMEL-24671
> URL: https://issues.apache.org/jira/browse/CAMEL-24671
> Project: Camel
> Issue Type: Improvement
> Components: camel-openai
> Reporter: Federico Mariani
> Priority: Major
>
> Follow-up of CAMEL-24670, which covers the Responses API. The items below
> exist in openai-java 4.55 (the version Camel uses) and in the OpenAI API
> spec, but not in camel-openai.
> * *PDF and audio input on chat-completion*: the API accepts {{file}} and
> {{input_audio}} content parts, but {{OpenAIProducer}} only sends text and
> images. A PDF body is rejected, or silently dropped when {{userMessage}} is
> set, the same bug CAMEL-24670 fixes for {{responses}}.
> * *Speaker-labelled transcription*: a {{diarized_json}} response falls
> through to {{response.toString()}}, and {{diarized_json}} is missing from the
> {{audioResponseFormat}} values. {{chunkingStrategy}} (required for
> {{gpt-4o-transcribe-diarize}} input over 30 seconds), {{knownSpeakerNames}} /
> {{knownSpeakerReferences}}, {{keywords}}, {{languages}} and {{include}}
> (logprobs) are not exposed.
> * *Audio upload filename*: a {{byte[]}} or {{InputStream}} body without
> {{CamelFileNameOnly}} is uploaded as {{audio}}, with no extension or content
> type, while the spec asks for both.
> * *Error metadata and observability*: the docs say {{CamelAiErrorCategory}}
> and {{CamelAiRetryAfterMillis}} are set when any OpenAI producer call fails,
> but audio-transcription, audio-translation, audio-speech, image-generation,
> image-edit and tool-execution never call {{GenAiErrorSupport}} and record no
> GenAI observation ({{GenAiOperationName}} has no audio or image value).
> _Claude Code on behalf of Federico Mariani (Croway)_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
