bpattarkine opened a new issue, #1431:
URL: https://github.com/apache/flink-agents/issues/1431

   ### Description
   
   Can Flink Agents support jobs with `pipeline.generic-types=false` when their 
inputs, keys, outputs, and application payloads have explicit serialization 
contracts?
   
   #1425 reported the limitation and was closed as not planned. #1426 documents 
the current requirement after review of an approach that retained Kryo. #1430 
centralizes the recovery marker state definition while preserving its current 
format. This proposal asks whether maintainers would consider a narrower, 
opt-in feature that removes generic fallback from the Agents paths used by 
those jobs.
   
   ### Proposed scope
   
   - Use the Flink input key serializer for runtime state that stores job keys.
   - Define explicit formats for runtime-owned recovery markers and pending 
task metadata. Include both Kafka and Fluss markers and offline Kafka 
checkpoint cleanup.
   - Require explicit representations for application values in events, memory, 
and Java outputs. Preserve concrete value types across checkpoint and durable 
replay, and keep the Python bridge contract explicit.
   - Keep the default configuration and its behavior for existing jobs. Define 
the upgrade contract for checkpoints, savepoints, and durable records before 
changing their formats. A fresh-state-only opt-in may be a practical first 
release if documented and enforced.
   
   ### Acceptance criteria
   
   A keyed Java job and a PyFlink job run with generic types disabled, 
checkpoint, and restore pending events, tasks, and memory after failure. 
Include a durable recovery check and cover unsupported payloads with actionable 
errors. Exercise Kafka and Fluss marker handling and verify the supported 
upgrade path.
   
   ### Are you willing to submit a PR?
   
   - [x] Yes. I plan to develop an experimental implementation on my fork for 
review.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to