GitHub user amuldotexe edited a comment on the discussion: Iggy Connector Ecosystem Tracking
One thing that may help connector contributions is a short behavior checklist for every new connector PR. At minimum, I think each connector should answer these questions up front: 1. Is this a source, sink, or both? 2. What is the external unit of data? 3. How does it map to an Iggy message? 4. What field or identity defines progress? 5. Is that field unique and monotonic? 6. What exact event counts as success? 7. Can success be partial? 8. Where is the commit frontier? 9. When can duplicates happen? 10. When can loss happen? 11. What happens on restart? 12. What semantics can the current runtime honestly claim? I think this would help for a few reasons: - Most connector review friction seems to come from behavioral boundaries rather than code structure. - We already have implementation examples, but not yet a shared behavior contract. - A short checklist would let maintainers review explicit claims instead of inferring intent from code. <details> <summary>Possible small v1</summary> A minimal capability manifest could be: ```yaml direction: source | sink delivery: at_most_once | at_least_once | effectively_once_with_idempotent_key restart_resume: committed_state_only | best_effort | not_supported ordering_scope: none | per_partition | per_key | global idempotency_basis: none | deterministic_key | upsert | destination_native partial_batch_behavior: impossible | possible_detectable | possible_ambiguous destructive_side_effects: none | delete_after_commit | mark_after_commit | other ``` And the minimum behavioral tests could be: - happy-path integration test - failure-boundary test - restart test - unsupported-config test - for sources: commit-after-send and replay-after-failure - for sinks: partial external failure is not silent, plus deterministic identity / duplicate-collision coverage where relevant </details> If this seems useful, I can turn it into: - a short connector behavior template in the docs - a review checklist for connector PRs - a source-first conformance checklist, then a sink-side version after that GitHub link: https://github.com/apache/iggy/discussions/2756#discussioncomment-16033487 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
