GitHub user amuldotexe added a comment to the discussion: Iggy Connector Ecosystem Tracking
I think it would help to standardize a short **connector behavior template** for every new connector PR. Short version: - Every connector should declare its delivery semantics, restart frontier, partial-success behavior, and identity strategy before code is written. - Every connector should answer the same small question set and ship the same minimum behavioral tests. - This should reduce review churn on the kinds of bugs that are really boundary/expectation problems, not code-style problems. Why I think this is worth doing: 1. The hardest connector bugs are usually behavioral, not syntactic. They show up around checkpoint timing, offset advancement, partial success, restart behavior, and duplicate/loss windows. 2. Today we have implementation examples, but not yet a shared behavior contract. Contributors can copy crate structure while still missing the runtime boundary. 3. A short template is cheap to adopt and should make reviews faster. Maintainers can review against explicit claims instead of inferring intent from code. <details> <summary>Proposed fast-start template</summary> A possible v1 could be just these 12 questions: 1. Is this a source, sink, or both? 2. What is the external unit of data? 3. How does it map to an Iggy message? 4. What field or identity defines progress? 5. Is that field unique and monotonic? 6. What exact event counts as success? 7. Can success be partial? 8. Where is the commit frontier? 9. When can duplicates happen? 10. When can loss happen? 11. What happens on restart? 12. What semantics can the current runtime honestly claim? Suggested minimal capability manifest: ```yaml direction: source | sink delivery: at_most_once | at_least_once | effectively_once_with_idempotent_key restart_resume: committed_state_only | best_effort | not_supported ordering_scope: none | per_partition | per_key | global idempotency_basis: none | deterministic_key | upsert | destination_native partial_batch_behavior: impossible | possible_detectable | possible_ambiguous destructive_side_effects: none | delete_after_commit | mark_after_commit | other ``` Suggested minimum test matrix: - happy-path integration test - failure-boundary test - restart test - unsupported-config test - for sources: commit-after-send and replay-after-failure - for sinks: partial external failure is not silent, plus deterministic identity / duplicate-collision coverage where relevant </details> If maintainers think this is useful, I can turn it into: - a short PRD / behavior template in the docs - a review checklist for connector PRs - a conformance-style test checklist for sources first, then sinks That feels like a higher-leverage improvement than only adding more connectors one by one, because it should make all future connector contributions easier and less error-prone. GitHub link: https://github.com/apache/iggy/discussions/2756#discussioncomment-16033487 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
