Hi Kafka Community, Could you please review this KIP proposal when you have a chance?
Thank you, Pritam On Sat, May 3, 2025 at 10:53 PM pritam kumar <kumarpritamm...@gmail.com> wrote: > Hi Chris, > Can you please review this one too? > > On Sat, Apr 5, 2025 at 7:00 AM pritam kumar <kumarpritamm...@gmail.com> > wrote: > >> Hi Kafka Community, >> Sorry due to some changes I had to change the link to the kip. >> Here is the updated KIP link: >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1154%3A+Extending+support+for+Microsecond+Precision+for+Kafka+Connect >> >> On Sat, Apr 5, 2025 at 12:14 AM pritam kumar <kumarpritamm...@gmail.com> >> wrote: >> >>> Hi Kafka Community, >>> >>> I’d like to start a discussion on KIP-1153: Extending Support for >>> Microsecond Precision for Kafka Connect >>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-1153%3A+Extending+Support+for+Microsecond+Precision+for+Kafka+Connect> >>> . >>> >>> The primary motivation behind this KIP is to enhance the precision of >>> timestamp handling in Kafka Connect. Currently, Kafka Connect is limited to >>> millisecond-level precision for timestamps. However, many modern data >>> formats and platforms have moved beyond this limitation: >>> >>> - >>> >>> Formats such as *Avro*, *Parquet*, and *ORC* support microsecond >>> (and even nanosecond) precision. For example, Avro specifies support for >>> timestamp-micros (spec link >>> <https://avro.apache.org/docs/1.11.0/spec.html#timestamp-micros>). >>> - >>> >>> Sink systems like *Apache Iceberg*, *Delta Lake*, and *Apache Hudi* >>> offer *microsecond and nanosecond precision* for time-based fields, >>> making millisecond precision inadequate for accurate data replication and >>> analytics in some use cases. >>> >>> This gap can lead to *loss of precision* when transferring data through >>> Kafka Connect, especially when interacting with systems that depend on >>> high-resolution timestamps for change tracking, event ordering, or >>> deduplication. >>> >>> The goal of this KIP is to: >>> >>> - >>> >>> Introduce microsecond-level timestamp handling in Kafka Connect >>> schema and data representation. >>> - >>> >>> Ensure connectors (both source and sink) can leverage this precision >>> when supported by the underlying data systems. >>> - >>> >>> Maintain backward compatibility with existing millisecond-based >>> configurations and data. >>> >>> We welcome community feedback on: >>> >>> - >>> >>> Potential implementation concerns or edge cases we should address >>> - >>> >>> Suggestions for schema enhancements or conversion strategies >>> - >>> >>> Impacts on connector compatibility and testing >>> >>> Looking forward to your thoughts and input on this proposal! >>> >>> Thanks. >>> Link to the KIP. >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1153%3A+Extending+Support+for+Microsecond+Precision+for+Kafka+Connect >>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-1152%3A+Add+transactional+ID+prefix+filter+to+ListTransactions+API> >>> >>>
