Hi Kafka Community, Sorry due to some changes I had to change the link to the kip. Here is the updated KIP link: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1154%3A+Extending+support+for+Microsecond+Precision+for+Kafka+Connect
On Sat, Apr 5, 2025 at 12:14 AM pritam kumar <kumarpritamm...@gmail.com> wrote: > Hi Kafka Community, > > I’d like to start a discussion on KIP-1153: Extending Support for > Microsecond Precision for Kafka Connect > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-1153%3A+Extending+Support+for+Microsecond+Precision+for+Kafka+Connect> > . > > The primary motivation behind this KIP is to enhance the precision of > timestamp handling in Kafka Connect. Currently, Kafka Connect is limited to > millisecond-level precision for timestamps. However, many modern data > formats and platforms have moved beyond this limitation: > > - > > Formats such as *Avro*, *Parquet*, and *ORC* support microsecond (and > even nanosecond) precision. For example, Avro specifies support for > timestamp-micros (spec link > <https://avro.apache.org/docs/1.11.0/spec.html#timestamp-micros>). > - > > Sink systems like *Apache Iceberg*, *Delta Lake*, and *Apache Hudi* > offer *microsecond and nanosecond precision* for time-based fields, > making millisecond precision inadequate for accurate data replication and > analytics in some use cases. > > This gap can lead to *loss of precision* when transferring data through > Kafka Connect, especially when interacting with systems that depend on > high-resolution timestamps for change tracking, event ordering, or > deduplication. > > The goal of this KIP is to: > > - > > Introduce microsecond-level timestamp handling in Kafka Connect schema > and data representation. > - > > Ensure connectors (both source and sink) can leverage this precision > when supported by the underlying data systems. > - > > Maintain backward compatibility with existing millisecond-based > configurations and data. > > We welcome community feedback on: > > - > > Potential implementation concerns or edge cases we should address > - > > Suggestions for schema enhancements or conversion strategies > - > > Impacts on connector compatibility and testing > > Looking forward to your thoughts and input on this proposal! > > Thanks. > Link to the KIP. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1153%3A+Extending+Support+for+Microsecond+Precision+for+Kafka+Connect > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-1152%3A+Add+transactional+ID+prefix+filter+to+ListTransactions+API> > >
