Hi Chia-Ping Tsai, Thankyou very much for the review. chia_00: the description "Workers running the old version will not recognize schemas with the new logical type names and will treat them as raw INT64, potentially causing data misinterpretation." seems to be wrong since the old worker will throw a NoClassDefFoundError, right?
Ans :: Sorry for the confusion but I meant old worker in the sense of old data plane but I totally agree that if a connector/converter compiled against the new connect-api (its bytecode references TimestampMicros) is deployed on an old worker, the first reference to TimestampMicros.SCHEMA links against the worker's old connect-api, which lacks the class → NoClassDefFoundError. I have updated the KIP to reflect the same. chia_01: does this KIP handle timestamps before 1970? The test plan mentions negative values, but the KIP does not define how fromLogical/toLogical convert them. Ans :: : Yes, negative (pre-1970) values are handled correctly, but the KIP currently only implies it via the test plan. The encoded long is a signed count since the Unix epoch; negative = before 1970. The reason it works cleanly is that Instant stores an epochSecond (which may be negative) and a nanoOfSecond that is always in [0, 999_999_999] — so getNano() is never negative, even for pre-1970 instants. I have updated to KIP with same including a sample worked pre-1970 example. chia_02: I guess an old sink can use TimestampConverter to convert the new types back to Timestamp to avoid compatibility issues? If so, it would be nice to document that in the migration path. Ans :: Yes totally agree and worth documenting. I request you to please go through the update KIP once and review the changes. Thanks, Pritam On Mon, Sep 28, 2026 at 7:53 PM Chia-Ping Tsai <[email protected]> wrote: > Hi > > Thanks for this KIP. I have a couple of comments > > chia_00: the description "Workers running the old version will not > recognize schemas with the new logical type names and will treat them as > raw INT64, potentially causing data misinterpretation." seems to be wrong > since the old worker will throw a NoClassDefFoundError, right? > > chia_01: does this KIP handle timestamps before 1970? The test plan > mentions negative values, but the KIP does not define how > fromLogical/toLogical convert them. > > chia_02: I guess an old sink can use TimestampConverter to convert the new > types back to Timestamp to avoid compatibility issues? If so, it would be > nice to document that in the migration path. > > Best, > Chia-Ping > > On 2025/04/04 18:44:52 pritam kumar wrote: > > Hi Kafka Community, > > > > I’d like to start a discussion on KIP-1153: Extending Support for > > Microsecond Precision for Kafka Connect > > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1153%3A+Extending+Support+for+Microsecond+Precision+for+Kafka+Connect > > > > . > > > > The primary motivation behind this KIP is to enhance the precision of > > timestamp handling in Kafka Connect. Currently, Kafka Connect is limited > to > > millisecond-level precision for timestamps. However, many modern data > > formats and platforms have moved beyond this limitation: > > > > - > > > > Formats such as *Avro*, *Parquet*, and *ORC* support microsecond (and > > even nanosecond) precision. For example, Avro specifies support for > > timestamp-micros (spec link > > <https://avro.apache.org/docs/1.11.0/spec.html#timestamp-micros>). > > - > > > > Sink systems like *Apache Iceberg*, *Delta Lake*, and *Apache Hudi* > > offer *microsecond and nanosecond precision* for time-based fields, > > making millisecond precision inadequate for accurate data replication > and > > analytics in some use cases. > > > > This gap can lead to *loss of precision* when transferring data through > > Kafka Connect, especially when interacting with systems that depend on > > high-resolution timestamps for change tracking, event ordering, or > > deduplication. > > > > The goal of this KIP is to: > > > > - > > > > Introduce microsecond-level timestamp handling in Kafka Connect schema > > and data representation. > > - > > > > Ensure connectors (both source and sink) can leverage this precision > > when supported by the underlying data systems. > > - > > > > Maintain backward compatibility with existing millisecond-based > > configurations and data. > > > > We welcome community feedback on: > > > > - > > > > Potential implementation concerns or edge cases we should address > > - > > > > Suggestions for schema enhancements or conversion strategies > > - > > > > Impacts on connector compatibility and testing > > > > Looking forward to your thoughts and input on this proposal! > > > > Thanks. > > Link to the KIP. > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1153%3A+Extending+Support+for+Microsecond+Precision+for+Kafka+Connect > > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1152%3A+Add+transactional+ID+prefix+filter+to+ListTransactions+API > > > > >
