Re: data exchange format to use with Apache Kakfa that provides schema validation

Hi Rohit, Use avro. Please refer the link . On Fri, Feb 8, 2019 at 11:14 AM Rohit Sharma wrote: > Hi, > what is the best message format to use with apachhe kafka so that > producers and consumers can define contract and validate data and > s

data exchange format to use with Apache Kakfa that provides schema validation

Hi, what is the best message format to use with apachhe kafka so that producers and consumers can define contract and validate data and serialize/deserialize data? for example in xml we have xsd. but in json there is no universal schema.. i read about using apache avro but not sure how fast will

Re: [jira] [Resolved] (KAFKA-7872) Consider allowing transformations on GlobalKTable before materialization

unsubscribe On Fri, Feb 8, 2019 at 6:58 AM Guozhang Wang (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/KAFKA-7872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel > ] > > Guozhang Wang resolved KAFKA-7872. > -- > Resolution:

Re: [DISCUSS] KIP-428: Add in-memory window store

Thanks Sophie for proposing this new feature! In-memory window store is very useful in long term. One meta comment is that we don't need to include implementation details in the public interface section, and those validation steps are pretty trivial. Boyang Fro

[DISCUSS] KIP-428: Add in-memory window store

Streams currently only has support for a RocksDB window store, but users have been requesting an in-memory version. This KIP introduces a design for an in-memory window store implementation. https://cwiki.apache.org/confluence/display/KAFKA/KIP-428%3A+Add+in-memory+window+store

[jira] [Resolved] (KAFKA-7872) Consider allowing transformations on GlobalKTable before materialization

[ https://issues.apache.org/jira/browse/KAFKA-7872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-7872. -- Resolution: Duplicate > Consider allowing transformations on GlobalKTable before materializatio

Build failed in Jenkins: kafka-2.2-jdk8 #5

See Changes: [rajinisivaram] KAFKA-7902: Replace original loginContext if SASL/OAUTHBEARER refresh -- [...truncated 1.30 MB...] kafka.zk.KafkaZkClientTest > testGetBrokerMeth

Re: [VOTE] KIP-425: Add some Log4J Kafka Appender Properties for Producing to Secured Brokers

+1 (binding). Thanks! Guozhang On Thu, Feb 7, 2019 at 7:30 AM Dongjin Lee wrote: > binding +1: Gwen, Manikumar > non-binding +1: Dongjin, Bill > > We need one more binding +1. > > On Wed, Feb 6, 2019 at 3:41 AM Bill Bejeck wrote: > > > Thanks for the KIP, Rohan. > > > > +1 (non-binding). > > >

Build failed in Jenkins: kafka-trunk-jdk8 #3371

See Changes: [rajinisivaram] KAFKA-7902: Replace original loginContext if SASL/OAUTHBEARER refresh -- [...truncated 4.82 MB...] org.apache.kafka.streams.test.OutputVerifi

Build failed in Jenkins: kafka-trunk-jdk11 #269

See Changes: [rajinisivaram] KAFKA-7902: Replace original loginContext if SASL/OAUTHBEARER refresh -- [...truncated 2.30 MB...] org.apache.kafka.trogdor.agent.AgentTest >

Re: Statestore restoration & scaling questions - possible KIP as well.

Adam, I don’t have an answer for you but I would also be interested in clarification of this process if anyone can provide more details. If your reading is correct I would welcome the KIP to reduce the scaling pauses. Cheers, Rhys McCaig > On Feb 6, 2019, at 7:44 AM, Adam Bellemare wrote: >

Build failed in Jenkins: kafka-trunk-jdk11 #268

See Changes: [vahid.hashemian] MINOR: Add sentence to command line documentation of producer -- [...truncated 2.29 MB...] org.apache.kafka.streams.test.OutputVerifierTest

[jira] [Resolved] (KAFKA-7907) KafkaDistributed - Unknown datum class: class java.lang.Short

[ https://issues.apache.org/jira/browse/KAFKA-7907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-7907. -- Resolution: Invalid Closing this issue, since it is not a problem in Kafka Connect but rather a

Re: [jira] [Created] (KAFKA-7907) KafkaDistributed

This is a regression in Confluent's AvroConverter that shipped in Confluent Platform 5.1.0; earlier versions were not affected. Please see https://github.com/confluentinc/schema-registry/issues/968 for more details. It has already been fixed, and will be in the upcoming Confluent Platform 5.1.1 rel

[jira] [Resolved] (KAFKA-7902) SASL/OAUTHBEARER can become unable to connect: javax.security.sasl.SaslException: Unable to find OAuth Bearer token in Subject's private credentials (size=2)

[ https://issues.apache.org/jira/browse/KAFKA-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-7902. --- Resolution: Fixed Fix Version/s: 2.1.1 2.2.0 > SASL/OAUTHBEARER can

Build failed in Jenkins: kafka-trunk-jdk8 #3370

See Changes: [vahid.hashemian] MINOR: Add sentence to command line documentation of producer -- [...truncated 2.29 MB...] org.apache.kafka.streams.test.OutputVerifierTest

Re: [VOTE] KIP-425: Add some Log4J Kafka Appender Properties for Producing to Secured Brokers

binding +1: Gwen, Manikumar non-binding +1: Dongjin, Bill We need one more binding +1. On Wed, Feb 6, 2019 at 3:41 AM Bill Bejeck wrote: > Thanks for the KIP, Rohan. > > +1 (non-binding). > > -Bill > > On Tue, Feb 5, 2019 at 1:24 PM Manikumar > wrote: > > > +1 (binding). Thanks for the KIP. >

Re: [DISCUSS] KIP-390: Add producer option to adjust compression level

Hi Becket, Understood. You are right, introducing map type is just a clearer way, but not a strong prerequisite. It can be discussed later. @Committers, PMC members: Could you please give us some valuable feedbacks and suggestions for this topic? Thanks, Dongjin On Fri, Feb 1, 2019 at 11:37

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi Yaodong, thanks for the KIP! If I understand your intentions correctly then this KIP would only address a fairly specific use case, namely SASL-PLAIN with users defined in Zookeeper. For all other authentication mechanisms like SSL, SASL-GSSAPI or SASL-PLAIN with users defined in jaas files I

[jira] [Created] (KAFKA-7908) retention.ms and message.timestamp.difference.max.ms are tied

Ciprian Pascu created KAFKA-7908: Summary: retention.ms and message.timestamp.difference.max.ms are tied Key: KAFKA-7908 URL: https://issues.apache.org/jira/browse/KAFKA-7908 Project: Kafka

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

This KIP seems to duplicate some of the functionality proposed in KIP-248 . KIP-248 has been stuck in a vote thread since July 2018. Viktor, do you plan on working on the KIP? On Th

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hey there Yaodong, thanks for the KIP! I'm not too familiar with the user/client configurations we currently allow, is there a KIP describing the initial feature? If there is, it would be useful to include in KIP-422. I also didn't see any authorization in the PR, have we thought about needing to

Build failed in Jenkins: kafka-trunk-jdk11 #267

See Changes: [github] KAFKA-7540; Retry coordinator lookup to fix transient failure in -- [...truncated 2.30 MB...] org.apache.kafka.trogdor.rest.RestExceptionMapperTest

Build failed in Jenkins: kafka-trunk-jdk8 #3369

See Changes: [github] KAFKA-7540; Retry coordinator lookup to fix transient failure in -- [...truncated 2.29 MB...] org.apache.kafka.connect.transforms.TimestampConverter

[jira] [Created] (KAFKA-7907) KafkaDistributed

Blanchard Matthieu created KAFKA-7907: - Summary: KafkaDistributed Key: KAFKA-7907 URL: https://issues.apache.org/jira/browse/KAFKA-7907 Project: Kafka Issue Type: Bug Components

[jira] [Created] (KAFKA-7906) Improve failed leader election logging

Jason Gustafson created KAFKA-7906: -- Summary: Improve failed leader election logging Key: KAFKA-7906 URL: https://issues.apache.org/jira/browse/KAFKA-7906 Project: Kafka Issue Type: Improvem

[jira] [Resolved] (KAFKA-7540) Transient failure: kafka.api.ConsumerBounceTest.testClose

[ https://issues.apache.org/jira/browse/KAFKA-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7540. Resolution: Fixed Going to go on a limb and resolve this. Please reopen if the problem reoc