Re: Experiencing trouble with KafkaCSVMetricsReporter

Hi Dongjin, 1. I'm not familiar with `KafkaCSVMetricsReporter`, but it seems to delete and recreate the csv dir before starting `CsvReporter` which then creates the csv files. Assuming the permissions are correct, not sure why it would fail to create the files. It may be worth filing a JIRA (and

Re: custom offsets in ProduceRequest

Hi! > On 27 Dec 2016, at 19:35, radai wrote: > > IIUC if you replicate from a single source cluster to a single target > cluster, the topic has the same number of partitions on both, and no one > writes directly to the target cluster (so master --> slave) the offsets

Re: custom offsets in ProduceRequest

so, if i follow your suggested logic correctly, there would be some sort of : produce(partition, msg, requestedOffset) which would fail if requestedOffset is already taken (by another previous such explicit call or by another regular call that just happened to get assigned that offset by the

[jira] [Updated] (KAFKA-4575) Transient failure in ConnectDistributedTest.test_pause_and_resume_sink in consuming messages after resuming source connector

[ https://issues.apache.org/jira/browse/KAFKA-4575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan updated KAFKA-4575: --- Component/s: system tests KafkaConnect > Transient failure in

[jira] [Assigned] (KAFKA-4575) Transient failure in ConnectDistributedTest.test_pause_and_resume_sink in consuming messages after resuming source connector

[ https://issues.apache.org/jira/browse/KAFKA-4575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan reassigned KAFKA-4575: -- Assignee: Shikhar Bhushan > Transient failure in

[jira] [Created] (KAFKA-4575) Transient failure in ConnectDistributedTest.test_pause_and_resume_sink in consuming messages after resuming source connector

Shikhar Bhushan created KAFKA-4575: -- Summary: Transient failure in ConnectDistributedTest.test_pause_and_resume_sink in consuming messages after resuming source connector Key: KAFKA-4575 URL:

Some questions about providing corrections to documentation.

Hi, _Observations:_ While going through the design documentation I noticed a minor error of duplication in points 4 and 5 here https://kafka.apache.org/documentation/#design_compactionguarantees. When I looked up the files in git, I see the duplication in all of the below files in

[GitHub] kafka pull request #2238: MINOR: Sync up 'kafka-run-class.bat' with 'kafka-r...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2238 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Build failed in Jenkins: kafka-trunk-jdk8 #1134

See Changes: [me] MINOR: Sync up 'kafka-run-class.bat' with 'kafka-run-class.sh' -- [...truncated 17618 lines...] org.apache.kafka.streams.KafkaStreamsTest >

[jira] [Created] (KAFKA-4574) Transient failure in ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade with security_protocol = SASL_PLAINTEXT, SSL

Shikhar Bhushan created KAFKA-4574: -- Summary: Transient failure in ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade with security_protocol = SASL_PLAINTEXT, SSL Key: KAFKA-4574 URL:

Build failed in Jenkins: kafka-trunk-jdk7 #1787

See Changes: [me] MINOR: Sync up 'kafka-run-class.bat' with 'kafka-run-class.sh' -- [...truncated 17594 lines...] org.apache.kafka.streams.processor.internals.StreamsMetadataStateTest >

[GitHub] kafka pull request #2295: MINOR: Mx4jLoader always returns false even if mx4...

GitHub user eribeiro opened a pull request: https://github.com/apache/kafka/pull/2295 MINOR: Mx4jLoader always returns false even if mx4j is loaded & started Mx4jLoader.scala should explicitly `return true` if the class is successfully loaded and started, otherwise it will return

[GitHub] kafka pull request #2296: KAFKA-4404: Add javadocs to document core Connect ...

GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/2296 KAFKA-4404: Add javadocs to document core Connect types, especially that integer types are signed You can merge this pull request into a Git repository by running: $ git pull

[jira] [Updated] (KAFKA-4115) Grow default heap settings for distributed Connect from 256M to 1G

[ https://issues.apache.org/jira/browse/KAFKA-4115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-4115: - Fix Version/s: 0.10.2.0 > Grow default heap settings for distributed Connect from

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786213#comment-15786213 ] Jiangjie Qin commented on KAFKA-4497: - [~sandieg] I cannot think of a good workaround on this without

Re: custom offsets in ProduceRequest

> On 29 Dec 2016, at 20:43, radai wrote: > > so, if i follow your suggested logic correctly, there would be some sort of > : > > produce(partition, msg, requestedOffset) > > which would fail if requestedOffset is already taken (by another previous > such explicit

[jira] [Commented] (KAFKA-4404) Add knowledge of sign to numeric schema types

[ https://issues.apache.org/jira/browse/KAFKA-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786227#comment-15786227 ] ASF GitHub Bot commented on KAFKA-4404: --- GitHub user ewencp opened a pull request:

[jira] [Updated] (KAFKA-4404) Add knowledge of sign to numeric schema types

[ https://issues.apache.org/jira/browse/KAFKA-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-4404: - Fix Version/s: 0.10.2.0 Status: Patch Available (was: Open) > Add

[jira] [Commented] (KAFKA-3947) kafka-reassign-partitions.sh should support dumping current assignment

[ https://issues.apache.org/jira/browse/KAFKA-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786244#comment-15786244 ] Ewen Cheslack-Postava commented on KAFKA-3947: -- [~kawamuray] Any progress on this? It got

[jira] [Commented] (KAFKA-4570) How to transfer extended fields in producing or consuming requests.

[ https://issues.apache.org/jira/browse/KAFKA-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786713#comment-15786713 ] Ewen Cheslack-Postava commented on KAFKA-4570: -- [~zander] You're correct that Kafka does not

Re: [DISCUSS] KIP-99: Add Global Tables to Kafka Streams

1/2: Sounds good, let's remove the joins within KGlobalTable for now. 3. I see, makes sense. Unfortunately since TopologyBuilder is a public class we cannot separate its internal usage only functions like build / buildWithGlobalTables / etc with other user functions like stream / table / etc. We

Re: [VOTE] KIP-99: Add Global Tables to Kafka Streams

Besides my comments on the other DISCUSS email thread, I'm +1. On Mon, Dec 12, 2016 at 9:32 AM, Bill Bejeck wrote: > +1 > > On Mon, Dec 12, 2016 at 12:29 PM, Matthias J. Sax > wrote: > > > +1 > > > > On 12/12/16 3:45 AM, Damian Guy wrote: > > > Hi

[jira] [Comment Edited] (KAFKA-4567) Connect Producer and Consumer ignore ssl parameters configured for worker

[ https://issues.apache.org/jira/browse/KAFKA-4567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786745#comment-15786745 ] Ewen Cheslack-Postava edited comment on KAFKA-4567 at 12/30/16 3:13 AM:

[jira] [Commented] (KAFKA-4567) Connect Producer and Consumer ignore ssl parameters configured for worker

[ https://issues.apache.org/jira/browse/KAFKA-4567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786745#comment-15786745 ] Ewen Cheslack-Postava commented on KAFKA-4567: -- Given some future security features we may

[jira] [Commented] (KAFKA-4571) Consumer fails to retrieve messages if started before producer

[ https://issues.apache.org/jira/browse/KAFKA-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786707#comment-15786707 ] Ewen Cheslack-Postava commented on KAFKA-4571: -- Since another consumer group ID consumes the

[jira] [Commented] (KAFKA-2260) Allow specifying expected offset on produce

[ https://issues.apache.org/jira/browse/KAFKA-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786719#comment-15786719 ] Ewen Cheslack-Postava commented on KAFKA-2260: -- [~wwarshaw] That sounds right -- the epoch

[jira] [Updated] (KAFKA-2331) Kafka does not spread partitions in a topic among all consumers evenly

[ https://issues.apache.org/jira/browse/KAFKA-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2331: - Component/s: (was: core) consumer clients >

[jira] [Updated] (KAFKA-2019) Old consumer RoundRobinAssignor clusters by consumer

[ https://issues.apache.org/jira/browse/KAFKA-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2019: - Summary: Old consumer RoundRobinAssignor clusters by consumer (was:

[jira] [Updated] (KAFKA-3297) More optimally balanced partition assignment strategy (new consumer)

[ https://issues.apache.org/jira/browse/KAFKA-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3297: - Component/s: consumer > More optimally balanced partition assignment strategy

[jira] [Commented] (KAFKA-3297) More optimally balanced partition assignment strategy (new consumer)

[ https://issues.apache.org/jira/browse/KAFKA-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786948#comment-15786948 ] Ewen Cheslack-Postava commented on KAFKA-3297: -- [~jeffwidman] I don't think KIP-54 supercedes

Re: custom offsets in ProduceRequest

or even better - if topic creation is done dynamically by the replicator, setting the initial offsets for partitions could be made part of topic creation ? even less API changes this way On Thu, Dec 29, 2016 at 10:49 PM, radai wrote: > ah, I didnt realize we are

Build failed in Jenkins: kafka-trunk-jdk8 #1135

See Changes: [me] MINOR: Mx4jLoader always returns false even if mx4j is loaded & started [me] MINOR: Rephrase Javadoc summary for ConsumerRecord -- [...truncated 32958 lines...]

[jira] [Commented] (KAFKA-2331) Kafka does not spread partitions in a topic among all consumers evenly

[ https://issues.apache.org/jira/browse/KAFKA-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786920#comment-15786920 ] Ewen Cheslack-Postava commented on KAFKA-2331: -- This seems like a separate issue -- it sounds

[jira] [Commented] (KAFKA-2019) Old consumer RoundRobinAssignor clusters by consumer

[ https://issues.apache.org/jira/browse/KAFKA-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786931#comment-15786931 ] Ewen Cheslack-Postava commented on KAFKA-2019: -- [~jeffwidman] I updated the title to clarify.

[jira] [Commented] (KAFKA-2196) remove roundrobin identical topic constraint in consumer coordinator

[ https://issues.apache.org/jira/browse/KAFKA-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786960#comment-15786960 ] Ewen Cheslack-Postava commented on KAFKA-2196: -- Based on git history, looks like 0.9.0.0,

[jira] [Commented] (KAFKA-3297) More optimally balanced partition assignment strategy (new consumer)

[ https://issues.apache.org/jira/browse/KAFKA-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786986#comment-15786986 ] Jeff Widman commented on KAFKA-3297: [~ewencp] KIP-54 tackles both sticky re-assignments and a more

Jenkins build is back to normal : kafka-trunk-jdk7 #1788

See

[GitHub] kafka pull request #2295: MINOR: Mx4jLoader always returns false even if mx4...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2295 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #2290: MINOR: Rephrase Javadoc summary for ConsumerRecord

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2290 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3502) Build is killed during kafka streams tests due to `pure virtual method called` error

[ https://issues.apache.org/jira/browse/KAFKA-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786941#comment-15786941 ] Ewen Cheslack-Postava commented on KAFKA-3502: -- I've seen this in a number of Jenkins builds

Re: Some questions about providing corrections to documentation.

Hello Dhwani, Yes since it is a hotfix on the website, you need to submit a patch to kafka-site on all the error directories instead of on kafka repo's docs folder. Details about web docs contribution can be found here:

[jira] [Updated] (KAFKA-2196) remove roundrobin identical topic constraint in consumer coordinator

[ https://issues.apache.org/jira/browse/KAFKA-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2196: - Fix Version/s: 0.9.0.0 > remove roundrobin identical topic constraint in consumer

Re: custom offsets in ProduceRequest

ah, I didnt realize we are limiting the discussion to master --> slave. but - if we're talking about master-slave replication, and under the conditions i outlined earlier (src and dest match in #partitions, no foreign writes to dest) it "just works", seems to me the only thing youre really