[jira] [Commented] (KAFKA-2078) Getting Selector [WARN] Error in I/O with host java.io.EOFException

2016-02-21 Thread Mikael Sundberg (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156595#comment-15156595
 ] 

Mikael Sundberg commented on KAFKA-2078:


Is there any activity on this issue? A major bugg unresolved after almost one 
year makes me a bit scared?
Im getting this error using the 0.8.2 client both towards a 0.9 cluster and a 
0.8 cluster. Hoping it is solved in the 0.9 client.

> Getting Selector [WARN] Error in I/O with host java.io.EOFException
> ---
>
> Key: KAFKA-2078
> URL: https://issues.apache.org/jira/browse/KAFKA-2078
> Project: Kafka
>  Issue Type: Bug
>  Components: producer 
>Affects Versions: 0.8.2.0
> Environment: OS Version: 2.6.39-400.209.1.el5uek and Hardware: 8 x 
> Intel(R) Xeon(R) CPU X5660  @ 2.80GHz/44GB
>Reporter: Aravind
>Assignee: Jun Rao
>
> When trying to Produce 1000 (10 MB) messages, getting this below error some 
> where between 997 to 1000th message. There is no pattern but able to 
> reproduce.
> [PDT] 2015-03-31 13:53:50 Selector [WARN] Error in I/O with "our host" 
> java.io.EOFException at 
> org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62)
>  at org.apache.kafka.common.network.Selector.poll(Selector.java:248) at 
> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) at 
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) at 
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) at 
> java.lang.Thread.run(Thread.java:724)
> This error I am getting some times @ 997th message or 999th message. There is 
> no pattern but able to reproduce.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3235) Unclosed stream in AppInfoParser static block

2016-02-21 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156564#comment-15156564
 ] 

ASF GitHub Bot commented on KAFKA-3235:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/914


> Unclosed stream in AppInfoParser static block
> -
>
> Key: KAFKA-3235
> URL: https://issues.apache.org/jira/browse/KAFKA-3235
> Project: Kafka
>  Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
> Fix For: 0.9.1.0
>
>
> {code}
> static {
> try {
> Properties props = new Properties();
> 
> props.load(AppInfoParser.class.getResourceAsStream("/kafka/kafka-version.properties"));
> version = props.getProperty("version", version).trim();
> commitId = props.getProperty("commitId", commitId).trim();
> {code}
> The stream returned by getResourceAsStream() should be closed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (KAFKA-3235) Unclosed stream in AppInfoParser static block

2016-02-21 Thread Gwen Shapira (JIRA) 

 [ 
https://issues.apache.org/jira/browse/KAFKA-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gwen Shapira resolved KAFKA-3235.
-
Resolution: Fixed

Issue resolved by pull request 914
[https://github.com/apache/kafka/pull/914]

> Unclosed stream in AppInfoParser static block
> -
>
> Key: KAFKA-3235
> URL: https://issues.apache.org/jira/browse/KAFKA-3235
> Project: Kafka
>  Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
> Fix For: 0.9.1.0
>
>
> {code}
> static {
> try {
> Properties props = new Properties();
> 
> props.load(AppInfoParser.class.getResourceAsStream("/kafka/kafka-version.properties"));
> version = props.getProperty("version", version).trim();
> commitId = props.getProperty("commitId", commitId).trim();
> {code}
> The stream returned by getResourceAsStream() should be closed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request: KAFKA-3235: Unclosed stream in AppInfoParser s...

2016-02-21 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/914


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] kafka pull request: added a few more variables and descriptions to...

2016-02-21 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/942


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Work started] (KAFKA-3196) KIP-42 (part 2): add record size and CRC to RecordMetadata and ConsumerRecords

2016-02-21 Thread Anna Povzner (JIRA) 

 [ 
https://issues.apache.org/jira/browse/KAFKA-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on KAFKA-3196 started by Anna Povzner.
---
> KIP-42 (part 2): add record size and CRC to RecordMetadata and ConsumerRecords
> --
>
> Key: KAFKA-3196
> URL: https://issues.apache.org/jira/browse/KAFKA-3196
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Anna Povzner
>Assignee: Anna Povzner
>
> This is the second (smaller) part of KIP-42, which includes: Add record size 
> and CRC to RecordMetadata and ConsumerRecord.
> See details in KIP-42 wiki: 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-42%3A+Add+Producer+and+Consumer+Interceptors
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3251) Requesting committed offsets results in inconsistent results

2016-02-21 Thread Daichi HIRATA (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156425#comment-15156425
 ] 

Daichi HIRATA commented on KAFKA-3251:
--

Hi. I was also investigating a similar problem. sarama sets 0 not -1 in 
RetentionTime of OffsetCommitRequest. So, I think that 
delete-expired-consumer-offsets always remove expired offset.

> Requesting committed offsets results in inconsistent results
> 
>
> Key: KAFKA-3251
> URL: https://issues.apache.org/jira/browse/KAFKA-3251
> Project: Kafka
>  Issue Type: Bug
>  Components: offset manager
>Affects Versions: 0.9.0.0
>Reporter: Dimitrij Denissenko
>Assignee: Jason Gustafson
>
> Hi,
> I am using github.com/Shopify/sarama to retrieve the committed offsets for a 
> high-volume topic, but the bug seems to be actually originating in Kafka 
> itself.
> I have written a little test to query the offsets of all partitions of one 
> topic, every second. The request looks like this:
> {code}
> OffsetFetchRequest{
>   ConsumerGroup: "my-group-name", 
>   Version: 1,
>   TopicPartitions: []TopicPartition{
>  {TopicName: "logs", Partitions: []int32{0,1,2,3,4,5,6,7}
>   }
> }
> {code}
> For most of the time, the responses are correct, but every 10 minutes or so, 
> there is a little glitch. I am not familiar with the Kafka internals, but it 
> looks like a little race. Here's my log output:
> {code}
> ...
> 2016/02/19 09:48:10 topic=logs partition=00 error=0 offset=206567925
> 2016/02/19 09:48:10 topic=logs partition=01 error=0 offset=206671019
> 2016/02/19 09:48:10 topic=logs partition=02 error=0 offset=206567995
> 2016/02/19 09:48:10 topic=logs partition=03 error=0 offset=205785315
> 2016/02/19 09:48:10 topic=logs partition=04 error=0 offset=206526677
> 2016/02/19 09:48:10 topic=logs partition=05 error=0 offset=206713764
> 2016/02/19 09:48:10 topic=logs partition=06 error=0 offset=206524006
> 2016/02/19 09:48:10 topic=logs partition=07 error=0 offset=206629121
> 2016/02/19 09:48:11 topic=logs partition=00 error=0 offset=206572870
> 2016/02/19 09:48:11 topic=logs partition=01 error=0 offset=206675966
> 2016/02/19 09:48:11 topic=logs partition=02 error=0 offset=206573267
> 2016/02/19 09:48:11 topic=logs partition=03 error=0 offset=205790613
> 2016/02/19 09:48:11 topic=logs partition=04 error=0 offset=206531841
> 2016/02/19 09:48:11 topic=logs partition=05 error=0 offset=206718513
> 2016/02/19 09:48:11 topic=logs partition=06 error=0 offset=206529762
> 2016/02/19 09:48:11 topic=logs partition=07 error=0 offset=206634037
> 2016/02/19 09:48:12 topic=logs partition=00 error=0 offset=-1
> 2016/02/19 09:48:12 topic=logs partition=01 error=0 offset=-1
> 2016/02/19 09:48:12 topic=logs partition=02 error=0 offset=-1
> 2016/02/19 09:48:12 topic=logs partition=03 error=0 offset=-1
> 2016/02/19 09:48:12 topic=logs partition=04 error=0 offset=-1
> 2016/02/19 09:48:12 topic=logs partition=05 error=0 offset=-1
> 2016/02/19 09:48:12 topic=logs partition=06 error=0 offset=-1
> 2016/02/19 09:48:12 topic=logs partition=07 error=0 offset=-1
> 2016/02/19 09:48:13 topic=logs partition=00 error=0 offset=-1
> 2016/02/19 09:48:13 topic=logs partition=01 error=0 offset=206686020
> 2016/02/19 09:48:13 topic=logs partition=02 error=0 offset=206583861
> 2016/02/19 09:48:13 topic=logs partition=03 error=0 offset=205800480
> 2016/02/19 09:48:13 topic=logs partition=04 error=0 offset=206542733
> 2016/02/19 09:48:13 topic=logs partition=05 error=0 offset=206728251
> 2016/02/19 09:48:13 topic=logs partition=06 error=0 offset=206534794
> 2016/02/19 09:48:13 topic=logs partition=07 error=0 offset=206643853
> 2016/02/19 09:48:14 topic=logs partition=00 error=0 offset=206584533
> 2016/02/19 09:48:14 topic=logs partition=01 error=0 offset=206690275
> 2016/02/19 09:48:14 topic=logs partition=02 error=0 offset=206588902
> 2016/02/19 09:48:14 topic=logs partition=03 error=0 offset=205805413
> 2016/02/19 09:48:14 topic=logs partition=04 error=0 offset=206542733
> 2016/02/19 09:48:14 topic=logs partition=05 error=0 offset=206733144
> 2016/02/19 09:48:14 topic=logs partition=06 error=0 offset=206540275
> 2016/02/19 09:48:14 topic=logs partition=07 error=0 offset=206649392
> ...
> {code}
> As you can see, the returned error code is 0 and there is no obvious reason 
> why the returned offsets are suddenly wrong/blank. 
> I have also added some debugging to our offset committer to make absolutely 
> sure the numbers we are sending are absolutely correct and they are. 
> Any help is greatly appreciated!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3256) Large number of system test failures

2016-02-21 Thread Jiangjie Qin (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156340#comment-15156340
 ] 

Jiangjie Qin commented on KAFKA-3256:
-

Thanks for the help, Geoff.

> Large number of system test failures
> 
>
> Key: KAFKA-3256
> URL: https://issues.apache.org/jira/browse/KAFKA-3256
> Project: Kafka
>  Issue Type: Bug
>Reporter: Geoff Anderson
>Assignee: Jiangjie Qin
>
> Confluent's nightly run of the kafka system tests reported a large number of 
> failures beginning 2/20/2016
> Test run: 2016-02-19--001.1455897182--apache--trunk--eee9522/
> Link: 
> http://confluent-kafka-system-test-results.s3-us-west-2.amazonaws.com/2016-02-19--001.1455897182--apache--trunk--eee9522/report.html
> Pass: 136
> Fail: 0
> Test run: 2016-02-20--001.1455979842--apache--trunk--5caa800/
> Link: 
> http://confluent-kafka-system-test-results.s3-us-west-2.amazonaws.com/2016-02-20--001.1455979842--apache--trunk--5caa800/report.html
> Pass: 72
> Fail: 64
> I.e. trunk@eee9522 was the last passing run, and trunk@5caa800 had a large 
> number of failures.
> Given its complexity, the most likely culprit is 45c8195fa, and I confirmed 
> this is the first commit with failures on a small number of tests.
> [~becket_qin] do you mind investigating?
> {code}
> commit 5caa800e217c6b83f62ee3e6b5f02f56e331b309
> Author: Jun Rao 
> Date:   Fri Feb 19 09:40:59 2016 -0800
> trivial fix to authorization CLI table
> commit 45c8195fa14c766b200c720f316836dbb84e9d8b
> Author: Jiangjie Qin 
> Date:   Fri Feb 19 07:56:40 2016 -0800
> KAFKA-3025; Added timetamp to Message and use relative offset.
> commit eee95228fabe1643baa016a2d49fb0a9fe2c66bd
> Author: Yasuhiro Matsuda 
> Date:   Thu Feb 18 09:39:30 2016 +0800
> MINOR: remove streams config params from producer/consumer configs
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3256) Large number of system test failures

2016-02-21 Thread Geoff Anderson (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156331#comment-15156331
 ] 

Geoff Anderson commented on KAFKA-3256:
---

[~becket_qin] It appears your first attempt failed while building kafka for 
some reason.

I cleared the workspace and kicked off another run here: 
https://jenkins.confluent.io/job/kafka_system_tests_branch_builder/356/console 

> Large number of system test failures
> 
>
> Key: KAFKA-3256
> URL: https://issues.apache.org/jira/browse/KAFKA-3256
> Project: Kafka
>  Issue Type: Bug
>Reporter: Geoff Anderson
>Assignee: Jiangjie Qin
>
> Confluent's nightly run of the kafka system tests reported a large number of 
> failures beginning 2/20/2016
> Test run: 2016-02-19--001.1455897182--apache--trunk--eee9522/
> Link: 
> http://confluent-kafka-system-test-results.s3-us-west-2.amazonaws.com/2016-02-19--001.1455897182--apache--trunk--eee9522/report.html
> Pass: 136
> Fail: 0
> Test run: 2016-02-20--001.1455979842--apache--trunk--5caa800/
> Link: 
> http://confluent-kafka-system-test-results.s3-us-west-2.amazonaws.com/2016-02-20--001.1455979842--apache--trunk--5caa800/report.html
> Pass: 72
> Fail: 64
> I.e. trunk@eee9522 was the last passing run, and trunk@5caa800 had a large 
> number of failures.
> Given its complexity, the most likely culprit is 45c8195fa, and I confirmed 
> this is the first commit with failures on a small number of tests.
> [~becket_qin] do you mind investigating?
> {code}
> commit 5caa800e217c6b83f62ee3e6b5f02f56e331b309
> Author: Jun Rao 
> Date:   Fri Feb 19 09:40:59 2016 -0800
> trivial fix to authorization CLI table
> commit 45c8195fa14c766b200c720f316836dbb84e9d8b
> Author: Jiangjie Qin 
> Date:   Fri Feb 19 07:56:40 2016 -0800
> KAFKA-3025; Added timetamp to Message and use relative offset.
> commit eee95228fabe1643baa016a2d49fb0a9fe2c66bd
> Author: Yasuhiro Matsuda 
> Date:   Thu Feb 18 09:39:30 2016 +0800
> MINOR: remove streams config params from producer/consumer configs
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3256) Large number of system test failures

2016-02-21 Thread Jiangjie Qin (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156272#comment-15156272
 ] 

Jiangjie Qin commented on KAFKA-3256:
-

[~junrao] Yes, I was trying to run the integration test with the patch. 

Not sure if I did something wrong, but the jenkins test seems not quite stable. 
Yesterday night the build failed because some ssh issue. I kicked off a new 
build this morning but somehow it seems stuck at the 21st test. I just started 
a new build and see if that works.

> Large number of system test failures
> 
>
> Key: KAFKA-3256
> URL: https://issues.apache.org/jira/browse/KAFKA-3256
> Project: Kafka
>  Issue Type: Bug
>Reporter: Geoff Anderson
>Assignee: Jiangjie Qin
>
> Confluent's nightly run of the kafka system tests reported a large number of 
> failures beginning 2/20/2016
> Test run: 2016-02-19--001.1455897182--apache--trunk--eee9522/
> Link: 
> http://confluent-kafka-system-test-results.s3-us-west-2.amazonaws.com/2016-02-19--001.1455897182--apache--trunk--eee9522/report.html
> Pass: 136
> Fail: 0
> Test run: 2016-02-20--001.1455979842--apache--trunk--5caa800/
> Link: 
> http://confluent-kafka-system-test-results.s3-us-west-2.amazonaws.com/2016-02-20--001.1455979842--apache--trunk--5caa800/report.html
> Pass: 72
> Fail: 64
> I.e. trunk@eee9522 was the last passing run, and trunk@5caa800 had a large 
> number of failures.
> Given its complexity, the most likely culprit is 45c8195fa, and I confirmed 
> this is the first commit with failures on a small number of tests.
> [~becket_qin] do you mind investigating?
> {code}
> commit 5caa800e217c6b83f62ee3e6b5f02f56e331b309
> Author: Jun Rao 
> Date:   Fri Feb 19 09:40:59 2016 -0800
> trivial fix to authorization CLI table
> commit 45c8195fa14c766b200c720f316836dbb84e9d8b
> Author: Jiangjie Qin 
> Date:   Fri Feb 19 07:56:40 2016 -0800
> KAFKA-3025; Added timetamp to Message and use relative offset.
> commit eee95228fabe1643baa016a2d49fb0a9fe2c66bd
> Author: Yasuhiro Matsuda 
> Date:   Thu Feb 18 09:39:30 2016 +0800
> MINOR: remove streams config params from producer/consumer configs
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request: added a few more variables and descriptions to...

2016-02-21 Thread christian-posta 
GitHub user christian-posta opened a pull request:

https://github.com/apache/kafka/pull/942

added a few more variables and descriptions to what you can customize…

… with the Vagranfile.local file

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/christian-posta/kafka ceposta-doco

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/942.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #942


commit f506b967010ba37de828fd3d681997f2567bacc9
Author: Christian Posta 
Date:   2016-02-21T22:27:30Z

added a few more variables and descriptions to what you can customize with 
the Vagranfile.local file




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (KAFKA-3256) Large number of system test failures

2016-02-21 Thread Jun Rao (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156184#comment-15156184
 ] 

Jun Rao commented on KAFKA-3256:


[~becket_qin], could you try the fix in ConsoleConsumer and run all system 
tests again? Geoff has given you permission to Confluence jenkins and you can 
run the tests on your branch.

> Large number of system test failures
> 
>
> Key: KAFKA-3256
> URL: https://issues.apache.org/jira/browse/KAFKA-3256
> Project: Kafka
>  Issue Type: Bug
>Reporter: Geoff Anderson
>Assignee: Jiangjie Qin
>
> Confluent's nightly run of the kafka system tests reported a large number of 
> failures beginning 2/20/2016
> Test run: 2016-02-19--001.1455897182--apache--trunk--eee9522/
> Link: 
> http://confluent-kafka-system-test-results.s3-us-west-2.amazonaws.com/2016-02-19--001.1455897182--apache--trunk--eee9522/report.html
> Pass: 136
> Fail: 0
> Test run: 2016-02-20--001.1455979842--apache--trunk--5caa800/
> Link: 
> http://confluent-kafka-system-test-results.s3-us-west-2.amazonaws.com/2016-02-20--001.1455979842--apache--trunk--5caa800/report.html
> Pass: 72
> Fail: 64
> I.e. trunk@eee9522 was the last passing run, and trunk@5caa800 had a large 
> number of failures.
> Given its complexity, the most likely culprit is 45c8195fa, and I confirmed 
> this is the first commit with failures on a small number of tests.
> [~becket_qin] do you mind investigating?
> {code}
> commit 5caa800e217c6b83f62ee3e6b5f02f56e331b309
> Author: Jun Rao 
> Date:   Fri Feb 19 09:40:59 2016 -0800
> trivial fix to authorization CLI table
> commit 45c8195fa14c766b200c720f316836dbb84e9d8b
> Author: Jiangjie Qin 
> Date:   Fri Feb 19 07:56:40 2016 -0800
> KAFKA-3025; Added timetamp to Message and use relative offset.
> commit eee95228fabe1643baa016a2d49fb0a9fe2c66bd
> Author: Yasuhiro Matsuda 
> Date:   Thu Feb 18 09:39:30 2016 +0800
> MINOR: remove streams config params from producer/consumer configs
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)