[jira] [Commented] (KAFKA-1241) Cryptic serde error messages in new producer

2014-02-06 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13894188#comment-13894188 ] Jay Kreps commented on KAFKA-1241: -- I improved the error for that case. I think the

[jira] [Commented] (KAFKA-1171) Gradle build for Kafka

2014-02-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890828#comment-13890828 ] Jay Kreps commented on KAFKA-1171: -- My impression is that people who use Scala have

[jira] [Commented] (KAFKA-1171) Gradle build for Kafka

2014-02-03 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13889696#comment-13889696 ] Jay Kreps commented on KAFKA-1171: -- Shouldn't we default to scala 2.9 or something newer

[jira] [Updated] (KAFKA-1238) New producer hangs in a loop detecting metadata for auto created topics

2014-02-03 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1238: - Attachment: KAFKA-1238.patch Fix attached. New producer hangs in a loop detecting metadata for auto

[jira] [Updated] (KAFKA-1227) Code dump of new producer

2014-02-03 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1227: - Attachment: KAFKA-1227.patch Code dump of new producer -

[jira] [Commented] (KAFKA-1227) Code dump of new producer

2014-02-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13889247#comment-13889247 ] Jay Kreps commented on KAFKA-1227: -- Created reviewboard

[jira] [Updated] (KAFKA-1227) Code dump of new producer

2014-02-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1227: - Attachment: KAFKA-1227.patch Code dump of new producer -

[jira] [Commented] (KAFKA-1227) Code dump of new producer

2014-01-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883481#comment-13883481 ] Jay Kreps commented on KAFKA-1227: -- Hey Edward, What I actually want for the Cluster

[jira] [Commented] (KAFKA-1229) Reload broker config without a restart

2014-01-26 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882439#comment-13882439 ] Jay Kreps commented on KAFKA-1229: -- The problem with doing this in general beyond the

[jira] [Created] (KAFKA-1227) Code dump of new producer

2014-01-23 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-1227: Summary: Code dump of new producer Key: KAFKA-1227 URL: https://issues.apache.org/jira/browse/KAFKA-1227 Project: Kafka Issue Type: New Feature

[jira] [Commented] (KAFKA-1227) Code dump of new producer

2014-01-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880343#comment-13880343 ] Jay Kreps commented on KAFKA-1227: -- Created reviewboard

[jira] [Updated] (KAFKA-1227) Code dump of new producer

2014-01-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1227: - Attachment: KAFKA-1227.patch Code dump of new producer -

[jira] [Commented] (KAFKA-1227) Code dump of new producer

2014-01-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880717#comment-13880717 ] Jay Kreps commented on KAFKA-1227: -- Hey Jun, quick responses. I'll try to get a patch up

[jira] [Commented] (KAFKA-1227) Code dump of new producer

2014-01-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880752#comment-13880752 ] Jay Kreps commented on KAFKA-1227: -- Hey Guozhang, thanks for the detailed stylistic

[jira] [Commented] (KAFKA-1171) Gradle build for Kafka

2013-12-12 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13846415#comment-13846415 ] Jay Kreps commented on KAFKA-1171: -- These build discussions tend to blow up due to the

[jira] [Commented] (KAFKA-1137) Problem of FetchRequest API

2013-11-20 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13828487#comment-13828487 ] Jay Kreps commented on KAFKA-1137: -- One thing to be aware of is that we process requests

[jira] [Commented] (KAFKA-1112) broker can not start itself after kafka is killed with -9

2013-11-18 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13825708#comment-13825708 ] Jay Kreps commented on KAFKA-1112: -- Yeah at a high-level there are a couple of things we

[jira] [Commented] (KAFKA-1112) broker can not start itself after kafka is killed with -9

2013-11-18 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13825922#comment-13825922 ] Jay Kreps commented on KAFKA-1112: -- +1 lgtm. broker can not start itself after kafka is

[jira] [Commented] (KAFKA-1112) broker can not start itself after kafka is killed with -9

2013-11-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13823822#comment-13823822 ] Jay Kreps commented on KAFKA-1112: -- David, this should not be happening in 0.8. If it is

[jira] [Commented] (KAFKA-1112) broker can not start itself after kafka is killed with -9

2013-11-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13823829#comment-13823829 ] Jay Kreps commented on KAFKA-1112: -- Jun, this is true. However, if you think about it

[jira] [Updated] (KAFKA-1112) broker can not start itself after kafka is killed with -9

2013-11-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1112: - Attachment: KAFKA-1112-v3.patch Okay here is a maximally paranoid patch. broker can not start itself

[jira] [Commented] (KAFKA-1112) broker can not start itself after kafka is killed with -9

2013-11-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13824068#comment-13824068 ] Jay Kreps commented on KAFKA-1112: -- Yeah I would not be opposed to that as an

[jira] [Commented] (KAFKA-1130) log.dirs is a confusing property name

2013-11-12 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13820222#comment-13820222 ] Jay Kreps commented on KAFKA-1130: -- Fair enough. If we are going to make the change let's

[jira] [Updated] (KAFKA-1112) broker can not start itself after kafka is killed with -9

2013-11-12 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1112: - Attachment: KAFKA-1112-v1.patch The way the check was supposed to work was this: if the last offset in

[jira] [Commented] (KAFKA-1130) log.dirs is a confusing property name

2013-11-11 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13819810#comment-13819810 ] Jay Kreps commented on KAFKA-1130: -- I agree this is a tad bit ambiguous. However the

[jira] [Commented] (KAFKA-1132) Reduce size and amount of server access logs

2013-11-11 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13819811#comment-13819811 ] Jay Kreps commented on KAFKA-1132: -- Surely this is just for 0.8.1, no? Any change we

[jira] [Commented] (KAFKA-1119) Kafka 0.8.1 overwrites previous per topic config changes

2013-11-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13813655#comment-13813655 ] Jay Kreps commented on KAFKA-1119: -- Hmm, but it looks like the current behavior is to

[jira] [Commented] (KAFKA-1106) HighwaterMarkCheckpoint failure puting broker into a bad state

2013-10-29 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13808133#comment-13808133 ] Jay Kreps commented on KAFKA-1106: -- Do you have the highwatermark checkpoint file that

[jira] [Updated] (KAFKA-1098) Unit test failure in 0.8.1 related to LogCleaner

2013-10-24 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1098: - Attachment: KAFKA-1098-v3.patch Ack, fixed 2.10 issue. Frustrating. Do we have a single sbt command that

[jira] [Resolved] (KAFKA-1098) Unit test failure in 0.8.1 related to LogCleaner

2013-10-24 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-1098. -- Resolution: Fixed Unit test failure in 0.8.1 related to LogCleaner

[jira] [Updated] (KAFKA-1098) Unit test failure in 0.8.1 related to LogCleaner

2013-10-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1098: - Attachment: KAFKA-1098-v2.patch Here is a patch that seems to fix the issue. Unit test failure in

[jira] [Commented] (KAFKA-1098) Unit test failure in 0.8.1 related to LogCleaner

2013-10-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802606#comment-13802606 ] Jay Kreps commented on KAFKA-1098: -- Sorry about the churn btw I was only compiling for

[jira] [Updated] (KAFKA-1042) Fix segment flush logic

2013-10-18 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1042: - Attachment: KAFKA-1042-v2.patch Rebased that patch with minor changes. Also I found a subtle problem

[jira] [Resolved] (KAFKA-1042) Fix segment flush logic

2013-10-18 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-1042. -- Resolution: Fixed Fix segment flush logic --- Key: KAFKA-1042

[jira] [Resolved] (KAFKA-1081) kafka-run-class.sh is broken

2013-10-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-1081. -- Resolution: Fixed Checked in on both 0.8 and trunk. kafka-run-class.sh is broken

[jira] [Resolved] (KAFKA-1036) Unable to rename replication offset checkpoint in windows

2013-10-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-1036. -- Resolution: Fixed Checked in fix on trunk. Note that I don't have access to a windows box so I can't

[jira] [Resolved] (KAFKA-1008) Unmap before resizing

2013-10-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-1008. -- Resolution: Fixed Checked in on 0.8 and trunk. Unmap before resizing -

[jira] [Commented] (KAFKA-1081) kafka-run-class.sh is broken

2013-10-11 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13792936#comment-13792936 ] Jay Kreps commented on KAFKA-1081: -- Yeah I agree, rerunning the command to get the error

[jira] [Created] (KAFKA-1085) run-class.sh and other shell scripts are copy pasted all over the place

2013-10-11 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-1085: Summary: run-class.sh and other shell scripts are copy pasted all over the place Key: KAFKA-1085 URL: https://issues.apache.org/jira/browse/KAFKA-1085 Project: Kafka

[jira] [Updated] (KAFKA-1081) kafka-run-class.sh is broken

2013-10-11 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1081: - Attachment: KAFKA-1081-v1.patch Here is a proposed patch that uses exec for all the main commands and

[jira] [Updated] (KAFKA-1008) Unmap before resizing

2013-10-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1008: - Attachment: KAFKA-1008-v9-trunk.patch Attached KAFKA-1008-v9-trunk.patch which ports the final 0.8 patch

[jira] [Commented] (KAFKA-369) remove ZK dependency on producer

2013-10-08 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13789801#comment-13789801 ] Jay Kreps commented on KAFKA-369: - I guess all distributed systems have a problem of

[jira] [Commented] (KAFKA-1076) system tests in 0.8 are broken due to wrong log4j config

2013-10-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787416#comment-13787416 ] Jay Kreps commented on KAFKA-1076: -- +1 system tests in 0.8 are broken due to wrong

[jira] [Resolved] (KAFKA-982) Logo for Kafka

2013-10-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-982. - Resolution: Fixed Logo for Kafka -- Key: KAFKA-982 URL:

[jira] [Created] (KAFKA-1070) Auto-assign node id

2013-10-02 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-1070: Summary: Auto-assign node id Key: KAFKA-1070 URL: https://issues.apache.org/jira/browse/KAFKA-1070 Project: Kafka Issue Type: Bug Reporter: Jay Kreps

[jira] [Updated] (KAFKA-1070) Auto-assign node id

2013-10-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1070: - Description: It would be nice to have Kafka brokers auto-assign node ids rather than having that be a

[jira] [Commented] (KAFKA-1070) Auto-assign node id

2013-10-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13784113#comment-13784113 ] Jay Kreps commented on KAFKA-1070: -- I think the right way to do this is to have a

[jira] [Updated] (KAFKA-1008) Unmap before resizing

2013-09-20 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1008: - Attachment: KAFKA-0.8-1008-v8.patch Unmap before resizing -

[jira] [Commented] (KAFKA-1036) Unable to rename replication offset checkpoint in windows

2013-09-20 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13773527#comment-13773527 ] Jay Kreps commented on KAFKA-1036: -- I'm a little confused. I don't see any file locking

[jira] [Commented] (KAFKA-1008) Unmap before resizing

2013-09-20 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13773503#comment-13773503 ] Jay Kreps commented on KAFKA-1008: -- Sriram: 1.1 This is because this.mmap can be null so

[jira] [Commented] (KAFKA-1008) Unmap before resizing

2013-09-20 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13773504#comment-13773504 ] Jay Kreps commented on KAFKA-1008: -- Jun, added private on that method. Not sure about

[jira] [Comment Edited] (KAFKA-1008) Unmap before resizing

2013-09-20 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13773503#comment-13773503 ] Jay Kreps edited comment on KAFKA-1008 at 9/20/13 10:17 PM:

[jira] [Commented] (KAFKA-1043) Time-consuming FetchRequest could block other request in the response queue

2013-09-18 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13771557#comment-13771557 ] Jay Kreps commented on KAFKA-1043: -- Why not just configure the socket buffer size based

[jira] [Commented] (KAFKA-1056) Evenly Distribute Intervals in OffsetIndex

2013-09-17 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13769664#comment-13769664 ] Jay Kreps commented on KAFKA-1056: -- Agreed. This becomes a problem in the case where

[jira] [Commented] (KAFKA-1050) Support for no data loss mode

2013-09-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13762070#comment-13762070 ] Jay Kreps commented on KAFKA-1050: -- Hey Justin, yeah the two things I wanted to clarify:

[jira] [Commented] (KAFKA-1049) Encoder implementations are required to provide an undocumented constructor.

2013-09-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13762193#comment-13762193 ] Jay Kreps commented on KAFKA-1049: -- I agree that an init() method avoids the

[jira] [Commented] (KAFKA-1050) Support for no data loss mode

2013-09-08 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761528#comment-13761528 ] Jay Kreps commented on KAFKA-1050: -- I think there may be some confusion. Does this

[jira] [Commented] (KAFKA-1050) Support for no data loss mode

2013-09-08 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761530#comment-13761530 ] Jay Kreps commented on KAFKA-1050: -- Or actually, perhaps I am confused. The data loss

[jira] [Commented] (KAFKA-1024) possible memory leak in 0.8 beta1 producer with G1GC

2013-08-28 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13752578#comment-13752578 ] Jay Kreps commented on KAFKA-1024: -- Interesting possible memory

[jira] [Commented] (KAFKA-1026) Dynamically Adjust Batch Size Upon Receiving MessageSizeTooLargeException

2013-08-28 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13752577#comment-13752577 ] Jay Kreps commented on KAFKA-1026: -- This is a good idea, but I recommend we put this off

[jira] [Updated] (KAFKA-1008) Unmap before resizing

2013-08-28 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1008: - Attachment: KAFKA-0.8-1008-v7.patch Attached a version rebased to 0.8. Unmap before

[jira] [Commented] (KAFKA-1023) Allow Injectable LogManager

2013-08-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13751715#comment-13751715 ] Jay Kreps commented on KAFKA-1023: -- My recommendation is just to increase the retention.

[jira] [Commented] (KAFKA-1024) possible memory leak in 0.8 beta1 producer with G1GC

2013-08-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13751723#comment-13751723 ] Jay Kreps commented on KAFKA-1024: -- Your serialized message size is 2KB, perhaps but I

[jira] [Commented] (KAFKA-955) After a leader change, messages sent with ack=0 are lost

2013-08-26 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13750375#comment-13750375 ] Jay Kreps commented on KAFKA-955: - Great fix. A few minor comments, mostly stylistic.

[jira] [Commented] (KAFKA-1024) possible memory leak in 0.8 beta1 producer with G1GC

2013-08-26 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13750470#comment-13750470 ] Jay Kreps commented on KAFKA-1024: -- 64MB is a really small heap. Can you validate that

[jira] [Commented] (KAFKA-1024) possible memory leak in 0.8 beta1 producer with G1GC

2013-08-26 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13750681#comment-13750681 ] Jay Kreps commented on KAFKA-1024: -- Yes. possible memory leak in 0.8

[jira] [Updated] (KAFKA-925) Add optional partition key override in producer

2013-08-25 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-925: Attachment: KAFKA-925-post-commit-v1.patch Yeah fair point. I didn't realize we had a special infinity

[jira] [Commented] (KAFKA-1023) Allow Injectable LogManager

2013-08-25 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13749738#comment-13749738 ] Jay Kreps commented on KAFKA-1023: -- Yeah I would also be reticent to do this. The reason

[jira] [Updated] (KAFKA-1000) Inbuilt consumer offset management feature for kakfa

2013-08-24 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1000: - Description: Kafka currently stores offsets in zookeeper. This is a problem for several reasons. First

[jira] [Updated] (KAFKA-1008) Unmap before resizing

2013-08-21 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1008: - Attachment: KAFKA-1008-v6.patch Hey Elizabeth, this basically looks good. A couple of minor things. One

[jira] [Commented] (KAFKA-1008) Unmap before resizing

2013-08-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741035#comment-13741035 ] Jay Kreps commented on KAFKA-1008: -- Yeah, the problem is reads continue while the log is

[jira] [Commented] (KAFKA-1008) Unmap before resizing

2013-08-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741037#comment-13741037 ] Jay Kreps commented on KAFKA-1008: -- Here is a list of os.name values (at least according

[jira] [Commented] (KAFKA-1008) Unmap before resizing

2013-08-14 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740611#comment-13740611 ] Jay Kreps commented on KAFKA-1008: -- But my understanding is that these copies are just

[jira] [Commented] (KAFKA-1008) Unmap before resizing

2013-08-13 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13739197#comment-13739197 ] Jay Kreps commented on KAFKA-1008: -- Hey Elizabeth, thanks for the patch. Two issues.

[jira] [Updated] (KAFKA-718) kafka-run-class.sh should use reasonable gc settings

2013-08-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-718: Resolution: Fixed Fix Version/s: 0.8 Assignee: Jay Kreps (was: Neha Narkhede)

[jira] [Commented] (KAFKA-615) Avoid fsync on log segment roll

2013-08-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729684#comment-13729684 ] Jay Kreps commented on KAFKA-615: - 1.1 Good catch 2.1 My desire is to ensure that

[jira] [Commented] (KAFKA-615) Avoid fsync on log segment roll

2013-08-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729700#comment-13729700 ] Jay Kreps commented on KAFKA-615: - Jun. 52.2 In most filesystems there is no guarantee

[jira] [Updated] (KAFKA-615) Avoid fsync on log segment roll

2013-08-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-615: Attachment: KAFKA-615-v7.patch Patch which uses the correct but slow approach of synchronously committing

[jira] [Updated] (KAFKA-615) Avoid fsync on log segment roll

2013-08-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-615: Attachment: KAFKA-615-v8.patch Ack, yes, I did mean to fix the recoveryPoint/logEndOffset issue, I just

[jira] [Created] (KAFKA-1001) Handle follower transition in batch

2013-08-05 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-1001: Summary: Handle follower transition in batch Key: KAFKA-1001 URL: https://issues.apache.org/jira/browse/KAFKA-1001 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-615) Avoid fsync on log segment roll

2013-08-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13730155#comment-13730155 ] Jay Kreps commented on KAFKA-615: - Also, Jun, basic system tests all pass with v8.

[jira] [Commented] (KAFKA-615) Avoid fsync on log segment roll

2013-08-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13730154#comment-13730154 ] Jay Kreps commented on KAFKA-615: - Filed KAFKA-1001 to handle the follow-up work to

[jira] [Resolved] (KAFKA-615) Avoid fsync on log segment roll

2013-08-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-615. - Resolution: Fixed Fix Version/s: 0.8.1 Assignee: Jay Kreps (was: Neha Narkhede) Ah, nice

[jira] [Updated] (KAFKA-615) Avoid fsync on log segment roll

2013-08-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-615: Attachment: KAFKA-615-v6.patch Updated patch: - Removed bad scaladoc - Improved log corruption test to

[jira] [Updated] (KAFKA-718) kafka-run-class.sh should use reasonable gc settings

2013-08-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-718: Attachment: KAFKA-718-v5.patch Patch v5: move all logs to a logs/ directory to avoid polluting the main

[jira] [Commented] (KAFKA-718) kafka-run-class.sh should use reasonable gc settings

2013-08-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729076#comment-13729076 ] Jay Kreps commented on KAFKA-718: - Sorry, I had missed that file in v4. I had fixed it in

[jira] [Commented] (KAFKA-615) Avoid fsync on log segment roll

2013-08-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13729087#comment-13729087 ] Jay Kreps commented on KAFKA-615: - Ah thanks for the detailed review: 50.2 Yes, nice. 50.3

[jira] [Commented] (KAFKA-527) Compression support does numerous byte copies

2013-08-03 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13728575#comment-13728575 ] Jay Kreps commented on KAFKA-527: - The general idea of the refactoring would be to allow

[jira] [Commented] (KAFKA-996) Capitalize first letter for log entries

2013-08-03 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13728577#comment-13728577 ] Jay Kreps commented on KAFKA-996: - +1 but can we apply this on trunk? We are trying to

[jira] [Commented] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

2013-08-03 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13728583#comment-13728583 ] Jay Kreps commented on KAFKA-992: - This is good, this can go on trunk, right?

[jira] [Updated] (KAFKA-995) Enforce that the value for replica.fetch.max.bytes is always = the value for message.max.bytes

2013-08-03 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-995: Fix Version/s: 0.8.1 Enforce that the value for replica.fetch.max.bytes is always = the value for

[jira] [Commented] (KAFKA-995) Enforce that the value for replica.fetch.max.bytes is always = the value for message.max.bytes

2013-08-03 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13728585#comment-13728585 ] Jay Kreps commented on KAFKA-995: - Committed on trunk. Enforce that the

[jira] [Resolved] (KAFKA-985) Increasing log retention quickly overflows scala Int

2013-08-03 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-985. - Resolution: Fixed Fix Version/s: (was: 0.9) 0.8.1 Ack, nice catch.

[jira] [Commented] (KAFKA-984) Avoid a full rebalance in cases when a new topic is discovered but container/broker set stay the same

2013-08-03 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13728597#comment-13728597 ] Jay Kreps commented on KAFKA-984: - Guys I don't think we should be doing massive surgery on

[jira] [Updated] (KAFKA-718) kafka-run-class.sh should use reasonable gc settings

2013-08-03 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-718: Attachment: KAFKA-718-v4.patch New patch. Rebased. Fixed logging in tools so that we no longer spew crazy

[jira] [Updated] (KAFKA-527) Compression support does numerous byte copies

2013-08-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-527: Description: The data path for compressing or decompressing messages is extremely inefficient. We do

[jira] [Updated] (KAFKA-527) Compression support does numerous byte copies

2013-08-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-527: Attachment: java.hprof.no-compression.txt Compression support does numerous byte copies

[jira] [Commented] (KAFKA-527) Compression support does numerous byte copies

2013-08-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13728452#comment-13728452 ] Jay Kreps commented on KAFKA-527: - To test performance checkout trunk and do ./sbt package

[jira] [Updated] (KAFKA-615) Avoid fsync on log segment roll

2013-08-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-615: Attachment: KAFKA-615-v5.patch Attach updated patch v5. Rebased against trunk and with added support for

[jira] [Commented] (KAFKA-994) High level consumer doesn't throw an exception when the message it is trying to fetch exceeds the configured fetch size

2013-08-01 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13726536#comment-13726536 ] Jay Kreps commented on KAFKA-994: - Ack, nice catch. High level consumer

<    5   6   7   8   9   10   11   12   >