[jira] [Commented] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec
[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017149#comment-14017149 ] Joe Stein commented on KAFKA-1456: -- it shouldn't since the imports are all within the case class like snappy is > Add LZ4 and LZ4C as a compression codec > --- > > Key: KAFKA-1456 > URL: https://issues.apache.org/jira/browse/KAFKA-1456 > Project: Kafka > Issue Type: Improvement >Reporter: Joe Stein > Labels: newbie > Fix For: 0.8.2 > > Attachments: KAFKA-1456.patch, KAFKA-1456_2014-05-19_15:01:10.patch, > KAFKA-1456_2014-05-19_16:39:01.patch, KAFKA-1456_2014-05-19_18:19:32.patch, > KAFKA-1456_2014-05-19_23:24:27.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec
[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016476#comment-14016476 ] Joe Stein commented on KAFKA-1456: -- Looks like a good point. I have been looking through some other codecs also we should add another few and update this also appropriately. [~joliver] Can you work on that? > Add LZ4 and LZ4C as a compression codec > --- > > Key: KAFKA-1456 > URL: https://issues.apache.org/jira/browse/KAFKA-1456 > Project: Kafka > Issue Type: Improvement >Reporter: Joe Stein > Labels: newbie > Fix For: 0.8.2 > > Attachments: KAFKA-1456.patch, KAFKA-1456_2014-05-19_15:01:10.patch, > KAFKA-1456_2014-05-19_16:39:01.patch, KAFKA-1456_2014-05-19_18:19:32.patch, > KAFKA-1456_2014-05-19_23:24:27.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (KAFKA-1478) Remove spam comment from wiki
[ https://issues.apache.org/jira/browse/KAFKA-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1478. -- Resolution: Fixed I took care of this just now. > Remove spam comment from wiki > - > > Key: KAFKA-1478 > URL: https://issues.apache.org/jira/browse/KAFKA-1478 > Project: Kafka > Issue Type: Bug >Reporter: Jon Bringhurst > > A spam comment exists on the wiki at: > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Replication?focusedCommentId=33294292#comment-33294292 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (KAFKA-1477) add authentication layer and initial JKS x509 implementation for brokers, producers and consumer for network communication
Joe Stein created KAFKA-1477: Summary: add authentication layer and initial JKS x509 implementation for brokers, producers and consumer for network communication Key: KAFKA-1477 URL: https://issues.apache.org/jira/browse/KAFKA-1477 Project: Kafka Issue Type: New Feature Reporter: Joe Stein Assignee: Ivan Lyutov Fix For: 0.8.2 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Reopened] (KAFKA-1454) Update for a Scala 2.11 build
[ https://issues.apache.org/jira/browse/KAFKA-1454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein reopened KAFKA-1454: -- Assignee: Joe Stein > Update for a Scala 2.11 build > - > > Key: KAFKA-1454 > URL: https://issues.apache.org/jira/browse/KAFKA-1454 > Project: Kafka > Issue Type: Improvement >Reporter: Joe Stein >Assignee: Joe Stein >Priority: Critical > Labels: newbie > Fix For: 0.8.2 > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec
[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008923#comment-14008923 ] Joe Stein commented on KAFKA-1456: -- committed to trunk > Add LZ4 and LZ4C as a compression codec > --- > > Key: KAFKA-1456 > URL: https://issues.apache.org/jira/browse/KAFKA-1456 > Project: Kafka > Issue Type: Improvement >Reporter: Joe Stein > Labels: newbie > Fix For: 0.8.2 > > Attachments: KAFKA-1456.patch, KAFKA-1456_2014-05-19_15:01:10.patch, > KAFKA-1456_2014-05-19_16:39:01.patch, KAFKA-1456_2014-05-19_18:19:32.patch, > KAFKA-1456_2014-05-19_23:24:27.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec
Joe Stein created KAFKA-1456: Summary: Add LZ4 and LZ4C as a compression codec Key: KAFKA-1456 URL: https://issues.apache.org/jira/browse/KAFKA-1456 Project: Kafka Issue Type: Improvement Reporter: Joe Stein Fix For: 0.8.2 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (KAFKA-1454) Update for a Scala 2.11 build
[ https://issues.apache.org/jira/browse/KAFKA-1454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1454. -- Resolution: Not a Problem just tested a Scala 2.11.0 project and it worked with the 'org.apache.kafka:kafka_2.10:0.8.1.1' import so don't see a reason we need to-do anything > Update for a Scala 2.11 build > - > > Key: KAFKA-1454 > URL: https://issues.apache.org/jira/browse/KAFKA-1454 > Project: Kafka > Issue Type: Improvement >Reporter: Joe Stein >Priority: Critical > Labels: newbie > Fix For: 0.8.2 > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (KAFKA-1454) Update for a Scala 2.11 build
Joe Stein created KAFKA-1454: Summary: Update for a Scala 2.11 build Key: KAFKA-1454 URL: https://issues.apache.org/jira/browse/KAFKA-1454 Project: Kafka Issue Type: Improvement Reporter: Joe Stein Priority: Critical Fix For: 0.8.2 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1308) Publish jar of test utilities to Maven
[
https://issues.apache.org/jira/browse/KAFKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1308:
-
Fix Version/s: 0.8.2
> Publish jar of test utilities to Maven
> --
>
> Key: KAFKA-1308
> URL: https://issues.apache.org/jira/browse/KAFKA-1308
> Project: Kafka
> Issue Type: Wish
>Affects Versions: 0.8.1
>Reporter: Martin Kleppmann
> Fix For: 0.8.2
>
> Attachments: KAFKA-1308.patch
>
>
> For projects that use Kafka, and want to write tests that exercise Kafka (in
> our case, Samza), it's useful to have access to Kafka's test utility classes
> such as kafka.zk.EmbeddedZookeeper and kafka.utils.TestUtils. We can use
> {{./gradlew testJar}} to build jar files that contain those classes, but as
> far as I know, these are currently not made available in a binary release.
> At the moment, we have to check those kafka*-test.jar files into the Samza
> repository. To avoid that, would it be possible to publish those jars of
> tests to Maven, so that they fit into the normal dependency management?
> Or perhaps, if publishing the tests themselves is not appropriate, we could
> move the test utilities into a separate module that is published, and make
> the tests depend on that module?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (KAFKA-1413) bump kafka version to 0.8.1.1 in gradle.properties
[ https://issues.apache.org/jira/browse/KAFKA-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1413. -- Resolution: Fixed > bump kafka version to 0.8.1.1 in gradle.properties > -- > > Key: KAFKA-1413 > URL: https://issues.apache.org/jira/browse/KAFKA-1413 > Project: Kafka > Issue Type: Task >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1.1 > > Attachments: KAFKA-1413.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1413) bump kafka version to 0.8.1.1 in gradle.properties
[ https://issues.apache.org/jira/browse/KAFKA-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13977161#comment-13977161 ] Joe Stein commented on KAFKA-1413: -- Created reviewboard https://reviews.apache.org/r/20560/ against branch 0.8.1 > bump kafka version to 0.8.1.1 in gradle.properties > -- > > Key: KAFKA-1413 > URL: https://issues.apache.org/jira/browse/KAFKA-1413 > Project: Kafka > Issue Type: Task >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1.1 > > Attachments: KAFKA-1413.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1413) bump kafka version to 0.8.1.1 in gradle.properties
[ https://issues.apache.org/jira/browse/KAFKA-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1413: - Attachment: KAFKA-1413.patch > bump kafka version to 0.8.1.1 in gradle.properties > -- > > Key: KAFKA-1413 > URL: https://issues.apache.org/jira/browse/KAFKA-1413 > Project: Kafka > Issue Type: Task >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1.1 > > Attachments: KAFKA-1413.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (KAFKA-1413) bump kafka version to 0.8.1.1 in gradle.properties
Joe Stein created KAFKA-1413: Summary: bump kafka version to 0.8.1.1 in gradle.properties Key: KAFKA-1413 URL: https://issues.apache.org/jira/browse/KAFKA-1413 Project: Kafka Issue Type: Task Reporter: Joe Stein Priority: Blocker Fix For: 0.8.1.1 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1362) Publish sources and javadoc jars
[ https://issues.apache.org/jira/browse/KAFKA-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13973535#comment-13973535 ] Joe Stein commented on KAFKA-1362: -- ok > Publish sources and javadoc jars > > > Key: KAFKA-1362 > URL: https://issues.apache.org/jira/browse/KAFKA-1362 > Project: Kafka > Issue Type: Bug > Components: packaging >Affects Versions: 0.8.1 >Reporter: Stevo Slavic >Assignee: Joel Koshy > Labels: build > Fix For: 0.8.1.1 > > Attachments: KAFKA-1362.patch > > > Currently just binaries jars get published on Maven Central (see > http://repo1.maven.org/maven2/org/apache/kafka/kafka_2.10/0.8.1/ ). Please > also publish sources and javadoc jars. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1362) Publish sources and javadoc jars
[ https://issues.apache.org/jira/browse/KAFKA-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13973474#comment-13973474 ] Joe Stein commented on KAFKA-1362: -- 2.8.0 is so much older and worse (arguably) and not used if we drop support for 2.8.2 we might as well do it for 2.8.0, we should have some consensus to-do that though IMHO it falls under "release plan" and would require a Lazy majority to pass. > Publish sources and javadoc jars > > > Key: KAFKA-1362 > URL: https://issues.apache.org/jira/browse/KAFKA-1362 > Project: Kafka > Issue Type: Bug > Components: packaging >Affects Versions: 0.8.1 >Reporter: Stevo Slavic >Assignee: Joel Koshy > Labels: build > Fix For: 0.8.1.1 > > Attachments: KAFKA-1362.patch > > > Currently just binaries jars get published on Maven Central (see > http://repo1.maven.org/maven2/org/apache/kafka/kafka_2.10/0.8.1/ ). Please > also publish sources and javadoc jars. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1399) Drop Scala 2.8.x support
[ https://issues.apache.org/jira/browse/KAFKA-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13973469#comment-13973469 ] Joe Stein commented on KAFKA-1399: -- +1 to stop supporting Scala 2.8.0 and 2.8.2 > Drop Scala 2.8.x support > > > Key: KAFKA-1399 > URL: https://issues.apache.org/jira/browse/KAFKA-1399 > Project: Kafka > Issue Type: Task > Components: packaging >Affects Versions: 0.8.1 >Reporter: Stevo Slavic > Labels: gradle, scala > > It's been almost 4 years since [Scala 2.8 has been > released|http://www.scala-lang.org/old/node/7009] and 3 years since [Scala > 2.9 has been released|http://www.scala-lang.org/old/node/9483], so there was > more than plenty of time to migrate. > Continued support of old Scala 2.8 is causing issues like > [this|https://issues.apache.org/jira/browse/KAFKA-1362?focusedCommentId=13970390&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13970390]. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1362) Publish sources and javadoc jars
[ https://issues.apache.org/jira/browse/KAFKA-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13970390#comment-13970390 ] Joe Stein commented on KAFKA-1362: -- i ran with stacktrace, debug and info https://gist.github.com/joestein/10803377#file-kafka-1362-L296 googled around a bit looks like a bug in 2.8.1 scaladoc itself https://issues.scala-lang.org/browse/SI-4284 but fixed in 2.9.0 not sure yet if there is a workaround or what exactly in the code is causing this that we can change to subvert it. > Publish sources and javadoc jars > > > Key: KAFKA-1362 > URL: https://issues.apache.org/jira/browse/KAFKA-1362 > Project: Kafka > Issue Type: Bug > Components: packaging >Affects Versions: 0.8.1 >Reporter: Stevo Slavic >Assignee: Joel Koshy > Labels: build > Fix For: 0.8.1.1 > > Attachments: KAFKA-1362.patch > > > Currently just binaries jars get published on Maven Central (see > http://repo1.maven.org/maven2/org/apache/kafka/kafka_2.10/0.8.1/ ). Please > also publish sources and javadoc jars. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1362) Publish sources and javadoc jars
[ https://issues.apache.org/jira/browse/KAFKA-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13970367#comment-13970367 ] Joe Stein commented on KAFKA-1362: -- ./gradlew srcJar worked great however the docsJar_2_8_2 error is stopping ./gradlew releaseTarGzAll from working :( trying to get a stacktrace out of it > Publish sources and javadoc jars > > > Key: KAFKA-1362 > URL: https://issues.apache.org/jira/browse/KAFKA-1362 > Project: Kafka > Issue Type: Bug > Components: packaging >Affects Versions: 0.8.1 >Reporter: Stevo Slavic >Assignee: Joel Koshy > Labels: build > Fix For: 0.8.1.1 > > Attachments: KAFKA-1362.patch > > > Currently just binaries jars get published on Maven Central (see > http://repo1.maven.org/maven2/org/apache/kafka/kafka_2.10/0.8.1/ ). Please > also publish sources and javadoc jars. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (KAFKA-1264) Make ConsoleProducer compression codec configurable
[ https://issues.apache.org/jira/browse/KAFKA-1264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1264. -- Resolution: Duplicate Fix Version/s: 0.8.2 duplicate of KAFKA-1344 which is committed to master for 0.8.2 > Make ConsoleProducer compression codec configurable > --- > > Key: KAFKA-1264 > URL: https://issues.apache.org/jira/browse/KAFKA-1264 > Project: Kafka > Issue Type: Improvement > Components: producer >Affects Versions: 0.8.0 >Reporter: Kieren Hynd >Assignee: Jun Rao >Priority: Minor > Fix For: 0.8.2 > > Attachments: kafka-1264-v1.patch > > > The ConsoleProducer --compress option only gives a toggle between > DefaultCompressionCodec (GZIP) and NoCompressionCodec. > Since the snappy-java .jar is bundled with Kafka, could the ConsoleProducer > mimic the ProducerPerformance.scala options to be an Int (0 = None, 1 = GZIP, > 2 = SNAPPY)? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1324) Debian packaging
[ https://issues.apache.org/jira/browse/KAFKA-1324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1324: - Fix Version/s: 0.8.2 > Debian packaging > > > Key: KAFKA-1324 > URL: https://issues.apache.org/jira/browse/KAFKA-1324 > Project: Kafka > Issue Type: Improvement > Components: packaging > Environment: linux >Reporter: David Stendardi >Priority: Minor > Labels: deb, debian, fpm, packaging > Fix For: 0.8.2 > > Attachments: packaging.patch > > > The following patch add a task releaseDeb to the gradle build : > ./gradlew releaseDeb > This task should create a debian package in core/build/distributions using > fpm : > https://github.com/jordansissel/fpm. > We decided to use fpm so other package types would be easy to provide in > further iterations (eg : rpm). > *Some implementations details* : > - We splitted the releaseTarGz in two tasks : distDir, releaseTarGz. > - We tried to use gradle builtin variables (project.name etc...) > - By default the service will not start automatically so the user is free to > setup the service with custom configuration. > Notes : > * FPM is required and should be in the path. > * FPM does not allow yet to declare /etc/default/kafka as a conffiles (see : > https://github.com/jordansissel/fpm/issues/668) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1324) Debian packaging
[ https://issues.apache.org/jira/browse/KAFKA-1324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13947880#comment-13947880 ] Joe Stein commented on KAFKA-1324: -- Hi David, I got a failure when running releaseDeb the gradle task is looking for fsm. So, I jumped to look at your fsm link and there are a few dependencies in there too that have to be fleshed out. What were your thoughts on how to include those dependencies in the patch? Maybe a gradle task to check your Ruby version and gem install fsm when doing the deb release (or something of the sort)? Let me know, will finish through trying this later on a vm using fsm and let you know if any other issues. Looking at the patch got me excited to try it out, could be really great to have such a nice quick simple support on ubuntu and an easy path for someone to hook rpm in from your pre-work also. > Debian packaging > > > Key: KAFKA-1324 > URL: https://issues.apache.org/jira/browse/KAFKA-1324 > Project: Kafka > Issue Type: Improvement > Components: packaging > Environment: linux >Reporter: David Stendardi >Priority: Minor > Labels: deb, debian, fpm, packaging > Attachments: packaging.patch > > > The following patch add a task releaseDeb to the gradle build : > ./gradlew releaseDeb > This task should create a debian package in core/build/distributions using > fpm : > https://github.com/jordansissel/fpm. > We decided to use fpm so other package types would be easy to provide in > further iterations (eg : rpm). > *Some implementations details* : > - We splitted the releaseTarGz in two tasks : distDir, releaseTarGz. > - We tried to use gradle builtin variables (project.name etc...) > - By default the service will not start automatically so the user is free to > setup the service with custom configuration. > Notes : > * FPM is required and should be in the path. > * FPM does not allow yet to declare /etc/default/kafka as a conffiles (see : > https://github.com/jordansissel/fpm/issues/668) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1275) fixes for quickstart documentation
[ https://issues.apache.org/jira/browse/KAFKA-1275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13924661#comment-13924661 ] Joe Stein commented on KAFKA-1275: -- [~jkreps] looks fantastic! > fixes for quickstart documentation > -- > > Key: KAFKA-1275 > URL: https://issues.apache.org/jira/browse/KAFKA-1275 > Project: Kafka > Issue Type: Bug > Components: website >Affects Versions: 0.8.1 >Reporter: Evan Zacks >Assignee: Jay Kreps >Priority: Minor > Labels: documentation > Fix For: 0.8.1 > > Attachments: KAFKA-1275-quickstart-doc.patch > > > The quickstart guide refers to commands that no longer exist in the master > git branch per changes in KAFKA-554. > If changes for the documentation to match 0.8.1 are already in development > elsewhere, please feel free to discard this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (KAFKA-1275) fixes for quickstart documentation
[ https://issues.apache.org/jira/browse/KAFKA-1275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923060#comment-13923060 ] Joe Stein edited comment on KAFKA-1275 at 3/6/14 9:05 PM: -- Maybe an example after the paragraph "Log compaction is a mechanism to give finer-grained per-record retention, rather than the coarser-grained time-based retention. The idea is to selectively remove records where we have a more recent update with the same primary key. This way the log is guaranteed to have at least the last state for each key." Something like... Lets say you have a topic that every time inventory information for a product changes it is sent to a topic Message1 with Primary Key == PRODUCTID99876 is produced the inventory for PRODUCTID99876 changes Message 2 with Primary Key == PRODUCTID99876 is produced the inventory changes again for PRODUCTID99876 Message 3 with Primary Key == PRODUCTID99876 Now, since this data stream is only having the inventory information as a snapshot every old one is likely to not have relevance (since for this example we are only interested in the latest). Compaction would get rid of message 1 and 2 only leaving 3 as it is the current state... or pick something from the uses cases you mentioned above in the docs (my example is not very realistic just trying to express the thought though) I think it looks good (but now that I understand it I may no longer be very objective). Another option (instead of example or in addition too, dunno) is after the log compaction diagram (the before and after compaction) maybe explain that 0,2 are not copied because the primary K1 is latest state is 3 (so that is kept), 1,5 are not copied because the primary key K2 has the latest state as offset 9, etc... was (Author: joestein): Maybe an example after the paragraph "Log compaction is a mechanism to give finer-grained per-record retention, rather than the coarser-grained time-based retention. The idea is to selectively remove records where we have a more recent update with the same primary key. This way the log is guaranteed to have at least the last state for each key." Something like... Lets say you have a topic that every time inventory information for a product changes it is sent to a topic Message1 with Primary Key == PRODUCTID99876 is produced the inventory for PRODUCTID99876 changes Message 2 with Primary Key == PRODUCTID99876 is produced the inventory changes again for PRODUCTID99876 Message 3 with Primary Key == PRODUCTID99876 Now, since this data stream is only having the inventory information as a snapshot every old one is likely to not have relevance (since for this example we are only interested in the latest). Compaction would get rid of message 1 and 2 only leaving 3 as it is the current state... or pick something from the uses cases you mentioned above in the docs (my example is not very realistic just trying to express the thought though) I think it looks good (but now that I understand it I may no longer be very objective). Another option (instead of example or in addition too, dunno) is after the log compaction (before and after) diagram maybe explain that 0,2 are not copied because the primary K1 is latest state is 3 (so that is kept), 1,5 are not copied because the primary key K2 has the latest state as offset 9, etc... > fixes for quickstart documentation > -- > > Key: KAFKA-1275 > URL: https://issues.apache.org/jira/browse/KAFKA-1275 > Project: Kafka > Issue Type: Bug > Components: website >Affects Versions: 0.8.1 >Reporter: Evan Zacks >Assignee: Jay Kreps >Priority: Minor > Labels: documentation > Fix For: 0.8.1 > > Attachments: KAFKA-1275-quickstart-doc.patch > > > The quickstart guide refers to commands that no longer exist in the master > git branch per changes in KAFKA-554. > If changes for the documentation to match 0.8.1 are already in development > elsewhere, please feel free to discard this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1275) fixes for quickstart documentation
[ https://issues.apache.org/jira/browse/KAFKA-1275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923060#comment-13923060 ] Joe Stein commented on KAFKA-1275: -- Maybe an example after the paragraph "Log compaction is a mechanism to give finer-grained per-record retention, rather than the coarser-grained time-based retention. The idea is to selectively remove records where we have a more recent update with the same primary key. This way the log is guaranteed to have at least the last state for each key." Something like... Lets say you have a topic that every time inventory information for a product changes it is sent to a topic Message1 with Primary Key == PRODUCTID99876 is produced the inventory for PRODUCTID99876 changes Message 2 with Primary Key == PRODUCTID99876 is produced the inventory changes again for PRODUCTID99876 Message 3 with Primary Key == PRODUCTID99876 Now, since this data stream is only having the inventory information as a snapshot every old one is likely to not have relevance (since for this example we are only interested in the latest). Compaction would get rid of message 1 and 2 only leaving 3 as it is the current state... or pick something from the uses cases you mentioned above in the docs (my example is not very realistic just trying to express the thought though) I think it looks good (but now that I understand it I may no longer be very objective). Another option (instead of example or in addition too, dunno) is after the log compaction (before and after) diagram maybe explain that 0,2 are not copied because the primary K1 is latest state is 3 (so that is kept), 1,5 are not copied because the primary key K2 has the latest state as offset 9, etc... > fixes for quickstart documentation > -- > > Key: KAFKA-1275 > URL: https://issues.apache.org/jira/browse/KAFKA-1275 > Project: Kafka > Issue Type: Bug > Components: website >Affects Versions: 0.8.1 >Reporter: Evan Zacks >Assignee: Jay Kreps >Priority: Minor > Labels: documentation > Fix For: 0.8.1 > > Attachments: KAFKA-1275-quickstart-doc.patch > > > The quickstart guide refers to commands that no longer exist in the master > git branch per changes in KAFKA-554. > If changes for the documentation to match 0.8.1 are already in development > elsewhere, please feel free to discard this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1275) fixes for quickstart documentation
[ https://issues.apache.org/jira/browse/KAFKA-1275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923034#comment-13923034 ] Joe Stein commented on KAFKA-1275: -- ah ha! got it now :) VERY COOL. So the KeyedMessage val key:K is the primaryKey key (which is what is used to compact all older matching primary key messages leaving only the latest value (which for the use case is the only thing that would matter)) and the val partKey: Any is the partition key thrown away for deciding what partition to go on (same as < 0.8.1) Thanks > fixes for quickstart documentation > -- > > Key: KAFKA-1275 > URL: https://issues.apache.org/jira/browse/KAFKA-1275 > Project: Kafka > Issue Type: Bug > Components: website >Affects Versions: 0.8.1 >Reporter: Evan Zacks >Assignee: Jay Kreps >Priority: Minor > Labels: documentation > Fix For: 0.8.1 > > Attachments: KAFKA-1275-quickstart-doc.patch > > > The quickstart guide refers to commands that no longer exist in the master > git branch per changes in KAFKA-554. > If changes for the documentation to match 0.8.1 are already in development > elsewhere, please feel free to discard this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1275) fixes for quickstart documentation
[ https://issues.apache.org/jira/browse/KAFKA-1275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13922944#comment-13922944 ] Joe Stein commented on KAFKA-1275: -- I am a bit confused by "3.The offset for a message never changes, it is the permanent identifier for a position in the log. 4. Any read from offset 0 will contain see the final state of all records provided it completes in less than a configurable time period (even if further compaction occurs during this read). The third guarantee seems somewhat esoteric but this ensures that a reader will end in a coherent state (equivalent to reading the full uncompacted log) even though the underlying log may be compacted while this read occurs" shouldn't it be the fourth guarantee? I am not sure I understand number 4 (haven't looked at the code) does it mean you are reading the log and not the copy of the log since you started prior to compaction ending? Also does this mean now that the compaction could mean you might need ~ twice ~ as much disk space (assuming a scenario where almost the entire log is copied (nothing not held back from writing)... also the word "Offset" is missing from the diagram Log After Compaction. lastly it is not entirely clear exactly what will happen when you turn log.cleanup.policy=compact on for a topic, is there a specific class that gets executed ? what gets deleted? how do you control it? or do you have to build a compaction strategy or use an existing one? > fixes for quickstart documentation > -- > > Key: KAFKA-1275 > URL: https://issues.apache.org/jira/browse/KAFKA-1275 > Project: Kafka > Issue Type: Bug > Components: website >Affects Versions: 0.8.1 >Reporter: Evan Zacks >Assignee: Jay Kreps >Priority: Minor > Labels: documentation > Fix For: 0.8.1 > > Attachments: KAFKA-1275-quickstart-doc.patch > > > The quickstart guide refers to commands that no longer exist in the master > git branch per changes in KAFKA-554. > If changes for the documentation to match 0.8.1 are already in development > elsewhere, please feel free to discard this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1289) Misc. nitpicks in log cleaner
[ https://issues.apache.org/jira/browse/KAFKA-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13919889#comment-13919889 ] Joe Stein commented on KAFKA-1289: -- committed to 0.8.1 and trunk, thanks!!! > Misc. nitpicks in log cleaner > - > > Key: KAFKA-1289 > URL: https://issues.apache.org/jira/browse/KAFKA-1289 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.1 >Reporter: Jay Kreps > Fix For: 0.8.1 > > Attachments: KAFKA-1289-v1.patch > > > There are a couple of minor annoyances in the log cleaner in 0.8.1. Since > this is one of the major features it would be nice to address these. > Problems: > 1. Logging is no longer going to the kafka-cleaner.log > 2. Shutdown when the log cleaner is enabled is very slow > 3. TestLogCleaner uses obsolete properties for the producer and consumer > In addition I want to change the configuration from "dedupe" to "compact" as > we don't use the terminology dedupe anywhere else and I think it is less > intuitive. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (KAFKA-1288) add enclosing dir in release tar gz
[ https://issues.apache.org/jira/browse/KAFKA-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1288. -- Resolution: Fixed Fix Version/s: 0.8.1 applying to 0.8.1 > add enclosing dir in release tar gz > --- > > Key: KAFKA-1288 > URL: https://issues.apache.org/jira/browse/KAFKA-1288 > Project: Kafka > Issue Type: Bug > Components: packaging >Affects Versions: 0.8.1 >Reporter: Jun Rao >Assignee: Jun Rao > Fix For: 0.8.1 > > Attachments: KAFKA-1288.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (KAFKA-1289) Misc. nitpicks in log cleaner
[ https://issues.apache.org/jira/browse/KAFKA-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1289. -- Resolution: Fixed Fix Version/s: 0.8.1 applying to 0.8.1 branch > Misc. nitpicks in log cleaner > - > > Key: KAFKA-1289 > URL: https://issues.apache.org/jira/browse/KAFKA-1289 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.1 >Reporter: Jay Kreps > Fix For: 0.8.1 > > Attachments: KAFKA-1289-v1.patch > > > There are a couple of minor annoyances in the log cleaner in 0.8.1. Since > this is one of the major features it would be nice to address these. > Problems: > 1. Logging is no longer going to the kafka-cleaner.log > 2. Shutdown when the log cleaner is enabled is very slow > 3. TestLogCleaner uses obsolete properties for the producer and consumer > In addition I want to change the configuration from "dedupe" to "compact" as > we don't use the terminology dedupe anywhere else and I think it is less > intuitive. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1180) WhiteList topic filter gets a NullPointerException on complex Regex
[ https://issues.apache.org/jira/browse/KAFKA-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1180: - Fix Version/s: (was: 0.8.1) 0.8.2 > WhiteList topic filter gets a NullPointerException on complex Regex > --- > > Key: KAFKA-1180 > URL: https://issues.apache.org/jira/browse/KAFKA-1180 > Project: Kafka > Issue Type: Bug > Components: consumer >Affects Versions: 0.8.0 >Reporter: Jason Rosenberg >Assignee: Neha Narkhede > Fix For: 0.8.2 > > Attachments: KAFKA-1180.patch, KAFKA-1180.patch, > KAFKA-1180_2013-12-22_01:24:57.patch, KAFKA-1180_2014-02-09_20:21:49.patch, > KAFKA-1180_2014-02-13_14:50:40.patch, KAFKA-1180_2014-02-13_15:13:17.patch, > KAFKA-1180_2014-02-13_15:21:51.patch, KAFKA-1180_2014-02-13_15:23:28.patch > > > We are needing to create a stream selector that essentially combines the > logic of the BlackList and WhiteList classes (which is not easily exposed in > the high-level consumer api). That is, we want to select a topic that > contains a certain prefix, as long as it doesn't also contain a secondary > string. > This should be easy to do with ordinary java Regex's, but we're running into > some issues, trying to do this with the WhiteList class only. > We have a pattern that uses negative lookahead, like this: > "test-(?!bad\\b)[\\w]+" > So this should select a topic like: "test-good", but exclude a topic like > "test-bad", and also exclude a topic without the "test" prefix, like > "foo-bar". > Instead, what we see is a NullPointerException in the call to > createMessageStreamsByFilter (after having previously sent a message to > "test-good" followed by a message to "test-bad"): > 21700 > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683] > ERROR kafka.consumer.ConsumerFetcherThread - > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683], > Error due to > kafka.common.KafkaException: error processing data for partition [test-bad,0] > offset 0 > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:137) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$mcV$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:565) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > Caused by: java.lang.NullPointerException > at > kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > ... 9 more -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1210) Windows Bat files are not working properly
[ https://issues.apache.org/jira/browse/KAFKA-1210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1210: - Fix Version/s: (was: 0.8.1) 0.8.2 > Windows Bat files are not working properly > -- > > Key: KAFKA-1210 > URL: https://issues.apache.org/jira/browse/KAFKA-1210 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.1 > Environment: Windows >Reporter: Håkan Canberger >Priority: Trivial > Fix For: 0.8.2 > > Attachments: 1210-v1.patch > > > The bat files are not working properly. > The paths in them are invalid. > They have not been updated to reflect the changes made to the shell scripts. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-930) Integrate preferred replica election logic into kafka
[ https://issues.apache.org/jira/browse/KAFKA-930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-930: Fix Version/s: (was: 0.9.0) 0.8.1 > Integrate preferred replica election logic into kafka > - > > Key: KAFKA-930 > URL: https://issues.apache.org/jira/browse/KAFKA-930 > Project: Kafka > Issue Type: Bug >Reporter: Sriram Subramanian >Assignee: Sriram Subramanian > Fix For: 0.8.1 > > Attachments: KAFKA-930.patch, KAFKA-930_2013-11-19_17:37:29.patch, > KAFKA-930_2013-11-19_17:38:49.patch, KAFKA-930_2013-11-21_09:42:11.patch, > KAFKA-930_2013-12-09_22:51:57.patch, KAFKA-930_2013-12-20_11:13:01.patch, > KAFKA-930_2013-12-20_11:22:36.patch, KAFKA-930_2014-01-27_13:28:51.patch > > > It seems useful to integrate the preferred replica election logic into kafka > controller. A simple way to implement this would be to have a background > thread that periodically finds the topic partitions that are not assigned to > the preferred broker and initiate the move. We could come up with some > heuristics to initiate the move only if the imbalance over a specific > threshold in order to avoid rebalancing too aggressively. Making the software > do this reduces operational cost. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1271) controller logs exceptions during ZK session expiration
[ https://issues.apache.org/jira/browse/KAFKA-1271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1271: - Fix Version/s: 0.8.1 > controller logs exceptions during ZK session expiration > --- > > Key: KAFKA-1271 > URL: https://issues.apache.org/jira/browse/KAFKA-1271 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.1 >Reporter: Jun Rao >Assignee: Jun Rao > Fix For: 0.8.1 > > Attachments: KAFKA-1271.patch > > > Saw the following issues when there is ZK session expiration in the > controller. > 1. > ERROR Error handling event ZkEvent[Children of > /admin/delete_topics changed sent to > kafka.controller.PartitionStateMachine$DeleteTopicsListener@39abdac9] > (org.I0Itec.zkclient.ZkEventThread) > java.lang.NullPointerException > at > scala.collection.JavaConversions$JListWrapper.iterator(JavaConversions.scala:524) > at scala.collection.IterableLike$class.foreach(IterableLike.scala:79) > at > scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:521) > at > scala.collection.TraversableOnce$class.foldLeft(TraversableOnce.scala:176) > at > scala.collection.JavaConversions$JListWrapper.foldLeft(JavaConversions.scala:521) > at > scala.collection.TraversableOnce$class.$div$colon(TraversableOnce.scala:139) > at > scala.collection.JavaConversions$JListWrapper.$div$colon(JavaConversions.scala:521) > at scala.collection.generic.Addable$class.$plus$plus(Addable.scala:54) > at scala.collection.immutable.Set$EmptySet$.$plus$plus(Set.scala:47) > at scala.collection.TraversableOnce$class.toSet(TraversableOnce.scala:436) > at > scala.collection.JavaConversions$JListWrapper.toSet(JavaConversions.scala:521) > at > kafka.controller.PartitionStateMachine$DeleteTopicsListener$$anonfun$handleChildChange$2.apply$mcV$sp(PartitionStateMachine.scala:448) > at > kafka.controller.PartitionStateMachine$DeleteTopicsListener$$anonfun$handleChildChange$2.apply(PartitionStateMachine.scala:445) > at > kafka.controller.PartitionStateMachine$DeleteTopicsListener$$anonfun$handleChildChange$2.apply(PartitionStateMachine.scala:445) > at kafka.utils.Utils$.inLock(Utils.scala:538) > at > kafka.controller.PartitionStateMachine$DeleteTopicsListener.handleChildChange(PartitionStateMachine.scala:445) > at org.I0Itec.zkclient.ZkClient$7.run(ZkClient.java:570) > at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71) > 2. IllegalStateException due to "Kafka scheduler has not been started". -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1275) fixes for quickstart documentation
[ https://issues.apache.org/jira/browse/KAFKA-1275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13906635#comment-13906635 ] Joe Stein commented on KAFKA-1275: -- +1 > fixes for quickstart documentation > -- > > Key: KAFKA-1275 > URL: https://issues.apache.org/jira/browse/KAFKA-1275 > Project: Kafka > Issue Type: Bug > Components: website >Affects Versions: 0.8.1 >Reporter: Evan Zacks >Priority: Minor > Labels: documentation > Fix For: 0.8.1 > > Attachments: KAFKA-1275-quickstart-doc.patch > > > The quickstart guide refers to commands that no longer exist in the master > git branch per changes in KAFKA-554. > If changes for the documentation to match 0.8.1 are already in development > elsewhere, please feel free to discard this issue. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (KAFKA-1243) Gradle issues for release
[ https://issues.apache.org/jira/browse/KAFKA-1243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1243. -- Resolution: Fixed everything related to this has been taken care of and is committed to 0.8.1 and trunk > Gradle issues for release > - > > Key: KAFKA-1243 > URL: https://issues.apache.org/jira/browse/KAFKA-1243 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.1 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > Attachments: KAFKA-1243.patch, KAFKA-1243_2014-02-11_13:37:25.patch > > > This is the parent issue for all of the sub tasks found for the release that > are required to update on gradle. I think some of the changes are going to > be minor and in some cases not blockers but there are a bunch of them so we > can identify each minuscule item but have it a bit organized. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1254) remove vestigial sbt
[ https://issues.apache.org/jira/browse/KAFKA-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1254: - Resolution: Fixed Status: Resolved (was: Patch Available) double committed to 0.8.1 and trunk felt so good > remove vestigial sbt > > > Key: KAFKA-1254 > URL: https://issues.apache.org/jira/browse/KAFKA-1254 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.0 >Reporter: Joe Stein > Fix For: 0.8.1 > > Attachments: KAFKA-1254.patch, KAFKA-1254_2014-02-09_21:05:28.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1254) remove vestigial sbt
[ https://issues.apache.org/jira/browse/KAFKA-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13906605#comment-13906605 ] Joe Stein commented on KAFKA-1254: -- Yes! will remove the TODO also. > remove vestigial sbt > > > Key: KAFKA-1254 > URL: https://issues.apache.org/jira/browse/KAFKA-1254 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.0 >Reporter: Joe Stein > Fix For: 0.8.1 > > Attachments: KAFKA-1254.patch, KAFKA-1254_2014-02-09_21:05:28.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1274) gradle.properties needs the variables used in the build.gradle
[ https://issues.apache.org/jira/browse/KAFKA-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1274: - Resolution: Fixed Status: Resolved (was: Patch Available) double committed on 0.8.1 and trunk > gradle.properties needs the variables used in the build.gradle > -- > > Key: KAFKA-1274 > URL: https://issues.apache.org/jira/browse/KAFKA-1274 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.1 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > Attachments: KAFKA-1274.patch > > > These get overridden from the global .gradle/gradle.properties so having them > as blank is fine when not used or for folks that want to use them in the repo > project directly -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1274) gradle.properties needs the variables used in the build.gradle
[ https://issues.apache.org/jira/browse/KAFKA-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1274: - Status: Patch Available (was: Open) > gradle.properties needs the variables used in the build.gradle > -- > > Key: KAFKA-1274 > URL: https://issues.apache.org/jira/browse/KAFKA-1274 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.1 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > Attachments: KAFKA-1274.patch > > > These get overridden from the global .gradle/gradle.properties so having them > as blank is fine when not used or for folks that want to use them in the repo > project directly -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1274) gradle.properties needs the variables used in the build.gradle
[ https://issues.apache.org/jira/browse/KAFKA-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13905531#comment-13905531 ] Joe Stein commented on KAFKA-1274: -- Created reviewboard https://reviews.apache.org/r/18269/ against branch origin/0.8.1 > gradle.properties needs the variables used in the build.gradle > -- > > Key: KAFKA-1274 > URL: https://issues.apache.org/jira/browse/KAFKA-1274 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.1 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > Attachments: KAFKA-1274.patch > > > These get overridden from the global .gradle/gradle.properties so having them > as blank is fine when not used or for folks that want to use them in the repo > project directly -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1274) gradle.properties needs the variables used in the build.gradle
[ https://issues.apache.org/jira/browse/KAFKA-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1274: - Attachment: KAFKA-1274.patch > gradle.properties needs the variables used in the build.gradle > -- > > Key: KAFKA-1274 > URL: https://issues.apache.org/jira/browse/KAFKA-1274 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.1 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > Attachments: KAFKA-1274.patch > > > These get overridden from the global .gradle/gradle.properties so having them > as blank is fine when not used or for folks that want to use them in the repo > project directly -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (KAFKA-1274) gradle.properties needs the variables used in the build.gradle
Joe Stein created KAFKA-1274: Summary: gradle.properties needs the variables used in the build.gradle Key: KAFKA-1274 URL: https://issues.apache.org/jira/browse/KAFKA-1274 Project: Kafka Issue Type: Sub-task Affects Versions: 0.8.1 Reporter: Joe Stein Fix For: 0.8.1 These get overridden from the global .gradle/gradle.properties so having them as blank is fine when not used or for folks that want to use them in the repo project directly -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1274) gradle.properties needs the variables used in the build.gradle
[ https://issues.apache.org/jira/browse/KAFKA-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1274: - Priority: Blocker (was: Major) > gradle.properties needs the variables used in the build.gradle > -- > > Key: KAFKA-1274 > URL: https://issues.apache.org/jira/browse/KAFKA-1274 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.1 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > > These get overridden from the global .gradle/gradle.properties so having them > as blank is fine when not used or for folks that want to use them in the repo > project directly -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (KAFKA-1267) Corrupt 2.8.0_0.8.0 artifacts in maven central
[ https://issues.apache.org/jira/browse/KAFKA-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1267. -- Resolution: Duplicate Steve, this is a known issue KAFKA-1163 with a few workarounds. You can use any Scala version (like 2.10) in your client and it will work fine with the broker even though the broker is compiled with 2.8.0 (the same how python, c, c++ and the rest work with the broker everything is through the wire protocol). The issue was with the release not with the code so you can create your own artifacts if you are using Scala 2.8.0 and need that version. Any other questions feel free to ping me or send on the mailing list, happy to assist. > Corrupt 2.8.0_0.8.0 artifacts in maven central > -- > > Key: KAFKA-1267 > URL: https://issues.apache.org/jira/browse/KAFKA-1267 > Project: Kafka > Issue Type: Bug > Components: clients >Affects Versions: 0.8.0 > Environment: Affects 0.8.0 for Scala 2.8.0 only >Reporter: Steve Robenalt > Fix For: 0.8.0 > > > The kafka 0.8.0 jars on maven central for Scala 2.8.0 are corrupt ( the main > jar is 5.1 kb in size when it should be 2.4 Mb). Since 2.8.0_0.8.0 is the > only official downloadable Kafka version, and the corresponding jar for > client apps is the only one that is corrupted, it's impossible to do a maven > build without building a custom version of Kafka or overriding the maven > central client jar. > Please update the maven central artifacts associated with this release. > http://search.maven.org/#artifactdetails|org.apache.kafka|kafka_2.8.0|0.8.0|jar -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1245) the jar files and pom are not being signed so nexus is failing to publish them
[ https://issues.apache.org/jira/browse/KAFKA-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1245: - Resolution: Fixed Status: Resolved (was: Patch Available) double committed to 0.8.1 and trunk > the jar files and pom are not being signed so nexus is failing to publish them > -- > > Key: KAFKA-1245 > URL: https://issues.apache.org/jira/browse/KAFKA-1245 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.1 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > Attachments: KAFKA-1245.patch, KAFKA-1245_2014-02-13_14:41:18.patch > > > I have a fix for this I did already in a previous patch, just need to dig it > out and see about integrating it again and where it was left off -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (KAFKA-1263) Snazzy up the README markdown for better visibility on github
[ https://issues.apache.org/jira/browse/KAFKA-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1263. -- Resolution: Fixed double committed to 0.8.1 and trunk > Snazzy up the README markdown for better visibility on github > - > > Key: KAFKA-1263 > URL: https://issues.apache.org/jira/browse/KAFKA-1263 > Project: Kafka > Issue Type: Bug >Reporter: Joe Stein > Fix For: 0.8.1 > > Attachments: KAFKA-1263.patch > > > Here are my suggested changes so you can see them on github how they look > https://github.com/stealthly/samplereadmekafka/blob/master/README.md -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1180) WhiteList topic filter gets a NullPointerException on complex Regex
[ https://issues.apache.org/jira/browse/KAFKA-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900700#comment-13900700 ] Joe Stein commented on KAFKA-1180: -- version 3,4,5 were some issue with my local branches. cleaned that up and fixed in version 6 (latest) which is reviewable now added test cases and comments reviewed by Joel & Neha and missing Blacklist test reviewed by Guozhang > WhiteList topic filter gets a NullPointerException on complex Regex > --- > > Key: KAFKA-1180 > URL: https://issues.apache.org/jira/browse/KAFKA-1180 > Project: Kafka > Issue Type: Bug > Components: consumer >Affects Versions: 0.8.0 >Reporter: Jason Rosenberg >Assignee: Neha Narkhede > Fix For: 0.8.1 > > Attachments: KAFKA-1180.patch, KAFKA-1180.patch, > KAFKA-1180_2013-12-22_01:24:57.patch, KAFKA-1180_2014-02-09_20:21:49.patch, > KAFKA-1180_2014-02-13_14:50:40.patch, KAFKA-1180_2014-02-13_15:13:17.patch, > KAFKA-1180_2014-02-13_15:21:51.patch, KAFKA-1180_2014-02-13_15:23:28.patch > > > We are needing to create a stream selector that essentially combines the > logic of the BlackList and WhiteList classes (which is not easily exposed in > the high-level consumer api). That is, we want to select a topic that > contains a certain prefix, as long as it doesn't also contain a secondary > string. > This should be easy to do with ordinary java Regex's, but we're running into > some issues, trying to do this with the WhiteList class only. > We have a pattern that uses negative lookahead, like this: > "test-(?!bad\\b)[\\w]+" > So this should select a topic like: "test-good", but exclude a topic like > "test-bad", and also exclude a topic without the "test" prefix, like > "foo-bar". > Instead, what we see is a NullPointerException in the call to > createMessageStreamsByFilter (after having previously sent a message to > "test-good" followed by a message to "test-bad"): > 21700 > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683] > ERROR kafka.consumer.ConsumerFetcherThread - > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683], > Error due to > kafka.common.KafkaException: error processing data for partition [test-bad,0] > offset 0 > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:137) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$mcV$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:565) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > Caused by: java.lang.NullPointerException > at > kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > ... 9 more -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1180) WhiteList topic filter gets a NullPointerException on complex Regex
[ https://issues.apache.org/jira/browse/KAFKA-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900698#comment-13900698 ] Joe Stein commented on KAFKA-1180: -- Updated reviewboard https://reviews.apache.org/r/16718/ against branch origin/0.8.1 > WhiteList topic filter gets a NullPointerException on complex Regex > --- > > Key: KAFKA-1180 > URL: https://issues.apache.org/jira/browse/KAFKA-1180 > Project: Kafka > Issue Type: Bug > Components: consumer >Affects Versions: 0.8.0 >Reporter: Jason Rosenberg >Assignee: Neha Narkhede > Fix For: 0.8.1 > > Attachments: KAFKA-1180.patch, KAFKA-1180.patch, > KAFKA-1180_2013-12-22_01:24:57.patch, KAFKA-1180_2014-02-09_20:21:49.patch, > KAFKA-1180_2014-02-13_14:50:40.patch, KAFKA-1180_2014-02-13_15:13:17.patch, > KAFKA-1180_2014-02-13_15:21:51.patch, KAFKA-1180_2014-02-13_15:23:28.patch > > > We are needing to create a stream selector that essentially combines the > logic of the BlackList and WhiteList classes (which is not easily exposed in > the high-level consumer api). That is, we want to select a topic that > contains a certain prefix, as long as it doesn't also contain a secondary > string. > This should be easy to do with ordinary java Regex's, but we're running into > some issues, trying to do this with the WhiteList class only. > We have a pattern that uses negative lookahead, like this: > "test-(?!bad\\b)[\\w]+" > So this should select a topic like: "test-good", but exclude a topic like > "test-bad", and also exclude a topic without the "test" prefix, like > "foo-bar". > Instead, what we see is a NullPointerException in the call to > createMessageStreamsByFilter (after having previously sent a message to > "test-good" followed by a message to "test-bad"): > 21700 > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683] > ERROR kafka.consumer.ConsumerFetcherThread - > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683], > Error due to > kafka.common.KafkaException: error processing data for partition [test-bad,0] > offset 0 > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:137) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$mcV$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:565) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > Caused by: java.lang.NullPointerException > at > kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > ... 9 more -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1180) WhiteList topic filter gets a NullPointerException on complex Regex
[ https://issues.apache.org/jira/browse/KAFKA-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1180: - Attachment: KAFKA-1180_2014-02-13_15:23:28.patch > WhiteList topic filter gets a NullPointerException on complex Regex > --- > > Key: KAFKA-1180 > URL: https://issues.apache.org/jira/browse/KAFKA-1180 > Project: Kafka > Issue Type: Bug > Components: consumer >Affects Versions: 0.8.0 >Reporter: Jason Rosenberg >Assignee: Neha Narkhede > Fix For: 0.8.1 > > Attachments: KAFKA-1180.patch, KAFKA-1180.patch, > KAFKA-1180_2013-12-22_01:24:57.patch, KAFKA-1180_2014-02-09_20:21:49.patch, > KAFKA-1180_2014-02-13_14:50:40.patch, KAFKA-1180_2014-02-13_15:13:17.patch, > KAFKA-1180_2014-02-13_15:21:51.patch, KAFKA-1180_2014-02-13_15:23:28.patch > > > We are needing to create a stream selector that essentially combines the > logic of the BlackList and WhiteList classes (which is not easily exposed in > the high-level consumer api). That is, we want to select a topic that > contains a certain prefix, as long as it doesn't also contain a secondary > string. > This should be easy to do with ordinary java Regex's, but we're running into > some issues, trying to do this with the WhiteList class only. > We have a pattern that uses negative lookahead, like this: > "test-(?!bad\\b)[\\w]+" > So this should select a topic like: "test-good", but exclude a topic like > "test-bad", and also exclude a topic without the "test" prefix, like > "foo-bar". > Instead, what we see is a NullPointerException in the call to > createMessageStreamsByFilter (after having previously sent a message to > "test-good" followed by a message to "test-bad"): > 21700 > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683] > ERROR kafka.consumer.ConsumerFetcherThread - > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683], > Error due to > kafka.common.KafkaException: error processing data for partition [test-bad,0] > offset 0 > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:137) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$mcV$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:565) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > Caused by: java.lang.NullPointerException > at > kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > ... 9 more -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1180) WhiteList topic filter gets a NullPointerException on complex Regex
[ https://issues.apache.org/jira/browse/KAFKA-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900697#comment-13900697 ] Joe Stein commented on KAFKA-1180: -- Updated reviewboard https://reviews.apache.org/r/16718/ against branch origin/0.8.1 > WhiteList topic filter gets a NullPointerException on complex Regex > --- > > Key: KAFKA-1180 > URL: https://issues.apache.org/jira/browse/KAFKA-1180 > Project: Kafka > Issue Type: Bug > Components: consumer >Affects Versions: 0.8.0 >Reporter: Jason Rosenberg >Assignee: Neha Narkhede > Fix For: 0.8.1 > > Attachments: KAFKA-1180.patch, KAFKA-1180.patch, > KAFKA-1180_2013-12-22_01:24:57.patch, KAFKA-1180_2014-02-09_20:21:49.patch, > KAFKA-1180_2014-02-13_14:50:40.patch, KAFKA-1180_2014-02-13_15:13:17.patch, > KAFKA-1180_2014-02-13_15:21:51.patch > > > We are needing to create a stream selector that essentially combines the > logic of the BlackList and WhiteList classes (which is not easily exposed in > the high-level consumer api). That is, we want to select a topic that > contains a certain prefix, as long as it doesn't also contain a secondary > string. > This should be easy to do with ordinary java Regex's, but we're running into > some issues, trying to do this with the WhiteList class only. > We have a pattern that uses negative lookahead, like this: > "test-(?!bad\\b)[\\w]+" > So this should select a topic like: "test-good", but exclude a topic like > "test-bad", and also exclude a topic without the "test" prefix, like > "foo-bar". > Instead, what we see is a NullPointerException in the call to > createMessageStreamsByFilter (after having previously sent a message to > "test-good" followed by a message to "test-bad"): > 21700 > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683] > ERROR kafka.consumer.ConsumerFetcherThread - > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683], > Error due to > kafka.common.KafkaException: error processing data for partition [test-bad,0] > offset 0 > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:137) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$mcV$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:565) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > Caused by: java.lang.NullPointerException > at > kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > ... 9 more -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1180) WhiteList topic filter gets a NullPointerException on complex Regex
[ https://issues.apache.org/jira/browse/KAFKA-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1180: - Attachment: KAFKA-1180_2014-02-13_15:21:51.patch > WhiteList topic filter gets a NullPointerException on complex Regex > --- > > Key: KAFKA-1180 > URL: https://issues.apache.org/jira/browse/KAFKA-1180 > Project: Kafka > Issue Type: Bug > Components: consumer >Affects Versions: 0.8.0 >Reporter: Jason Rosenberg >Assignee: Neha Narkhede > Fix For: 0.8.1 > > Attachments: KAFKA-1180.patch, KAFKA-1180.patch, > KAFKA-1180_2013-12-22_01:24:57.patch, KAFKA-1180_2014-02-09_20:21:49.patch, > KAFKA-1180_2014-02-13_14:50:40.patch, KAFKA-1180_2014-02-13_15:13:17.patch, > KAFKA-1180_2014-02-13_15:21:51.patch > > > We are needing to create a stream selector that essentially combines the > logic of the BlackList and WhiteList classes (which is not easily exposed in > the high-level consumer api). That is, we want to select a topic that > contains a certain prefix, as long as it doesn't also contain a secondary > string. > This should be easy to do with ordinary java Regex's, but we're running into > some issues, trying to do this with the WhiteList class only. > We have a pattern that uses negative lookahead, like this: > "test-(?!bad\\b)[\\w]+" > So this should select a topic like: "test-good", but exclude a topic like > "test-bad", and also exclude a topic without the "test" prefix, like > "foo-bar". > Instead, what we see is a NullPointerException in the call to > createMessageStreamsByFilter (after having previously sent a message to > "test-good" followed by a message to "test-bad"): > 21700 > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683] > ERROR kafka.consumer.ConsumerFetcherThread - > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683], > Error due to > kafka.common.KafkaException: error processing data for partition [test-bad,0] > offset 0 > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:137) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$mcV$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:565) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > Caused by: java.lang.NullPointerException > at > kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > ... 9 more -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1180) WhiteList topic filter gets a NullPointerException on complex Regex
[ https://issues.apache.org/jira/browse/KAFKA-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1180: - Attachment: KAFKA-1180_2014-02-13_15:13:17.patch > WhiteList topic filter gets a NullPointerException on complex Regex > --- > > Key: KAFKA-1180 > URL: https://issues.apache.org/jira/browse/KAFKA-1180 > Project: Kafka > Issue Type: Bug > Components: consumer >Affects Versions: 0.8.0 >Reporter: Jason Rosenberg >Assignee: Neha Narkhede > Fix For: 0.8.1 > > Attachments: KAFKA-1180.patch, KAFKA-1180.patch, > KAFKA-1180_2013-12-22_01:24:57.patch, KAFKA-1180_2014-02-09_20:21:49.patch, > KAFKA-1180_2014-02-13_14:50:40.patch, KAFKA-1180_2014-02-13_15:13:17.patch > > > We are needing to create a stream selector that essentially combines the > logic of the BlackList and WhiteList classes (which is not easily exposed in > the high-level consumer api). That is, we want to select a topic that > contains a certain prefix, as long as it doesn't also contain a secondary > string. > This should be easy to do with ordinary java Regex's, but we're running into > some issues, trying to do this with the WhiteList class only. > We have a pattern that uses negative lookahead, like this: > "test-(?!bad\\b)[\\w]+" > So this should select a topic like: "test-good", but exclude a topic like > "test-bad", and also exclude a topic without the "test" prefix, like > "foo-bar". > Instead, what we see is a NullPointerException in the call to > createMessageStreamsByFilter (after having previously sent a message to > "test-good" followed by a message to "test-bad"): > 21700 > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683] > ERROR kafka.consumer.ConsumerFetcherThread - > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683], > Error due to > kafka.common.KafkaException: error processing data for partition [test-bad,0] > offset 0 > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:137) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$mcV$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:565) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > Caused by: java.lang.NullPointerException > at > kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > ... 9 more -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1180) WhiteList topic filter gets a NullPointerException on complex Regex
[ https://issues.apache.org/jira/browse/KAFKA-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900685#comment-13900685 ] Joe Stein commented on KAFKA-1180: -- Updated reviewboard https://reviews.apache.org/r/16718/ against branch origin/0.8.1 > WhiteList topic filter gets a NullPointerException on complex Regex > --- > > Key: KAFKA-1180 > URL: https://issues.apache.org/jira/browse/KAFKA-1180 > Project: Kafka > Issue Type: Bug > Components: consumer >Affects Versions: 0.8.0 >Reporter: Jason Rosenberg >Assignee: Neha Narkhede > Fix For: 0.8.1 > > Attachments: KAFKA-1180.patch, KAFKA-1180.patch, > KAFKA-1180_2013-12-22_01:24:57.patch, KAFKA-1180_2014-02-09_20:21:49.patch, > KAFKA-1180_2014-02-13_14:50:40.patch, KAFKA-1180_2014-02-13_15:13:17.patch > > > We are needing to create a stream selector that essentially combines the > logic of the BlackList and WhiteList classes (which is not easily exposed in > the high-level consumer api). That is, we want to select a topic that > contains a certain prefix, as long as it doesn't also contain a secondary > string. > This should be easy to do with ordinary java Regex's, but we're running into > some issues, trying to do this with the WhiteList class only. > We have a pattern that uses negative lookahead, like this: > "test-(?!bad\\b)[\\w]+" > So this should select a topic like: "test-good", but exclude a topic like > "test-bad", and also exclude a topic without the "test" prefix, like > "foo-bar". > Instead, what we see is a NullPointerException in the call to > createMessageStreamsByFilter (after having previously sent a message to > "test-good" followed by a message to "test-bad"): > 21700 > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683] > ERROR kafka.consumer.ConsumerFetcherThread - > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683], > Error due to > kafka.common.KafkaException: error processing data for partition [test-bad,0] > offset 0 > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:137) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$mcV$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:565) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > Caused by: java.lang.NullPointerException > at > kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > ... 9 more -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1180) WhiteList topic filter gets a NullPointerException on complex Regex
[ https://issues.apache.org/jira/browse/KAFKA-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900661#comment-13900661 ] Joe Stein commented on KAFKA-1180: -- The last patch uploaded just now is rebased on 0.8.1 and added blacklist test case that was missing too > WhiteList topic filter gets a NullPointerException on complex Regex > --- > > Key: KAFKA-1180 > URL: https://issues.apache.org/jira/browse/KAFKA-1180 > Project: Kafka > Issue Type: Bug > Components: consumer >Affects Versions: 0.8.0 >Reporter: Jason Rosenberg >Assignee: Neha Narkhede > Fix For: 0.8.1 > > Attachments: KAFKA-1180.patch, KAFKA-1180.patch, > KAFKA-1180_2013-12-22_01:24:57.patch, KAFKA-1180_2014-02-09_20:21:49.patch, > KAFKA-1180_2014-02-13_14:50:40.patch > > > We are needing to create a stream selector that essentially combines the > logic of the BlackList and WhiteList classes (which is not easily exposed in > the high-level consumer api). That is, we want to select a topic that > contains a certain prefix, as long as it doesn't also contain a secondary > string. > This should be easy to do with ordinary java Regex's, but we're running into > some issues, trying to do this with the WhiteList class only. > We have a pattern that uses negative lookahead, like this: > "test-(?!bad\\b)[\\w]+" > So this should select a topic like: "test-good", but exclude a topic like > "test-bad", and also exclude a topic without the "test" prefix, like > "foo-bar". > Instead, what we see is a NullPointerException in the call to > createMessageStreamsByFilter (after having previously sent a message to > "test-good" followed by a message to "test-bad"): > 21700 > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683] > ERROR kafka.consumer.ConsumerFetcherThread - > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683], > Error due to > kafka.common.KafkaException: error processing data for partition [test-bad,0] > offset 0 > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:137) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$mcV$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:565) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > Caused by: java.lang.NullPointerException > at > kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > ... 9 more -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1180) WhiteList topic filter gets a NullPointerException on complex Regex
[ https://issues.apache.org/jira/browse/KAFKA-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900660#comment-13900660 ] Joe Stein commented on KAFKA-1180: -- Updated reviewboard https://reviews.apache.org/r/16718/ against branch origin/0.8.1 > WhiteList topic filter gets a NullPointerException on complex Regex > --- > > Key: KAFKA-1180 > URL: https://issues.apache.org/jira/browse/KAFKA-1180 > Project: Kafka > Issue Type: Bug > Components: consumer >Affects Versions: 0.8.0 >Reporter: Jason Rosenberg >Assignee: Neha Narkhede > Fix For: 0.8.1 > > Attachments: KAFKA-1180.patch, KAFKA-1180.patch, > KAFKA-1180_2013-12-22_01:24:57.patch, KAFKA-1180_2014-02-09_20:21:49.patch, > KAFKA-1180_2014-02-13_14:50:40.patch > > > We are needing to create a stream selector that essentially combines the > logic of the BlackList and WhiteList classes (which is not easily exposed in > the high-level consumer api). That is, we want to select a topic that > contains a certain prefix, as long as it doesn't also contain a secondary > string. > This should be easy to do with ordinary java Regex's, but we're running into > some issues, trying to do this with the WhiteList class only. > We have a pattern that uses negative lookahead, like this: > "test-(?!bad\\b)[\\w]+" > So this should select a topic like: "test-good", but exclude a topic like > "test-bad", and also exclude a topic without the "test" prefix, like > "foo-bar". > Instead, what we see is a NullPointerException in the call to > createMessageStreamsByFilter (after having previously sent a message to > "test-good" followed by a message to "test-bad"): > 21700 > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683] > ERROR kafka.consumer.ConsumerFetcherThread - > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683], > Error due to > kafka.common.KafkaException: error processing data for partition [test-bad,0] > offset 0 > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:137) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$mcV$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:565) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > Caused by: java.lang.NullPointerException > at > kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > ... 9 more -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1180) WhiteList topic filter gets a NullPointerException on complex Regex
[ https://issues.apache.org/jira/browse/KAFKA-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1180: - Attachment: KAFKA-1180_2014-02-13_14:50:40.patch > WhiteList topic filter gets a NullPointerException on complex Regex > --- > > Key: KAFKA-1180 > URL: https://issues.apache.org/jira/browse/KAFKA-1180 > Project: Kafka > Issue Type: Bug > Components: consumer >Affects Versions: 0.8.0 >Reporter: Jason Rosenberg >Assignee: Neha Narkhede > Fix For: 0.8.1 > > Attachments: KAFKA-1180.patch, KAFKA-1180.patch, > KAFKA-1180_2013-12-22_01:24:57.patch, KAFKA-1180_2014-02-09_20:21:49.patch, > KAFKA-1180_2014-02-13_14:50:40.patch > > > We are needing to create a stream selector that essentially combines the > logic of the BlackList and WhiteList classes (which is not easily exposed in > the high-level consumer api). That is, we want to select a topic that > contains a certain prefix, as long as it doesn't also contain a secondary > string. > This should be easy to do with ordinary java Regex's, but we're running into > some issues, trying to do this with the WhiteList class only. > We have a pattern that uses negative lookahead, like this: > "test-(?!bad\\b)[\\w]+" > So this should select a topic like: "test-good", but exclude a topic like > "test-bad", and also exclude a topic without the "test" prefix, like > "foo-bar". > Instead, what we see is a NullPointerException in the call to > createMessageStreamsByFilter (after having previously sent a message to > "test-good" followed by a message to "test-bad"): > 21700 > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683] > ERROR kafka.consumer.ConsumerFetcherThread - > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683], > Error due to > kafka.common.KafkaException: error processing data for partition [test-bad,0] > offset 0 > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:137) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$mcV$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:565) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > Caused by: java.lang.NullPointerException > at > kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > ... 9 more -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1263) Snazzy up the README markdown for better visibility on github
[ https://issues.apache.org/jira/browse/KAFKA-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1263: - Attachment: KAFKA-1263.patch > Snazzy up the README markdown for better visibility on github > - > > Key: KAFKA-1263 > URL: https://issues.apache.org/jira/browse/KAFKA-1263 > Project: Kafka > Issue Type: Bug >Reporter: Joe Stein > Fix For: 0.8.1 > > Attachments: KAFKA-1263.patch > > > Here are my suggested changes so you can see them on github how they look > https://github.com/stealthly/samplereadmekafka/blob/master/README.md -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1263) Snazzy up the README markdown for better visibility on github
[ https://issues.apache.org/jira/browse/KAFKA-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900653#comment-13900653 ] Joe Stein commented on KAFKA-1263: -- Created reviewboard https://reviews.apache.org/r/18092/ against branch origin/0.8.1 > Snazzy up the README markdown for better visibility on github > - > > Key: KAFKA-1263 > URL: https://issues.apache.org/jira/browse/KAFKA-1263 > Project: Kafka > Issue Type: Bug >Reporter: Joe Stein > Fix For: 0.8.1 > > Attachments: KAFKA-1263.patch > > > Here are my suggested changes so you can see them on github how they look > https://github.com/stealthly/samplereadmekafka/blob/master/README.md -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (KAFKA-1248) jars are missing from maven upload that were previously there
[ https://issues.apache.org/jira/browse/KAFKA-1248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1248. -- Resolution: Won't Fix agree with Jun this is not necessary > jars are missing from maven upload that were previously there > - > > Key: KAFKA-1248 > URL: https://issues.apache.org/jira/browse/KAFKA-1248 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.0 >Reporter: Joe Stein > Fix For: 0.8.1 > > > I think the only one that perhaps matter is the test jar? > But here is what is there now for 0.8.0 but not in 0.8.1 > * xyz -javadoc.jar > * xyz -sources.jar > * xyz -test-javadoc.jar > * xyz -test-sources.jar > * xyz -test.jar > not sure what we want to have in there that isn't since none of them have POM > one could argue it was bad to have them there in the first place and the way > we are doing it now is right ... here is what exists today > http://central.maven.org/maven2/org/apache/kafka/kafka_2.10/0.8.0/ -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1245) the jar files and pom are not being signed so nexus is failing to publish them
[ https://issues.apache.org/jira/browse/KAFKA-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1245: - Attachment: KAFKA-1245_2014-02-13_14:41:18.patch > the jar files and pom are not being signed so nexus is failing to publish them > -- > > Key: KAFKA-1245 > URL: https://issues.apache.org/jira/browse/KAFKA-1245 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.1 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > Attachments: KAFKA-1245.patch, KAFKA-1245_2014-02-13_14:41:18.patch > > > I have a fix for this I did already in a previous patch, just need to dig it > out and see about integrating it again and where it was left off -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1245) the jar files and pom are not being signed so nexus is failing to publish them
[ https://issues.apache.org/jira/browse/KAFKA-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900648#comment-13900648 ] Joe Stein commented on KAFKA-1245: -- The artifacts were successfully re-staged to https://repository.apache.org/content/groups/staging > the jar files and pom are not being signed so nexus is failing to publish them > -- > > Key: KAFKA-1245 > URL: https://issues.apache.org/jira/browse/KAFKA-1245 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.1 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > Attachments: KAFKA-1245.patch, KAFKA-1245_2014-02-13_14:41:18.patch > > > I have a fix for this I did already in a previous patch, just need to dig it > out and see about integrating it again and where it was left off -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1245) the jar files and pom are not being signed so nexus is failing to publish them
[ https://issues.apache.org/jira/browse/KAFKA-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900647#comment-13900647 ] Joe Stein commented on KAFKA-1245: -- Updated reviewboard https://reviews.apache.org/r/17879/ against branch origin/0.8.1 > the jar files and pom are not being signed so nexus is failing to publish them > -- > > Key: KAFKA-1245 > URL: https://issues.apache.org/jira/browse/KAFKA-1245 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.1 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > Attachments: KAFKA-1245.patch, KAFKA-1245_2014-02-13_14:41:18.patch > > > I have a fix for this I did already in a previous patch, just need to dig it > out and see about integrating it again and where it was left off -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (KAFKA-1263) Snazzy up the README markdown for better visibility on github
Joe Stein created KAFKA-1263: Summary: Snazzy up the README markdown for better visibility on github Key: KAFKA-1263 URL: https://issues.apache.org/jira/browse/KAFKA-1263 Project: Kafka Issue Type: Bug Reporter: Joe Stein Fix For: 0.8.1 Here are my suggested changes so you can see them on github how they look https://github.com/stealthly/samplereadmekafka/blob/master/README.md -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1243) Gradle issues for release
[ https://issues.apache.org/jira/browse/KAFKA-1243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13899282#comment-13899282 ] Joe Stein commented on KAFKA-1243: -- Thanks Jun! Sorry I have been swamped the last few days. KAFKA-121 was there just so folks could review the POM as it is now and I agree with you it looks good so I resolved that ticket. I will rebase 1245 and give another release dry run through hopefully later today/tonight/tomorrow. Thanks! > Gradle issues for release > - > > Key: KAFKA-1243 > URL: https://issues.apache.org/jira/browse/KAFKA-1243 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.1 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > Attachments: KAFKA-1243.patch, KAFKA-1243_2014-02-11_13:37:25.patch > > > This is the parent issue for all of the sub tasks found for the release that > are required to update on gradle. I think some of the changes are going to > be minor and in some cases not blockers but there are a bunch of them so we > can identify each minuscule item but have it a bit organized. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (KAFKA-121) pom should include standard maven niceties
[ https://issues.apache.org/jira/browse/KAFKA-121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-121. - Resolution: Won't Fix These have all been taken care. > pom should include standard maven niceties > -- > > Key: KAFKA-121 > URL: https://issues.apache.org/jira/browse/KAFKA-121 > Project: Kafka > Issue Type: Sub-task > Components: packaging >Reporter: Chris Burroughs >Priority: Minor > Fix For: 0.8.1 > > Attachments: kafka_2.10.1-0.8.1.pom > > > * license info, name, description, etc > * webpage link > * parent tags for sub-modules > * groupid pending mailing list discussion > One refernce: > http://vasilrem.com/blog/software-development/from-sbt-to-maven-in-one-move/ -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1254) remove vestigial sbt
[ https://issues.apache.org/jira/browse/KAFKA-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1254: - Status: Patch Available (was: Open) > remove vestigial sbt > > > Key: KAFKA-1254 > URL: https://issues.apache.org/jira/browse/KAFKA-1254 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.0 >Reporter: Joe Stein > Fix For: 0.8.1 > > Attachments: KAFKA-1254.patch, KAFKA-1254_2014-02-09_21:05:28.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1254) remove vestigial sbt
[ https://issues.apache.org/jira/browse/KAFKA-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13896163#comment-13896163 ] Joe Stein commented on KAFKA-1254: -- Updated reviewboard https://reviews.apache.org/r/17885/ against branch origin/0.8.1 > remove vestigial sbt > > > Key: KAFKA-1254 > URL: https://issues.apache.org/jira/browse/KAFKA-1254 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.0 >Reporter: Joe Stein > Fix For: 0.8.1 > > Attachments: KAFKA-1254.patch, KAFKA-1254_2014-02-09_21:05:28.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1254) remove vestigial sbt
[ https://issues.apache.org/jira/browse/KAFKA-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1254: - Attachment: KAFKA-1254_2014-02-09_21:05:28.patch > remove vestigial sbt > > > Key: KAFKA-1254 > URL: https://issues.apache.org/jira/browse/KAFKA-1254 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.0 >Reporter: Joe Stein > Fix For: 0.8.1 > > Attachments: KAFKA-1254.patch, KAFKA-1254_2014-02-09_21:05:28.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1254) remove vestigial sbt
[ https://issues.apache.org/jira/browse/KAFKA-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1254: - Attachment: KAFKA-1254.patch > remove vestigial sbt > > > Key: KAFKA-1254 > URL: https://issues.apache.org/jira/browse/KAFKA-1254 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.0 >Reporter: Joe Stein > Fix For: 0.8.1 > > Attachments: KAFKA-1254.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1254) remove vestigial sbt
[ https://issues.apache.org/jira/browse/KAFKA-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13896160#comment-13896160 ] Joe Stein commented on KAFKA-1254: -- Created reviewboard https://reviews.apache.org/r/17885/ against branch origin/0.8.1 > remove vestigial sbt > > > Key: KAFKA-1254 > URL: https://issues.apache.org/jira/browse/KAFKA-1254 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.0 >Reporter: Joe Stein > Fix For: 0.8.1 > > Attachments: KAFKA-1254.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (KAFKA-1254) remove vestigial sbt
Joe Stein created KAFKA-1254: Summary: remove vestigial sbt Key: KAFKA-1254 URL: https://issues.apache.org/jira/browse/KAFKA-1254 Project: Kafka Issue Type: Sub-task Affects Versions: 0.8.0 Reporter: Joe Stein Fix For: 0.8.1 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1159) try to get the bin tar smaller
[ https://issues.apache.org/jira/browse/KAFKA-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1159: - Resolution: Won't Fix Status: Resolved (was: Patch Available) this is already done in the gradle build these are the libs left now jopt-simple-3.2.jar log4j-1.2.15.jar metrics-core-2.2.0.jar slf4j-api-1.7.2.jar zkclient-0.3.jar kafka_2.8.0-0.8.1.jar metrics-annotation-2.2.0.jar scala-library-2.8.0.jar snappy-java-1.0.5.jar zookeeper-3.3.4.jar > try to get the bin tar smaller > -- > > Key: KAFKA-1159 > URL: https://issues.apache.org/jira/browse/KAFKA-1159 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.8.0 >Reporter: Joe Stein > Labels: release > Fix For: 0.8.1 > > Attachments: KAFKA-1159.patch > > > "probably it makes sense to drop > libs/scala-compiler.jar -- kafka do not perform compilations during runtime > and this step will trim some fat from the resulting release (from 17 mb > down to 9.5 mb*). > * by the way using the best possible compression method (-9 instead of > default -6) + drop of compiler lib gave me the very same result -- 9.5 Mb" -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1180) WhiteList topic filter gets a NullPointerException on complex Regex
[ https://issues.apache.org/jira/browse/KAFKA-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13896140#comment-13896140 ] Joe Stein commented on KAFKA-1180: -- Updated reviewboard https://reviews.apache.org/r/16718/ against branch origin/0.8.1 > WhiteList topic filter gets a NullPointerException on complex Regex > --- > > Key: KAFKA-1180 > URL: https://issues.apache.org/jira/browse/KAFKA-1180 > Project: Kafka > Issue Type: Bug > Components: consumer >Affects Versions: 0.8.0 >Reporter: Jason Rosenberg >Assignee: Neha Narkhede > Fix For: 0.8.1 > > Attachments: KAFKA-1180.patch, KAFKA-1180.patch, > KAFKA-1180_2013-12-22_01:24:57.patch, KAFKA-1180_2014-02-09_20:21:49.patch > > > We are needing to create a stream selector that essentially combines the > logic of the BlackList and WhiteList classes (which is not easily exposed in > the high-level consumer api). That is, we want to select a topic that > contains a certain prefix, as long as it doesn't also contain a secondary > string. > This should be easy to do with ordinary java Regex's, but we're running into > some issues, trying to do this with the WhiteList class only. > We have a pattern that uses negative lookahead, like this: > "test-(?!bad\\b)[\\w]+" > So this should select a topic like: "test-good", but exclude a topic like > "test-bad", and also exclude a topic without the "test" prefix, like > "foo-bar". > Instead, what we see is a NullPointerException in the call to > createMessageStreamsByFilter (after having previously sent a message to > "test-good" followed by a message to "test-bad"): > 21700 > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683] > ERROR kafka.consumer.ConsumerFetcherThread - > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683], > Error due to > kafka.common.KafkaException: error processing data for partition [test-bad,0] > offset 0 > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:137) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$mcV$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:565) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > Caused by: java.lang.NullPointerException > at > kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > ... 9 more -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1180) WhiteList topic filter gets a NullPointerException on complex Regex
[ https://issues.apache.org/jira/browse/KAFKA-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1180: - Attachment: KAFKA-1180_2014-02-09_20:21:49.patch > WhiteList topic filter gets a NullPointerException on complex Regex > --- > > Key: KAFKA-1180 > URL: https://issues.apache.org/jira/browse/KAFKA-1180 > Project: Kafka > Issue Type: Bug > Components: consumer >Affects Versions: 0.8.0 >Reporter: Jason Rosenberg >Assignee: Neha Narkhede > Fix For: 0.8.1 > > Attachments: KAFKA-1180.patch, KAFKA-1180.patch, > KAFKA-1180_2013-12-22_01:24:57.patch, KAFKA-1180_2014-02-09_20:21:49.patch > > > We are needing to create a stream selector that essentially combines the > logic of the BlackList and WhiteList classes (which is not easily exposed in > the high-level consumer api). That is, we want to select a topic that > contains a certain prefix, as long as it doesn't also contain a secondary > string. > This should be easy to do with ordinary java Regex's, but we're running into > some issues, trying to do this with the WhiteList class only. > We have a pattern that uses negative lookahead, like this: > "test-(?!bad\\b)[\\w]+" > So this should select a topic like: "test-good", but exclude a topic like > "test-bad", and also exclude a topic without the "test" prefix, like > "foo-bar". > Instead, what we see is a NullPointerException in the call to > createMessageStreamsByFilter (after having previously sent a message to > "test-good" followed by a message to "test-bad"): > 21700 > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683] > ERROR kafka.consumer.ConsumerFetcherThread - > [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683], > Error due to > kafka.common.KafkaException: error processing data for partition [test-bad,0] > offset 0 > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:137) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$mcV$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:565) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > Caused by: java.lang.NullPointerException > at > kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > ... 9 more -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1244) The LICENSE and NOTICE are missing from the jar files
[ https://issues.apache.org/jira/browse/KAFKA-1244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1244: - Description: The LICENSE and NOTICE are in the release tar but are missing from the jar META-INF published to maven and the jar files within the release tar.(was: The LICENSE and NOTICE are in the release tar but are missing from the jar files published to maven and within the release tar. ) > The LICENSE and NOTICE are missing from the jar files > - > > Key: KAFKA-1244 > URL: https://issues.apache.org/jira/browse/KAFKA-1244 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.0 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > > The LICENSE and NOTICE are in the release tar but are missing from the jar > META-INF published to maven and the jar files within the release tar. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1244) The LICENSE and NOTICE are missing from the jar files
[ https://issues.apache.org/jira/browse/KAFKA-1244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1244: - Description: The LICENSE and NOTICE are in the release tar but are missing from the jar files published to maven and within the release tar.(was: This is both the release tar and the uploaded maven archive) > The LICENSE and NOTICE are missing from the jar files > - > > Key: KAFKA-1244 > URL: https://issues.apache.org/jira/browse/KAFKA-1244 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.0 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > > The LICENSE and NOTICE are in the release tar but are missing from the jar > files published to maven and within the release tar. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1245) the jar files and pom are not being signed so nexus is failing to publish them
[ https://issues.apache.org/jira/browse/KAFKA-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1245: - Affects Version/s: (was: 0.8.0) 0.8.1 Status: Patch Available (was: Open) The artifacts were successfully staged to https://repository.apache.org/content/groups/staging I did some initial testing and no issues that have not already been opened up as tickets sibling to this one as of yet > the jar files and pom are not being signed so nexus is failing to publish them > -- > > Key: KAFKA-1245 > URL: https://issues.apache.org/jira/browse/KAFKA-1245 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.1 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > Attachments: KAFKA-1245.patch > > > I have a fix for this I did already in a previous patch, just need to dig it > out and see about integrating it again and where it was left off -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1245) the jar files and pom are not being signed so nexus is failing to publish them
[ https://issues.apache.org/jira/browse/KAFKA-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895913#comment-13895913 ] Joe Stein commented on KAFKA-1245: -- Created reviewboard https://reviews.apache.org/r/17879/ against branch origin/0.8.1 > the jar files and pom are not being signed so nexus is failing to publish them > -- > > Key: KAFKA-1245 > URL: https://issues.apache.org/jira/browse/KAFKA-1245 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.0 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > Attachments: KAFKA-1245.patch > > > I have a fix for this I did already in a previous patch, just need to dig it > out and see about integrating it again and where it was left off -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1245) the jar files and pom are not being signed so nexus is failing to publish them
[ https://issues.apache.org/jira/browse/KAFKA-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1245: - Attachment: KAFKA-1245.patch > the jar files and pom are not being signed so nexus is failing to publish them > -- > > Key: KAFKA-1245 > URL: https://issues.apache.org/jira/browse/KAFKA-1245 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.0 >Reporter: Joe Stein >Priority: Blocker > Fix For: 0.8.1 > > Attachments: KAFKA-1245.patch > > > I have a fix for this I did already in a previous patch, just need to dig it > out and see about integrating it again and where it was left off -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1246) The 2.10 version is showing up as 2.10.1
[ https://issues.apache.org/jira/browse/KAFKA-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895912#comment-13895912 ] Joe Stein commented on KAFKA-1246: -- I tested this and it works in that "org.apache.kafka" % "kafka_2.10.1" % "0.8.1" works with a Scala 2.10.3 project so this ticket now comes down to aesthetics/consistency with 0.8.0 and if we want it to say 2.10 instead of 2.10.1 > The 2.10 version is showing up as 2.10.1 > > > Key: KAFKA-1246 > URL: https://issues.apache.org/jira/browse/KAFKA-1246 > Project: Kafka > Issue Type: Sub-task >Affects Versions: 0.8.0 >Reporter: Joe Stein > Fix For: 0.8.1 > > > This is arguably not a bad thing it is just different so I wanted to create a > ticket for it... If we could keep it consistent as 2.10 that would make > sense since Scala is now minor version compatible so either way really. > Need to confirm that the 2.10.1 is in fact working in 2.10.2 and 2.10.3 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (KAFKA-1247) 2.9.2 maven jars are not showing up
[ https://issues.apache.org/jira/browse/KAFKA-1247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1247. -- Resolution: Cannot Reproduce These were there just off the nexus repo screen, so no issue > 2.9.2 maven jars are not showing up > --- > > Key: KAFKA-1247 > URL: https://issues.apache.org/jira/browse/KAFKA-1247 > Project: Kafka > Issue Type: Bug >Reporter: Joe Stein >Priority: Blocker > > 2.8.0 > 2.8.1 > 2.9.1 > 2.10.1 > are all that is uploaded -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1247) 2.9.2 maven jars are not showing up
[ https://issues.apache.org/jira/browse/KAFKA-1247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1247: - Summary: 2.9.2 maven jars are not showing up (was: 2.9.2 maven jars are now showing up) > 2.9.2 maven jars are not showing up > --- > > Key: KAFKA-1247 > URL: https://issues.apache.org/jira/browse/KAFKA-1247 > Project: Kafka > Issue Type: Bug >Reporter: Joe Stein >Priority: Blocker > > 2.8.0 > 2.8.1 > 2.9.1 > 2.10.1 > are all that is uploaded -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1158) remove bin/run-rat.sh
[ https://issues.apache.org/jira/browse/KAFKA-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1158: - Resolution: Fixed Status: Resolved (was: Patch Available) double committed to 0.8.1 and trunk > remove bin/run-rat.sh > - > > Key: KAFKA-1158 > URL: https://issues.apache.org/jira/browse/KAFKA-1158 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.8.0 >Reporter: Joe Stein > Labels: release > Fix For: 0.8.1 > > Attachments: KAFKA-1158.patch > > > its vestigial we don't even use it during a release, just run rat from the > jar on its own... -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (KAFKA-1249) release tar name is different than 0.8.0
Joe Stein created KAFKA-1249: Summary: release tar name is different than 0.8.0 Key: KAFKA-1249 URL: https://issues.apache.org/jira/browse/KAFKA-1249 Project: Kafka Issue Type: Sub-task Affects Versions: 0.8.1 Reporter: Joe Stein Priority: Minor Fix For: 0.8.1 kafka_2.10.1-0.8.1-dist.tgz I think this is a minor issue since we can just rename prior to upload and signing -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (KAFKA-1248) jars are missing from maven upload that were previously there
Joe Stein created KAFKA-1248: Summary: jars are missing from maven upload that were previously there Key: KAFKA-1248 URL: https://issues.apache.org/jira/browse/KAFKA-1248 Project: Kafka Issue Type: Sub-task Affects Versions: 0.8.0 Reporter: Joe Stein Fix For: 0.8.1 I think the only one that perhaps matter is the test jar? But here is what is there now for 0.8.0 but not in 0.8.1 * xyz -javadoc.jar * xyz -sources.jar * xyz -test-javadoc.jar * xyz -test-sources.jar * xyz -test.jar not sure what we want to have in there that isn't since none of them have POM one could argue it was bad to have them there in the first place and the way we are doing it now is right ... here is what exists today http://central.maven.org/maven2/org/apache/kafka/kafka_2.10/0.8.0/ -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (KAFKA-1162) handle duplicate entry for ZK in the pom file
[ https://issues.apache.org/jira/browse/KAFKA-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1162. -- Resolution: Won't Fix moved from sbt to gradle, this was a sbt problem > handle duplicate entry for ZK in the pom file > - > > Key: KAFKA-1162 > URL: https://issues.apache.org/jira/browse/KAFKA-1162 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.8.0 >Reporter: Joe Stein > Labels: release > Fix For: 0.8.1 > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (KAFKA-1160) have the pom reference the exclusions necessary so folks don't have to
[ https://issues.apache.org/jira/browse/KAFKA-1160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1160. -- Resolution: Won't Fix moved from sbt to gradle, this was a sbt problem > have the pom reference the exclusions necessary so folks don't have to > -- > > Key: KAFKA-1160 > URL: https://issues.apache.org/jira/browse/KAFKA-1160 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.8.0 >Reporter: Joe Stein > Labels: release > Fix For: 0.8.1 > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-121) pom should include standard maven niceties
[ https://issues.apache.org/jira/browse/KAFKA-121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-121: Issue Type: Sub-task (was: Improvement) Parent: KAFKA-1243 > pom should include standard maven niceties > -- > > Key: KAFKA-121 > URL: https://issues.apache.org/jira/browse/KAFKA-121 > Project: Kafka > Issue Type: Sub-task > Components: packaging >Reporter: Chris Burroughs >Priority: Minor > Fix For: 0.8.1 > > Attachments: kafka_2.10.1-0.8.1.pom > > > * license info, name, description, etc > * webpage link > * parent tags for sub-modules > * groupid pending mailing list discussion > One refernce: > http://vasilrem.com/blog/software-development/from-sbt-to-maven-in-one-move/ -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-121) pom should include standard maven niceties
[ https://issues.apache.org/jira/browse/KAFKA-121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-121: Attachment: kafka_2.10.1-0.8.1.pom attached is the pom generated by the gradle build we can use this ticket to decide if the pom is looking how we want and if not make changes appropriately... or not if it is good to go all pom specific issues I think should be encompassed in this ticket > pom should include standard maven niceties > -- > > Key: KAFKA-121 > URL: https://issues.apache.org/jira/browse/KAFKA-121 > Project: Kafka > Issue Type: Sub-task > Components: packaging >Reporter: Chris Burroughs >Priority: Minor > Fix For: 0.8.1 > > Attachments: kafka_2.10.1-0.8.1.pom > > > * license info, name, description, etc > * webpage link > * parent tags for sub-modules > * groupid pending mailing list discussion > One refernce: > http://vasilrem.com/blog/software-development/from-sbt-to-maven-in-one-move/ -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1247) 2.9.2 maven jars are now showing up
[ https://issues.apache.org/jira/browse/KAFKA-1247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1247: - Priority: Blocker (was: Major) > 2.9.2 maven jars are now showing up > --- > > Key: KAFKA-1247 > URL: https://issues.apache.org/jira/browse/KAFKA-1247 > Project: Kafka > Issue Type: Bug >Reporter: Joe Stein >Priority: Blocker > > 2.8.0 > 2.8.1 > 2.9.1 > 2.10.1 > are all that is uploaded -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (KAFKA-1247) 2.9.2 maven jars are now showing up
Joe Stein created KAFKA-1247: Summary: 2.9.2 maven jars are now showing up Key: KAFKA-1247 URL: https://issues.apache.org/jira/browse/KAFKA-1247 Project: Kafka Issue Type: Bug Reporter: Joe Stein 2.8.0 2.8.1 2.9.1 2.10.1 are all that is uploaded -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (KAFKA-1246) The 2.10 version is showing up as 2.10.1
Joe Stein created KAFKA-1246: Summary: The 2.10 version is showing up as 2.10.1 Key: KAFKA-1246 URL: https://issues.apache.org/jira/browse/KAFKA-1246 Project: Kafka Issue Type: Sub-task Affects Versions: 0.8.0 Reporter: Joe Stein Fix For: 0.8.1 This is arguably not a bad thing it is just different so I wanted to create a ticket for it... If we could keep it consistent as 2.10 that would make sense since Scala is now minor version compatible so either way really. Need to confirm that the 2.10.1 is in fact working in 2.10.2 and 2.10.3 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (KAFKA-1245) the jar files and pom are not being signed so nexus is failing to publish them
Joe Stein created KAFKA-1245: Summary: the jar files and pom are not being signed so nexus is failing to publish them Key: KAFKA-1245 URL: https://issues.apache.org/jira/browse/KAFKA-1245 Project: Kafka Issue Type: Sub-task Affects Versions: 0.8.0 Reporter: Joe Stein Priority: Blocker Fix For: 0.8.1 I have a fix for this I did already in a previous patch, just need to dig it out and see about integrating it again and where it was left off -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (KAFKA-1244) The LICENSE and NOTICE are missing from the jar files
Joe Stein created KAFKA-1244: Summary: The LICENSE and NOTICE are missing from the jar files Key: KAFKA-1244 URL: https://issues.apache.org/jira/browse/KAFKA-1244 Project: Kafka Issue Type: Sub-task Affects Versions: 0.8.0 Reporter: Joe Stein Priority: Blocker Fix For: 0.8.1 This is both the release tar and the uploaded maven archive -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (KAFKA-1243) Gradle issues for release
Joe Stein created KAFKA-1243: Summary: Gradle issues for release Key: KAFKA-1243 URL: https://issues.apache.org/jira/browse/KAFKA-1243 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Joe Stein Priority: Blocker Fix For: 0.8.1 This is the parent issue for all of the sub tasks found for the release that are required to update on gradle. I think some of the changes are going to be minor and in some cases not blockers but there are a bunch of them so we can identify each minuscule item but have it a bit organized. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13894934#comment-13894934 ] Joe Stein commented on KAFKA-1171: -- yup, great, thanks that worked :) > Gradle build for Kafka > -- > > Key: KAFKA-1171 > URL: https://issues.apache.org/jira/browse/KAFKA-1171 > Project: Kafka > Issue Type: Improvement > Components: packaging >Affects Versions: 0.8.1, 0.9.0 >Reporter: David Arthur >Assignee: David Arthur >Priority: Blocker > Attachments: 0001-Adding-basic-Gradle-build.patch, > 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, > 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, > 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, > kafka-1171_v10.patch, kafka-1171_v11.patch, kafka-1171_v12.patch, > kafka-1171_v13.patch, kafka-1171_v14.patch, kafka-1171_v15.patch, > kafka-1171_v6.patch, kafka-1171_v7.patch, kafka-1171_v8.patch, > kafka-1171_v9.patch > > > We have previously discussed moving away from SBT to an > easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a > patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted > to check out Gradle as well. > 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13894916#comment-13894916 ] Joe Stein commented on KAFKA-1171: -- I tried to apply v15 (off a fresh trunk clone) git am --signoff < kafka-1171_v15.patch and got the "Patch does not have a valid e-mail address." error so I tried patch -p1 < kafka-1171_v15.patch and got all sorts of other errors from that... ./gradlew uploadArchivesAll Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain probably doing something wrong let me know so I can try out the upload artifacts, releases, build and the rest... didn't have more time to dig into I gotta run out will be back later tonight/tomorrow if anyone can help out thanks! > Gradle build for Kafka > -- > > Key: KAFKA-1171 > URL: https://issues.apache.org/jira/browse/KAFKA-1171 > Project: Kafka > Issue Type: Improvement > Components: packaging >Affects Versions: 0.8.1, 0.9.0 >Reporter: David Arthur >Assignee: David Arthur >Priority: Blocker > Attachments: 0001-Adding-basic-Gradle-build.patch, > 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, > 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, > 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, > kafka-1171_v10.patch, kafka-1171_v11.patch, kafka-1171_v12.patch, > kafka-1171_v13.patch, kafka-1171_v14.patch, kafka-1171_v15.patch, > kafka-1171_v6.patch, kafka-1171_v7.patch, kafka-1171_v8.patch, > kafka-1171_v9.patch > > > We have previously discussed moving away from SBT to an > easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a > patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted > to check out Gradle as well. > 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (KAFKA-1129) if we have a script to run the jar then we should include it in the build or remove it during release
[ https://issues.apache.org/jira/browse/KAFKA-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1129. -- Resolution: Incomplete > if we have a script to run the jar then we should include it in the build or > remove it during release > - > > Key: KAFKA-1129 > URL: https://issues.apache.org/jira/browse/KAFKA-1129 > Project: Kafka > Issue Type: Bug >Reporter: Joe Stein >Assignee: Joe Stein > Fix For: 0.8.1 > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1159) try to get the bin tar smaller
[ https://issues.apache.org/jira/browse/KAFKA-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1159: - Attachment: KAFKA-1159.patch > try to get the bin tar smaller > -- > > Key: KAFKA-1159 > URL: https://issues.apache.org/jira/browse/KAFKA-1159 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.8.0 >Reporter: Joe Stein > Labels: release > Fix For: 0.8.1 > > Attachments: KAFKA-1159.patch > > > "probably it makes sense to drop > libs/scala-compiler.jar -- kafka do not perform compilations during runtime > and this step will trim some fat from the resulting release (from 17 mb > down to 9.5 mb*). > * by the way using the best possible compression method (-9 instead of > default -6) + drop of compiler lib gave me the very same result -- 9.5 Mb" -- This message was sent by Atlassian JIRA (v6.1.5#6160)
