Hi Taylor, I apologize that this objection is a bit long, incomplete, and late for this release vote. Also, I by no means intend this as an attack on the good folks that develop and maintain storm-kafka-client. That being said, I'm uncomfortable with the situation of storm-kafka-client in the 1.0.x-branch (and possibly 1.1.x-branch too) and I wonder if everyone is aware of the situation. There seem to be a number of important storm-kafka-client changes that haven't been backported to the 1.0.x-branch. e.g.,
(1) We discovered in storm-1.0.3 that a spout can get stuck forever if its stored offsets fall behind the earliest available and FirstPollOffsetStrategy is set to UNCOMMITTED_EARLIEST or UNCOMMITTED_LATEST. We believe this behavior to be fixed in newer branches, but not in 1.0.x-branch. The issue is here (note that the fetchOffset doesn't get updated in the case that it was out of bounds for the seek): * https://github.com/apache/storm/blob/v1.0.6/external/ storm-kafka-client/src/main/java/org/apache/storm/kafka/ spout/KafkaSpout.java#L188-L192 (2) storm-1.0.x is using kafka-clients-0.9.0.1, which isn't acceptable when using Kafka 0.10 due to the performance impact on the Kafka cluster. I believe, perhaps naively, that we could use kafka-clients-0.10.x just fine in storm-kafka-client-1.0.x, even when speaking to a Kafka 0.9 cluster. (Obviously we'd have to fix any issues with the Kafka API usage from storm-kafka-clients.) Notably, this situation is especially problematic for storm-on-mesos, since we *cannot* run anything newer than storm-1.0.x for the daemons (Nimbus, Supervisor, Worker), because of a fundamental change that was made to the storm-core scheduling logic. I haven't yet spent the time to fully analyze the set of changes made to storm-kafka-client in the various active branches, so I apologize for a lack of links to existing PRs and Jira tickets. I also need to file tickets for some of this stuff, as I believe issue (1) above was fixed in newer branches as part of a large patch for another issue. I also suppose that this isn't super actionable, since it might not be fair to hold back the entire storm release just for these issues. However, if we can get these issues fixed in 1.0.x-branch I hope we can cut the 1.0.7 release soon thereafter. Thanks! - Erik On Wed, Jan 24, 2018 at 10:41 AM, P. Taylor Goetz <[email protected]> wrote: > This is a call to vote on releasing Apache Storm 1.0.6 (rc1) > > Full list of changes in this release: > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1. > 0.6-rc1/RELEASE_NOTES.html > > The tag/commit to be voted upon is v1.0.6: > > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h > =24a421e34a71353dc6c750b1f026d06df8ead3f2;hb=bce45993f8622e4 > d3e9ccba96cc78e4ef76e48ae > > The source archive being voted upon can be found here: > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1. > 0.6-rc1/apache-storm-1.0.6-src.tar.gz > > Other release files, signatures and digests can be found here: > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.0.6-rc1/ > > The release artifacts are signed with the following key: > > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_p > lain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd > > The Nexus staging repository for this release is: > > https://repository.apache.org/content/repositories/orgapachestorm-1054 > > Please vote on releasing this package as Apache Storm 1.0.6. > > When voting, please list the actions taken to verify the release. > > This vote will be open for at least 72 hours. > > [ ] +1 Release this package as Apache Storm 1.0.6 > [ ] 0 No opinion > [ ] -1 Do not release this package because... > > Thanks to everyone who contributed to this release. > > -Taylor >
