IMHO, I have been wondering that it is still a good strategy to couple Storm core and connectors with one version. That has been making painful situations in storm-kafka-client, where it could be marked as beta and release often with free to break backward compatibility until it is fairly stable, but it can't be released often because it requires Storm itself to be released which requires huge bootstrap.
2018년 1월 25일 (목) 오후 12:37, Jungtaek Lim <kabh...@gmail.com>님이 작성: > Hi Erik, > > Thanks for bring up issue. Yes you're right that all the patches are going > through 1.x-branch (1.2.0), but not 1.1.x-branch because of huge > divergence. We have limited resource so I'm afraid to say folks to port > back issues with much pain of conflicts. > > Maybe we could make somewhat crazy but easiest decision: break backward > compatibility once again for storm-kafka-client (sad), and just copy the > code on 1.x-branch to 1.1.x-branch and also 1.0.x-branch. 1.0.x-branch was > support for Kafka 0.9 (odd strategy btw), but once Kafka published > 1.0.0, I'd rather not thinking about Kafka 0.9. > > To. PMCs > Makes sense? Need to initiate another discussion thread? > > Thanks, > Jungtaek Lim (HeartSaVioR) > > 2018년 1월 25일 (목) 오전 10:48, P. Taylor Goetz <ptgo...@gmail.com>님이 작성: > >> Erik, >> >> Open and honest opinions are always welcome. Even if they may come across >> as harsh (and yours did not). We value your opinion, and that of the mesos >> community. >> >> I’d like to get these issues resolved and would ask others to help where >> they can. >> >> You’ve been very forthcoming with information to help resolve this. >> Please don’t stop. The more information the better. >> >> Thanks again, >> >> -Taylor >> >> >> >> >> >> > On Jan 24, 2018, at 8:12 PM, Erik Weathers >> <eweath...@groupon.com.INVALID> wrote: >> > >> > Hi Taylor, >> > >> > I apologize that this objection is a bit long, incomplete, and late for >> > this release vote. Also, I by no means intend this as an attack on the >> > good folks that develop and maintain storm-kafka-client. That being >> said, >> > I'm uncomfortable with the situation of storm-kafka-client in the >> > 1.0.x-branch (and possibly 1.1.x-branch too) and I wonder if everyone is >> > aware of the situation. There seem to be a number of important >> > storm-kafka-client changes that haven't been backported to the >> > 1.0.x-branch. e.g., >> > >> > (1) We discovered in storm-1.0.3 that a spout can get stuck forever if >> its >> > stored offsets fall behind the earliest available and >> > FirstPollOffsetStrategy is set to UNCOMMITTED_EARLIEST or >> > UNCOMMITTED_LATEST. We believe this behavior to be fixed in newer >> > branches, but not in 1.0.x-branch. The issue is here (note that the >> > fetchOffset doesn't get updated in the case that it was out of bounds >> for >> > the seek): >> > * https://github.com/apache/storm/blob/v1.0.6/external/ >> > storm-kafka-client/src/main/java/org/apache/storm/kafka/ >> > spout/KafkaSpout.java#L188-L192 >> > >> > (2) storm-1.0.x is using kafka-clients-0.9.0.1, which isn't acceptable >> when >> > using Kafka 0.10 due to the performance impact on the Kafka cluster. I >> > believe, perhaps naively, that we could use kafka-clients-0.10.x just >> fine >> > in storm-kafka-client-1.0.x, even when speaking to a Kafka 0.9 cluster. >> > (Obviously we'd have to fix any issues with the Kafka API usage from >> > storm-kafka-clients.) >> > >> > >> > Notably, this situation is especially problematic for storm-on-mesos, >> since >> > we *cannot* run anything newer than storm-1.0.x for the daemons (Nimbus, >> > Supervisor, Worker), because of a fundamental change that was made to >> the >> > storm-core scheduling logic. >> > >> > I haven't yet spent the time to fully analyze the set of changes made to >> > storm-kafka-client in the various active branches, so I apologize for a >> > lack of links to existing PRs and Jira tickets. I also need to file >> > tickets for some of this stuff, as I believe issue (1) above was fixed >> in >> > newer branches as part of a large patch for another issue. >> > >> > I also suppose that this isn't super actionable, since it might not be >> fair >> > to hold back the entire storm release just for these issues. However, >> if >> > we can get these issues fixed in 1.0.x-branch I hope we can cut the >> 1.0.7 >> > release soon thereafter. >> > >> > Thanks! >> > >> > - Erik >> > >> >> On Wed, Jan 24, 2018 at 10:41 AM, P. Taylor Goetz <ptgo...@gmail.com> >> wrote: >> >> >> >> This is a call to vote on releasing Apache Storm 1.0.6 (rc1) >> >> >> >> Full list of changes in this release: >> >> >> >> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1. >> >> 0.6-rc1/RELEASE_NOTES.html >> >> >> >> The tag/commit to be voted upon is v1.0.6: >> >> >> >> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h >> >> =24a421e34a71353dc6c750b1f026d06df8ead3f2;hb=bce45993f8622e4 >> >> d3e9ccba96cc78e4ef76e48ae >> >> >> >> The source archive being voted upon can be found here: >> >> >> >> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1. >> >> 0.6-rc1/apache-storm-1.0.6-src.tar.gz >> >> >> >> Other release files, signatures and digests can be found here: >> >> >> >> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.0.6-rc1/ >> >> >> >> The release artifacts are signed with the following key: >> >> >> >> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_p >> >> lain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd >> >> >> >> The Nexus staging repository for this release is: >> >> >> >> https://repository.apache.org/content/repositories/orgapachestorm-1054 >> >> >> >> Please vote on releasing this package as Apache Storm 1.0.6. >> >> >> >> When voting, please list the actions taken to verify the release. >> >> >> >> This vote will be open for at least 72 hours. >> >> >> >> [ ] +1 Release this package as Apache Storm 1.0.6 >> >> [ ] 0 No opinion >> >> [ ] -1 Do not release this package because... >> >> >> >> Thanks to everyone who contributed to this release. >> >> >> >> -Taylor >> >> >> >
