Hi Stig,
After having included commons-lang-2.5.jar in my topologies, I still have
11 topologies failing with this new message, this time a
NullPointerException:
Running: /usr/local/jdk/bin/java -client -Ddaemon.name= -Dstorm.options=
-Dstorm.home=/usr/local/Storm/storm-stable
-Dstorm.log.dir=/usr/local/Storm/storm-stable/logs
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
-Dstorm.conf.file= -cp
/usr/local/Storm/storm-stable/*:/usr/local/Storm/storm-stable/lib/*:/usr/local/Storm/storm-stable/extlib/*:/usr/local/Storm/StormTopologiesBJ.jar:/usr/local/Storm/storm-stable/conf:/usr/local/Storm/storm-stable/bin
-Dstorm.jar=/usr/local/Storm/StormTopologiesBJ.jar -Dstorm.dependency.jars=
-Dstorm.dependency.artifacts={}
com.acme.storm.evaluator.SLAEventsInterceptorTopology slaEventsInterceptor
1939 [main] WARN c.a.s.u.StormUtils - Couldn't read
/usr/local/Storm/flux/slaEventsInterceptor_kafkaConsumer.properties file:
/usr/local/Storm/flux/slaEventsInterceptor_kafkaConsumer.properties (No
such file or directory)
1978 [main] WARN o.a.s.k.s.KafkaSpoutConfig - Do not set
enable.auto.commit manually. Instead use
KafkaSpoutConfig.Builder.setProcessingGuarantee. This will be treated as an
error in the next major release. For now the spout will be configured to
behave like it would have in pre-1.2.0 releases.
1982 [main] INFO c.a.s.u.StormUtils - Use Kafka Spout:
com.acme.storm.evaluator.spout.BasicKafkaSpout
Exception in thread "main" java.lang.NullPointerException
at
com.acme.storm.evaluator.spout.BasicKafkaSpout.<init>(BasicKafkaSpout.java:72)
at com.acme.storm.util.StormUtils.getKafkaSpout(StormUtils.java:888)
at
com.acme.storm.evaluator.SLAEventsInterceptorTopology.main(SLAEventsInterceptorTopology.java:120)
and here's the line from our source from which this exception was triggered:
IRichSpout kafkaSpoutForMainTopic = StormUtils
.getKafkaSpout(spoutConfigForMainTopic, kafkaSpoutClass);
Looks like upgrade from 1.1.0 (our current production setup) to 1.2.0 is a
bit more difficult than I expected.
But anyway, I greatly appreciate Storm, so I hope my feedbacks will help
keeping it as good as possible :)
Please let me know if further details are required to solve this issue with
storm-kafka-client 1.2.0 snapshot.
Best regards,
Alexandre
2017-11-19 13:26 GMT+01:00 Stig Rohde Døssing <stigdoess...@gmail.com>:
> Using Maven and controlling dependencies aren't mutually exclusive. Tools
> like Nexus (https://help.sonatype.com/display/NXRM2/Procurement+Suite)
> allow you to control access to dependencies without manually putting jars
> in an SCM.
>
> I realize this doesn't help you right now, just thought I'd mention it so
> you don't get the idea that I'm advocating a build process that blindly
> includes new dependencies.
>
> 2017-11-19 13:05 GMT+01:00 Alexandre Vermeerbergen <
> avermeerber...@gmail.com
> >:
>
> > Believe it or not, we don't internally use Maven : we strictly control
> our
> > dependencies.
> >
> > So okay, I have delivered a "storm-commons-lang2.jar" in our SCM system
> in
> > order to specifically add it to our topologies big jars depending on
> > storm-kafka-client (feeling sad).
> >
> > I'll tell you the outcome ASAP.
> >
> > Best regard,
> > Alexandre Vermeerbergen
> >
> >
> > 2017-11-19 12:57 GMT+01:00 Stig Rohde Døssing <stigdoess...@gmail.com>:
> >
> > > I also don't think that adding dependencies constitutes a breaking
> > change,
> > > nor should it. Dependency management tooling like Maven will handle
> > pulling
> > > the right dependencies automatically.
> > >
> > > 2017-11-19 12:34 GMT+01:00 Jungtaek Lim <kabh...@gmail.com>:
> > >
> > > > Alexandre,
> > > >
> > > > There's so much pain on using relocated artifact in other module in a
> > > > project. You need to refer the class as relocated name, and your IDE
> > (at
> > > > least IntelliJ) complains and the compilation in IDE will not
> succeed.
> > > (The
> > > > build via Maven will succeed though.)
> > > > There was same issue on 2.0.0 as well, and for now we don't shade
> > > anything
> > > > in 2.0.0. We need to discuss which things to shade before discussing
> > > about
> > > > releasing, indeed.
> > > >
> > > > And normally we have been allowing adding dependency as minor
> release.
> > I
> > > > understand someone could say that it is breaking dependency, and
> should
> > > > lead another relocation as well. But my 2 cents the requirement is
> too
> > > > restrictive, and may end up with Guava-like versioning. (releasing
> > major
> > > > versions sooner...)
> > > > I think relocating is not an ideal solution, and eventually we may
> have
> > > > nice solution (like classloader isolation) and then we could sort out
> > all
> > > > the related issues. Before that, I could not block adding
> dependencies
> > in
> > > > minor release if it is necessary.
> > > >
> > > > Thanks,
> > > > Jungtaek Lim (HeartSaVioR)
> > > >
> > > > 2017년 11월 19일 (일) 오후 8:02, Stig Rohde Døssing <
> stigdoess...@gmail.com
> > >님이
> > > > 작성:
> > > >
> > > > > Alexandre,
> > > > >
> > > > > Thanks for trying. I think this time it's not a problem with
> > > > > storm-kafka-client. It has a dependency on commons-lang 2.5, which
> is
> > > > > declared in the storm-kafka-client pom. We usually recommend that
> > > people
> > > > > use Maven/Gradle/Ivy or similar systems to ensure that they get all
> > > > > dependencies when they build their topologies. In such a case Maven
> > (or
> > > > > similar) would see that your topology depends on
> storm-kafka-client,
> > > and
> > > > > also download and include commons-lang in the topology jar. When
> you
> > > put
> > > > > together dependencies manually you need to include commons-lang
> > > manually
> > > > as
> > > > > well.
> > > > >
> > > > > You can get the commons-lang 2.5 jar here
> > > > >
> > > > > http://central.maven.org/maven2/commons-lang/commons-
> > > > lang/2.5/commons-lang-2.5.jar
> > > > >
> > > > > 2017-11-19 11:49 GMT+01:00 Alexandre Vermeerbergen <
> > > > > avermeerber...@gmail.com
> > > > > >:
> > > > >
> > > > > > Hello Stig,
> > > > > >
> > > > > > Than you very much for you quick answer & fix.
> > > > > > Unfortunaly, I still have topologies failing to start but with a
> > > > > different
> > > > > > exception this time:
> > > > > >
> > > > > > Exception in thread "main" java.lang.NoClassDefFoundError:
> > > > > > org/apache/commons/lang/StringUtils
> > > > > > at
> > > > > > org.apache.storm.kafka.spout.NamedTopicFilter.getTopicsString(
> > > > > > NamedTopicFilter.java:66)
> > > > > > at
> > > > > > org.apache.storm.kafka.spout.ManualPartitionSubscription.
> > > > getTopicsString(
> > > > > > ManualPartitionSubscription.java:70)
> > > > > > at
> > > > > > com.acme.storm.evaluator.spout.BasicKafkaSpout.<init>(
> > > > > > BasicKafkaSpout.java:59)
> > > > > > at com.acme.storm.util.StormUtils.getKafkaSpout(
> > > > > > StormUtils.java:888)
> > > > > > at
> > > > > > com.acme.storm.evaluator.SLAEventsInterceptorTopology.main(
> > > > > > SLAEventsInterceptorTopology.java:120)
> > > > > > Caused by: java.lang.ClassNotFoundException:
> > > > > > org.apache.commons.lang.StringUtils
> > > > > > at java.net.URLClassLoader.
> findClass(URLClassLoader.java:
> > > 381)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> > > > > > at sun.misc.Launcher$AppClassLoader.loadClass(
> > > > Launcher.java:338)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> > > > > >
> > > > > > Looks like you introduced a dependency on commons lang that
> wasn't
> > > part
> > > > > of
> > > > > > Storm, isn't it ?
> > > > > >
> > > > > > Keep faith, next time will be better :)
> > > > > >
> > > > > > Best regards,
> > > > > > Alexandre
> > > > > >
> > > > > > 2017-11-19 10:44 GMT+01:00 Stig Rohde Døssing <
> > > stigdoess...@gmail.com
> > > > >:
> > > > > >
> > > > > > > I've put up a fix here https://github.com/apache/
> storm/pull/2426
> > .
> > > > > > There's
> > > > > > > an updated storm-kafka-client jar at
> > > > > > > https://drive.google.com/file/d/1DgJWjhWwczYgZS82YGd63V3GT2G_
> > > > > > > v9fd/view?usp=sharing
> > > > > > > if you'd like to try it out.
> > > > > > >
> > > > > > > 2017-11-19 10:17 GMT+01:00 Stig Rohde Døssing <
> > > > stigdoess...@gmail.com
> > > > > >:
> > > > > > >
> > > > > > > > Oops. The exception is not intentional, it's a bug. In 1.2.0
> we
> > > > check
> > > > > > for
> > > > > > > > the "enable.auto.commit" key in the kafkaConsumerProp map,
> and
> > if
> > > > it
> > > > > is
> > > > > > > set
> > > > > > > > we warn in the log that it shouldn't be, because users should
> > use
> > > > the
> > > > > > > > KafkaSpoutConfig.Builder.setProcessingGuarantee method
> > instead.
> > > > When
> > > > > > the
> > > > > > > > property is set we try to set the processing guarantee to
> match
> > > the
> > > > > > > > pre-1.2.0 behavior, but I made a mistake and assumed the
> > property
> > > > key
> > > > > > is
> > > > > > > a
> > > > > > > > boolean when it might actually be either a boolean or a
> string.
> > > > I'll
> > > > > > put
> > > > > > > up
> > > > > > > > a fix ASAP.
> > > > > > > >
> > > > > > > > 2017-11-19 <20%2017%2011%2019> 9:55 GMT+01:00 Alexandre
> > > > Vermeerbergen
> > > > > > <
> > > > > > > > avermeerber...@gmail.com>:
> > > > > > > >
> > > > > > > >> Hello Stig,
> > > > > > > >>
> > > > > > > >> Here's my first feedback on this Storm 1.2.0 preview on my
> > > > > Supervision
> > > > > > > >> system based on Storm : I have 11 topologies KO (not even
> able
> > > to
> > > > > > > start),
> > > > > > > >> and 4 topologies which seem to be unaffected.
> > > > > > > >>
> > > > > > > >> Details:
> > > > > > > >> - I used the binaries posted on dropbox by Jungteak and your
> > > > binary
> > > > > > for
> > > > > > > >> storm-kafka-client
> > > > > > > >> - Rebuilt all our topologies using these storm-core.jar &
> > > > > > > >> storm-kafka-client.jar from these storm-1.2.0-snapshot (and
> I
> > > got
> > > > > > > >> absolutely no error at build time).
> > > > > > > >> Meaning that I have new "big jar" files including this
> newer
> > > > > > > >> storm-kafka-client.jar (except for 1 topology which depends
> on
> > > > > another
> > > > > > > >> build system)
> > > > > > > >> - The 11 topologies which fail to start show up the
> following
> > > > trace
> > > > > > when
> > > > > > > >> they are submitted:
> > > > > > > >>
> > > > > > > >> Running: java -client -Ddaemon.name= -Dstorm.options=
> > > > > > > >> -Dstorm.home=/usr/local/Storm/storm-stable
> > > > > > > >> -Dstorm.log.dir=/usr/local/Storm/storm-stable/logs
> > > > > > > >> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
> > > > > > > >> -Dstorm.conf.file= -cp
> > > > > > > >> /usr/local/Storm/storm-stable/
> *:/usr/local/Storm/storm-stabl
> > > > > > > >> e/lib/*:/usr/local/Storm/storm-stable/extlib/*:/usr/local/
> > > > > > > >> Storm/StormTopologiesBJ.jar:/usr/local/Storm/storm-stable/
> > > > > > > >> conf:/usr/local/Storm/storm-stable/bin
> > > > > > > >> -Dstorm.jar=/usr/local/Storm/StormTopologiesBJ.jar
> > > > > > > >> -Dstorm.dependency.jars=
> > > > > > > >> -Dstorm.dependency.artifacts={}
> > > > > > > >> com.acme.storm.evaluator.SLAEventsInterceptorTopology
> > > > > > > >> slaEventsInterceptor
> > > > > > > >> Latest
> > > > > > > >> 1022 [main] INFO c.a.s.e.SLAEventsInterceptorTopology -
> > > > Topology :
> > > > > > > >> 'slaEventsInterceptor', KafkaOffsetTimeStrategy = 'Latest'
> > > > > > > >> 1026 [main] WARN c.a.s.u.StormUtils - Couldn't read
> > > > > > > >> /usr/local/Storm/flux/slaEventsInterceptor_
> > > > kafkaConsumer.properties
> > > > > > > file:
> > > > > > > >> /usr/local/Storm/flux/slaEventsInterceptor_
> > > > kafkaConsumer.properties
> > > > > > (No
> > > > > > > >> such file or directory)
> > > > > > > >> 1037 [main] WARN o.a.s.k.s.KafkaSpoutConfig - Do not set
> > > > > > > >> enable.auto.commit manually. Instead use
> > > > > > > >> KafkaSpoutConfig.Builder.setProcessingGuarantee. This will
> be
> > > > > treated
> > > > > > > as
> > > > > > > >> an
> > > > > > > >> error in the next major release. For now the spout will be
> > > > > configured
> > > > > > to
> > > > > > > >> behave like it would have in pre-1.2.0 releases.
> > > > > > > >> Exception in thread "main" java.lang.ClassCastException:
> > > > > > > java.lang.String
> > > > > > > >> cannot be cast to java.lang.Boolean
> > > > > > > >> at
> > > > > > > >> org.apache.storm.kafka.spout.KafkaSpoutConfig.
> setAutoCommitM
> > > > > > > >> ode(KafkaSpoutConfig.java:721)
> > > > > > > >> at
> > > > > > > >> org.apache.storm.kafka.spout.KafkaSpoutConfig.<init>(
> KafkaSp
> > > > > > > >> outConfig.java:97)
> > > > > > > >> at
> > > > > > > >> org.apache.storm.kafka.spout.KafkaSpoutConfig$Builder.
> build(
> > > > > > > >> KafkaSpoutConfig.java:671)
> > > > > > > >> at
> > > > > > > >> com.acme.storm.evaluator.SLAEventsInterceptorTopology.
> main(S
> > > > > > > >> LAEventsInterceptorTopology.java:118)
> > > > > > > >>
> > > > > > > >> The line part of our application which triggered above stack
> > > trace
> > > > > is
> > > > > > > the
> > > > > > > >> following one:
> > > > > > > >>
> > > > > > > >> KafkaSpoutConfig<String, Event>
> > spoutConfigForMainTopic
> > > =
> > > > > > > >> KafkaSpoutConfig
> > > > > > > >> .builder(elasticKafkaBrokers,
> > KafkaTopics.MAIN)
> > > > > > > >> .setValue(EventKafkaDeserializer.class)
> > > > > > > >> .setGroupId(consumerId + "_" +
> > KafkaTopics.MAIN)
> > > > > > > >> .setFirstPollOffsetStrategy(strategy)
> > > > > > > >> .setProp(kafkaConsumerProp)
> > > > > > > >> .setRecordTranslator(
> > > > > > > >> new SLAEventsInterceptorKafkaRecor
> > > > > > > dTranslator())
> > > > > > > >> .build();
> > > > > > > >>
> > > > > > > >> I understand that there's a WARN message, but if it's a
> > warning,
> > > > > then
> > > > > > > why
> > > > > > > >> do we exit with such a brutal exception?
> > > > > > > >>
> > > > > > > >> Best regards,
> > > > > > > >> Alexandre Vermeerbergen
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> 2017-11-18 14:54 GMT+01:00 Stig Rohde Døssing <
> > > > > stigdoess...@gmail.com
> > > > > > >:
> > > > > > > >>
> > > > > > > >> > Alexandre,
> > > > > > > >> >
> > > > > > > >> > I've uploaded the 1.2.0-SNAPSHOT storm-kafka-client jar
> here
> > > > > > > >> > https://drive.google.com/file/d/
> > 1DgJWjhWwczYgZS82YGd63V3GT2G_
> > > > > > > >> > v9fd/view?usp=sharing.
> > > > > > > >> > You can probably keep the 1.1.0 versions of storm-hbase
> and
> > > > > > > storm-hdfs,
> > > > > > > >> > since the core Storm API hasn't changed.
> > > > > > > >> >
> > > > > > > >> > If you want to build the jars yourself, clone the
> repository
> > > at
> > > > > > > >> > https://github.com/apache/storm and check out the
> > 1.x-branch
> > > > > > branch.
> > > > > > > >> You
> > > > > > > >> > can build the entire project by running "mvn clean install
> > > > > > > -DskipTests"
> > > > > > > >> in
> > > > > > > >> > the root. The individual module jars will be available in
> > the
> > > > > target
> > > > > > > >> > directories for each module, e.g. storm-kafka-client.jar
> > will
> > > be
> > > > > in
> > > > > > > >> > external/storm-kafka-client/target.
> > > > > > > >> >
> > > > > > > >> > If you need to build the regular Storm distribution (the
> > > tarball
> > > > > > > >> Jungtaek
> > > > > > > >> > sent you), you can go to storm-dist/binary and run "mvn
> > clean
> > > > > > install
> > > > > > > >> > -Dgpg.skip". The tar/zip will be in
> > storm-dist/binary/target.
> > > > > > > >> >
> > > > > > > >> > Further reference in case you need it
> > > > > > > >> > https://github.com/apache/storm/blob/master/DEVELOPER.
> > > > > > > >> > md#build-the-code-and-run-the-tests
> > > > > > > >> > .
> > > > > > > >> >
> > > > > > > >> > 2017-11-18 12:57 GMT+01:00 Alexandre Vermeerbergen <
> > > > > > > >> > avermeerber...@gmail.com
> > > > > > > >> > >:
> > > > > > > >> >
> > > > > > > >> > > Hello Jungtaek,
> > > > > > > >> > >
> > > > > > > >> > > Thanks for the link to 1.2.0 preview binaries.
> > > > > > > >> > > However, since we heavily depend on Storm Kafka Client,
> > > would
> > > > > you
> > > > > > > >> please
> > > > > > > >> > > either add it or remind me how to build the jars of this
> > > > > external
> > > > > > > >> lib, at
> > > > > > > >> > > same "snapshot" version as the rest ?
> > > > > > > >> > > On a side note, some of our topologies also rely on
> storm
> > > > hbase
> > > > > &
> > > > > > > >> storm
> > > > > > > >> > > hdfs : does it matters if these later ones stay in 1.1.0
> > > > version
> > > > > > for
> > > > > > > >> this
> > > > > > > >> > > test?
> > > > > > > >> > > Indeed, I want to focus on Storm 1.2.0 "core" + Storm
> > Kafka
> > > > > Client
> > > > > > > >> 1.2.0
> > > > > > > >> > >
> > > > > > > >> > > Best regards,
> > > > > > > >> > > Alexandre Vermeerbergen
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > > 2017-11-18 <20%2017%2011%2018> 7:41 GMT+01:00 Jungtaek
> > Lim <
> > > > > > > >> kabh...@gmail.com>:
> > > > > > > >> > >
> > > > > > > >> > > > Alexandre,
> > > > > > > >> > > >
> > > > > > > >> > > > https://www.dropbox.com/s/
> mg2gnunk24oesyc/apache-storm-
> > > > > > > >> > > > 1.2.0-SNAPSHOT.tar.gz?dl=0
> > > > > > > >> > > >
> > > > > > > >> > > > Above link is custom binary distribution of current
> > > > 1.x-branch
> > > > > > > >> > (SNAPSHOT
> > > > > > > >> > > of
> > > > > > > >> > > > 1.2.0). Could you run the test in your environment
> first
> > > and
> > > > > let
> > > > > > > us
> > > > > > > >> > know
> > > > > > > >> > > > about the result? Regardless of including metrics V2
> to
> > > > 1.2.0
> > > > > or
> > > > > > > >> not,
> > > > > > > >> > > your
> > > > > > > >> > > > test report should be valuable for us.
> > > > > > > >> > > >
> > > > > > > >> > > > And please refer my analysis of current metrics:
> > > > > > > >> > > > https://cwiki.apache.org/confluence/display/STORM/
> > > > > > > >> > > Limitations+of+current+
> > > > > > > >> > > > metrics+feature
> > > > > > > >> > > > to
> > > > > > > >> > > > see why we want to move toward to Metrics V2. Current
> PR
> > > on
> > > > > > > Metrics
> > > > > > > >> V2
> > > > > > > >> > is
> > > > > > > >> > > > for initial state and it could cover only some of
> issues
> > > in
> > > > > > list,
> > > > > > > >> but I
> > > > > > > >> > > > expect we will address another issues as well based on
> > the
> > > > PR.
> > > > > > > >> > > >
> > > > > > > >> > > > Thanks in advance!
> > > > > > > >> > > > Jungtaek Lim (HeartSaVioR)
> > > > > > > >> > > >
> > > > > > > >> > > > 2017년 11월 18일 (토) 오전 6:40, Alexandre Vermeerbergen <
> > > > > > > >> > > > avermeerber...@gmail.com>님이
> > > > > > > >> > > > 작성:
> > > > > > > >> > > >
> > > > > > > >> > > > > Hello Hugo,
> > > > > > > >> > > > >
> > > > > > > >> > > > > As I already posted, I was getting ready to test the
> > > > > upcoming
> > > > > > > RC,
> > > > > > > >> so
> > > > > > > >> > > > yeah :
> > > > > > > >> > > > > as soon as I have some binaries for testing, I can
> > run
> > > > any
> > > > > > > Storm
> > > > > > > >> > 1.2.0
> > > > > > > >> > > > > preview on my pre-production (well stressed)
> > > environment.
> > > > Is
> > > > > > > there
> > > > > > > >> > some
> > > > > > > >> > > > URL
> > > > > > > >> > > > > from where I can download such "snapshot" binaries?
> > > > > > > >> > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > > > Best regards,
> > > > > > > >> > > > > Alexandre Vermeerbergen
> > > > > > > >> > > > >
> > > > > > > >> > > > > PS: By the way, I have no interest in the
> > > metrics-related
> > > > > > > >> > enhancements
> > > > > > > >> > > > you
> > > > > > > >> > > > > mentionned - I can understand it is of interest for
> > > people
> > > > > > aware
> > > > > > > >> of
> > > > > > > >> > it
> > > > > > > >> > > > and
> > > > > > > >> > > > > who are expecting "something" related to it - but I
> > > would
> > > > be
> > > > > > sad
> > > > > > > >> to
> > > > > > > >> > > see a
> > > > > > > >> > > > > postponing of the imminent Storm 1.2.0 release for
> > > > something
> > > > > > we
> > > > > > > do
> > > > > > > >> > not
> > > > > > > >> > > > use
> > > > > > > >> > > > > at all.
> > > > > > > >> > > > > PPS: I would love to learn more about these
> "metrics",
> > > > maybe
> > > > > > > it's
> > > > > > > >> > > > something
> > > > > > > >> > > > > worth which never quite understood... any link with
> > > clear
> > > > > > > >> explanation
> > > > > > > >> > > of
> > > > > > > >> > > > > this feature?
> > > > > > > >> > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > > > 2017-11-17 17:44 GMT+01:00 Hugo Da Cruz Louro <
> > > > > > > >> > hlo...@hortonworks.com
> > > > > > > >> > > >:
> > > > > > > >> > > > >
> > > > > > > >> > > > > > I am also in agreement that we should not delay
> the
> > > > 1.2.0
> > > > > > > >> release
> > > > > > > >> > for
> > > > > > > >> > > > too
> > > > > > > >> > > > > > long, but in order to release it a few things need
> > to
> > > be
> > > > > > kept
> > > > > > > in
> > > > > > > >> > > mind:
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > - If we want to avoid releasing 1.3.0 soon or at
> > all,
> > > > > that
> > > > > > > >> means
> > > > > > > >> > > that
> > > > > > > >> > > > we
> > > > > > > >> > > > > > should include in 1.2.0 as many important features
> > > (e.g.
> > > > > > > >> metricsV2)
> > > > > > > >> > > as
> > > > > > > >> > > > > > possible within a reasonable time frame. Also to
> > avoid
> > > > as
> > > > > > much
> > > > > > > >> back
> > > > > > > >> > > > > porting
> > > > > > > >> > > > > > as possible one should really get all features
> that
> > we
> > > > > > foresee
> > > > > > > >> we
> > > > > > > >> > > want
> > > > > > > >> > > > to
> > > > > > > >> > > > > > maintain in 1.2.0 right away, and then simply
> > maintain
> > > > > them
> > > > > > > >> there.
> > > > > > > >> > > > > However
> > > > > > > >> > > > > > it’s a fact of life that if a bug is found and it
> > is a
> > > > > > > blocker,
> > > > > > > >> it
> > > > > > > >> > > will
> > > > > > > >> > > > > > have to be back ported.
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > - STORM-2153 details that the new metrics
> > requirements
> > > > > were
> > > > > > > >> driven
> > > > > > > >> > by
> > > > > > > >> > > > the
> > > > > > > >> > > > > > users. Therefore if this feature is highly
> important
> > > and
> > > > > > > sought
> > > > > > > >> > > after,
> > > > > > > >> > > > it
> > > > > > > >> > > > > > may not make much sense to release 1.2.0 without
> it.
> > > If
> > > > we
> > > > > > do
> > > > > > > >> > release
> > > > > > > >> > > > > 1.2.0
> > > > > > > >> > > > > > without the metrics, does it mean that metrics
> will
> > go
> > > > > only
> > > > > > in
> > > > > > > >> > 2.0? I
> > > > > > > >> > > > > would
> > > > > > > >> > > > > > conjecture that most production deployments will
> > take
> > > a
> > > > > > while
> > > > > > > to
> > > > > > > >> > > > upgrade
> > > > > > > >> > > > > to
> > > > > > > >> > > > > > 2.0 even after it is released. That means that
> they
> > > will
> > > > > > still
> > > > > > > >> be
> > > > > > > >> > > > running
> > > > > > > >> > > > > > Storm without the benefit of the new metrics.
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > - Several fixes have got in storm-kafka-client.
> > There
> > > > > have
> > > > > > > >> been a
> > > > > > > >> > > lot
> > > > > > > >> > > > of
> > > > > > > >> > > > > > changes and I wonder to which degree they have
> been
> > > > system
> > > > > > > >> tested
> > > > > > > >> > in
> > > > > > > >> > > > > > addition the existing unit tests
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > @Alexandre, since you are using storm-kafka-client
> > and
> > > > > have
> > > > > > > >> filed
> > > > > > > >> > > some
> > > > > > > >> > > > > > bugs and driven some feature requests, I would
> like
> > to
> > > > ask
> > > > > > if
> > > > > > > >> you
> > > > > > > >> > > could
> > > > > > > >> > > > > > help us within what is reasonably possible for you
> > > with
> > > > > the
> > > > > > > >> > > following:
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > - system test the latest storm-kafka-client
> changes
> > > in
> > > > > your
> > > > > > > >> > > > > > test/pre-production environment
> > > > > > > >> > > > > > - provide some information on about your setup,
> > > > > > > >> > > > > > - how you are using storm-kafka-client
> > > > > > > >> > > > > > - how it is performing
> > > > > > > >> > > > > > - Kafka brokers, number of
> topics/partitions,
> > > > storm
> > > > > > > >> > > parallelism,
> > > > > > > >> > > > > and
> > > > > > > >> > > > > > some info about your network
> > > > > > > >> > > > > > - if it is reasonable to do so share some of your
> > > tests
> > > > > such
> > > > > > > >> that
> > > > > > > >> > we
> > > > > > > >> > > > can
> > > > > > > >> > > > > > also test it at our end.
> > > > > > > >> > > > > > - Tell us specific things that you would like us
> to
> > > test
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > Thanks,
> > > > > > > >> > > > > > Hugo
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > > On Nov 16, 2017, at 11:49 PM, Stig Rohde
> Døssing <
> > > > > > > >> > > > > stigdoess...@gmail.com>
> > > > > > > >> > > > > > wrote:
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > I agree with Jungtaek, if metrics v2 can go in
> > very
> > > > soon
> > > > > > it
> > > > > > > >> > should
> > > > > > > >> > > go
> > > > > > > >> > > > > in,
> > > > > > > >> > > > > > > otherwise I'd rather release 1.2.0 now and work
> on
> > > > > getting
> > > > > > > >> 2.0.0
> > > > > > > >> > > > ready
> > > > > > > >> > > > > > for
> > > > > > > >> > > > > > > release.
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > 1.x and master have drifted pretty far apart,
> and
> > > it's
> > > > > > > >> causing a
> > > > > > > >> > > lot
> > > > > > > >> > > > of
> > > > > > > >> > > > > > > porting work at this point. 1.x still has a lot
> of
> > > > > Clojure
> > > > > > > >> code,
> > > > > > > >> > > > > > storm-core
> > > > > > > >> > > > > > > hasn't been split yet in that branch and it's
> also
> > > > > > targeting
> > > > > > > >> JDK
> > > > > > > >> > > 1.7.
> > > > > > > >> > > > > > It's
> > > > > > > >> > > > > > > seems rare at this point that a PR cherry picks
> > > > cleanly
> > > > > > onto
> > > > > > > >> 1.x
> > > > > > > >> > > from
> > > > > > > >> > > > > > > master.
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > I've linked the list of issues that are only
> fixed
> > > in
> > > > > > 2.0.0
> > > > > > > >> and
> > > > > > > >> > not
> > > > > > > >> > > > > 1.x,
> > > > > > > >> > > > > > to
> > > > > > > >> > > > > > > illustrate how far ahead 2.0.0 is.
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > https://pste.eu/p/9CJT.html
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > 2017-11-17 5:41 GMT+01:00 Arun Iyer <
> > > > > > ai...@hortonworks.com
> > > > > > > >:
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > >> Hi Taylor,
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> Is it https://github.com/apache/
> storm/pull/2203
> > ?
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> I think it would be great to get this in 1.2
> > > release.
> > > > > Can
> > > > > > > we
> > > > > > > >> try
> > > > > > > >> > > to
> > > > > > > >> > > > > > >> address the issues in a week or so and get this
> > in?
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> Thanks,
> > > > > > > >> > > > > > >> Arun
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> On 11/17/17, 7:13 AM, "P. Taylor Goetz" <
> > > > > > ptgo...@gmail.com
> > > > > > > >
> > > > > > > >> > > wrote:
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >>> The original idea for the 1.2 release was that
> > > would
> > > > > > > >> introduce
> > > > > > > >> > > the
> > > > > > > >> > > > > > >> metrics v2 work (and there is additional work
> > > > required
> > > > > > > there,
> > > > > > > >> > so I
> > > > > > > >> > > > > > >> understand a desire not to delay the release).
> Do
> > > we
> > > > > want
> > > > > > > to
> > > > > > > >> > stick
> > > > > > > >> > > > > with
> > > > > > > >> > > > > > >> that, or deviate? If the latter, would we do a
> > 1.3
> > > > > > release
> > > > > > > >> for
> > > > > > > >> > > > > metrics?
> > > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >>> As far as a 1.1.2 release I’m fine with
> > releasing
> > > > that
> > > > > > at
> > > > > > > >> any
> > > > > > > >> > > time.
> > > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >>> -Taylor
> > > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >>>> On Nov 15, 2017, at 6:24 PM, Jungtaek Lim <
> > > > > > > >> kabh...@gmail.com>
> > > > > > > >> > > > > wrote:
> > > > > > > >> > > > > > >>>>
> > > > > > > >> > > > > > >>>> I think we could start release phase for both
> > > 1.1.2
> > > > > and
> > > > > > > >> 1.2.0
> > > > > > > >> > > when
> > > > > > > >> > > > > > >>>> https://github.com/apache/storm/pull/2423
> will
> > > be
> > > > > > > merged.
> > > > > > > >> > > > > > >>>>
> > > > > > > >> > > > > > >>>> Thanks,
> > > > > > > >> > > > > > >>>> Jungtaek Lim (HeartSaVioR)
> > > > > > > >> > > > > > >>>>
> > > > > > > >> > > > > > >>>> 2017년 11월 16일 (목) 오전 7:00, Alexandre
> > > Vermeerbergen
> > > > <
> > > > > > > >> > > > > > >> avermeerber...@gmail.com>님이
> > > > > > > >> > > > > > >>>> 작성:
> > > > > > > >> > > > > > >>>>
> > > > > > > >> > > > > > >>>>> Hello,
> > > > > > > >> > > > > > >>>>>
> > > > > > > >> > > > > > >>>>> I'd love to see a Storm 1.2.0 released:
> it's a
> > > > > perfect
> > > > > > > >> > schedule
> > > > > > > >> > > > for
> > > > > > > >> > > > > > me
> > > > > > > >> > > > > > >> for
> > > > > > > >> > > > > > >>>>> test any Release Candidate that might be
> > > available
> > > > > if
> > > > > > it
> > > > > > > >> > > happens
> > > > > > > >> > > > > > soon.
> > > > > > > >> > > > > > >>>>>
> > > > > > > >> > > > > > >>>>> Best regards,
> > > > > > > >> > > > > > >>>>> Alexandre
> > > > > > > >> > > > > > >>>>>
> > > > > > > >> > > > > > >>>>> 2017-11-15 7:29 GMT+01:00 Arun Mahadevan <
> > > > > > > >> ar...@apache.org>:
> > > > > > > >> > > > > > >>>>>
> > > > > > > >> > > > > > >>>>>> Hi,
> > > > > > > >> > > > > > >>>>>>
> > > > > > > >> > > > > > >>>>>> Looks like we are only waiting on
> > > > > > > >> > https://issues.apache.org/
> > > > > > > >> > > > > > >>>>>> jira/browse/STORM-2546 .
> > > > > > > >> > > > > > >>>>>>
> > > > > > > >> > > > > > >>>>>> Are there any other issues which are
> blockers
> > > for
> > > > > > Storm
> > > > > > > >> > 1.2.0?
> > > > > > > >> > > > > Would
> > > > > > > >> > > > > > >> be
> > > > > > > >> > > > > > >>>>>> great to see the 1.2.0 release out soon as
> it
> > > > has a
> > > > > > lot
> > > > > > > >> of
> > > > > > > >> > > > > critical
> > > > > > > >> > > > > > >>>>> fixes.
> > > > > > > >> > > > > > >>>>>>
> > > > > > > >> > > > > > >>>>>> Thanks,
> > > > > > > >> > > > > > >>>>>> Arun
> > > > > > > >> > > > > > >>>>>>
> > > > > > > >> > > > > > >>>>>>
> > > > > > > >> > > > > > >>>>>>
> > > > > > > >> > > > > > >>>>>
> > > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
