Re: registerMetrics on TopologyContext

[Stig Rohde Døssing]

Hi Satheesh,

I think you're looking for the new metrics API, described here
https://storm.apache.org/releases/2.0.0/metrics_v2.html, which is based on
Dropwizard Metrics. As you note, the old API is deprecated. It will still
work, so feel free to upgrade to 2.x first, and then switching to the new
metrics API later.

Den fre. 8. nov. 2019 kl. 20.39 skrev Satheesh Akkinepally
:

> Hi storm devs, we have code that registers metrics using
> TopologyContext.registerMetrics. This was deprecated in 1.x versions of
> storm. It still seems to be available in Storm 2.x though its deprecated. I
> was looking at the documentation to fix this while we are trying to upgrade
> to 2.x.  What would be the preferred and recommended way to do this? I
> searched in the docs but could not find anything related to it. Could any
> of you point me to the right direction?
>
> Thank You!
>

Re: [ANNOUNCE] Apache Storm 2.1.0 Released

[Stig Rohde Døssing]

Great job with the release.

Den tor. 31. okt. 2019 kl. 22.48 skrev Ethan Li :

> The Apache Storm community is pleased to announce the release of Apache
> Storm version 2.1.0.
>
> Apache Storm is a distributed, fault-tolerant, and high-performance
> realtime computation system that provides strong guarantees on the
> processing of data. You can read more about Storm on the project website:
>
> http://storm.apache.org
>
> Downloads of source and binary distributions are listed in our download
> section:
>
> http://storm.apache.org/downloads.html
>
> You can read more about this release in the following blog post:
>
> https://storm.apache.org/2019/10/31/storm210-released.html
>
> Distribution artifacts are available in Maven Central at the following
> coordinates:
>
> groupId: org.apache.storm
> artifactId: storm-{component}
> version: 2.1.0
>
> The full list of changes is available here[1]. Please let us know [2] if
> you encounter any problems.
>
> Regards,
>
> The Apache Storm Team
>
> [1]:
> http://www.us.apache.org/dist/storm/apache-storm-2.1.0/RELEASE_NOTES.html
> [2]: https://issues.apache.org/jira/browse/STORM
>

Re: [VOTE] Release Apache Storm 2.1.0 (rc5)

[Stig Rohde Døssing]

Derek,

Are you using Maven 3.6.2? There's a bug causing it to fail to look in the
right repositories for dependencies in some cases
https://github.com/apache/maven/pull/294. If you're using 3.6.2, please try
building with 3.6.1 until that issue is fixed. The kafka-avro-serializer
jar is in Confluent's Maven repo.

Regarding the illegal reflective access, is the test failing or just
warning about the illegal access? The Kryo guys seem to think it's just a
little annoying, but shouldn't break anything
https://github.com/EsotericSoftware/kryo/issues/543#issuecomment-395894516.
Our CI builds with Java 11 (excluding some Hadoop and Cassandra stuff), so
am a little surprised if one of the tests fail on Java 9+.

Den tor. 31. okt. 2019 kl. 02.43 skrev Derek Dagit :

> Differencing the source from version control versus the source in the
> tar.gz:
>
> * We are distributing several dependency-reduced-pom.xml files that are
>   not part of the source.
> * There is what appears to be a test directory accidentally included at
>   apache-storm-2.1.0/storm-core/null/
>
> These are not blockers for a release.
>
> When building the source with `mvn clean install -DskipTests=true`:
>
> * Build fails for flux-examples due to a missing dependency for
>   storm-hdfs: io.confluent:kafka-avro-serializer:jar:1.0
>
>   Clearing the dependency-reduced-pom.xml files mentioned above does not
>   resolve the problem.
>
>   This may be a blocker if it prevents the build generally. Has anyone
>   else had trouble downloading this dependency? If not, from where is
>   the dependency downloaded?
>
>
> When testing with `mvn clean install`
>
> * The following warning fails test-clojure in storm-core:
>   WARNING: Illegal reflective access by
> com.esotericsoftware.kryo.util.UnsafeUtil
>
>   This may have to do with changes beginning with Java 9:
>   http://openjdk.java.net/jeps/261#Relaxed-strong-encapsulation
>
>   When using Java 8 I do not see this failure.
>
>   This probably should not be a blocker, but we may have to have a
>   work-around.
>
> --
> Derek
>
>
> On Wed, Oct 30, 2019 at 04:23:11PM -0400, Kishorkumar Patil wrote:
> >
> > +1.
> >
> > Built from code. Executed based wordcount and drpc topologies. I executed
> > and compared Based ThroughtputVsLatency and performance and resource uses
> > is same as 2.0.0 release.
> >
> > -Kishor
> >
> > On Wed, Oct 30, 2019 at 3:44 PM Aaron Gresch  wrote:
> >
> > > +1
> > >
> > > Built and ran tests from source.
> > > Ran local WordCountTopology, ran a rebalance and kill and looked at
> logs
> > > for ERRORs.
> > > Validated storm blobstore list and jar command with -c option worked.
> > >
> > >
> > > On Fri, Oct 25, 2019 at 3:09 PM Ethan Li  wrote:
> > >
> > > > This is a call to vote on releasing Apache Storm 2.1.0 (rc5)
> > > >
> > > > Full list of changes in this release:
> > > >
> > > >
> > > >
> > >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc5/RELEASE_NOTES.html
> > > >
> > > > The tag/commit to be voted upon is v2.1.0:
> > > >
> > > >
> > > >
> > >
> https://gitbox.apache.org/repos/asf?p=storm.git;a=tree;h=9a4a245efeb6d74dc01b0feac77bda0e5709b3f6;hb=79fc9b3e6aeec623b42d165536a936d28f2b12f1
> > > >
> > > > The source archive being voted upon can be found here:
> > > >
> > > >
> > > >
> > >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc5/apache-storm-2.1.0-src.tar.gz
> > > >
> > > > Other release files, signatures and digests can be found here:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc5
> > > >
> > > > The release artifacts are signed with the following key:
> > > >
> > > > https://www.apache.org/dist/storm/KEYS
> > > >
> > > > The Nexus staging repository for this release is:
> > > >
> > > >
> https://repository.apache.org/content/repositories/orgapachestorm-1089/
> > > >
> > > > Please vote on releasing this package as Apache Storm 2.1.0.
> > > >
> > > > When voting, please list the actions taken to verify the release.
> > > >
> > > > This vote will be open for at least 72 hours.
> > > >
> > > > [ ] +1 Release this package as Apache Storm 2.1.0
> > > > [ ]  0 No opinion
> > > > [ ] -1 Do not release this package because...
> > > >
> > > > Thanks to everyone who contributed to this release.
> > > >
> > >
>

Re: [VOTE] Release Apache Storm 2.1.0 (rc5)

[Stig Rohde Døssing]

+1

Verified signatures and checksums for zips and tars.
Installed locally using the binary zip.
Ran the build and tests from the source zip.
Ran ExclamationTopology.
Checked activate/deactivate/kill, dynamic log levels, log search, verified
that there are no error logs anywhere in the log output until topology is
shut down, where a few error logs are expected due to connections shutting
down.

Den fre. 25. okt. 2019 kl. 22.09 skrev Ethan Li :

> This is a call to vote on releasing Apache Storm 2.1.0 (rc5)
>
> Full list of changes in this release:
>
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc5/RELEASE_NOTES.html
>
> The tag/commit to be voted upon is v2.1.0:
>
>
> https://gitbox.apache.org/repos/asf?p=storm.git;a=tree;h=9a4a245efeb6d74dc01b0feac77bda0e5709b3f6;hb=79fc9b3e6aeec623b42d165536a936d28f2b12f1
>
> The source archive being voted upon can be found here:
>
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc5/apache-storm-2.1.0-src.tar.gz
>
> Other release files, signatures and digests can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc5
>
> The release artifacts are signed with the following key:
>
> https://www.apache.org/dist/storm/KEYS
>
> The Nexus staging repository for this release is:
>
> https://repository.apache.org/content/repositories/orgapachestorm-1089/
>
> Please vote on releasing this package as Apache Storm 2.1.0.
>
> When voting, please list the actions taken to verify the release.
>
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 2.1.0
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>

Re: Any hope for a fix of STORM-3032 ?

[Stig Rohde Døssing]

Sounds good, thanks for letting other people know about the JDK part.

Den ons. 25. sep. 2019 kl. 09.09 skrev Alexandre Vermeerbergen <
avermeerber...@gmail.com>:

> Hello Stig,
>
> I did not realize that these API exist in Storm 1.2.x, if so then I
> can start working on this adaptation to prepare 1.x/2.x transition of
> our topologies.
> On a side note, I realized that I could only reproduce STORM-3032 only
> if running Supervisor processes using AdoptOpenJDK 8 update 222 with
> OpenJ9 ; whereas I don't get these exceptions when using AdopOpenJDK 8
> update 222 with HotSpot
> => let me update the JIRA with this piece of information
>
> Kind regards,
> Alexandre
>
>
> Le mer. 25 sept. 2019 à 08:21, Stig Rohde Døssing
>  a écrit :
> >
> > I believe those API changes are in Storm 1.2.0+ as well. You can upgrade
> to
> > one of those versions, and update your topologies so they don't use
> > deprecated methods. Then the topologies should be compatible with 2.x
> > without rebuilding.
> >
> > Den tir. 24. sep. 2019 kl. 22.34 skrev Alexandre Vermeerbergen <
> > avermeerber...@gmail.com>:
> >
> > > Hello Stig,
> > >
> > > The API changes are the ones mentionned in this thread:
> > >
> > >
> http://mail-archives.apache.org/mod_mbox/storm-dev/201905.mbox/%3cCAG09ER3iDfXY7+SixSsCDo=a+h0z-sd6hjwnfnfp+zfaw-r...@mail.gmail.com%3e
> > >
> > > My problem is that if I migrate my topologies to 2.x to try it, then I
> > > have to duplicate some code to keep our topologies compatible with
> > > Storm 1.x until we have 100% tested our migration.
> > >
> > > Unless I am missing something? Is there a way to run Storm
> > > 1.x-compiled topologies on Storm 2.x without rebuilding?
> > >
> > > Kind regards,
> > > Alexandre
> > >
> > >
> > > Le dim. 22 sept. 2019 à 22:56, Stig Rohde Døssing
> > >  a écrit :
> > > >
> > > > I thought the API changes for Storm 2.x were mostly minor. Which API
> > > > changes are you being blocked by?
> > > >
> > > > Den søn. 22. sep. 2019 kl. 21.27 skrev Alexandre Vermeerbergen <
> > > > avermeerber...@gmail.com>:
> > > >
> > > > > Hello Stig,
> > > > >
> > > > > Thank you very much for your answer.
> > > > >
> > > > > Upgrading to Storm 2.x is something I would like to try, but I know
> > > > > that building with Storm 2.x API requires changing our topologies
> > > > > since some API have changes (I used to try, and was kind of
> blocked by
> > > > > the un-reversible changes that were required).
> > > > >
> > > > > Question: could our Storm 1.x topology work with Storm 2.x ?
> > > > >
> > > > > Kind regards,
> > > > > Alexandre
> > > > >
> > > > > Le jeu. 19 sept. 2019 à 19:05, Stig Rohde Døssing
> > > > >  a écrit :
> > > > > >
> > > > > > Looking at the stack trace, the issue is not in the Kafka bolt,
> but
> > > in
> > > > > some
> > > > > > internal Storm statistics code. It seems likely to me that it has
> > > been
> > > > > > fixed in 2.0.0, as we haven't heard of anyone encounter it there.
> > > Are you
> > > > > > able to upgrade to 2.x to see if that fixes this?
> > > > > >
> > > > > > If not, the exception is coming from
> > > > > >
> > > > >
> > >
> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131
> > > > > .
> > > > > > Looks like either "amt" or " (stats-rate stats) " in that line is
> > > null.
> > > > > My
> > > > > > guess would be stats. The equivalent code in 2.x uses primitive
> > > ints, so
> > > > > > won't be throwing NPEs
> > > > > >
> > > > >
> > >
> https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63
> > > > > >
> > > > > > If you have the ability to upgrade to 2.0.0 that would be best.
> > > > > >
> > > > > > I don't think simply catching the NPE is a good solution, we
> should
> > > > > prevent
> > > > > > the NPE from being thrown in the first place. If you want to
> take a
> > > look
> > > > > at
> > > > > > it, feel free to raise a PR. May want to also lobby to get a
> 1.2.4
> > > > > release
> > > > > > out after that.
> > > > > >
> > > > > > Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
> > > > > > avermeerber...@gmail.com>:
> > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > I have seen couple of occurrences of the NullPointerException
> in
> > > Storm
> > > > > > > Kafka Spout with same stack trace as in
> > > > > > > https://issues.apache.org/jira/browse/STORM-3032
> > > > > > >
> > > > > > > Using Storm 1.2.3, with Kafka Client at 2.0.1 version,
> connected
> > > to a
> > > > > > > cluster of Kafka Brokers at 2.2.1 version.
> > > > > > >
> > > > > > > Since the stack traces seems to tell that the
> NullPointerException
> > > > > > > occurs in some code related to statistics computation,
> wouldn't it
> > > be
> > > > > > > better to fix the issue by catching the NullPointerException to
> > > avoid
> > > > > > > blocking Kafka consumption ?
> > > > > > >
> > > > > > > Kind regards,
> > > > > > > Alexandre Vermeerbergen
> > > > > > >
> > > > >
> > >
>

Re: Any hope for a fix of STORM-3032 ?

[Stig Rohde Døssing]

I believe those API changes are in Storm 1.2.0+ as well. You can upgrade to
one of those versions, and update your topologies so they don't use
deprecated methods. Then the topologies should be compatible with 2.x
without rebuilding.

Den tir. 24. sep. 2019 kl. 22.34 skrev Alexandre Vermeerbergen <
avermeerber...@gmail.com>:

> Hello Stig,
>
> The API changes are the ones mentionned in this thread:
>
> http://mail-archives.apache.org/mod_mbox/storm-dev/201905.mbox/%3cCAG09ER3iDfXY7+SixSsCDo=a+h0z-sd6hjwnfnfp+zfaw-r...@mail.gmail.com%3e
>
> My problem is that if I migrate my topologies to 2.x to try it, then I
> have to duplicate some code to keep our topologies compatible with
> Storm 1.x until we have 100% tested our migration.
>
> Unless I am missing something? Is there a way to run Storm
> 1.x-compiled topologies on Storm 2.x without rebuilding?
>
> Kind regards,
> Alexandre
>
>
> Le dim. 22 sept. 2019 à 22:56, Stig Rohde Døssing
>  a écrit :
> >
> > I thought the API changes for Storm 2.x were mostly minor. Which API
> > changes are you being blocked by?
> >
> > Den søn. 22. sep. 2019 kl. 21.27 skrev Alexandre Vermeerbergen <
> > avermeerber...@gmail.com>:
> >
> > > Hello Stig,
> > >
> > > Thank you very much for your answer.
> > >
> > > Upgrading to Storm 2.x is something I would like to try, but I know
> > > that building with Storm 2.x API requires changing our topologies
> > > since some API have changes (I used to try, and was kind of blocked by
> > > the un-reversible changes that were required).
> > >
> > > Question: could our Storm 1.x topology work with Storm 2.x ?
> > >
> > > Kind regards,
> > > Alexandre
> > >
> > > Le jeu. 19 sept. 2019 à 19:05, Stig Rohde Døssing
> > >  a écrit :
> > > >
> > > > Looking at the stack trace, the issue is not in the Kafka bolt, but
> in
> > > some
> > > > internal Storm statistics code. It seems likely to me that it has
> been
> > > > fixed in 2.0.0, as we haven't heard of anyone encounter it there.
> Are you
> > > > able to upgrade to 2.x to see if that fixes this?
> > > >
> > > > If not, the exception is coming from
> > > >
> > >
> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131
> > > .
> > > > Looks like either "amt" or " (stats-rate stats) " in that line is
> null.
> > > My
> > > > guess would be stats. The equivalent code in 2.x uses primitive
> ints, so
> > > > won't be throwing NPEs
> > > >
> > >
> https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63
> > > >
> > > > If you have the ability to upgrade to 2.0.0 that would be best.
> > > >
> > > > I don't think simply catching the NPE is a good solution, we should
> > > prevent
> > > > the NPE from being thrown in the first place. If you want to take a
> look
> > > at
> > > > it, feel free to raise a PR. May want to also lobby to get a 1.2.4
> > > release
> > > > out after that.
> > > >
> > > > Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
> > > > avermeerber...@gmail.com>:
> > > >
> > > > > Hello,
> > > > >
> > > > > I have seen couple of occurrences of the NullPointerException in
> Storm
> > > > > Kafka Spout with same stack trace as in
> > > > > https://issues.apache.org/jira/browse/STORM-3032
> > > > >
> > > > > Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected
> to a
> > > > > cluster of Kafka Brokers at 2.2.1 version.
> > > > >
> > > > > Since the stack traces seems to tell that the NullPointerException
> > > > > occurs in some code related to statistics computation, wouldn't it
> be
> > > > > better to fix the issue by catching the NullPointerException to
> avoid
> > > > > blocking Kafka consumption ?
> > > > >
> > > > > Kind regards,
> > > > > Alexandre Vermeerbergen
> > > > >
> > >
>

Re: Location for 2.0 blogs

[Stig Rohde Døssing]

We could put it in the regular news feed, similar to the release posts, or
maybe we could add another button next to news for blog posts?

Den søn. 22. sep. 2019 kl. 12.32 skrev Roshan Naik
:

> I see that we have never published any blogs previouslyanywhere on the
> Apache Storm website.So wondering where would be a good location to publish
> the (overdue) 2.0 perf blog that I am close to completing. I guess that
> might dictate the format (html / wiki) the final version needs to be in.
> Suggestions ?
> -roshan

Re: Any hope for a fix of STORM-3032 ?

[Stig Rohde Døssing]

I thought the API changes for Storm 2.x were mostly minor. Which API
changes are you being blocked by?

Den søn. 22. sep. 2019 kl. 21.27 skrev Alexandre Vermeerbergen <
avermeerber...@gmail.com>:

> Hello Stig,
>
> Thank you very much for your answer.
>
> Upgrading to Storm 2.x is something I would like to try, but I know
> that building with Storm 2.x API requires changing our topologies
> since some API have changes (I used to try, and was kind of blocked by
> the un-reversible changes that were required).
>
> Question: could our Storm 1.x topology work with Storm 2.x ?
>
> Kind regards,
> Alexandre
>
> Le jeu. 19 sept. 2019 à 19:05, Stig Rohde Døssing
>  a écrit :
> >
> > Looking at the stack trace, the issue is not in the Kafka bolt, but in
> some
> > internal Storm statistics code. It seems likely to me that it has been
> > fixed in 2.0.0, as we haven't heard of anyone encounter it there. Are you
> > able to upgrade to 2.x to see if that fixes this?
> >
> > If not, the exception is coming from
> >
> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131
> .
> > Looks like either "amt" or " (stats-rate stats) " in that line is null.
> My
> > guess would be stats. The equivalent code in 2.x uses primitive ints, so
> > won't be throwing NPEs
> >
> https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63
> >
> > If you have the ability to upgrade to 2.0.0 that would be best.
> >
> > I don't think simply catching the NPE is a good solution, we should
> prevent
> > the NPE from being thrown in the first place. If you want to take a look
> at
> > it, feel free to raise a PR. May want to also lobby to get a 1.2.4
> release
> > out after that.
> >
> > Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
> > avermeerber...@gmail.com>:
> >
> > > Hello,
> > >
> > > I have seen couple of occurrences of the NullPointerException in Storm
> > > Kafka Spout with same stack trace as in
> > > https://issues.apache.org/jira/browse/STORM-3032
> > >
> > > Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected to a
> > > cluster of Kafka Brokers at 2.2.1 version.
> > >
> > > Since the stack traces seems to tell that the NullPointerException
> > > occurs in some code related to statistics computation, wouldn't it be
> > > better to fix the issue by catching the NullPointerException to avoid
> > > blocking Kafka consumption ?
> > >
> > > Kind regards,
> > > Alexandre Vermeerbergen
> > >
>

Re: Any hope for a fix of STORM-3032 ?

[Stig Rohde Døssing]

Looking at the stack trace, the issue is not in the Kafka bolt, but in some
internal Storm statistics code. It seems likely to me that it has been
fixed in 2.0.0, as we haven't heard of anyone encounter it there. Are you
able to upgrade to 2.x to see if that fixes this?

If not, the exception is coming from
https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131.
Looks like either "amt" or " (stats-rate stats) " in that line is null. My
guess would be stats. The equivalent code in 2.x uses primitive ints, so
won't be throwing NPEs
https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63

If you have the ability to upgrade to 2.0.0 that would be best.

I don't think simply catching the NPE is a good solution, we should prevent
the NPE from being thrown in the first place. If you want to take a look at
it, feel free to raise a PR. May want to also lobby to get a 1.2.4 release
out after that.

Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
avermeerber...@gmail.com>:

> Hello,
>
> I have seen couple of occurrences of the NullPointerException in Storm
> Kafka Spout with same stack trace as in
> https://issues.apache.org/jira/browse/STORM-3032
>
> Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected to a
> cluster of Kafka Brokers at 2.2.1 version.
>
> Since the stack traces seems to tell that the NullPointerException
> occurs in some code related to statistics computation, wouldn't it be
> better to fix the issue by catching the NullPointerException to avoid
> blocking Kafka consumption ?
>
> Kind regards,
> Alexandre Vermeerbergen
>

Re: [VOTE] Release Apache Storm 2.1.0 (rc3)

[Stig Rohde Døssing]

Sounds great, thanks Ethan.

Den tor. 5. sep. 2019 kl. 16.55 skrev Ethan Li :

> Once another bug fix https://github.com/apache/storm/pull/3123 <
> https://github.com/apache/storm/pull/3123> is merged, I think we can run
> another release candidate.
>
>
>
> > On Aug 28, 2019, at 9:27 AM, Ethan Li  wrote:
> >
> > Thanks for finding the issue.
> >
> > I just uploaded my key to http://pgp.mit.edu/ <http://pgp.mit.edu/>.
> It’s a good idea to link to https://www.apache.org/info/verification.html
> <https://www.apache.org/info/verification.html>  on download page.
> >
> >> On Aug 27, 2019, at 3:35 PM, Stig Rohde Døssing  <mailto:stigdoess...@gmail.com>> wrote:
> >>
> >> -1, I think this issue is a blocker. It's caused by
> >> https://github.com/apache/storm/pull/3093/files <
> https://github.com/apache/storm/pull/3093/files>, which prevents the
> >> localizer from deleting blobs that are deleted from Nimbus, even if
> those
> >> blobs are only scheduled for download.
> >>
> >> Have opened https://github.com/apache/storm/pull/3119 <
> https://github.com/apache/storm/pull/3119>
> >>
> >> Den tir. 27. aug. 2019 kl. 19.54 skrev Stig Rohde Døssing <
> >> stigdoess...@gmail.com <mailto:stigdoess...@gmail.com>>:
> >>
> >>> I think there may be an issue with topology cleanup
> >>> I ran WordCountTopology for a bit, then killed it. The supervisor
> seems to
> >>> keep trying to download the topology jar indefinitely.
> >>>
> >>> 2019-08-27 19:49:53.459 o.a.s.l.AsyncLocalizer AsyncLocalizer Executor
> - 1
> >>> [WARN] Failed to download blob LOCAL TOPO BLOB TOPO_JAR
> >>> word-count-1-1566926844 will try again in 100 ms
> >>> org.apache.storm.generated.KeyNotFoundException: null
> >>> at
> >>>
> org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25919)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25887)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.generated.Nimbus$getBlobMeta_result.read(Nimbus.java:25818)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:88)
> >>> ~[storm-shaded-deps-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.generated.Nimbus$Client.recv_getBlobMeta(Nimbus.java:794)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> org.apache.storm.generated.Nimbus$Client.getBlobMeta(Nimbus.java:781)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.blobstore.NimbusBlobStore.getBlobMeta(NimbusBlobStore.java:85)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.localizer.LocallyCachedTopologyBlob.getRemoteVersion(LocallyCachedTopologyBlob.java:127)
> >>> ~[storm-server-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:264)
> >>> ~[storm-server-2.1.0.jar:2.1.0]
> >>> at
> >>>
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
> >>> [?:1.8.0_144]
> >>> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> [?:1.8.0_144]
> >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [?:1.8.0_144]
> >>> at
> >>>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> >>> [?:1.8.0_144]
> >>> at
> >>>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> >>> [?:1.8.0_144]
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >>> [?:1.8.0_144]
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >>> [?:1.8.0_144]
> >>> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
> >>>
> >>> These repeat, intermittently broken up by
> >>>
> >>> Caused by: java.lang.RuntimeException: Could not download...
> >>>

Re: Unable to locate classes which are coming from shaded jar

[Stig Rohde Døssing]

Hi Prashant,

You don't need to send emails to me personally, I check the Storm mailing
lists regularly. Several of the addresses you used don't belong to me
either.

Did you click the update icon in the Maven project window (the two circling
arrows in the right part of your screenshot)?

If that doesn't help, maybe other people using IntelliJ can share how they
are getting IntelliJ to find shaded classes?

Den tor. 29. aug. 2019 kl. 01.24 skrev Prashant Bhardwaj <
amu.prash...@gmail.com>:

> Hi Stig Rohde,
>
> I activated IntelliJ maven profile. Still, it didn't help. Am I missing
> anything?
>
>
> [image: image.png]
>
> Thanks & regards
>
> *Prashant Bhardwaj*
>
> +44 74247 18513 (UK),
> +91 92894 36265 (India)
>
>
>
> On Mon, Aug 26, 2019 at 10:09 PM Prashant Bhardwaj 
> wrote:
>
>> Hi,
>>
>> I am in the process of my first contribution to Apache Storm. I was
>> writing an example for a combination of Storm, Cassandra and Storm + how to
>> test this combo using Storm's Testing classes.
>>
>> I found some of the classes are using shaded classes, for example -
>> ConfigUtils class is using
>> org.apache.storm.shade.com.google.common.collect.Maps.
>>
>> In my InteliiJ setup, ConfigUtils is unable to find this import.
>>
>> I have run mvn clean install which was executed successfully and saw
>> apache-storm-shaded jar was created. Even I looked inside that jar and all
>> the referred classes are part of this jar.
>>
>> Could you please suggest how to fix this issue because I am unable to run
>> my tests, these are failing on these compilations errors in other classes?
>>
>> [image: image.png]
>>
>> Thanks & regards
>>
>> *Prashant Bhardwaj*
>>
>

Re: [VOTE] Release Apache Storm 2.1.0 (rc3)

[Stig Rohde Døssing]

-1, I think this issue is a blocker. It's caused by
https://github.com/apache/storm/pull/3093/files, which prevents the
localizer from deleting blobs that are deleted from Nimbus, even if those
blobs are only scheduled for download.

Have opened https://github.com/apache/storm/pull/3119

Den tir. 27. aug. 2019 kl. 19.54 skrev Stig Rohde Døssing <
stigdoess...@gmail.com>:

> I think there may be an issue with topology cleanup
> I ran WordCountTopology for a bit, then killed it. The supervisor seems to
> keep trying to download the topology jar indefinitely.
>
> 2019-08-27 19:49:53.459 o.a.s.l.AsyncLocalizer AsyncLocalizer Executor - 1
> [WARN] Failed to download blob LOCAL TOPO BLOB TOPO_JAR
> word-count-1-1566926844 will try again in 100 ms
> org.apache.storm.generated.KeyNotFoundException: null
> at
> org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25919)
> ~[storm-client-2.1.0.jar:2.1.0]
> at
> org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25887)
> ~[storm-client-2.1.0.jar:2.1.0]
> at
> org.apache.storm.generated.Nimbus$getBlobMeta_result.read(Nimbus.java:25818)
> ~[storm-client-2.1.0.jar:2.1.0]
> at
> org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:88)
> ~[storm-shaded-deps-2.1.0.jar:2.1.0]
> at
> org.apache.storm.generated.Nimbus$Client.recv_getBlobMeta(Nimbus.java:794)
> ~[storm-client-2.1.0.jar:2.1.0]
> at org.apache.storm.generated.Nimbus$Client.getBlobMeta(Nimbus.java:781)
> ~[storm-client-2.1.0.jar:2.1.0]
> at
> org.apache.storm.blobstore.NimbusBlobStore.getBlobMeta(NimbusBlobStore.java:85)
> ~[storm-client-2.1.0.jar:2.1.0]
> at
> org.apache.storm.localizer.LocallyCachedTopologyBlob.getRemoteVersion(LocallyCachedTopologyBlob.java:127)
> ~[storm-server-2.1.0.jar:2.1.0]
> at
> org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:264)
> ~[storm-server-2.1.0.jar:2.1.0]
> at
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
> [?:1.8.0_144]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [?:1.8.0_144]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_144]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> [?:1.8.0_144]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [?:1.8.0_144]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_144]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_144]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
>
> These repeat, intermittently broken up by
>
> Caused by: java.lang.RuntimeException: Could not download...
> at
> org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:285)
> ~[storm-server-2.1.0.jar:2.1.0]
> at
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
> ~[?:1.8.0_144]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[?:1.8.0_144]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_144]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> ~[?:1.8.0_144]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> ~[?:1.8.0_144]
> ... 3 more
> Caused by: org.apache.storm.generated.KeyNotFoundException
> at
> org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25919)
> ~[storm-client-2.1.0.jar:2.1.0]
> at
> org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25887)
> ~[storm-client-2.1.0.jar:2.1.0]
> at
> org.apache.storm.generated.Nimbus$getBlobMeta_result.read(Nimbus.java:25818)
> ~[storm-client-2.1.0.jar:2.1.0]
> at
> org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:88)
> ~[storm-shaded-deps-2.1.0.jar:2.1.0]
> at
> org.apache.storm.generated.Nimbus$Client.recv_getBlobMeta(Nimbus.java:794)
> ~[storm-client-2.1.0.jar:2.1.0]
> at org.apache.storm.generated.Nimbus$Client.getBlobMeta(Nimbus.java:781)
> ~[storm-client-2.1.0.jar:2.1.0]
> at
> org.apache.storm.blobstore.NimbusBlobStore.getBlobMeta(NimbusBlobStore.java:85)
> ~[storm-client-2.1.0.jar:2.1.0]
> at
> org.apache.storm.localizer.LocallyCachedTopologyBlob.getRemoteVersion(LocallyCachedTopologyBlob.java:127)
> ~[storm-server-2.1.0.jar:2.1.0]

Re: [VOTE] Release Apache Storm 2.1.0 (rc3)

[Stig Rohde Døssing]

I think there may be an issue with topology cleanup
I ran WordCountTopology for a bit, then killed it. The supervisor seems to
keep trying to download the topology jar indefinitely.

2019-08-27 19:49:53.459 o.a.s.l.AsyncLocalizer AsyncLocalizer Executor - 1
[WARN] Failed to download blob LOCAL TOPO BLOB TOPO_JAR
word-count-1-1566926844 will try again in 100 ms
org.apache.storm.generated.KeyNotFoundException: null
at
org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25919)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25887)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.generated.Nimbus$getBlobMeta_result.read(Nimbus.java:25818)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:88)
~[storm-shaded-deps-2.1.0.jar:2.1.0]
at
org.apache.storm.generated.Nimbus$Client.recv_getBlobMeta(Nimbus.java:794)
~[storm-client-2.1.0.jar:2.1.0]
at org.apache.storm.generated.Nimbus$Client.getBlobMeta(Nimbus.java:781)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.blobstore.NimbusBlobStore.getBlobMeta(NimbusBlobStore.java:85)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.localizer.LocallyCachedTopologyBlob.getRemoteVersion(LocallyCachedTopologyBlob.java:127)
~[storm-server-2.1.0.jar:2.1.0]
at
org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:264)
~[storm-server-2.1.0.jar:2.1.0]
at
java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
[?:1.8.0_144]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_144]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_144]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[?:1.8.0_144]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[?:1.8.0_144]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_144]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_144]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]

These repeat, intermittently broken up by

Caused by: java.lang.RuntimeException: Could not download...
at
org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:285)
~[storm-server-2.1.0.jar:2.1.0]
at
java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
~[?:1.8.0_144]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_144]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_144]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
~[?:1.8.0_144]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
~[?:1.8.0_144]
... 3 more
Caused by: org.apache.storm.generated.KeyNotFoundException
at
org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25919)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25887)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.generated.Nimbus$getBlobMeta_result.read(Nimbus.java:25818)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:88)
~[storm-shaded-deps-2.1.0.jar:2.1.0]
at
org.apache.storm.generated.Nimbus$Client.recv_getBlobMeta(Nimbus.java:794)
~[storm-client-2.1.0.jar:2.1.0]
at org.apache.storm.generated.Nimbus$Client.getBlobMeta(Nimbus.java:781)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.blobstore.NimbusBlobStore.getBlobMeta(NimbusBlobStore.java:85)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.localizer.LocallyCachedTopologyBlob.getRemoteVersion(LocallyCachedTopologyBlob.java:127)
~[storm-server-2.1.0.jar:2.1.0]
at
org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:264)
~[storm-server-2.1.0.jar:2.1.0]
at
java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
~[?:1.8.0_144]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_144]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_144]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
~[?:1.8.0_144]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
~[?:1.8.0_144]
... 3 more

Restarting the supervisor seems to fix it.

Here's what I did before hitting this issue

Verified asc and sha512 files.
Built Storm from the zipped source.
Extracted zipped binary and started 

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

Sounds good

Den ons. 21. aug. 2019 kl. 17.27 skrev Ethan Li :

> As we agreed to create a 2.1.x-branch and any later critical bug fixes for
> 2.1.x will go into that branch, I think there is not point for master
> branch to freeze.
>
> There are many pull requests I’d like to merge and it potentially blocks
> some of our developments (for up to 20 days now). So I am proposing to free
> master branch and start merging pull requests. And if it’s a critical bug
> fix, we should merge it to 2.1.x-branch too.
>
> > On Aug 19, 2019, at 3:01 PM, Ethan Li  wrote:
> >
> > Yes we can revert the two commits before merging in a new commit if we
> decide to include this new commit to current version.
> >
> >
> >> On Aug 19, 2019, at 2:57 PM, Stig Rohde Døssing  <mailto:stigdoess...@gmail.com>> wrote:
> >>
> >>> Now if we need to merge something in (2.1.0 is not released yet), we
> >> merge it to 2.1.x-branch.  But the current pom version is
> 2.1.1-SNAPSHOT,
> >> which should really be 2.1.0-SNAPSHOT, because this commit goes into
> 2.1.0
> >> version.  To fix this, we need to revert last two commits before we
> merge
> >> the new commit.
> >>
> >> Sure, but if we're merging anything to 2.1.x-branch during the release
> >> process, we're either expecting it to go in 2.1.1 so 2.1.1-SNAPSHOT is
> the
> >> right version, or we're cancelling the 2.1.0 vote in order to include
> it,
> >> and in that case we're reverting those two commits anyway, right? I
> don't
> >> think there's a problem with merging something in, and then reverting
> the
> >> version bump in a later commit?
> >>
> >> Den man. 19. aug. 2019 kl. 20.38 skrev Ethan Li <
> ethanopensou...@gmail.com <mailto:ethanopensou...@gmail.com>>:
> >>
> >>> You are right. But I was talking about the pom version.
> >>>
> >>> So if we have 2.1.x-branch and the pom version is 2.1.0-SNAPSHOT. If we
> >>> start release process here, i.e. run “mvn release:prepare”,  it will
> create
> >>> and push two commits automatically.
> >>>
> >>> First commit: change pom version to 2.1.0; and create a v2.1.0 git tag.
> >>> Second commit: change pom version to 2.1.1-SNAPSHOT.
> >>>
> >>> So now in 2.1.x-branch, the pom version is 2.1.1-SNAPSHOT.
> >>>
> >>> Now if we need to merge something in (2.1.0 is not released yet), we
> merge
> >>> it to 2.1.x-branch.  But the current pom version is 2.1.1-SNAPSHOT,
> which
> >>> should really be 2.1.0-SNAPSHOT, because this commit goes into 2.1.0
> >>> version.  To fix this, we need to revert last two commits before we
> merge
> >>> the new commit.
> >>>
> >>> If we use 2.1-SNAPSHOT, we don’t need to revert.
> >>>
> >>> With that being said, I am fine with reverting if needed.  It’s
> probably
> >>> safe to not change the convention.
> >>>
> >>>
> >>>
> >>>> On Aug 19, 2019, at 1:24 PM, Stig Rohde Døssing <
> stigdoess...@gmail.com <mailto:stigdoess...@gmail.com>>
> >>> wrote:
> >>>>
> >>>> Ok, that makes sense. I still don't understand the need for
> 2.1-SNAPSHOT?
> >>>>
> >>>> If we branch off 2.1.x-branch and then roll master to 2.2.0-SNAPSHOT
> >>>> immediately, we can merge any commits into master and mark them in
> Jira
> >>>> with fix version 2.2.0. If we then get a bugfix that needs to go in
> e.g.
> >>>> 2.1.1, we can merge it to master and cherry pick the commit back to
> >>>> 2.1.x-branch, and update the fix version to both 2.2.0 and 2.1.1.
> This is
> >>>> basically similar to how it worked for the 1.x-branch branches.
> >>>>
> >>>> Why do we need 2.1-SNAPSHOT?
> >>>>
> >>>> Den man. 19. aug. 2019 kl. 20.03 skrev Ethan Li <
> >>> ethanopensou...@gmail.com <mailto:ethanopensou...@gmail.com>  ethanopensou...@gmail.com <mailto:ethanopensou...@gmail.com>>>:
> >>>>
> >>>>> We don’t create 2.1.0 branch.
> >>>>>
> >>>>> I was thinking (and have been doing) about creating a 2.1.x-branch
> >>> before
> >>>>> doing any thing release related.  Then we use 2.1.x-branch to release
> >>>>> 2.1.0, and 2.1.1, etc.
> >>>>>
> >>>>> When we create a 2.1.x-branch,  we mov

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

> Now if we need to merge something in (2.1.0 is not released yet), we
merge it to 2.1.x-branch.  But the current pom version is 2.1.1-SNAPSHOT,
which should really be 2.1.0-SNAPSHOT, because this commit goes into 2.1.0
version.  To fix this, we need to revert last two commits before we merge
the new commit.

Sure, but if we're merging anything to 2.1.x-branch during the release
process, we're either expecting it to go in 2.1.1 so 2.1.1-SNAPSHOT is the
right version, or we're cancelling the 2.1.0 vote in order to include it,
and in that case we're reverting those two commits anyway, right? I don't
think there's a problem with merging something in, and then reverting the
version bump in a later commit?

Den man. 19. aug. 2019 kl. 20.38 skrev Ethan Li :

> You are right. But I was talking about the pom version.
>
> So if we have 2.1.x-branch and the pom version is 2.1.0-SNAPSHOT. If we
> start release process here, i.e. run “mvn release:prepare”,  it will create
> and push two commits automatically.
>
> First commit: change pom version to 2.1.0; and create a v2.1.0 git tag.
> Second commit: change pom version to 2.1.1-SNAPSHOT.
>
> So now in 2.1.x-branch, the pom version is 2.1.1-SNAPSHOT.
>
> Now if we need to merge something in (2.1.0 is not released yet), we merge
> it to 2.1.x-branch.  But the current pom version is 2.1.1-SNAPSHOT, which
> should really be 2.1.0-SNAPSHOT, because this commit goes into 2.1.0
> version.  To fix this, we need to revert last two commits before we merge
> the new commit.
>
> If we use 2.1-SNAPSHOT, we don’t need to revert.
>
> With that being said, I am fine with reverting if needed.  It’s probably
> safe to not change the convention.
>
>
>
> > On Aug 19, 2019, at 1:24 PM, Stig Rohde Døssing 
> wrote:
> >
> > Ok, that makes sense. I still don't understand the need for 2.1-SNAPSHOT?
> >
> > If we branch off 2.1.x-branch and then roll master to 2.2.0-SNAPSHOT
> > immediately, we can merge any commits into master and mark them in Jira
> > with fix version 2.2.0. If we then get a bugfix that needs to go in e.g.
> > 2.1.1, we can merge it to master and cherry pick the commit back to
> > 2.1.x-branch, and update the fix version to both 2.2.0 and 2.1.1. This is
> > basically similar to how it worked for the 1.x-branch branches.
> >
> > Why do we need 2.1-SNAPSHOT?
> >
> > Den man. 19. aug. 2019 kl. 20.03 skrev Ethan Li <
> ethanopensou...@gmail.com <mailto:ethanopensou...@gmail.com>>:
> >
> >> We don’t create 2.1.0 branch.
> >>
> >> I was thinking (and have been doing) about creating a 2.1.x-branch
> before
> >> doing any thing release related.  Then we use 2.1.x-branch to release
> >> 2.1.0, and 2.1.1, etc.
> >>
> >> When we create a 2.1.x-branch,  we move master to 2.2.x-branch. It’s a
> >> little different than what we did in the past. But it makes more sense
> as
> >> it follows semantic versioning more strictly.
> >>
> >>
> >>
> >>> On Aug 19, 2019, at 12:53 PM, Stig Rohde Døssing <
> stigdoess...@gmail.com>
> >> wrote:
> >>>
> >>> I would be fine with just freezing (non-critical) merges during the
> >>> release process. These mails are public, so it's easy for anyone to see
> >>> whether a release is in progress.
> >>>
> >>> If we want to do merges while releases are going on, I think your
> >> proposal
> >>> of using a release branch is fine. I'm not sure I understand why we
> need
> >>> the 2.1-SNAPSHOT version though?
> >>>
> >>> Let's say we create release branch 2.1.0-branch from master. We roll
> the
> >>> master version to 2.1.1-SNAPSHOT (for example) after the 2.1.0 branch
> is
> >>> created. We then get a bug fix that should go in 2.1.0. In this case we
> >>> should merge it to 2.1.0-branch, then merge 2.1.0-branch to master. In
> >> Jira
> >>> we mark it as going in 2.1.0.
> >>>
> >>> We then get a PR that shouldn't be included in 2.1.0. We just merge it
> to
> >>> master, and mark it as 2.1.1 in Jira.
> >>>
> >>> Finally once 2.1.0 is released, we merge 2.1.0-branch into master and
> >>> delete 2.1.0-branch.
> >>>
> >>> Why is there a need to 2.1-SNAPSHOT?
> >>>
> >>> Den man. 19. aug. 2019 kl. 18.15 skrev Ethan Li <
> >> ethanopensou...@gmail.com <mailto:ethanopensou...@gmail.com>  ethanopensou...@gmail.com <mailto:ethanopensou...@gmail.com>>>:
> >>>
> >>>> M

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

Ok, that makes sense. I still don't understand the need for 2.1-SNAPSHOT?

If we branch off 2.1.x-branch and then roll master to 2.2.0-SNAPSHOT
immediately, we can merge any commits into master and mark them in Jira
with fix version 2.2.0. If we then get a bugfix that needs to go in e.g.
2.1.1, we can merge it to master and cherry pick the commit back to
2.1.x-branch, and update the fix version to both 2.2.0 and 2.1.1. This is
basically similar to how it worked for the 1.x-branch branches.

Why do we need 2.1-SNAPSHOT?

Den man. 19. aug. 2019 kl. 20.03 skrev Ethan Li :

> We don’t create 2.1.0 branch.
>
> I was thinking (and have been doing) about creating a 2.1.x-branch before
> doing any thing release related.  Then we use 2.1.x-branch to release
> 2.1.0, and 2.1.1, etc.
>
> When we create a 2.1.x-branch,  we move master to 2.2.x-branch. It’s a
> little different than what we did in the past. But it makes more sense as
> it follows semantic versioning more strictly.
>
>
>
> > On Aug 19, 2019, at 12:53 PM, Stig Rohde Døssing 
> wrote:
> >
> > I would be fine with just freezing (non-critical) merges during the
> > release process. These mails are public, so it's easy for anyone to see
> > whether a release is in progress.
> >
> > If we want to do merges while releases are going on, I think your
> proposal
> > of using a release branch is fine. I'm not sure I understand why we need
> > the 2.1-SNAPSHOT version though?
> >
> > Let's say we create release branch 2.1.0-branch from master. We roll the
> > master version to 2.1.1-SNAPSHOT (for example) after the 2.1.0 branch is
> > created. We then get a bug fix that should go in 2.1.0. In this case we
> > should merge it to 2.1.0-branch, then merge 2.1.0-branch to master. In
> Jira
> > we mark it as going in 2.1.0.
> >
> > We then get a PR that shouldn't be included in 2.1.0. We just merge it to
> > master, and mark it as 2.1.1 in Jira.
> >
> > Finally once 2.1.0 is released, we merge 2.1.0-branch into master and
> > delete 2.1.0-branch.
> >
> > Why is there a need to 2.1-SNAPSHOT?
> >
> > Den man. 19. aug. 2019 kl. 18.15 skrev Ethan Li <
> ethanopensou...@gmail.com <mailto:ethanopensou...@gmail.com>>:
> >
> >> Many issues came up while I was preparing for 2.1.0 release. Freezing
> >> merge until the release is done is not practical.  So I am proposing:
> >>
> >> 1. Once we decide to make a release, we create a new branch based on
> >> master and start release process.
> >> 2. During the new process, master is still open for backwards
> >> compatibility commits, including new features, bu g fixes, etc.
> >> 3. Only bug fixes will be merged to the new branch and we need to
> restart
> >> the release process to include the bug fixes.
> >> 4. To avoid too much confusion on pom versions when we merge new commits
> >> during the release process,  we should use less concrete version. For
> >> example, in 2.1.x-branch, the pom version should be 2.1-SNAPSHOT,
> instead
> >> of 2.1.0-SNAPSHOT.
> >>
> >> For example,
> >>
> >> Let’s say we have a new branch 2.1.x-branch, the pom version is
> >> 2.1.0-SNAPSHOT. We start the release process and after running the mvn
> >> release:prepare -Pxxx command, the pom version changes to
> 2.1.1-SNAPSHOT,
> >> and before that a git tag v2.1.0 is created.  Now if there is a bug fix
> we
> >> have to merge in, so we merge in and it’s actually merged in to
> >> 2.1.1-SNAPSHOT at this point if we don’t manually revert the pom
> version,
> >> which can be confusing.  We can avoid this problem by just using
> >> 2.1-SNAPSHOT as the pom version. So it doesn’t have to change.
> >>
> >> Please let me know if there is any potential risk for doing this.
> >>
> >> Thanks
> >> Ethan
> >>
> >>> On Aug 19, 2019, at 10:25 AM, Ethan Li 
> >> wrote:
> >>>
> >>> The pull request for the fix is
> >> https://github.com/apache/storm/pull/3106 <
> >> https://github.com/apache/storm/pull/3106 <
> https://github.com/apache/storm/pull/3106>>
> >>>
> >>>> On Aug 19, 2019, at 10:15 AM, Ethan Li  <mailto:ethanopensou...@gmail.com>
> >> <mailto:ethanopensou...@gmail.com <mailto:ethanopensou...@gmail.com>>>
> wrote:
> >>>>
> >>>> So I was preparing for a new release candidate i.e. rc3. I can build
> it
> >> from source without any problem. Then I set up a standalone cluster and
> >&g

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

r/storm-client/src/jvm/org/apache/storm/task/ShellBolt.java#L365-L367
> <
> https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/task/ShellBolt.java#L365-L367>
> I believe we shouldn’t throw exceptions here.
> >>
> >> Will make a pull request to fix it.
> >>
> >>
> >>
> >>> On Aug 14, 2019, at 11:11 PM, Ethan Li  <mailto:ethanopensou...@gmail.com>> wrote:
> >>>
> >>> Hi Taylor,
> >>>
> >>> Mostly I was able to follow the doc
> https://github.com/apache/storm/blob/master/RELEASING.md#setting-up-a-vote
> <
> https://github.com/apache/storm/blob/master/RELEASING.md#setting-up-a-vote>,
> except:
> >>>
> >>> For Step 1,  I used the following command as suggested.
> >>> mvn release:prepare -P dist,rat,externals,examples
> >>> mvn release:perform -P dist,rat,externals,examples
> >>>
> >>>
> >>> For Step3,  I had to run "git checkout tags/v2.1.0 -b v2.1.0” so that
> I can run “mvn package” for storm-dist/binary and storm-dist/source based
> on v2.1.0. Otherwise if I run “mvn package” on 2.1.x-branch, it will fail
> because the pom version is “2.1.x-SNAPSHOT”.
> >>>
> >>> Then I find packages in storm-dist/binary/final-package/target and
> storm-dist/source/target, sign them, generate checksum and copy them to svn.
> >>>
> >>> I think there are something I should do. But please let me know if I
> was doing something wrong.  I will  also update the doc after the release
> is complete. Thank you very much!
> >>>
> >>> Ethan
> >>>
> >>>
> >>>> On Aug 14, 2019, at 12:24 PM, Ethan Li  <mailto:ethanopensou...@gmail.com>> wrote:
> >>>>
> >>>> I am continuing for another release candidate since the pr is merged.
> >>>>
> >>>>> On Aug 13, 2019, at 11:58 AM, Stig Rohde Døssing <
> stigdoess...@gmail.com <mailto:stigdoess...@gmail.com>> wrote:
> >>>>>
> >>>>> Updated https://github.com/apache/storm/pull/3053 <
> https://github.com/apache/storm/pull/3053> so we don't have
> >>>>> org.apache.storm artifacts in the DEPENDENCY-LICENSES file. It's
> ready for
> >>>>> review if someone wants to look at it.
> >>>>>
> >>>>> Den tir. 13. aug. 2019 kl. 18.21 skrev Ethan Li <
> ethanopensou...@gmail.com <mailto:ethanopensou...@gmail.com>>:
> >>>>>
> >>>>>> Thank you, Taylor. Will delete them.
> >>>>>>
> >>>>>>> On Aug 13, 2019, at 11:19 AM, P. Taylor Goetz  <mailto:ptgo...@gmail.com>> wrote:
> >>>>>>>
> >>>>>>> Those can be safely deleted.
> >>>>>>>
> >>>>>>> -Taylor
> >>>>>>>
> >>>>>>>> On Aug 13, 2019, at 12:15 PM, Ethan Li  <mailto:ethanopensou...@gmail.com>>
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Do we need/want to clean up the release candidate from
> >>>>>> https://dist.apache.org/repos/dist/dev/storm <
> https://dist.apache.org/repos/dist/dev/storm> <
> >>>>>> https://dist.apache.org/repos/dist/dev/storm <
> https://dist.apache.org/repos/dist/dev/storm>> ?
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> https://github.com/apache/storm/blob/master/RELEASING.md#cleaning-up-if-the-vote-fails
> <
> https://github.com/apache/storm/blob/master/RELEASING.md#cleaning-up-if-the-vote-fails
> >
> >>>>>> <
> >>>>>>
> https://github.com/apache/storm/blob/master/RELEASING.md#cleaning-up-if-the-vote-fails
> <
> https://github.com/apache/storm/blob/master/RELEASING.md#cleaning-up-if-the-vote-fails
> >>
> >>>>>> says we do. But we have a lot of rc in
> >>>>>> https://dist.apache.org/repos/dist/dev/storm/ <
> https://dist.apache.org/repos/dist/dev/storm/> <
> >>>>>> https://dist.apache.org/repos/dist/dev/storm/ <
> https://dist.apache.org/repos/dist/dev/storm/>>
> >>>>>>>>
> >>>>>>>> Just want to be absolutely sure about this. Thanks.
> >>>>>>>>
> >>>>>>>>> On Aug 13, 2019, at 10:56 AM, Ethan Li <
> ethanopensou...@gmail.c

Re: [VOTE] Release Apache Storm 2.1.0 (rc2)

[Stig Rohde Døssing]

Ok, makes sense.

Den tor. 15. aug. 2019 kl. 17.16 skrev Ethan Li :

> It was added for backwards compatibility.
>
> Apache Storm 2.x supports running older version of workers. So users can
> submit 1.x topology on a 2.x Storm cluster. We have been using it in this
> way until we move all our users to use 2.x topology.  This is nice feature
> so that users can migrate gradually.
>
> To support this, Apache Storm needs to be set up with
> *supervisor.worker.version.classpath.map *config which tells supervisors
> where to find the classes for this specific version of topology . If the
> topology is 1.x, there is no storm-client jar. The version info is found in
> storm-core jar.  That’s why it’s introduced. We should probably keep it.
>
>
>
>
> On Aug 15, 2019, at 9:57 AM, Stig Rohde Døssing 
> wrote:
>
> Oops, was wrong. The revision stuff is also being read by Storm UI.
>
> Yes, we should just fix VersionInfoMojo to handle running outside SCM. I'll
> put up a PR to fix it shortly. I think we can also delete the version info
> file in storm-core. I can't think of any reason why storm-core would be on
> the classpath, but storm-client wouldn't be.
>
> Den tor. 15. aug. 2019 kl. 16.24 skrev Ethan Li  >:
>
> Sorry for typos. I meant "If it’s not built within git or SCM"
>
> On Aug 15, 2019, at 9:23 AM, Ethan Li  wrote:
>
> I prefer to revert the change in VersionInfoMojo. These fields
>
> version-info.properties are introduced on purpose in
> https://github.com/apache/storm/pull/2844 <
> https://github.com/apache/storm/pull/2844>
> https://github.com/apache/storm/pull/2891 <
> https://github.com/apache/storm/pull/2891> so we should probably keep it.
>
>
> It makes sense to return unknown if it’s built within git or SCM as
>
> there is no other way to tell.
>
>
>
> On Aug 15, 2019, at 9:12 AM, Derek Dagit 
> da...@apache.org>> wrote:
>
>
> Well, the change seems to be to throw an IllegalArgumentException
> instead of what was the default behavior: to return a String "Unknown".
>
> It is unclear how it would work outside of a Git working tree (as when
> the source is built from a download), so it seems the easiest fix would
> be to change the default labels to do nothing and revert to the previous
> behavior—if we can convince checkstyle that it is OK.
>
> A better fix might be as you suggested. I like the idea of the build not
> behaving differently depending on if it is executed within SCM, at least
> if we can help it.
>
> In any case, I am inclined to think that this should be fixed before the
> release, as it is a bug that prevents the code from building as per our
> instructions.
>
> On Thu, Aug 15, 2019 at 02:08:17PM +0200, Stig Rohde Døssing wrote:
>
>
> As far as I can tell, the version plugin is supposed to populate these
> files
>
>
> https://github.com/apache/storm/blob/master/storm-client/src/resources/storm-client-version-info.properties
> <
>
> https://github.com/apache/storm/blob/master/storm-client/src/resources/storm-client-version-info.properties
>
>
> and
>
>
> https://github.com/apache/storm/blob/master/storm-core/src/resources/storm-core-version-info.properties
> .
>
> I'm not sure why we need this file in both storm-client and
>
> storm-core, I'm
>
> not sure anything reads the storm-core file, so maybe we can delete it?
>
> When the build runs, those files are supposed to be populated with
>
> version
>
> info (commit hash, author info and so on) from the plugin. The version
>
> info
>
> is then read into
>
>
> https://github.com/apache/storm/blob/925422a5b5ad1c3329a2c2b44db460ae94f70806/storm-client/src/jvm/org/apache/storm/utils/VersionInfo.java
> .
>
> The VersionInfo file also loads the Storm version from the pom, where
>
> it's
>
> used to populate e.g. the version in Storm UI for Nimbus and
>
> Supervisors.
>
>
> It looks like this:
>
> version=2.0.0
> revision=2ba95bbd1c911d4fc6363b1c4b9c4c6d86ac9aae
> branch=(no branch)
> user=tgoetz
> date=2019-04-29T21:17Z
> url=https://git-wip-us.apache.org/repos/asf/storm.git
> srcChecksum=58779abf76d8a8bbd20a2bd5f466
>
> The only field I see read is the Storm version from the pom, I'm not
>
> sure
>
> we use the other stuff (e.g. commit hash, commit author). For 2.0.0
> something broke when splitting out storm-client, so the file for that
> module doesn't have all fields populated:
>
> version=2.0.0
> revision=${version-info.scm.commit}
> branch=${version-info.scm.branch}
> user=tgoetz
> date=${version-info.build.time}
> url=${version-info.scm.uri}
> srcChecksum=${version-info.source.m

Re: [VOTE] Release Apache Storm 2.1.0 (rc2)

[Stig Rohde Døssing]

Oops, was wrong. The revision stuff is also being read by Storm UI.

Yes, we should just fix VersionInfoMojo to handle running outside SCM. I'll
put up a PR to fix it shortly. I think we can also delete the version info
file in storm-core. I can't think of any reason why storm-core would be on
the classpath, but storm-client wouldn't be.

Den tor. 15. aug. 2019 kl. 16.24 skrev Ethan Li :

> Sorry for typos. I meant "If it’s not built within git or SCM"
>
> > On Aug 15, 2019, at 9:23 AM, Ethan Li  wrote:
> >
> > I prefer to revert the change in VersionInfoMojo. These fields
> version-info.properties are introduced on purpose in
> https://github.com/apache/storm/pull/2844 <
> https://github.com/apache/storm/pull/2844>
> https://github.com/apache/storm/pull/2891 <
> https://github.com/apache/storm/pull/2891> so we should probably keep it.
> >
> > It makes sense to return unknown if it’s built within git or SCM as
> there is no other way to tell.
> >
> >
> >> On Aug 15, 2019, at 9:12 AM, Derek Dagit  da...@apache.org>> wrote:
> >>
> >> Well, the change seems to be to throw an IllegalArgumentException
> >> instead of what was the default behavior: to return a String "Unknown".
> >>
> >> It is unclear how it would work outside of a Git working tree (as when
> >> the source is built from a download), so it seems the easiest fix would
> >> be to change the default labels to do nothing and revert to the previous
> >> behavior—if we can convince checkstyle that it is OK.
> >>
> >> A better fix might be as you suggested. I like the idea of the build not
> >> behaving differently depending on if it is executed within SCM, at least
> >> if we can help it.
> >>
> >> In any case, I am inclined to think that this should be fixed before the
> >> release, as it is a bug that prevents the code from building as per our
> >> instructions.
> >>
> >> On Thu, Aug 15, 2019 at 02:08:17PM +0200, Stig Rohde Døssing wrote:
> >>>
> >>> As far as I can tell, the version plugin is supposed to populate these
> >>> files
> >>>
> https://github.com/apache/storm/blob/master/storm-client/src/resources/storm-client-version-info.properties
> <
> https://github.com/apache/storm/blob/master/storm-client/src/resources/storm-client-version-info.properties
> >
> >>> and
> >>>
> https://github.com/apache/storm/blob/master/storm-core/src/resources/storm-core-version-info.properties
> .
> >>> I'm not sure why we need this file in both storm-client and
> storm-core, I'm
> >>> not sure anything reads the storm-core file, so maybe we can delete it?
> >>>
> >>> When the build runs, those files are supposed to be populated with
> version
> >>> info (commit hash, author info and so on) from the plugin. The version
> info
> >>> is then read into
> >>>
> https://github.com/apache/storm/blob/925422a5b5ad1c3329a2c2b44db460ae94f70806/storm-client/src/jvm/org/apache/storm/utils/VersionInfo.java
> .
> >>> The VersionInfo file also loads the Storm version from the pom, where
> it's
> >>> used to populate e.g. the version in Storm UI for Nimbus and
> Supervisors.
> >>>
> >>> It looks like this:
> >>>
> >>> version=2.0.0
> >>> revision=2ba95bbd1c911d4fc6363b1c4b9c4c6d86ac9aae
> >>> branch=(no branch)
> >>> user=tgoetz
> >>> date=2019-04-29T21:17Z
> >>> url=https://git-wip-us.apache.org/repos/asf/storm.git
> >>> srcChecksum=58779abf76d8a8bbd20a2bd5f466
> >>>
> >>> The only field I see read is the Storm version from the pom, I'm not
> sure
> >>> we use the other stuff (e.g. commit hash, commit author). For 2.0.0
> >>> something broke when splitting out storm-client, so the file for that
> >>> module doesn't have all fields populated:
> >>>
> >>> version=2.0.0
> >>> revision=${version-info.scm.commit}
> >>> branch=${version-info.scm.branch}
> >>> user=tgoetz
> >>> date=${version-info.build.time}
> >>> url=${version-info.scm.uri}
> >>> srcChecksum=${version-info.source.md5}
> >>>
> >>> It's most likely just because storm-client is missing configuration to
> run
> >>> the plugin when building.
> >>>
> >>> I think if we're not using these extra fields from git, we might as
> well
> >>> drop the version info mojo. If we only need to 

Re: [VOTE] Release Apache Storm 2.1.0 (rc2)

[Stig Rohde Døssing]

As far as I can tell, the version plugin is supposed to populate these
files
https://github.com/apache/storm/blob/master/storm-client/src/resources/storm-client-version-info.properties
and
https://github.com/apache/storm/blob/master/storm-core/src/resources/storm-core-version-info.properties.
I'm not sure why we need this file in both storm-client and storm-core, I'm
not sure anything reads the storm-core file, so maybe we can delete it?

When the build runs, those files are supposed to be populated with version
info (commit hash, author info and so on) from the plugin. The version info
is then read into
https://github.com/apache/storm/blob/925422a5b5ad1c3329a2c2b44db460ae94f70806/storm-client/src/jvm/org/apache/storm/utils/VersionInfo.java.
The VersionInfo file also loads the Storm version from the pom, where it's
used to populate e.g. the version in Storm UI for Nimbus and Supervisors.

It looks like this:

version=2.0.0
revision=2ba95bbd1c911d4fc6363b1c4b9c4c6d86ac9aae
branch=(no branch)
user=tgoetz
date=2019-04-29T21:17Z
url=https://git-wip-us.apache.org/repos/asf/storm.git
srcChecksum=58779abf76d8a8bbd20a2bd5f466

The only field I see read is the Storm version from the pom, I'm not sure
we use the other stuff (e.g. commit hash, commit author). For 2.0.0
something broke when splitting out storm-client, so the file for that
module doesn't have all fields populated:

version=2.0.0
revision=${version-info.scm.commit}
branch=${version-info.scm.branch}
user=tgoetz
date=${version-info.build.time}
url=${version-info.scm.uri}
srcChecksum=${version-info.source.md5}

It's most likely just because storm-client is missing configuration to run
the plugin when building.

I think if we're not using these extra fields from git, we might as well
drop the version info mojo. If we only need to set the "version" field, we
don't need to invoke git during the build.

If we do need these fields for something, we should just make the mojo set
the fields to "Unknown" if it fails to invoke git (i.e. if SCM.NONE is
set). I'm guessing what's happening for Derek is that he downloaded the
sources and is trying to run the build with files that aren't part of a git
repo. So when the mojo tries to do "git branch" it gets an error.

What do you think?

Den tor. 15. aug. 2019 kl. 06.02 skrev Ethan Li :

> Derek ran into this error when building from source:
>
> [ERROR] Failed to execute goal
> org.apache.storm:storm-maven-plugins:2.1.0:version-info (version-info) on
> project storm-core: java.lang.IllegalArgumentException: SCM NONE is not
> supported -> [Help 1]
>
>
> The log before this is
>
> [INFO] --- storm-maven-plugins:2.1.0:version-info (version-info) @
> storm-core ---
> [WARNING] [svn, info] failed with error code 1
> [WARNING] [git, branch] failed with error code 128
> [INFO] SCM: NONE
>
>
> I compared with apache-storm-2.0.0 and found that it can pass although
> there are similar WARNINGs:
>
> [INFO] --- storm-maven-plugins:2.0.0:version-info (version-info) @
> storm-core ---
> [WARNING] [svn, info] failed with error code 1
> [WARNING] [git, branch] failed with error code 128
> [INFO] SCM: NONE
> [INFO] Computed MD5: 58779abf76d8a8bbd20a2bd5f466
>
> This is the PR that changed the behavior
> https://github.com/apache/storm/pull/3061/files#diff-f1197661326b04376e94e3b25cc3eedcR197
> <
> https://github.com/apache/storm/pull/3061/files#diff-f1197661326b04376e94e3b25cc3eedcR197>.
> I am not sure what this storm-maven-plugin should do though. Will need to
> look into it. Or if anyone has better idea, please let me know.
>
>
>
> > On Aug 14, 2019, at 6:53 PM, P. Taylor Goetz  wrote:
> >
> > Nice work Ethan!
> >
> > I’ll try to take a look in the next few days.
> >
> > -Taylor
> >
> >> On Aug 14, 2019, at 2:13 PM, Ethan Li  wrote:
> >>
> >> This is a call to vote on releasing Apache Storm 2.1.0 (rc2)
> >>
> >> Full list of changes in this release:
> >>
> >>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc2/RELEASE_NOTES.html
> >>
> >> The tag/commit to be voted upon is v2.1.0:
> >>
> >>
> https://gitbox.apache.org/repos/asf?p=storm.git;a=tree;h=6c7adb8c7095cb718d5c49d07331034ae0e8434a;hb=f875619fc1017e401d65bd2900b195738f90e754
> >>
> >> The source archive being voted upon can be found here:
> >>
> >>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc2/apache-storm-2.1.0-src.tar.gz
> >>
> >> Other release files, signatures and digests can be found here:
> >>
> >> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc2
> >>
> >> The release artifacts are signed with the following key:
> >>
> >> https://www.apache.org/dist/storm/KEYS
> >>
> >> The Nexus staging repository for this release is:
> >>
> >> https://repository.apache.org/content/repositories/orgapachestorm-1085/
> >>
> >> Please vote on releasing this package as Apache Storm 2.1.0.
> >>
> >> When voting, please list the actions taken to verify the release.
> >>
> >> This vote will be open for at least 72 hours.
> >>
> >> [ 

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

Updated https://github.com/apache/storm/pull/3053 so we don't have
org.apache.storm artifacts in the DEPENDENCY-LICENSES file. It's ready for
review if someone wants to look at it.

Den tir. 13. aug. 2019 kl. 18.21 skrev Ethan Li :

> Thank you, Taylor. Will delete them.
>
> > On Aug 13, 2019, at 11:19 AM, P. Taylor Goetz  wrote:
> >
> > Those can be safely deleted.
> >
> > -Taylor
> >
> >> On Aug 13, 2019, at 12:15 PM, Ethan Li 
> wrote:
> >>
> >> Do we need/want to clean up the release candidate from
> https://dist.apache.org/repos/dist/dev/storm <
> https://dist.apache.org/repos/dist/dev/storm> ?
> >>
> >>
> https://github.com/apache/storm/blob/master/RELEASING.md#cleaning-up-if-the-vote-fails
> <
> https://github.com/apache/storm/blob/master/RELEASING.md#cleaning-up-if-the-vote-fails>
> says we do. But we have a lot of rc in
> https://dist.apache.org/repos/dist/dev/storm/ <
> https://dist.apache.org/repos/dist/dev/storm/>
> >>
> >> Just want to be absolutely sure about this. Thanks.
> >>
> >>> On Aug 13, 2019, at 10:56 AM, Ethan Li 
> wrote:
> >>>
> >>> That sounds better. It will be easier for release. Thank you for
> looking into this.
> >>>
> >>>> On Aug 13, 2019, at 10:54 AM, Stig Rohde Døssing <
> stigdoess...@gmail.com> wrote:
> >>>>
> >>>> Makes sense. I'll take a look at it. Ideally I'd want the license
> files to
> >>>> just not include org.apache.storm artifacts. I don't think we need to
> tell
> >>>> people that the project depends on itself.
> >>>>
> >>>> If we can exclude these artifacts from the lists, I don't think the
> release
> >>>> plugin needs to update these files.
> >>>>
> >>>> Den tir. 13. aug. 2019 kl. 17.36 skrev Ethan Li <
> ethanopensou...@gmail.com>:
> >>>>
> >>>>> Thanks Stig.
> >>>>>
> >>>>> In this case, we probably should abort release process and get this
> merged
> >>>>> into master first. Also we need to make sure it works with “mvn
> >>>>> release:prepare” since “ mvn release:prepare” will automatically
> push two
> >>>>> commits. For example,
> >>>>>
> >>>>> (1) [maven-release-plugin] prepare release v2.1.0:  this commit
> changes
> >>>>> the pom version to 2.1.0, push the commit, and then create a git tag
> v2.1.0.
> >>>>>
> >>>>> (2)  [maven-release-plugin] prepare for next development iteration:
> this
> >>>>> commit changes the pom version to 2.1.1-SNAPSHOT.
> >>>>>
> >>>>>
> >>>>> We need DEPENDENCY-LICENSES to be updated on every step.
> >>>>>
> >>>>>
> >>>>>
> >>>>>> On Aug 13, 2019, at 10:24 AM, Stig Rohde Døssing <
> stigdoess...@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>> Oops, sorry that's not right. The license plugin setup was disabled
> in
> >>>>>> https://github.com/apache/storm/pull/3031 due to a bug in the
> license
> >>>>>> plugin. It is added back in
> https://github.com/apache/storm/pull/3053,
> >>>>>> where we've upgraded the plugin. Until that PR goes in, we can't
> easily
> >>>>>> regenerate DEPENDENCY-LICENSES.
> >>>>>>
> >>>>>> Den tir. 13. aug. 2019 kl. 17.15 skrev Stig Rohde Døssing <
> >>>>>> stigdoess...@gmail.com>:
> >>>>>>
> >>>>>>> There's a script that can help you do it in
> >>>>>>> https://github.com/apache/storm/pull/3053. It checks the
> >>>>>>> DEPENDENCY-LICENSES and LICENSE-binary contain the right
> dependencies,
> >>>>> and
> >>>>>>> prints which licenses are redundant or should be added.
> >>>>>>>
> >>>>>>> For DEPENDENCY-LICENSES specifically you can run "mvn
> >>>>>>> license:aggregate-add-third-party@generate-and-check-licenses
> >>>>>>> -Dlicense.skipAggregateAddThirdParty=false -B" in the project
> root. It
> >>>>>>> should update the file for you.
> >>>>>>>
> >>>>>>> Den tir. 13. aug. 2019 

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

Makes sense. I'll take a look at it. Ideally I'd want the license files to
just not include org.apache.storm artifacts. I don't think we need to tell
people that the project depends on itself.

If we can exclude these artifacts from the lists, I don't think the release
plugin needs to update these files.

Den tir. 13. aug. 2019 kl. 17.36 skrev Ethan Li :

> Thanks Stig.
>
> In this case, we probably should abort release process and get this merged
> into master first. Also we need to make sure it works with “mvn
> release:prepare” since “ mvn release:prepare” will automatically push two
> commits. For example,
>
> (1) [maven-release-plugin] prepare release v2.1.0:  this commit changes
> the pom version to 2.1.0, push the commit, and then create a git tag v2.1.0.
>
> (2)  [maven-release-plugin] prepare for next development iteration:  this
> commit changes the pom version to 2.1.1-SNAPSHOT.
>
>
> We need DEPENDENCY-LICENSES to be updated on every step.
>
>
>
> > On Aug 13, 2019, at 10:24 AM, Stig Rohde Døssing 
> wrote:
> >
> > Oops, sorry that's not right. The license plugin setup was disabled in
> > https://github.com/apache/storm/pull/3031 due to a bug in the license
> > plugin. It is added back in https://github.com/apache/storm/pull/3053,
> > where we've upgraded the plugin. Until that PR goes in, we can't easily
> > regenerate DEPENDENCY-LICENSES.
> >
> > Den tir. 13. aug. 2019 kl. 17.15 skrev Stig Rohde Døssing <
> > stigdoess...@gmail.com>:
> >
> >> There's a script that can help you do it in
> >> https://github.com/apache/storm/pull/3053. It checks the
> >> DEPENDENCY-LICENSES and LICENSE-binary contain the right dependencies,
> and
> >> prints which licenses are redundant or should be added.
> >>
> >> For DEPENDENCY-LICENSES specifically you can run "mvn
> >> license:aggregate-add-third-party@generate-and-check-licenses
> >> -Dlicense.skipAggregateAddThirdParty=false -B" in the project root. It
> >> should update the file for you.
> >>
> >> Den tir. 13. aug. 2019 kl. 17.06 skrev Ethan Li <
> ethanopensou...@gmail.com
> >>> :
> >>
> >>> Hi Stig,
> >>>
> >>> How do we update
> >>> https://github.com/apache/storm/blob/v2.1.0/DEPENDENCY-LICENSES? <
> >>> https://github.com/apache/storm/blob/v2.1.0/DEPENDENCY-LICENSES?> Is
> >>> there a command I can use? Or I just replace strings manually?
> >>>
> >>> Thanks
> >>> Ethan
> >>>
> >>>> On Aug 12, 2019, at 3:54 PM, Ethan Li 
> >>> wrote:
> >>>>
> >>>> Yes my public keys in that file. Thanks! Ready to set up a vote.
> >>>>
> >>>>> On Aug 12, 2019, at 3:52 PM, P. Taylor Goetz 
> >>> wrote:
> >>>>>
> >>>>> Yes. Is you public key in that file? If not you should add it.
> >>>>>
> >>>>> -Taylor
> >>>>>
> >>>>>> On Aug 12, 2019, at 4:15 PM, Ethan Li 
> >>> wrote:
> >>>>>>
> >>>>>> I have one more question before I can start a vote.
> >>>>>>
> >>>>>> In previous [VOTE] thread, Taylor always has this:
> >>>>>>
> >>>>>> The release artifacts are signed with the following key:
> >>>>>>
> >>>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >>> <
> >>>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >>>>
> >>>>>>
> >>>>>>
> >>>>>> Does anyone know where to find the updated KEYs file? Should I just
> >>> use https://www.apache.org/dist/storm/KEYS <
> >>> https://www.apache.org/dist/storm/KEYS> ?
> >>>>>>
> >>>>>> Thanks very much
> >>>>>>
> >>>>>>> On Aug 12, 2019, at 9:44 AM, Ethan Li 
> >>> wrote:
> >>>>>>>
> >>>>>>> Thanks Stig and Taylor! It worked. I will continue the process now
> >>> and update the documentation later.
> >>>>>>>
> >>>>>>>
> >>>>>>>> On Aug 12, 2019, at 9:40 AM, P. Taylor Goetz 
> >>> wrote:
> >>>>>>>>
> >>>>>

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

 Oops, sorry that's not right. The license plugin setup was disabled in
https://github.com/apache/storm/pull/3031 due to a bug in the license
plugin. It is added back in https://github.com/apache/storm/pull/3053,
where we've upgraded the plugin. Until that PR goes in, we can't easily
regenerate DEPENDENCY-LICENSES.

Den tir. 13. aug. 2019 kl. 17.15 skrev Stig Rohde Døssing <
stigdoess...@gmail.com>:

> There's a script that can help you do it in
> https://github.com/apache/storm/pull/3053. It checks the
> DEPENDENCY-LICENSES and LICENSE-binary contain the right dependencies, and
> prints which licenses are redundant or should be added.
>
> For DEPENDENCY-LICENSES specifically you can run "mvn
> license:aggregate-add-third-party@generate-and-check-licenses
> -Dlicense.skipAggregateAddThirdParty=false -B" in the project root. It
> should update the file for you.
>
> Den tir. 13. aug. 2019 kl. 17.06 skrev Ethan Li  >:
>
>> Hi Stig,
>>
>> How do we update
>> https://github.com/apache/storm/blob/v2.1.0/DEPENDENCY-LICENSES? <
>> https://github.com/apache/storm/blob/v2.1.0/DEPENDENCY-LICENSES?> Is
>> there a command I can use? Or I just replace strings manually?
>>
>> Thanks
>> Ethan
>>
>> > On Aug 12, 2019, at 3:54 PM, Ethan Li 
>> wrote:
>> >
>> > Yes my public keys in that file. Thanks! Ready to set up a vote.
>> >
>> >> On Aug 12, 2019, at 3:52 PM, P. Taylor Goetz 
>> wrote:
>> >>
>> >> Yes. Is you public key in that file? If not you should add it.
>> >>
>> >> -Taylor
>> >>
>> >>> On Aug 12, 2019, at 4:15 PM, Ethan Li 
>> wrote:
>> >>>
>> >>> I have one more question before I can start a vote.
>> >>>
>> >>> In previous [VOTE] thread, Taylor always has this:
>> >>>
>> >>> The release artifacts are signed with the following key:
>> >>>
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>> <
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>> >
>> >>>
>> >>>
>> >>> Does anyone know where to find the updated KEYs file? Should I just
>> use https://www.apache.org/dist/storm/KEYS <
>> https://www.apache.org/dist/storm/KEYS> ?
>> >>>
>> >>> Thanks very much
>> >>>
>> >>>> On Aug 12, 2019, at 9:44 AM, Ethan Li 
>> wrote:
>> >>>>
>> >>>> Thanks Stig and Taylor! It worked. I will continue the process now
>> and update the documentation later.
>> >>>>
>> >>>>
>> >>>>> On Aug 12, 2019, at 9:40 AM, P. Taylor Goetz 
>> wrote:
>> >>>>>
>> >>>>> For the 2.x version lines, there are a bunch of profiles you need
>> to enable. This is what I use when preparing a release:
>> >>>>>
>> >>>>> -P dist,include-shaded-deps,rat,externals,examples
>> >>>>>
>> >>>>> -Taylor
>> >>>>>
>> >>>>>> On Aug 12, 2019, at 9:27 AM, Stig Rohde Døssing <
>> stigdoess...@gmail.com> wrote:
>> >>>>>>
>> >>>>>> Try enabling the "dist" profile by adding "-P
>> dist,externals,examples" to
>> >>>>>> the command you're running. See
>> >>>>>> https://github.com/apache/storm/blob/master/pom.xml#L518. Unless
>> you enable
>> >>>>>> that profile, the storm-dist modules aren't in the reactor.
>> >>>>>>
>> >>>>>> Den man. 12. aug. 2019 kl. 15.15 skrev Ethan Li <
>> ethanopensou...@gmail.com>:
>> >>>>>>
>> >>>>>>> Just in case if anyone happens to know the answer:
>> >>>>>>>
>> >>>>>>> I created a branch
>> https://github.com/apache/storm/tree/2.1.x-branch <
>> >>>>>>> https://github.com/apache/storm/tree/2.1.x-branch> and ran
>> “mvn:prepare”
>> >>>>>>> and “mvn:perform”. It created two commits and created a v2.1.0
>> git tag. But
>> >>>>>>> I realized that the pom version under storm-dist was not updated
>> and it’s
>> >>>>>>> still 2.0.1-SNAPSHOT. I checked the comm

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

There's a script that can help you do it in
https://github.com/apache/storm/pull/3053. It checks the
DEPENDENCY-LICENSES and LICENSE-binary contain the right dependencies, and
prints which licenses are redundant or should be added.

For DEPENDENCY-LICENSES specifically you can run "mvn
license:aggregate-add-third-party@generate-and-check-licenses
-Dlicense.skipAggregateAddThirdParty=false -B" in the project root. It
should update the file for you.

Den tir. 13. aug. 2019 kl. 17.06 skrev Ethan Li :

> Hi Stig,
>
> How do we update
> https://github.com/apache/storm/blob/v2.1.0/DEPENDENCY-LICENSES? <
> https://github.com/apache/storm/blob/v2.1.0/DEPENDENCY-LICENSES?> Is
> there a command I can use? Or I just replace strings manually?
>
> Thanks
> Ethan
>
> > On Aug 12, 2019, at 3:54 PM, Ethan Li  wrote:
> >
> > Yes my public keys in that file. Thanks! Ready to set up a vote.
> >
> >> On Aug 12, 2019, at 3:52 PM, P. Taylor Goetz  wrote:
> >>
> >> Yes. Is you public key in that file? If not you should add it.
> >>
> >> -Taylor
> >>
> >>> On Aug 12, 2019, at 4:15 PM, Ethan Li 
> wrote:
> >>>
> >>> I have one more question before I can start a vote.
> >>>
> >>> In previous [VOTE] thread, Taylor always has this:
> >>>
> >>> The release artifacts are signed with the following key:
> >>>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> <
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >
> >>>
> >>>
> >>> Does anyone know where to find the updated KEYs file? Should I just
> use https://www.apache.org/dist/storm/KEYS <
> https://www.apache.org/dist/storm/KEYS> ?
> >>>
> >>> Thanks very much
> >>>
> >>>> On Aug 12, 2019, at 9:44 AM, Ethan Li 
> wrote:
> >>>>
> >>>> Thanks Stig and Taylor! It worked. I will continue the process now
> and update the documentation later.
> >>>>
> >>>>
> >>>>> On Aug 12, 2019, at 9:40 AM, P. Taylor Goetz 
> wrote:
> >>>>>
> >>>>> For the 2.x version lines, there are a bunch of profiles you need to
> enable. This is what I use when preparing a release:
> >>>>>
> >>>>> -P dist,include-shaded-deps,rat,externals,examples
> >>>>>
> >>>>> -Taylor
> >>>>>
> >>>>>> On Aug 12, 2019, at 9:27 AM, Stig Rohde Døssing <
> stigdoess...@gmail.com> wrote:
> >>>>>>
> >>>>>> Try enabling the "dist" profile by adding "-P
> dist,externals,examples" to
> >>>>>> the command you're running. See
> >>>>>> https://github.com/apache/storm/blob/master/pom.xml#L518. Unless
> you enable
> >>>>>> that profile, the storm-dist modules aren't in the reactor.
> >>>>>>
> >>>>>> Den man. 12. aug. 2019 kl. 15.15 skrev Ethan Li <
> ethanopensou...@gmail.com>:
> >>>>>>
> >>>>>>> Just in case if anyone happens to know the answer:
> >>>>>>>
> >>>>>>> I created a branch
> https://github.com/apache/storm/tree/2.1.x-branch <
> >>>>>>> https://github.com/apache/storm/tree/2.1.x-branch> and ran
> “mvn:prepare”
> >>>>>>> and “mvn:perform”. It created two commits and created a v2.1.0 git
> tag. But
> >>>>>>> I realized that the pom version under storm-dist was not updated
> and it’s
> >>>>>>> still 2.0.1-SNAPSHOT. I checked the commits in other tags like
> >>>>>>> https://github.com/apache/storm/commits/v2.0.0 <
> >>>>>>> https://github.com/apache/storm/commits/v2.0.0> and it looks like
> >>>>>>> storm-dist got updated within the “mvn:prepare” step.  How do I get
> >>>>>>> storm-dist pom version updated with “mv:prepare”?
> >>>>>>>
> >>>>>>> I am currently blocked on this. Any help will be appreciated.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Ethan
> >>>>>>>
> >>>>>>>
> >>>>>>>> On Aug 12, 2019, at 2:38 AM, Stig Rohde Døssing <
> stigdoess...@gmai

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

Try enabling the "dist" profile by adding "-P dist,externals,examples" to
the command you're running. See
https://github.com/apache/storm/blob/master/pom.xml#L518. Unless you enable
that profile, the storm-dist modules aren't in the reactor.

Den man. 12. aug. 2019 kl. 15.15 skrev Ethan Li :

> Just in case if anyone happens to know the answer:
>
> I created a branch https://github.com/apache/storm/tree/2.1.x-branch <
> https://github.com/apache/storm/tree/2.1.x-branch> and ran “mvn:prepare”
> and “mvn:perform”. It created two commits and created a v2.1.0 git tag. But
> I realized that the pom version under storm-dist was not updated and it’s
> still 2.0.1-SNAPSHOT. I checked the commits in other tags like
> https://github.com/apache/storm/commits/v2.0.0 <
> https://github.com/apache/storm/commits/v2.0.0> and it looks like
> storm-dist got updated within the “mvn:prepare” step.  How do I get
> storm-dist pom version updated with “mv:prepare”?
>
> I am currently blocked on this. Any help will be appreciated.
>
> Thanks,
> Ethan
>
>
> > On Aug 12, 2019, at 2:38 AM, Stig Rohde Døssing 
> wrote:
> >
> > Sounds good Ethan.
> >
> > Derek, I also think being clear about how long we expect to support a
> > release line is a good idea. Maybe we should do a separate discuss thread
> > on this, or if you already have a good idea for what the policy should
> look
> > like you could put it up as a PR for either the Downloads page or a new
> > page on the Storm site?
> >
> > Den søn. 11. aug. 2019 kl. 08.37 skrev Ethan Li <
> ethanopensou...@gmail.com>:
> >
> >> I am doing the release following
> >> https://github.com/apache/storm/blob/master/RELEASING.md <
> >> https://github.com/apache/storm/blob/master/RELEASING.md> . I made some
> >> progress but I have some questions so I just sent an email to Taylor.
> >>
> >> Please don’t merge anything to master or 2.1.x-branch for now. Thanks
> >>
> >> Best,
> >> Ethan
> >>
> >>
> >>> On Aug 9, 2019, at 4:45 PM, Ethan Li 
> wrote:
> >>>
> >>> Currently we have two outstanding PRs that we wanted to include in the
> >> new release:
> >>>
> >>> https://github.com/apache/storm/pull/3096 <
> >> https://github.com/apache/storm/pull/3096> (Travis has some issues
> >> connecting to repository.apache.org <http://repository.apache.org/>
> >> recently. Travis build fails consistently)
> >>> https://github.com/apache/storm/pull/2878 <
> >> https://github.com/apache/storm/pull/2878> (Some comments are to be
> >> addressed but Author hasn’t responded yet.)
> >>>
> >>> It’s not absolutely necessary to include them in this new release so I
> >> think I should move forward with the release process. These two and
> other
> >> PRs can be included in the next release.
> >>>
> >>> For the release, I will create a new branch 2.1.x-branch based on
> >> current master branch, and release 2.1.0 from there.   I will then move
> >> master to 2.2.0-SNAPSHOT.  Please let me know if there is any
> objections.
> >>>
> >>> Thanks,
> >>> Ethan
> >>>
> >>>
> >>>> On Aug 9, 2019, at 2:53 PM, Derek Dagit  >> da...@apache.org>> wrote:
> >>>>
> >>>>> We might not able to say how long we want to support a specific
> >>>>> release line but I would love to see a clear release policy being
> >>>>> made.
> >>>>
> >>>> That makes sense. It seems better for us not to choose a specific
> >>>> calendar date or duration of time.
> >>>>
> >>>> - We do not want too many release lines supported concurrently.
> >>>> - We want devs and users to know what to expect.
> >>>>
> >>>> --
> >>>> Derek
> >>>>
> >>>> On Fri, Aug 09, 2019 at 10:32:59AM -0500, Ethan Li wrote:
> >>>>>
> >>>>> Derek,
> >>>>>
> >>>>> I think it’s a good idea to be more clear on the versioning and
> >> release process so users and developers know what to expect. We might
> not
> >> able to say how long we want to support a specific release line but I
> would
> >> love to see a clear release policy being made.
> >>>>>
> >>>>> Thanks,
> >>>>> Ethan
> >>>>>

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

Sounds good Ethan.

Derek, I also think being clear about how long we expect to support a
release line is a good idea. Maybe we should do a separate discuss thread
on this, or if you already have a good idea for what the policy should look
like you could put it up as a PR for either the Downloads page or a new
page on the Storm site?

Den søn. 11. aug. 2019 kl. 08.37 skrev Ethan Li :

> I am doing the release following
> https://github.com/apache/storm/blob/master/RELEASING.md <
> https://github.com/apache/storm/blob/master/RELEASING.md> . I made some
> progress but I have some questions so I just sent an email to Taylor.
>
> Please don’t merge anything to master or 2.1.x-branch for now. Thanks
>
> Best,
> Ethan
>
>
> > On Aug 9, 2019, at 4:45 PM, Ethan Li  wrote:
> >
> > Currently we have two outstanding PRs that we wanted to include in the
> new release:
> >
> > https://github.com/apache/storm/pull/3096 <
> https://github.com/apache/storm/pull/3096> (Travis has some issues
> connecting to repository.apache.org <http://repository.apache.org/>
> recently. Travis build fails consistently)
> > https://github.com/apache/storm/pull/2878 <
> https://github.com/apache/storm/pull/2878> (Some comments are to be
> addressed but Author hasn’t responded yet.)
> >
> > It’s not absolutely necessary to include them in this new release so I
> think I should move forward with the release process. These two and other
> PRs can be included in the next release.
> >
> > For the release, I will create a new branch 2.1.x-branch based on
> current master branch, and release 2.1.0 from there.   I will then move
> master to 2.2.0-SNAPSHOT.  Please let me know if there is any objections.
> >
> > Thanks,
> > Ethan
> >
> >
> >> On Aug 9, 2019, at 2:53 PM, Derek Dagit  da...@apache.org>> wrote:
> >>
> >>> We might not able to say how long we want to support a specific
> >>> release line but I would love to see a clear release policy being
> >>> made.
> >>
> >> That makes sense. It seems better for us not to choose a specific
> >> calendar date or duration of time.
> >>
> >> - We do not want too many release lines supported concurrently.
> >> - We want devs and users to know what to expect.
> >>
> >> --
> >> Derek
> >>
> >> On Fri, Aug 09, 2019 at 10:32:59AM -0500, Ethan Li wrote:
> >>>
> >>> Derek,
> >>>
> >>> I think it’s a good idea to be more clear on the versioning and
> release process so users and developers know what to expect. We might not
> able to say how long we want to support a specific release line but I would
> love to see a clear release policy being made.
> >>>
> >>> Thanks,
> >>> Ethan
> >>>
> >>>> On Aug 9, 2019, at 8:53 AM, Derek Dagit  da...@apache.org>> wrote:
> >>>>
> >>>> On Fri, Aug 09, 2019 at 02:35:57PM +0200, Stig Rohde Døssing wrote:
> >>>>> Where on the Traffic Server page do they list how long their release
> >>>>> trains survive? I only see dates of release, not how long e.g. 7.x is
> >>>>> supposed to receive support.  Derek,
> >>>>
> >>>> This is a better link:
> https://cwiki.apache.org/confluence/display/TS/Release+Management <
> https://cwiki.apache.org/confluence/display/TS/Release+Management>
> >>>>
> >>>> This example, where "RM" means "Release Manager":
> >>>>
> >>>>> 1. We promise to make 1 major release / year, but the RM and
> community
> >>>>>  can of course make more as necessary
> >>>>>
> >>>>> 2. We only make releases off the LTS branches, which are cut once a
> >>>>>  year off master
> >>>>>
> >>>>> 3. Master is always open, for any type of change (including
> >>>>>  incompatible changes). But don't break compatibility just for fun!
> >>>>>
> >>>>> 4. Master is always stable, i.e. commits should be properly tested
> and
> >>>>>  reviewed before committed to master.
> >>>>>
> >>>>> 5. All releases are stable releases, following strict Semantic
> >>>>>  Versioning.
> >>>>>
> >>>>> 6. Minor releases are made at the discretion at the discretion of the
> >>>>>  community and the RM.
> >>>>>
> >>>>> 7. M

[ANNOUNCE] New Committer/PMC Member Aaron Gresch

[Stig Rohde Døssing]

I'm happy to announce that Aaron Gresch is now a member of the Storm PMC.

Aaron has been active contributing to Storm for a while now. His
contributions have a good spread around the codebase, in particular having
to do with the blob store, metrics system and scheduling. He helped
implement metrics v2.

Please join me in welcoming Aaron to the PMC.

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

Ethan,

I think the idea has been that master is just the latest unreleased
version. The same goes for 1.x-branch, which is the latest unreleased 1.x
version (so 1.2.4 right now). I think we've been branching when necessary
rather than proactively, so e.g. when work requiring breaking changes to
1.x started, the 1.x-branch was created and master became 2.0.0-SNAPSHOT. I
don't see an issue branching proactively by cutting a 2.1.x-branch
immediately, but I'm not sure it's necessary. It means that any change
going in 2.1.1 needs to be merged to master and also 2.1.x, instead of
master just being 2.1.x until we need to bump to 2.2.x. I see it as just
adding an extra unnecessary branch, because until master contains something
that should go in 2.2.x and not 2.1.x, the branches have the same contents.
Branching at the point where 2.2.x and 2.1.x would diverge makes more sense
IMO.

I think at least for 2.1.0, I'd rather we ask users to upgrade to 2.1.0.
Otherwise, we need to create the 2.0.x branch from v2.0.0 and then go
cherry pick any changes that are already in master that should be in a
2.0.1 release. Is there anything in 2.1.0 that's risky enough that we want
to put the effort of also doing a 2.0.1 in?

Derek,

I think the period of support so far has been based mostly around whether
anyone wants to put in the work to support old branches. We've been
dropping support once no one shows an interest in supporting the old
branches anymore.

When you say a branch is LTS, I assume you mean we're promising to keep
putting out bugfix releases for that line? It seems like a fine idea,
assuming we limit ourselves to a couple of branches (maintaining 2.x and a
bunch of 1.x branches has been unpleasant in the past, but that probably
has more to do with slow release cadence). Regarding setting a date for
when a release line will no longer be supported, I'm not sure how easy it
is to set a date up front. For instance, I would have a hard time setting a
date for the end of 2.x support until there's a 3.x release on the horizon.

If we're going to promise that a release line survives for a given amount
of time, I think we should do it at the major version level only, this also
seems to be what Traffic Server is doing. If we're following semver,
upgrading to minor versions should be safe, and I don't see the value of
promising to maintain e.g. both 2.0.x and 2.1.x for long periods.

Where on the Traffic Server page do they list how long their release trains
survive? I only see dates of release, not how long e.g. 7.x is supposed to
receive support.

Den fre. 9. aug. 2019 kl. 00.15 skrev Derek Dagit :

> What do we think about defining Long-Term Support branches with a fixed
> period of support?
>
> For example, we could say 2.0.x is an LTS release line with support
> ending no earlier than a certain calendar date.
>
> The date could be extended, and it might ultimately be governed by the
> timing of the subsequent release (e.g., 2.1.x or 3.0.x). Keeping things
> clear would imply semantic versioning as mentioned earlier
> (https://semver.org/).
>
> Apache Traffic Server does something like this, to name one project:
>
> https://trafficserver.apache.org/downloads
>
> Having a regular cadence of releases might also help make the process
> easier and help set expectations for users and devs.
>
> --
> Derek
>
> On Thu, Aug 08, 2019 at 02:50:07PM -0500, Ethan Li wrote:
> >
> > Currently we don’t have a 2.0.x-branch and master is actually
> “2.0.1-SNAPSHOT”.
> >
> > So if we  do a 2.1.0 release,  we will create a 2.1.x-branch based on
> current master, release from there. And we change master to
> “2.2.0-SNAPSHOT”.
> >
> > But we will have one problem: we will lose 2.0.x release line.
> >
> > There are two things I can do:
> >
> > 1) create a 2.0.x-branch based on v2.0.0 tag.
> >
> > 2) ignore it. If there is an issue with 2.0.x release,  ask users to
> upgrade to 2.1.0.
> >
> > I prefer 1) but not sure if it’s the right way to make things right. Or
> please let me know if I misunderstood something and it’s not an issue.
> >
> > Btw, I am seeing the same issue with 1.x-branch. We shouldn’t have
> 1.x-branch. Instead, we should have 1.2.x-branch. But this is not a problem
> since we will not release 1.3.x.
> >
> > Thanks,
> > Ethan
> >
> >
> > > On Aug 7, 2019, at 10:43 AM, Ethan Li 
> wrote:
> > >
> > > Yes thanks.
> > >
> > >> On Aug 7, 2019, at 10:39 AM, Stig Rohde Døssing <
> stigdoess...@gmail.com> wrote:
> > >>
> > >> Sounds great. Remember to add your key to
> > >> https://dist.apache.org/repos/dist/release/storm/KEYS, you should be
> able
> > >> to update it with an SVN client. See also
>

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

Sounds great. Remember to add your key to
https://dist.apache.org/repos/dist/release/storm/KEYS, you should be able
to update it with an SVN client. See also
https://www.apache.org/dev/openpgp.html#update.

Den ons. 7. aug. 2019 kl. 15.05 skrev Ethan Li :

> I got my pgp key signed by Bryan W. Call  bc...@apache.org>> (Thanks to him).
>
> My pgp key:
> http://pgp.surfnet.nl/pks/lookup?op=vindex=on=0xA4A672F11B5050C8
> <
> http://pgp.surfnet.nl/pks/lookup?op=vindex=on=0xA4A672F11B5050C8
> >
>
> My understanding is that I am good to do release with this key now.
>
>
> Here is a list of PRs that we might want to include in the new release:
>
> https://github.com/apache/storm/pull/3098 <
> https://github.com/apache/storm/pull/3098>
> https://github.com/apache/storm/pull/3096 <
> https://github.com/apache/storm/pull/3096>
> https://github.com/apache/storm/pull/2878 <
> https://github.com/apache/storm/pull/2878>
>
>
> Please review if you get a chance.  Thanks
>
>
> Ethan
>
>
>
> > On Aug 1, 2019, at 4:19 AM, Stig Rohde Døssing 
> wrote:
> >
> > Thanks Ethan, yes 2.1.0 makes sense.
> >
> > Den man. 29. jul. 2019 kl. 23.43 skrev Ethan Li <
> ethanopensou...@gmail.com>:
> >
> >> It’s a good point. I will start a discussion thread for it.
> >>
> >>
> >> For the new release, I went through the list:
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20STORM%20AND%20fixVersion%20%3D%202.0.1
> >> <
> >>
> https://issues.apache.org/jira/issues/?jql=project%20=%20STORM%20AND%20fixVersion%20=%202.0.1
> >>>
> >>
> >> We introduced some new functionalities, including
> >> https://issues.apache.org/jira/browse/STORM-2720 <
> >> https://issues.apache.org/jira/browse/STORM-2720>
> >> https://issues.apache.org/jira/browse/STORM-3412 <
> >> https://issues.apache.org/jira/browse/STORM-3412>
> >> https://issues.apache.org/jira/browse/STORM-3411 <
> >> https://issues.apache.org/jira/browse/STORM-3411>
> >> https://issues.apache.org/jira/browse/STORM-3442 <
> >> https://issues.apache.org/jira/browse/STORM-3442>
> >> https://issues.apache.org/jira/browse/STORM-3396 <
> >> https://issues.apache.org/jira/browse/STORM-3396>
> >> https://issues.apache.org/jira/browse/STORM-3392 <
> >> https://issues.apache.org/jira/browse/STORM-3392>
> >> https://issues.apache.org/jira/browse/STORM-3395 <
> >> https://issues.apache.org/jira/browse/STORM-3395>
> >>
> >> So I think we should release 2.1.0 rather than 2.0.1.
> >>
> >> There are a few pull requests we may want to review before the next
> >> release:
> >>
> >> https://github.com/apache/storm/pull/3094 <
> >> https://github.com/apache/storm/pull/3094>
> >> https://github.com/apache/storm/pull/2990 <
> >> https://github.com/apache/storm/pull/2990>
> >> https://github.com/apache/storm/pull/2878 <
> >> https://github.com/apache/storm/pull/2878>
> >>
> >>
> >> Thanks
> >> Ethan
> >>
> >>
> >>> On Jul 29, 2019, at 10:11 AM, Hugo Louro  wrote:
> >>>
> >>> +1
> >>>
> >>> I think it would facilitate more frequent releases to summarize in a
> page
> >>> the testing that all contributors/committers do in anticipation of the
> >>> release, plus any "new" testing that may become relevant for the newer
> >>> releases. Doing so would make it easy to create a check form or or
> email
> >>> template that what we feel should be done to guarantee a stable
> release.
> >>>
> >>> Thanks,
> >>> Hugo
> >>>
> >>> On Mon, Jul 29, 2019 at 7:15 AM Ethan Li 
> >> wrote:
> >>>
> >>>> Thanks Stig. I will look into it.
> >>>>
> >>>>> On Jul 26, 2019, at 3:06 PM, Stig Rohde Døssing <
> >> stigdoess...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>> I think ideally we've been trying for semver, but it's been pretty
> >> loose,
> >>>>> e.g. there were breaking changes in one of the 1.2.x releases for
> >>>>> storm-kafka-client. I don't know what rules we've actually been
> using,
> >> if
> >>>>> any.
> >>>>>
> >>>>> Semver for binary compatibility would probably be a good 

Re: [DISCUSS] Create a checklist for testing new Storm release

[Stig Rohde Døssing]

The list looks good. I think we should drop the .md5 files, the ASF have
been telling people not to use them for a while.

I don't know that everyone needs to do it, but someone should check that
the license files are up to date, and that we're not including any
dependencies with incompatible licenses (
https://apache.org/legal/resolved.html). Ideally we can automate most of
the work.

Den man. 29. jul. 2019 kl. 23.56 skrev Ethan Li :

>
> Per Hugo’s suggestion, we should probably document the testings that every
> contributors/committers should do to guarantee a stable release. It also
> helps creating a check form or email template.
>
>
>  The testings I did for 2.0.0 release are
>
> 1. Verify files such as *.asc, *.md5, *.sha512
> 2. Build Storm source code and run unit tests; create a Storm distribution.
> 3. Set up a standalone cluster using apache-storm-xxx.zip,
> apache-storm-xxx.tar.gz, the Storm distribution created from step 2,
> separately
> 4. Launch WordCountTopology and ThroughputVsLatency topology and check
> logs, UI metrics
> 5. Test basic UI functionalities such as jstack, heap dump, deactivate,
> activate, rebalance, change log level, kill topology
>
>
> Please suggest anything that should be added so that we can document a
> base checklist that everyone can follow to test a new Storm release.
>
>
> Thanks,
> Ethan

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

Thanks Ethan, yes 2.1.0 makes sense.

Den man. 29. jul. 2019 kl. 23.43 skrev Ethan Li :

> It’s a good point. I will start a discussion thread for it.
>
>
> For the new release, I went through the list:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20STORM%20AND%20fixVersion%20%3D%202.0.1
> <
> https://issues.apache.org/jira/issues/?jql=project%20=%20STORM%20AND%20fixVersion%20=%202.0.1
> >
>
> We introduced some new functionalities, including
> https://issues.apache.org/jira/browse/STORM-2720 <
> https://issues.apache.org/jira/browse/STORM-2720>
> https://issues.apache.org/jira/browse/STORM-3412 <
> https://issues.apache.org/jira/browse/STORM-3412>
> https://issues.apache.org/jira/browse/STORM-3411 <
> https://issues.apache.org/jira/browse/STORM-3411>
> https://issues.apache.org/jira/browse/STORM-3442 <
> https://issues.apache.org/jira/browse/STORM-3442>
> https://issues.apache.org/jira/browse/STORM-3396 <
> https://issues.apache.org/jira/browse/STORM-3396>
> https://issues.apache.org/jira/browse/STORM-3392 <
> https://issues.apache.org/jira/browse/STORM-3392>
> https://issues.apache.org/jira/browse/STORM-3395 <
> https://issues.apache.org/jira/browse/STORM-3395>
>
> So I think we should release 2.1.0 rather than 2.0.1.
>
> There are a few pull requests we may want to review before the next
> release:
>
> https://github.com/apache/storm/pull/3094 <
> https://github.com/apache/storm/pull/3094>
> https://github.com/apache/storm/pull/2990 <
> https://github.com/apache/storm/pull/2990>
> https://github.com/apache/storm/pull/2878 <
> https://github.com/apache/storm/pull/2878>
>
>
> Thanks
> Ethan
>
>
> > On Jul 29, 2019, at 10:11 AM, Hugo Louro  wrote:
> >
> > +1
> >
> > I think it would facilitate more frequent releases to summarize in a page
> > the testing that all contributors/committers do in anticipation of the
> > release, plus any "new" testing that may become relevant for the newer
> > releases. Doing so would make it easy to create a check form or or email
> > template that what we feel should be done to guarantee a stable release.
> >
> > Thanks,
> > Hugo
> >
> > On Mon, Jul 29, 2019 at 7:15 AM Ethan Li 
> wrote:
> >
> >> Thanks Stig. I will look into it.
> >>
> >>> On Jul 26, 2019, at 3:06 PM, Stig Rohde Døssing <
> stigdoess...@gmail.com>
> >> wrote:
> >>>
> >>> I think ideally we've been trying for semver, but it's been pretty
> loose,
> >>> e.g. there were breaking changes in one of the 1.2.x releases for
> >>> storm-kafka-client. I don't know what rules we've actually been using,
> if
> >>> any.
> >>>
> >>> Semver for binary compatibility would probably be a good rule of thumb.
> >>>
> >>> Den fre. 26. jul. 2019 kl. 20.01 skrev Ethan Li <
> >> ethanopensou...@gmail.com>:
> >>>
> >>>>
> >>>> Stig,
> >>>>
> >>>> Do you know what’s the versioning standard we have been following (to
> >>>> determine a 2.0.1 release or 2.1.0 release) ?
> >>>>
> >>>>
> >>>>> On Jul 26, 2019, at 12:26 PM, Stig Rohde Døssing <
> >> stigdoess...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>> Sounds great, thanks Ethan.
> >>>>>
> >>>>> Den fre. 26. jul. 2019 kl. 19.16 skrev Ethan Li <
> >>>> ethanopensou...@gmail.com>:
> >>>>>
> >>>>>> It’s good idea to do more frequent release. I can run the next
> >> release.
> >>>>>>
> >>>>>> I will take a look at both PRs. Other than that, I think we should
> >> also
> >>>>>> get https://github.com/apache/storm/pull/3093 <
> >>>>>> https://github.com/apache/storm/pull/3093>  in the new release.
> >>>>>>
> >>>>>>
> >>>>>>> On Jul 26, 2019, at 11:58 AM, Stig Rohde Døssing <
> >>>> stigdoess...@gmail.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I think we've talked about more frequent releases before. Releasing
> >> new
> >>>>>>> versions every few months means people don't have to wait long for
> >>>> fixes
> >>>>>> to
> >>>>>>> get out, and smaller releases are probably also easier for users to
> >> get
> >>>>>> to
> >>>>>>> grips with (the fix list for 2.0.0 is enormous).
> >>>>>>>
> >>>>>>> With that in mind, I think we should start looking at the next 2.x
> >>>>>> release
> >>>>>>> (2.0.1 or 2.1.0?), since it's been a couple of months since 2.0.0
> >>>>>> released.
> >>>>>>> The fix list would be
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20STORM%20AND%20fixVersion%20%3D%202.0.1
> >>>>>>> .
> >>>>>>>
> >>>>>>> Govind and Ethan have offered to run the next release, and help
> >>>> validate
> >>>>>>> our release process guidelines. Would one of you have time to work
> >> on a
> >>>>>>> release in the near future?
> >>>>>>>
> >>>>>>> It would be good to take a look at currently open PRs and decide
> >> which
> >>>> we
> >>>>>>> feel need to get merged before the next release.
> >>>>>>>
> >>>>>>> I would like to see at least
> >> https://github.com/apache/storm/pull/2990
> >>>>>>> merged
> >>>>>>>
> >>>>>>> https://github.com/apache/storm/pull/2878 seems like it's close to
> >> be
> >>>>>>> mergeable too?
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>
> >>
>
>

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

I think ideally we've been trying for semver, but it's been pretty loose,
e.g. there were breaking changes in one of the 1.2.x releases for
storm-kafka-client. I don't know what rules we've actually been using, if
any.

Semver for binary compatibility would probably be a good rule of thumb.

Den fre. 26. jul. 2019 kl. 20.01 skrev Ethan Li :

>
> Stig,
>
> Do you know what’s the versioning standard we have been following (to
> determine a 2.0.1 release or 2.1.0 release) ?
>
>
> > On Jul 26, 2019, at 12:26 PM, Stig Rohde Døssing 
> wrote:
> >
> > Sounds great, thanks Ethan.
> >
> > Den fre. 26. jul. 2019 kl. 19.16 skrev Ethan Li <
> ethanopensou...@gmail.com>:
> >
> >> It’s good idea to do more frequent release. I can run the next release.
> >>
> >> I will take a look at both PRs. Other than that, I think we should also
> >> get https://github.com/apache/storm/pull/3093 <
> >> https://github.com/apache/storm/pull/3093>  in the new release.
> >>
> >>
> >>> On Jul 26, 2019, at 11:58 AM, Stig Rohde Døssing <
> stigdoess...@gmail.com>
> >> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I think we've talked about more frequent releases before. Releasing new
> >>> versions every few months means people don't have to wait long for
> fixes
> >> to
> >>> get out, and smaller releases are probably also easier for users to get
> >> to
> >>> grips with (the fix list for 2.0.0 is enormous).
> >>>
> >>> With that in mind, I think we should start looking at the next 2.x
> >> release
> >>> (2.0.1 or 2.1.0?), since it's been a couple of months since 2.0.0
> >> released.
> >>> The fix list would be
> >>>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20STORM%20AND%20fixVersion%20%3D%202.0.1
> >>> .
> >>>
> >>> Govind and Ethan have offered to run the next release, and help
> validate
> >>> our release process guidelines. Would one of you have time to work on a
> >>> release in the near future?
> >>>
> >>> It would be good to take a look at currently open PRs and decide which
> we
> >>> feel need to get merged before the next release.
> >>>
> >>> I would like to see at least https://github.com/apache/storm/pull/2990
> >>> merged
> >>>
> >>> https://github.com/apache/storm/pull/2878 seems like it's close to be
> >>> mergeable too?
> >>
> >>
>
>

Re: [DISCUSS] Next 2.x release

[Stig Rohde Døssing]

Sounds great, thanks Ethan.

Den fre. 26. jul. 2019 kl. 19.16 skrev Ethan Li :

> It’s good idea to do more frequent release. I can run the next release.
>
> I will take a look at both PRs. Other than that, I think we should also
> get https://github.com/apache/storm/pull/3093 <
> https://github.com/apache/storm/pull/3093>  in the new release.
>
>
> > On Jul 26, 2019, at 11:58 AM, Stig Rohde Døssing 
> wrote:
> >
> > Hi,
> >
> > I think we've talked about more frequent releases before. Releasing new
> > versions every few months means people don't have to wait long for fixes
> to
> > get out, and smaller releases are probably also easier for users to get
> to
> > grips with (the fix list for 2.0.0 is enormous).
> >
> > With that in mind, I think we should start looking at the next 2.x
> release
> > (2.0.1 or 2.1.0?), since it's been a couple of months since 2.0.0
> released.
> > The fix list would be
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20STORM%20AND%20fixVersion%20%3D%202.0.1
> > .
> >
> > Govind and Ethan have offered to run the next release, and help validate
> > our release process guidelines. Would one of you have time to work on a
> > release in the near future?
> >
> > It would be good to take a look at currently open PRs and decide which we
> > feel need to get merged before the next release.
> >
> > I would like to see at least https://github.com/apache/storm/pull/2990
> > merged
> >
> > https://github.com/apache/storm/pull/2878 seems like it's close to be
> > mergeable too?
>
>

[DISCUSS] Next 2.x release

[Stig Rohde Døssing]

Hi,

I think we've talked about more frequent releases before. Releasing new
versions every few months means people don't have to wait long for fixes to
get out, and smaller releases are probably also easier for users to get to
grips with (the fix list for 2.0.0 is enormous).

With that in mind, I think we should start looking at the next 2.x release
(2.0.1 or 2.1.0?), since it's been a couple of months since 2.0.0 released.
The fix list would be
https://issues.apache.org/jira/issues/?jql=project%20%3D%20STORM%20AND%20fixVersion%20%3D%202.0.1
.

Govind and Ethan have offered to run the next release, and help validate
our release process guidelines. Would one of you have time to work on a
release in the near future?

It would be good to take a look at currently open PRs and decide which we
feel need to get merged before the next release.

I would like to see at least https://github.com/apache/storm/pull/2990
merged

https://github.com/apache/storm/pull/2878 seems like it's close to be
mergeable too?

[CVE-2018-11779] Apache Storm UI Java deserialization vulnerability

[Stig Rohde Døssing]

[CVEID]:CVE-2018-11779[PRODUCT]:Apache Storm[VERSION]:Apache Storm
1.1.0 to 1.2.2[PROBLEMTYPE]:CWE-502: Deserialization of Untrusted
Data[DESCRIPTION]:In Apache Storm versions 1.1.0 to 1.2.2,
  when the user is using the storm-kafka-client or
storm-kafka modules,
  it is possible to cause the Storm UI daemon to
deserialize user provided bytes into a Java class.

Mitigation: Upgrade to Apache Storm 1.2.3 or later.

Credit: Bobby Evans for discovery and fix

[CVE-2019-0202] Apache Storm Logviewer file system access vulnerability

[Stig Rohde Døssing]

[CVEID]:CVE-2019-0202[PRODUCT]:Apache Storm[VERSION]:Apache Storm
0.9.1-incubating to 1.2.2[PROBLEMTYPE]:CWE-200: Information
Exposure[DESCRIPTION]:The Apache Storm Logviewer daemon exposes
HTTP-accessible endpoints to read/search log files on hosts running
Storm.
  In Apache Storm versions 0.9.1-incubating to 1.2.2, it
is possible to read files off the
  host's file system that were not intended to be
accessible via these endpoints.

Mitigation: Upgrade to Apache Storm 1.2.3 or later.

Credit: Stig Rohde Døssing for discovery and fix

[CVE-2018-1320] Apache Storm vulnerable Thrift version

[Stig Rohde Døssing]

[CVEID]:CVE-2018-1320[PRODUCT]:Apache Storm[VERSION]:Apache Storm
0.9.1-incubating to 1.2.2[PROBLEMTYPE]:CWE-20: Input
Validation[DESCRIPTION]:Apache Storm versions 0.9.1-incubating to
1.2.2
  use Thrift library versions vulnerable to CVE-2018-1320.

Mitigation: Upgrade to Apache Storm 1.2.3 or later.

Credit: Arun Mahadevan for discovery and fix

Re: [ANNOUNCE] Apache Storm 1.2.3 Released

[Stig Rohde Døssing]

This should now be fixed, thanks Sebb

Den tor. 18. jul. 2019 kl. 20.59 skrev Stig Rohde Døssing <
stigdoess...@gmail.com>:

> Thanks for letting me know, I'll fix it.
>
> Den tor. 18. jul. 2019 kl. 20.48 skrev sebbaz :
>
>>
>>
>> On 2019/07/18 17:49:55, Stig Rohde Døssing  wrote:
>> >  The Apache Storm community is pleased to announce the release of Apache
>> > Storm version 1.2.3.
>> >
>> > Storm is a distributed, fault-tolerant, and high-performance realtime
>> > computation system that provides strong guarantees on the processing of
>> > data. You can read more about Storm on the project website:
>> >
>> > http://storm.apache.org
>> >
>> > Downloads of source and binary distributions are listed in our download
>> > section:
>> >
>> > http://storm.apache.org/downloads.html
>>
>> The download page does *not* currently have links to 1.2.3
>>
>> > You can read more about this release in the following blog post:
>> >
>> > http://storm.apache.org/2019/07/18/storm123-released.html
>> > <http://storm.apache.org/2019/05/30/storm200-released.html>
>> >
>> > Distribution artifacts are available in Maven Central at the following
>> > coordinates:
>> >
>> > groupId: org.apache.storm
>> > artifactId: storm-core
>> > version: 1.2.3
>> >
>> > The full list of changes is available here[1]. Please let us know [2] if
>> > you encounter any problems.
>> >
>> > Regards,
>> >
>> > The Apache Storm Team
>> >
>> > [1]:
>> >
>> http://www.us.apache.org/dist/storm/apache-storm-1.2.3/RELEASE_NOTES.html
>> > <
>> http://www.us.apache.org/dist/storm/apache-storm-2.0.0/RELEASE_NOTES.html
>> >
>> > [2]: https://issues.apache.org/jira/browse/STORM
>> >
>>
>

Site updates

[Stig Rohde Døssing]

Just so it doesn't come as a surprise to anyone: I've done a bit of cleanup
on the site. The downloads page was linking to a bunch of old releases,
which we've been asked to remove from the mirroring system
https://issues.apache.org/jira/browse/STORM-2956.

As the downloads page already links to the archival system, I don't think
there's a reason to list out old release files explicitly, people can just
click the link to the archive.

I've removed documentation for outdated releases, so we now only have the
documentation for the newest release branches. Let me know if this was too
aggressive, and we need to restore some of the old docs.

There was a https://storm.apache.org/releases.html page, that was listing
releases, but nothing linked to it. I've deleted it, as equivalent
information can be found on the downloads page.

Re: [ANNOUNCE] Apache Storm 1.2.3 Released

[Stig Rohde Døssing]

Thanks for letting me know, I'll fix it.

Den tor. 18. jul. 2019 kl. 20.48 skrev sebbaz :

>
>
> On 2019/07/18 17:49:55, Stig Rohde Døssing  wrote:
> >  The Apache Storm community is pleased to announce the release of Apache
> > Storm version 1.2.3.
> >
> > Storm is a distributed, fault-tolerant, and high-performance realtime
> > computation system that provides strong guarantees on the processing of
> > data. You can read more about Storm on the project website:
> >
> > http://storm.apache.org
> >
> > Downloads of source and binary distributions are listed in our download
> > section:
> >
> > http://storm.apache.org/downloads.html
>
> The download page does *not* currently have links to 1.2.3
>
> > You can read more about this release in the following blog post:
> >
> > http://storm.apache.org/2019/07/18/storm123-released.html
> > <http://storm.apache.org/2019/05/30/storm200-released.html>
> >
> > Distribution artifacts are available in Maven Central at the following
> > coordinates:
> >
> > groupId: org.apache.storm
> > artifactId: storm-core
> > version: 1.2.3
> >
> > The full list of changes is available here[1]. Please let us know [2] if
> > you encounter any problems.
> >
> > Regards,
> >
> > The Apache Storm Team
> >
> > [1]:
> >
> http://www.us.apache.org/dist/storm/apache-storm-1.2.3/RELEASE_NOTES.html
> > <
> http://www.us.apache.org/dist/storm/apache-storm-2.0.0/RELEASE_NOTES.html>
> > [2]: https://issues.apache.org/jira/browse/STORM
> >
>

[ANNOUNCE] Apache Storm 1.2.3 Released

[Stig Rohde Døssing]

 The Apache Storm community is pleased to announce the release of Apache
Storm version 1.2.3.

Storm is a distributed, fault-tolerant, and high-performance realtime
computation system that provides strong guarantees on the processing of
data. You can read more about Storm on the project website:

http://storm.apache.org

Downloads of source and binary distributions are listed in our download
section:

http://storm.apache.org/downloads.html

You can read more about this release in the following blog post:

http://storm.apache.org/2019/07/18/storm123-released.html


Distribution artifacts are available in Maven Central at the following
coordinates:

groupId: org.apache.storm
artifactId: storm-core
version: 1.2.3

The full list of changes is available here[1]. Please let us know [2] if
you encounter any problems.

Regards,

The Apache Storm Team

[1]:
http://www.us.apache.org/dist/storm/apache-storm-1.2.3/RELEASE_NOTES.html

[2]: https://issues.apache.org/jira/browse/STORM

Re: [Discuss] ARM CI for Storm

[Stig Rohde Døssing]

Jungtaek, does the workflow outlined by Yikun work for you?

Den tor. 27. jun. 2019 kl. 18.00 skrev Roshan Naik
:

> Thanks for volunteering.
>
>
> Sent from Yahoo Mail for iPhone
>
>
> On Wednesday, June 26, 2019, 7:08 PM, Yikun Jiang 
> wrote:
>
> Yes, we will definitely help to fix ARM test failures in PRs, and the
> OpenLab CI will tell us which PR perhaps has ARM compatible problem, if
> it's an easy fix problem, I think authors can fix it by themselves. If not,
> we will help them to address it.
>
> For the lack of ARM environment, if the developer are interested in testing
> and debugging their storm patch in ARM  environment but don't have ARM
> environment , we can also provide tmp ARM environment to them for testing.
>
> Regards,
> Yikun
> ----
> Jiang Yikun（Kero）
> Mail: yikunk...@gmail.com
>
>
> Stig Rohde Døssing  于2019年6月26日周三 上午1:48写道：
>
> > It sounds pretty low risk for us, if you're volunteering to help fix any
> > ARM-specific CI failures that may crop up. Will this include helping to
> fix
> > ARM test failures in PRs? Most people are unlikely to have an ARM
> > environment they can use to test, and I'd prefer not to ask contributors
> to
> > fix ARM test failures themselves.
> >
> > Den tir. 25. jun. 2019 kl. 11.08 skrev Yikun Jiang  >:
> >
> > > Sorry for late reply, actually, we have a developer team that willing
> to
> > > work on this, and I'm the owner of storm ARM CI in the OpenLab team. We
> > not
> > > only want to enable the OpenLab CI, but also want to maintain the arm
> CI
> > > job and fix the CI issue in Storm project. That means if the storm
> > project
> > > has some ARM compatible problem, I will very happy to fix it.
> > >
> > > As I mentioned before, one of the OpenLab goal is to make more open
> > source
> > > software to be more compatible for aarch64 platform. And the Storm
> > project
> > > is one of the most important one in BigData area, so we would like to
> > > propose to work on aarch64 related works in Storm. we plan to start the
> > > aarch64 related work from to add aarch64 build job for Storm, and our
> > > initial plan as below:
> > >
> > > 1. We first propose enable a periodical job in OpenLab project to make
> > sure
> > > the storm can compile and build successfully in arm64 env.
> > > 2. We hope can run the OpenLab CI in every pr to ensure that any pr
> after
> > > Step 1 will not break the arm64 build.
> > > 3. We can add more complex test cases on aarch64, like unit tests and
> > > functional tests, step by step. It's a log term works.
> > >
> > > Of course, welcome another developers join to maintain the aarch64 CI,
> > and
> > > take effort on aarch64 work together.
> > >
> > > Thanks for your attention.
> > >
> > > Regards,
> > > Yikun
> > > 
> > > Jiang Yikun（Kero）
> > > Mail: yikunk...@gmail.com
> > >
> > >
> > > Stig Rohde Døssing  于2019年6月21日周五 上午2:23写道：
> > >
> > > > I guess there is no interest in maintaining ARM compatibility, or at
> > > least
> > > > no one currently wants to take on the effort. Let's not add the
> Openlab
> > > CI
> > > > then, we can always do it later if someone expresses interest (and
> > > > willingness to maintain).
> > > >
> > > > Den ons. 12. jun. 2019 kl. 17.11 skrev Stig Rohde Døssing <
> > > > stigdoess...@gmail.com>:
> > > >
> > > > > Good point, let's see if there's anyone with an ARM environment.
> > > > >
> > > > > Den tir. 11. jun. 2019 kl. 23.07 skrev Jungtaek Lim <
> > kabh...@gmail.com
> > > >:
> > > > >
> > > > >> I guess the point is not related to open CI build. The point is
> > > whether
> > > > we
> > > > >> really want to support ARM. I'm seeing OpenLab request for other
> > > Apache
> > > > >> projects as well, so I'd rather not treat their request as
> > commitment
> > > of
> > > > >> putting efforts to make builds on ARM green.
> > > > >>
> > > > >> I'm a bit hesitant to add some environment on the support list
> > unless
> > > > >> there're enough engineers willing to work on maintaining. If the
> > work
> > > is
> > > > >&g

Re: Intellij can't resolve shaded dependencies

[Stig Rohde Døssing]

There's an intellij profile in the root pom. Does it no longer work?
https://github.com/apache/storm/blob/fb76dd1c7dc39c4979f9cc921cf4a5930dfaa760/pom.xml#L377

Den tor. 27. jun. 2019 kl. 17.29 skrev Ethan Li :

> Hi,
>
> If you checked out the latest community master branch code and your
> intellij can’t resolve the shaded dependencies, you could probably try this
> work around:
> https://youtrack.jetbrains.com/issue/IDEA-126596#focus=streamItem-27-757181.0-0
> <
> https://youtrack.jetbrains.com/issue/IDEA-126596#focus=streamItem-27-757181.0-0>
> . It works for me. But be aware that with this, intellij will use
> storm-shaded-deps which's already built and installed in ~/.m2 directory.
>
> If there is a better way to solve it, please advice. I’d really appreciate
> it.
>
>
> Best,
> Ethan

Re: [Discuss] ARM CI for Storm

[Stig Rohde Døssing]

It sounds pretty low risk for us, if you're volunteering to help fix any
ARM-specific CI failures that may crop up. Will this include helping to fix
ARM test failures in PRs? Most people are unlikely to have an ARM
environment they can use to test, and I'd prefer not to ask contributors to
fix ARM test failures themselves.

Den tir. 25. jun. 2019 kl. 11.08 skrev Yikun Jiang :

> Sorry for late reply, actually, we have a developer team that willing to
> work on this, and I'm the owner of storm ARM CI in the OpenLab team. We not
> only want to enable the OpenLab CI, but also want to maintain the arm CI
> job and fix the CI issue in Storm project. That means if the storm project
> has some ARM compatible problem, I will very happy to fix it.
>
> As I mentioned before, one of the OpenLab goal is to make more open source
> software to be more compatible for aarch64 platform. And the Storm project
> is one of the most important one in BigData area, so we would like to
> propose to work on aarch64 related works in Storm. we plan to start the
> aarch64 related work from to add aarch64 build job for Storm, and our
> initial plan as below:
>
> 1. We first propose enable a periodical job in OpenLab project to make sure
> the storm can compile and build successfully in arm64 env.
> 2. We hope can run the OpenLab CI in every pr to ensure that any pr after
> Step 1 will not break the arm64 build.
> 3. We can add more complex test cases on aarch64, like unit tests and
> functional tests, step by step. It's a log term works.
>
> Of course, welcome another developers join to maintain the aarch64 CI, and
> take effort on aarch64 work together.
>
> Thanks for your attention.
>
> Regards,
> Yikun
> ------------
> Jiang Yikun（Kero）
> Mail: yikunk...@gmail.com
>
>
> Stig Rohde Døssing  于2019年6月21日周五 上午2:23写道：
>
> > I guess there is no interest in maintaining ARM compatibility, or at
> least
> > no one currently wants to take on the effort. Let's not add the Openlab
> CI
> > then, we can always do it later if someone expresses interest (and
> > willingness to maintain).
> >
> > Den ons. 12. jun. 2019 kl. 17.11 skrev Stig Rohde Døssing <
> > stigdoess...@gmail.com>:
> >
> > > Good point, let's see if there's anyone with an ARM environment.
> > >
> > > Den tir. 11. jun. 2019 kl. 23.07 skrev Jungtaek Lim  >:
> > >
> > >> I guess the point is not related to open CI build. The point is
> whether
> > we
> > >> really want to support ARM. I'm seeing OpenLab request for other
> Apache
> > >> projects as well, so I'd rather not treat their request as commitment
> of
> > >> putting efforts to make builds on ARM green.
> > >>
> > >> I'm a bit hesitant to add some environment on the support list unless
> > >> there're enough engineers willing to work on maintaining. If the work
> is
> > >> left to existing maintainers, we will be stuck on failing tests on ARM
> > CI
> > >> but no environment to play locally.
> > >>
> > >> So I'd be +1 if there're at least two folks experienced with ARM env.
> > >> volunteer to maintain the compatibility part. Otherwise -1 here.
> > >>
> > >> Thanks,
> > >> Jungtaek Lim (HeartSaVioR)
> > >>
> > >> On Mon, Jun 10, 2019 at 7:27 PM Stig Rohde Døssing <
> > >> stigdoess...@gmail.com>
> > >> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > An issue was opened suggesting adding ARM CI to the Storm build,
> using
> > >> > OpenLab https://issues.apache.org/jira/browse/STORM-3401,
> associated
> > PR
> > >> > here https://github.com/apache/storm/pull/3023
> > >> >
> > >> > As far as I can tell, we need to allow OpenLab access to the Github
> > PRs,
> > >> > similar to how Travis has permission to access our PRs. The app that
> > >> needs
> > >> > access is https://github.com/apps/theopenlab-ci. Note that the app
> > will
> > >> > have access to the PRs only, not the repo code.
> > >> >
> > >> > Is anyone opposed to me asking infra to allow OpenLab access to our
> > PRs?
> > >> > I'll let this thread sit for a week or so, if there is no opposition
> > >> I'll
> > >> > ask infra to give the app access.
> > >> >
> > >>
> > >>
> > >> --
> > >> Name : Jungtaek Lim
> > >> Blog : http://medium.com/@heartsavior
> > >> Twitter : http://twitter.com/heartsavior
> > >> LinkedIn : http://www.linkedin.com/in/heartsavior
> > >>
> > >
> >
>

Re: [Discuss] ARM CI for Storm

[Stig Rohde Døssing]

I guess there is no interest in maintaining ARM compatibility, or at least
no one currently wants to take on the effort. Let's not add the Openlab CI
then, we can always do it later if someone expresses interest (and
willingness to maintain).

Den ons. 12. jun. 2019 kl. 17.11 skrev Stig Rohde Døssing <
stigdoess...@gmail.com>:

> Good point, let's see if there's anyone with an ARM environment.
>
> Den tir. 11. jun. 2019 kl. 23.07 skrev Jungtaek Lim :
>
>> I guess the point is not related to open CI build. The point is whether we
>> really want to support ARM. I'm seeing OpenLab request for other Apache
>> projects as well, so I'd rather not treat their request as commitment of
>> putting efforts to make builds on ARM green.
>>
>> I'm a bit hesitant to add some environment on the support list unless
>> there're enough engineers willing to work on maintaining. If the work is
>> left to existing maintainers, we will be stuck on failing tests on ARM CI
>> but no environment to play locally.
>>
>> So I'd be +1 if there're at least two folks experienced with ARM env.
>> volunteer to maintain the compatibility part. Otherwise -1 here.
>>
>> Thanks,
>> Jungtaek Lim (HeartSaVioR)
>>
>> On Mon, Jun 10, 2019 at 7:27 PM Stig Rohde Døssing <
>> stigdoess...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > An issue was opened suggesting adding ARM CI to the Storm build, using
>> > OpenLab https://issues.apache.org/jira/browse/STORM-3401, associated PR
>> > here https://github.com/apache/storm/pull/3023
>> >
>> > As far as I can tell, we need to allow OpenLab access to the Github PRs,
>> > similar to how Travis has permission to access our PRs. The app that
>> needs
>> > access is https://github.com/apps/theopenlab-ci. Note that the app will
>> > have access to the PRs only, not the repo code.
>> >
>> > Is anyone opposed to me asking infra to allow OpenLab access to our PRs?
>> > I'll let this thread sit for a week or so, if there is no opposition
>> I'll
>> > ask infra to give the app access.
>> >
>>
>>
>> --
>> Name : Jungtaek Lim
>> Blog : http://medium.com/@heartsavior
>> Twitter : http://twitter.com/heartsavior
>> LinkedIn : http://www.linkedin.com/in/heartsavior
>>
>

Re: [Discuss] ARM CI for Storm

[Stig Rohde Døssing]

Good point, let's see if there's anyone with an ARM environment.

Den tir. 11. jun. 2019 kl. 23.07 skrev Jungtaek Lim :

> I guess the point is not related to open CI build. The point is whether we
> really want to support ARM. I'm seeing OpenLab request for other Apache
> projects as well, so I'd rather not treat their request as commitment of
> putting efforts to make builds on ARM green.
>
> I'm a bit hesitant to add some environment on the support list unless
> there're enough engineers willing to work on maintaining. If the work is
> left to existing maintainers, we will be stuck on failing tests on ARM CI
> but no environment to play locally.
>
> So I'd be +1 if there're at least two folks experienced with ARM env.
> volunteer to maintain the compatibility part. Otherwise -1 here.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> On Mon, Jun 10, 2019 at 7:27 PM Stig Rohde Døssing  >
> wrote:
>
> > Hi,
> >
> > An issue was opened suggesting adding ARM CI to the Storm build, using
> > OpenLab https://issues.apache.org/jira/browse/STORM-3401, associated PR
> > here https://github.com/apache/storm/pull/3023
> >
> > As far as I can tell, we need to allow OpenLab access to the Github PRs,
> > similar to how Travis has permission to access our PRs. The app that
> needs
> > access is https://github.com/apps/theopenlab-ci. Note that the app will
> > have access to the PRs only, not the repo code.
> >
> > Is anyone opposed to me asking infra to allow OpenLab access to our PRs?
> > I'll let this thread sit for a week or so, if there is no opposition I'll
> > ask infra to give the app access.
> >
>
>
> --
> Name : Jungtaek Lim
> Blog : http://medium.com/@heartsavior
> Twitter : http://twitter.com/heartsavior
> LinkedIn : http://www.linkedin.com/in/heartsavior
>

[Discuss] ARM CI for Storm

[Stig Rohde Døssing]

Hi,

An issue was opened suggesting adding ARM CI to the Storm build, using
OpenLab https://issues.apache.org/jira/browse/STORM-3401, associated PR
here https://github.com/apache/storm/pull/3023

As far as I can tell, we need to allow OpenLab access to the Github PRs,
similar to how Travis has permission to access our PRs. The app that needs
access is https://github.com/apps/theopenlab-ci. Note that the app will
have access to the PRs only, not the repo code.

Is anyone opposed to me asking infra to allow OpenLab access to our PRs?
I'll let this thread sit for a week or so, if there is no opposition I'll
ask infra to give the app access.

Re: Storm 2.0 Maven Artifacts

[Stig Rohde Døssing]

No, I think you are right. It doesn't look like the 2.0.0 artifacts are
present in Maven yet.

CC'ing to the dev list.

Den man. 3. jun. 2019 kl. 17.30 skrev Re'em Bensimhon :

> Hey everybody
>
> I'm trying to test out the new Storm release (very exciting :-) but I'm
> not managing to find the v2.0 artifacts for storm-client or storm-core in
> any public maven repository.
>
> https://repo.maven.apache.org/maven2/org/apache/storm/storm/
> https://mvnrepository.com/artifact/org.apache.storm/storm
>
> Am I missing something?
>
> Re'em
>

[Discuss] New release managers

[Stig Rohde Døssing]

Hi everyone,

I think we should have more people willing to act as release managers.
Taylor has expressed a desire to step back from the role (or at least to
not be the only RM), and it would be good if we could have a few people
ready to step up and wear the RM hat. If you're interested in this role,
this thread would be a good place to say so.

There's some general information about releasing under Apache at
http://www.apache.org/dev/release-publishing.html.

As I understand it, a new RM will need to at least generate and publish
their signature before they can act as RM. More info at
http://www.apache.org/dev/release-signing.html. I'm not sure whether
linking with the Apache web of trust is also mandatory.

Taylor, it would be helpful if you could explain any additional steps
beyond what is in the Apache guide to what a release manager needs to do to
release Storm.

Re: [ANNOUNCE] Apache Storm 2.0.0 Released

[Stig Rohde Døssing]

Thanks for running the release Taylor.

Most likely users will want the storm-client jar, rather than the
storm-core jar.

Den tor. 30. maj 2019 kl. 21.56 skrev P. Taylor Goetz :

> The Apache Storm community is pleased to announce the release of Apache
> Storm version 2.0.0.
>
> Storm is a distributed, fault-tolerant, and high-performance realtime
> computation system that provides strong guarantees on the processing of
> data. You can read more about Storm on the project website:
>
> http://storm.apache.org
>
> Downloads of source and binary distributions are listed in our download
> section:
>
> http://storm.apache.org/downloads.html
>
> You can read more about this release in the following blog post:
>
> http://storm.apache.org/2019/05/30/storm200-released.html
>
> Distribution artifacts are available in Maven Central at the following
> coordinates:
>
> groupId: org.apache.storm
> artifactId: storm-core
> version: 2.0.0
>
> The full list of changes is available here[1]. Please let us know [2] if
> you encounter any problems.
>
> Regards,
>
> The Apache Storm Team
>
> [1]:
> http://www.us.apache.org/dist/storm/apache-storm-2.0.0/RELEASE_NOTES.html
> [2]: https://issues.apache.org/jira/browse/STORM

Re: [VOTE] Release Apache Storm 2.0.0 RC7

[Stig Rohde Døssing]

Subscription has been replaced with TopicFilter. Subscription isn't
deprecated in 1.x because it is not possible to backport TopicFilter
without breaking changes. The replacement is the TopicFilter and
ManualPartitioner classes.

getKey/ValueDeserializer is deprecated in the latest 1.x release, the
Javadoc will say how to replace it.

For convenience: " Please use {@link #getKafkaProps() } and look up the
entry for {@link ConsumerConfig#KEY_DESERIALIZER_CLASS_CONFIG} instead".
There's a similar one for the value deserializer.

I suspect the last error is because getValueDeserializer no longer exists?

Den ons. 1. maj 2019 kl. 22.04 skrev Alexandre Vermeerbergen <
avermeerber...@gmail.com>:

> Hello,
>
> Thanks, and my step issue with Storm 2.0.0 is with some Storm Kafka
> Client breaking changes in KafkaSpoutConfig class.
> See attached source for our homebrewed BasicKafkaSpout, I have at
> least the following compilation errors:
>
>
> ./i3DXSupTopologiesCommons/i3DXSupTopologiesCommons.mj/src/com/dassault_systemes/storm/spout/BasicKafkaSpout.java:58:
> error: cannot find symbol
> logger.debug("topics are: {}", config.getSubscription()
>
>
> ./i3DXSupTopologiesCommons/i3DXSupTopologiesCommons.mj/src/com/dassault_systemes/storm/spout/BasicKafkaSpout.java:77:
> error: cannot find symbol
> if (config.getKeyDeserializer()==null) {
>
>
> ./i3DXSupTopologiesCommons/i3DXSupTopologiesCommons.mj/src/com/dassault_systemes/storm/spout/BasicKafkaSpout.java:91:
> error: cannot find symbol
> this.valueDeserializer =
> config.getValueDeserializer().getClass();
>
> and this more complex error:
>
>
> ./i3DXSupTopologiesCommons/i3DXSupTopologiesCommons.mj/src/com/dassault_systemes/storm/spout/BasicKafkaSpout.java:91:
> error: incompatible types: getClass cannot be converted to Class extends Deserializer>
> this.valueDeserializer =
> config.getValueDeserializer().getClass();
>
>
> What would be the migration path from Storm 1.x to Storm 2.x for this
> storm-kafka-client dependent class?
>
> Kind regards,
> Alexandre Vermeerbergen
>
> Le mer. 1 mai 2019 à 21:18, Stig Rohde Døssing
>  a écrit :
> >
> > Yes, we already have a check that the Maven version is above 3.0.0 (
> > https://github.com/apache/storm/blob/master/pom.xml#L1439), IMO we can
> just
> > bump it to something like 3.5.0. I'll raise an issue for it.
> >
> > Den ons. 1. maj 2019 kl. 20.59 skrev Alexandre Vermeerbergen <
> > avermeerber...@gmail.com>:
> >
> > > Hello Stig,
> > >
> > > Good catch: I upgraded Maven to version 3.6.1 and I rebuilt everything
> > > from source tar.gz without any trouble.
> > >
> > > So I delay my vote and I'm no longer blocked since I have
> > > storm-kafka-client-2.0.0.jar allowing me to rebuilt my topologies.
> > >
> > > Just a question: since my experience showed that build Storm 2.0.0
> > > break with Maven 3.3.9, wouldn't it be possible to improve Storm's
> > > pom.xml to include a Maven version check (see
> > > https://maven.apache.org/enforcer/enforcer-rules/versionRanges.html) ?
> > >
> > > This would improve Storm users' experience a little bit, and sounds
> > > very simple to do (not a blocker for 2.0.0 of course).
> > >
> > > How about that?
> > >
> > > Kind regards,
> > > Alexandre Vermeerbergen
> > >
> > > Le mer. 1 mai 2019 à 19:42, Stig Rohde Døssing
> > >  a écrit :
> > > >
> > > > I haven't seen the error you're getting before, but it looks like
> it's a
> > > > Maven issue, not a Storm issue. Could you try updating to the latest
> > > Maven
> > > > version?
> > > >
> > > > Den ons. 1. maj 2019 kl. 19.30 skrev Alexandre Vermeerbergen <
> > > > avermeerber...@gmail.com>:
> > > >
> > > > > Hello,
> > > > >
> > > > > I don't know if this is worth a -1 because maybe I did something
> > > > > wrong, but I got a failure when trying to build binaries from the
> > > > > sources, see attached log (zipped) with 4 errors, captured from the
> > > > > output of:
> > > > >
> > > > > mvn clean install -DskipTests 2>&1 build_Storm_2.0.0_rc7.log
> > > > >
> > > > > more details on my machine:
> > > > >
> > > > > $ cat /etc/redhat-release
> > > > > CentOS release 6.7 (Final)
> > > > >
> > > > > $ mvn -version

Re: [VOTE] Release Apache Storm 2.0.0 RC7

[Stig Rohde Døssing]

Yes, we already have a check that the Maven version is above 3.0.0 (
https://github.com/apache/storm/blob/master/pom.xml#L1439), IMO we can just
bump it to something like 3.5.0. I'll raise an issue for it.

Den ons. 1. maj 2019 kl. 20.59 skrev Alexandre Vermeerbergen <
avermeerber...@gmail.com>:

> Hello Stig,
>
> Good catch: I upgraded Maven to version 3.6.1 and I rebuilt everything
> from source tar.gz without any trouble.
>
> So I delay my vote and I'm no longer blocked since I have
> storm-kafka-client-2.0.0.jar allowing me to rebuilt my topologies.
>
> Just a question: since my experience showed that build Storm 2.0.0
> break with Maven 3.3.9, wouldn't it be possible to improve Storm's
> pom.xml to include a Maven version check (see
> https://maven.apache.org/enforcer/enforcer-rules/versionRanges.html) ?
>
> This would improve Storm users' experience a little bit, and sounds
> very simple to do (not a blocker for 2.0.0 of course).
>
> How about that?
>
> Kind regards,
> Alexandre Vermeerbergen
>
> Le mer. 1 mai 2019 à 19:42, Stig Rohde Døssing
>  a écrit :
> >
> > I haven't seen the error you're getting before, but it looks like it's a
> > Maven issue, not a Storm issue. Could you try updating to the latest
> Maven
> > version?
> >
> > Den ons. 1. maj 2019 kl. 19.30 skrev Alexandre Vermeerbergen <
> > avermeerber...@gmail.com>:
> >
> > > Hello,
> > >
> > > I don't know if this is worth a -1 because maybe I did something
> > > wrong, but I got a failure when trying to build binaries from the
> > > sources, see attached log (zipped) with 4 errors, captured from the
> > > output of:
> > >
> > > mvn clean install -DskipTests 2>&1 build_Storm_2.0.0_rc7.log
> > >
> > > more details on my machine:
> > >
> > > $ cat /etc/redhat-release
> > > CentOS release 6.7 (Final)
> > >
> > > $ mvn -version
> > > Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5;
> > > 2015-11-10T17:41:47+01:00)
> > > Maven home: /home/data/ave/maven/src/apache-maven-3.3.9
> > > Java version: 1.8.0_66, vendor: Oracle Corporation
> > > Java home: /usr/java/jdk1.8.0_66_x86_64/jdk1.8.0_66/jre
> > > Default locale: en_US, platform encoding: UTF-8
> > > OS name: "linux", version: "2.6.32-573.el6.x86_64", arch: "amd64",
> > > family: "unix"
> > >
> > > I got couple of JAR files built, but I'm missing the
> > > storm-kafka-client one, so am blocked because I need this "external"
> > > lib to test my topologies rebuilt with Storm 2.0.0 RC7
> > >
> > > Did I missed something that break my Storm 2.0.0 RC7 build?
> > >
> > > Kind regards,
> > > Alexandre Vermeerbergen
> > >
> > >
> > > Le mar. 30 avr. 2019 à 00:49, P. Taylor Goetz  a
> écrit
> > > :
> > > >
> > > > This is a call to vote on releasing Apache Storm 2.0.0 (rc7)
> > > >
> > > > Full list of changes in this release:
> > > >
> > > >
> > >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/RELEASE_NOTES.html
> > > >
> > > > The tag/commit to be voted upon is v2.0.0:
> > > >
> > > >
> > >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=007863edd95e838b3df414928c6fa3f28244ab49;hb=2ba95bbd1c911d4fc6363b1c4b9c4c6d86ac9aae
> > > >
> > > > The source archive being voted upon can be found here:
> > > >
> > > >
> > >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/apache-storm-2.0.0-src.tar.gz
> > > >
> > > > Other release files, signatures and digests can be found here:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/
> > > >
> > > > The release artifacts are signed with the following key:
> > > >
> > > >
> > >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> > > >
> > > > The Nexus staging repository for this release is:
> > > >
> > > >
> https://repository.apache.org/content/repositories/orgapachestorm-1079
> > > >
> > > > Please vote on releasing this package as Apache Storm 2.0.0.
> > > >
> > > > When voting, please list the actions taken to verify the release.
> > > >
> > > > This vote will be open for at least 72 hours.
> > > >
> > > > [ ] +1 Release this package as Apache Storm 2.0.0
> > > > [ ]  0 No opinion
> > > > [ ] -1 Do not release this package because...
> > > >
> > > > Thanks to everyone who contributed to this release.
> > > >
> > > > -Taylor
> > >
>

Re: [VOTE] Release Apache Storm 2.0.0 RC7

[Stig Rohde Døssing]

I haven't seen the error you're getting before, but it looks like it's a
Maven issue, not a Storm issue. Could you try updating to the latest Maven
version?

Den ons. 1. maj 2019 kl. 19.30 skrev Alexandre Vermeerbergen <
avermeerber...@gmail.com>:

> Hello,
>
> I don't know if this is worth a -1 because maybe I did something
> wrong, but I got a failure when trying to build binaries from the
> sources, see attached log (zipped) with 4 errors, captured from the
> output of:
>
> mvn clean install -DskipTests 2>&1 build_Storm_2.0.0_rc7.log
>
> more details on my machine:
>
> $ cat /etc/redhat-release
> CentOS release 6.7 (Final)
>
> $ mvn -version
> Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5;
> 2015-11-10T17:41:47+01:00)
> Maven home: /home/data/ave/maven/src/apache-maven-3.3.9
> Java version: 1.8.0_66, vendor: Oracle Corporation
> Java home: /usr/java/jdk1.8.0_66_x86_64/jdk1.8.0_66/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "2.6.32-573.el6.x86_64", arch: "amd64",
> family: "unix"
>
> I got couple of JAR files built, but I'm missing the
> storm-kafka-client one, so am blocked because I need this "external"
> lib to test my topologies rebuilt with Storm 2.0.0 RC7
>
> Did I missed something that break my Storm 2.0.0 RC7 build?
>
> Kind regards,
> Alexandre Vermeerbergen
>
>
> Le mar. 30 avr. 2019 à 00:49, P. Taylor Goetz  a écrit
> :
> >
> > This is a call to vote on releasing Apache Storm 2.0.0 (rc7)
> >
> > Full list of changes in this release:
> >
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/RELEASE_NOTES.html
> >
> > The tag/commit to be voted upon is v2.0.0:
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=007863edd95e838b3df414928c6fa3f28244ab49;hb=2ba95bbd1c911d4fc6363b1c4b9c4c6d86ac9aae
> >
> > The source archive being voted upon can be found here:
> >
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/apache-storm-2.0.0-src.tar.gz
> >
> > Other release files, signatures and digests can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/
> >
> > The release artifacts are signed with the following key:
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >
> > The Nexus staging repository for this release is:
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1079
> >
> > Please vote on releasing this package as Apache Storm 2.0.0.
> >
> > When voting, please list the actions taken to verify the release.
> >
> > This vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache Storm 2.0.0
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > -Taylor
>

Re: [VOTE] Release Apache Storm 2.0.0 RC7

[Stig Rohde Døssing]

Alexandre,

I think we tend to assume people use a dependency management tool:
https://storm.apache.org/releases/2.0.0-SNAPSHOT/Maven.html

The easiest way for you to get a list of the jars that need to be on the
classpath is to make a dummy Maven project, add the snippet I linked to the
POM, and then use the dependency plugin to get a list of dependencies (do
"mvn dependency:list" in the project root). You could also run that command
from the storm-client directory in the Storm source.

Regarding which dependencies you should have, the link snippet is enough
for a production jar. If you are running tests using a LocalCluster, you
will also need to have storm-server on the test classpath.

Neither jar should be included in your fat jar. Storm puts them on the
classpath when you deploy to a cluster.

Den ons. 1. maj 2019 kl. 09.30 skrev Alexandre Vermeerbergen <
avermeerber...@gmail.com>:

> Hello,
>
> I tried to rebuild my topologies with 2.0, but I hit a first trivial
> issue: I used to rely on storm-core-.jar in our classpath to
> resolve Storm API dependencies, but storm-core-2.0.0.jar doesn't seem
> to contain anymore classes like org.apache.storm.task.OutputCollector
> which is used in our topologies' code.
>
> I did a "grep org.apache.storm.task.OutputCollector  directory>" and found that this kind of classes are now in
> storm-client-2.0.0.jar
>
> I'm OK to change my dependencies, but then, to avoid other people
> falling into this trap, couldn't Storm 2.0.0 documentation tell which
> are the build-time dependencies?
>
> I also need to know whether or not my BigJars must embed
> storm-client.2.0.0.jar (we use to embed storm-core-.jar in
> our BigJar)
>
> Kind regards,
> Alexandre Vermeerbergen
>
> Le mar. 30 avr. 2019 à 23:49, Roshan Naik
>  a écrit :
> >
> >  Yes, you need to rebuild your topology jars against 2.0.
> > If you have some settings to tweak perf with 1.x,  refer to
> https://github.com/apache/storm/blob/master/docs/Performance.md for the
> 2.x configs.
> >
> > On Tuesday, April 30, 2019, 2:41:49 PM PDT, Alexandre Vermeerbergen <
> avermeerber...@gmail.com> wrote:
> >
> >  Hello,
> >
> > I'm eager to test Storm 2.0.0 with my complex topologies, but first of
> > all: do I need to rebuild all my topologies' Big Jars with Storm
> > 2.0.0, or may I try my existing Storm 1.2.3 (recent snapshot)-based
> > Big Jars ?
> >
> > Kind regards,
> > Alexandre Vermeerbergen
> >
> > Le mar. 30 avr. 2019 à 00:49, P. Taylor Goetz  a
> écrit :
> > >
> > > This is a call to vote on releasing Apache Storm 2.0.0 (rc7)
> > >
> > > Full list of changes in this release:
> > >
> > >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/RELEASE_NOTES.html
> > >
> > > The tag/commit to be voted upon is v2.0.0:
> > >
> > >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=007863edd95e838b3df414928c6fa3f28244ab49;hb=2ba95bbd1c911d4fc6363b1c4b9c4c6d86ac9aae
> > >
> > > The source archive being voted upon can be found here:
> > >
> > >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/apache-storm-2.0.0-src.tar.gz
> > >
> > > Other release files, signatures and digests can be found here:
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/
> > >
> > > The release artifacts are signed with the following key:
> > >
> > >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> > >
> > > The Nexus staging repository for this release is:
> > >
> > > https://repository.apache.org/content/repositories/orgapachestorm-1079
> > >
> > > Please vote on releasing this package as Apache Storm 2.0.0.
> > >
> > > When voting, please list the actions taken to verify the release.
> > >
> > > This vote will be open for at least 72 hours.
> > >
> > > [ ] +1 Release this package as Apache Storm 2.0.0
> > > [ ]  0 No opinion
> > > [ ] -1 Do not release this package because...
> > >
> > > Thanks to everyone who contributed to this release.
> > >
> > > -Taylor
>

Re: [VOTE] Release Apache Storm 2.0.0 RC7

[Stig Rohde Døssing]

+1

Ran tests from source zip
Installed and ran ExclamationTopology on single-node cluster from binary zip
Verified no unexpected error logs.
Clicked around a bit in UI, verified that logviewer works.
Checked out tag and ran the license check plugin to generate
DEPENDENCY-LICENSES. Verified that there are no category X licenses. A
number of dependency versions have changed, but I don't think this should
block the release. I'll see if we can automate generating the license files
a bit more, so we don't forget to update them.

Derek, I'm not sure about a missing stormconf.ser, but there are a couple
of open issues impacting tests stability. See
https://issues.apache.org/jira/browse/STORM-3376 and
https://issues.apache.org/jira/browse/STORM-3379. Maybe one of them fixes
it? I've run the integration test and localcluster tests a bunch of times
with these fixes, and the tests looks stable to me. If you get the errors
occasionally, it would be nice if you could see if they are still there
when applying the two fixes.

Den tir. 30. apr. 2019 kl. 20.16 skrev Derek Dagit :

> OK ran the distribution from the tarball, and built and ran from the src
> tarball on macOS with JDK8. Tested WordCountTopology on both and it seemed
> fine.
>
> +1
>
> On Tue, Apr 30, 2019 at 11:55 AM Derek Dagit  wrote:
>
> > Sent too soon:
> >
> > - I verified sigs and sums
> > - Built the source from the tarball on macOS
> >   - I saw a couple of test errors because the local stormconf.ser was
> gone
> > when the supervisor tried to open it, and this resulted in a crash. I
> > re-ran the tests and they passed.
> >   - I do not think this is a blocker, but it is probable worth looking
> > into.
> >
> > I'll continue to evaluate the RC, but wanted to send this out in case
> > anyone else has seen something similar.
> >
> > On Tue, Apr 30, 2019 at 11:52 AM Derek Dagit  wrote:
> >
> >> - I verified sigs and sums
> >>
> >> On Mon, Apr 29, 2019 at 5:49 PM P. Taylor Goetz 
> >> wrote:
> >>
> >>> This is a call to vote on releasing Apache Storm 2.0.0 (rc7)
> >>>
> >>> Full list of changes in this release:
> >>>
> >>>
> >>>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/RELEASE_NOTES.html
> >>>
> >>> The tag/commit to be voted upon is v2.0.0:
> >>>
> >>>
> >>>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=007863edd95e838b3df414928c6fa3f28244ab49;hb=2ba95bbd1c911d4fc6363b1c4b9c4c6d86ac9aae
> >>>
> >>> The source archive being voted upon can be found here:
> >>>
> >>>
> >>>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/apache-storm-2.0.0-src.tar.gz
> >>>
> >>> Other release files, signatures and digests can be found here:
> >>>
> >>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/
> >>>
> >>> The release artifacts are signed with the following key:
> >>>
> >>>
> >>>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >>>
> >>> The Nexus staging repository for this release is:
> >>>
> >>> https://repository.apache.org/content/repositories/orgapachestorm-1079
> >>>
> >>> Please vote on releasing this package as Apache Storm 2.0.0.
> >>>
> >>> When voting, please list the actions taken to verify the release.
> >>>
> >>> This vote will be open for at least 72 hours.
> >>>
> >>> [ ] +1 Release this package as Apache Storm 2.0.0
> >>> [ ]  0 No opinion
> >>> [ ] -1 Do not release this package because...
> >>>
> >>> Thanks to everyone who contributed to this release.
> >>>
> >>> -Taylor
> >>
> >>
>

Re: [VOTE] Release Apache Storm 2.0.0 (rc6)

[Stig Rohde Døssing]

The fix was merged, so hopefully we should be good to try again.

Den ons. 17. apr. 2019 kl. 13.28 skrev Stig Rohde Døssing <
stigdoess...@gmail.com>:

> Fix at https://github.com/apache/storm/pull/3000.
>
> Den ons. 17. apr. 2019 kl. 12.51 skrev Stig Rohde Døssing <
> stigdoess...@gmail.com>:
>
>> -1
>>
>> Upgrading to Zookeeper 3.4.14 added a transitive dependency on
>> spotbugs-annotations, which is LGPL licensed. I filed a bug for it at
>> https://issues.apache.org/jira/browse/STORM-3381, and a matching one for
>> Zookeeper https://issues.apache.org/jira/browse/ZOOKEEPER-3367.
>>
>> Sorry, should have caught this during the PR. In the longer run, I'll see
>> if we can automate license checking a bit more, it would be nice if Travis
>> failed unless the license files are up to date, so it is obvious when stuff
>> like this happens.
>>
>>
>> Den tir. 16. apr. 2019 kl. 20.42 skrev P. Taylor Goetz > >:
>>
>>> This is a call to vote on releasing Apache Storm 2.0.0 (rc6)
>>>
>>> Full list of changes in this release:
>>>
>>>
>>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc6/RELEASE_NOTES.html
>>>
>>> The tag/commit to be voted upon is v2.0.0:
>>>
>>>
>>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=fb277b90c8488aec017eed066a9fd221a247e4f9;hb=ac046ee6170d982f98b66f3ed21a82f979707d68
>>>
>>> The source archive being voted upon can be found here:
>>>
>>>
>>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc6/apache-storm-2.0.0-src.tar.gz
>>>
>>> Other release files, signatures and digests can be found here:
>>>
>>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc6/
>>>
>>> The release artifacts are signed with the following key:
>>>
>>>
>>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>>>
>>> The Nexus staging repository for this release is:
>>>
>>> https://repository.apache.org/content/repositories/orgapachestorm-1077
>>>
>>> Please vote on releasing this package as Apache Storm 2.0.0.
>>>
>>> When voting, please list the actions taken to verify the release.
>>>
>>> This vote will be open for at least 72 hours.
>>>
>>> [ ] +1 Release this package as Apache Storm 2.0.0
>>> [ ]  0 No opinion
>>> [ ] -1 Do not release this package because...
>>>
>>> Thanks to everyone who contributed to this release.
>>>
>>> -Taylor
>>
>>

Re: [VOTE] Release Apache Storm 2.0.0 (rc6)

[Stig Rohde Døssing]

Fix at https://github.com/apache/storm/pull/3000.

Den ons. 17. apr. 2019 kl. 12.51 skrev Stig Rohde Døssing <
stigdoess...@gmail.com>:

> -1
>
> Upgrading to Zookeeper 3.4.14 added a transitive dependency on
> spotbugs-annotations, which is LGPL licensed. I filed a bug for it at
> https://issues.apache.org/jira/browse/STORM-3381, and a matching one for
> Zookeeper https://issues.apache.org/jira/browse/ZOOKEEPER-3367.
>
> Sorry, should have caught this during the PR. In the longer run, I'll see
> if we can automate license checking a bit more, it would be nice if Travis
> failed unless the license files are up to date, so it is obvious when stuff
> like this happens.
>
>
> Den tir. 16. apr. 2019 kl. 20.42 skrev P. Taylor Goetz  >:
>
>> This is a call to vote on releasing Apache Storm 2.0.0 (rc6)
>>
>> Full list of changes in this release:
>>
>>
>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc6/RELEASE_NOTES.html
>>
>> The tag/commit to be voted upon is v2.0.0:
>>
>>
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=fb277b90c8488aec017eed066a9fd221a247e4f9;hb=ac046ee6170d982f98b66f3ed21a82f979707d68
>>
>> The source archive being voted upon can be found here:
>>
>>
>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc6/apache-storm-2.0.0-src.tar.gz
>>
>> Other release files, signatures and digests can be found here:
>>
>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc6/
>>
>> The release artifacts are signed with the following key:
>>
>>
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>>
>> The Nexus staging repository for this release is:
>>
>> https://repository.apache.org/content/repositories/orgapachestorm-1077
>>
>> Please vote on releasing this package as Apache Storm 2.0.0.
>>
>> When voting, please list the actions taken to verify the release.
>>
>> This vote will be open for at least 72 hours.
>>
>> [ ] +1 Release this package as Apache Storm 2.0.0
>> [ ]  0 No opinion
>> [ ] -1 Do not release this package because...
>>
>> Thanks to everyone who contributed to this release.
>>
>> -Taylor
>
>

Re: [VOTE] Release Apache Storm 2.0.0 (rc6)

[Stig Rohde Døssing]

-1

Upgrading to Zookeeper 3.4.14 added a transitive dependency on
spotbugs-annotations, which is LGPL licensed. I filed a bug for it at
https://issues.apache.org/jira/browse/STORM-3381, and a matching one for
Zookeeper https://issues.apache.org/jira/browse/ZOOKEEPER-3367.

Sorry, should have caught this during the PR. In the longer run, I'll see
if we can automate license checking a bit more, it would be nice if Travis
failed unless the license files are up to date, so it is obvious when stuff
like this happens.


Den tir. 16. apr. 2019 kl. 20.42 skrev P. Taylor Goetz :

> This is a call to vote on releasing Apache Storm 2.0.0 (rc6)
>
> Full list of changes in this release:
>
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc6/RELEASE_NOTES.html
>
> The tag/commit to be voted upon is v2.0.0:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=fb277b90c8488aec017eed066a9fd221a247e4f9;hb=ac046ee6170d982f98b66f3ed21a82f979707d68
>
> The source archive being voted upon can be found here:
>
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc6/apache-storm-2.0.0-src.tar.gz
>
> Other release files, signatures and digests can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc6/
>
> The release artifacts are signed with the following key:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>
> The Nexus staging repository for this release is:
>
> https://repository.apache.org/content/repositories/orgapachestorm-1077
>
> Please vote on releasing this package as Apache Storm 2.0.0.
>
> When voting, please list the actions taken to verify the release.
>
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 2.0.0
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>
> -Taylor

Re: Closing outdated PRs

[Stig Rohde Døssing]

Closed the mentioned PRs.

Den lør. 6. apr. 2019 kl. 16.57 skrev Stig Rohde Døssing <
stigdoess...@gmail.com>:

> I'd like to close the following PRs that have been superseded by other
> fixes, or are pulling older release branches into master :
>
> https://github.com/apache/storm/pull/1504 (old release branch into master)
> https://github.com/apache/storm/pull/2273 (old release branch into master)
> https://github.com/apache/storm/pull/2388 (fixed in STORM-2607)
> https://github.com/apache/storm/pull/2556 (fixed in STORM-3349)
> https://github.com/apache/storm/pull/2598 (targeting deleted module
> storm-kafka)
> https://github.com/apache/storm/pull/2599 (targeting deleted module
> storm-kafka)
> https://github.com/apache/storm/pull/2714 (fixed in STORM-3197)
> https://github.com/apache/storm/pull/2719 (fixed in STORM-3349)
> https://github.com/apache/storm/pull/2730 (Not sure what this is)
> https://github.com/apache/storm/pull/2764 (fixed in STORM-3197)
> https://github.com/apache/storm/pull/2773 (fixed in STORM-3252)
> https://github.com/apache/storm/pull/2800 (fixed in STORM-3162)
>
> Let me know if there are any objections to this. If not, I'll go ahead and
> close them in a week or so.
>

Re: [VOTE] Release Apache Storm 2.0.0 RC5

[Stig Rohde Døssing]

Thanks.

Den tir. 9. apr. 2019 kl. 19.21 skrev P. Taylor Goetz :

> Cool. I’ll put one together tomorrow.
>
> -Taylor
>
> > On Apr 9, 2019, at 1:00 PM, Stig Rohde Døssing 
> wrote:
> >
> > Merged the PR. I think we follow the ASF guidelines more closely now.
> >
> > I probably won't backport it to 1.x, and I'm not sure it's necessary.
> After
> > looking at a few other Apache projects, it looks like it's pretty
> variable
> > how detailed the included license information is. If someone feels that
> it
> > should be backported, they should be able to do so by following the
> README
> > instructions added in the PR.
> >
> > I think we can try another RC.
> >
> > Den man. 8. apr. 2019 kl. 22.43 skrev Stig Rohde Døssing <
> > stigdoess...@gmail.com>:
> >
> >> See the first few responses in this thread, as well as this comment
> >> https://github.com/apache/storm/pull/2980#issuecomment-480514979, I
> tried
> >> to point to the relevant ASF guidelines there. We are missing a bit of
> text
> >> in NOTICE, we aren't including enough in LICENSE for the binary
> releases,
> >> and we aren't listing dependencies with category B licenses anywhere
> users
> >> are likely to see it.
> >>
> >> Den man. 8. apr. 2019 kl. 20.59 skrev Roshan Naik
> >> :
> >>
> >>> ?
> >>>On Wednesday, March 27, 2019, 4:39:33 PM PDT, Roshan Naik
> >>>  wrote:
> >>>
> >>>  Will there be another RC or this one is good to continue with ?
> >>>
> >>>
> >>>
> >>> Sent from Yahoo Mail for iPhone
> >>>
> >>>
> >>> On Wednesday, March 27, 2019, 9:41 AM, Derek Dagit 
> >>> wrote:
> >>>
> >>> * Downloaded source ZIP, `mvn clean install`, all passed
> >>> * Verified signatures and checksums
> >>> * Packaged my build and ran a single-tenant (default) cluster
> >>> * Ran org.apache.storm.starter.WordCountTopology
> >>> * UI seemed OK, Visualization seemed OK, Logviewer seemed OK
> >>>
> >>> +1
> >>>
> >>> On Fri, Mar 22, 2019 at 3:23 PM P. Taylor Goetz 
> >>> wrote:
> >>>
> >>>> This is a call to vote on releasing Apache Storm 2.0.0 (rc5)
> >>>>
> >>>> Full list of changes in this release:
> >>>>
> >>>>
> >>>>
> >>>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/RELEASE_NOTES.html
> >>>>
> >>>> The tag/commit to be voted upon is v2.0.0:
> >>>>
> >>>>
> >>>>
> >>>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=7e0a711e4ed5315f04f9f726caff61e0f169e320;hb=b5823809bd4b438e789a36f163f318d4b161ad13
> >>>>
> >>>> The source archive being voted upon can be found here:
> >>>>
> >>>>
> >>>>
> >>>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/apache-storm-2.0.0-src.tar.gz
> >>>>
> >>>> Other release files, signatures and digests can be found here:
> >>>>
> >>>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/
> >>>>
> >>>> The release artifacts are signed with the following key:
> >>>>
> >>>>
> >>>>
> >>>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >>>>
> >>>> The Nexus staging repository for this release is:
> >>>>
> >>>>
> https://repository.apache.org/content/repositories/orgapachestorm-1076
> >>>>
> >>>> Please vote on releasing this package as Apache Storm 2.0.0.
> >>>>
> >>>> When voting, please list the actions taken to verify the release.
> >>>>
> >>>> This vote will be open for at least 72 hours.
> >>>>
> >>>> [ ] +1 Release this package as Apache Storm 2.0.0
> >>>> [ ]  0 No opinion
> >>>> [ ] -1 Do not release this package because...
> >>>>
> >>>> Thanks to everyone who contributed to this release.
> >>>>
> >>>> -Taylor
> >>>
> >>>
> >>>
> >>
> >>
>
>

Re: [VOTE] Release Apache Storm 2.0.0 RC5

[Stig Rohde Døssing]

Merged the PR. I think we follow the ASF guidelines more closely now.

I probably won't backport it to 1.x, and I'm not sure it's necessary. After
looking at a few other Apache projects, it looks like it's pretty variable
how detailed the included license information is. If someone feels that it
should be backported, they should be able to do so by following the README
instructions added in the PR.

I think we can try another RC.

Den man. 8. apr. 2019 kl. 22.43 skrev Stig Rohde Døssing <
stigdoess...@gmail.com>:

> See the first few responses in this thread, as well as this comment
> https://github.com/apache/storm/pull/2980#issuecomment-480514979, I tried
> to point to the relevant ASF guidelines there. We are missing a bit of text
> in NOTICE, we aren't including enough in LICENSE for the binary releases,
> and we aren't listing dependencies with category B licenses anywhere users
> are likely to see it.
>
> Den man. 8. apr. 2019 kl. 20.59 skrev Roshan Naik
> :
>
>>  ?
>> On Wednesday, March 27, 2019, 4:39:33 PM PDT, Roshan Naik
>>  wrote:
>>
>>   Will there be another RC or this one is good to continue with ?
>>
>>
>>
>> Sent from Yahoo Mail for iPhone
>>
>>
>> On Wednesday, March 27, 2019, 9:41 AM, Derek Dagit 
>> wrote:
>>
>> * Downloaded source ZIP, `mvn clean install`, all passed
>> * Verified signatures and checksums
>> * Packaged my build and ran a single-tenant (default) cluster
>> * Ran org.apache.storm.starter.WordCountTopology
>> * UI seemed OK, Visualization seemed OK, Logviewer seemed OK
>>
>> +1
>>
>> On Fri, Mar 22, 2019 at 3:23 PM P. Taylor Goetz 
>> wrote:
>>
>> > This is a call to vote on releasing Apache Storm 2.0.0 (rc5)
>> >
>> > Full list of changes in this release:
>> >
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/RELEASE_NOTES.html
>> >
>> > The tag/commit to be voted upon is v2.0.0:
>> >
>> >
>> >
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=7e0a711e4ed5315f04f9f726caff61e0f169e320;hb=b5823809bd4b438e789a36f163f318d4b161ad13
>> >
>> > The source archive being voted upon can be found here:
>> >
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/apache-storm-2.0.0-src.tar.gz
>> >
>> > Other release files, signatures and digests can be found here:
>> >
>> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/
>> >
>> > The release artifacts are signed with the following key:
>> >
>> >
>> >
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>> >
>> > The Nexus staging repository for this release is:
>> >
>> > https://repository.apache.org/content/repositories/orgapachestorm-1076
>> >
>> > Please vote on releasing this package as Apache Storm 2.0.0.
>> >
>> > When voting, please list the actions taken to verify the release.
>> >
>> > This vote will be open for at least 72 hours.
>> >
>> > [ ] +1 Release this package as Apache Storm 2.0.0
>> > [ ]  0 No opinion
>> > [ ] -1 Do not release this package because...
>> >
>> > Thanks to everyone who contributed to this release.
>> >
>> > -Taylor
>>
>>
>>
>
>

Re: [VOTE] Release Apache Storm 2.0.0 RC5

[Stig Rohde Døssing]

See the first few responses in this thread, as well as this comment
https://github.com/apache/storm/pull/2980#issuecomment-480514979, I tried
to point to the relevant ASF guidelines there. We are missing a bit of text
in NOTICE, we aren't including enough in LICENSE for the binary releases,
and we aren't listing dependencies with category B licenses anywhere users
are likely to see it.

Den man. 8. apr. 2019 kl. 20.59 skrev Roshan Naik
:

>  ?
> On Wednesday, March 27, 2019, 4:39:33 PM PDT, Roshan Naik
>  wrote:
>
>   Will there be another RC or this one is good to continue with ?
>
>
>
> Sent from Yahoo Mail for iPhone
>
>
> On Wednesday, March 27, 2019, 9:41 AM, Derek Dagit 
> wrote:
>
> * Downloaded source ZIP, `mvn clean install`, all passed
> * Verified signatures and checksums
> * Packaged my build and ran a single-tenant (default) cluster
> * Ran org.apache.storm.starter.WordCountTopology
> * UI seemed OK, Visualization seemed OK, Logviewer seemed OK
>
> +1
>
> On Fri, Mar 22, 2019 at 3:23 PM P. Taylor Goetz  wrote:
>
> > This is a call to vote on releasing Apache Storm 2.0.0 (rc5)
> >
> > Full list of changes in this release:
> >
> >
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/RELEASE_NOTES.html
> >
> > The tag/commit to be voted upon is v2.0.0:
> >
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=7e0a711e4ed5315f04f9f726caff61e0f169e320;hb=b5823809bd4b438e789a36f163f318d4b161ad13
> >
> > The source archive being voted upon can be found here:
> >
> >
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/apache-storm-2.0.0-src.tar.gz
> >
> > Other release files, signatures and digests can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/
> >
> > The release artifacts are signed with the following key:
> >
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >
> > The Nexus staging repository for this release is:
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1076
> >
> > Please vote on releasing this package as Apache Storm 2.0.0.
> >
> > When voting, please list the actions taken to verify the release.
> >
> > This vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache Storm 2.0.0
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > -Taylor
>
>
>

Re: [VOTE] Release Apache Storm 2.0.0 RC5

[Stig Rohde Døssing]

I think this RC is dead due to potential license issues. If someone gets a
chance, please take a look at https://github.com/apache/storm/pull/2980.

Den tor. 28. mar. 2019 kl. 00.39 skrev Roshan Naik
:

>  Will there be another RC or this one is good to continue with ?
>
>
>
> Sent from Yahoo Mail for iPhone
>
>
> On Wednesday, March 27, 2019, 9:41 AM, Derek Dagit 
> wrote:
>
> * Downloaded source ZIP, `mvn clean install`, all passed
> * Verified signatures and checksums
> * Packaged my build and ran a single-tenant (default) cluster
> * Ran org.apache.storm.starter.WordCountTopology
> * UI seemed OK, Visualization seemed OK, Logviewer seemed OK
>
> +1
>
> On Fri, Mar 22, 2019 at 3:23 PM P. Taylor Goetz  wrote:
>
> > This is a call to vote on releasing Apache Storm 2.0.0 (rc5)
> >
> > Full list of changes in this release:
> >
> >
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/RELEASE_NOTES.html
> >
> > The tag/commit to be voted upon is v2.0.0:
> >
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=7e0a711e4ed5315f04f9f726caff61e0f169e320;hb=b5823809bd4b438e789a36f163f318d4b161ad13
> >
> > The source archive being voted upon can be found here:
> >
> >
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/apache-storm-2.0.0-src.tar.gz
> >
> > Other release files, signatures and digests can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/
> >
> > The release artifacts are signed with the following key:
> >
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >
> > The Nexus staging repository for this release is:
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1076
> >
> > Please vote on releasing this package as Apache Storm 2.0.0.
> >
> > When voting, please list the actions taken to verify the release.
> >
> > This vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache Storm 2.0.0
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > -Taylor
>
>
>
>

Closing outdated PRs

[Stig Rohde Døssing]

I'd like to close the following PRs that have been superseded by other
fixes, or are pulling older release branches into master :

https://github.com/apache/storm/pull/1504 (old release branch into master)
https://github.com/apache/storm/pull/2273 (old release branch into master)
https://github.com/apache/storm/pull/2388 (fixed in STORM-2607)
https://github.com/apache/storm/pull/2556 (fixed in STORM-3349)
https://github.com/apache/storm/pull/2598 (targeting deleted module
storm-kafka)
https://github.com/apache/storm/pull/2599 (targeting deleted module
storm-kafka)
https://github.com/apache/storm/pull/2714 (fixed in STORM-3197)
https://github.com/apache/storm/pull/2719 (fixed in STORM-3349)
https://github.com/apache/storm/pull/2730 (Not sure what this is)
https://github.com/apache/storm/pull/2764 (fixed in STORM-3197)
https://github.com/apache/storm/pull/2773 (fixed in STORM-3252)
https://github.com/apache/storm/pull/2800 (fixed in STORM-3162)

Let me know if there are any objections to this. If not, I'll go ahead and
close them in a week or so.

Re: [VOTE] Release Apache Storm 2.0.0 RC5

[Stig Rohde Døssing]

Thanks for clarifying Taylor.

If I understand it correctly, the L & N for source should only contain
licenses for source we include. So the L & N files we have now are probably
fine for this, as they contain licensing for the Javascript we have in UI,
and I don't believe we include other kinds of external source in the source
dist. I'll do a quick check that all the JS we have is covered.

For the binary distribution, we should have a L & N that is the L & N for
the source, plus any licenses for binaries we include (e.g. everything in
the /lib directory). We should update the L & N in storm-dist/binary, and
I'd also like to move them to the project root, e.g. as "LICENSE-binary". I
think they're a little well hidden right now.

I get the impression that the binary license should not include licenses
for e.g. dependencies downloaded when a topology uses `storm-hbase`, as
those binaries aren't part of the distribution. Is this right? Do those
kinds of dependencies still need to have their licenses listed somewhere
(e.g. in the autogenerated dependency-licenses file), or is this not
necessary?

Den tir. 26. mar. 2019 kl. 00.45 skrev P. Taylor Goetz :

> L & N files usually differ between source and binary distributions.
> Usually due to shading, contents of lib directory, etc. Source
> distributions are simpler, since they can’t contain any binaries. For a
> binary distribution, the L & N files need to reflect everything in the
> binary dependencies.
>
> I think it’s important that we review licensing issues, so I’ll cancel
> this RC.
>
> Kudos to Stig for paying attention to license hygiene!
>
> -Taylor
>
> > On Mar 25, 2019, at 1:21 PM, Stig Rohde Døssing 
> wrote:
> >
> > Maybe something like this? https://github.com/apache/storm/pull/2980
> >
> > Den man. 25. mar. 2019 kl. 14.12 skrev Stig Rohde Døssing <
> > stigdoess...@gmail.com>:
> >
> >> The ASF guideline says the file "should identify the third-party
> product,
> >> its licensing, and a url to the its homepage". If we can get away with
> >> including the license name and not the license text, I think
> >> THIRD-PARTY.txt contains what we need. E.g. Spark also only lists the
> >> license names, and not the texts.
> >>
> >> We can clean up the output a bit more, e.g. collapse all the Apache
> >> licenses together under one header, I just didn't bother because I
> didn't
> >> expect the file to be user facing.
> >>
> >> Den man. 25. mar. 2019 kl. 13.47 skrev Jungtaek Lim  >:
> >>
> >>> According to how other projects are doing right now, looks like we are
> >>> not doing.
> >>>
> >>> https://github.com/apache/flink/blob/master/NOTICE-binary
> >>> https://github.com/apache/spark/blob/master/LICENSE-binary
> >>> https://github.com/apache/kafka/blob/trunk/LICENSE
> >>>
> >>> If I understand correct, aether in storm-submit-tool is also EPL v1.0.
> >>>
> >>> Btw, I just ran the command "mvn generate-resources
> >>> -Dlicense.skipAggregateAddThirdParty=false" and see the output: the
> format
> >>> of output looks good though the output is a bit verbose. (Attached the
> >>> output file.)
> >>>
> >>> Would it be OK to include the file without cleaning up?
> >>>
> >>> 2019년 3월 25일 (월) 오후 8:19, Stig Rohde Døssing  >님이
> >>> 작성:
> >>>
> >>>> 0
> >>>>
> >>>> Built and ran tests from source zip.
> >>>> Ran ExclamationTopology on local install set up from binary zip.
> >>>> Verified no unexpected error logs.
> >>>> Ran integration test locally.
> >>>> Clicked around in UI for a bit, checked that logviewer works.
> >>>> Ran the license check plugin, and verified that all dependency
> licenses
> >>>> are
> >>>> either category A or B.
> >>>>
> >>>> We have some category B dependencies, e.g. JAXB and Jersey under CDDL.
> >>>>
> https://apache.org/legal/resolved.html#appropriately-labelled-condition
> >>>> mentions that we should list category B dependencies somewhere
> visible to
> >>>> users. Do we do this currently?
> >>>>
> >>>> Den fre. 22. mar. 2019 kl. 21.23 skrev P. Taylor Goetz <
> >>>> ptgo...@gmail.com>:
> >>>>
> >>>>> This is a call to vote on releasing Apache Storm 2.0.0 (rc5)
> >>>>>
> >>>>> Full list of changes in th

Re: [VOTE] Release Apache Storm 2.0.0 RC5

[Stig Rohde Døssing]

Maybe something like this? https://github.com/apache/storm/pull/2980

Den man. 25. mar. 2019 kl. 14.12 skrev Stig Rohde Døssing <
stigdoess...@gmail.com>:

> The ASF guideline says the file "should identify the third-party product,
> its licensing, and a url to the its homepage". If we can get away with
> including the license name and not the license text, I think
> THIRD-PARTY.txt contains what we need. E.g. Spark also only lists the
> license names, and not the texts.
>
> We can clean up the output a bit more, e.g. collapse all the Apache
> licenses together under one header, I just didn't bother because I didn't
> expect the file to be user facing.
>
> Den man. 25. mar. 2019 kl. 13.47 skrev Jungtaek Lim :
>
>> According to how other projects are doing right now, looks like we are
>> not doing.
>>
>> https://github.com/apache/flink/blob/master/NOTICE-binary
>> https://github.com/apache/spark/blob/master/LICENSE-binary
>> https://github.com/apache/kafka/blob/trunk/LICENSE
>>
>> If I understand correct, aether in storm-submit-tool is also EPL v1.0.
>>
>> Btw, I just ran the command "mvn generate-resources
>> -Dlicense.skipAggregateAddThirdParty=false" and see the output: the format
>> of output looks good though the output is a bit verbose. (Attached the
>> output file.)
>>
>> Would it be OK to include the file without cleaning up?
>>
>> 2019년 3월 25일 (월) 오후 8:19, Stig Rohde Døssing 님이
>> 작성:
>>
>>> 0
>>>
>>> Built and ran tests from source zip.
>>> Ran ExclamationTopology on local install set up from binary zip.
>>> Verified no unexpected error logs.
>>> Ran integration test locally.
>>> Clicked around in UI for a bit, checked that logviewer works.
>>> Ran the license check plugin, and verified that all dependency licenses
>>> are
>>> either category A or B.
>>>
>>> We have some category B dependencies, e.g. JAXB and Jersey under CDDL.
>>> https://apache.org/legal/resolved.html#appropriately-labelled-condition
>>> mentions that we should list category B dependencies somewhere visible to
>>> users. Do we do this currently?
>>>
>>> Den fre. 22. mar. 2019 kl. 21.23 skrev P. Taylor Goetz <
>>> ptgo...@gmail.com>:
>>>
>>> > This is a call to vote on releasing Apache Storm 2.0.0 (rc5)
>>> >
>>> > Full list of changes in this release:
>>> >
>>> >
>>> >
>>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/RELEASE_NOTES.html
>>> >
>>> > The tag/commit to be voted upon is v2.0.0:
>>> >
>>> >
>>> >
>>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=7e0a711e4ed5315f04f9f726caff61e0f169e320;hb=b5823809bd4b438e789a36f163f318d4b161ad13
>>> >
>>> > The source archive being voted upon can be found here:
>>> >
>>> >
>>> >
>>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/apache-storm-2.0.0-src.tar.gz
>>> >
>>> > Other release files, signatures and digests can be found here:
>>> >
>>> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/
>>> >
>>> > The release artifacts are signed with the following key:
>>> >
>>> >
>>> >
>>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>>> >
>>> > The Nexus staging repository for this release is:
>>> >
>>> > https://repository.apache.org/content/repositories/orgapachestorm-1076
>>> >
>>> > Please vote on releasing this package as Apache Storm 2.0.0.
>>> >
>>> > When voting, please list the actions taken to verify the release.
>>> >
>>> > This vote will be open for at least 72 hours.
>>> >
>>> > [ ] +1 Release this package as Apache Storm 2.0.0
>>> > [ ]  0 No opinion
>>> > [ ] -1 Do not release this package because...
>>> >
>>> > Thanks to everyone who contributed to this release.
>>> >
>>> > -Taylor
>>>
>>

Re: [VOTE] Release Apache Storm 2.0.0 RC5

[Stig Rohde Døssing]

The ASF guideline says the file "should identify the third-party product,
its licensing, and a url to the its homepage". If we can get away with
including the license name and not the license text, I think
THIRD-PARTY.txt contains what we need. E.g. Spark also only lists the
license names, and not the texts.

We can clean up the output a bit more, e.g. collapse all the Apache
licenses together under one header, I just didn't bother because I didn't
expect the file to be user facing.

Den man. 25. mar. 2019 kl. 13.47 skrev Jungtaek Lim :

> According to how other projects are doing right now, looks like we are not
> doing.
>
> https://github.com/apache/flink/blob/master/NOTICE-binary
> https://github.com/apache/spark/blob/master/LICENSE-binary
> https://github.com/apache/kafka/blob/trunk/LICENSE
>
> If I understand correct, aether in storm-submit-tool is also EPL v1.0.
>
> Btw, I just ran the command "mvn generate-resources
> -Dlicense.skipAggregateAddThirdParty=false" and see the output: the format
> of output looks good though the output is a bit verbose. (Attached the
> output file.)
>
> Would it be OK to include the file without cleaning up?
>
> 2019년 3월 25일 (월) 오후 8:19, Stig Rohde Døssing 님이
> 작성:
>
>> 0
>>
>> Built and ran tests from source zip.
>> Ran ExclamationTopology on local install set up from binary zip.
>> Verified no unexpected error logs.
>> Ran integration test locally.
>> Clicked around in UI for a bit, checked that logviewer works.
>> Ran the license check plugin, and verified that all dependency licenses
>> are
>> either category A or B.
>>
>> We have some category B dependencies, e.g. JAXB and Jersey under CDDL.
>> https://apache.org/legal/resolved.html#appropriately-labelled-condition
>> mentions that we should list category B dependencies somewhere visible to
>> users. Do we do this currently?
>>
>> Den fre. 22. mar. 2019 kl. 21.23 skrev P. Taylor Goetz > >:
>>
>> > This is a call to vote on releasing Apache Storm 2.0.0 (rc5)
>> >
>> > Full list of changes in this release:
>> >
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/RELEASE_NOTES.html
>> >
>> > The tag/commit to be voted upon is v2.0.0:
>> >
>> >
>> >
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=7e0a711e4ed5315f04f9f726caff61e0f169e320;hb=b5823809bd4b438e789a36f163f318d4b161ad13
>> >
>> > The source archive being voted upon can be found here:
>> >
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/apache-storm-2.0.0-src.tar.gz
>> >
>> > Other release files, signatures and digests can be found here:
>> >
>> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/
>> >
>> > The release artifacts are signed with the following key:
>> >
>> >
>> >
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>> >
>> > The Nexus staging repository for this release is:
>> >
>> > https://repository.apache.org/content/repositories/orgapachestorm-1076
>> >
>> > Please vote on releasing this package as Apache Storm 2.0.0.
>> >
>> > When voting, please list the actions taken to verify the release.
>> >
>> > This vote will be open for at least 72 hours.
>> >
>> > [ ] +1 Release this package as Apache Storm 2.0.0
>> > [ ]  0 No opinion
>> > [ ] -1 Do not release this package because...
>> >
>> > Thanks to everyone who contributed to this release.
>> >
>> > -Taylor
>>
>

Re: [VOTE] Release Apache Storm 2.0.0 RC5

[Stig Rohde Døssing]

0

Built and ran tests from source zip.
Ran ExclamationTopology on local install set up from binary zip.
Verified no unexpected error logs.
Ran integration test locally.
Clicked around in UI for a bit, checked that logviewer works.
Ran the license check plugin, and verified that all dependency licenses are
either category A or B.

We have some category B dependencies, e.g. JAXB and Jersey under CDDL.
https://apache.org/legal/resolved.html#appropriately-labelled-condition
mentions that we should list category B dependencies somewhere visible to
users. Do we do this currently?

Den fre. 22. mar. 2019 kl. 21.23 skrev P. Taylor Goetz :

> This is a call to vote on releasing Apache Storm 2.0.0 (rc5)
>
> Full list of changes in this release:
>
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/RELEASE_NOTES.html
>
> The tag/commit to be voted upon is v2.0.0:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=7e0a711e4ed5315f04f9f726caff61e0f169e320;hb=b5823809bd4b438e789a36f163f318d4b161ad13
>
> The source archive being voted upon can be found here:
>
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/apache-storm-2.0.0-src.tar.gz
>
> Other release files, signatures and digests can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc5/
>
> The release artifacts are signed with the following key:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>
> The Nexus staging repository for this release is:
>
> https://repository.apache.org/content/repositories/orgapachestorm-1076
>
> Please vote on releasing this package as Apache Storm 2.0.0.
>
> When voting, please list the actions taken to verify the release.
>
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 2.0.0
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>
> -Taylor

Re: Storm 2.0 blogs ?

[Stig Rohde Døssing]

We should probably also highlight in the release notes that Java 8 is now
the minimum.

Here is the blurb for storm-kafka-client:

# Kafka integration changes

## Removal of storm-kafka
The most significant change to Storm's Kafka integration since 1.x, is that
storm-kafka has been removed. The module was deprecated a while back, due
to Kafka's deprecation of the underlying client library. Users will have to
move to the storm-kafka-client module, which uses Kafka's ´kafka-clients´
library for integration.

For the most part, the migration to storm-kafka-client is straightforward.
The documentation for storm-kafka-client contains a helpful mapping between
the old and new spout configurations. If you are using any of the
storm-kafka spouts, you will need to migrate offset checkpoints to the new
spout, to avoid the new spout starting from scratch on your partitions. You
can find a helper tool to do this at
https://github.com/apache/storm/tree/master/external/storm-kafka-migration.
You should stop your topology, run the migration tool, then redeploy your
topology with the storm-kafka-client spout.

## Move to using the KafkaConsumer.assign API
Storm-kafka-client in 1.x allowed you to use Kafka's own mechanism to
manage which spout tasks were responsible for which partitions. This
mechanism was a poor fit for Storm, and was deprecated in 1.2.0. It has
been removed entirely in 2.0
https://issues.apache.org/jira/browse/STORM-2542.

The storm-kafka-client Subscription interface has also been removed. It
offered too limited control over the subscription behavior. It has been
replaced with the TopicFilter and ManualPartitioner interfaces. Unless you
were using a custom Subscription implementation, this will likely not
affect you. If you were using a custom Subscription, the storm-kafka-client
documentation describes how to customize assignment
https://github.com/apache/storm/blob/master/docs/storm-kafka-client.md#manual-partition-assigment-advanced
.

## Other highlights
* The KafkaBolt now allows you to specify a callback that will be called
when a batch is written to Kafka
https://issues.apache.org/jira/browse/STORM-3175.
* The FirstPollOffsetStrategy behavior has been made consistent between the
non-Trident and Trident spouts. It is now always the case that
EARLIEST/LATEST only take effect on topology redeploy, and not when a
worker restarts https://issues.apache.org/jira/browse/STORM-2990.
* Storm-kafka-client now has a transactional non-opaque Trident spout
https://issues.apache.org/jira/browse/STORM-2974.
* There is a new examples module for storm-kafka-client at
https://github.com/apache/storm/tree/master/examples/storm-kafka-client-examples
.
* Deprecated methods in KafkaSpoutConfig have been removed. If you are
using one of the deprecated methods, check the Javadoc for the latest 1.2.x
release, which describes the replacement for each method.

Den ons. 23. jan. 2019 kl. 15.54 skrev P. Taylor Goetz :

> If you need to format (e.g. code examples, etc.) then markdown is fine. So
> is plain text.
>
> -Taylor
>
> > On Jan 23, 2019, at 5:24 AM, Stig Rohde Døssing 
> wrote:
> >
> > I'll write something for 5 - Kafka related changes.
> >
> > We have dropped Druid support, so 13 should be only Kinesis.
> >
> > Which format should the blurbs be written in? (Markdown?)
> >
> > Den ons. 23. jan. 2019 kl. 08.57 skrev Roshan Naik
> > :
> >
> >> Like Taylor’s suggestion of collectively contributing small blurbs on
> >> features for the Release announcement. Thats the first thing people
> look on
> >> hearing a release announcement. The jira list is not very
> understandable at
> >> first glance.
> >>
> >>
> >> Based on suggestions so far and a quick scan of the jiras in release
> notes
> >> here is a draft list in no particular order. I am sure I am missing a
> few
> >> impt ones. This can be pruned or modified as needed:
> >>
> >> 1- Re-architecture - [Roshan]
> >> 2- Windowing enhancements
> >> 3- SQL enhancements
> >> 4- Metrics
> >> 5- Kafka related changes
> >> 6- Security (nimbus admin groups, delegation tokens, optional
> >> impersonation)
> >> 7- PMML (Machine Learning) support.
> >> 8- Streams API
> >> 9- Module restructuring & dependency mitigation
> >> 10- Java porting
> >> 11- DRPC cmd line
> >> 12- Lambda support
> >> 13- New spouts: Kinesis & Druid ?
> >> 14- Changes to deployment and cli submission
> >> 15- RAS changes
> >> 16- Trident enhancements
> >> 17- New Admin cmds to debug cluster state
> >> 18 ... others ?
> >>
> >> Please pick the topics you can contribute blurbs for. I have put my name
> >> against one. It will help Taylor aggregate them and do the necessary
> final
> >> edits.
> >>
> >>
> >> -Roshan
> >>
> >>
> >>
> >>
> >>
>
>

Re: Storm 2.0 blogs ?

[Stig Rohde Døssing]

I'll write something for 5 - Kafka related changes.

We have dropped Druid support, so 13 should be only Kinesis.

Which format should the blurbs be written in? (Markdown?)

Den ons. 23. jan. 2019 kl. 08.57 skrev Roshan Naik
:

>  Like Taylor’s suggestion of collectively contributing small blurbs on
> features for the Release announcement. Thats the first thing people look on
> hearing a release announcement. The jira list is not very understandable at
> first glance.
>
>
> Based on suggestions so far and a quick scan of the jiras in release notes
> here is a draft list in no particular order. I am sure I am missing a few
> impt ones. This can be pruned or modified as needed:
>
> 1- Re-architecture - [Roshan]
> 2- Windowing enhancements
> 3- SQL enhancements
> 4- Metrics
> 5- Kafka related changes
> 6- Security (nimbus admin groups, delegation tokens, optional
> impersonation)
> 7- PMML (Machine Learning) support.
> 8- Streams API
> 9- Module restructuring & dependency mitigation
> 10- Java porting
> 11- DRPC cmd line
> 12- Lambda support
> 13- New spouts: Kinesis & Druid ?
> 14- Changes to deployment and cli submission
> 15- RAS changes
> 16- Trident enhancements
> 17- New Admin cmds to debug cluster state
> 18 ... others ?
>
> Please pick the topics you can contribute blurbs for. I have put my name
> against one. It will help Taylor aggregate them and do the necessary final
> edits.
>
>
> -Roshan
>
>
>
>
>

Re: [VOTE] Migration of git repos to gitbox.apache.org

[Stig Rohde Døssing]

+1

Den fre. 18. jan. 2019 kl. 16.59 skrev Kishorkumar Patil
:

> +1
>
> On Fri, Jan 18, 2019 at 10:47 AM Ethan Li 
> wrote:
>
> > This is a call to vote on migration of git repos to gitbox.apache.org <
> > http://gitbox.apache.org/>.
> >
> > The following repositories on git-wip-us belong to our project:
> > - storm-site.git
> > - storm.git
> >
> > All repositories not migrated on February 7th will be mass migrated
> > without warning. It’s better to work with the infrastructure team to
> avoid
> > the mass that day.
> >
> > The notice email is at
> >
> https://lists.apache.org/thread.html/2bb953946ee6652a054329df2814f512f9186e863ec841555f9cc26a@%3Cdev.storm.apache.org%3E
> > <
> >
> https://lists.apache.org/thread.html/2bb953946ee6652a054329df2814f512f9186e863ec841555f9cc26a@%3Cdev.storm.apache.org%3E
> > >
> >
> > This vote will be open for at least 24 hours.
> >
> > Thanks
> >
> > - Ethan
>
> --
> -Kishor
>

Re: [NOTICE] Mandatory migration of git repos to gitbox.apache.org - three weeks left!

[Stig Rohde Døssing]

+1. As far as I know we don't have anything in our setup we need to change
manually?

Den tir. 15. jan. 2019 kl. 17.35 skrev Ethan Li :

> Hi team,
>
> Should we go ahead to have our repositories moved?
>
> - Ethan
>
>
> > On Jan 15, 2019, at 1:50 AM, Apache Infrastructure Team <
> infrastruct...@apache.org> wrote:
> >
> > Hello, storm folks.
> > As stated earlier in 2018, and reiterated two weeks ago, all git
> > repositories must be migrated from the git-wip-us.apache.org URL to
> > gitbox.apache.org, as the old service is being decommissioned. Your
> > project is receiving this email because you still have repositories on
> > git-wip-us that needs to be migrated.
> >
> > The following repositories on git-wip-us belong to your project:
> > - storm-site.git
> > - storm.git
> >
> >
> > We are now entering the remaining three weeks of the mandated
> > (coordinated) move stage of the roadmap, and you are asked to please
> > coordinate migration with the Apache Infrastructure Team before February
> > 7th. All repositories not migrated on February 7th will be mass migrated
> > without warning, and we'd appreciate it if we could work together to
> > avoid a big mess that day :-).
> >
> > As stated earlier, moving to gitbox means you will get full write access
> > on GitHub as well, and be able to close/merge pull requests and much
> > more. The move is mandatory for all Apache projects using git.
> >
> > To have your repositories moved, please follow these steps:
> >
> > - Ensure consensus on the move (a link to a lists.apache.org thread will
> >  suffice for us as evidence).
> > - Create a JIRA ticket at https://issues.apache.org/jira/browse/INFRA
> >
> > Your migration should only take a few minutes. If you wish to migrate
> > at a specific time of day or date, please do let us know in the ticket,
> > otherwise we will migrate at the earliest convenient time.
> >
> > There will be redirects in place from git-wip to gitbox, so requests
> > using the old remote origins should still work (however we encourage
> > people to update their remotes once migration has completed).
> >
> > As always, we appreciate your understanding and patience as we move
> > things around and work to provide better services and features for
> > the Apache Family.
> >
> > Should you wish to contact us with feedback or questions, please do so
> > at: us...@infra.apache.org.
> >
> >
> > With regards,
> > Apache Infrastructure
> >
>
>

Re: Me going forward

[Stig Rohde Døssing]

Congratulations on the new job.

Den tir. 15. jan. 2019 kl. 14.29 skrev Bobby Evans :

> I wanted to give everyone a heads up that I have taken a new job at Nvidia
> and my day to day work, though still a bit up in the air, is not likely to
> concentrate on Storm as much as it did before.  I am still around and will
> still try to participate as much as I can, but you may need to explicitly
> call me out if you want me to look at something.
>
> Thanks,
>
> Bobby
>

Re: [VOTE] Release Apache Storm 2.0.0 (rc4)

[Stig Rohde Døssing]

Thanks Arun,

I don't have an opinion on it, just wanted to make sure that it wasn't an
oversight.

+1

Set up Storm from the binary zip file
Built storm-kafka-client-examples from the zip while pointing Maven at the
Nexus repo
Ran one of the example Kafka topologies with a single-node cluster against
a local Kafka. Verified there were no errors in the logs, clicked around in
UI.

Den tor. 10. jan. 2019 kl. 22.25 skrev Arun Mahadevan :

> This is for users to use the "auto credentials" mechanism (delegation
> tokens) with HDFS/Hive/Hbase.
>
> We have been shipping it since 1.x (I think since 1.2.0 release) so that
> users can just add that directory to class path rather than building it
> separately to get the right dependencies. We could consider removing it
> from the main binary and ship it separately but it will need changes to the
> build, release and documentation and users will need to download and
> install it separately.
>
>
> Thanks,
> Arun
>
> On Thu, 10 Jan 2019 at 10:28, Stig Rohde Døssing 
> wrote:
>
> > I think this was remarked on by Roshan in the last RC, but the binary
> > distribution has become significantly larger since 1.x. It looks like
> this
> > is down to storm-autocreds not being added to the exclusion list in
> > storm-dist/binary/final-package/src/main/assembly/binary.xml.
> >
> > Since the module isn't excluded, external/storm-autocreds contains the
> > module jar, plus all dependency jars. Is this an accident, or do we want
> to
> > include these jars in the distribution?
> >
> > Den ons. 9. jan. 2019 kl. 19.48 skrev Ethan Li <
> ethanopensou...@gmail.com
> > >:
> >
> > > +1
> > >
> > > - Built from the src, ran all the unit tests and integration tests.
> > > - Set up a single-node cluster and submit ThroughputVsLatency topology.
> > > - Checked the UI.
> > > They look good.
> > >
> > > Thanks
> > > Ethan
> > >
> > > > On Jan 9, 2019, at 8:48 AM, Bobby Evans  wrote:
> > > >
> > > > +1 built from the git tag.  Ran all of the unit tests and ran some
> > manual
> > > > tests they all passed.
> > > >
> > > > Thanks,
> > > >
> > > > Bobby
> > > >
> > > > On Tue, Jan 8, 2019 at 6:30 PM Xin Wang 
> > wrote:
> > > >
> > > >> +1
> > > >>
> > > >> Built it and ran all of the tests.  Everything passed.
> > > >>
> > > >> -Xin
> > > >>
> > > >> Kishorkumar Patil  于2019年1月9日周三 上午5:08写道：
> > > >>
> > > >>> +1
> > > >>>
> > > >>> - built from source code and deployment works.
> > > >>> -  Ran some of the tests for UI, DRPC, ThroughputVsLatency
> > > >>> -  Validated UI bugs reported in the recent past are fixed in this
> > > >> version
> > > >>>
> > > >>> -Kishor
> > > >>>
> > > >>>
> > > >>> On Tue, Jan 8, 2019 at 2:29 PM Arun Mahadevan 
> > > wrote:
> > > >>>
> > > >>>> +1
> > > >>>>
> > > >>>> - Downloaded the binaries and validated signatures.
> > > >>>> - Deployed the binaries, ran some sample topologies and checked
> the
> > > UI.
> > > >>>> - Ran top level build using the source zip.
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Arun
> > > >>>>
> > > >>>>
> > > >>>> On Tue, 8 Jan 2019 at 11:03, P. Taylor Goetz 
> > > >> wrote:
> > > >>>>
> > > >>>>> This is a call to vote on releasing Apache Storm 2.0.0 (rc4)
> > > >>>>>
> > > >>>>> Full list of changes in this release:
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc4/RELEASE_NOTES.html
> > > >>>>>
> > > >>>>> The tag/commit to be voted upon is v2.0.0:
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> >
> https://git-wip-us.apache.

Re: [VOTE] Release Apache Storm 2.0.0 (rc4)

[Stig Rohde Døssing]

I think this was remarked on by Roshan in the last RC, but the binary
distribution has become significantly larger since 1.x. It looks like this
is down to storm-autocreds not being added to the exclusion list in
storm-dist/binary/final-package/src/main/assembly/binary.xml.

Since the module isn't excluded, external/storm-autocreds contains the
module jar, plus all dependency jars. Is this an accident, or do we want to
include these jars in the distribution?

Den ons. 9. jan. 2019 kl. 19.48 skrev Ethan Li :

> +1
>
> - Built from the src, ran all the unit tests and integration tests.
> - Set up a single-node cluster and submit ThroughputVsLatency topology.
> - Checked the UI.
> They look good.
>
> Thanks
> Ethan
>
> > On Jan 9, 2019, at 8:48 AM, Bobby Evans  wrote:
> >
> > +1 built from the git tag.  Ran all of the unit tests and ran some manual
> > tests they all passed.
> >
> > Thanks,
> >
> > Bobby
> >
> > On Tue, Jan 8, 2019 at 6:30 PM Xin Wang  wrote:
> >
> >> +1
> >>
> >> Built it and ran all of the tests.  Everything passed.
> >>
> >> -Xin
> >>
> >> Kishorkumar Patil  于2019年1月9日周三 上午5:08写道：
> >>
> >>> +1
> >>>
> >>> - built from source code and deployment works.
> >>> -  Ran some of the tests for UI, DRPC, ThroughputVsLatency
> >>> -  Validated UI bugs reported in the recent past are fixed in this
> >> version
> >>>
> >>> -Kishor
> >>>
> >>>
> >>> On Tue, Jan 8, 2019 at 2:29 PM Arun Mahadevan 
> wrote:
> >>>
>  +1
> 
>  - Downloaded the binaries and validated signatures.
>  - Deployed the binaries, ran some sample topologies and checked the
> UI.
>  - Ran top level build using the source zip.
> 
>  Thanks,
>  Arun
> 
> 
>  On Tue, 8 Jan 2019 at 11:03, P. Taylor Goetz 
> >> wrote:
> 
> > This is a call to vote on releasing Apache Storm 2.0.0 (rc4)
> >
> > Full list of changes in this release:
> >
> >
> >
> 
> >>>
> >>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc4/RELEASE_NOTES.html
> >
> > The tag/commit to be voted upon is v2.0.0:
> >
> >
> >
> 
> >>>
> >>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=1eece73e8c9ed7f41d2f20f727bc7f644c499360;hb=ddee8decac57d1a4a0aa23cc76066609a2abc8d2
> >
> > The source archive being voted upon can be found here:
> >
> >
> >
> 
> >>>
> >>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc4/apache-storm-2.0.0-src.tar.gz
> >
> > Other release files, signatures and digests can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc4/
> >
> > The release artifacts are signed with the following key:
> >
> >
> >
> 
> >>>
> >>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >
> > The Nexus staging repository for this release is:
> >
> >
> >> https://repository.apache.org/content/repositories/orgapachestorm-1073
> >
> > Please vote on releasing this package as Apache Storm 2.0.0.
> >
> > When voting, please list the actions taken to verify the release.
> >
> > This vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache Storm 2.0.0
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > -Taylor
> 
> >>>
> >>
> >>
> >> --
> >> Thanks,
> >> Xin
> >>
>
>

Re: New Storm Committer and PMC Member

[Stig Rohde Døssing]

Congratulations.

Den ons. 9. jan. 2019 kl. 19.55 skrev Roshan Naik
:

> Congratulations Govind. Roshan
>
>
> Sent from Yahoo Mail for iPhone
>
>
> On Wednesday, January 9, 2019, 10:47 AM, Ethan Li <
> ethanopensou...@gmail.com> wrote:
>
> Congratulations! Govind. Well deserved!
>
> Ethan
>
> > On Jan 9, 2019, at 12:40 PM, Hugo Louro  wrote:
> >
> > Congratulations Govind. Very well deserved. Thank you for all your
> > contributions and dedication to the Storm project.
> >
> > Best,
> > Hugo
> >
> > On Wed, Jan 9, 2019 at 10:36 AM Bobby Evans  wrote:
> >
> >> I am happy to announce that Govind Menon has just been added as the
> latest
> >> Committer and PMC member to the Apache Storm Project.  Please join me in
> >> congratulating him on this and thanking him for his contributions so
> far.
> >>
> >> Thanks,
> >>
> >> Bobby Evans
> >>
>
>
>
>
>

Re: Maintenance releases

[Stig Rohde Døssing]

Sounds good, thanks.

Den man. 7. jan. 2019 kl. 15.58 skrev P. Taylor Goetz :

> Not that I’m aware of. I’ll cue up the 1.x releases behind the 2.0 release.
>
> -Taylor
>
> > On Jan 7, 2019, at 3:58 AM, Stig Rohde Døssing 
> wrote:
> >
> > I think we should try to get the next 1.x releases out soon. Are there
> any
> > blockers?
>
>

Maintenance releases

[Stig Rohde Døssing]

I think we should try to get the next 1.x releases out soon. Are there any
blockers?

Re: Storm 2.0.0 release?

[Stig Rohde Døssing]

+1

Den lør. 5. jan. 2019 kl. 07.42 skrev Govind Menon <
govindappume...@gmail.com>:

> +1
>
> On Jan 4, 2019 23:46, "Roshan Naik"  wrote:
>
> +1
>
>
> Sent from Yahoo Mail for iPhone
>
>
>
> On Friday, January 4, 2019, 9:02 PM, Ethan Li 
> wrote:
>
> +1 on this
>
> Ethan Li
>
> > On Jan 4, 2019, at 20:19, P. Taylor Goetz  wrote:
> >
> > If no one objects, I’ll kick off a release candidate.
> >
> > -Taylor
> >
> >> On Jan 4, 2019, at 9:06 PM, saurabh mimani 
> wrote:
> >>
> >> Hey, Any approximate date for 2.0 release given there are no blockers?
> >>
> >>
> >>
> >> Best Regards
> >>
> >> Saurabh Kumar Mimani
> >>
> >>
> >>
> >>
> >> On Sat, Dec 22, 2018 at 1:13 AM Stig Rohde Døssing <
> stigdoess...@gmail.com>
> >> wrote:
> >>
> >>> Looks good to me, the blockers list in JIRA is empty for 2.0.0.
> >>>
> >>>> Den fre. 21. dec. 2018 kl. 19.52 skrev Bobby Evans  >:
> >>>>
> >>>> I think all of the blockers are in now.  Please take a look and
> >>> hopefully,
> >>>> we can get a release out soon.
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Bobby
> >>>>
> >>>>
> >>>>> On Fri, Dec 14, 2018 at 9:02 AM Bobby Evans 
> wrote:
> >>>>>
> >>>>> Sorry I was out at a conference for the past week, and have been
> heads
> >>>> down
> >>>>> on a different project for a while before that.  I'll respond to the
> >>>> JIRA.
> >>>>> I am happy to let it go in.
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Bobby
> >>>>>
> >>>>> On Thu, Dec 13, 2018 at 1:07 PM Stig Rohde Døssing <
> >>>> stigdoess...@gmail.com
> >>>>>>
> >>>>> wrote:
> >>>>>
> >>>>>> I think STORM-2990/3279 is ready. Bobby had a question (
> >>>>>> https://github.com/apache/storm/pull/2907#discussion_r234329136)
> >>>>> regarding
> >>>>>> whether Kafka offsets loop, but I wasn't sure where he was going
> with
> >>>> it,
> >>>>>> so I didn't want to merge prematurely.
> >>>>>>
> >>>>>> I agree that we can postpone STORM-2720. As far as I know it's
> >>> waiting
> >>>>> for
> >>>>>> STORM-2990 to go in, since it's going to be touching the same code.
> >>>>>>
> >>>>>> Den tor. 13. dec. 2018 kl. 19.54 skrev Roshan Naik
> >>>>>> :
> >>>>>>
> >>>>>>> Sounds like -  https://github.com/apache/storm/pull/2913
> >>>> (STORM-3290)
> >>>>>> is
> >>>>>>> merged -  https://github.com/apache/storm/pull/2908 (STORM-3276)
> >>> is
> >>>>>>> nearly complete and may need some small tweaks. -
> >>>>>>> https://github.com/apache/storm/pull/2907  (STORM-2990,
> >>> STORM-3279)
> >>>>>>> appears ready to be committed ?
> >>>>>>>
> >>>>>>> and- https://github.com/apache/storm/pull/2911   - (STORM-2720)
> >>>>> seems a
> >>>>>>> bit inactive and may not be critical enough to wait on.
> >>>>>>>
> >>>>>>> -roshan
> >>>>>>>  On Monday, November 26, 2018, 9:49:30 AM PST, Stig Rohde
> >>> Døssing
> >>>> <
> >>>>>>> stigdoess...@gmail.com> wrote:
> >>>>>>>
> >>>>>>> I would like to get at least
> >>>>> https://github.com/apache/storm/pull/2913
> >>>>>>> (breaking changes) and https://github.com/apache/storm/pull/2908
> >>>>>>> (regression) in.
> >>>>>>>
> >>>>>>> I think it would be nice to also get
> >>>>>>> https://github.com/apache/storm/pull/2907 and
> >>>>>>> https://github.com/apache/storm/pull/2911 in, but if we're in a
> >>>> hurry
> >>>>>> they
> >>>>>>> could go in the next release.
> >>>>>>>
> >>>>>>> Den man. 26. nov. 2018 kl. 17.37 skrev Julien Nioche <
> >>>>>>> lists.digitalpeb...@gmail.com>:
> >>>>>>>
> >>>>>>>> Hi devs,
> >>>>>>>>
> >>>>>>>> Is there anything blocking the release of Storm 2.0? Any idea of
> >>>> when
> >>>>>> it
> >>>>>>>> could happen?
> >>>>>>>>
> >>>>>>>> Thanks
> >>>>>>>>
> >>>>>>>> Julien
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>>
> >>>>>>>> *Open Source Solutions for Text Engineering*
> >>>>>>>>
> >>>>>>>> http://www.digitalpebble.com
> >>>>>>>> http://digitalpebble.blogspot.com/
> >>>>>>>> #digitalpebble <http://twitter.com/digitalpebble>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
>

Re: Storm 2.0.0 release?

[Stig Rohde Døssing]

Looks good to me, the blockers list in JIRA is empty for 2.0.0.

Den fre. 21. dec. 2018 kl. 19.52 skrev Bobby Evans :

> I think all of the blockers are in now.  Please take a look and hopefully,
> we can get a release out soon.
>
> Thanks,
>
> Bobby
>
>
> On Fri, Dec 14, 2018 at 9:02 AM Bobby Evans  wrote:
>
> > Sorry I was out at a conference for the past week, and have been heads
> down
> > on a different project for a while before that.  I'll respond to the
> JIRA.
> > I am happy to let it go in.
> >
> > Thanks,
> >
> > Bobby
> >
> > On Thu, Dec 13, 2018 at 1:07 PM Stig Rohde Døssing <
> stigdoess...@gmail.com
> > >
> > wrote:
> >
> > > I think STORM-2990/3279 is ready. Bobby had a question (
> > > https://github.com/apache/storm/pull/2907#discussion_r234329136)
> > regarding
> > > whether Kafka offsets loop, but I wasn't sure where he was going with
> it,
> > > so I didn't want to merge prematurely.
> > >
> > > I agree that we can postpone STORM-2720. As far as I know it's waiting
> > for
> > > STORM-2990 to go in, since it's going to be touching the same code.
> > >
> > > Den tor. 13. dec. 2018 kl. 19.54 skrev Roshan Naik
> > > :
> > >
> > > >  Sounds like -  https://github.com/apache/storm/pull/2913
> (STORM-3290)
> > > is
> > > > merged -  https://github.com/apache/storm/pull/2908 (STORM-3276) is
> > > > nearly complete and may need some small tweaks. -
> > > > https://github.com/apache/storm/pull/2907  (STORM-2990, STORM-3279)
> > > > appears ready to be committed ?
> > > >
> > > > and- https://github.com/apache/storm/pull/2911   - (STORM-2720)
> > seems a
> > > > bit inactive and may not be critical enough to wait on.
> > > >
> > > > -roshan
> > > > On Monday, November 26, 2018, 9:49:30 AM PST, Stig Rohde Døssing
> <
> > > > stigdoess...@gmail.com> wrote:
> > > >
> > > >  I would like to get at least
> > https://github.com/apache/storm/pull/2913
> > > > (breaking changes) and https://github.com/apache/storm/pull/2908
> > > > (regression) in.
> > > >
> > > > I think it would be nice to also get
> > > > https://github.com/apache/storm/pull/2907 and
> > > > https://github.com/apache/storm/pull/2911 in, but if we're in a
> hurry
> > > they
> > > > could go in the next release.
> > > >
> > > > Den man. 26. nov. 2018 kl. 17.37 skrev Julien Nioche <
> > > > lists.digitalpeb...@gmail.com>:
> > > >
> > > > > Hi devs,
> > > > >
> > > > > Is there anything blocking the release of Storm 2.0? Any idea of
> when
> > > it
> > > > > could happen?
> > > > >
> > > > > Thanks
> > > > >
> > > > > Julien
> > > > >
> > > > > --
> > > > >
> > > > > *Open Source Solutions for Text Engineering*
> > > > >
> > > > > http://www.digitalpebble.com
> > > > > http://digitalpebble.blogspot.com/
> > > > > #digitalpebble <http://twitter.com/digitalpebble>
> > > > >
> > > >
> > >
> >
>

Re: Storm 2.0.0 release?

[Stig Rohde Døssing]

I think STORM-2990/3279 is ready. Bobby had a question (
https://github.com/apache/storm/pull/2907#discussion_r234329136) regarding
whether Kafka offsets loop, but I wasn't sure where he was going with it,
so I didn't want to merge prematurely.

I agree that we can postpone STORM-2720. As far as I know it's waiting for
STORM-2990 to go in, since it's going to be touching the same code.

Den tor. 13. dec. 2018 kl. 19.54 skrev Roshan Naik
:

>  Sounds like -  https://github.com/apache/storm/pull/2913 (STORM-3290) is
> merged -  https://github.com/apache/storm/pull/2908 (STORM-3276) is
> nearly complete and may need some small tweaks. -
> https://github.com/apache/storm/pull/2907  (STORM-2990, STORM-3279)
> appears ready to be committed ?
>
> and- https://github.com/apache/storm/pull/2911   - (STORM-2720)  seems a
> bit inactive and may not be critical enough to wait on.
>
> -roshan
> On Monday, November 26, 2018, 9:49:30 AM PST, Stig Rohde Døssing <
> stigdoess...@gmail.com> wrote:
>
>  I would like to get at least https://github.com/apache/storm/pull/2913
> (breaking changes) and https://github.com/apache/storm/pull/2908
> (regression) in.
>
> I think it would be nice to also get
> https://github.com/apache/storm/pull/2907 and
> https://github.com/apache/storm/pull/2911 in, but if we're in a hurry they
> could go in the next release.
>
> Den man. 26. nov. 2018 kl. 17.37 skrev Julien Nioche <
> lists.digitalpeb...@gmail.com>:
>
> > Hi devs,
> >
> > Is there anything blocking the release of Storm 2.0? Any idea of when it
> > could happen?
> >
> > Thanks
> >
> > Julien
> >
> > --
> >
> > *Open Source Solutions for Text Engineering*
> >
> > http://www.digitalpebble.com
> > http://digitalpebble.blogspot.com/
> > #digitalpebble <http://twitter.com/digitalpebble>
> >
>

Re: Storm 2.0.0 release?

[Stig Rohde Døssing]

I would like to get at least https://github.com/apache/storm/pull/2913
(breaking changes) and https://github.com/apache/storm/pull/2908
(regression) in.

I think it would be nice to also get
https://github.com/apache/storm/pull/2907 and
https://github.com/apache/storm/pull/2911 in, but if we're in a hurry they
could go in the next release.

Den man. 26. nov. 2018 kl. 17.37 skrev Julien Nioche <
lists.digitalpeb...@gmail.com>:

> Hi devs,
>
> Is there anything blocking the release of Storm 2.0? Any idea of when it
> could happen?
>
> Thanks
>
> Julien
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble 
>

Re: NullPointerException in KafkaOffsetMetric.getValueAndReset causing worker to die

[Stig Rohde Døssing]

It looks like KAFKA-7044 affects 1.1.0 and up, so people on earlier
versions aren't affected. I think we should either make a work around for
the issue by skipping the metrics if the bug occurs, or add a link to
KAFKA-7044 to the documentation.

Den man. 19. nov. 2018 kl. 09.42 skrev Alexandre Vermeerbergen <
avermeerber...@gmail.com>:

> Hello Stig,
>
> Thank you very much for your answer : I have tested our many
> topologies using Kafka Client 2.0.1 instead of Kafka Client 2.0.0, let
> them run at full charge for couple of days, and I can confirm that
> this exception no longer occurs !
>
> May I suggest storm-kafka-client documentation to mention that this
> version of Kafka Client is recommended ?
>
> Kind regards,
> Alexandre Vermeerbergen
>
> Le lun. 12 nov. 2018 à 23:17, Stig Rohde Døssing
>  a écrit :
> >
> > I don't think beginningOffsets is null. I think it's missing one of the
> > partitions, which would mean the right hand side of the line is null,
> which
> > gives an NPE when we try to assign it to a primitive long.
> >
> > I think this could be due to
> > https://issues.apache.org/jira/browse/KAFKA-7044, going by the commit
> > message for the fix
> >
> https://github.com/apache/kafka/commit/e2ec2d79c8d5adefc0c764583cec47144dbc5705#diff-b45245913eaae46aa847d2615d62cde0
> .
> > Specifically part 2 sounds a lot like what I think might be happening
> here.
> >
> > "
> >
> > `ConsumerGroupCommand.getLogEndOffsets()` and `getLogStartOffsets()`
> > assumed that endOffsets()/beginningOffsets() which eventually call
> > Fetcher.fetchOffsetsByTimes(), would return a map with all the topic
> > partitions passed to endOffsets()/beginningOffsets() and that values
> > are not null. Because of (1), null values were possible if some of the
> > topic partitions were already known (in metadata cache) and some not
> > (metadata cache did not have entries for some of the topic
> > partitions). However, even with fixing (1),
> > endOffsets()/beginningOffsets() may return a map with some topic
> > partitions missing, when list offset request returns a non-retriable
> > error.
> >
> > "
> >
> > Basically KafkaOffsetMetric also assumes that when
> beginningOffsets(topics)
> > is called, the returned map will contain a value for all requested
> topics.
> > Could you try upgrading to Kafka 2.0.1?
> >
> > If necessary we can also work around this on the Storm side by skipping
> the
> > metrics if the requested partition isn't in the return values for
> > beginningOffsets/endOffsets. Feel free to raise an issue for this.
> >
> > Den man. 12. nov. 2018 kl. 21.56 skrev Alexandre Vermeerbergen <
> > avermeerber...@gmail.com>:
> >
> > > Hello,
> > >
> > > Using Storm 1.2.3-snapshot of the 3rd of November 2018 with all libs
> > > (storm-core & storm-kafka-client) taken from same Git, we get the
> > > following crash coming from a NullPointerException in
> > > KafkaOffsetMetric.getValueAndReset :
> > >
> > > 2018-11-12 19:31:30.496 o.a.s.util
> > > Thread-9-metricsFromKafka-executor[13 13] [ERROR] Async loop died!
> > >
> > > java.lang.RuntimeException: java.lang.NullPointerException
> > >
> > >   at
> > >
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522)
> > > ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
> > >
> > >   at
> > >
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)
> > > ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
> > >
> > >   at
> > >
> org.apache.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:477)
> > > ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
> > >
> > >   at
> > > org.apache.storm.disruptor$consume_batch.invoke(disruptor.clj:70)
> > > ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
> > >
> > >   at
> > >
> org.apache.storm.daemon.executor$fn__9620$fn__9635$fn__9666.invoke(executor.clj:634)
> > > ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
> > >
> > >   at
> org.apache.storm.util$async_loop$fn__561.invoke(util.clj:484)
> > > [storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
> > >
> > >   at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
> > >
> > >   at java.lang.Thread.run(Thread.java:748) [?:1.8.0_192]
> > >
> > > Caused by: java.lang.NullPointer

Re: NullPointerException in KafkaOffsetMetric.getValueAndReset causing worker to die

[Stig Rohde Døssing]

I don't think beginningOffsets is null. I think it's missing one of the
partitions, which would mean the right hand side of the line is null, which
gives an NPE when we try to assign it to a primitive long.

I think this could be due to
https://issues.apache.org/jira/browse/KAFKA-7044, going by the commit
message for the fix
https://github.com/apache/kafka/commit/e2ec2d79c8d5adefc0c764583cec47144dbc5705#diff-b45245913eaae46aa847d2615d62cde0.
Specifically part 2 sounds a lot like what I think might be happening here.

"

`ConsumerGroupCommand.getLogEndOffsets()` and `getLogStartOffsets()`
assumed that endOffsets()/beginningOffsets() which eventually call
Fetcher.fetchOffsetsByTimes(), would return a map with all the topic
partitions passed to endOffsets()/beginningOffsets() and that values
are not null. Because of (1), null values were possible if some of the
topic partitions were already known (in metadata cache) and some not
(metadata cache did not have entries for some of the topic
partitions). However, even with fixing (1),
endOffsets()/beginningOffsets() may return a map with some topic
partitions missing, when list offset request returns a non-retriable
error.

"

Basically KafkaOffsetMetric also assumes that when beginningOffsets(topics)
is called, the returned map will contain a value for all requested topics.
Could you try upgrading to Kafka 2.0.1?

If necessary we can also work around this on the Storm side by skipping the
metrics if the requested partition isn't in the return values for
beginningOffsets/endOffsets. Feel free to raise an issue for this.

Den man. 12. nov. 2018 kl. 21.56 skrev Alexandre Vermeerbergen <
avermeerber...@gmail.com>:

> Hello,
>
> Using Storm 1.2.3-snapshot of the 3rd of November 2018 with all libs
> (storm-core & storm-kafka-client) taken from same Git, we get the
> following crash coming from a NullPointerException in
> KafkaOffsetMetric.getValueAndReset :
>
> 2018-11-12 19:31:30.496 o.a.s.util
> Thread-9-metricsFromKafka-executor[13 13] [ERROR] Async loop died!
>
> java.lang.RuntimeException: java.lang.NullPointerException
>
>   at
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522)
> ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
>
>   at
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)
> ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
>
>   at
> org.apache.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:477)
> ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
>
>   at
> org.apache.storm.disruptor$consume_batch.invoke(disruptor.clj:70)
> ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
>
>   at
> org.apache.storm.daemon.executor$fn__9620$fn__9635$fn__9666.invoke(executor.clj:634)
> ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
>
>   at org.apache.storm.util$async_loop$fn__561.invoke(util.clj:484)
> [storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
>
>   at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
>
>   at java.lang.Thread.run(Thread.java:748) [?:1.8.0_192]
>
> Caused by: java.lang.NullPointerException
>
>   at
> org.apache.storm.kafka.spout.metrics.KafkaOffsetMetric.getValueAndReset(KafkaOffsetMetric.java:89)
> ~[stormjar.jar:?]
>
>   at
> org.apache.storm.daemon.executor$metrics_tick$fn__9544.invoke(executor.clj:345)
> ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
>
>   at clojure.core$map$fn__4553.invoke(core.clj:2622)
> ~[clojure-1.7.0.jar:?]
>
>   at clojure.lang.LazySeq.sval(LazySeq.java:40)
> ~[clojure-1.7.0.jar:?]
>
>   at clojure.lang.LazySeq.seq(LazySeq.java:49)
> ~[clojure-1.7.0.jar:?]
>
>   at clojure.lang.RT.seq(RT.java:507) ~[clojure-1.7.0.jar:?]
>
>   at clojure.core$seq__4128.invoke(core.clj:137)
> ~[clojure-1.7.0.jar:?]
>
>   at clojure.core$filter$fn__4580.invoke(core.clj:2679)
> ~[clojure-1.7.0.jar:?]
>
>   at clojure.lang.LazySeq.sval(LazySeq.java:40)
> ~[clojure-1.7.0.jar:?]
>
>   at clojure.lang.LazySeq.seq(LazySeq.java:49)
> ~[clojure-1.7.0.jar:?]
>
>   at clojure.lang.Cons.next(Cons.java:39) ~[clojure-1.7.0.jar:?]
>
>   at clojure.lang.RT.next(RT.java:674) ~[clojure-1.7.0.jar:?]
>
>   at clojure.core$next__4112.invoke(core.clj:64)
> ~[clojure-1.7.0.jar:?]
>
>   at clojure.core.protocols$fn__6523.invoke(protocols.clj:170)
> ~[clojure-1.7.0.jar:?]
>
>   at
> clojure.core.protocols$fn__6478$G__6473__6487.invoke(protocols.clj:19)
> ~[clojure-1.7.0.jar:?]
>
>   at clojure.core.protocols$seq_reduce.invoke(protocols.clj:31)
> ~[clojure-1.7.0.jar:?]
>
>   at clojure.core.protocols$fn__6506.invoke(protocols.clj:101)
> ~[clojure-1.7.0.jar:?]
>
>   at
> clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13)
> ~[clojure-1.7.0.jar:?]
>
>   at clojure.core$reduce.invoke(core.clj:6519)
> ~[clojure-1.7.0.jar:?]
>
>   at 

Re: Storm Source Code

[Stig Rohde Døssing]

1) Storm-server is a new module in 2.0.0/the current master. In 1.x the
classes in storm-server were in the storm-core module. You should make your
changes there instead.

2-4) There is probably something wrong with either how you're building
Storm or how you're deploying it. "mvn clean install -DskipTests" in the
project root will build Storm. Then you should go into storm-dist/binary
(storm-dist/binary/final-package for master) and run "mvn clean package
-Dgpg.skip" to produce the final tar/zip, which will be somewhere in the
storm-dist/target directory. You can then extract this tar/zip to wherever
you want Storm installed.

5) Could you elaborate? If you don't do "git checkout ", you're
likely just building Storm from master, so remember to go into
storm-dist/binary/final-package instead of storm-dist/binary.

If it still doesn't work please list each command you run, starting from a
clean checkout of Storm, it will be easier to tell what's going wrong that
way.

Den fre. 9. nov. 2018 kl. 18.24 skrev Ashwin Siddharth :

> Hi,
> I am Ashwin,
> I am a Storm Developer,I want to make changes to the storm source code and
> build the Code with the existing changes,
>
> Problems faced:
> 1)Making changes to EvenScheduler.java,when I do git checkout tags/v1.2.2,
> the storm-server folder no longer exists, the final built code , doesn't
> reflect the changes made in the scheduler, the log files print the old
> log-messages , not the newly changed ones.
>
> 2)Secondly, I tried to make changes to EvenScheduler.clj as git checkout
> removes storm-server, I thought I could make changes in the storm-core
> directory.But again the changes made were not reflected in the log files.
>
> 3)Third try , I did git checkout tags/v1.2.2 and made changes to the
> nimbus.clj file followed by mvn clean install, with storm-server no longer
> present , I thought it would be better to make changes to the nimbus.clj
> file in the storm-core directory, as it was also seconded by the nimbus.log
> file which showed - "o.a.s.d- followed by a nimbus log message".Yet
> again , I could not see the changes made in the nimbus.log file.
>
> 4)I tried making similar changes to various other clj files and looking for
> changes in the corresponding log files, again I could not notice any
> changes.
>
> 5)Building without the git checkout command , does not produce the
> apache-tar file in the target directory.
>
> So,From all this , I couldn't infer where I am going wrong , am I making
> mistakes in building the code or am I making mistakes in making changes to
> the source code.It would be very helpful if I could get a way out of
> this.Can you Please help me figure out how to successfully make changes to
> code and observe the changes made.
>
>
> Warm regards,
> Ashwin S
>

Re: Cleaning Up Old Pull Requests STORM-3250

[Stig Rohde Døssing]

Thanks for explaining, that makes sense. It's probably easier to go over
the issues once the PRs are closed, instead of having to handle them all at
the same time.

Den ons. 10. okt. 2018 kl. 20.27 skrev Derek Dagit :

> > What is the value in keeping the associated Jira issues around though?
>
> 1) I had thought Jira issues could be valid even if they are stale, whereas
> pull requests typically are not valid when they go stale.
> 2) It is less effort. :)
>
> The simplest thing to do would be to run a similar query (not updated in
> 2018) and close the issues with a common message.
>
>
> https://issues.apache.org/jira/browse/STORM-579?jql=project%20%3D%20STORM%20AND%20statusCategory%20!%3D%20done%20AND%20updatedDate%20%3C%20startOfYear()%20ORDER%20BY%20updated%20ASC
>
> It seems there are currently 800+ issues that match. That is an awful lot.
> I would be open to handling these in bulk too if that is what we want.
>
>
> On Wed, Oct 10, 2018 at 1:19 PM Hugo Louro  wrote:
>
> > Derek, I am OK with closing them all. By phase I meant perhaps leaving
> some
> > of the most recent one's in case the author wants to resume them... but I
> > guess he can always reopen them.
> > Stig, the JIRAs I think the should be handled on an individual basis. If
> > they are still revenant, leave the JIRA open hoping someone will pick it
> > up. If they are no longer relevant, perhaps close as "will not fix" or
> > something like that.
> >
> > Hugo
> >
> > On Wed, Oct 10, 2018 at 11:12 AM Stig Rohde Døssing <
> > stigdoess...@gmail.com>
> > wrote:
> >
> > > +1 to close old PRs.
> > >
> > > What is the value in keeping the associated Jira issues around though?
> > >
> > > Den tir. 9. okt. 2018 kl. 17.18 skrev Derek Dagit
> >  > > >:
> > >
> > > > > Is the idea to remove them all in one batch, or have the removal
> > > process
> > > > through phases ?
> > > >
> > > > Yeah, the idea was to close them all in one batch. We do not want to
> > > close
> > > > pull requests that have value, and we want to balance this with the
> > > effort
> > > > required to review each one to see if it instead should be kept open.
> > > >
> > > > If we are interested in putting in more effort, then we could remove
> in
> > > > phases.
> > > >
> > > > On Mon, Oct 8, 2018 at 5:30 PM Hugo Louro 
> wrote:
> > > >
> > > > > +1 to remove old PRs. If any PRs still warrant any value we could
> try
> > > > > reaching out to the creator to see if he wants to follow up with
> it.
> > Is
> > > > the
> > > > > idea to remove them all in one batch, or have the removal process
> > > through
> > > > > phases ?
> > > > >
> > > > > On Mon, Oct 8, 2018 at 3:26 PM Jungtaek Lim 
> > wrote:
> > > > >
> > > > > > +1 It doesn't look like there're any critical PRs in the list,
> and
> > it
> > > > is
> > > > > > pretty less chance we could connect with PR authors.
> > > > > >
> > > > > > -Jungtaek Lim (HeartSaVioR)
> > > > > >
> > > > > > 2018년 10월 9일 (화) 오전 5:08, Kishorkumar Patil
> >  > > > >님이
> > > > > > 작성:
> > > > > >
> > > > > > > +1.
> > > > > > > It would be nice to clean up old clutter while we are getting
> > ready
> > > > for
> > > > > > > days past 2.x
> > > > > > >
> > > > > > > -Kishor
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Oct 8, 2018 at 3:52 PM Bobby Evans 
> > > wrote:
> > > > > > >
> > > > > > > > +1
> > > > > > > >
> > > > > > > > Sounds good to me
> > > > > > > >
> > > > > > > > On Mon, Oct 8, 2018 at 2:50 PM Derek Dagit
> > >  > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Currently there are over 150 open pull requests on the
> Apache
> > > > Storm
> > > > > > > > GitHub
> > > > > > > > > project. Over 100 of these have not been modified in 2018.
> > > > > > > > >
> > > > > > > > > It seems we are unlikely to handle each one of these
> without
> > > > > > > significant
> > > > > > > > > effort and time. Looking at many of them, they seem to be
> > > > abandoned
> > > > > > by
> > > > > > > > the
> > > > > > > > > requester.
> > > > > > > > >
> > > > > > > > > I propose in STORM-3250 that we close all pull requests
> that
> > > have
> > > > > not
> > > > > > > > been
> > > > > > > > > updated in 2018 and leave any corresponding Jira issues as
> > they
> > > > > are.
> > > > > > If
> > > > > > > > > there are any pull requests among these should remain open,
> > > > please
> > > > > > let
> > > > > > > me
> > > > > > > > > know. I plan to wait at least a week before requesting any
> > > > changes.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/storm/pulls?utf8=%E2%9C%93=is%3Apr+is%3Aopen+updated%3A%3C2018-01-01
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Derek
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Derek
> > > >
> > >
> >
>
>
> --
> Derek
>

Re: Cleaning Up Old Pull Requests STORM-3250

[Stig Rohde Døssing]

+1 to close old PRs.

What is the value in keeping the associated Jira issues around though?

Den tir. 9. okt. 2018 kl. 17.18 skrev Derek Dagit :

> > Is the idea to remove them all in one batch, or have the removal process
> through phases ?
>
> Yeah, the idea was to close them all in one batch. We do not want to close
> pull requests that have value, and we want to balance this with the effort
> required to review each one to see if it instead should be kept open.
>
> If we are interested in putting in more effort, then we could remove in
> phases.
>
> On Mon, Oct 8, 2018 at 5:30 PM Hugo Louro  wrote:
>
> > +1 to remove old PRs. If any PRs still warrant any value we could try
> > reaching out to the creator to see if he wants to follow up with it. Is
> the
> > idea to remove them all in one batch, or have the removal process through
> > phases ?
> >
> > On Mon, Oct 8, 2018 at 3:26 PM Jungtaek Lim  wrote:
> >
> > > +1 It doesn't look like there're any critical PRs in the list, and it
> is
> > > pretty less chance we could connect with PR authors.
> > >
> > > -Jungtaek Lim (HeartSaVioR)
> > >
> > > 2018년 10월 9일 (화) 오전 5:08, Kishorkumar Patil  >님이
> > > 작성:
> > >
> > > > +1.
> > > > It would be nice to clean up old clutter while we are getting ready
> for
> > > > days past 2.x
> > > >
> > > > -Kishor
> > > >
> > > >
> > > > On Mon, Oct 8, 2018 at 3:52 PM Bobby Evans  wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Sounds good to me
> > > > >
> > > > > On Mon, Oct 8, 2018 at 2:50 PM Derek Dagit  >
> > > > > wrote:
> > > > >
> > > > > > Currently there are over 150 open pull requests on the Apache
> Storm
> > > > > GitHub
> > > > > > project. Over 100 of these have not been modified in 2018.
> > > > > >
> > > > > > It seems we are unlikely to handle each one of these without
> > > > significant
> > > > > > effort and time. Looking at many of them, they seem to be
> abandoned
> > > by
> > > > > the
> > > > > > requester.
> > > > > >
> > > > > > I propose in STORM-3250 that we close all pull requests that have
> > not
> > > > > been
> > > > > > updated in 2018 and leave any corresponding Jira issues as they
> > are.
> > > If
> > > > > > there are any pull requests among these should remain open,
> please
> > > let
> > > > me
> > > > > > know. I plan to wait at least a week before requesting any
> changes.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/storm/pulls?utf8=%E2%9C%93=is%3Apr+is%3Aopen+updated%3A%3C2018-01-01
> > > > > >
> > > > > > --
> > > > > > Derek
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
> --
> Derek
>

Re: [VOTE] Release Apache Storm 2.0.0 (rc2)

[Stig Rohde Døssing]

+1

Built and ran unit tests from the tag.
Ran ExclamationTopology locally using the Storm tar, verified that UI looks
as expected, that logviewer works, and that there were no errors in the
logs.
Verified the signature and SHA512 for the source and binary tars.

We should consider deleting the md5 files, Apache's release policy
recommends against including them in a release
https://www.apache.org/dev/release-distribution#sigs-and-sums.


Den ons. 10. okt. 2018 kl. 17.02 skrev Bobby Evans :

> +1
>
> built and ran all of the unit tests from the tag.
> Ran some small perf tests on a single node cluster.  Things look really
> good there.
>
>
> On a side note our CI pipeline has been running and passing builds very
> close to this release too.  (we are following master currently) and it is
> looking good.
>
> Thanks,
>
> Bobby
>
> On Tue, Oct 9, 2018 at 4:02 PM Kishorkumar Patil 
> wrote:
>
> > +1  to release this package.
> >
> > I ran basic tests, and fucntionality tested manually some of the UI
> > features and profiling issues reported as part of the blockers. I did not
> > notice any silent failures either - or any failures/exception in the
> logs.
> >
> > Regards
> > -Kishor
> >
> >
> > On Tue, Oct 9, 2018 at 4:05 PM P. Taylor Goetz 
> wrote:
> >
> > > This is a call to vote on releasing Apache Storm 2.0.0 (rc2)
> > >
> > > Full list of changes in this release:
> > >
> > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc2/RELEASE_NOTES.html
> > >
> > > The tag/commit to be voted upon is v2.0.0:
> > >
> > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=f8d04910dc3fd14534c186232ecf7882d8916f67;hb=f8d04910dc3fd14534c186232ecf7882d8916f67
> > >
> > > The source archive being voted upon can be found here:
> > >
> > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc2/apache-storm-2.0.0-src.tar.gz
> > >
> > > Other release files, signatures and digests can be found here:
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc2/
> > >
> > > The release artifacts are signed with the following key:
> > >
> > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> > >
> > > The Nexus staging repository for this release is:
> > >
> > > https://repository.apache.org/content/repositories/orgapachestorm-1071
> > >
> > > Please vote on releasing this package as Apache Storm 2.0.0.
> > >
> > > When voting, please list the actions taken to verify the release.
> > >
> > > This vote will be open for at least 72 hours.
> > >
> > > [ ] +1 Release this package as Apache Storm 2.0.0
> > > [ ]  0 No opinion
> > > [ ] -1 Do not release this package because...
> > >
> > > Thanks to everyone who contributed to this release.
> > >
> > > -Taylor
> > >
> >
>

Re: Companies Using Apache Storm Listing - XenonStack

[Stig Rohde Døssing]

Hi XenonStack,

Thanks. We'll update the site.

Den tor. 13. sep. 2018 kl. 13.21 skrev XenonStack A Stack Innovator <
busin...@xenonstack.com>:

> At XenonStack we use Storm for building real-time data integration systems
> and Enabling  Predictive analysis
>

Is the powered-by file in the Storm repository necessary?

[Stig Rohde Døssing]

Hi,

We have a powered-by.md file in the Storm repository's /docs directory.
There's also a powered-by file in the storm-site repo root, as well as in
each release directory.

The storm-site root powered-by is the one linked to by the Storm site, and
I don't believe it's being updated based on the Storm repo powered-by. The
Storm repo powered-by ends up in the release-specific documentation
instead, e.g.
https://storm.apache.org/releases/2.0.0-SNAPSHOT/Powered-By.html.

I think we might as well treat Powered-By.md the same as the
getting-help.md file, which seems to only be present in the storm-site
repo. I'd like to delete powered-by from the Storm repo, so we only have
the one in the storm-site root.

Any opinions?

Re: Regarding releasing Apache Storm 2.0.0

[Stig Rohde Døssing]

+1 to cut an RC.

Here are a couple of PRs that could maybe go in

https://github.com/apache/storm/pull/2719
https://github.com/apache/storm/pull/2800 (this one requires some changes,
but we should be able to fix it pretty quickly)
also would like to get https://github.com/apache/storm/pull/2805 reviewed,
it might change some public methods.

Other than that, we should try to remove as much deprecated code as we can
before release

https://issues.apache.org/jira/browse/STORM-2947

Den man. 10. sep. 2018 kl. 21.59 skrev Alexandre Vermeerbergen <
avermeerber...@gmail.com>:

> +1 for an Storm 2.0 as soon as possible, let's jump into the future :)
> Le lun. 10 sept. 2018 à 21:50, Kishorkumar Patil
>  a écrit :
> >
> > Looking into all issues reported under epic
> > https://issues.apache.org/jira/browse/STORM-2714 are resolved/closed. I
> > don't see any open issues/blockers at this point for going ahead with 2.x
> > release.
> >
> > I am +1 to 2.0 release.
> >
> > Regards,
> > -Kishor
> >
> > On Mon, Sep 10, 2018 at 2:24 PM, P. Taylor Goetz 
> wrote:
> >
> > > I agree, and looking through the JIRAs against 2.0, I would say a
> majority
> > > of the ones marked critical are not critical.
> > >
> > > I’m +1 on moving forward with a 2.0 release, but will give others time
> to
> > > respond with any JIRAs they think should be included.
> > >
> > > > p.s. I don't want to create branch-2.x or branch-2.0.x until
> absolutely
> > > > necessary, I don't see any major features with pull requests up but
> if
> > > you
> > > > do run across one please send something out before merging it in, so
> we
> > > can
> > > > set up the branches properly at that time.
> > >
> > >
> > > Agree. We can always branch off the release tag/commit.
> > >
> > > -Taylor
> > >
> > >
> > > > On Sep 10, 2018, at 12:25 PM, Bobby Evans  wrote:
> > > >
> > > > It has been nearly a month since this was originally sent out, and
> this
> > > is
> > > > not the first of these kinds of emails to go out about a 2.0.0
> release.
> > > I
> > > > think we have made a lot of really good progress on getting ready
> for a
> > > 2.0
> > > > release, and I really would like to see it happen before another
> month
> > > > passes.
> > > >
> > > > We have a 2.0 based deploy in some of our staging clusters, currently
> > > > following the master branch with a little that is Yahoo specific on
> top.
> > > We
> > > > would like to start pushing towards production with it soon.
> > > >
> > > > There are a few issues that we are aware of.
> > > >
> > > >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20STORM%20AND%
> > > 20affectedVersion%20in%20(2.0.0)%20AND%20resolution%20%3D%
> > > 20Unresolved%20ORDER%20BY%20priority%20DESC
> > > >
> > > > There are no blockers still open, and only 4 issues listed as
> critical.
> > > >
> > > > If others have any open issues that feel need to be addressed prior
> to a
> > > > 2.0.0 release please respond to this with the JIRA number.  I would
> like
> > > to
> > > > set a goal/tentative date of Sep 17th (one week from today) to put
> > > together
> > > > a release candidate for a 2.0.0 release, and unless there are major
> > > > blockers that show up I think we can do it.
> > > >
> > > > Thanks,
> > > >
> > > > Bobby Evans
> > > >
> > > > p.s. I don't want to create branch-2.x or branch-2.0.x until
> absolutely
> > > > necessary, I don't see any major features with pull requests up but
> if
> > > you
> > > > do run across one please send something out before merging it in, so
> we
> > > can
> > > > set up the branches properly at that time.
> > > >
> > > >
> > > > On Wed, Jul 18, 2018 at 10:47 PM Jungtaek Lim 
> wrote:
> > > >
> > > >> I'd like to say first, thanks Stig to take up remaining issues.
> Thanks
> > > to
> > > >> his efforts, according to the epic, we have only one major issue
> left:
> > > >> porting UI to Java [1], and pull request [2] is available for that.
> > > >> There're another issues [3] [4] targeting 2.0.0 (since it is
> backward
> > > >> incompatible) but they are all about removing deprecated things, so
> > > easier
> > > >> to be reviewed and make decisions.
> > > >>
> > > >> Once we have a patch for that now, IMHO it would be good to review
> and
> > > ship
> > > >> in 2.0.0 if it wouldn't take a month or so. We could do some sanity
> > > tests
> > > >> in parallel, so waiting for UI port would not block much time on
> > > releasing
> > > >> Storm 2.0.0.
> > > >>
> > > >> - Jungtaek Lim (HeartSaVioR)
> > > >>
> > > >> 1. https://issues.apache.org/jira/browse/STORM-1311
> > > >> 2. https://github.com/apache/storm/pull/2752
> > > >> 3. https://issues.apache.org/jira/browse/STORM-2947
> > > >> 4. https://issues.apache.org/jira/browse/STORM-3156
> > > >>
> > > >>
> > > >> 2018년 7월 11일 (수) 오전 5:12, Alexandre Vermeerbergen <
> > > >> avermeerber...@gmail.com>님이
> > > >> 작성:
> > > >>
> > > >>> +1 would love to try it when an RC is avail!
> > > >>>
> > > >>> Alexandre Vermeerbergen
> > > >>>
> > > 

Re: [DISCUSS] Remove Ganglia metrics reporter

[Stig Rohde Døssing]

Doesn't seem like there's any opposition to this. I think we can go ahead
and merge https://github.com/apache/storm/pull/2807 in the next day or two.

Den man. 20. aug. 2018 kl. 06.49 skrev Jungtaek Lim :

> For Storm 2.0 I'm OK to drop supporting Ganglia reporter as same as why we
> removed storm-druid in Storm 2.0.
>
> When we drop supporting Ganglia reporter in 1.x it would be backward
> incompatible. We just introduced Metrics V2 in Storm 1.2 and I guess few
> people use Ganglia reporter, so I feel even OK to drop supporting for 1.x,
> but just want to be sure it has no outstanding impact.
>
> 2018년 8월 18일 (토) 오후 9:37, Stig Rohde Døssing 님이
> 작성:
>
>> Hi,
>>
>> The metrics v2 system contains a Ganglia reporter based on Dropwizard
>> Metrics' metrics-ganglia module. One of the dependencies of this module is
>> licensed under LGPL. In order to avoid including LGPL software in our
>> releases, we would have to exclude this module and ask users to manually
>> install the relevant jars.
>>
>> Dropwizard Metrics has dropped support for metrics-ganglia due to the
>> license on the LGPL dependency
>> https://github.com/dropwizard/metrics/issues/1319. We won't be able to
>> upgrade to Metrics version 4+ unless we remove the metrics-ganglia
>> dependency.
>>
>> I'd like to know whether anyone is not okay with removing the Ganglia
>> reporter from master and deprecating it in 1.x? If we want to keep Ganglia
>> reporting, we should try to find an alternative to metrics-ganglia so we
>> don't get stuck on a specific Metrics version.
>>
>

[DISCUSS] Remove Ganglia metrics reporter

[Stig Rohde Døssing]

Hi,

The metrics v2 system contains a Ganglia reporter based on Dropwizard
Metrics' metrics-ganglia module. One of the dependencies of this module is
licensed under LGPL. In order to avoid including LGPL software in our
releases, we would have to exclude this module and ask users to manually
install the relevant jars.

Dropwizard Metrics has dropped support for metrics-ganglia due to the
license on the LGPL dependency
https://github.com/dropwizard/metrics/issues/1319. We won't be able to
upgrade to Metrics version 4+ unless we remove the metrics-ganglia
dependency.

I'd like to know whether anyone is not okay with removing the Ganglia
reporter from master and deprecating it in 1.x? If we want to keep Ganglia
reporting, we should try to find an alternative to metrics-ganglia so we
don't get stuck on a specific Metrics version.

Re: [DISCUSS] Plans for releasing Storm 1.2.3

[Stig Rohde Døssing]

I think we need to resolve https://issues.apache.org/jira/browse/STORM-3199
before we can release.

2018-08-17 3:21 GMT+02:00 Aniket Alhat :

> +1
>
> On Wed, Aug 8, 2018 at 1:16 AM Hugo Louro  wrote:
>
> > +1
> >
> > On Tue, Aug 7, 2018 at 9:02 AM Alexandre Vermeerbergen <
> > avermeerber...@gmail.com> wrote:
> >
> > > +1 for a Storm release 1.2.3 first (non binding)
> > >
> > > Alexandre Vermeerbergen
> > > Le mar. 7 août 2018 à 17:47, Stig Rohde Døssing
> > >  a écrit :
> > > >
> > > > Hi devs,
> > > >
> > > > I've seen a couple of requests for releasing Storm 1.2.3 soon on the
> > > users
> > > > list. I know we're currently working toward a release for Storm
> 2.0.0,
> > > but
> > > > I think it would be nice to release 1.2.3 first. It contains some
> > decent
> > > > bugfixes, and also contains the deprecation of storm-druid. Last
> > release
> > > > was a couple of months ago, so I think it's a good time to do
> another.
> > > >
> > > > What do you think? Is there anything pending we should get in first?
> > >
> >
>
>
> --
>
> *Aniket Alhat*
>

[DISCUSS] Plans for releasing Storm 1.2.3

[Stig Rohde Døssing]

Hi devs,

I've seen a couple of requests for releasing Storm 1.2.3 soon on the users
list. I know we're currently working toward a release for Storm 2.0.0, but
I think it would be nice to release 1.2.3 first. It contains some decent
bugfixes, and also contains the deprecation of storm-druid. Last release
was a couple of months ago, so I think it's a good time to do another.

What do you think? Is there anything pending we should get in first?

Re: Regarding releasing Apache Storm 2.0.0

[Stig Rohde Døssing]

Sounds good to me. The list in JIRA also looks fine to me.

It might make sense to get https://github.com/apache/storm/pull/2443 in
before doing https://issues.apache.org/jira/browse/STORM-2972, so the first
might effectively block removing storm-kafka.

I think even if we can't remove everything listed in
https://issues.apache.org/jira/browse/STORM-2947, it would be good to
delete as much as possible before releasing 2.0.0. I might take a look at
it soon, unless you're already working on it.

2018-07-10 8:18 GMT+02:00 Jungtaek Lim :

> Hi devs,
>
> I hopefully have a time to sort out issues regarding Storm 2.0.0 and link
> to epic issue.
>
> https://issues.apache.org/jira/browse/STORM-2714
> (require login to Apache JIRA to see issues in epic)
>
> I guess we are close to the release, mostly left reviewing some pending
> pull requests, and some manual sanity tests.
>
> Given that master branch is relatively stabilized for Travis CI build, as
> well as style check and Java port make codebase better (at least for me), I
> would really want to make Storm 2.0.0 released sooner than later, and rely
> majorly on 2.x version line.
>
> So I would propose dev folks to concentrate on remaining tasks for Storm
> 2.0.0 till we announce release. WDYT?
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>

Re: Requesting review for a couple of PRs

[Stig Rohde Døssing]

Hugo,

Sounds good, thanks.

Jungtaek,

I don't think there are any missing tasks in STORM-2953. Once the listed
tasks are done, we should be good to delete storm-kafka. I've been holding
off on looking at https://issues.apache.org/jira/browse/STORM-2972 because
I was hoping that https://github.com/apache/storm/pull/2443 could go in
first.

2018-07-07 0:21 GMT+02:00 Hugo Louro :

> I’ll take a pass at reviewing them as well.
> Thanks,
> Hugo
>
> > On Jul 6, 2018, at 1:41 PM, Jungtaek Lim  wrote:
> >
> > Yeah I have been hoping that some other folks who are familiar with
> > storm-kafka-client jump in and review in time, but unfortunately it
> didn't
> > happen. Will try to review those PRs, hopefully within couple of days.
> >
> > Btw, if there're missing PRs to resolve STORM-2953
> > <https://issues.apache.org/jira/browse/STORM-2953> I'd like to ask you
> to
> > also add them here, so we can resolve the one which is effectively
> blocker
> > for Storm 2.0.
> >
> > Thanks,
> > Jungtaek Lim (HeartSaVioR)
> >
> > 2018년 7월 6일 (금) 오후 9:08, Stig Rohde Døssing 님이
> 작성:
> >
> >> Hi devs,
> >>
> >> There are a couple of PRs open for changes in storm-kafka-client that
> would
> >> be nice to get reviewed. The PRs are
> >> https://github.com/apache/storm/pull/2652
> >> https://github.com/apache/storm/pull/2648 and
> >> https://github.com/apache/storm/pull/2590. If someone could find a
> couple
> >> of minutes to give them a look, I'd appreciate it.
> >>
> >> Thanks.
> >>
>

Requesting review for a couple of PRs

[Stig Rohde Døssing]

Hi devs,

There are a couple of PRs open for changes in storm-kafka-client that would
be nice to get reviewed. The PRs are
https://github.com/apache/storm/pull/2652
https://github.com/apache/storm/pull/2648 and
https://github.com/apache/storm/pull/2590. If someone could find a couple
of minutes to give them a look, I'd appreciate it.

Thanks.

Re: Looking for current Storm 1.2.3 snapshot build or git infos to retrieve its sources

[Stig Rohde Døssing]

The easiest way to get the build artifacts would probably be to build Storm
yourself. Clone https://github.com/apache/storm and check out 1.x-branch.
Then do "mvn clean install -DskipTests" in the project root. This will put
all the artifacts (e.g. stuff like storm-kafka-client) in your local Maven
repository (likely in ~/.m2/repository unless you've changed some
settings), and also in the /target directories of the respective modules.

If you also need to generate a distribution like what you would find on
https://storm.apache.org/downloads.html, once you've built the rest of
Storm you can go into storm-dist/binary and do "mvn clean package
-Dgpg.skip". This will put the distribution files in
storm-dist/binary/final-package/target.

See the documentation at
https://github.com/apache/storm/blob/1.x-branch/DEVELOPER.md#building and
https://github.com/apache/storm/blob/1.x-branch/DEVELOPER.md#create-a-storm-distribution-packaging
.

2018-06-26 21:14 GMT+02:00 Alexandre Vermeerbergen :

> Hello,
>
> I am looking for a simple way to validate couple of JIRAs marked as
> resolved on Apache Storm 1.2.3 with our large topologies & setup.
>
> Is there a way to access binaries from a snapshot build, and/or the
> latest 1.2.3 sources ?
>
> What would be the URLs to retrieves either binary artifacts or sources
> corresponding to latest commits in 1.2.3 branch ?
>
> Note: I have checked on http://storm.apache.org/downloads.html but I
> found no clue about where to find such items.
>
> Best regards,
> Alexandre Vermeerbergen
>

Re: Request to update Apache Storm "Powered By" page [was: CrowdFlower - Figure Eight Link Update]

[Stig Rohde Døssing]

Hi Anna,

Yes, we can definitely update the site. Should I just replace the
CrowdFlower company name and update the links, or would you like to provide
a new blurb for your company?

2018-05-16 17:30 GMT+02:00 Sally Khudairi :

> Thank you, Anna.
>
> I'm copying the Apache Storm team on this message for their attention.
>
> Kind regards,
> Sally
>
> - - -
> Vice President Marketing & Publicity
> The Apache Software Foundation
>
> Tel +1 617 921 8656
> Skype sallykhudairi
>
>
> On Wed, May 16, 2018, at 08:41, Anna Davis wrote:
> > Hi there,
> >
> >  I was reaching back out to see if you could help me out with updating
> >  Figure Eight's (formerly CrowdFlower) link on your
> >  http://storm.apache.org/Powered-By.html page?>
> >
> >  Please see the details below:
> >
> >
> >  Thanks,
> >
> >  Anna Davis
> >  On Thu, May 10, 2018 at 5:21 PM, Anna Davis
> >   wrote:>> Hi there,
> >>
> >> I am Anna, a member of Figure Eight’s digital marketing team. I
> >> noticed that your http://storm.apache.org/Powered-By.html Page has
> >> them listed under their old domain and brand name. Good news!
> >> CrowdFlower has undergone a rebrand as well as a domain change to
> >> https://www.figure-eight.com/.>>
> >> We are very excited for the transition and we would appreciate it if
> >> you are willing to update your site to reflect our new domain please
> >> https://www.figure-eight.com/.>>
> >> Let me know if you are able to do so or if we need to figure out
> >> another solution for updating your site to reflect our new domain.>>
> >> Thanks!
> >>
> >> They are found on the following page
> >>  http://storm.apache.org/Powered-By.html
> >>
> >>
> >>  Anna Davis
> >>
> >>
> >> Don't want emails from us anymore? Reply to this email with the word
> >> "UNSUBSCRIBE" in the subject line.>
> >
> >
> > Don't want emails from us anymore? Reply to this email with the word
> > "UNSUBSCRIBE" in the subject line.
>