Re: Licensing of the Redis Module

[Alexandre Vermeerbergen]

Hello,

Yes Redis server licensing became non-OSS friendly, but I don't
understand what would be Storm's dependency on Redis server : don't we
we just need a Redis client, such as Jedis (this later has a MIT
License) ?

Alex

Le lun. 13 mai 2024 à 11:51, Richard Zowalla  a écrit :
>
> Hi all,
>
> REDIS switched it's license, which isn't compatible anymore [1]
>
> We have two options here (imho):
>
> (1) Drop the REDIS module
> (2) Migrate to the OSS fork: https://github.com/valkey-io/valkey
>
>
> WDYT?
>
> Gruß
> Richard
>
> [1] https://redis.io/blog/redis-adopts-dual-source-available-licensing/

Re: [RESULT] [VOTE] Apache Storm 2.6.2 (rc1)

[Alexandre Vermeerbergen]

Thanks a lot Richard for this very smooth & efficient release !

Alexandre

Le ven. 5 avr. 2024 à 19:37, Richard Zowalla  a écrit :
>
> Hi,
>
> this vote passes with the following votes:
>
> +1 Julien Nioche (binding)
> +1 Alexandre Vermeerbergen (binding)
> +1 Richard Zowalla (binding)
>
> I'll proceed with the steps. Thx for your time!
>
> Gruß
> Richard

Re: [VOTE] Apache Storm 2.6.2 (rc1)

[Alexandre Vermeerbergen]

+1 (binding)

tested on my pre-production Storm cluster with 15 topologies on 6
supervisor nodes (and 3 zookeeper nodes), since all topologies relying
on Apache Kafka topics (with kafka-monitoring showing lags per
partitions & per topic in Storm UI, this time it's not broken ;) and
some topologies relying on old Storm HDFS stuff. All this powevered by
IBM Semeru 21 (aka OpenJ9 JVM, at Java runtime 21, but code is still
built with a JDK17) on Redhat Linux 8.

I have observed no functional regression, topologies global latency is
same as with Storm 2.6.1, same for Supervision CPU/Memory/IOPS usage.

Disclaimer : I haven't yet updated built-time dependencies of
topologies, whose "BigJars" are still relying on Storm 2.6.1
dependencies (but that's good to check that topologies built from
version N-1 still run with Storm version N, for those willing to do
"rolling upgrades".

Many thanks Richard for the great work at providing this release for votes!

Alexandre

Le mer. 3 avr. 2024 à 09:53, Julien Nioche
 a écrit :
>
> Thanks Richard
>
> +1 (binding) Release this package as Apache Storm 2.6.2
>
>
>- checked the signatures for the release candidate
>- successfully built Storm from the source
>- ran a local and distributed topology
>
>
> On Tue, 2 Apr 2024 at 09:32, Richard Zowalla  wrote:
>
> > Hi all,
> >
> > I have posted a first release candidate for Apache Storm 2.6.2 and it
> > is ready for testing.
> >
> > We upgraded libraries and pruned some dependencies which came in as
> > transient dependencies.
> >
> > Full list of changes in this release:
> >
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.6.2-rc1/RELEASE_NOTES.html
> >
> >
> > The Nexus staging repository is here:
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-/
> >
> > Storm Source and Binary Release with sha512 signature files are here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.6.2-rc1/
> >
> > The release artifacts are signed with my key contained in this file:
> >
> > https://dist.apache.org/repos/dist/release/storm/KEYS
> >
> > The release was made from the Apache Storm v2.6.2 tag at:
> >
> > https://github.com/apache/storm/releases/tag/v2.6.2
> >
> >
> > To use it in a maven build set the version for Storm to 2.6.2 and add
> > the following to your POM:
> >
> >
> > 
> > 
> > storm-2.6.2-rc1
> > Testing Storm 2.6.2 RC1 
> > 
> > https://repository.apache.org/content/repositories/orgapachestorm-/
> >
> > 
> > 
> > 
> >
> >
> > The release was made using the Storm release process, documented on the
> > GitHub repository:
> > https://github.com/apache/storm/blob/master/RELEASING.md
> >
> > Please vote on releasing these packages as Apache Storm 2.6.2.
> >
> > The vote is open for at least the next 72 hours or as long as needed.
> >
> > "How to vote" is described here:
> >
> > https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> >
> > When voting, please list the actions taken to verify the release.
> >
> > Only votes from the Storm PMC are binding, but everyone is welcome to
> > check the release candidate and vote.
> >
> > The vote passes if at least three binding +1 votes are cast.
> >
> > [ ] +1 Release this package as Apache Storm 2.6.2
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > Thanks!
> >
> > Richard
> >
> >
> >
>
> --
> *Julien Nioche *
>
>
> digitalpebble.com 

Re: [DISCUSS] Release 2.6.2 ?

[Alexandre Vermeerbergen]

+1 and I commit that I will test 2.6.2RC with by large set of
topologies on my preproduction !

Alexandre

Le jeu. 28 mars 2024 à 15:45, Julien Nioche
 a écrit :
>
> +1 from me
>
>
> On Thu, 28 Mar 2024 at 13:58, Richard Zowalla  wrote:
>
> > Hi,
> >
> > any objections in doing a 2.6.2 after eastern?
> >
> > We have some CVE fixes, so might be worth making a small update?
> >
> > WDYT?
> >
> > Best
> > Richard
> >
>
>
> --
> *Julien Nioche *
>
>
> digitalpebble.com 

Re: [Discuss] Move Storm from JIRA to GitHub Issues

[Alexandre Vermeerbergen]

+1 I agree that JIRA is overkill for the simple needs of incidents /
enhancements tracking of Apache Storm

Alexandre

Le jeu. 22 févr. 2024 à 12:18, Richard Zowalla  a écrit :
>
> Hi all,
>
> The ASF JIRA instance no longer allows users to self-sign-up (though it
> DOES allow us to invite others). This is largely due to spam.
> Most developers have a GitHub account (and that is where most folks
> interact with Storm's code base), and this would meet folks where they
> are, as opposed to making them go somewhere else to dig through issues.
>
> Tentative suggestion:
>
> * Go through existing Storm JIRA issues and close any issues that are
> no longer relevant.
> * Migrate open issues to GitHub Issues in apache/storm (linking back to
> the
> JIRA issue)
> * All new issues will only be created in GitHub Issues
> * When an existing JIRA issue is fixed, it will be updated in both
> locations
> * Make JIRA read-only for normal users
>
> Potential process changes:
> * Release notes/change logs are currently generated through JIRA. We
> would need to change that.
>
> NOTE: This is not final; the above text is just to start the
> discussion.
>
> Feel free to pick holes in the above, suggest changes, or propose
> something else!
>
> Thoughts and feedback are welcome!
> Richard

Re: [VOTE] Release Apache Storm 2.6.1 (RC1)

[Alexandre Vermeerbergen]

[+1] binding

Tested this 2.6.1 RC1 using binary artifacts on my Storm preproduction
cluster (6 Supervisor VMs) after having rebuilt my 14 Storm topologies
with updated 2.6.1 RC1 dependencies (including storm-kafka-client and
storm-hdfs), on Redhat 8.7 Linux x64 and with IBM Semeru JDK  21.0.1
(aka OpenJ9 0.42.0)
=> everything runs fine compared to same setup with Storm 2.6.0, VM
resources (cpu, memory,iops) usage on Supervisors is quite the same as
usual
=> the very useful fix of [STORM-4025] - ClassCastException when
changing log level in Storm UI is fine with 2.6.1

Tthanks for this release !

Alexandre

Le mer. 31 janv. 2024 à 16:24, Julien Nioche
 a écrit :
>
> +1 to release this package as Apache Storm 2.6.1 (binding)
>
> Checked the *.asc and *.sha512
> Built from the source code and ran a StormCrawler topology successfully
>
> Thanks
>
> Julien
>
> On Tue, 30 Jan 2024 at 08:42, Richard Zowalla  wrote:
>
> > Hi ladies & lords,
> >
> > I have posted a first release candidate for Apache Storm 2.6.1 release
> > and it is ready for testing.
> >
> > We fixed a few bugs and upgraded libraries.
> >
> > Full list of changes in this release:
> >
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.6.1-rc1/RELEASE_NOTES.html
> >
> >
> > The Nexus staging repository is here:
> >
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1110/
> >
> > Storm Source and Binary Release with sha512 signature files are here:
> >
> > The release artifacts are signed with my key contained in this file:
> >
> > https://dist.apache.org/repos/dist/release/storm/KEYS
> >
> > The release was made from the Apache Storm v2.6.1 tag at:
> >
> > https://github.com/apache/storm/releases/tag/v2.6.1
> >
> >
> > To use it in a maven build set the version for Storm to 2.6.1 and add
> > the following to your POM:
> >
> >
> > 
> > 
> > storm-2.6.1-rc1
> > Testing Storm 2.6.1 RC1 
> > 
> > https://repository.apache.org/content/repositories/orgapachestorm-1110/
> >
> > 
> > 
> > 
> >
> >
> > The release was made using the Storm release process, documented on the
> > GitHub repository:
> > https://github.com/apache/storm/blob/master/RELEASING.md
> >
> > Please vote on releasing these packages as Apache Storm 2.6.0.
> >
> > The vote is open for at least the next 72 hours or as long as needed.
> >
> > "How to vote" is described here:
> >
> > https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> >
> > When voting, please list the actions taken to verify the release.
> >
> > Only votes from the Storm PMC are binding, but everyone is welcome to
> > check the release candidate and vote.
> >
> > The vote passes if at least three binding +1 votes are cast.
> >
> > [ ] +1 Release this package as Apache Storm 2.6.1
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > Thanks!
> >
> > Richard
> >
> >
>
> --
> *Julien Nioche *
>
>
> digitalpebble.com 

Re: [VOTE] Release Apache Storm 2.6.1 (RC1)

[Alexandre Vermeerbergen]

Thanks for the clarification
=> I have upgraded my preproduction cluster with this 2.6.1
RC1,rebuilt all my topologies with 2.6.1 dependencies and deployed on
this preproduction cluster
=> so far sounds good, but I want to let it run at least for 24 hours
"at scale" (with a full day of processed data) & review the resulting
monitoring (on topologies métrics + supporting VMs cpu/memory/iops)
before casting my vote

Thanks,
Alexandre

Le mar. 30 janv. 2024 à 20:05, Richard Zowalla  a écrit :
>
> Hi,
>
> we do not ship "storm-kafka-client-*" since a few versions.
> It isn't contained in 2.3.0, 2.4.0, 2.5.0 and 2.6.0 (didn't check the
> others), so yes, it is intentional. (It is a collection of spouts /
> bolts, trident - so more or less something a user may want to use (or
> not)).
>
> The JAR itself is available via Maven:
>
> https://repository.apache.org/content/repositories/orgapachestorm-1110/org/apache/storm/storm-kafka-client/2.6.1/
>
> Hope it helps
> Richard
>
>
> Am Dienstag, dem 30.01.2024 um 16:02 +0100 schrieb Alexandre
> Vermeerbergen:
> > Other question: to test
> > https://issues.apache.org/jira/browse/STORM-4016 I would like to have
> > storm-kafka-client--2.6.1.jar, but it's not in
> > apache-storm-2.6.1.tar.gz archive (binary artifacts)
> > => is that intentional ?
> > => from where can I find torm-kafka-client--2.6.1.jar ?
> > (sorry if my questions sound trivial..)
> >
> > Alexandre
> >
> > Le mar. 30 janv. 2024 à 14:12, Richard Zowalla 
> > a écrit :
> > >
> > > Here is the missing link. It is located at the usual location:
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.6.1-rc1/
> > >
> > > Thx for the hint!
> > >
> > >
> > > Am Dienstag, dem 30.01.2024 um 14:09 +0100 schrieb Alexandre
> > > Vermeerbergen:
> > > > Hi Richard,
> > > >
> > > > The link to "Storm Source and Binary Release with sha512
> > > > signature
> > > > files" is missing, can you please re-send the vote mail ?
> > > >
> > > > Thanks,
> > > > Alexandre
> > > >
> > > > Le mar. 30 janv. 2024 à 09:42, Richard Zowalla 
> > > > a
> > > > écrit :
> > > > >
> > > > > Hi ladies & lords,
> > > > >
> > > > > I have posted a first release candidate for Apache Storm 2.6.1
> > > > > release
> > > > > and it is ready for testing.
> > > > >
> > > > > We fixed a few bugs and upgraded libraries.
> > > > >
> > > > > Full list of changes in this release:
> > > > >
> > > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.6.1-rc1/RELEASE_NOTES.html
> > > > >
> > > > >
> > > > > The Nexus staging repository is here:
> > > > >
> > > > >
> > > > > https://repository.apache.org/content/repositories/orgapachestorm-1110/
> > > > >
> > > > > Storm Source and Binary Release with sha512 signature files are
> > > > > here:
> > > > >
> > > > > The release artifacts are signed with my key contained in this
> > > > > file:
> > > > >
> > > > > https://dist.apache.org/repos/dist/release/storm/KEYS
> > > > >
> > > > > The release was made from the Apache Storm v2.6.1 tag at:
> > > > >
> > > > > https://github.com/apache/storm/releases/tag/v2.6.1
> > > > >
> > > > >
> > > > > To use it in a maven build set the version for Storm to 2.6.1
> > > > > and
> > > > > add
> > > > > the following to your POM:
> > > > >
> > > > >
> > > > > 
> > > > > 
> > > > > storm-2.6.1-rc1
> > > > > Testing Storm 2.6.1 RC1 
> > > > > 
> > > > > https://repository.apache.org/content/repositories/orgapachestorm
> > > > > -
> > > > > 1110/
> > > > >
> > > > > 
> > > > > 
> > > > > 
> > > > >
> > > > >
> > > > > The release was made using the Storm release process,
> > > > > documented on
> > > > > the
> > > > > GitHub repository:
> > > > > https://github.com/apache/storm/blob/master/RELEASING.md
> > > > >
> > > > > Please vote on releasing these packages as Apache Storm 2.6.0.
> > > > >
> > > > > The vote is open for at least the next 72 hours or as long as
> > > > > needed.
> > > > >
> > > > > "How to vote" is described here:
> > > > > https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> > > > >
> > > > > When voting, please list the actions taken to verify the
> > > > > release.
> > > > >
> > > > > Only votes from the Storm PMC are binding, but everyone is
> > > > > welcome
> > > > > to
> > > > > check the release candidate and vote.
> > > > >
> > > > > The vote passes if at least three binding +1 votes are cast.
> > > > >
> > > > > [ ] +1 Release this package as Apache Storm 2.6.1
> > > > > [ ]  0 No opinion
> > > > > [ ] -1 Do not release this package because...
> > > > >
> > > > > Thanks to everyone who contributed to this release.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Richard
> > > > >
> > >
>

Re: [VOTE] Release Apache Storm 2.6.1 (RC1)

[Alexandre Vermeerbergen]

Other question: to test
https://issues.apache.org/jira/browse/STORM-4016 I would like to have
storm-kafka-client--2.6.1.jar, but it's not in
apache-storm-2.6.1.tar.gz archive (binary artifacts)
=> is that intentional ?
=> from where can I find torm-kafka-client--2.6.1.jar ?
(sorry if my questions sound trivial..)

Alexandre

Le mar. 30 janv. 2024 à 14:12, Richard Zowalla  a écrit :
>
> Here is the missing link. It is located at the usual location:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.6.1-rc1/
>
> Thx for the hint!
>
>
> Am Dienstag, dem 30.01.2024 um 14:09 +0100 schrieb Alexandre
> Vermeerbergen:
> > Hi Richard,
> >
> > The link to "Storm Source and Binary Release with sha512 signature
> > files" is missing, can you please re-send the vote mail ?
> >
> > Thanks,
> > Alexandre
> >
> > Le mar. 30 janv. 2024 à 09:42, Richard Zowalla  a
> > écrit :
> > >
> > > Hi ladies & lords,
> > >
> > > I have posted a first release candidate for Apache Storm 2.6.1
> > > release
> > > and it is ready for testing.
> > >
> > > We fixed a few bugs and upgraded libraries.
> > >
> > > Full list of changes in this release:
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.6.1-rc1/RELEASE_NOTES.html
> > >
> > >
> > > The Nexus staging repository is here:
> > >
> > >
> > > https://repository.apache.org/content/repositories/orgapachestorm-1110/
> > >
> > > Storm Source and Binary Release with sha512 signature files are
> > > here:
> > >
> > > The release artifacts are signed with my key contained in this
> > > file:
> > >
> > > https://dist.apache.org/repos/dist/release/storm/KEYS
> > >
> > > The release was made from the Apache Storm v2.6.1 tag at:
> > >
> > > https://github.com/apache/storm/releases/tag/v2.6.1
> > >
> > >
> > > To use it in a maven build set the version for Storm to 2.6.1 and
> > > add
> > > the following to your POM:
> > >
> > >
> > > 
> > > 
> > > storm-2.6.1-rc1
> > > Testing Storm 2.6.1 RC1 
> > > 
> > > https://repository.apache.org/content/repositories/orgapachestorm-
> > > 1110/
> > >
> > > 
> > > 
> > > 
> > >
> > >
> > > The release was made using the Storm release process, documented on
> > > the
> > > GitHub repository:
> > > https://github.com/apache/storm/blob/master/RELEASING.md
> > >
> > > Please vote on releasing these packages as Apache Storm 2.6.0.
> > >
> > > The vote is open for at least the next 72 hours or as long as
> > > needed.
> > >
> > > "How to vote" is described here:
> > > https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> > >
> > > When voting, please list the actions taken to verify the release.
> > >
> > > Only votes from the Storm PMC are binding, but everyone is welcome
> > > to
> > > check the release candidate and vote.
> > >
> > > The vote passes if at least three binding +1 votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Storm 2.6.1
> > > [ ]  0 No opinion
> > > [ ] -1 Do not release this package because...
> > >
> > > Thanks to everyone who contributed to this release.
> > >
> > > Thanks!
> > >
> > > Richard
> > >
>

Re: [VOTE] Release Apache Storm 2.6.1 (RC1)

[Alexandre Vermeerbergen]

Hi Richard,

The link to "Storm Source and Binary Release with sha512 signature
files" is missing, can you please re-send the vote mail ?

Thanks,
Alexandre

Le mar. 30 janv. 2024 à 09:42, Richard Zowalla  a écrit :
>
> Hi ladies & lords,
>
> I have posted a first release candidate for Apache Storm 2.6.1 release
> and it is ready for testing.
>
> We fixed a few bugs and upgraded libraries.
>
> Full list of changes in this release:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.6.1-rc1/RELEASE_NOTES.html
>
>
> The Nexus staging repository is here:
>
>
> https://repository.apache.org/content/repositories/orgapachestorm-1110/
>
> Storm Source and Binary Release with sha512 signature files are here:
>
> The release artifacts are signed with my key contained in this file:
>
> https://dist.apache.org/repos/dist/release/storm/KEYS
>
> The release was made from the Apache Storm v2.6.1 tag at:
>
> https://github.com/apache/storm/releases/tag/v2.6.1
>
>
> To use it in a maven build set the version for Storm to 2.6.1 and add
> the following to your POM:
>
>
> 
> 
> storm-2.6.1-rc1
> Testing Storm 2.6.1 RC1 
> https://repository.apache.org/content/repositories/orgapachestorm-1110/
>
> 
> 
> 
>
>
> The release was made using the Storm release process, documented on the
> GitHub repository:
> https://github.com/apache/storm/blob/master/RELEASING.md
>
> Please vote on releasing these packages as Apache Storm 2.6.0.
>
> The vote is open for at least the next 72 hours or as long as needed.
>
> "How to vote" is described here:
> https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
>
> When voting, please list the actions taken to verify the release.
>
> Only votes from the Storm PMC are binding, but everyone is welcome to
> check the release candidate and vote.
>
> The vote passes if at least three binding +1 votes are cast.
>
> [ ] +1 Release this package as Apache Storm 2.6.1
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>
> Thanks!
>
> Richard
>

Re: Storm 2.6.1 ?

[Alexandre Vermeerbergen]

I also updated my preproduction 10+ topologies with Storm snapshot
generated 20240128
=> everything runs fine and I confirm STORM-4025 is fixed !

Alex

Le lun. 29 janv. 2024 à 14:37, Julien Nioche
 a écrit :
>
> Hi,
>
> Ran a StormCrawler toplogy with a nightly - looking good, no issues found
> Can't see any outstanding PRs to merge in the short term so would +1 to
> releasing what we currently have as 2.6.1
>
> Thanks
>
> Julien
>
> On Fri, 26 Jan 2024 at 11:59, Julien Nioche 
> wrote:
>
> > +1 to a 2.6.1 release
> >
> > The nightlies are at https://nightlies.apache.org/storm/
> > A couple of changes were merged today, I'll try a nightly next week
> >
> > J
> >
> > On Fri, 26 Jan 2024 at 07:40, Richard Zowalla  wrote:
> >
> >> Hi all,
> >>
> >> we had some fixes and dependency updates in Storm since our last
> >> release.
> >>
> >> Any objections in going for a 2.6.1 soon? Any lib upgrades we need to
> >> include prior?
> >>
> >> Did someone test our nightlies already?
> >>
> >>
> >> Gruß
> >> Richard
> >>
> >
> >
> > --
> > *Julien Nioche *
> >
> >
> > digitalpebble.com 
> >
>
>
> --
> *Julien Nioche *
>
>
> digitalpebble.com 

Re: Community over Code EU 2024 Travel Assistance Applications now open!

[Alexandre Vermeerbergen]

Hi Julien,

It's very nice that your submitted a Sorm-related topic for this
conference, I wish your proposal to be accepted !

Unfortunately I won't be able to travel at that time, I hope to see
you in broadcasts or records of your future session (betting it'll be
accepted :)

Alex

Le sam. 27 janv. 2024 à 09:50, Julien Nioche
 a écrit :
>
> Dear Apache Storm community
>
> Below is a message from the Travel Assistance Committee (TAC) to announce
> that
> travel assistance applications for Community over Code EU 2024 are now open!
>
> I will be at the conference if my presentation on StormCrawler and Apache
> Storm is accepted.
> Anyone using or contributing to Apache Storm planning to attend?
>
> Cheers
>
> Julien
>
> ***
>
> We will be supporting Community over Code EU, Bratislava, Slovakia,
> June 3th - 5th, 2024.
>
> TAC exists to help those that would like to attend Community over Code
> events, but are unable to do so for financial reasons. For more info
> on this years applications and qualifying criteria, please visit the
> TAC website at < https://tac.apache.org/ >. Applications are already
> open on https://tac-apply.apache.org/, so don't delay!
>
> The Apache Travel Assistance Committee will only be accepting
> applications from those people that are able to attend the full event.
>
> Important: Applications close on Friday, March 1st, 2024.
>
> Applicants have until the the closing date above to submit their
> applications (which should contain as much supporting material as
> required to efficiently and accurately process their request), this
> will enable TAC to announce successful applications shortly
> afterwards.
>
> As usual, TAC expects to deal with a range of applications from a
> diverse range of backgrounds; therefore, we encourage (as always)
> anyone thinking about sending in an application to do so ASAP.
>
> We look forward to greeting many of you in Bratislava, Slovakia in June,
> 2024!
>
> Kind Regards,
>
> Chris
>
> (On behalf of the Travel Assistance Committee)
>
> When replying, please reply to travel-assista...@apache.org
>
> --
> *Julien Nioche *
> Director
>
>
> digitalpebble.com 

Re: Storm 2.6.1 ?

[Alexandre Vermeerbergen]

I have no objection for such Storm 2.7.0 to Storm 2.6.1 renaming !
(so then nightly build shall automatically lead to 2.6.1 one renaming is done ?)

Le ven. 26 janv. 2024 à 09:50, Richard Zowalla  a écrit :
>
> Hi Alexandre,
>
> I think, that we can rename the milestone/version from 2.7.0 to 2.6.1
> as we do not have any public facing breaking changes in the API
> (afaik). So instead of going 2.7.0, we can just do a 2.6.1 (imho).
>
> Gruß
> Richard
>
> Am Freitag, dem 26.01.2024 um 09:48 +0100 schrieb Alexandre
> Vermeerbergen:
> > Hello Richard,
> >
> > +1, you must read my mind :)
> >
> > I am especially interest to get
> > https://issues.apache.org/jira/browse/STORM-4025 backported to 2.6.1
> > ;
> > and your recent dependency update won Kryo, please.
> >
> > Kind regards,
> > ALexandre
> >
> > Le ven. 26 janv. 2024 à 08:40, Richard Zowalla  a
> > écrit :
> > >
> > > Hi all,
> > >
> > > we had some fixes and dependency updates in Storm since our last
> > > release.
> > >
> > > Any objections in going for a 2.6.1 soon? Any lib upgrades we need
> > > to
> > > include prior?
> > >
> > > Did someone test our nightlies already?
> > >
> > >
> > > Gruß
> > > Richard
>

Re: Storm 2.6.1 ?

[Alexandre Vermeerbergen]

Hello Richard,

+1, you must read my mind :)

I am especially interest to get
https://issues.apache.org/jira/browse/STORM-4025 backported to 2.6.1 ;
and your recent dependency update won Kryo, please.

Kind regards,
ALexandre

Le ven. 26 janv. 2024 à 08:40, Richard Zowalla  a écrit :
>
> Hi all,
>
> we had some fixes and dependency updates in Storm since our last
> release.
>
> Any objections in going for a 2.6.1 soon? Any lib upgrades we need to
> include prior?
>
> Did someone test our nightlies already?
>
>
> Gruß
> Richard

Re: [VOTE] Docker images for Apache Storm

[Alexandre Vermeerbergen]

Hello,

I would have voted [0] if possible, but only [+1] or [-1] are proposed ;)

Indeed, regarding Docker images, I tend to be reluctant to use
community images appart from OS base ones and built mine based on my
actual needs which are:
1. To use a specific JVM (I'm more found of OpenJ9 - aka IBM Semeru
JRE than HotSpot - aka no matter which)
2. To package only what's needed in my Docker images. Given that the
best practive in the word of containers in to have one process per
container, I would build a image with only Storm process, another one
with only Storm UI, etc (one tickly case is how to mix supervisor and
logview ; and by the way worker processes should be spawned by Storm
process acting as a k8s operator, to be really "cloud native" with k8s
stack...)

If the goal of the Docker image for which the vote is ran is to
provide an all-in-one image embedding all Storm processes, then I'm
neutral on it, it could help newbies in evaluating Storm...

Given that I'm so undecided, I just confirm [0]
(sorry for putting a mess in this vote)

Alexandre

Le mar. 12 déc. 2023 à 08:31, Richard Zowalla  a écrit :
>
> +1 (think it would be good to "officially" be responsible and
> distribute those on our own instead of relying on 3rd party repos)
>
> Am Montag, dem 11.12.2023 um 14:57 + schrieb Julien Nioche:
> > Dear Storm community,
> >
> > There was a recent discussion on the dev list [1] about a roadmap for
> > the
> > next release (major or minor). One of the items was to release Docker
> > images for Apache Storm ourselves. There are Docker images but
> > maintained
> > by a third party [2]. By managing the images ourselves, we would have
> > more
> > control on when the images are available and also be able to fix
> > issues
> > quicker.
> >
> > The first step is to ask Infra to give our project seats on DockerHub
> > [3].
> >
> > Could we have a quick vote to make sure everyone is happy to go ahead
> > with this?
> >
> > *[+1] publish Docker images ourselves and ask infra for seats to do
> > so on
> > Dockerhub*
> >
> > *or *
> >
> > *[-1] do not publish Docker images ourselves?*
> >
> > The vote is open for the next 72 hours and will finish on Thursday,
> > December 14 at  3PM UTC.
> >
> > Best regards,
> >
> > Julien Nioche
> >
> >
> > [1] https://lists.apache.org/thread/g78xg0k1m86m3o9jtj4g6p4cvvz3dhgr
> > [2] https://github.com/31z4/storm-docker
> > [3] https://infra.apache.org/docker-hub-policy.html
>

Re: Roadmap Apache Storm 3.x

[Alexandre Vermeerbergen]

Hello Julien,

Sorry for asking so late, but one detail puzzles my in this
discussion: according to semantic versioning, the change of major
version number is meant to introduce changes breaking compatibility.

Unless I have missed something, I do not see any changes that would
break compatibility from current Storm 2.x.y versions to this future
Storm 3.0.0 version.

Did I miss something?

Kind regards,
Alexandre

Le jeu. 7 déc. 2023 à 15:19, Bipin Prasad
 a écrit :
>
> I agree on the need to fix logviewer functionality so that it works in a 
> Kubernetes deployment (eg AWS EKS or GKE).
>
>
> Sent from Yahoo Mail for iPhone
>
>
> On Thursday, December 7, 2023, 4:26 AM, Richard Zowalla  
> wrote:
>
> It might be possible to build an abstraction layer + java 21-based
> module, so ppl can decide (for ex by configuration) to switch to it or
> not. If you do not activley decide to use it, it won't be a difference
> to how it is currently working. Might need some carefully thinking and
> some eyes on other projects.
>
> Am Donnerstag, dem 07.12.2023 um 13:24 +0100 schrieb Alexandre
> Vermeerbergen:
> > Agreed for leaving fiber projet aside from 3.x
> >
> > Re-logviewer : how about my idea to use storm-ui's app server and
> > write some kind of proxy servlet to make all traffic related to logs
> > retrieval going to storm UI ?
> >
> > Le jeu. 7 déc. 2023 à 13:06, Julien Nioche
> >  a écrit :
> > >
> > > Thanks Alexandre,
> > >
> > > I haven't looked nor know anything about fibers but it sounds
> > > promising.
> > > Forcing Java 21 would be problematic I think.
> > > Given that it is a fundamental change, maybe this could live in a
> > > separate
> > > branch until thoroughly tested and Java 21 is the norm. What do you
> > > think?
> > >
> > > Re-logviewer: it is something that has been bugging people for a
> > > while and
> > > that a lot of us would like to see fixed I think. Sadly, I don't
> > > have a
> > > clue on how to do it.
> > >
> > > Julien
> > >
> > > On Thu, 7 Dec 2023 at 11:01, Alexandre Vermeerbergen <
> > > avermeerber...@gmail.com> wrote:
> > >
> > > > Hello Julien,
> > > >
> > > > Excellent initiative indeed !
> > > >
> > > > I am very fond of "Find a way of having the Logviewer work in
> > > > Docker"
> > > > topic, because I think it's similar to a shorter time need which
> > > > I
> > > > have when deploying AWS EC2 with a VPN to avoid exposure of
> > > > "internal
> > > > VMs" : in this case, the public FQDN of Supervisor VMs is not
> > > > usable
> > > > for logviewer, so i'm trying to figure out a way to do some
> > > > proxying
> > > > through Nimbus/Nimbus UI process... it's only brainstorming at
> > > > the
> > > > moment.
> > > >
> > > > I also have recently  launched the idea to use Java fibers and
> > > > got a
> > > > positive feedback from Bipin, but I have no idea on whether it
> > > > could
> > > > be affordable for 3.x. One problem being is that unless we un-
> > > > support
> > > > Storm on Java < 21, we can't just build everything using JDK21 to
> > > > get
> > > > access to fibers API...
> > > >
> > > > That were my 2 cents...
> > > >
> > > > Alexandre
> > > >
> > > >
> > > >
> > > > Le jeu. 7 déc. 2023 à 10:43, Julien Nioche
> > > >  a écrit :
> > > > >
> > > > > Hi,
> > > > >
> > > > > Now that Storm is active again and we are getting regular
> > > > > releases,
> > > > should
> > > > > we try to come up with a roadmap for the Storm 3.x?
> > > > >
> > > > > Here are a few things that come to mind
> > > > >
> > > > >- Finish the migration from Clojure to Java (it is still
> > > > > used in the
> > > > >tests I think)
> > > > >- Find a way of having the Logviewer work in Docker
> > > > ><https://issues.apache.org/jira/browse/STORM-1759>
> > > > >- Publish the Docker images ourselves
> > > > >- UI Refresh (at least the libs) - suggested by R. Zowalla
> > > > >- Fix Maven build  - suggested by R. Zowalla
> > > > >
> > > > > Anything else? Who would like to get involved? Where would be a
> > > > > good
> > > > place
> > > > > to formalise a roadmap? JIRA? GitHub projects
> > > > > <https://github.com/apache/storm/projects?query=is%3Aopen>?
> > > > >
> > > > > Best regards
> > > > >
> > > > > --
> > > > > *Julien Nioche *
> > > > >
> > > > >
> > > > > digitalpebble.com <http://www.digitalpebble.com/>
> > > >
> > >
> > >
> > > --
> > > *Julien Nioche *
> > >
> > >
> > > digitalpebble.com <http://www.digitalpebble.com/>
>
>
>
>

Re: Roadmap Apache Storm 3.x

[Alexandre Vermeerbergen]

Agreed for leaving fiber projet aside from 3.x

Re-logviewer : how about my idea to use storm-ui's app server and
write some kind of proxy servlet to make all traffic related to logs
retrieval going to storm UI ?

Le jeu. 7 déc. 2023 à 13:06, Julien Nioche
 a écrit :
>
> Thanks Alexandre,
>
> I haven't looked nor know anything about fibers but it sounds promising.
> Forcing Java 21 would be problematic I think.
> Given that it is a fundamental change, maybe this could live in a separate
> branch until thoroughly tested and Java 21 is the norm. What do you think?
>
> Re-logviewer: it is something that has been bugging people for a while and
> that a lot of us would like to see fixed I think. Sadly, I don't have a
> clue on how to do it.
>
> Julien
>
> On Thu, 7 Dec 2023 at 11:01, Alexandre Vermeerbergen <
> avermeerber...@gmail.com> wrote:
>
> > Hello Julien,
> >
> > Excellent initiative indeed !
> >
> > I am very fond of "Find a way of having the Logviewer work in Docker"
> > topic, because I think it's similar to a shorter time need which I
> > have when deploying AWS EC2 with a VPN to avoid exposure of "internal
> > VMs" : in this case, the public FQDN of Supervisor VMs is not usable
> > for logviewer, so i'm trying to figure out a way to do some proxying
> > through Nimbus/Nimbus UI process... it's only brainstorming at the
> > moment.
> >
> > I also have recently  launched the idea to use Java fibers and got a
> > positive feedback from Bipin, but I have no idea on whether it could
> > be affordable for 3.x. One problem being is that unless we un-support
> > Storm on Java < 21, we can't just build everything using JDK21 to get
> > access to fibers API...
> >
> > That were my 2 cents...
> >
> > Alexandre
> >
> >
> >
> > Le jeu. 7 déc. 2023 à 10:43, Julien Nioche
> >  a écrit :
> > >
> > > Hi,
> > >
> > > Now that Storm is active again and we are getting regular releases,
> > should
> > > we try to come up with a roadmap for the Storm 3.x?
> > >
> > > Here are a few things that come to mind
> > >
> > >- Finish the migration from Clojure to Java (it is still used in the
> > >tests I think)
> > >- Find a way of having the Logviewer work in Docker
> > ><https://issues.apache.org/jira/browse/STORM-1759>
> > >- Publish the Docker images ourselves
> > >- UI Refresh (at least the libs) - suggested by R. Zowalla
> > >- Fix Maven build  - suggested by R. Zowalla
> > >
> > > Anything else? Who would like to get involved? Where would be a good
> > place
> > > to formalise a roadmap? JIRA? GitHub projects
> > > <https://github.com/apache/storm/projects?query=is%3Aopen>?
> > >
> > > Best regards
> > >
> > > --
> > > *Julien Nioche *
> > >
> > >
> > > digitalpebble.com <http://www.digitalpebble.com/>
> >
>
>
> --
> *Julien Nioche *
>
>
> digitalpebble.com <http://www.digitalpebble.com/>

Re: Roadmap Apache Storm 3.x

[Alexandre Vermeerbergen]

Hello Julien,

Excellent initiative indeed !

I am very fond of "Find a way of having the Logviewer work in Docker"
topic, because I think it's similar to a shorter time need which I
have when deploying AWS EC2 with a VPN to avoid exposure of "internal
VMs" : in this case, the public FQDN of Supervisor VMs is not usable
for logviewer, so i'm trying to figure out a way to do some proxying
through Nimbus/Nimbus UI process... it's only brainstorming at the
moment.

I also have recently  launched the idea to use Java fibers and got a
positive feedback from Bipin, but I have no idea on whether it could
be affordable for 3.x. One problem being is that unless we un-support
Storm on Java < 21, we can't just build everything using JDK21 to get
access to fibers API...

That were my 2 cents...

Alexandre



Le jeu. 7 déc. 2023 à 10:43, Julien Nioche
 a écrit :
>
> Hi,
>
> Now that Storm is active again and we are getting regular releases, should
> we try to come up with a roadmap for the Storm 3.x?
>
> Here are a few things that come to mind
>
>- Finish the migration from Clojure to Java (it is still used in the
>tests I think)
>- Find a way of having the Logviewer work in Docker
>
>- Publish the Docker images ourselves
>- UI Refresh (at least the libs) - suggested by R. Zowalla
>- Fix Maven build  - suggested by R. Zowalla
>
> Anything else? Who would like to get involved? Where would be a good place
> to formalise a roadmap? JIRA? GitHub projects
> ?
>
> Best regards
>
> --
> *Julien Nioche *
>
>
> digitalpebble.com 

Would Java Fibers (aka Loom Project) be of interest for improving Storm capacities?

[Alexandre Vermeerbergen]

Hello guys,

I have a weird idea bugging my mind, so let me share it and if that's
improductive/dead end, then I'll forget it :)

I have been reading a few sources about the notion of Fibers in Java
(aka Loom Project) which are generally available with Java 21.

There's an optional "virtual threads executor" in Apache Tomcat since
a while: 
https://tomcat.apache.org/tomcat-8.5-doc/config/executor.html#Virtual_Thread_Implementation
 ; would it be worth to have something similar in Apache Storm ?

Note: Storm is a scheduler, so normally the main pain of Java threads
which is that they are OS threads is part of Storm's mission to
schedule a potentially larger set of spouts & bolts than there are
physical threads on all supervisor nodes, right?  does it mean that
virtual threads would be useless for Storm ?

Sorry for pushing headaches to some of you with these questions, but
I'd like to challenge the idea :)

Alexandre

Re: [VOTE] Storm 2.6.0 (RC1)

[Alexandre Vermeerbergen]

Hello,

+1 (binding)

tested with 13 topologies running on a 7 nodes cluster of Supervisors
on Redhat Linux 8.7 with IBM Semeru 17.0.8.1, with dependencies on:
  storm-kafka-client
  storm-kafka-monitoring
  storm-hdfs

and... all dependencies on Java EE (mostly javax.json) migrated on our
side to equivalent jakarta APIs.

We have no regressions with this Storm 2.6.0 candidate release, and
topologies latency is smoother than with previous 2.6.0 snapshots.
CPU/Memory/IOPS usage on Supervisor machine is "as usual", no
particular regressions seen here.

Thanks a lot Richard for pushing this release!

Kind regards,
Alexandre

Le mer. 22 nov. 2023 à 12:06, Richard Zowalla  a écrit :
>
> Here is my own +1
>
> Am Donnerstag, dem 16.11.2023 um 10:19 +0100 schrieb Richard Zowalla:
> > Hi folks,
> >
> > I have posted a first release candidate for the Apache Storm 2.6.0
> > release and it is ready for testing.
> >
> > We updated a lot of dependencies, fixes some bugs and removed some
> > /external modules. We are now also testing with Java 21, which looks
> > promising.
> >
> > The Nexus staging repository is here:
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1109/
> >
> >
> > Storm Source and Binary Release with sha512 signature files are here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.6.0-rc1/
> >
> > The release artifacts are signed with my key contained in this file:
> >
> > https://dist.apache.org/repos/dist/release/storm/KEYS
> >
> > The release was made from the Apache Storm v2.6.0 tag at:
> >
> > https://github.com/apache/storm/releases/tag/v2.6.0
> >
> > Full list of changes in this release:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.6.0-rc1/RELEASE_NOTES.html
> >
> >
> > To use it in a maven build set the version for Storm to 2.6.0 and add
> > the following to your POM:
> >
> >
> > 
> > 
> > storm-2.6.0-rc1
> > Testing Storm 2.6.0 RC1 
> > 
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1109/
> > 
> > 
> > 
> >
> >
> > The release was made using the Storm release process, documented on
> > the
> > GitHub repository:
> > https://github.com/apache/storm/blob/master/RELEASING.md
> >
> > Please vote on releasing these packages as Apache Storm 2.6.0.
> >
> > The vote is open for at least the next 72 hours or as long as needed.
> >
> > "How to vote" is described here:
> > https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> >
> > When voting, please list the actions taken to verify the release.
> >
> > Only votes from the Storm PMC are binding, but everyone is welcome to
> > check the release candidate and vote.
> >
> > The vote passes if at least three binding +1 votes are cast.
> >
> > [ ] +1 Release this package as Apache Storm 2.6.0
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > Thanks!
> >
> > Richard
>

Re: [VOTE] Storm 2.6.0 (RC1)

[Alexandre Vermeerbergen]

Hello,

Yesterday, I have upgraded my pre-production cluster "at scale" with
Storm 2.6.0 RC1, and *this time* this is with all my 10+ topologies
rebuilt using dependencies at same version (not based on previous
2.5.0 dependencies).

So far everything is running smoothly (CPU/IOPS/Memory loads are
smooth, so are topologies latencies), but I want to wait until
mid-next week to give a definitive vote. All issues which I previously
discovered using 2.6.0 snapshots are solved.

Thanks Richard for putting in place this vote !

Kind regards,
Alexandre

Le ven. 17 nov. 2023 à 13:36, Julien Nioche
 a écrit :
>
> Thanks Richard!
>
> +1 (binding) to release
>
>- checked the *.asc, *.sha512
>- built Storm using the code in apache-storm-2.6.0-src.tar.gz
>- ran a complex topology in local and distributed modes
>
> Julien
>
> On Thu, 16 Nov 2023 at 09:21, Richard Zowalla  wrote:
>
> > Hi folks,
> >
> > I have posted a first release candidate for the Apache Storm 2.6.0
> > release and it is ready for testing.
> >
> > We updated a lot of dependencies, fixes some bugs and removed some
> > /external modules. We are now also testing with Java 21, which looks
> > promising.
> >
> > The Nexus staging repository is here:
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1109/
> >
> >
> > Storm Source and Binary Release with sha512 signature files are here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.6.0-rc1/
> >
> > The release artifacts are signed with my key contained in this file:
> >
> > https://dist.apache.org/repos/dist/release/storm/KEYS
> >
> > The release was made from the Apache Storm v2.6.0 tag at:
> >
> > https://github.com/apache/storm/releases/tag/v2.6.0
> >
> > Full list of changes in this release:
> >
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.6.0-rc1/RELEASE_NOTES.html
> >
> >
> > To use it in a maven build set the version for Storm to 2.6.0 and add
> > the following to your POM:
> >
> >
> > 
> > 
> > storm-2.6.0-rc1
> > Testing Storm 2.6.0 RC1 
> > 
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1109/
> > 
> > 
> > 
> >
> >
> > The release was made using the Storm release process, documented on the
> > GitHub repository:
> > https://github.com/apache/storm/blob/master/RELEASING.md
> >
> > Please vote on releasing these packages as Apache Storm 2.6.0.
> >
> > The vote is open for at least the next 72 hours or as long as needed.
> >
> > "How to vote" is described here:
> >
> > https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> >
> > When voting, please list the actions taken to verify the release.
> >
> > Only votes from the Storm PMC are binding, but everyone is welcome to
> > check the release candidate and vote.
> >
> > The vote passes if at least three binding +1 votes are cast.
> >
> > [ ] +1 Release this package as Apache Storm 2.6.0
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > Thanks!
> >
> > Richard
> >
>
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble 

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT (a7aac90fdc4b87eba8dce3b851b55c86c4a390cf)

[Alexandre Vermeerbergen]

Great !
On a side note, my preproduction has been running with 2.6.0 build
20231114 at scale since the last 10 hours, using my usual set:
  RedHat Linux 8
  IBM Java Semeru 17.0.8.1 (aka OpenJ9)  - 6 big machines for Supervisors
  14 topologies (with one having 5 different spouts and 9 bolts, all
with lots of executors, using various field grouping stratefies)
   + all topologies migrated at source level to use Jakarta EE APIs
(like jakarta.json instead of javax.son)

=> all runs smoothly with no regression.  On Supervisor machines, CPU
is even a bit smoother (less peaks) than before this upgrade to 2.6.0
built 2023/11/14.

So from my standpoint, we should be in good shape to have a 2.6.0 release ASAP !

Note: I wish I had time to test all this with Java 21, but to be
realistic I don't think it will happen before next month...

Many thanks Richard for pushing us to try snapshot releases before
entering a vote cycle!

Alexandre


Le mer. 15 nov. 2023 à 16:49, Richard Zowalla  a écrit :
>
> Thx. The related PR is merged.
>
> Am Mittwoch, dem 15.11.2023 um 15:13 +0100 schrieb Richard Zowalla:
> > Yes, feel free to provide a fix / pr + jira issue.
> >
> > It is just a matter of adding
> >
> >  
> >  jakarta.xml.bind
> >  jakarta.xml.bind-api
> >  
> >  
> >
> > to the related pom in the /external module [1]
> >
> > Gruß
> > Richard
> >
> > [1]
> > https://github.com/apache/storm/blob/master/external/storm-kafka-monitor/pom.xml
> >
> >
> > Am Mittwoch, dem 15.11.2023 um 15:02 +0100 schrieb Alexandre
> > Vermeerbergen:
> > > I had found yet another thing that must be fixed before starting
> > > 2.6.0
> > > vote: while reviewing my shells to install Storm on my Nimbus hosts
> > > and Supervision hosts, I found this old by-pass which I used to put
> > > in
> > > place when switching from Java 11 to Java 17 at runtime:
> > >
> > > cp storm-stable/lib-worker/jakarta.xml.bind-api-2.3.2.jar
> > > storm-stable/lib-tools/storm-kafka-monitor
> > >
> > > (storm-stable is the name of the directory where I extract Storm's
> > > binaries)
> > >
> > > It turns out that this ugly by-pass is still required when running
> > > with Java 17 (which is my case since more than 1 year), otherwise
> > > the
> > > display of Kafka lags in Storm UI is broken, and we have the
> > > following
> > > exceptions in ui.log:
> > >
> > > 2023-11-15 10:54:27.473 o.e.j.s.AbstractConnector main [INFO]
> > > Started
> > > ServerConnector@e3c9c02f{HTTP/1.1, (http/1.1)}{0.0.0.0:8070}
> > > 2023-11-15 10:54:27.474 o.e.j.s.Server main [INFO] Started @4256ms
> > > 2023-11-15 10:55:02.781 o.a.s.u.NimbusClient qtp-1075191296-25
> > > [INFO]
> > > Found leader nimbus :
> > > ec23-1-251-0-65.eu-west-1.compute.amazonaws.com:6627
> > > 2023-11-15 10:59:31.128 o.a.s.u.ShellUtils qtp-1075191296-24 [INFO]
> > > Failed running command
> > > [/usr/local/Storm/storm-stable/bin/storm-kafka-monitor, -t, audit,
> > > -
> > > g,
> > > StormAuditPublisherTopology_SbxRealTimeSupervisionAVEEEZZeuw1, -b,
> > > ec23-1-251-0-66.eu-west-1.compute.amazonaws.com,ec23-1-251-0-65.eu-
> > > west-1.compute.amazonaws.com,ec2-52-18-172-77,
> > > -s, SASL_SSL, -c, /tmp/kafka-consumer-
> > > extra6421423768198678110props]
> > > org.apache.storm.utils.ShellUtils$ExitCodeException: SLF4J: Failed
> > > to
> > > load class "org.slf4j.impl.StaticLoggerBinder".
> > > SLF4J: Defaulting to no-operation (NOP) logger implementation
> > > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
> > > further details.
> > > Exception in thread "main" java.lang.NoClassDefFoundError:
> > > javax.xml.bind.DatatypeConverter
> > >  at
> > > org.apache.kafka.common.security.scram.ScramMessages$ServerFirstMes
> > > sa
> > > ge.(ScramMessages.java:143)
> > >  at
> > > org.apache.kafka.common.security.scram.ScramSaslClient.evaluateChal
> > > le
> > > nge(ScramSaslClient.java:112)
> > >  at
> > > org.apache.kafka.common.security.authenticator.SaslClientAuthentica
> > > to
> > > r$2.run(SaslClientAuthenticator.java:280)
> > >  at
> > > org.apache.kafka.common.security.authenticator.SaslClientAuthentica
> > > to
> > > r$2.run(SaslClientAuthenticator.java:278)
> > >  at
> > > java.base/java.security.AccessController.doPrivileged(AccessControl
> > > le
&g

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT (a7aac90fdc4b87eba8dce3b851b55c86c4a390cf)

[Alexandre Vermeerbergen]

Okay!
I just created https://issues.apache.org/jira/browse/STORM-4003 for
the tracking.

Le mer. 15 nov. 2023 à 15:16, Richard Zowalla  a écrit :
>
> Yes, feel free to provide a fix / pr + jira issue.
>
> It is just a matter of adding
>
>  
>  jakarta.xml.bind
>  jakarta.xml.bind-api
>  
>  
>
> to the related pom in the /external module [1]
>
> Gruß
> Richard
>
> [1]
> https://github.com/apache/storm/blob/master/external/storm-kafka-monitor/pom.xml
>
>
> Am Mittwoch, dem 15.11.2023 um 15:02 +0100 schrieb Alexandre
> Vermeerbergen:
> > I had found yet another thing that must be fixed before starting
> > 2.6.0
> > vote: while reviewing my shells to install Storm on my Nimbus hosts
> > and Supervision hosts, I found this old by-pass which I used to put
> > in
> > place when switching from Java 11 to Java 17 at runtime:
> >
> > cp storm-stable/lib-worker/jakarta.xml.bind-api-2.3.2.jar
> > storm-stable/lib-tools/storm-kafka-monitor
> >
> > (storm-stable is the name of the directory where I extract Storm's
> > binaries)
> >
> > It turns out that this ugly by-pass is still required when running
> > with Java 17 (which is my case since more than 1 year), otherwise the
> > display of Kafka lags in Storm UI is broken, and we have the
> > following
> > exceptions in ui.log:
> >
> > 2023-11-15 10:54:27.473 o.e.j.s.AbstractConnector main [INFO] Started
> > ServerConnector@e3c9c02f{HTTP/1.1, (http/1.1)}{0.0.0.0:8070}
> > 2023-11-15 10:54:27.474 o.e.j.s.Server main [INFO] Started @4256ms
> > 2023-11-15 10:55:02.781 o.a.s.u.NimbusClient qtp-1075191296-25 [INFO]
> > Found leader nimbus :
> > ec23-1-251-0-65.eu-west-1.compute.amazonaws.com:6627
> > 2023-11-15 10:59:31.128 o.a.s.u.ShellUtils qtp-1075191296-24 [INFO]
> > Failed running command
> > [/usr/local/Storm/storm-stable/bin/storm-kafka-monitor, -t, audit, -
> > g,
> > StormAuditPublisherTopology_SbxRealTimeSupervisionAVEEEZZeuw1, -b,
> > ec23-1-251-0-66.eu-west-1.compute.amazonaws.com,ec23-1-251-0-65.eu-
> > west-1.compute.amazonaws.com,ec2-52-18-172-77,
> > -s, SASL_SSL, -c, /tmp/kafka-consumer-extra6421423768198678110props]
> > org.apache.storm.utils.ShellUtils$ExitCodeException: SLF4J: Failed to
> > load class "org.slf4j.impl.StaticLoggerBinder".
> > SLF4J: Defaulting to no-operation (NOP) logger implementation
> > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
> > further details.
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> > javax.xml.bind.DatatypeConverter
> >  at
> > org.apache.kafka.common.security.scram.ScramMessages$ServerFirstMessa
> > ge.(ScramMessages.java:143)
> >  at
> > org.apache.kafka.common.security.scram.ScramSaslClient.evaluateChalle
> > nge(ScramSaslClient.java:112)
> >  at
> > org.apache.kafka.common.security.authenticator.SaslClientAuthenticato
> > r$2.run(SaslClientAuthenticator.java:280)
> >  at
> > org.apache.kafka.common.security.authenticator.SaslClientAuthenticato
> > r$2.run(SaslClientAuthenticator.java:278)
> >  at
> > java.base/java.security.AccessController.doPrivileged(AccessControlle
> > r.java:784)
> >  at java.base/javax.security.auth.Subject.doAs(Subject.java:439)
> >  at
> > org.apache.kafka.common.security.authenticator.SaslClientAuthenticato
> > r.createSaslToken(SaslClientAuthenticator.java:278)
> >  at
> > org.apache.kafka.common.security.authenticator.SaslClientAuthenticato
> > r.sendSaslToken(SaslClientAuthenticator.java:215)
> >  at
> > org.apache.kafka.common.security.authenticator.SaslClientAuthenticato
> > r.authenticate(SaslClientAuthenticator.java:189)
> >  at
> > org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.jav
> > a:76)
> >  at
> > org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.j
> > ava:385)
> >  at org.apache.kafka.common.network.Selector.poll(Selector.java:334)
> >  at
> > org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:433)
> >  at
> > org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.pol
> > l(ConsumerNetworkClient.java:232)
> >  at
> > org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.pol
> > l(ConsumerNetworkClient.java:208)
> >  at
> > org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.pol
> > l(ConsumerNetworkClient.java:184)
> >  at
> > org.apache.kafka.clients.consumer.internals.Fetcher.getTopicMetadata(
> > Fetcher.java:314)
> >  at
> > org.

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT (a7aac90fdc4b87eba8dce3b851b55c86c4a390cf)

[Alexandre Vermeerbergen]

I had found yet another thing that must be fixed before starting 2.6.0
vote: while reviewing my shells to install Storm on my Nimbus hosts
and Supervision hosts, I found this old by-pass which I used to put in
place when switching from Java 11 to Java 17 at runtime:

cp storm-stable/lib-worker/jakarta.xml.bind-api-2.3.2.jar
storm-stable/lib-tools/storm-kafka-monitor

(storm-stable is the name of the directory where I extract Storm's binaries)

It turns out that this ugly by-pass is still required when running
with Java 17 (which is my case since more than 1 year), otherwise the
display of Kafka lags in Storm UI is broken, and we have the following
exceptions in ui.log:

2023-11-15 10:54:27.473 o.e.j.s.AbstractConnector main [INFO] Started
ServerConnector@e3c9c02f{HTTP/1.1, (http/1.1)}{0.0.0.0:8070}
2023-11-15 10:54:27.474 o.e.j.s.Server main [INFO] Started @4256ms
2023-11-15 10:55:02.781 o.a.s.u.NimbusClient qtp-1075191296-25 [INFO]
Found leader nimbus :
ec23-1-251-0-65.eu-west-1.compute.amazonaws.com:6627
2023-11-15 10:59:31.128 o.a.s.u.ShellUtils qtp-1075191296-24 [INFO]
Failed running command
[/usr/local/Storm/storm-stable/bin/storm-kafka-monitor, -t, audit, -g,
StormAuditPublisherTopology_SbxRealTimeSupervisionAVEEEZZeuw1, -b,
ec23-1-251-0-66.eu-west-1.compute.amazonaws.com,ec23-1-251-0-65.eu-west-1.compute.amazonaws.com,ec2-52-18-172-77,
-s, SASL_SSL, -c, /tmp/kafka-consumer-extra6421423768198678110props]
org.apache.storm.utils.ShellUtils$ExitCodeException: SLF4J: Failed to
load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
further details.
Exception in thread "main" java.lang.NoClassDefFoundError:
javax.xml.bind.DatatypeConverter
 at 
org.apache.kafka.common.security.scram.ScramMessages$ServerFirstMessage.(ScramMessages.java:143)
 at 
org.apache.kafka.common.security.scram.ScramSaslClient.evaluateChallenge(ScramSaslClient.java:112)
 at 
org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$2.run(SaslClientAuthenticator.java:280)
 at 
org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$2.run(SaslClientAuthenticator.java:278)
 at 
java.base/java.security.AccessController.doPrivileged(AccessController.java:784)
 at java.base/javax.security.auth.Subject.doAs(Subject.java:439)
 at 
org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslToken(SaslClientAuthenticator.java:278)
 at 
org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.sendSaslToken(SaslClientAuthenticator.java:215)
 at 
org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.authenticate(SaslClientAuthenticator.java:189)
 at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:76)
 at 
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:385)
 at org.apache.kafka.common.network.Selector.poll(Selector.java:334)
 at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:433)
 at 
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:232)
 at 
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:208)
 at 
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:184)
 at 
org.apache.kafka.clients.consumer.internals.Fetcher.getTopicMetadata(Fetcher.java:314)
 at 
org.apache.kafka.clients.consumer.KafkaConsumer.partitionsFor(KafkaConsumer.java:1386)
 at 
org.apache.storm.kafka.monitor.KafkaOffsetLagUtil.getOffsetLags(KafkaOffsetLagUtil.java:165)
 at 
org.apache.storm.kafka.monitor.KafkaOffsetLagUtil.main(KafkaOffsetLagUtil.java:74)
Caused by: java.lang.ClassNotFoundException: javax.xml.bind.DatatypeConverter
 at 
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:827)
 at 
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:1095)
 ... 20 more

 at org.apache.storm.utils.ShellUtils.runCommand(ShellUtils.java:264)
~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]

copying jakarta.xml.bind-api-2.3.2.jar to
storm-stable/lib-tools/storm-kafka-monitor and restarting storm-ui
solves the problem.

=> Would it be possible to include jakarta.xml.bind-api-2.3.2.jar in
lib-tools/storm-kafka-monitor/ directory of binary Storm archive out
of the box?

Thanks,
Alexandre

Le mar. 14 nov. 2023 à 17:50, Alexandre Vermeerbergen
 a écrit :
>
> Good news on my side:
> * First test with 2.6.0 snapshot 2023/11/14 with a single Supervisor host is 
> OK
>=> I will soon (tomorrow) be able to upgrade my pre production
> cluster to have a wider scope & performance results "at scale"
> * My crazy attempt to convert all my topologies code from Java EE API
> to Jakarta EE API is finally 

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT (a7aac90fdc4b87eba8dce3b851b55c86c4a390cf)

[Alexandre Vermeerbergen]

Good news on my side:
* First test with 2.6.0 snapshot 2023/11/14 with a single Supervisor host is OK
   => I will soon (tomorrow) be able to upgrade my pre production
cluster to have a wider scope & performance results "at scale"
* My crazy attempt to convert all my topologies code from Java EE API
to Jakarta EE API is finally working *fine*, the issue was lying
between my keyboard and my seat (I used a misconfigured Kafka cluster
with my mostly Kafka-depending topologies). So it means that
regardless of the fact that Storm internally still relies on Java EE
artefacts, a bunch of code migrated to Jakarta EE runs anway - that's
what I like (smooth transitions)

I update when I'll have results on my pre-production cluster at scale
(max Thursday)

Alexandre

Le mar. 14 nov. 2023 à 16:46, Julien Nioche
 a écrit :
>
> Thanks Richard,
>
> I have tested the latest snapshot with a simple crawl topology in local and
> deployed mode + a more complex one which generates WARC files (and
> therefore the HDFS resouces), again both in local and remote mode.
> No issues found, everything is looking fine
>
> Julien
>
> On Tue, 14 Nov 2023 at 09:52, Richard Zowalla  wrote:
>
> > Hi all,
> >
> > as written in the other thread, I have prepared another SNAPSHOT
> > artifact incorporting the feedbach received in the first round.
> >
> > I would like to receive some feedback on the current SNAPSHOT build
> > again.
> >
> > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> >
> > https://github.com/apache/storm/commit/a7aac90fdc4b87eba8dce3b851b55c86c4a390cf
> >
> >
> > It is available via the ASF Snapshot repository:
> > https://repository.apache.org/content/repositories/snapshots/
> >
> > You can consume by adding
> >
> >  
> >   apache.snapshots
> >   Apache Snapshot Repository
> >   https://repository.apache.org/snapshots
> >   
> > false
> >   
> >  
> >
> > to your project pom. As we do not deploy SNAPSHOTS automatically, it
> > should be easy to just consume the latest SNAPSHOT.
> >
> > The packaged binaries are available at nightlies.apache.org:
> >
> >
> > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/a7aac90fdc4b87eba8dce3b851b55c86c4a390cf/
> >
> >
> > If you have some minutes left: Please test and report any issues with
> > this binaries, so we can fix before attempting to release.
> >
> > The most significant changes are some fixes of ui-related bugs +
> > upgrades of dependencies with CVEs, etc.
> >
> > Here is a diff:
> >
> >
> > https://github.com/apache/storm/compare/8f883086032669a8f04b09a3b312d60af5b44533..a7aac90fdc4b87eba8dce3b851b55c86c4a390cf
> >
> > A summary in Jira is here:
> >
> >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820=12353484
> >
> > Gruß
> > Richard
> >
> >
> >
> >
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble 

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT (a7aac90fdc4b87eba8dce3b851b55c86c4a390cf)

[Alexandre Vermeerbergen]

Thanks Richard !

I am starting a release of my code based on this new Storm 2.6.0
snapshot 20231114, and I will upgrade our pre-production (running with
previous 2.6.0 snapshot of 2nd of November + few hand applies patches)

On a side note, I have made a crazy test yesterday, which was to
migrate my topologies from Java EE APIs to Jakarta EE ones
(jakarta.json, etc with Johnzon version compatible with Jakarta).
Strangely, my topologies are failing so much that Nimbus thinks the
Supervisor (test made with a single Supervision type of deployment) in
unhealthy and restarts are failing. Even more strange is that I don't
find any useful log, which could complain that I had forgot to migrate
from javax. to jakartax. something that should be so... I will update
dev list on a separate thread for this topic...

Thanks,
Alexandre

Le mar. 14 nov. 2023 à 10:52, Richard Zowalla  a écrit :
>
> Hi all,
>
> as written in the other thread, I have prepared another SNAPSHOT
> artifact incorporting the feedbach received in the first round.
>
> I would like to receive some feedback on the current SNAPSHOT build
> again.
>
> I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> https://github.com/apache/storm/commit/a7aac90fdc4b87eba8dce3b851b55c86c4a390cf
>
>
> It is available via the ASF Snapshot repository:
> https://repository.apache.org/content/repositories/snapshots/
>
> You can consume by adding
>
>  
>   apache.snapshots
>   Apache Snapshot Repository
>   https://repository.apache.org/snapshots
>   
> false
>   
>  
>
> to your project pom. As we do not deploy SNAPSHOTS automatically, it
> should be easy to just consume the latest SNAPSHOT.
>
> The packaged binaries are available at nightlies.apache.org:
>
> https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/a7aac90fdc4b87eba8dce3b851b55c86c4a390cf/
>
>
> If you have some minutes left: Please test and report any issues with
> this binaries, so we can fix before attempting to release.
>
> The most significant changes are some fixes of ui-related bugs +
> upgrades of dependencies with CVEs, etc.
>
> Here is a diff:
>
> https://github.com/apache/storm/compare/8f883086032669a8f04b09a3b312d60af5b44533..a7aac90fdc4b87eba8dce3b851b55c86c4a390cf
>
> A summary in Jira is here:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820=12353484
>
> Gruß
> Richard
>
>
>

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Yet another regression on in-work 2.6.0 : the supervisor.html view is
broken, there's a Javascript syntax error (probably my fault again).
It' s bit late for me, I'll check that tomorrow...

Alexandre

Le lun. 13 nov. 2023 à 15:36, Julien Nioche
 a écrit :
>
> I agree with Richard. The next release contains enough big changes and it
> would be good to release it ASAP before embarking on another big
> modification.
>
> Julien
>
> On Mon, 13 Nov 2023 at 12:08, Richard Zowalla  wrote:
>
> > Hi,
> >
> > yes - in principal I agree, that switching to slf4j 2.x would be great
> > in the near future. However, it requires some more work due to our old
> > dependency stack ("external modules", ...) and most likely a lot of
> > exclusions.
> >
> > Therefore, personally, I wouldn't see it in a 2.6.0 release but as a
> > task on which we could work after having a new version in the pipeline
> >
> > wdyt?
> >
> > Gruß
> > Richard
> >
> > Am Sonntag, dem 12.11.2023 um 09:40 +0100 schrieb Alexandre
> > Vermeerbergen:
> > > Thank Richard for the great work to cleanup CVEs on in-work 2.6.0 !
> > > One last "regret" I have with current in-work 2.6.0 is that we still
> > > rely on very old slf4j 1.x.
> > > While it was not on slf4j itself, the log4shell CVE should remind us
> > > that we should keep loggers as up to date as possible.
> > >
> > > I see in a deployed Storm runtime the following hits:
> > >
> > > [root]# find . -name "slf*jar" -print
> > > ./lib/slf4j-api-1.7.36.jar
> > > ./lib-tools/storm-kafka-monitor/slf4j-api-1.7.36.jar
> > > ./lib-tools/submit-tools/slf4j-api-1.7.36.jar
> > > ./lib-tools/sql/core/slf4j-api-1.7.36.jar
> > > ./lib-tools/sql/runtime/slf4j-api-1.7.36.jar
> > > ./external/storm-autocreds/slf4j-api-1.7.36.jar
> > > ./lib-worker/slf4j-api-1.7.36.jar
> > > ./lib-webapp/slf4j-api-1.7.36.jar
> > >
> > > From https://www.slf4j.org/download.html it is said that "The older
> > > stable SLF4J version is 1.7.36. It is no longer actively developed."
> > >
> > > The most current version of slf4j is 2.0.9.
> > >
> > > But I think at same time we would need to update these ones too,
> > > otherwise we will have a mismatch between SLF4J v1 logger
> > > implementations vs new v2 APIs:
> > >
> > > [root@ip-172-31-43-21 storm-stable]# find . -name "log*jar" -print
> > > ./lib/log4j-over-slf4j-1.7.36.jar
> > > ./lib/log4j-api-2.20.0.jar
> > > ./lib/log4j-slf4j-impl-2.20.0.jar
> > > ./lib/log4j-core-2.20.0.jar
> > > ./external/storm-autocreds/log4j-api-2.20.0.jar
> > > ./external/storm-autocreds/log4j-web-2.20.0.jar
> > > ./external/storm-autocreds/log4j-1.2-api-2.20.0.jar
> > > ./external/storm-autocreds/log4j-core-2.20.0.jar
> > > ./lib-worker/log4j-over-slf4j-1.7.36.jar
> > > ./lib-worker/log4j-api-2.20.0.jar
> > > ./lib-worker/log4j-slf4j-impl-2.20.0.jar
> > > ./lib-worker/log4j-core-2.20.0.jar
> > > ./lib-webapp/log4j-over-slf4j-1.7.36.jar
> > > ./lib-webapp/log4j-api-2.20.0.jar
> > > ./lib-webapp/log4j-core-2.20.0.jar
> > >
> > > Disclaimer: I have no idea how hard this update would be.
> > >
> > > Anyone with good experience at updating SLF4J & related loggers could
> > > react on this proposal ?
> > >
> > > Thanks,
> > > Alexandre
> > >
> > >
> > > Le jeu. 9 nov. 2023 à 15:04, Richard Zowalla  a
> > > écrit :
> > > >
> > > > Ok, most of the stuff reported by grype should be fixed now.
> > > > I might be able to push a new snapshot within the next week, so we
> > > > can
> > > > do some testing and than go for a release candiate / vote :)
> > > >
> > > > Gruß
> > > > Richard
> > > >
> > > >
> > > >
> > > > Am Donnerstag, dem 09.11.2023 um 08:53 +0100 schrieb Richard
> > > > Zowalla:
> > > > > Maybe just run grype [1] on the lib directories and post the
> > > > > output
> > > > > on
> > > > > the list / thread, so we can create tickets for it
> > > > >
> > > > > There might be some false positives but will give good insights.
> > > > >
> > > > > Gruß
> > > > > Richard
> > > > >
> > > > >
> > > > > [1] https://g

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Thank Richard for the great work to cleanup CVEs on in-work 2.6.0 !
One last "regret" I have with current in-work 2.6.0 is that we still
rely on very old slf4j 1.x.
While it was not on slf4j itself, the log4shell CVE should remind us
that we should keep loggers as up to date as possible.

I see in a deployed Storm runtime the following hits:

[root]# find . -name "slf*jar" -print
./lib/slf4j-api-1.7.36.jar
./lib-tools/storm-kafka-monitor/slf4j-api-1.7.36.jar
./lib-tools/submit-tools/slf4j-api-1.7.36.jar
./lib-tools/sql/core/slf4j-api-1.7.36.jar
./lib-tools/sql/runtime/slf4j-api-1.7.36.jar
./external/storm-autocreds/slf4j-api-1.7.36.jar
./lib-worker/slf4j-api-1.7.36.jar
./lib-webapp/slf4j-api-1.7.36.jar

>From https://www.slf4j.org/download.html it is said that "The older
stable SLF4J version is 1.7.36. It is no longer actively developed."

The most current version of slf4j is 2.0.9.

But I think at same time we would need to update these ones too,
otherwise we will have a mismatch between SLF4J v1 logger
implementations vs new v2 APIs:

[root@ip-172-31-43-21 storm-stable]# find . -name "log*jar" -print
./lib/log4j-over-slf4j-1.7.36.jar
./lib/log4j-api-2.20.0.jar
./lib/log4j-slf4j-impl-2.20.0.jar
./lib/log4j-core-2.20.0.jar
./external/storm-autocreds/log4j-api-2.20.0.jar
./external/storm-autocreds/log4j-web-2.20.0.jar
./external/storm-autocreds/log4j-1.2-api-2.20.0.jar
./external/storm-autocreds/log4j-core-2.20.0.jar
./lib-worker/log4j-over-slf4j-1.7.36.jar
./lib-worker/log4j-api-2.20.0.jar
./lib-worker/log4j-slf4j-impl-2.20.0.jar
./lib-worker/log4j-core-2.20.0.jar
./lib-webapp/log4j-over-slf4j-1.7.36.jar
./lib-webapp/log4j-api-2.20.0.jar
./lib-webapp/log4j-core-2.20.0.jar

Disclaimer: I have no idea how hard this update would be.

Anyone with good experience at updating SLF4J & related loggers could
react on this proposal ?

Thanks,
Alexandre


Le jeu. 9 nov. 2023 à 15:04, Richard Zowalla  a écrit :
>
> Ok, most of the stuff reported by grype should be fixed now.
> I might be able to push a new snapshot within the next week, so we can
> do some testing and than go for a release candiate / vote :)
>
> Gruß
> Richard
>
>
>
> Am Donnerstag, dem 09.11.2023 um 08:53 +0100 schrieb Richard Zowalla:
> > Maybe just run grype [1] on the lib directories and post the output
> > on
> > the list / thread, so we can create tickets for it
> >
> > There might be some false positives but will give good insights.
> >
> > Gruß
> > Richard
> >
> >
> > [1] https://github.com/anchore/grype
> > Am Donnerstag, dem 09.11.2023 um 08:49 +0100 schrieb Alexandre
> > Vermeerbergen:
> > > Yet another finding on my side to "not call yet for vote on a Storm
> > > 2.6.0", we have some CVEs and we shoudn't release a version having
> > > known CVEs.
> > >
> > > For example, we have
> > >
> > >  find . -name "commons-text*jar" -print
> > > ./lib-webapp/commons-text-1.9.jar
> > > ./external/storm-autocreds/commons-text-1.10.0.jar
> > >
> > > The second one is okay from
> > > https://www.cvedetails.com/cve/CVE-2022-42889/ perspective, but not
> > > the first one.
> > >
> > > I will continue checking if there are other CVEs...
> > >
> > > Thanks,
> > > Alexandre
> > >
> > > Le mar. 7 nov. 2023 à 17:43, Alexandre Vermeerbergen
> > >  a écrit :
> > > >
> > > > Oh thank you very much for this cherry-pinking, I was too busy
> > > > doing
> > > > other unrelated stuff..
> > > >
> > > > Le mar. 7 nov. 2023 à 17:35, Richard Zowalla  a
> > > > écrit :
> > > > >
> > > > > I cherry-picked your commit and put that on master
> > > > >
> > > > >
> > > > > Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard
> > > > > Zowalla:
> > > > > > You pushed on your own branch. You would need to push to
> > > > > > apache/storm
> > > > > > main branch or open a PR ;-)
> > > > > >
> > > > > > Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb Alexandre
> > > > > > Vermeerbergen:
> > > > > > > Just did a 'git push' with that trivial fix... is that
> > > > > > > enough?
> > > > > > >
> > > > > > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla
> > > > > > > 
> > > > > > > a
> > > > > > > écrit :
> > > > > > > >
> > > > > > > > You can do a

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

At this moment that's the CVE which I found using Storm 2.6.0 snapshot 20231102.
And since there's a commons-text-1.11.0.jar available, I guess we
should directly jump to this version.

Le jeu. 9 nov. 2023 à 08:49, Alexandre Vermeerbergen
 a écrit :
>
> Yet another finding on my side to "not call yet for vote on a Storm
> 2.6.0", we have some CVEs and we shoudn't release a version having
> known CVEs.
>
> For example, we have
>
>  find . -name "commons-text*jar" -print
> ./lib-webapp/commons-text-1.9.jar
> ./external/storm-autocreds/commons-text-1.10.0.jar
>
> The second one is okay from
> https://www.cvedetails.com/cve/CVE-2022-42889/ perspective, but not
> the first one.
>
> I will continue checking if there are other CVEs...
>
> Thanks,
> Alexandre
>
> Le mar. 7 nov. 2023 à 17:43, Alexandre Vermeerbergen
>  a écrit :
> >
> > Oh thank you very much for this cherry-pinking, I was too busy doing
> > other unrelated stuff..
> >
> > Le mar. 7 nov. 2023 à 17:35, Richard Zowalla  a écrit :
> > >
> > > I cherry-picked your commit and put that on master
> > >
> > >
> > > Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard Zowalla:
> > > > You pushed on your own branch. You would need to push to apache/storm
> > > > main branch or open a PR ;-)
> > > >
> > > > Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb Alexandre
> > > > Vermeerbergen:
> > > > > Just did a 'git push' with that trivial fix... is that enough?
> > > > >
> > > > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla 
> > > > > a
> > > > > écrit :
> > > > > >
> > > > > > You can do as you like ;-)
> > > > > >
> > > > > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen
> > > > > > :
> > > > > > > I mean :
> > > > > > >
> > > > > > > Can I keep using my
> > > > > > > 'STORM-3958-Capacity-to-set-St
> > > > > > > orm-UI's-title-in-conf/storm.yaml'
> > > > > > > branch to push this fix, or should I create a new one?
> > > > > > >
> > > > > > > Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > > > > >  a écrit :
> > > > > > > >
> > > > > > > > Can you keep using my
> > > > > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > > > > > conf/storm.yaml'
> > > > > > > > branch to push this fix, or should I create a new one?
> > > > > > > >
> > > > > > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > > > > >  a écrit :
> > > > > > > > >
> > > > > > > > > Just push a fix for it (referencing the original issue) ;-)
> > > > > > > > > -
> > > > > > > > > we hadn't a release in between, so not a big deal.
> > > > > > > > >
> > > > > > > > > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre
> > > > > > > > > Vermeerbergen :
> > > > > > > > > > Temporary workaround for the Storm UI title's regression:
> > > > > > > > > > run this
> > > > > > > > > > command on the title-template.html file (in
> > > > > > > > > > storm-stable/public/templates directory) after having
> > > > > > > > > > extracted the
> > > > > > > > > > binary archive:
> > > > > > > > > >
> > > > > > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > > > > > > > > >
> > > > > > > > > > Again sorry for having introduced this regression.
> > > > > > > > > >
> > > > > > > > > > Alexandre
> > > > > > > > > >
> > > > > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > > > > > > > > >  a écrit :
> > > > > > > > > > >
> > > > > > > > > > > Updates regarding my test with Storm 2.6.0 Snapshot
> > > > > > > > > > > 2023/

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Yet another finding on my side to "not call yet for vote on a Storm
2.6.0", we have some CVEs and we shoudn't release a version having
known CVEs.

For example, we have

 find . -name "commons-text*jar" -print
./lib-webapp/commons-text-1.9.jar
./external/storm-autocreds/commons-text-1.10.0.jar

The second one is okay from
https://www.cvedetails.com/cve/CVE-2022-42889/ perspective, but not
the first one.

I will continue checking if there are other CVEs...

Thanks,
Alexandre

Le mar. 7 nov. 2023 à 17:43, Alexandre Vermeerbergen
 a écrit :
>
> Oh thank you very much for this cherry-pinking, I was too busy doing
> other unrelated stuff..
>
> Le mar. 7 nov. 2023 à 17:35, Richard Zowalla  a écrit :
> >
> > I cherry-picked your commit and put that on master
> >
> >
> > Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard Zowalla:
> > > You pushed on your own branch. You would need to push to apache/storm
> > > main branch or open a PR ;-)
> > >
> > > Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb Alexandre
> > > Vermeerbergen:
> > > > Just did a 'git push' with that trivial fix... is that enough?
> > > >
> > > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla 
> > > > a
> > > > écrit :
> > > > >
> > > > > You can do as you like ;-)
> > > > >
> > > > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen
> > > > > :
> > > > > > I mean :
> > > > > >
> > > > > > Can I keep using my
> > > > > > 'STORM-3958-Capacity-to-set-St
> > > > > > orm-UI's-title-in-conf/storm.yaml'
> > > > > > branch to push this fix, or should I create a new one?
> > > > > >
> > > > > > Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > > > >  a écrit :
> > > > > > >
> > > > > > > Can you keep using my
> > > > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > > > > conf/storm.yaml'
> > > > > > > branch to push this fix, or should I create a new one?
> > > > > > >
> > > > > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > > > >  a écrit :
> > > > > > > >
> > > > > > > > Just push a fix for it (referencing the original issue) ;-)
> > > > > > > > -
> > > > > > > > we hadn't a release in between, so not a big deal.
> > > > > > > >
> > > > > > > > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre
> > > > > > > > Vermeerbergen :
> > > > > > > > > Temporary workaround for the Storm UI title's regression:
> > > > > > > > > run this
> > > > > > > > > command on the title-template.html file (in
> > > > > > > > > storm-stable/public/templates directory) after having
> > > > > > > > > extracted the
> > > > > > > > > binary archive:
> > > > > > > > >
> > > > > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > > > > > > > >
> > > > > > > > > Again sorry for having introduced this regression.
> > > > > > > > >
> > > > > > > > > Alexandre
> > > > > > > > >
> > > > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > > > > > > > >  a écrit :
> > > > > > > > > >
> > > > > > > > > > Updates regarding my test with Storm 2.6.0 Snapshot
> > > > > > > > > > 2023/11/02:
> > > > > > > > > >  1. I have modified my KO bolts using this pattern in
> > > > > > > > > > their prepare() method:
> > > > > > > > > >
> > > > > > > > > >long cacheMaxSize =
> > > > > > > > > > Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZ
> > > > > > > > > > E)
> > > > > > > > > > .toString());
> > > > > > > > > >
> > > > > > > > > >instead of:
> > > > > > > > > >
> > > >

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Oh thank you very much for this cherry-pinking, I was too busy doing
other unrelated stuff..

Le mar. 7 nov. 2023 à 17:35, Richard Zowalla  a écrit :
>
> I cherry-picked your commit and put that on master
>
>
> Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard Zowalla:
> > You pushed on your own branch. You would need to push to apache/storm
> > main branch or open a PR ;-)
> >
> > Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb Alexandre
> > Vermeerbergen:
> > > Just did a 'git push' with that trivial fix... is that enough?
> > >
> > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla 
> > > a
> > > écrit :
> > > >
> > > > You can do as you like ;-)
> > > >
> > > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen
> > > > :
> > > > > I mean :
> > > > >
> > > > > Can I keep using my
> > > > > 'STORM-3958-Capacity-to-set-St
> > > > > orm-UI's-title-in-conf/storm.yaml'
> > > > > branch to push this fix, or should I create a new one?
> > > > >
> > > > > Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > > >  a écrit :
> > > > > >
> > > > > > Can you keep using my
> > > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > > > conf/storm.yaml'
> > > > > > branch to push this fix, or should I create a new one?
> > > > > >
> > > > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > > >  a écrit :
> > > > > > >
> > > > > > > Just push a fix for it (referencing the original issue) ;-)
> > > > > > > -
> > > > > > > we hadn't a release in between, so not a big deal.
> > > > > > >
> > > > > > > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre
> > > > > > > Vermeerbergen :
> > > > > > > > Temporary workaround for the Storm UI title's regression:
> > > > > > > > run this
> > > > > > > > command on the title-template.html file (in
> > > > > > > > storm-stable/public/templates directory) after having
> > > > > > > > extracted the
> > > > > > > > binary archive:
> > > > > > > >
> > > > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > > > > > > >
> > > > > > > > Again sorry for having introduced this regression.
> > > > > > > >
> > > > > > > > Alexandre
> > > > > > > >
> > > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > > > > > > >  a écrit :
> > > > > > > > >
> > > > > > > > > Updates regarding my test with Storm 2.6.0 Snapshot
> > > > > > > > > 2023/11/02:
> > > > > > > > >  1. I have modified my KO bolts using this pattern in
> > > > > > > > > their prepare() method:
> > > > > > > > >
> > > > > > > > >long cacheMaxSize =
> > > > > > > > > Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZ
> > > > > > > > > E)
> > > > > > > > > .toString());
> > > > > > > > >
> > > > > > > > >instead of:
> > > > > > > > >
> > > > > > > > >long cacheMaxSize = (Long)
> > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > >
> > > > > > > > > 2. I have found the cause of the regression in loss of
> > > > > > > > > 'Storm UI'
> > > > > > > > > title at the top left-hand side of all Storm UI's
> > > > > > > > > pages.
> > > > > > > > > This is a mistake of mine in this PR:
> > > > > > > > >
> > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > > > > > > More precisely, at line 19 of this file:
> > > > > > > > > https://github.com/apache/storm/pu

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Hello Julien,

Thanks for your suggestion to rebuild my code, but as I mentioned, my
first test with Storm 2.6.0 SNAPSHOT was to deploy existing topologies
without rebuilding these topologies.
It turned out to be a worthy experience, given the "old bad practices"
in my topologies' code that this exercice allowed me to find.

Thanks,
Alexandre

Le dim. 5 nov. 2023 à 16:32, Julien Nioche
 a écrit :
>
> Alexandre,
>
> I had a similar issue when trying the initial snapshot made by Richard but
> it seems to have disappeared with the new one.
> Maybe try building your code with *mvn clean package -U  *or delete the
> content of* ~/.m2/repository/org/apache/storm/ *beforehand?
>
> Julien
>
> On Sat, 4 Nov 2023 at 19:57, Alexandre Vermeerbergen <
> avermeerber...@gmail.com> wrote:
>
> > Yes another regression I found with my topologies (built with 2.5.0
> > dependencies) running to Storm 2.6.0 SNAPSHOT 2023/11/02: I have seen
> > several exceptions like this one:
> >
> > 2023-11-04 18:43:31.696 o.a.s.e.e.ReportError
> > Thread-16-alertsHistoryUMToKafka-executor[27, 27] [ERROR] Error
> > java.lang.RuntimeException: java.lang.NoClassDefFoundError:
> > org.apache.storm.shade.org.json.simple.JSONObject
> > at org.apache.storm.utils.Utils$1.run(Utils.java:413)
> > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > at java.lang.Thread.run(Thread.java:857) [?:?]
> > Caused by: java.lang.NoClassDefFoundError:
> > org.apache.storm.shade.org.json.simple.JSONObject
> > at
> > com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessageFromTuple(AlertHistoryTupleToKafkaMapper.java:25)
> > ~[stormjar.jar:?]
> > at
> > com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessageFromTuple(AlertHistoryTupleToKafkaMapper.java:11)
> > ~[stormjar.jar:?]
> > at org.apache.storm.kafka.bolt.KafkaBolt.process(KafkaBolt.java:177)
> > ~[stormjar.jar:?]
> >
> > and here's the related source code:
> >
> > package com.acme.storm.evaluator.bolt;
> >
> > import org.apache.storm.kafka.bolt.mapper.TupleToKafkaMapper;
> > import org.apache.storm.shade.org.json.simple.JSONObject;
> > import org.apache.storm.tuple.Tuple;
> > import org.slf4j.Logger;
> > import org.slf4j.LoggerFactory;
> >
> > import com.acme.storm.util.TupleAlertHistoryFields;
> >
> > public class AlertHistoryTupleToKafkaMapper implements
> > TupleToKafkaMapper {
> > private static final long serialVersionUID = 5668103889461721134L;
> >
> > public static final Logger logger =
> > LoggerFactory.getLogger(AlertHistoryTupleToKafkaMapper.class);
> >
> > @Override
> > public String getKeyFromTuple(Tuple tuple) {
> > return
> > tuple.getStringByField(TupleAlertHistoryFields.SERVICE_DEFINITION_NAME);
> > }
> >
> > @SuppressWarnings("unchecked")
> > @Override
> > public String getMessageFromTuple(Tuple tuple)
> > {
> > JSONObject tupleJson = new JSONObject();
> >
> > tuple.getFields().forEach(f -> {
> > tupleJson.put(f, tuple.getValueByField(f));
> > });
> >
> > String jsonString = tupleJson.toJSONString();
> >
> > return jsonString;
> > }
> > }
> >
> > line #25 is this one:
> >
> > JSONObject tupleJson = new JSONObject();
> >
> > so, from what I understand, Storm 2.6.0 no longer exposes some shaded
> > classes such as:
> >
> > org.apache.storm.shade.org.json.simple.JSONObject;
> >
> > Interestingly, it looks similar to this issue:
> >
> > https://stackoverflow.com/questions/33377291/java-lang-noclassdeffounderror-org-json-simple-jsonvalue-at-storm-kafka-dynamic
> >
> > Whatever, I guess I can only blame our own code for using Storm's
> > shaded classes, right ?
> > Note it's been running as is for long time and not breaking since many
> > upgrades, of course it's not an excuse...
> >
> > I'll fix our code & continue checking if I see other impacts of 2.5.0
> > -> 2.6.0 upgrade...
> >
> > Thanks,
> > Alexandre
> >
> > Le sam. 4 nov. 2023 à 10:36, Alexandre Vermeerbergen
> >  a écrit :
> > >
> > > Just did a 'git push' with that trivial fix... is that enough?
> > >
> > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla  a
> > écrit :
> > > >
> > > > You can do as you like ;-)
> > > >
> > > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen 

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Interestingly enough, to clean our topologies' code from its use of
Storm shaded classes, I removed storm-shaded-deps.jar from our
dependencies... only to find that in such case we have a compilation
error:

The type org.apache.storm.thrift.TException cannot be resolved. It is
indirectly referenced from required type
org.apache.storm.StormSubmitter

from this type of code (at the last line)

  // 
  // Submit Topology
  String displayName =
TopologyHelper.getTopologyDisplayName(topologyShortName,
kafkaBrokersHosts);
  StormSubmitter.submitTopology(displayName, topologyConfig,
builder.createTopology());
}


I would have excepted such indirect dependency to be part of
non-shared storm JARs... am I missing something?

Note: in the meantime, let me re-introduce this dependency and use
"find in code" to fix uses of shaded classes..

Thanks,
Alexandre

Le sam. 4 nov. 2023 à 20:57, Alexandre Vermeerbergen
 a écrit :
>
> Yes another regression I found with my topologies (built with 2.5.0
> dependencies) running to Storm 2.6.0 SNAPSHOT 2023/11/02: I have seen
> several exceptions like this one:
>
> 2023-11-04 18:43:31.696 o.a.s.e.e.ReportError
> Thread-16-alertsHistoryUMToKafka-executor[27, 27] [ERROR] Error
> java.lang.RuntimeException: java.lang.NoClassDefFoundError:
> org.apache.storm.shade.org.json.simple.JSONObject
> at org.apache.storm.utils.Utils$1.run(Utils.java:413)
> ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> at java.lang.Thread.run(Thread.java:857) [?:?]
> Caused by: java.lang.NoClassDefFoundError:
> org.apache.storm.shade.org.json.simple.JSONObject
> at 
> com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessageFromTuple(AlertHistoryTupleToKafkaMapper.java:25)
> ~[stormjar.jar:?]
> at 
> com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessageFromTuple(AlertHistoryTupleToKafkaMapper.java:11)
> ~[stormjar.jar:?]
> at org.apache.storm.kafka.bolt.KafkaBolt.process(KafkaBolt.java:177)
> ~[stormjar.jar:?]
>
> and here's the related source code:
>
> package com.acme.storm.evaluator.bolt;
>
> import org.apache.storm.kafka.bolt.mapper.TupleToKafkaMapper;
> import org.apache.storm.shade.org.json.simple.JSONObject;
> import org.apache.storm.tuple.Tuple;
> import org.slf4j.Logger;
> import org.slf4j.LoggerFactory;
>
> import com.acme.storm.util.TupleAlertHistoryFields;
>
> public class AlertHistoryTupleToKafkaMapper implements
> TupleToKafkaMapper {
> private static final long serialVersionUID = 5668103889461721134L;
>
> public static final Logger logger =
> LoggerFactory.getLogger(AlertHistoryTupleToKafkaMapper.class);
>
> @Override
> public String getKeyFromTuple(Tuple tuple) {
> return 
> tuple.getStringByField(TupleAlertHistoryFields.SERVICE_DEFINITION_NAME);
> }
>
> @SuppressWarnings("unchecked")
> @Override
> public String getMessageFromTuple(Tuple tuple)
> {
> JSONObject tupleJson = new JSONObject();
>
> tuple.getFields().forEach(f -> {
> tupleJson.put(f, tuple.getValueByField(f));
> });
>
> String jsonString = tupleJson.toJSONString();
>
> return jsonString;
> }
> }
>
> line #25 is this one:
>
> JSONObject tupleJson = new JSONObject();
>
> so, from what I understand, Storm 2.6.0 no longer exposes some shaded
> classes such as:
>
> org.apache.storm.shade.org.json.simple.JSONObject;
>
> Interestingly, it looks similar to this issue:
> https://stackoverflow.com/questions/33377291/java-lang-noclassdeffounderror-org-json-simple-jsonvalue-at-storm-kafka-dynamic
>
> Whatever, I guess I can only blame our own code for using Storm's
> shaded classes, right ?
> Note it's been running as is for long time and not breaking since many
> upgrades, of course it's not an excuse...
>
> I'll fix our code & continue checking if I see other impacts of 2.5.0
> -> 2.6.0 upgrade...
>
> Thanks,
> Alexandre
>
> Le sam. 4 nov. 2023 à 10:36, Alexandre Vermeerbergen
>  a écrit :
> >
> > Just did a 'git push' with that trivial fix... is that enough?
> >
> > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla  a écrit :
> > >
> > > You can do as you like ;-)
> > >
> > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen 
> > > :
> > > >I mean :
> > > >
> > > >Can I keep using my
> > > >'STORM-3958-Capacity-to-set-St
> > > >orm-UI's-title-in-conf/storm.yaml'
> > > >branch to push this fix, or should I create a new one?
> > > >
> > > >Le sam. 4 nov. 2023 à 08:53, Alexandre Verme

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Yes another regression I found with my topologies (built with 2.5.0
dependencies) running to Storm 2.6.0 SNAPSHOT 2023/11/02: I have seen
several exceptions like this one:

2023-11-04 18:43:31.696 o.a.s.e.e.ReportError
Thread-16-alertsHistoryUMToKafka-executor[27, 27] [ERROR] Error
java.lang.RuntimeException: java.lang.NoClassDefFoundError:
org.apache.storm.shade.org.json.simple.JSONObject
at org.apache.storm.utils.Utils$1.run(Utils.java:413)
~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
at java.lang.Thread.run(Thread.java:857) [?:?]
Caused by: java.lang.NoClassDefFoundError:
org.apache.storm.shade.org.json.simple.JSONObject
at 
com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessageFromTuple(AlertHistoryTupleToKafkaMapper.java:25)
~[stormjar.jar:?]
at 
com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessageFromTuple(AlertHistoryTupleToKafkaMapper.java:11)
~[stormjar.jar:?]
at org.apache.storm.kafka.bolt.KafkaBolt.process(KafkaBolt.java:177)
~[stormjar.jar:?]

and here's the related source code:

package com.acme.storm.evaluator.bolt;

import org.apache.storm.kafka.bolt.mapper.TupleToKafkaMapper;
import org.apache.storm.shade.org.json.simple.JSONObject;
import org.apache.storm.tuple.Tuple;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.acme.storm.util.TupleAlertHistoryFields;

public class AlertHistoryTupleToKafkaMapper implements
TupleToKafkaMapper {
private static final long serialVersionUID = 5668103889461721134L;

public static final Logger logger =
LoggerFactory.getLogger(AlertHistoryTupleToKafkaMapper.class);

@Override
public String getKeyFromTuple(Tuple tuple) {
return 
tuple.getStringByField(TupleAlertHistoryFields.SERVICE_DEFINITION_NAME);
}

@SuppressWarnings("unchecked")
@Override
public String getMessageFromTuple(Tuple tuple)
{
JSONObject tupleJson = new JSONObject();

tuple.getFields().forEach(f -> {
tupleJson.put(f, tuple.getValueByField(f));
});

String jsonString = tupleJson.toJSONString();

return jsonString;
}
}

line #25 is this one:

JSONObject tupleJson = new JSONObject();

so, from what I understand, Storm 2.6.0 no longer exposes some shaded
classes such as:

org.apache.storm.shade.org.json.simple.JSONObject;

Interestingly, it looks similar to this issue:
https://stackoverflow.com/questions/33377291/java-lang-noclassdeffounderror-org-json-simple-jsonvalue-at-storm-kafka-dynamic

Whatever, I guess I can only blame our own code for using Storm's
shaded classes, right ?
Note it's been running as is for long time and not breaking since many
upgrades, of course it's not an excuse...

I'll fix our code & continue checking if I see other impacts of 2.5.0
-> 2.6.0 upgrade...

Thanks,
Alexandre

Le sam. 4 nov. 2023 à 10:36, Alexandre Vermeerbergen
 a écrit :
>
> Just did a 'git push' with that trivial fix... is that enough?
>
> Le sam. 4 nov. 2023 à 09:34, Richard Zowalla  a écrit :
> >
> > You can do as you like ;-)
> >
> > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen 
> > :
> > >I mean :
> > >
> > >Can I keep using my
> > >'STORM-3958-Capacity-to-set-St
> > >orm-UI's-title-in-conf/storm.yaml'
> > >branch to push this fix, or should I create a new one?
> > >
> > >Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > a écrit :
> > >>
> > >> Can you keep using my
> > >> 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-conf/storm.yaml'
> > >> branch to push this fix, or should I create a new one?
> > >>
> > >> Le sam. 4 nov. 2023 à 08:14, Richard Zowalla  a 
> > >> écrit :
> > >> >
> > >> > Just push a fix for it (referencing the original issue) ;-) - we 
> > >> > hadn't a release in between, so not a big deal.
> > >> >
> > >> > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre Vermeerbergen 
> > >> > :
> > >> > >Temporary workaround for the Storm UI title's regression: run this
> > >> > >command on the title-template.html file (in
> > >> > >storm-stable/public/templates directory) after having extracted the
> > >> > >binary archive:
> > >> > >
> > >> > >sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > >> > >
> > >> > >Again sorry for having introduced this regression.
> > >> > >
> > >> > >Alexandre
> > >> > >
> > >> > >Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > >> > > a écrit :
> > >> > >>
> &

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Just did a 'git push' with that trivial fix... is that enough?

Le sam. 4 nov. 2023 à 09:34, Richard Zowalla  a écrit :
>
> You can do as you like ;-)
>
> Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen 
> :
> >I mean :
> >
> >Can I keep using my
> >'STORM-3958-Capacity-to-set-St
> >orm-UI's-title-in-conf/storm.yaml'
> >branch to push this fix, or should I create a new one?
> >
> >Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > a écrit :
> >>
> >> Can you keep using my
> >> 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-conf/storm.yaml'
> >> branch to push this fix, or should I create a new one?
> >>
> >> Le sam. 4 nov. 2023 à 08:14, Richard Zowalla  a écrit 
> >> :
> >> >
> >> > Just push a fix for it (referencing the original issue) ;-) - we hadn't 
> >> > a release in between, so not a big deal.
> >> >
> >> > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre Vermeerbergen 
> >> > :
> >> > >Temporary workaround for the Storm UI title's regression: run this
> >> > >command on the title-template.html file (in
> >> > >storm-stable/public/templates directory) after having extracted the
> >> > >binary archive:
> >> > >
> >> > >sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> >> > >
> >> > >Again sorry for having introduced this regression.
> >> > >
> >> > >Alexandre
> >> > >
> >> > >Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> >> > > a écrit :
> >> > >>
> >> > >> Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
> >> > >>  1. I have modified my KO bolts using this pattern in their prepare() 
> >> > >> method:
> >> > >>
> >> > >>long cacheMaxSize =
> >> > >> Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
> >> > >>
> >> > >>instead of:
> >> > >>
> >> > >>long cacheMaxSize = (Long) 
> >> > >> stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> >> > >>
> >> > >> 2. I have found the cause of the regression in loss of 'Storm UI'
> >> > >> title at the top left-hand side of all Storm UI's pages.
> >> > >> This is a mistake of mine in this PR:
> >> > >> 
> >> > >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> >> > >> More precisely, at line 19 of this file:
> >> > >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> >> > >> we must have:
> >> > >> {{title}}
> >> > >>
> >> > >> instead of:
> >> > >>
> >> > >> {{ui.title}}
> >> > >>
> >> > >>I think I missed this during my tests because of browser's cache
> >> > >> (bad excuse, I know...): probably I tested by-patching my deployed
> >> > >> Storm cluster, and I finally forgot to checkout this latest change of
> >> > >> my delivery.
> >> > >>
> >> > >>I had to use "private navigation" to make sure this time it fixes 
> >> > >> this issue
> >> > >>
> >> > >> What would be the quickest way for this very simple fix to be 
> >> > >> delivered ?
> >> > >>
> >> > >> Thanks,
> >> > >> Alexandre
> >> > >>
> >> > >> Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> >> > >>  a écrit :
> >> > >> >
> >> > >> > Thanks Richard for the analysis.
> >> > >> >
> >> > >> > My thought on this are very simple: let me fix my own old code, 
> >> > >> > after
> >> > >> > all this was impacting only one of my topologies, the other one were
> >> > >> > using more careful Integer.parseInt or Integer.parseLong to avoid
> >> > >> > making assumption on the way the values in the config Map were
> >> > >> 

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

I mean :

Can I keep using my
'STORM-3958-Capacity-to-set-St
orm-UI's-title-in-conf/storm.yaml'
branch to push this fix, or should I create a new one?

Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
 a écrit :
>
> Can you keep using my
> 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-conf/storm.yaml'
> branch to push this fix, or should I create a new one?
>
> Le sam. 4 nov. 2023 à 08:14, Richard Zowalla  a écrit :
> >
> > Just push a fix for it (referencing the original issue) ;-) - we hadn't a 
> > release in between, so not a big deal.
> >
> > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre Vermeerbergen 
> > :
> > >Temporary workaround for the Storm UI title's regression: run this
> > >command on the title-template.html file (in
> > >storm-stable/public/templates directory) after having extracted the
> > >binary archive:
> > >
> > >sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > >
> > >Again sorry for having introduced this regression.
> > >
> > >Alexandre
> > >
> > >Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > > a écrit :
> > >>
> > >> Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
> > >>  1. I have modified my KO bolts using this pattern in their prepare() 
> > >> method:
> > >>
> > >>long cacheMaxSize =
> > >> Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
> > >>
> > >>instead of:
> > >>
> > >>long cacheMaxSize = (Long) 
> > >> stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > >>
> > >> 2. I have found the cause of the regression in loss of 'Storm UI'
> > >> title at the top left-hand side of all Storm UI's pages.
> > >> This is a mistake of mine in this PR:
> > >> 
> > >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > >> More precisely, at line 19 of this file:
> > >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > >> we must have:
> > >> {{title}}
> > >>
> > >> instead of:
> > >>
> > >> {{ui.title}}
> > >>
> > >>I think I missed this during my tests because of browser's cache
> > >> (bad excuse, I know...): probably I tested by-patching my deployed
> > >> Storm cluster, and I finally forgot to checkout this latest change of
> > >> my delivery.
> > >>
> > >>I had to use "private navigation" to make sure this time it fixes 
> > >> this issue
> > >>
> > >> What would be the quickest way for this very simple fix to be delivered ?
> > >>
> > >> Thanks,
> > >> Alexandre
> > >>
> > >> Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> > >>  a écrit :
> > >> >
> > >> > Thanks Richard for the analysis.
> > >> >
> > >> > My thought on this are very simple: let me fix my own old code, after
> > >> > all this was impacting only one of my topologies, the other one were
> > >> > using more careful Integer.parseInt or Integer.parseLong to avoid
> > >> > making assumption on the way the values in the config Map were
> > >> > serialized.
> > >> >
> > >> > My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
> > >> > UI" title at the top left hand side of each page is missing. This may
> > >> > be the result of one of my own PR, I'll have to check why...
> > >> >
> > >> > Alex
> > >> >
> > >> > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla  a écrit 
> > >> > :
> > >> > >
> > >> > > Hi,
> > >> > >
> > >> > > thanks for the feedback.
> > >> > >
> > >> > > The main difference to 2.5.0 is, that we switched the JSON
> > >> > > implementation in Utils, which relied on a
> > >> > > super old JSON library. What is happening here is, that the new JSON
> > >> > > library (net.minidev) behaves differently.
> > >> > >
> > >> > > We can see it by writing some simple code without St

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Can you keep using my
'STORM-3958-Capacity-to-set-Storm-UI's-title-in-conf/storm.yaml'
branch to push this fix, or should I create a new one?

Le sam. 4 nov. 2023 à 08:14, Richard Zowalla  a écrit :
>
> Just push a fix for it (referencing the original issue) ;-) - we hadn't a 
> release in between, so not a big deal.
>
> Am 4. November 2023 08:01:54 MEZ schrieb Alexandre Vermeerbergen 
> :
> >Temporary workaround for the Storm UI title's regression: run this
> >command on the title-template.html file (in
> >storm-stable/public/templates directory) after having extracted the
> >binary archive:
> >
> >sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> >
> >Again sorry for having introduced this regression.
> >
> >Alexandre
> >
> >Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > a écrit :
> >>
> >> Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
> >>  1. I have modified my KO bolts using this pattern in their prepare() 
> >> method:
> >>
> >>long cacheMaxSize =
> >> Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
> >>
> >>instead of:
> >>
> >>long cacheMaxSize = (Long) 
> >> stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> >>
> >> 2. I have found the cause of the regression in loss of 'Storm UI'
> >> title at the top left-hand side of all Storm UI's pages.
> >> This is a mistake of mine in this PR:
> >> 
> >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> >> More precisely, at line 19 of this file:
> >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> >> we must have:
> >> {{title}}
> >>
> >> instead of:
> >>
> >> {{ui.title}}
> >>
> >>I think I missed this during my tests because of browser's cache
> >> (bad excuse, I know...): probably I tested by-patching my deployed
> >> Storm cluster, and I finally forgot to checkout this latest change of
> >> my delivery.
> >>
> >>I had to use "private navigation" to make sure this time it fixes this 
> >> issue
> >>
> >> What would be the quickest way for this very simple fix to be delivered ?
> >>
> >> Thanks,
> >> Alexandre
> >>
> >> Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> >>  a écrit :
> >> >
> >> > Thanks Richard for the analysis.
> >> >
> >> > My thought on this are very simple: let me fix my own old code, after
> >> > all this was impacting only one of my topologies, the other one were
> >> > using more careful Integer.parseInt or Integer.parseLong to avoid
> >> > making assumption on the way the values in the config Map were
> >> > serialized.
> >> >
> >> > My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
> >> > UI" title at the top left hand side of each page is missing. This may
> >> > be the result of one of my own PR, I'll have to check why...
> >> >
> >> > Alex
> >> >
> >> > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla  a écrit :
> >> > >
> >> > > Hi,
> >> > >
> >> > > thanks for the feedback.
> >> > >
> >> > > The main difference to 2.5.0 is, that we switched the JSON
> >> > > implementation in Utils, which relied on a
> >> > > super old JSON library. What is happening here is, that the new JSON
> >> > > library (net.minidev) behaves differently.
> >> > >
> >> > > We can see it by writing some simple code without Storm (The related
> >> > > code line with the parsing is taken from Storm Utils)
> >> > >
> >> > >
> >> > > Map conf = new HashMap<>();
> >> > > conf.put("test", 1L);
> >> > >
> >> > > Map deserTopoConf = (Map)
> >> > > JSONValue.parseWithException(JSONValue.toJSONString(conf));
> >> > >
> >> > > for(Map.Entry e : deserTopoConf.entrySet()) {
> >> > >   System.out.println(e.getValue().getClass().getSimpleName());
> >> > > }
> >> > >
> >> > >

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Temporary workaround for the Storm UI title's regression: run this
command on the title-template.html file (in
storm-stable/public/templates directory) after having extracted the
binary archive:

sed -i 's/{{ui\.title}}/{{title}}/' title-template.html

Again sorry for having introduced this regression.

Alexandre

Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
 a écrit :
>
> Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
>  1. I have modified my KO bolts using this pattern in their prepare() method:
>
>long cacheMaxSize =
> Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
>
>instead of:
>
>long cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
>
> 2. I have found the cause of the regression in loss of 'Storm UI'
> title at the top left-hand side of all Storm UI's pages.
> This is a mistake of mine in this PR:
> 
> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> More precisely, at line 19 of this file:
> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> we must have:
> {{title}}
>
> instead of:
>
> {{ui.title}}
>
>I think I missed this during my tests because of browser's cache
> (bad excuse, I know...): probably I tested by-patching my deployed
> Storm cluster, and I finally forgot to checkout this latest change of
> my delivery.
>
>I had to use "private navigation" to make sure this time it fixes this 
> issue
>
> What would be the quickest way for this very simple fix to be delivered ?
>
> Thanks,
> Alexandre
>
> Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
>  a écrit :
> >
> > Thanks Richard for the analysis.
> >
> > My thought on this are very simple: let me fix my own old code, after
> > all this was impacting only one of my topologies, the other one were
> > using more careful Integer.parseInt or Integer.parseLong to avoid
> > making assumption on the way the values in the config Map were
> > serialized.
> >
> > My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
> > UI" title at the top left hand side of each page is missing. This may
> > be the result of one of my own PR, I'll have to check why...
> >
> > Alex
> >
> > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla  a écrit :
> > >
> > > Hi,
> > >
> > > thanks for the feedback.
> > >
> > > The main difference to 2.5.0 is, that we switched the JSON
> > > implementation in Utils, which relied on a
> > > super old JSON library. What is happening here is, that the new JSON
> > > library (net.minidev) behaves differently.
> > >
> > > We can see it by writing some simple code without Storm (The related
> > > code line with the parsing is taken from Storm Utils)
> > >
> > >
> > > Map conf = new HashMap<>();
> > > conf.put("test", 1L);
> > >
> > > Map deserTopoConf = (Map)
> > > JSONValue.parseWithException(JSONValue.toJSONString(conf));
> > >
> > > for(Map.Entry e : deserTopoConf.entrySet()) {
> > >   System.out.println(e.getValue().getClass().getSimpleName());
> > > }
> > >
> > > This will print "Integer". Using "1L" as a value will
> > > result in "Long".
> > >
> > > The main issue derives from the fact, that we serialize the config map
> > > into a JSON String and parse the result.
> > > Based on the String representation, "1L" will still fit into an
> > > Integer, so the parser will not use Long ;-)
> > >
> > > If the number is big enough, the parser will switch to a Long.
> > > The old library (> 10 years old now):
> > >
> > > deserTopoConf = (Map)
> > > org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
> > > toJSONString(conf));
> > >
> > > for(Map.Entry e : deserTopoConf.entrySet()) {
> > >   System.out.println(e.getValue().getClass().getSimpleName());
> > > }
> > >
> > > This will print "Long".
> > >
> > > The difference between these libraries is, that the super old
> > > org.json.simple just always returns Long as the type.
> > >
> > > If can eventually tune the new JSON parser to something like:
> > >
> > >   deser

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
 1. I have modified my KO bolts using this pattern in their prepare() method:

   long cacheMaxSize =
Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());

   instead of:

   long cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);

2. I have found the cause of the regression in loss of 'Storm UI'
title at the top left-hand side of all Storm UI's pages.
This is a mistake of mine in this PR:

https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
More precisely, at line 19 of this file:
https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
we must have:
{{title}}

instead of:

{{ui.title}}

   I think I missed this during my tests because of browser's cache
(bad excuse, I know...): probably I tested by-patching my deployed
Storm cluster, and I finally forgot to checkout this latest change of
my delivery.

   I had to use "private navigation" to make sure this time it fixes this issue

What would be the quickest way for this very simple fix to be delivered ?

Thanks,
Alexandre

Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
 a écrit :
>
> Thanks Richard for the analysis.
>
> My thought on this are very simple: let me fix my own old code, after
> all this was impacting only one of my topologies, the other one were
> using more careful Integer.parseInt or Integer.parseLong to avoid
> making assumption on the way the values in the config Map were
> serialized.
>
> My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
> UI" title at the top left hand side of each page is missing. This may
> be the result of one of my own PR, I'll have to check why...
>
> Alex
>
> Le ven. 3 nov. 2023 à 20:23, Richard Zowalla  a écrit :
> >
> > Hi,
> >
> > thanks for the feedback.
> >
> > The main difference to 2.5.0 is, that we switched the JSON
> > implementation in Utils, which relied on a
> > super old JSON library. What is happening here is, that the new JSON
> > library (net.minidev) behaves differently.
> >
> > We can see it by writing some simple code without Storm (The related
> > code line with the parsing is taken from Storm Utils)
> >
> >
> > Map conf = new HashMap<>();
> > conf.put("test", 1L);
> >
> > Map deserTopoConf = (Map)
> > JSONValue.parseWithException(JSONValue.toJSONString(conf));
> >
> > for(Map.Entry e : deserTopoConf.entrySet()) {
> >   System.out.println(e.getValue().getClass().getSimpleName());
> > }
> >
> > This will print "Integer". Using "1L" as a value will
> > result in "Long".
> >
> > The main issue derives from the fact, that we serialize the config map
> > into a JSON String and parse the result.
> > Based on the String representation, "1L" will still fit into an
> > Integer, so the parser will not use Long ;-)
> >
> > If the number is big enough, the parser will switch to a Long.
> > The old library (> 10 years old now):
> >
> > deserTopoConf = (Map)
> > org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
> > toJSONString(conf));
> >
> > for(Map.Entry e : deserTopoConf.entrySet()) {
> >   System.out.println(e.getValue().getClass().getSimpleName());
> > }
> >
> > This will print "Long".
> >
> > The difference between these libraries is, that the super old
> > org.json.simple just always returns Long as the type.
> >
> > If can eventually tune the new JSON parser to something like:
> >
> >   deserTopoConf = (Map) new
> > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
> > ing(conf), JSONValue.defaultReader.DEFAULT);
> >
> > This will result in "Long" but might have other side-effects.
> >
> > From my point of view, the newer JSON library works as expected
> > (although it might break backward compatibility).
> > Unchecked casts are always evil, imho. In your specific case, it might
> > be more save to cast to "Number" and than invoke "longValue()" instead.
> >
> > Nevertheless, we might being able to restore the old behaviour by
> > tuning the JSONParser configuration.
> >
> > Thoughts?
> >
> > Gruß
> > Richard
> >
> >
> >
> >
> > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
> > Verm

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Thanks Richard for the analysis.

My thought on this are very simple: let me fix my own old code, after
all this was impacting only one of my topologies, the other one were
using more careful Integer.parseInt or Integer.parseLong to avoid
making assumption on the way the values in the config Map were
serialized.

My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
UI" title at the top left hand side of each page is missing. This may
be the result of one of my own PR, I'll have to check why...

Alex

Le ven. 3 nov. 2023 à 20:23, Richard Zowalla  a écrit :
>
> Hi,
>
> thanks for the feedback.
>
> The main difference to 2.5.0 is, that we switched the JSON
> implementation in Utils, which relied on a
> super old JSON library. What is happening here is, that the new JSON
> library (net.minidev) behaves differently.
>
> We can see it by writing some simple code without Storm (The related
> code line with the parsing is taken from Storm Utils)
>
>
> Map conf = new HashMap<>();
> conf.put("test", 1L);
>
> Map deserTopoConf = (Map)
> JSONValue.parseWithException(JSONValue.toJSONString(conf));
>
> for(Map.Entry e : deserTopoConf.entrySet()) {
>   System.out.println(e.getValue().getClass().getSimpleName());
> }
>
> This will print "Integer". Using "1L" as a value will
> result in "Long".
>
> The main issue derives from the fact, that we serialize the config map
> into a JSON String and parse the result.
> Based on the String representation, "1L" will still fit into an
> Integer, so the parser will not use Long ;-)
>
> If the number is big enough, the parser will switch to a Long.
> The old library (> 10 years old now):
>
> deserTopoConf = (Map)
> org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
> toJSONString(conf));
>
> for(Map.Entry e : deserTopoConf.entrySet()) {
>   System.out.println(e.getValue().getClass().getSimpleName());
> }
>
> This will print "Long".
>
> The difference between these libraries is, that the super old
> org.json.simple just always returns Long as the type.
>
> If can eventually tune the new JSON parser to something like:
>
>   deserTopoConf = (Map) new
> JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
> ing(conf), JSONValue.defaultReader.DEFAULT);
>
> This will result in "Long" but might have other side-effects.
>
> From my point of view, the newer JSON library works as expected
> (although it might break backward compatibility).
> Unchecked casts are always evil, imho. In your specific case, it might
> be more save to cast to "Number" and than invoke "longValue()" instead.
>
> Nevertheless, we might being able to restore the old behaviour by
> tuning the JSONParser configuration.
>
> Thoughts?
>
> Gruß
> Richard
>
>
>
>
> Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
> Vermeerbergen:
> > On a side note, the Long value that "MyKOBolt" is set by a
> > DefaultTopologyConfig to a value with is a java.lang.Long instance...
> > hence it's surprising to see it internally converted into a
> > java.lang.Integer in the Map passed to prepare() method...
> >
> >
> > Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
> >  a écrit :
> > >
> > > Hello,
> > >
> > > I found a breaking change using Apache Storm 2.6.0 snapshot build
> > > dated 20231103 and without changing my topologies' code (keeping
> > > the
> > > same big jars).
> > >
> > > This seems to be tied to a datatype change in Map argument of
> > > prepare() method of Bolt.
> > >
> > > For example, this Bolt is KO at runtime:
> > >
> > > public class MyKOBolt extends BaseRichBolt {
> > >
> > > @Override
> > > public void prepare(Map stormConf, TopologyContext context,
> > > OutputCollector collector) {
> > > cacheMaxSize = (Long)
> > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > cacheTtlMin = (Long)
> > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > >
> > > as it throws this error:
> > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > > Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
> > > __system:[-1]
> > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop
> > > died!
> > > java.lang.ClassCastException: java.lang.Integer incompati

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

On a side note, the Long value that "MyKOBolt" is set by a
DefaultTopologyConfig to a value with is a java.lang.Long instance...
hence it's surprising to see it internally converted into a
java.lang.Integer in the Map passed to prepare() method...


Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
 a écrit :
>
> Hello,
>
> I found a breaking change using Apache Storm 2.6.0 snapshot build
> dated 20231103 and without changing my topologies' code (keeping the
> same big jars).
>
> This seems to be tied to a datatype change in Map argument of
> prepare() method of Bolt.
>
> For example, this Bolt is KO at runtime:
>
> public class MyKOBolt extends BaseRichBolt {
>
> @Override
> public void prepare(Map stormConf, TopologyContext context,
> OutputCollector collector) {
> cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> cacheTtlMin = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
>
> as it throws this error:
> 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
> __system:[-1]
> 2023-11-03 16:42:20.193 o.a.s.u.Utils
> Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop died!
> java.lang.ClassCastException: java.lang.Integer incompatible with 
> java.lang.Long
>  at com.acme.storm.alerting.MyKOBolt .prepare(EvaluationBolt.java:71)
> ~[stormjar.jar:?]
>  at org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:128)
> ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
>  at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:138)
> ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
>  at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:54)
> ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
>  at org.apache.storm.utils.Utils$1.run(Utils.java:393)
> [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
>  at java.lang.Thread.run(Thread.java:857) [?:?]
>
> whereas this bolt is fine with Storm 2.6.0:
>
> public class MyOKBolt extends BaseBasicBolt {
>
> @Override
> public void prepare(final Map stormConf, final TopologyContext context) {
> super.prepare(stormConf, context);
>
> this.redisPort   = Integer.parseInt((String)
> stormConf.get(ConfKeys.REDIS_PORT));
>
> That said, I can modify the code of "MyKOBolt" to use same
> Integer.parseInt  or Integer.parseLong trick, but it's the first time
> in my long history of upgrades that I have seen such runtime
> incompatibility.
>
> Maybe "MyKOBolt" was badly written since the beginning and I have just
> hit the punishment for it: is there a documentation which clarifies
> datatypes of the Map argument of prepare() method?
>
> Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
> "MyOKBolt" derives from BaseBasicBolt => could this have any impact on
> this finding ?
>
> Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK 17.0.8.1.
>
> Kind regards,
> Alexandre
>
> Le ven. 3 nov. 2023 à 15:16, Julien Nioche
>  a écrit :
> >
> > Thanks Richard.
> >
> > Tried the latest snapshot with StormCrawler both in local and deployed mode
> > and did not find any issues.
> > Will try it on a topology generating WARC files next week to check that the
> > dependency changes on Hadoop have not broken anything.
> >
> > Have a good week end
> >
> > Julien
> >
> > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla  wrote:
> >
> > > Short update. Sorted out an issue with the tar.gz/zip with Julien today
> > > and re-uploaded them to the nightlies area.
> > >
> > > This new bundle works as expected in my deployment but happy to receive
> > > additional feedback before getting up a first release candidate ;-)
> > >
> > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard Zowalla:
> > > > Hi Alexandre,
> > > >
> > > > we are not in a hurry here :) - take as much as time you need.
> > > >
> > > > Gruß
> > > > Richard
> > > >
> > > >
> > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > > > Vermeerbergen:
> > > > > Hello Richard,
> > > > >
> > > > > Okay, I'm more than happy to do that with my pre-production cluster
> > > > > (~10 topologies) using the binary artifacts.
> > > > > Would it be OK if I can use up to end of this week  so that I'll be
> > > > > able to have enough time to check all potential issues that this
> > > > > upg

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Hello,

I found a breaking change using Apache Storm 2.6.0 snapshot build
dated 20231103 and without changing my topologies' code (keeping the
same big jars).

This seems to be tied to a datatype change in Map argument of
prepare() method of Bolt.

For example, this Bolt is KO at runtime:

public class MyKOBolt extends BaseRichBolt {

@Override
public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
cacheTtlMin = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);

as it throws this error:
2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
__system:[-1]
2023-11-03 16:42:20.193 o.a.s.u.Utils
Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop died!
java.lang.ClassCastException: java.lang.Integer incompatible with java.lang.Long
 at com.acme.storm.alerting.MyKOBolt .prepare(EvaluationBolt.java:71)
~[stormjar.jar:?]
 at org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:128)
~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
 at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:138)
~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
 at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:54)
~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
 at org.apache.storm.utils.Utils$1.run(Utils.java:393)
[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
 at java.lang.Thread.run(Thread.java:857) [?:?]

whereas this bolt is fine with Storm 2.6.0:

public class MyOKBolt extends BaseBasicBolt {

@Override
public void prepare(final Map stormConf, final TopologyContext context) {
super.prepare(stormConf, context);

this.redisPort   = Integer.parseInt((String)
stormConf.get(ConfKeys.REDIS_PORT));

That said, I can modify the code of "MyKOBolt" to use same
Integer.parseInt  or Integer.parseLong trick, but it's the first time
in my long history of upgrades that I have seen such runtime
incompatibility.

Maybe "MyKOBolt" was badly written since the beginning and I have just
hit the punishment for it: is there a documentation which clarifies
datatypes of the Map argument of prepare() method?

Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
"MyOKBolt" derives from BaseBasicBolt => could this have any impact on
this finding ?

Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK 17.0.8.1.

Kind regards,
Alexandre

Le ven. 3 nov. 2023 à 15:16, Julien Nioche
 a écrit :
>
> Thanks Richard.
>
> Tried the latest snapshot with StormCrawler both in local and deployed mode
> and did not find any issues.
> Will try it on a topology generating WARC files next week to check that the
> dependency changes on Hadoop have not broken anything.
>
> Have a good week end
>
> Julien
>
> On Thu, 2 Nov 2023 at 19:25, Richard Zowalla  wrote:
>
> > Short update. Sorted out an issue with the tar.gz/zip with Julien today
> > and re-uploaded them to the nightlies area.
> >
> > This new bundle works as expected in my deployment but happy to receive
> > additional feedback before getting up a first release candidate ;-)
> >
> > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard Zowalla:
> > > Hi Alexandre,
> > >
> > > we are not in a hurry here :) - take as much as time you need.
> > >
> > > Gruß
> > > Richard
> > >
> > >
> > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > > Vermeerbergen:
> > > > Hello Richard,
> > > >
> > > > Okay, I'm more than happy to do that with my pre-production cluster
> > > > (~10 topologies) using the binary artifacts.
> > > > Would it be OK if I can use up to end of this week  so that I'll be
> > > > able to have enough time to check all potential issues that this
> > > > upgrade bring?
> > > >
> > > > Thanks,
> > > > Alexandre
> > > >
> > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla  a
> > > > écrit :
> > > > >
> > > > > Hi all,
> > > > >
> > > > > before starting a release of 2.6.0-SNAPSHOT, I would like to
> > > > > receive
> > > > > some feedback on the current SNAPSHOT build.
> > > > >
> > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > > > >
> > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > >
> > > > > It is available via the ASF Snapshot repository:
> > > > > https://repos

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Apologies: I have been way too busy many unrelated stuff (including,
but not limited to bad weather, which is ironic when thinking about
Storm ;); so I haven't yet started testing pre-vote Storm 2.6.0.
Trying to bootstrap this activity tomorrow, if the sky is nice enough
not try falling on my head twice!

Alexandre

Le jeu. 2 nov. 2023 à 20:24, Richard Zowalla  a écrit :
>
> Short update. Sorted out an issue with the tar.gz/zip with Julien today
> and re-uploaded them to the nightlies area.
>
> This new bundle works as expected in my deployment but happy to receive
> additional feedback before getting up a first release candidate ;-)
>
> Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard Zowalla:
> > Hi Alexandre,
> >
> > we are not in a hurry here :) - take as much as time you need.
> >
> > Gruß
> > Richard
> >
> >
> > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > Vermeerbergen:
> > > Hello Richard,
> > >
> > > Okay, I'm more than happy to do that with my pre-production cluster
> > > (~10 topologies) using the binary artifacts.
> > > Would it be OK if I can use up to end of this week  so that I'll be
> > > able to have enough time to check all potential issues that this
> > > upgrade bring?
> > >
> > > Thanks,
> > > Alexandre
> > >
> > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla  a
> > > écrit :
> > > >
> > > > Hi all,
> > > >
> > > > before starting a release of 2.6.0-SNAPSHOT, I would like to
> > > > receive
> > > > some feedback on the current SNAPSHOT build.
> > > >
> > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > >
> > > > It is available via the ASF Snapshot repository:
> > > > https://repository.apache.org/content/repositories/snapshots/
> > > >
> > > > You can consume by adding
> > > >
> > > >  
> > > >   apache.snapshots
> > > >   Apache Snapshot Repository
> > > >   https://repository.apache.org/snapshots
> > > >   
> > > > false
> > > >   
> > > >  
> > > >
> > > > to your project pom. As we do not deploy SNAPSHOTS automatically,
> > > > it
> > > > should be easy to just consume the latest SNAPSHOT.
> > > >
> > > > The packaged binaries are available at nightlies.apache.org:
> > > >
> > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > >
> > > > If you have some minutes left: Please test and report any issues
> > > > with
> > > > this binaries, so we can fix before attempting to release.
> > > >
> > > > The most significant changes are in the are of hadoop/hbase/hdfs
> > > > as
> > > > we
> > > > upgraded from 2.x to 3.x - our own test coverage within the build
> > > > looks
> > > > good but would be  nice to get some real world use-case feedback
> > > > on
> > > > this. In addition, we had quite a lof of 3rd party dependency
> > > > upgrades.
> > > >
> > > > In addition, it contains the pruning of external modules as
> > > > listed
> > > > in
> > > >
> > > > https://issues.apache.org/jira/browse/STORM-3988
> > > >
> > > > A summaryin Jira is here:
> > > >
> > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820=12353484
> > > >
> > > > Gruß
> > > > Richard
> > > >
> > > >
> >
>

Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT

[Alexandre Vermeerbergen]

Hello Richard,

Okay, I'm more than happy to do that with my pre-production cluster
(~10 topologies) using the binary artifacts.
Would it be OK if I can use up to end of this week  so that I'll be
able to have enough time to check all potential issues that this
upgrade bring?

Thanks,
Alexandre

Le lun. 30 oct. 2023 à 07:35, Richard Zowalla  a écrit :
>
> Hi all,
>
> before starting a release of 2.6.0-SNAPSHOT, I would like to receive
> some feedback on the current SNAPSHOT build.
>
> I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
>
> It is available via the ASF Snapshot repository:
> https://repository.apache.org/content/repositories/snapshots/
>
> You can consume by adding
>
>  
>   apache.snapshots
>   Apache Snapshot Repository
>   https://repository.apache.org/snapshots
>   
> false
>   
>  
>
> to your project pom. As we do not deploy SNAPSHOTS automatically, it
> should be easy to just consume the latest SNAPSHOT.
>
> The packaged binaries are available at nightlies.apache.org:
>
> https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
>
> If you have some minutes left: Please test and report any issues with
> this binaries, so we can fix before attempting to release.
>
> The most significant changes are in the are of hadoop/hbase/hdfs as we
> upgraded from 2.x to 3.x - our own test coverage within the build looks
> good but would be  nice to get some real world use-case feedback on
> this. In addition, we had quite a lof of 3rd party dependency upgrades.
>
> In addition, it contains the pruning of external modules as listed in
>
> https://issues.apache.org/jira/browse/STORM-3988
>
> A summaryin Jira is here:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820=12353484
>
> Gruß
> Richard
>
>

Re: [VOTE] [RESULT] Removal of modules residing in /external (2nd attempt)

[Alexandre Vermeerbergen]

Thank you very much for casting this vote Richard !

Le jeu. 19 oct. 2023 à 08:52, Richard Zowalla  a écrit :
>
> Hi all,
>
> this vote passes with the following +1
>
> Alexandre Vermeerbergen (binding)
> Julien Nioche (binding)
> Richard Zowalla (binding)
>
> I'll prepare a Jira + PR for the change.
>
> Gruß
> Richard
>
> Am Donnerstag, dem 12.10.2023 um 10:59 +0200 schrieb Richard Zowalla:
> > Hello all,
> >
> > This is the 2nd attempt due to an oversight in the components to be
> > voted on.
> >
> > A few weeks ago we conducted a survey [1] and discussed the results
> > [2]. Corresponding details about the background can be found there.
> > The
> > short version: We have too few resources to keep all modules up to
> > date. Old dependencies make it difficult to update the project.
> >
> > Now it's time for us to make an appropriate decision. For the sake of
> > simplicity, let's vote on the whole package. If there are differences
> > (ie. a VETO for this code change), we can still vote per module.
> >
> > Please keep in mind that corresponding modules can also exist outside
> > the main project (e.g. as a fork). A removed module can also be added
> > back at some point if it is updated and maintained appropriately.
> >
> > We are voting on the following proposal:
> >
> > Remove the following external components:
> >
> > - storm-cassandra
> > - storm-eventhubs
> > - storm-hbase
> > - storm-hive
> > - storm-kinesis
> > - storm-mongodb
> > - storm-mqtt
> > - storm-openmtsdb
> > - storm-pmml
> > - storm-pulsar
> > - storm-rocketmq
> > - storm-solr
> >
> > Keep the following components:
> >
> > - storm-autocreds (required by UI)
> > - storm-blobstore-migration
> > - storm-elasticsearch
> > - storm-jdbc
> > - storm-hdfs-*
> > - storm-jms
> > - storm-kafka-*
> > - storm-metrics
> > - storm-redis
> >
> >
> > "How to vote" on code modification is described here:
> > https://www.apache.org/foundation/voting.html#votes-on-code-modification
> >
> > Only votes from the Storm PMC are binding, but everyone is welcome to
> > vote. The vote passes if at least three binding +1 votes are cast.
> >
> > Please note: A -1 vote by a qualified voter (PMC) stops a code-
> > modification proposal in its tracks. This constitutes a veto, and it
> > cannot be overruled nor overridden by anyone. Vetoes stand until and
> > unless the individual withdraws their veto. To prevent vetoes from
> > being used capriciously, the voter must provide with the veto a
> > technical justification showing why the change is bad (opens a
> > security
> > exposure, negatively affects performance, etc. ). A veto without a
> > justification is invalid and has no weight.
> >
> >
> > Please VOTE on the removal of the modules mentioned above. The vote
> > is
> > open for at least the next 72 hours or as long as needed.
> >
> > Please vote:
> >
> > [ ] +1 Remove the modules mentioned above
> > [ ]  0 No opinion
> > [ ] -1 Do NOT remove because {VETO REASON}
> >
> > Gruß
> > Richard
> >
> > [1] https://lists.apache.org/thread/f0396c98ttt1688ys9jlxd3wx3ykvch4
> > [2] https://lists.apache.org/thread/s7nsrq4byn7z1b4504l8hy7vlj7f0bpl
> >
>

Re: [VOTE] Removal of modules residing in /external (2nd attempt)

[Alexandre Vermeerbergen]

Hello,

+1 (binding)

Thanks,
Alexandre

Le jeu. 12 oct. 2023 à 10:59, Richard Zowalla  a écrit :
>
> Hello all,
>
> This is the 2nd attempt due to an oversight in the components to be
> voted on.
>
> A few weeks ago we conducted a survey [1] and discussed the results
> [2]. Corresponding details about the background can be found there. The
> short version: We have too few resources to keep all modules up to
> date. Old dependencies make it difficult to update the project.
>
> Now it's time for us to make an appropriate decision. For the sake of
> simplicity, let's vote on the whole package. If there are differences
> (ie. a VETO for this code change), we can still vote per module.
>
> Please keep in mind that corresponding modules can also exist outside
> the main project (e.g. as a fork). A removed module can also be added
> back at some point if it is updated and maintained appropriately.
>
> We are voting on the following proposal:
>
> Remove the following external components:
>
> - storm-cassandra
> - storm-eventhubs
> - storm-hbase
> - storm-hive
> - storm-kinesis
> - storm-mongodb
> - storm-mqtt
> - storm-openmtsdb
> - storm-pmml
> - storm-pulsar
> - storm-rocketmq
> - storm-solr
>
> Keep the following components:
>
> - storm-autocreds (required by UI)
> - storm-blobstore-migration
> - storm-elasticsearch
> - storm-jdbc
> - storm-hdfs-*
> - storm-jms
> - storm-kafka-*
> - storm-metrics
> - storm-redis
>
>
> "How to vote" on code modification is described here:
> https://www.apache.org/foundation/voting.html#votes-on-code-modification
>
> Only votes from the Storm PMC are binding, but everyone is welcome to
> vote. The vote passes if at least three binding +1 votes are cast.
>
> Please note: A -1 vote by a qualified voter (PMC) stops a code-
> modification proposal in its tracks. This constitutes a veto, and it
> cannot be overruled nor overridden by anyone. Vetoes stand until and
> unless the individual withdraws their veto. To prevent vetoes from
> being used capriciously, the voter must provide with the veto a
> technical justification showing why the change is bad (opens a security
> exposure, negatively affects performance, etc. ). A veto without a
> justification is invalid and has no weight.
>
>
> Please VOTE on the removal of the modules mentioned above. The vote is
> open for at least the next 72 hours or as long as needed.
>
> Please vote:
>
> [ ] +1 Remove the modules mentioned above
> [ ]  0 No opinion
> [ ] -1 Do NOT remove because {VETO REASON}
>
> Gruß
> Richard
>
> [1] https://lists.apache.org/thread/f0396c98ttt1688ys9jlxd3wx3ykvch4
> [2] https://lists.apache.org/thread/s7nsrq4byn7z1b4504l8hy7vlj7f0bpl
>

Re: [VOTE] Removal of modules residing in /external

[Alexandre Vermeerbergen]

Hello,

+1 (binding)

Thanks,
Alexandre

Le jeu. 12 oct. 2023 à 10:15, Edward Armes  a écrit :
>
> -1 (non-binding)
>  I would maybe think about keeping JDBC connector and dropping the Redis
> connector as the JDBC connector could potentially include Redis.
>
> Edward
>
> On Thu, 12 Oct 2023, 08:33 Richard Zowalla,  wrote:
>
> > Hello all,
> >
> > A few weeks ago we conducted a survey [1] and discussed the results
> > [2]. Corresponding details about the background can be found there. The
> > short version: We have too few resources to keep all modules up to
> > date. Old dependencies make it difficult to update the project.
> >
> > Now it's time for us to make an appropriate decision. For the sake of
> > simplicity, let's vote on the whole package. If there are differences
> > (ie. a VETO for this code change), we can still vote per module.
> >
> > Please keep in mind that corresponding modules can also exist outside
> > the main project (e.g. as a fork). A removed module can also be added
> > back at some point if it is updated and maintained appropriately.
> >
> > We are voting on the following proposal:
> >
> > Remove the following external components:
> >
> > - storm-cassandra
> > - storm-eventhubs
> > - storm-hbase
> > - storm-hive
> > - storm-jdbc
> > - storm-kinesis
> > - storm-mongodb
> > - storm-mqtt
> > - storm-openmtsdb
> > - storm-pmml
> > - storm-pulsar
> > - storm-rocketmq
> > - storm-solr
> >
> > Keep the following components:
> >
> > - storm-autocreds
> > - storm-blobstore-migration
> > - storm-hdfs-*
> > - storm-jms
> > - storm-kafka-*
> > - storm-metrics
> > - storm-redis
> >
> >
> > "How to vote" on code modification is described here:
> > https://www.apache.org/foundation/voting.html#votes-on-code-modification
> >
> > Only votes from the Storm PMC are binding, but everyone is welcome to
> > vote. The vote passes if at least three binding +1 votes are cast.
> >
> > Please note: A -1 vote by a qualified voter (PMC) stops a code-
> > modification proposal in its tracks. This constitutes a veto, and it
> > cannot be overruled nor overridden by anyone. Vetoes stand until and
> > unless the individual withdraws their veto. To prevent vetoes from
> > being used capriciously, the voter must provide with the veto a
> > technical justification showing why the change is bad (opens a security
> > exposure, negatively affects performance, etc. ). A veto without a
> > justification is invalid and has no weight.
> >
> >
> > Please VOTE on the removal of the modules mentioned above. The vote is
> > open for at least the next 72 hours or as long as needed.
> >
> > Please vote:
> >
> > [ ] +1 Remove the modules mentioned above
> > [ ]  0 No opinion
> > [ ] -1 Do NOT remove because {VETO REASON}
> >
> > Gruß
> > Richard
> >
> > [1] https://lists.apache.org/thread/f0396c98ttt1688ys9jlxd3wx3ykvch4
> > [2] https://lists.apache.org/thread/s7nsrq4byn7z1b4504l8hy7vlj7f0bpl
> >

Re: [DISCUSS] Future of /external modules -> next steps?

[Alexandre Vermeerbergen]

I agree with Julien.

Le mar. 3 oct. 2023 à 09:20, Julien Nioche
 a écrit :
>
> Hi
>
> Thanks Richard. There are some dubious votes indeed.
>
> My views is that we could keep only the modules with 4 votes or more :
> cassandra, elasticsearch, hdfs-*, jms, kafka*, metrics, redis
>
> The rest can be removed. If people feel strongly about them, they can
> always copy the code and maintain it in external repos (which can of course
> be contributed back later on if they prove to be popular).
> Given the low number of active committers on the project, any reduction on
> the size of the code or number of dependencies to maintain is a good thing.
>
> Julien
>
>
>
> On Mon, 2 Oct 2023 at 07:04, Richard Zowalla  wrote:
>
> > Hi all,
> >
> > our survey is closed and here are the results. We only have a sample
> > size of n=11, which isn't great but somehow represents the current
> > state of Storm (just revived from the Attic).
> >
> > You find the results here: [1] (csv + image). User comments are also
> > added to the gist.
> >
> > We had some votes from people, who aren't using Storm at the moment and
> > just need a technical comparison between Storm, Flink and Spark to
> > decide, which one to use. Therefore, it seems, that they marked every
> > module as "must have" ;-) - as a take-a-way: it might be good to
> > provide such comparison on our website.
> >
> > We should discuss, which modules are worth being kept in the main
> > repository and which modules can be extracted (for example into a
> > "storm-external" repository).
> >
> > Due to the tight coupling for security (in the UI), we need to keep the
> > hadoop-related stuff. Kafka things are also a thing, it seems.
> >
> > Some of the moduels are used by (new) PMC members, so they might have
> > the energy to start an update initiative on these modules.
> >
> > Happy to hear your thoughts!
> >
> > Gruß
> > Richard
> >
> >
> >
> > [1] https://gist.github.com/rzo1/2a1d20acfc20da050ea515770c6606df
> >
>
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble 

Re: How about javax to jakarta migration in Storm?

[Alexandre Vermeerbergen]

Thanks!
 I created https://issues.apache.org/jira/browse/STORM-3985

Le jeu. 28 sept. 2023 à 20:04, Richard Zowalla  a écrit :
>
> Hi,
>
> yes, the coordinate move came with EE9, although EE8 will still be around 
> some day.
>
> It will require to update Jetty _and_ check transient dependencies for 
> compatibility. I didn't check but it might be that some auth libs (from 
> hadoop) might rely on javax dependencies causing issues at runtime.
>
> Nothing against a Jira to track.
>
> Gruß
> Richard
>
>
>
> Am 28. September 2023 15:34:36 MESZ schrieb Alexandre Vermeerbergen 
> :
> >Hello,
> >
> >I noticed we still rely on javax.servlet in storm-server/pom.xml.
> >
> >Unless I am missing something, Java EE" APIs are supposed to be
> >deprecated in favor in Jakarta EE APIs.
> >
> >While I don't know the scope of needed changes in Storm to move to
> >Jakarta EE APIs, is there any objection if I create a JIRA to track
> >such change (potentially needing Jettty update, etc) ?
> >
> >Having a look to https://issues.apache.org/jira/browse/STORM-3961, I
> >see that Richard had this topic in mind, but I found no STORM JIRA
> >explicitly meaning "Upgrade to Jarkarta EE APIs"
> >
> >Thanks,
> >Alexandre

How about javax to jakarta migration in Storm?

[Alexandre Vermeerbergen]

Hello,

I noticed we still rely on javax.servlet in storm-server/pom.xml.

Unless I am missing something, Java EE" APIs are supposed to be
deprecated in favor in Jakarta EE APIs.

While I don't know the scope of needed changes in Storm to move to
Jakarta EE APIs, is there any objection if I create a JIRA to track
such change (potentially needing Jettty update, etc) ?

Having a look to https://issues.apache.org/jira/browse/STORM-3961, I
see that Richard had this topic in mind, but I found no STORM JIRA
explicitly meaning "Upgrade to Jarkarta EE APIs"

Thanks,
Alexandre

Re: [PR] [STORM-3970] Changes to storm-sql-core/pom.xml to avoid errors in Eclipse (storm)

[Alexandre Vermeerbergen]

Hi Bipin,

That will be wonderful if you find the way to fix these last 3 errors
in pom.xml files, thanks !!

By the way, I never experienced Eclipse IDE being confused about
files' content : just make sure if you edit the file outside Eclipse
IDE to use "Refresh" (when I doubt, I select all projects from Package
Explorer view, and I use [F5] to get everything on filesystem synced
to Eclipse IDE).

Kind regards,
Alexandre

Le ven. 1 sept. 2023 à 12:30, Bipin Prasad
 a écrit :
>
> Yes, I have fixes for those. They will be up there shortly. But at least you 
> are up and running and can use the power of UI.
> One thing surprised me with the latest eclipse - it is very easy for the 
> Editor to get confused about the content of a file (and loose the trailing 
> end of a file). I had to go a couple times in “vi” just to make sure that the 
> file did not end where eclipse screen showed.
>
>
> Sent from Yahoo Mail for iPhone
>
>
> On Friday, September 1, 2023, 8:53 AM, Alexandre Vermeerbergen 
>  wrote:
>
> And by the way, after copying the changes from STORM-3970 (and closing
> storm-cassandra project, for which I'm anticipating low community
> interest, so I'm skipping it), I only have 3 errors left in Eclipse
> IDE for Storm:
>
> On /storm-client/pom.xml:
>  line 206 (which just contains  tag) error message is:
> - Plugin execution not covered by lifecycle configuration:
> org.apache.maven.plugins:maven-dependency-plugin:2.8:unpack
> (execution: unpack, phase: process-test-resources)
> - Failed to execute mojo
> org.apache.maven.plugins:maven-dependency-plugin:2.8:unpack
> {execution: unpack}
> (org.apache.maven.plugins:maven-dependency-plugin:2.8:unpack:unpack:process-test-resources)
>
> On /storm-core/pom.xml:
>  line 367 (which just contains  tag) error message is:
>  - Failed to execute mojo
> org.apache.maven.plugins:maven-dependency-plugin:3.6.0:copy-dependencies
> {execution: copy-dependencies}
> (org.apache.maven.plugins:maven-dependency-plugin:3.6.0:copy-dependencies:copy-dependencies:package)
>
> On /storm-core/pom.xml:
>  line 381 (which just contains  tag) error message is:
>  - Failed to execute mojo
> org.apache.maven.plugins:maven-dependency-plugin:3.6.0:unpack
> {execution: unpack}
> (org.apache.maven.plugins:maven-dependency-plugin:3.6.0:unpack:unpack:process-test-resources)
>
> What is strange is although there are these 3 last errors, there are
> no more Java compilation errors...
>
> Any idea how to get rid of these 3 last ones (I'm intentionally
> forgetting storm-cassandra) ?
>
> Kind regards,
> Alexandre
>
> Le ven. 1 sept. 2023 à 11:44, avermeer (via GitHub)  a écrit 
> :
> >
> >
> > avermeer commented on PR #3578:
> > URL: https://github.com/apache/storm/pull/3578#issuecomment-1702951565
> >
> >I verified that this change greatly improves the situation is Eclipse 
> > IDE (by manually copying the changes to my local temporary branch for test 
> > purposes).
> >
> >May I merge it ?
> >
> >
> > --
> > This is an automated message from the Apache Git Service.
> > To respond to the message, please log on to GitHub and use the
> > URL above to go to the specific comment.
> >
> > To unsubscribe, e-mail: dev-unsubscr...@storm.apache.org
> >
> > For queries about this service, please contact Infrastructure at:
> > us...@infra.apache.org
> >
>
>
>

Re: [PR] [STORM-3970] Changes to storm-sql-core/pom.xml to avoid errors in Eclipse (storm)

[Alexandre Vermeerbergen]

And by the way, after copying the changes from STORM-3970 (and closing
storm-cassandra project, for which I'm anticipating low community
interest, so I'm skipping it), I only have 3 errors left in Eclipse
IDE for Storm:

On /storm-client/pom.xml:
 line 206 (which just contains  tag) error message is:
- Plugin execution not covered by lifecycle configuration:
org.apache.maven.plugins:maven-dependency-plugin:2.8:unpack
(execution: unpack, phase: process-test-resources)
- Failed to execute mojo
org.apache.maven.plugins:maven-dependency-plugin:2.8:unpack
{execution: unpack}
(org.apache.maven.plugins:maven-dependency-plugin:2.8:unpack:unpack:process-test-resources)

On /storm-core/pom.xml:
 line 367 (which just contains  tag) error message is:
 - Failed to execute mojo
org.apache.maven.plugins:maven-dependency-plugin:3.6.0:copy-dependencies
{execution: copy-dependencies}
(org.apache.maven.plugins:maven-dependency-plugin:3.6.0:copy-dependencies:copy-dependencies:package)

On /storm-core/pom.xml:
 line 381 (which just contains  tag) error message is:
 - Failed to execute mojo
org.apache.maven.plugins:maven-dependency-plugin:3.6.0:unpack
{execution: unpack}
(org.apache.maven.plugins:maven-dependency-plugin:3.6.0:unpack:unpack:process-test-resources)

What is strange is although there are these 3 last errors, there are
no more Java compilation errors...

Any idea how to get rid of these 3 last ones (I'm intentionally
forgetting storm-cassandra) ?

Kind regards,
Alexandre

Le ven. 1 sept. 2023 à 11:44, avermeer (via GitHub)  a écrit :
>
>
> avermeer commented on PR #3578:
> URL: https://github.com/apache/storm/pull/3578#issuecomment-1702951565
>
>I verified that this change greatly improves the situation is Eclipse IDE 
> (by manually copying the changes to my local temporary branch for test 
> purposes).
>
>May I merge it ?
>
>
> --
> This is an automated message from the Apache Git Service.
> To respond to the message, please log on to GitHub and use the
> URL above to go to the specific comment.
>
> To unsubscribe, e-mail: dev-unsubscr...@storm.apache.org
>
> For queries about this service, please contact Infrastructure at:
> us...@infra.apache.org
>

Re: Any clue to import Storm's pom.xml in Eclipse for contributing purposes?

[Alexandre Vermeerbergen]

All Storm projects imported in Eclipse are using "JRE System Library
[JavaSE-11]", in all projects, Java compiler compliance level,
Generated classes compatibility and Source compatibility are all set
to 11.

I hope it answers your question.

Kind regards,
Alexandre

Le jeu. 31 août 2023 à 13:30, Bipin Prasad
 a écrit :
>
>  I will look into this later today.Your project (subproject) jvm shows 
> jdk/jvm-11 (as visible in eclipse in Package Explorer window?
> On Thursday, August 31, 2023 at 06:15:55 AM PDT, Alexandre Vermeerbergen 
>  wrote:
>
>  Hello,
>
> Thank you very much Bipin for your advice.
> I did it, and good news, now the number of errors is much lower : from
> several thousand errors to 73 errors.
>
> First set of errors is very strange: in
> /storm-cassandra/src/test/java/org/apache/storm/cassandra/trident/WeatherBatchSpout.java,
> I have these imports:
>
> import java.util.ArrayList;
> import java.util.HashMap;
> import java.util.List;
> import java.util.Map;
> import java.util.Random;
> import java.util.UUID;
>
> with all the same errror: "The package java.util is accessible from
> more than one module: , java.base"
> => this look like an issue with the notion of Java modules
>
> Another error is on /storm-client/pom.xml, at line 206 with this long
> error message:
> -
> Multiple markers at this line - Plugin execution not covered by
> lifecycle configuration: org.apache.maven.plugins:maven-
> dependency-plugin:2.8:unpack (execution: unpack, phase:
> process-test-resources) - Failed to execute mojo
> org.apache.maven.plugins:maven-dependency-plugin:2.8:unpack
> {execution: unpack} (org.apache.maven.plugins:maven-
> dependency-plugin:2.8:unpack:unpack:process-test-resources)
> org.eclipse.core.runtime.CoreException: Failed to execute
> mojo org.apache.maven.plugins:maven-dependency-plugin:2.8:unpack
> {execution: unpack} at
> org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeMojo(MavenExecutionContext.java:340)
> at
> ... (more lines)
> -
>
> Any idea how to solve these 2 types of errors ? anything missing or to
> fix in related pom.xml files?
>
> Kind regards,
> Alexandre
>
> Le mer. 30 août 2023 à 23:41, Bipin Prasad
>  a écrit :
> >
> > Looks like eclipse (Also IntelliJ) doesn’t know about the relocated 
> > (shaded) class. Solution that I have used is to build Storm-shaded-deps 
> > directly in maven (mvn install), and then closing the storm-shaded-deps 
> > subproject in eclipse/IntelliJ. This forces the IDE to use the 
> > storm-shaded-jar from -/.m2 and errors go away.
> >
> >
> > Sent from Yahoo Mail for iPhone
> >
> >
> > On Wednesday, August 30, 2023, 8:19 PM, Alexandre Vermeerbergen 
> >  wrote:
> >
> > Hello;
> >
> > I have created a branch in my own fork of storm Github projet, and
> > merged Bipin's changes from these two branches on his own fork:
> >
> >  [STORM-3963] Add m2e plugin to storm-shaded-deps and few minor
> > pom.xml fixes => remotes/origin/storm-3963 =>
> > storm-shaded-deps/pom.xml
> >  [STORM-3967] Specify generated sources directory
> >=> remotes/origin/storm-3967 => pom.xml
> >
> > I cleaned everything that I previously had in Eclipse, then I tried to
> > again create an blank Eclipse workspace, then, using Eclipse's import
> > projects... / Maven / Existing Maven Projects and point to the root
> > directory of my local Storm git directory.
> >
> > Maybe one interesting point at this step is that I leave everything
> > selected by default "as is", including the "[x} Add project(s) to
> > working set", with "storm" as the associated value.
> >
> > then I clic on "Finish".
> >
> > After certain time of processing/downloading dependencies, Eclipse
> > shows me the "Discover m2e connectors" dialog with same list as in my
> > initial post, with again, no matter what I select, there is no way to
> > use "Finish" button
> >
> > => from here, clicking "on cancel", brings me to Eclipse IDE showing
> > tons of errors in the various storm projects created by the import
> > process.
> >
> > first one is the following line:
> >
> > /storm-server/src/main/java/org/apache/storm/zookeeper/AclEnforcement.java
> >
> >  The errors are because the following imports are not found:
> >
> >
> > import org.apache.storm.

Re: Any clue to import Storm's pom.xml in Eclipse for contributing purposes?

[Alexandre Vermeerbergen]

Hello,

Thank you very much Bipin for your advice.
I did it, and good news, now the number of errors is much lower : from
several thousand errors to 73 errors.

First set of errors is very strange: in
/storm-cassandra/src/test/java/org/apache/storm/cassandra/trident/WeatherBatchSpout.java,
I have these imports:

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Random;
import java.util.UUID;

with all the same errror: "The package java.util is accessible from
more than one module: , java.base"
=> this look like an issue with the notion of Java modules

Another error is on /storm-client/pom.xml, at line 206 with this long
error message:
-
Multiple markers at this line - Plugin execution not covered by
lifecycle configuration: org.apache.maven.plugins:maven-
dependency-plugin:2.8:unpack (execution: unpack, phase:
process-test-resources) - Failed to execute mojo
org.apache.maven.plugins:maven-dependency-plugin:2.8:unpack
{execution: unpack} (org.apache.maven.plugins:maven-
dependency-plugin:2.8:unpack:unpack:process-test-resources)
org.eclipse.core.runtime.CoreException: Failed to execute
mojo org.apache.maven.plugins:maven-dependency-plugin:2.8:unpack
{execution: unpack} at
org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeMojo(MavenExecutionContext.java:340)
at
... (more lines)
-

Any idea how to solve these 2 types of errors ? anything missing or to
fix in related pom.xml files?

Kind regards,
Alexandre

Le mer. 30 août 2023 à 23:41, Bipin Prasad
 a écrit :
>
> Looks like eclipse (Also IntelliJ) doesn’t know about the relocated (shaded) 
> class. Solution that I have used is to build Storm-shaded-deps directly in 
> maven (mvn install), and then closing the storm-shaded-deps subproject in 
> eclipse/IntelliJ. This forces the IDE to use the storm-shaded-jar from -/.m2 
> and errors go away.
>
>
> Sent from Yahoo Mail for iPhone
>
>
> On Wednesday, August 30, 2023, 8:19 PM, Alexandre Vermeerbergen 
>  wrote:
>
> Hello;
>
> I have created a branch in my own fork of storm Github projet, and
> merged Bipin's changes from these two branches on his own fork:
>
>  [STORM-3963] Add m2e plugin to storm-shaded-deps and few minor
> pom.xml fixes => remotes/origin/storm-3963 =>
> storm-shaded-deps/pom.xml
>  [STORM-3967] Specify generated sources directory
> => remotes/origin/storm-3967 => pom.xml
>
> I cleaned everything that I previously had in Eclipse, then I tried to
> again create an blank Eclipse workspace, then, using Eclipse's import
> projects... / Maven / Existing Maven Projects and point to the root
> directory of my local Storm git directory.
>
> Maybe one interesting point at this step is that I leave everything
> selected by default "as is", including the "[x} Add project(s) to
> working set", with "storm" as the associated value.
>
> then I clic on "Finish".
>
> After certain time of processing/downloading dependencies, Eclipse
> shows me the "Discover m2e connectors" dialog with same list as in my
> initial post, with again, no matter what I select, there is no way to
> use "Finish" button
>
> => from here, clicking "on cancel", brings me to Eclipse IDE showing
> tons of errors in the various storm projects created by the import
> process.
>
> first one is the following line:
>
> /storm-server/src/main/java/org/apache/storm/zookeeper/AclEnforcement.java
>
>  The errors are because the following imports are not found:
>
>
> import org.apache.storm.shade.org.apache.curator.framework.CuratorFramework;
> import org.apache.storm.shade.org.apache.zookeeper.KeeperException;
> import org.apache.storm.shade.org.apache.zookeeper.ZooDefs;
> import org.apache.storm.shade.org.apache.zookeeper.data.ACL;
> import org.apache.storm.shade.org.apache.zookeeper.data.Id;
> import 
> org.apache.storm.shade.org.apache.zookeeper.server.auth.DigestAuthenticationProvider;
>
> It is like storm-shaded-deps's pom.xml was not automatically
> generating the shared dependencies for storm-server module.
>
> Any idea what to do next ?
>
> Reminder, I'm using latest Eclipse IDE stable version:
>
> Eclipse IDE for Java Developers (includes Incubating components)
> Version: 2023-06 (4.28.0)
> Build id: 20230608-1333
>
>
> Kind regards,
> Alexandre
>
>
>
>
>
>
> Le mer. 30 août 2023 à 03:35, Richard Zowalla  a écrit :
> >
> > I think you can fork the project, merge Bipin's changes an

Re: Any clue to import Storm's pom.xml in Eclipse for contributing purposes?

[Alexandre Vermeerbergen]

Hello;

I have created a branch in my own fork of storm Github projet, and
merged Bipin's changes from these two branches on his own fork:

 [STORM-3963] Add m2e plugin to storm-shaded-deps and few minor
pom.xml fixes => remotes/origin/storm-3963 =>
storm-shaded-deps/pom.xml
 [STORM-3967] Specify generated sources directory
=> remotes/origin/storm-3967 => pom.xml

I cleaned everything that I previously had in Eclipse, then I tried to
again create an blank Eclipse workspace, then, using Eclipse's import
projects... / Maven / Existing Maven Projects and point to the root
directory of my local Storm git directory.

Maybe one interesting point at this step is that I leave everything
selected by default "as is", including the "[x} Add project(s) to
working set", with "storm" as the associated value.

then I clic on "Finish".

After certain time of processing/downloading dependencies, Eclipse
shows me the "Discover m2e connectors" dialog with same list as in my
initial post, with again, no matter what I select, there is no way to
use "Finish" button

=> from here, clicking "on cancel", brings me to Eclipse IDE showing
tons of errors in the various storm projects created by the import
process.

first one is the following line:

/storm-server/src/main/java/org/apache/storm/zookeeper/AclEnforcement.java

 The errors are because the following imports are not found:


import org.apache.storm.shade.org.apache.curator.framework.CuratorFramework;
import org.apache.storm.shade.org.apache.zookeeper.KeeperException;
import org.apache.storm.shade.org.apache.zookeeper.ZooDefs;
import org.apache.storm.shade.org.apache.zookeeper.data.ACL;
import org.apache.storm.shade.org.apache.zookeeper.data.Id;
import 
org.apache.storm.shade.org.apache.zookeeper.server.auth.DigestAuthenticationProvider;

It is like storm-shaded-deps's pom.xml was not automatically
generating the shared dependencies for storm-server module.

Any idea what to do next ?

Reminder, I'm using latest Eclipse IDE stable version:

Eclipse IDE for Java Developers (includes Incubating components)
Version: 2023-06 (4.28.0)
Build id: 20230608-1333


Kind regards,
Alexandre






Le mer. 30 août 2023 à 03:35, Richard Zowalla  a écrit :
>
> I think you can fork the project, merge Bipin's changes and test with
> Eclipse locally. If the changes solve the issues, it might be a good
> thing otherwise we need to ensure, Eclipse users have a chance to be
> onboarded :)
>
> Am Dienstag, dem 29.08.2023 um 23:14 -0400 schrieb Alexandre
> Vermeerbergen:
> > Hello Julien,
> >
> > Here's the precise version of Eclipse IDE which I am using:
> > ==
> > Eclipse IDE for Java Developers (includes Incubating components)
> >
> > Version: 2023-06 (4.28.0)
> > Build id: 20230608-1333
> > ===
> >
> > And here are the versions of my M2E - Maven Integration for Eclipse's
> > components:
> >  Eclipse.org - m2eM2E Maven Integration for Eclipse Core
> > 2.1.2.20230523-2106org.eclipse.m2e.logback.feature
> >  Eclipse.org - m2eM2E Maven Integration for Eclipse Core
> > 2.3.0.20230523-2033org.eclipse.m2e.feature
> >
> >
> > Unlike Binpin, I can wait forever, the tons of compile issues are
> > still there.
> >
> > I have seen that Binpin is currently doing a pull request to importe
> > stuff in Storm's pom.xml files, maybe should I wait until this is
> > merged to master and then re-try ?
> >
> > Kind regards,
> > Alexandre
> >
> > Le mar. 29 août 2023 à 05:39, Julien Nioche
> >  a écrit :
> > >
> > > Hi Alexandre
> > >
> > > I use Eclipse and have imported Storm with Import -> Existing Maven
> > > project
> > > etc.. but am not getting the error message you mentioned. Which
> > > version of
> > > Eclipse are you using?
> > > I then get tons of compilation errors, like you do.
> > >
> > > Julien
> > >
> > >
> > > On Mon, 28 Aug 2023 at 02:23, Alexandre Vermeerbergen <
> > > avermeerber...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > I am trying to use my Eclipse IDE to work on a contribution on
> > > > storm
> > > > project.
> > > >
> > > > I must be missing something, because I though I was just a matter
> > > > of
> > > > using Eclipse's import projects... / Maven / Existing Maven
> > > > Projects
> > > > and point to the root directory of my local Storm git directory.
> > > >
> &g

Re: Any clue to import Storm's pom.xml in Eclipse for contributing purposes?

[Alexandre Vermeerbergen]

Hello Julien,

Here's the precise version of Eclipse IDE which I am using:
==
Eclipse IDE for Java Developers (includes Incubating components)

Version: 2023-06 (4.28.0)
Build id: 20230608-1333
===

And here are the versions of my M2E - Maven Integration for Eclipse's
components:
 Eclipse.org - m2eM2E Maven Integration for Eclipse Core
2.1.2.20230523-2106org.eclipse.m2e.logback.feature
 Eclipse.org - m2eM2E Maven Integration for Eclipse Core
2.3.0.20230523-2033org.eclipse.m2e.feature


Unlike Binpin, I can wait forever, the tons of compile issues are still there.

I have seen that Binpin is currently doing a pull request to importe
stuff in Storm's pom.xml files, maybe should I wait until this is
merged to master and then re-try ?

Kind regards,
Alexandre

Le mar. 29 août 2023 à 05:39, Julien Nioche
 a écrit :
>
> Hi Alexandre
>
> I use Eclipse and have imported Storm with Import -> Existing Maven project
> etc.. but am not getting the error message you mentioned. Which version of
> Eclipse are you using?
> I then get tons of compilation errors, like you do.
>
> Julien
>
>
> On Mon, 28 Aug 2023 at 02:23, Alexandre Vermeerbergen <
> avermeerber...@gmail.com> wrote:
>
> > Hello,
> >
> > I am trying to use my Eclipse IDE to work on a contribution on storm
> > project.
> >
> > I must be missing something, because I though I was just a matter of
> > using Eclipse's import projects... / Maven / Existing Maven Projects
> > and point to the root directory of my local Storm git directory.
> >
> > But this import project wizard shows me a 'Setup Maven plugin
> > connectors' dialog with the following table, having two columns:
> > "Maven Build" and "Action" and 9 lines.
> >
> > Here are the 2 first lines in JSON:
> >
> > [{"Maven
> > Build":"build-helper-maven-plugin:1.5:add-source","Action":"install
> > buildhelper"},
> > {"Maven Build":"clojure-maven-plugin:1.8.4:compile","Action":"Use
> > Workspace Default"}]
> >
> > and from there I cannot click on Finish button, no matter what I try.
> >
> > If I close this dialog, Eclipse seem to have imported all sources, but
> > I get tons of compilation errors, like the ones related to missing
> > shaded packages, such as:
> >
> > import org.apache.storm.shade.org
> > .apache.curator.framework.CuratorFramework;
> > import org.apache.storm.shade.org.apache.zookeeper.KeeperException;
> > import org.apache.storm.shade.org.apache.zookeeper.ZooDefs;
> > import org.apache.storm.shade.org.apache.zookeeper.data.ACL;
> > import org.apache.storm.shade.org.apache.zookeeper.data.Id;
> > import org.apache.storm.shade.org
> > .apache.zookeeper.server.auth.DigestAuthenticationProvider;
> >
> > Sorry if that sounds trivial to current Storm developers, but I'm
> > confused what to do to get a "clean" Eclipse workspace to start
> > contributing ; and I once I will have understood (which you help, you
> > Storm developers who already solved that kind of issue) I will
> > document this in Contributing page.
> >
> > Kind regards,
> > Alexandre
> >
>
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble <http://twitter.com/digitalpebble>

Any clue to import Storm's pom.xml in Eclipse for contributing purposes?

[Alexandre Vermeerbergen]

Hello,

I am trying to use my Eclipse IDE to work on a contribution on storm project.

I must be missing something, because I though I was just a matter of
using Eclipse's import projects... / Maven / Existing Maven Projects
and point to the root directory of my local Storm git directory.

But this import project wizard shows me a 'Setup Maven plugin
connectors' dialog with the following table, having two columns:
"Maven Build" and "Action" and 9 lines.

Here are the 2 first lines in JSON:

[{"Maven Build":"build-helper-maven-plugin:1.5:add-source","Action":"install
buildhelper"},
{"Maven Build":"clojure-maven-plugin:1.8.4:compile","Action":"Use
Workspace Default"}]

and from there I cannot click on Finish button, no matter what I try.

If I close this dialog, Eclipse seem to have imported all sources, but
I get tons of compilation errors, like the ones related to missing
shaded packages, such as:

import org.apache.storm.shade.org.apache.curator.framework.CuratorFramework;
import org.apache.storm.shade.org.apache.zookeeper.KeeperException;
import org.apache.storm.shade.org.apache.zookeeper.ZooDefs;
import org.apache.storm.shade.org.apache.zookeeper.data.ACL;
import org.apache.storm.shade.org.apache.zookeeper.data.Id;
import 
org.apache.storm.shade.org.apache.zookeeper.server.auth.DigestAuthenticationProvider;

Sorry if that sounds trivial to current Storm developers, but I'm
confused what to do to get a "clean" Eclipse workspace to start
contributing ; and I once I will have understood (which you help, you
Storm developers who already solved that kind of issue) I will
document this in Contributing page.

Kind regards,
Alexandre

Is it normal not to receive notifications on dev list when new Storm JIRA are created?

[Alexandre Vermeerbergen]

Hello,

I recently created https://issues.apache.org/jira/browse/STORM-3958,
and I realize that this creation is not automatically associated with
a mail to dev@storm.apache.org.

Maybe this is normal, but I'm wondering: aren't we risking to miss
interesting feature requests and/or bug reports if people open Storm
JIRA and think somebody in Storm DEV will notice it?

Sorry if that's a naive question...

Thanks;
Alexandre

Re: [Discuss] updating Community/People page on storm-site

[Alexandre Vermeerbergen]

Okay thanks, so I guess I need to wait until next Tuesday to get this
pull request to be merged, right ?
Next naive question: who should click on "Merge Pull Request" here
https://github.com/apache/storm-site/pull/45#pullrequestreview-1585813522
?

Thank,
Alexandre

Le dim. 20 août 2023 à 09:10, Richard Zowalla  a écrit :
>
> Wait for reviews + 2-3 days before merging (timezones, holidays, etc), so 
> people have a chance to look.
>
> In case of "storm-site":
>
> It uses "asf-site" branch and will automatically picked up by the ASF site 
> builder and gets published once a new commit is put to that branch ;-)
>
> Am 19. August 2023 22:04:07 MESZ schrieb Alexandre Vermeerbergen 
> :
> >I just sent pull request https://github.com/apache/storm-site/pull/45
> >for this change to storm-site.
> >
> >Since it's my first pull request, I am curious: what is the next step
> >to get this change merged, and then Apache Storm site to be actually
> >updated?
> >
> >Note: I have tested my change on my machine using:
> >bundle exec jekyll serve -w
> >(as documented in storm-site/README.md)
> >
> >Also, I have includes in my pull request a simple change to
> >.gitignore, so as to avoid Gemfile.lock file to be part of the
> >repository (it's a transient file generated by the bundle XXX command)
> >
> >Kind regards,
> >Alexandre
> >
> >Le ven. 18 août 2023 à 22:00, Alexandre Vermeerbergen
> > a écrit :
> >>
> >> OK, created https://issues.apache.org/jira/browse/STORM-3957 for the
> >> tracking (self assigned)
> >>
> >> Le ven. 18 août 2023 à 21:48, Richard Zowalla  a écrit :
> >> >
> >> > I am fine with it.
> >> >
> >> > It might be a nice to have to track website changes in Jira, imho.
> >> > We won'T include such changes in a release's changelog but might be a
> >> > good thing to know, that someone is working on that task.
> >> >
> >> > Gruß
> >> > Richard
> >> >
> >> > Am Freitag, dem 18.08.2023 um 19:59 +0200 schrieb Alexandre
> >> > Vermeerbergen:
> >> > > Hello,
> >> > >
> >> > > Community/People page on storm-site
> >> > > (https://storm.apache.org/contribute/People.html) is currently
> >> > > outdated.
> >> > >
> >> > > For example, it is missing Julien Nioche who recently joined as a
> >> > > PMC.
> >> > >
> >> > > I volunteer to fix this page, by replacing the static table fed by
> >> > > _data/committers.yml file, by a link to ASF's
> >> > > https://projects.apache.org/committee.html?storm
> >> > >
> >> > > My idea is to replace existing People.md file's content by this
> >> > > content:
> >> > >
> >> > > ---
> >> > > title: People
> >> > > layout: documentation
> >> > > documentation: true
> >> > > ---
> >> > >
> >> > > Current list of committers and PMC's are
> >> > > [here](https://projects.apache.org/committee.html?storm)
> >> > >
> >> > >
> >> > > and to remove _data/committers.yml file.
> >> > >
> >> > > Any objection?
> >> > >
> >> > > Do I need to create a JIRA and then a branch prefixed by JIRA number,
> >> > > like it's required for code changes ?
> >> > >
> >> > > Note: I am aware that there is the same issue with Project Leader &
> >> > > Committers list at https://github.com/apache/storm#readme, but here I
> >> > > am starting with storm-site.
> >> > >
> >> > > Kind regards,
> >> > > Alexandre
> >> >

Re: [Discuss] updating Community/People page on storm-site

[Alexandre Vermeerbergen]

I just sent pull request https://github.com/apache/storm-site/pull/45
for this change to storm-site.

Since it's my first pull request, I am curious: what is the next step
to get this change merged, and then Apache Storm site to be actually
updated?

Note: I have tested my change on my machine using:
bundle exec jekyll serve -w
(as documented in storm-site/README.md)

Also, I have includes in my pull request a simple change to
.gitignore, so as to avoid Gemfile.lock file to be part of the
repository (it's a transient file generated by the bundle XXX command)

Kind regards,
Alexandre

Le ven. 18 août 2023 à 22:00, Alexandre Vermeerbergen
 a écrit :
>
> OK, created https://issues.apache.org/jira/browse/STORM-3957 for the
> tracking (self assigned)
>
> Le ven. 18 août 2023 à 21:48, Richard Zowalla  a écrit :
> >
> > I am fine with it.
> >
> > It might be a nice to have to track website changes in Jira, imho.
> > We won'T include such changes in a release's changelog but might be a
> > good thing to know, that someone is working on that task.
> >
> > Gruß
> > Richard
> >
> > Am Freitag, dem 18.08.2023 um 19:59 +0200 schrieb Alexandre
> > Vermeerbergen:
> > > Hello,
> > >
> > > Community/People page on storm-site
> > > (https://storm.apache.org/contribute/People.html) is currently
> > > outdated.
> > >
> > > For example, it is missing Julien Nioche who recently joined as a
> > > PMC.
> > >
> > > I volunteer to fix this page, by replacing the static table fed by
> > > _data/committers.yml file, by a link to ASF's
> > > https://projects.apache.org/committee.html?storm
> > >
> > > My idea is to replace existing People.md file's content by this
> > > content:
> > >
> > > ---
> > > title: People
> > > layout: documentation
> > > documentation: true
> > > ---
> > >
> > > Current list of committers and PMC's are
> > > [here](https://projects.apache.org/committee.html?storm)
> > >
> > >
> > > and to remove _data/committers.yml file.
> > >
> > > Any objection?
> > >
> > > Do I need to create a JIRA and then a branch prefixed by JIRA number,
> > > like it's required for code changes ?
> > >
> > > Note: I am aware that there is the same issue with Project Leader &
> > > Committers list at https://github.com/apache/storm#readme, but here I
> > > am starting with storm-site.
> > >
> > > Kind regards,
> > > Alexandre
> >

Re: [Discuss] updating Community/People page on storm-site

[Alexandre Vermeerbergen]

OK, created https://issues.apache.org/jira/browse/STORM-3957 for the
tracking (self assigned)

Le ven. 18 août 2023 à 21:48, Richard Zowalla  a écrit :
>
> I am fine with it.
>
> It might be a nice to have to track website changes in Jira, imho.
> We won'T include such changes in a release's changelog but might be a
> good thing to know, that someone is working on that task.
>
> Gruß
> Richard
>
> Am Freitag, dem 18.08.2023 um 19:59 +0200 schrieb Alexandre
> Vermeerbergen:
> > Hello,
> >
> > Community/People page on storm-site
> > (https://storm.apache.org/contribute/People.html) is currently
> > outdated.
> >
> > For example, it is missing Julien Nioche who recently joined as a
> > PMC.
> >
> > I volunteer to fix this page, by replacing the static table fed by
> > _data/committers.yml file, by a link to ASF's
> > https://projects.apache.org/committee.html?storm
> >
> > My idea is to replace existing People.md file's content by this
> > content:
> >
> > ---
> > title: People
> > layout: documentation
> > documentation: true
> > ---
> >
> > Current list of committers and PMC's are
> > [here](https://projects.apache.org/committee.html?storm)
> >
> >
> > and to remove _data/committers.yml file.
> >
> > Any objection?
> >
> > Do I need to create a JIRA and then a branch prefixed by JIRA number,
> > like it's required for code changes ?
> >
> > Note: I am aware that there is the same issue with Project Leader &
> > Committers list at https://github.com/apache/storm#readme, but here I
> > am starting with storm-site.
> >
> > Kind regards,
> > Alexandre
>

[Discuss] updating Community/People page on storm-site

[Alexandre Vermeerbergen]

Hello,

Community/People page on storm-site
(https://storm.apache.org/contribute/People.html) is currently
outdated.

For example, it is missing Julien Nioche who recently joined as a PMC.

I volunteer to fix this page, by replacing the static table fed by
_data/committers.yml file, by a link to ASF's
https://projects.apache.org/committee.html?storm

My idea is to replace existing People.md file's content by this content:

---
title: People
layout: documentation
documentation: true
---

Current list of committers and PMC's are
[here](https://projects.apache.org/committee.html?storm)


and to remove _data/committers.yml file.

Any objection?

Do I need to create a JIRA and then a branch prefixed by JIRA number,
like it's required for code changes ?

Note: I am aware that there is the same issue with Project Leader &
Committers list at https://github.com/apache/storm#readme, but here I
am starting with storm-site.

Kind regards,
Alexandre

Re: [DISCUSS] Migrate to GitHub Issues (and stop using Jira)

[Alexandre Vermeerbergen]

Hello Richard,

On the one hand, I have nothing against the use of GitHub issues
instead of JIRA issues. I agree that JIRA is kind of overkill for the
simple need of tracking issues for Storm project.

On the other hand, the migration of existing JIRA issues to GitHub
issues is likely to take time.
Given that recently Storm project was threatened to go attic because
of its decreasing activity, I'm wondering if such migration is a fair
use of Storm's contributors.

Maybe an estimation of work to be done would help:
- How many opened JIRA issues are there?
- Is there a way to (at least partially) automate JIRA to GitHub
issues migration?
   I guess that manually opening a GitHub issue and copying both the
title, initial content description, copy URL to initial JIRA, add
attached materiel (if any) and copying related discussions will be an
hassle...

Thanks,
Alexandre

Le mer. 16 août 2023 à 09:24, Richard Zowalla  a écrit :
>
> Hi all,
>
> I'd like to take a moment to discuss something that I believe will
> significantly enhance our project's collaboration and attract new
> contributors: migrating from Jira to GitHub Issues and GitHub
> Discussions.
>
> We aren't using Jira very well in most cases, and the requirement for a
> Jira ticket for a code change leads to people just creating new Jira
> tickets, rather than searching to see if there already exists a ticket
> for that feature. Additionally most of the committers follow a similar
> path of "work on feature, open Jira ticket just before creating PR".
>
> One downside is we might get people opening issues for "help, how do I
> do this" -- I think we can address that by having an issue template
> saying something like "DO NOT OPEN AN ISSUE ASKING FOR HELP - ask on
> users@ or in GitHub Discussions. Another downside would be the vendor
> lock related to GitHub.
>
> In addition, the ASF JIRA instance no longer allows users to self-sign-
> up (though it DOES allow us to invite others). This is largely due to
> spam.
>
> Most developers have a GitHub account (and that is where most folks
> interact with Storm's code base), and this would meet folks where they
> are, as opposed to making them go somewhere else to dig through issues
> or to start working on them. Enabling GitHub discussions would also
> enable users to work where they are and we can still sent everything to
> the related mailing lists to ensure archivability.
>
>
> Tentative suggestion:
> * Go through existing Storm JIRA issues and close any issues that are
> no
> longer relevant.
> * Migrate open issues to GitHub Issues in apache/storm (linking back to
> the
> JIRA issue)
> * All new issues will only be created in GitHub Issues
> * When an existing JIRA issue is fixed, it will be updated in both
> locations
>
> Potential process changes:
> * Release notes/change logs are currently generated through JIRA, which
> we would need to change in order to make it happen.
>
> Thoughts?
>
> Gruß
> Richard

Re: Addendum to Storm's release process

[Alexandre Vermeerbergen]

Hello,

Ah sorry: I indeed accepted Richard's invitation to join as a PMC, but I
thought I had to wait for some formal confirmation before I could cast a
binding vote - hence my mention as "non-binding " with my vote.

Should I recast it, or I am right to wait for some formal annouce of my PMC
status?

Alexandre



Le sam. 5 août 2023, 19:16, Richard Zowalla  a écrit :

> Hey Bipin,
>
> think it is sufficient, if you put your own +1 to the original thread.
> Given that Alexandre has also accepted his invitation, we should be good
> now.
>
> Thanks for doing the release!
>
> Gruß
> Richard
>
>
>
>
> Am 5. August 2023 19:09:40 MESZ schrieb Bipin Prasad <
> bipinpra...@apache.org>:
> >Richard,
> >Thanks for this note on the vote process. I will ask for further
> votes on the to ensure compliance - even though RC4 was moved to release
> area. Will send out an email with the new locations. Once an additional
> vote is registered, I will send out the vote result email.
> >
> >--Bipin
> >
> >On 2023/08/04 20:05:35 Richard Zowalla wrote:
> >> Hi all,
> >>
> >>
> >> During the last release vote, I noticed the following things that we
> >> should do better next time to adhere to the ASF release guidelines
> >> [1,2,3]:
> >>
> >> - We need at least 3 (binding) votes for a release. A binding vote
> >> originates from PMC members only (although community is encouraged to
> >> also cast non-binding votes and test the binaries). For 2.5.0, we only
> >> had 2 (binding) votes for RC4 and an implicit one (at least I assume
> >> it) by our release manager, which would need to be cast next time, so
> >> we can formally do the release according to the foundation's policies.
> >>
> >> - A successful vote should be followed by a [VOTE] [RESULT] mail to
> >> indicate, that the vote was successful and the release process
> >> continues.
> >>
> >> - If multiple RC's are needed to do a release, we would need to sent a
> >> mail with [VOTE] [CANCELLED] to indicate the failed attempt and start a
> >> new mail thread with the next release candidate. This make the process
> >> more transparent to everyone.
> >>
> >> No blaming, just my observation. I think, that we can do better next
> >> time :)
> >>
> >> Gruß
> >> Richard
> >>
> >> [1] https://infra.apache.org/release-publishing.html
> >> [2] https://www.apache.org/legal/release-policy.html
> >> [3] https://www.apache.org/foundation/voting.html
> >>
>

Re: Storm 2.5.0 Release Candidate Vote

[Alexandre Vermeerbergen]

+1 [non binding]

Alexandre

Le sam. 5 août 2023, 19:24, Bipin Prasad  a écrit :

> Note: In order to be compliant with ASF procedures, we need an additional
> vote for Storm 2.5.0 RC4 (which was released yesterday) . Please test the
> following and vote:
>
> Storm Release candidate version 2.5.0 rc4 was earlier released yesterday to
> here: https://dist.apache.org/repos/dist/release/storm/apache-storm-2.5.0
> Please vote on this release.
>
>1. How to vote is described here:
>
> https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
>2. Release notes are here:
>
> https://dist.apache.org/repos/dist/release/storm/apache-storm-2.5.0/RELEASE_NOTES.html
>.
>3. The tag/commit to be voted upon is v2.5.0:
>4. The source archive being votes upon is here:
>
> https://dist.apache.org/repos/dist/release/storm/apache-storm-2.5.0/apache-storm-2.5.0-src.tar.gz
>5. The release artifacts are signed with the following key:
>
> https://keyserver.ubuntu.com/pks/lookup?op=get=0x134716af768d9b6e
> in
>this file https://www.apache.org/dist/storm/KEYS
>
>
> When voting, please list the actions taken to verify the release.
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 2.5.0
> [ ]   0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>
> --Bipin Prasad
>
> On Thu, Aug 3, 2023 at 2:41 PM Bipin Prasad 
> wrote:
>
> > Storm Release candidate version 2.5.0 rc4 is here:
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc4
> > Please vote on this release.
> >
> >1. How to vote is described here:
> >
> https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> >2. Release notes are here:
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc4/RELEASE_NOTES.html
> >.
> >3. The tag/commit to be voted upon is v2.5.0:
> >4. The source archive being votes upon is here:
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc4/apache-storm-2.5.0-src.tar.gz
> >5. The release artifacts are signed with the following key:
> >
> https://keyserver.ubuntu.com/pks/lookup?op=get=0x134716af768d9b6e
> > in this file https://www.apache.org/dist/storm/KEYS
> >
> >
> > When voting, please list the actions taken to verify the release.
> > This vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache Storm 2.5.0
> > [ ]   0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > --Bipin Prasad
> >
> > On Wed, Aug 2, 2023 at 4:07 PM Bipin Prasad 
> > wrote:
> >
> >> Storm Release candidate version 2.5.0 rc3 is here:
> >> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc3
> >> Please vote on this release.
> >>
> >>1. How to vote is described here:
> >>
> https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> >>2. Release notes are here:
> >>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc3/RELEASE_NOTES.html
> >>.
> >>3. The tag/commit to be voted upon is v2.5.0:
> >>4. The source archive being votes upon is here:
> >>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc3/apache-storm-2.5.0-src.tar.gz
> >>5. The release artifacts are signed with the following key:
> >>
> https://keyserver.ubuntu.com/pks/lookup?op=get=0x134716af768d9b6e
> >> in this file https://www.apache.org/dist/storm/KEYS
> >>
> >>
> >> When voting, please list the actions taken to verify the release.
> >> This vote will be open for at least 72 hours.
> >>
> >> [ ] +1 Release this package as Apache Storm 2.5.0
> >> [ ]   0 No opinion
> >> [ ] -1 Do not release this package because...
> >>
> >> Thanks to everyone who contributed to this release.
> >>
> >> --Bipin Prasad
> >>
> >> On Tue, Jul 25, 2023 at 8:28 AM Bipin Prasad 
> >> wrote:
> >>
> >>> Storm Release candidate version 2.5.0 rc2 is here:
> >>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2
> >>> Please vote on this release.
> >>>
> >>>1. How to vote is described here:
> >>>
> https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> >>>2. Release notes are here:
> >>>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/RELEASE_NOTES.html
> >>>.
> >>>3. The tag/commit to be voted upon is v2.5.0:
> >>>4. The source archive being votes upon is here:
> >>>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/apache-storm-2.5.0-src.tar.gz
> >>>5. The release artifacts are signed with the following key:
> >>>
> https://keyserver.ubuntu.com/pks/lookup?op=get=0x134716af768d9b6e
> >>> in this file https://www.apache.org/dist/storm/KEYS
> >>>
> >>>
> >>> When voting, please list 

Re: Storm 2.5.0 Release Candidate Vote

[Alexandre Vermeerbergen]

Hello,

So far, +1 [non binding] for 2.5.0 RC4 based on my tests, which are to
use binaries to upgrade my pre production distributed cluster running
10+ topologies at scale (> 1 million tuples per minute) running on
Rocky Linux x64 on AWS EC2 VMs  with IBM Semeru JDK 17.0.7.

Since I have been running with pre production setup with 2.5.0 since
RC1, then updated to RC2 (I skipped RC3 as it was so short lived) and
got no performance regressions nor feature regressions, I'm confident
that this 2.5.0 RC4 is stable enough vs. 2.4.0

I checked the SHA512 signatures => OK.

Disclaimer: I haven't yet done source compilation, will do next week
but maybe vote will be passed.

Kind regards,
Alexandre

Le ven. 4 août 2023 à 16:45, Bipin Prasad  a écrit :
>
> Note added in RELEASE_NOTES.html, updated sha512 for RELEASE_NOTES.html.
> Will go thru the rest of the release process.
>
> On 2023/08/04 14:19:15 Bipin Prasad wrote:
> > Thank you for checking on this RC4 and the vote. I will add note about 
> > python3 in the README.
> >
> > On 2023/08/04 08:58:12 Richard Zowalla wrote:
> > > Hi,
> > >
> > > 1. Checked SHA512 sums: OK
> > > 2. Checked ASC signatures: OK
> > > 3. Build from sources with the following setup
> > >
> > > Apache Maven 3.8.7 (b89d5959fcde851dcb1c8946a785a163f14e1e29)
> > > Maven home: /usr/share/maven
> > > Java version: 11.0.20, vendor: Ubuntu, runtime: /usr/lib/jvm/java-11-
> > > openjdk-amd64
> > > Default locale: de_DE, platform encoding: UTF-8
> > > OS name: "linux", version: "5.19.0-50-generic", arch: "amd64", family:
> > > "unix"
> > >
> > > similar to GH actions: SUCCESS
> > >
> > > 4. Run all tests (similar to GH actions) locally: SUCCESS
> > >
> > > 5. Run a dockerized Storm topology for a while
> > >
> > > (A) First I saw "/usr/bin/env: ‘python3’: No such file or directory" or
> > > "/apache-storm-2.5.0/bin/storm: line 42: 10 * ‘python3’: + ‘python3’::
> > > syntax error: operand expected (error token is "‘python3’: +
> > > ‘python3’:")" in the logs indicating that my docker containers did not
> > > have python3 available. I fixed it locally but might be worth to
> > > mention that in the announcement so people don't confused but isn't a
> > > blocker for 2.5.0-rc4 IMHO.
> > >
> > > (B) Storm UI is showing my cluster, topology viz looks good, no script
> > > errors in browser console
> > >
> > > (C) Started the topology using flux, don't see any issues (didn't test
> > > log viewer).
> > >
> > > Thanks for preparing RC4, Bipin!!
> > >
> > > So here is my +1 (binding) for RC4.
> > >
> > > Gruß
> > > Richard
> > >
> > >
> > >
> > > Am Donnerstag, dem 03.08.2023 um 14:41 -0700 schrieb Bipin Prasad:
> > > > Storm Release candidate version 2.5.0 rc4 is here:
> > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc4
> > > > Please vote on this release.
> > > >
> > > >1. How to vote is described here:
> > > >
> > > > https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> > > >2. Release notes are here:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc4/RELEASE_NOTES.html
> > > >.
> > > >3. The tag/commit to be voted upon is v2.5.0:
> > > >4. The source archive being votes upon is here:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc4/apache-storm-2.5.0-src.tar.gz
> > > >5. The release artifacts are signed with the following key:
> > > >
> > > > https://keyserver.ubuntu.com/pks/lookup?op=get=0x134716af768d9b6e
> > > >  in
> > > >this file https://www.apache.org/dist/storm/KEYS
> > > >
> > > >
> > > > When voting, please list the actions taken to verify the release.
> > > > This vote will be open for at least 72 hours.
> > > >
> > > > [ ] +1 Release this package as Apache Storm 2.5.0
> > > > [ ]   0 No opinion
> > > > [ ] -1 Do not release this package because...
> > > >
> > > > Thanks to everyone who contributed to this release.
> > > >
> > > > --Bipin Prasad
> > > >
> > > > On Wed, Aug 2, 2023 at 4:07 PM Bipin Prasad 
> > > > wrote:
> > > >
> > > > > Storm Release candidate version 2.5.0 rc3 is here:
> > > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc3
> > > > > Please vote on this release.
> > > > >
> > > > >1. How to vote is described here:
> > > > >
> > > > > https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> > > > >2. Release notes are here:
> > > > >
> > > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc3/RELEASE_NOTES.html
> > > > >.
> > > > >3. The tag/commit to be voted upon is v2.5.0:
> > > > >4. The source archive being votes upon is here:
> > > > >
> > > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc3/apache-storm-2.5.0-src.tar.gz
> > > > >5. The release artifacts are signed with the following key:
> > > > >
> > > > > https://keyserver.ubuntu.com/pks/lookup?op=get=0x134716af768d9b6e
> > 

Re: Storm 2.5.0 Release Candidate Vote

[Alexandre Vermeerbergen]

Hello,

Regarding my tests with Apache Storm 2.5.0 RC2 on a pre-production
cluster at scale, with 10+ topologies, running on RockyLinux 8 with
IBM Semeru 17.0.7 (based on OpenJ9 JVM) on Linux x64, I have detected
no functional regression, not any noticeable performance changes.

On side node, our topologies's Java source code is still compiled with
JDK11, and soon we will try compiling it with JDK17.
(with Java 21 LTS coming soon - end of September this year, it's time
to upgrade ;)

I still need to test compiling Storm from sources, my previous attempt
with RC1 was broken, and I have read some feedbacks that RC2 isn't
perfect on this side, so if an RC3 is planned, then I'd like to know
it so as to organize myself for Storm compiling tests.

Kind regards,
Alexandre

Le mar. 1 août 2023 à 10:49, Julien Nioche
 a écrit :
>
> Hi,
>
> Have experienced the same problems as Derek when trying to build. This is
> due to the dependency *pentaho-aggdesigner-algorithm *not being available.
> This dependency is excluded in some places of the poms but not
> systematically, I will open an issue and submit a PR shortly.
>
> Otherwise, I managed to run a StormCrawler topology with the 2.5 RC both in
> local and deployed mode.
>
> It would be good to fix the build before releasing IMHO.
>
> Julien
>
> On Mon, 31 Jul 2023 at 16:35, Bipin Prasad  wrote:
>
> > Hello Derek,
> > Looks like some shared libraries may be missing from the jdk11
> > install. Could be similar to this issue
> > https://github.com/pytorch/serve/issues/1352
> >
> > --Bipin
> >
> > On 2023/07/28 04:52:21 Derek Dagit wrote:
> > > I downloaded the release files:
> > >
> > > ∴ wget -r -np '
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/'
> > >
> > > I checked signatures:
> > >
> > > ∴ ls *.asc|xargs -n1 gpg --verify
> > > gpg: assuming signed data in 'apache-storm-2.5.0.pom'
> > > gpg: Signature made Tue 25 Jul 2023 09:33:30 AM CDT
> > > gpg:using RSA key
> > 51379DA8A7AE5B02674EF15C134716AF768D9B6E
> > > gpg: Good signature from "Bipin Prasad (For Signing Apache Storm
> > release) " [full]
> > > gpg: assuming signed data in 'apache-storm-2.5.0-src.tar.gz'
> > > gpg: Signature made Tue 25 Jul 2023 09:34:52 AM CDT
> > > gpg:using RSA key
> > 51379DA8A7AE5B02674EF15C134716AF768D9B6E
> > > gpg: Good signature from "Bipin Prasad (For Signing Apache Storm
> > release) " [full]
> > > gpg: assuming signed data in 'apache-storm-2.5.0-src.zip'
> > > gpg: Signature made Tue 25 Jul 2023 09:34:52 AM CDT
> > > gpg:using RSA key
> > 51379DA8A7AE5B02674EF15C134716AF768D9B6E
> > > gpg: Good signature from "Bipin Prasad (For Signing Apache Storm
> > release) " [full]
> > > gpg: assuming signed data in 'apache-storm-2.5.0.tar.gz'
> > > gpg: Signature made Tue 25 Jul 2023 09:33:39 AM CDT
> > > gpg:using RSA key
> > 51379DA8A7AE5B02674EF15C134716AF768D9B6E
> > > gpg: Good signature from "Bipin Prasad (For Signing Apache Storm
> > release) " [full]
> > > gpg: assuming signed data in 'apache-storm-2.5.0.zip'
> > > gpg: Signature made Tue 25 Jul 2023 09:33:39 AM CDT
> > > gpg:using RSA key
> > 51379DA8A7AE5B02674EF15C134716AF768D9B6E
> > > gpg: Good signature from "Bipin Prasad (For Signing Apache Storm
> > release) " [full]
> > > gpg: assuming signed data in 'RELEASE_NOTES.html'
> > > gpg: Signature made Tue 25 Jul 2023 09:47:54 AM CDT
> > > gpg:using RSA key
> > 51379DA8A7AE5B02674EF15C134716AF768D9B6E
> > > gpg: Good signature from "Bipin Prasad (For Signing Apache Storm
> > release) " [full]
> > >
> > >
> > >
> > > I checked checksums:
> > >
> > > ∴ ls *.sha512|while read f; do gpg --print-md SHA512 "${f%%.sha512}" |
> > diff -s - "$f"; done
> > >
> > > Each one of these failed because the format does not match what results
> > from
> > > following the instructions at
> > >
> > https://github.com/apache/storm/blob/master/RELEASING.md#setting-up-a-vote
> > .
> > >
> > > I retried this way:
> > >
> > > ∴ ls *.sha512|while read f; do sha512sum "${f%%.sha512}" | diff -s -
> > "$f"; done
> > > Files - and apache-storm-2.5.0.pom.asc.sha512 are identical
> > > Files - and apache-storm-2.5.0.pom.sha512 are identical
> > > Files - and apache-storm-2.5.0-src.tar.gz.asc.sha512 are identical
> > > Files - and apache-storm-2.5.0-src.tar.gz.sha512 are identical
> > > Files - and apache-storm-2.5.0-src.zip.asc.sha512 are identical
> > > Files - and apache-storm-2.5.0-src.zip.sha512 are identical
> > > Files - and apache-storm-2.5.0.tar.gz.asc.sha512 are identical
> > > Files - and apache-storm-2.5.0.tar.gz.sha512 are identical
> > > Files - and apache-storm-2.5.0.zip.asc.sha512 are identical
> > > Files - and apache-storm-2.5.0.zip.sha512 are identical
> > > Files - and RELEASE_NOTES.html.asc.sha512 are identical
> > > Files - and RELEASE_NOTES.html.sha512 are identical
> > >
> > > Looks OK.
> > >
> > >
> > >
> > > Unzipped the source, built and ran tests with `mvn 

Re: Storm 2.5.0 Release Candidate Vote

[Alexandre Vermeerbergen]

Okay for this related to SHA512 files, which remains a mystery.

Anyway I installed a cluster with Storm 2.5.0 RC2, and good news, I
see a first improvement vs. Storm 2.5.0 RC1 : storm command no more
complains because 'python' is not in PATH (I see now it seeks for
python3, which is installed on my RedHat 8 VMs).

However, here's another mystery : with Storm 2.5.0 RC1, the "storm"
command was not finding 'python' command, but it was just printing
some garbage because python version check was failing without exiting
storm shell. But then, no issue at running Storm 2.5.0 RC1 with 10+
topologies.

So my question is: why "storm" shell command check for python version
at all?  Storm various processes don't seem to use python, so isn't it
a "useless legacy check" that could be removed (not blocking 2.5.0
anyway, of course, but worth noting?)

In the meantime, I'm continuing 2.5.0 RC2 with test at higher scale...
stay tuned.

Thanks,
Alex

Le mar. 25 juil. 2023 à 20:21, Bipin Prasad  a écrit :
>
> Thanks for pointing this out. I have reverted to using individual sha512 
> files - generated via sha512sum command. These have been pushed to svn link.
>
> On 2023/07/25 18:03:06 Alexandre Vermeerbergen wrote:
> > Thanks... but that's not quite usable unless one download *all artefacts*, 
> > see:
> >
> > wget 
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/apache-storm-2.5.0.tar.gz
> > wget 
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/sha512sums.sha512
> > /usr/bin/sha512sum -c sha512sums.sha512
> >
> > /usr/bin/sha512sum: RELEASE_NOTES.html: No such file or directory
> > RELEASE_NOTES.html: FAILED open or read
> > /usr/bin/sha512sum: RELEASE_NOTES.html.asc: No such file or directory
> > RELEASE_NOTES.html.asc: FAILED open or read
> > /usr/bin/sha512sum: apache-storm-2.5.0-src.tar.gz: No such file or directory
> > apache-storm-2.5.0-src.tar.gz: FAILED open or read
> > /usr/bin/sha512sum: apache-storm-2.5.0-src.tar.gz.asc: No such file or 
> > directory
> > apache-storm-2.5.0-src.tar.gz.asc: FAILED open or read
> > /usr/bin/sha512sum: apache-storm-2.5.0-src.zip: No such file or directory
> > apache-storm-2.5.0-src.zip: FAILED open or read
> > /usr/bin/sha512sum: apache-storm-2.5.0-src.zip.asc: No such file or 
> > directory
> > apache-storm-2.5.0-src.zip.asc: FAILED open or read
> > /usr/bin/sha512sum: apache-storm-2.5.0.pom: No such file or directory
> > apache-storm-2.5.0.pom: FAILED open or read
> > /usr/bin/sha512sum: apache-storm-2.5.0.pom.asc: No such file or directory
> > apache-storm-2.5.0.pom.asc: FAILED open or read
> > apache-storm-2.5.0.tar.gz: OK
> > /usr/bin/sha512sum: apache-storm-2.5.0.tar.gz.asc: No such file or directory
> > apache-storm-2.5.0.tar.gz.asc: FAILED open or read
> > /usr/bin/sha512sum: apache-storm-2.5.0.zip: No such file or directory
> > apache-storm-2.5.0.zip: FAILED open or read
> > /usr/bin/sha512sum: apache-storm-2.5.0.zip.asc: No such file or directory
> > apache-storm-2.5.0.zip.asc: FAILED open or read
> > /usr/bin/sha512sum: WARNING: 11 listed files could not be read
> >
> > I tried to "corrupt" apache-storm-2.5.0.tar.gz and run the same
> > command, the corrupted file is detected, but that key information is a
> > bit burried into the many lines of 'sha512sum -c sha512sums.sha512'
> > output...
> >
> > Couldn't we have one .sha512 per file but with some sed or awk command
> > that post-processes gpg's output to turn the hexadecimal into a
> > one-liner compatible with sha512sum ?
> >
> > On a side note, the use of sha512sum is advertised on Apache
> > Fundation's site to verify sha512's signatures:
> > https://www.apache.org/info/verification.html
> >
> > Even more puzzling is the fact that Apache Kafka's recent signature
> > are also multi-line, see at: https://kafka.apache.org/downloads and I
> > test:
> >
> > wget https://downloads.apache.org/kafka/3.5.1/kafka-3.5.1-src.tgz
> > wget https://downloads.apache.org/kafka/3.5.1/kafka-3.5.1-src.tgz.sha512
> > sha512sum -c kafka-3.5.1-src.tgz.sha512
> > sha512sum: kafka-3.5.1-src.tgz.sha512: no properly formatted SHA512
> > checksum lines found
> >
> > => So your .sha512 files initially published aren't worse than Apache
> > Kafka's... but now I'm wondering why somebody seems to care??
> >
> > Anyway, in parallel I'm running tests with this 2.5.0 RC2, I'll will
> > provide feedbacks on that one too ASAP
> >
> > Thanks,
> > Alex
> >
> > Le mar. 25 juil. 2023 à 19:40, Bipin Prasad  a 
> > écrit :
> > >

Re: Storm 2.5.0 Release Candidate Vote

[Alexandre Vermeerbergen]

Thanks... but that's not quite usable unless one download *all artefacts*, see:

wget 
https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/apache-storm-2.5.0.tar.gz
wget 
https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/sha512sums.sha512
/usr/bin/sha512sum -c sha512sums.sha512

/usr/bin/sha512sum: RELEASE_NOTES.html: No such file or directory
RELEASE_NOTES.html: FAILED open or read
/usr/bin/sha512sum: RELEASE_NOTES.html.asc: No such file or directory
RELEASE_NOTES.html.asc: FAILED open or read
/usr/bin/sha512sum: apache-storm-2.5.0-src.tar.gz: No such file or directory
apache-storm-2.5.0-src.tar.gz: FAILED open or read
/usr/bin/sha512sum: apache-storm-2.5.0-src.tar.gz.asc: No such file or directory
apache-storm-2.5.0-src.tar.gz.asc: FAILED open or read
/usr/bin/sha512sum: apache-storm-2.5.0-src.zip: No such file or directory
apache-storm-2.5.0-src.zip: FAILED open or read
/usr/bin/sha512sum: apache-storm-2.5.0-src.zip.asc: No such file or directory
apache-storm-2.5.0-src.zip.asc: FAILED open or read
/usr/bin/sha512sum: apache-storm-2.5.0.pom: No such file or directory
apache-storm-2.5.0.pom: FAILED open or read
/usr/bin/sha512sum: apache-storm-2.5.0.pom.asc: No such file or directory
apache-storm-2.5.0.pom.asc: FAILED open or read
apache-storm-2.5.0.tar.gz: OK
/usr/bin/sha512sum: apache-storm-2.5.0.tar.gz.asc: No such file or directory
apache-storm-2.5.0.tar.gz.asc: FAILED open or read
/usr/bin/sha512sum: apache-storm-2.5.0.zip: No such file or directory
apache-storm-2.5.0.zip: FAILED open or read
/usr/bin/sha512sum: apache-storm-2.5.0.zip.asc: No such file or directory
apache-storm-2.5.0.zip.asc: FAILED open or read
/usr/bin/sha512sum: WARNING: 11 listed files could not be read

I tried to "corrupt" apache-storm-2.5.0.tar.gz and run the same
command, the corrupted file is detected, but that key information is a
bit burried into the many lines of 'sha512sum -c sha512sums.sha512'
output...

Couldn't we have one .sha512 per file but with some sed or awk command
that post-processes gpg's output to turn the hexadecimal into a
one-liner compatible with sha512sum ?

On a side note, the use of sha512sum is advertised on Apache
Fundation's site to verify sha512's signatures:
https://www.apache.org/info/verification.html

Even more puzzling is the fact that Apache Kafka's recent signature
are also multi-line, see at: https://kafka.apache.org/downloads and I
test:

wget https://downloads.apache.org/kafka/3.5.1/kafka-3.5.1-src.tgz
wget https://downloads.apache.org/kafka/3.5.1/kafka-3.5.1-src.tgz.sha512
sha512sum -c kafka-3.5.1-src.tgz.sha512
sha512sum: kafka-3.5.1-src.tgz.sha512: no properly formatted SHA512
checksum lines found

=> So your .sha512 files initially published aren't worse than Apache
Kafka's... but now I'm wondering why somebody seems to care??

Anyway, in parallel I'm running tests with this 2.5.0 RC2, I'll will
provide feedbacks on that one too ASAP

Thanks,
Alex

Le mar. 25 juil. 2023 à 19:40, Bipin Prasad  a écrit :
>
> I have removed the .sha512 files generated via gpg and created one single 
> sha512 file with all the checksums. Please check again. Thanks
>
> On 2023/07/25 17:27:14 Bipin Prasad wrote:
> > Looks like gpg does not generate checksum in the format expected by 
> > sha512sum.
> > sha512sum expects one line of the format:
> > space 
> >
> > gpg generates:
> >: 
> >
> > So when I take the checksum file:
> > apache-storm-2.5.0-src.zip: 8ABE6595 4C51B310 BA226021 29253788 0201BF9D
> > 92DC24D7 A00FEBAE 809D0F91 EC772FAC C433F16B
> > B56F958E 2D2EDF21 0814A02B 58819949 E26B574B
> > A75BF1B9
> > and change it to:
> > 8ABE65954C51B310BA226021292537880201BF9D92DC24D7A00FEBAE809D0F91EC772FACC433F16BB56F958E2D2EDF210814A02B58819949E26B574BA75BF1B9
> >  apache-storm-2.5.0-src.zip
> >
> > and run sha512sum it works.
> >
> > On 2023/07/25 17:13:15 Bipin Prasad wrote:
> > > The sha512 files were generated by "gpg --print-md SHA512" command. Will 
> > > check why sha512sum does not think it is valid file.
> > >
> > > On 2023/07/25 16:50:50 Alexandre Vermeerbergen wrote:
> > > > Hello,
> > > >
> > > > Great !
> > > > But I'm facing again an issue with SHA512 validation of the source 
> > > > archive:
> > > >
> > > > wget 
> > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/apache-storm-2.5.0-src.tar.gz
> > > > wget 
> > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/apache-storm-2.5.0-src.tar.gz.sha512
> > > > sha512sum -c apache-storm-2.5.0-src.tar.gz.sha512
> > > > sha512sum: apache-st

Re: Storm 2.5.0 Release Candidate Vote

[Alexandre Vermeerbergen]

Hello,

Great !
But I'm facing again an issue with SHA512 validation of the source archive:

wget 
https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/apache-storm-2.5.0-src.tar.gz
wget 
https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/apache-storm-2.5.0-src.tar.gz.sha512
sha512sum -c apache-storm-2.5.0-src.tar.gz.sha512
sha512sum: apache-storm-2.5.0-src.tar.gz.sha512: no properly formatted
SHA512 checksum lines found

same issue with this binary artifact:

wget 
https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/apache-storm-2.5.0.tar.gz
wget 
https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/apache-storm-2.5.0.tar.gz.sha512
 sha512sum -c apache-storm-2.5.0.tar.gz.sha512
sha512sum: apache-storm-2.5.0.tar.gz.sha512: no properly formatted
SHA512 checksum lines found

So I guess that's not good for a final 2.5.0 release isn't it ?

Thanks,
Alex

Le mar. 25 juil. 2023 à 17:29, Bipin Prasad  a écrit :
>
> Storm Release candidate version 2.5.0 rc2 is here:
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2
> Please vote on this release.
>
>1. How to vote is described here:
>
> https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
>2. Release notes are here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/RELEASE_NOTES.html
>.
>3. The tag/commit to be voted upon is v2.5.0:
>4. The source archive being votes upon is here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc2/apache-storm-2.5.0-src.tar.gz
>5. The release artifacts are signed with the following key:
>https://keyserver.ubuntu.com/pks/lookup?op=get=0x134716af768d9b6e in
>this file https://www.apache.org/dist/storm/KEYS
>
>
> When voting, please list the actions taken to verify the release.
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 2.5.0
> [ ]   0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>
> --Bipin Prasad
>
> On Mon, Jun 26, 2023 at 4:15 PM Bipin Prasad  wrote:
>
> > Storm Release candidate version 2.5.0 rc1 is here:
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1
> > Please vote on this release.
> >
> >1. How to vote is described here:
> >
> > https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> >2. Release notes are here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/RELEASE_NOTES.html
> >.
> >3. The tag/commit to be voted upon is v2.5.0:
> >4. The source archive being votes upon is here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/apache-storm-2.5.0-src.tar.gz
> >5. The release artifacts are signed with the following key:
> >https://keyserver.ubuntu.com/pks/lookup?op=get=0x134716af768d9b6e
> > in this file https://www.apache.org/dist/storm/KEYS
> >
> >
> > When voting, please list the actions taken to verify the release.
> > This vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache Storm 2.5.0
> > [ ]   0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > --Bipin Prasad
> >

Re: Storm 2.5.0 Release Candidate Vote

[Alexandre Vermeerbergen]

Great for jdk11/jdk17 !
Attached the mvn clean install's GZIP logs which I forgot to attach in
previous post, in case that's of any help as a feedback on RC1

Le sam. 8 juil. 2023 à 18:54, Bipin Prasad
 a écrit :
>
> About jdk11,jdk17 this is coming shortly. Because of some downstream 
> dependencies, there were some issues with tests. After this release, we need 
> to add jdk11 and 17 to GitHub actions (and then at some point drop jdk8).
>
>
> Sent from Yahoo Mail for iPhone
>
>
> On Saturday, July 8, 2023, 11:44 AM, Alexandre Vermeerbergen 
>  wrote:
>
> More tests on Storm 2.5.0-RC1:
> * Using the binary distribution, I upgrade my Storm cluster with 6
> workers/supervisor dedicated machines (without changing the big JARs
> which are still compiled with Storm 2.4.0 dependencies)
>   => everything work fine so far
> * I tried to compile from sources on a RedHat 8.6 machine with JDK8
> (OpenJ9's) and Maven 3.9.2, got errors but that looks like what you
> guys are trying to fix to Python.
>   => I have attached GZIPed version of mvn clean install
>
> Question: can we use JDK11 or JDK17 to compile Storm ?
>
> Thanks,
> Alex
>
> Le sam. 8 juil. 2023 à 13:55, Bipin Prasad
>  a écrit :
> >
> > I am working on release candidate 2, but will also look at this issue this 
> > weekend and get back to you.Thanks for checking.
> >
> >
> > Sent from Yahoo Mail for iPhone
> >
> >
> > On Saturday, July 8, 2023, 1:15 AM, Alexandre Vermeerbergen 
> >  wrote:
> >
> > Hello,
> >
> > I tried to check apache-storm-2.5.0.tar.gz.sha512(from
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/),
> > but I get this error:
> >
> > $ /usr/bin/sha512sum -c apache-storm-2.5.0.tar.gz.sha512
> > /usr/bin/sha512sum: apache-storm-2.5.0.tar.gz.sha512: no properly
> > formatted SHA512 checksum lines found
> > $ cat apache-storm-2.5.0.tar.gz.sha512
> > apache-storm-2.5.0.tar.gz: 93066657 9585BF11 9F71444D 355A2FBB B1B2FAF5
> >  EA2B8FDA B783C375 391D6926 C27D9BA3 2269
> >  32D22B77 AEB00E79 EBDC5563 910740DC 35E77DD9
> >  625C0DDD
> >
> > Am I doing it wrong? (to create the contents of
> > apache-storm-2.5.0.tar.gz.sha512, I copy-pasted the output of
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/apache-storm-2.5.0.tar.gz.sha512)
> >
> > thanks,
> > Alex
> >
> > Le jeu. 29 juin 2023 à 21:18, Aaron Gresch  a écrit :
> > >
> > > I found some issues with python vs python3 with the release candidate that
> > > Bipin is addressing.
> > >
> > > On Mon, Jun 26, 2023 at 6:15 PM Bipin Prasad  
> > > wrote:
> > >
> > > > Storm Release candidate version 2.5.0 rc1 is here:
> > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1
> > > > Please vote on this release.
> > > >
> > > >1. How to vote is described here:
> > > >
> > > > https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> > > >2. Release notes are here:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/RELEASE_NOTES.html
> > > >.
> > > >3. The tag/commit to be voted upon is v2.5.0:
> > > >4. The source archive being votes upon is here:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/apache-storm-2.5.0-src.tar.gz
> > > >5. The release artifacts are signed with the following key:
> > > >
> > > > https://keyserver.ubuntu.com/pks/lookup?op=get=0x134716af768d9b6e
> > > > in
> > > >this file https://www.apache.org/dist/storm/KEYS
> > > >
> > > >
> > > > When voting, please list the actions taken to verify the release.
> > > > This vote will be open for at least 72 hours.
> > > >
> > > >[ ] +1 Release this package as Apache Storm 2.5.0
> > > >[ ]  0 No opinion
> > > >[ ] -1 Do not release this package because...
> > > >
> > > > Thanks to everyone who contributed to this release.
> > > >
> > > > --Bipin Prasad
> > > >
> >
> >
> >
>
>
>


log.mvn.gz
Description: GNU Zip compressed data

Re: Storm 2.5.0 Release Candidate Vote

[Alexandre Vermeerbergen]

More tests on Storm 2.5.0-RC1:
* Using the binary distribution, I upgrade my Storm cluster with 6
workers/supervisor dedicated machines (without changing the big JARs
which are still compiled with Storm 2.4.0 dependencies)
   => everything work fine so far
* I tried to compile from sources on a RedHat 8.6 machine with JDK8
(OpenJ9's) and Maven 3.9.2, got errors but that looks like what you
guys are trying to fix to Python.
  => I have attached GZIPed version of mvn clean install

Question: can we use JDK11 or JDK17 to compile Storm ?

Thanks,
Alex

Le sam. 8 juil. 2023 à 13:55, Bipin Prasad
 a écrit :
>
> I am working on release candidate 2, but will also look at this issue this 
> weekend and get back to you.Thanks for checking.
>
>
> Sent from Yahoo Mail for iPhone
>
>
> On Saturday, July 8, 2023, 1:15 AM, Alexandre Vermeerbergen 
>  wrote:
>
> Hello,
>
> I tried to check apache-storm-2.5.0.tar.gz.sha512(from
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/),
> but I get this error:
>
> $ /usr/bin/sha512sum -c apache-storm-2.5.0.tar.gz.sha512
> /usr/bin/sha512sum: apache-storm-2.5.0.tar.gz.sha512: no properly
> formatted SHA512 checksum lines found
> $ cat apache-storm-2.5.0.tar.gz.sha512
> apache-storm-2.5.0.tar.gz: 93066657 9585BF11 9F71444D 355A2FBB B1B2FAF5
>   EA2B8FDA B783C375 391D6926 C27D9BA3 2269
>   32D22B77 AEB00E79 EBDC5563 910740DC 35E77DD9
>   625C0DDD
>
> Am I doing it wrong? (to create the contents of
> apache-storm-2.5.0.tar.gz.sha512, I copy-pasted the output of
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/apache-storm-2.5.0.tar.gz.sha512)
>
> thanks,
> Alex
>
> Le jeu. 29 juin 2023 à 21:18, Aaron Gresch  a écrit :
> >
> > I found some issues with python vs python3 with the release candidate that
> > Bipin is addressing.
> >
> > On Mon, Jun 26, 2023 at 6:15 PM Bipin Prasad  wrote:
> >
> > > Storm Release candidate version 2.5.0 rc1 is here:
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1
> > > Please vote on this release.
> > >
> > >1. How to vote is described here:
> > >
> > > https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> > >2. Release notes are here:
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/RELEASE_NOTES.html
> > >.
> > >3. The tag/commit to be voted upon is v2.5.0:
> > >4. The source archive being votes upon is here:
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/apache-storm-2.5.0-src.tar.gz
> > >5. The release artifacts are signed with the following key:
> > >
> > > https://keyserver.ubuntu.com/pks/lookup?op=get=0x134716af768d9b6e
> > > in
> > >this file https://www.apache.org/dist/storm/KEYS
> > >
> > >
> > > When voting, please list the actions taken to verify the release.
> > > This vote will be open for at least 72 hours.
> > >
> > >[ ] +1 Release this package as Apache Storm 2.5.0
> > >[ ]  0 No opinion
> > >[ ] -1 Do not release this package because...
> > >
> > > Thanks to everyone who contributed to this release.
> > >
> > > --Bipin Prasad
> > >
>
>
>

Re: Storm 2.5.0 Release Candidate Vote

[Alexandre Vermeerbergen]

Hello,

I tried to check apache-storm-2.5.0.tar.gz.sha512(from
https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/),
but I get this error:

$ /usr/bin/sha512sum -c apache-storm-2.5.0.tar.gz.sha512
/usr/bin/sha512sum: apache-storm-2.5.0.tar.gz.sha512: no properly
formatted SHA512 checksum lines found
$ cat apache-storm-2.5.0.tar.gz.sha512
apache-storm-2.5.0.tar.gz: 93066657 9585BF11 9F71444D 355A2FBB B1B2FAF5
   EA2B8FDA B783C375 391D6926 C27D9BA3 2269
   32D22B77 AEB00E79 EBDC5563 910740DC 35E77DD9
   625C0DDD

Am I doing it wrong? (to create the contents of
apache-storm-2.5.0.tar.gz.sha512, I copy-pasted the output of
https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/apache-storm-2.5.0.tar.gz.sha512)

thanks,
Alex

Le jeu. 29 juin 2023 à 21:18, Aaron Gresch  a écrit :
>
> I found some issues with python vs python3 with the release candidate that
> Bipin is addressing.
>
> On Mon, Jun 26, 2023 at 6:15 PM Bipin Prasad  wrote:
>
> > Storm Release candidate version 2.5.0 rc1 is here:
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1
> > Please vote on this release.
> >
> >1. How to vote is described here:
> >
> > https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
> >2. Release notes are here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/RELEASE_NOTES.html
> >.
> >3. The tag/commit to be voted upon is v2.5.0:
> >4. The source archive being votes upon is here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/apache-storm-2.5.0-src.tar.gz
> >5. The release artifacts are signed with the following key:
> >
> > https://keyserver.ubuntu.com/pks/lookup?op=get=0x134716af768d9b6e
> > in
> >this file https://www.apache.org/dist/storm/KEYS
> >
> >
> > When voting, please list the actions taken to verify the release.
> > This vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache Storm 2.5.0
> > [ ]   0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > --Bipin Prasad
> >

Re: Storm 2.5.0 Release Candidate Vote

[Alexandre Vermeerbergen]

Great news!

I will start testing this Storm 2.5.0 RC  later this week, as i do not have
access to my machines before.

Thanks!
Alex

Le mar. 27 juin 2023, 01:15, Bipin Prasad  a écrit :

> Storm Release candidate version 2.5.0 rc1 is here:
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1
> Please vote on this release.
>
>1. How to vote is described here:
>
> https://github.com/apache/storm/blob/master/RELEASING.md#how-to-vote-on-a-release-candidate
>2. Release notes are here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/RELEASE_NOTES.html
>.
>3. The tag/commit to be voted upon is v2.5.0:
>4. The source archive being votes upon is here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.5.0-rc1/apache-storm-2.5.0-src.tar.gz
>5. The release artifacts are signed with the following key:
>
> https://keyserver.ubuntu.com/pks/lookup?op=get=0x134716af768d9b6e
> in
>this file https://www.apache.org/dist/storm/KEYS
>
>
> When voting, please list the actions taken to verify the release.
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 2.5.0
> [ ]   0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>
> --Bipin Prasad
>

Re: [DISCUSS] Call for PMC Members and Contributors

[Alexandre Vermeerbergen]

Agree with Bipin Prasad, since it seems that we're close to a Storm
2.5.0 candidate release...

Le mar. 20 juin 2023 à 17:38, Bipin Prasad
 a écrit :
>
> I would urge PMC Chair to give us some guidance on this.
>
>
> Sent from Yahoo Mail for iPhone
>
>
> On Tuesday, June 20, 2023, 8:32 AM, 6harat  
> wrote:
>
> Hey,
>
> Just wanted to follow-up on the thread initiated by Richard, if there were
> any updates from the PMC circle that can be shared with the wider audience.
> Also, curious about if we have any previous roadmap available for the
> further evolution of the project. (I checked on JIRA but wasn't able to
> find one)
>
> I would be interested to help contribute to the project in any way possible
> out of my personal capacity. (Have used Storm in the enterprise environment
> for many years)
> I would suggest if the PMC circle can syncup over google meet / zoom along
> with interested new folks and help decide the way forward. Please let us
> know your thoughts on the same.
>
> Regards
> 6harat
>
> On Mon, Jun 19, 2023 at 12:37 PM Richard Zowalla  wrote:
>
> > Don't want to come across as an annoying user, but have we made any
> > progress in this area yet? Everyone who has expressed interest
> > (usually) doesn't have read access to private@, so no one knows what
> > the current status is regarding the continuation of the project by
> > expanding the committer / pmc circle. CC'ing private@, so we can
> > hopefully get an update.
> >
> > I am asking because I read about the intention to cut a storm 2.5.0
> > release and I am wondering, if we get the necessary votes in a timely
> > manner.
> >
> > Gruß
> > Richard
> >
> > Am Montag, dem 12.06.2023 um 12:06 +0200 schrieb Alexandre
> > Vermeerbergen:
> > > +1
> > >
> > > Le lun. 12 juin 2023 à 11:57, Richard Zowalla  a
> > > écrit :
> > > >
> > > > Some weeks passed, any updates?
> > > >
> > > > Gruß
> > > > Richard
> > > >
> > > > Am Donnerstag, dem 11.05.2023 um 19:16 -0400 schrieb P. Taylor
> > > > Goetz:
> > > > > Thanks for the reminder Richard,
> > > > >
> > > > > In response, I’ve bumped the PMC-private discussion around adding
> > > > > new
> > > > > members interested in contributing.
> > > > >
> > > > > - Taylor
> > > > >
> > > > > > On May 11, 2023, at 8:39 AM, Richard Zowalla 
> > > > > > wrote:
> > > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I know we are all busy with our day jobs, but is there any
> > > > > > news?
> > > > > >
> > > > > > We had some activity after a move to the attic was proposed. In
> > > > > > the
> > > > > > process, a few people volunteered to help with the project.
> > > > > >
> > > > > > A few weeks later, IMHO we are back to where we were before the
> > > > > > proposal. Mails are going nowhere and I've heard of a few
> > > > > > people
> > > > > > planning to just fork Storm to get an updated version into
> > > > > > production.
> > > > > >
> > > > > > If we want to prevent that and Storm should really be put into
> > > > > > maintenance mode, then we should urgently take care of
> > > > > > expanding
> > > > > > the PMC/Committers circle, so that we are able to put out
> > > > > > timely
> > > > > > releases again and the work regarding maintenance is
> > > > > > distributed on
> > > > > > more than just 2 shoulders.
> > > > > >
> > > > > > Don't take it as a rant, it's just my opinion as someone who
> > > > > > runs a
> > > > > > Storm Cluster in production and would love to help the project
> > > > > > to
> > > > > > sustain.
> > > > > >
> > > > > > Gruß
> > > > > > Richard
> > > > > >
> > > > > > On 2023/05/02 12:58:00 Richard Zowalla wrote:
> > > > > > > Any updates? :-)
> > > > > > >
> > > > > > > Think Storm would benefit from a new release soon...
> > > > > > >
> > > > > > > Am Mittwoch, dem 22.02.2023 um 18:

Re: Storm Cassandra

[Alexandre Vermeerbergen]

+1 to exclude Storm Cassandra
(my 2 cents)

Thanks,
Alex

Le ven. 16 juin 2023 à 17:58, Bipin Prasad  a écrit :
>
> All,
>I am working on creating a new release for Storm 2.5.0. Unfortunately,
> storm-cassandra has been failing tests since we moved to Git Actions. Can
> someone take a look at it and possibly get it to work - with a newer
> version of Cassandra if possible and/or upgraded tests?
>
>If no one is using Storm Cassandra (under xternal/storm-cassandra), I
> can exclude this from the build and release). Open to other suggestions.
>
> Thanks
> --Bipin

Re: [DISCUSS] Call for PMC Members and Contributors

[Alexandre Vermeerbergen]

+1

Le lun. 12 juin 2023 à 11:57, Richard Zowalla  a écrit :
>
> Some weeks passed, any updates?
>
> Gruß
> Richard
>
> Am Donnerstag, dem 11.05.2023 um 19:16 -0400 schrieb P. Taylor Goetz:
> > Thanks for the reminder Richard,
> >
> > In response, I’ve bumped the PMC-private discussion around adding new
> > members interested in contributing.
> >
> > - Taylor
> >
> > > On May 11, 2023, at 8:39 AM, Richard Zowalla 
> > > wrote:
> > >
> > > Hello,
> > >
> > > I know we are all busy with our day jobs, but is there any news?
> > >
> > > We had some activity after a move to the attic was proposed. In the
> > > process, a few people volunteered to help with the project.
> > >
> > > A few weeks later, IMHO we are back to where we were before the
> > > proposal. Mails are going nowhere and I've heard of a few people
> > > planning to just fork Storm to get an updated version into
> > > production.
> > >
> > > If we want to prevent that and Storm should really be put into
> > > maintenance mode, then we should urgently take care of expanding
> > > the PMC/Committers circle, so that we are able to put out timely
> > > releases again and the work regarding maintenance is distributed on
> > > more than just 2 shoulders.
> > >
> > > Don't take it as a rant, it's just my opinion as someone who runs a
> > > Storm Cluster in production and would love to help the project to
> > > sustain.
> > >
> > > Gruß
> > > Richard
> > >
> > > On 2023/05/02 12:58:00 Richard Zowalla wrote:
> > > > Any updates? :-)
> > > >
> > > > Think Storm would benefit from a new release soon...
> > > >
> > > > Am Mittwoch, dem 22.02.2023 um 18:02 -0500 schrieb P. Taylor
> > > > Goetz:
> > > > > Quick update:
> > > > >
> > > > > The ASF Board voted to accept both our report and the
> > > > > resolution to
> > > > > change PMC Chair. Board feedback on our report was positive.
> > > > >
> > > > > The next step is to expand the the PMC/Committers group.
> > > > > Official
> > > > > votes will necessarily be private, but I see no reason
> > > > > nominations
> > > > > can’t be public.
> > > > >
> > > > > So lets open a thread to talk about adding new contributors to
> > > > > the
> > > > > PMC. Feel free to:
> > > > >
> > > > > 1. Volunteer to support the project as a PMC Member/Committer..
> > > > > 2. Nominate someone become PMC/Committer
> > > > >
> > > > > I have a few candidates in mind, and will follow up
> > > > > accordingly.
> > > > > Votes for new members will be private, but successful votes for
> > > > > new
> > > > > members will be announced on the public lists.
> > > > >
> > > > > One important way to contribute is to volunteer as a release
> > > > > manager
> > > > > for any given release, even if we release infrequently (e.g.
> > > > > only in
> > > > > response to a serious bug or security issue. I don’t have the
> > > > > bandwidth to act in that role, though I will commit to support
> > > > > voting
> > > > > on releases, etc. Ideally we would want at least two folks
> > > > > ready to
> > > > > step into that role, but we can get by with one if necessary.
> > > > >
> > > > > Thanks again to everyone who’s expressed interest in
> > > > > contributing to
> > > > > the project.
> > > > >
> > > > > - Taylor
> > > >
> > > >
> >
>

Any comment on my proposal to update snakeyaml to 2.0 ?

[Alexandre Vermeerbergen]

Hello,

I just created https://issues.apache.org/jira/browse/STORM-3918
Any objection for this dependency update ?

I guess I can take care of this - I just need a reminder about how to
properly create a git branch to deliver an update to .pom and make
this branch linked to the JIRA

Thanks,
Alex

Re: [DISCUSSION] Apache Storm and moving to the Attic

[Alexandre Vermeerbergen]

+1 for whatever makes Apache Storm to be maintained as long as
possible, including my time if needed !

Alexandre

Le ven. 3 févr. 2023 à 14:54, Richard Zowalla  a écrit :
>
> Hi all,
>
> I am also +1 for Taylor as PMC and please count me in as well (my first
> comment was a bit vague about it).
>
> Looks like, that the discussion is going well. We know see a lot of
> involvement ;-) (didn't see that for years on this list!).
>
> Gruß
> Richard
>
>
> Am Freitag, dem 03.02.2023 um 13:20 + schrieb Julien Nioche:
> > Hi,
> >
> > I'd be happy to get involved. I haven't delved deep into Storm's code
> > but have been using it for years and my project StormCrawler relies
> > heavily on it. I have been a committer on several Apache projects but
> > not been active in any of them for a while.
> > I know that there are loads of StormCrawler users and of course that
> > also makes them Storm users. Even though a number of people might
> > have moved to other platforms, I don't believe Storm is dead and
> > maintenance mode is probably an accurate description. So if you are
> > willing to lower the bar, please count me in and am definitely +1 for
> > Taylor as a PMC chair.
> >
> > Cheers
> >
> > Julien
> >
> > On Thu, 2 Feb 2023 at 05:07, P. Taylor Goetz 
> > wrote:
> > > IMO, there are tow scenarios:
> > >
> > > 1. Storm is in dead. No one uses it anymore, and no one needs
> > > security patches, etc.
> > > 2. Storm is in maintenance mode. While new features may not be
> > > added, there are enough contributors left tp at least address any
> > > security concerns.
> > >
> > > Wether we pursue the attic o projectr continuation is really up to
> > > the community. Anyone who want to step up and help continue the
> > > project should be considered a PMC candidate.
> > >
> > > As I’ve said in other threads, I’m willing to act as PMC chair
> > > whatever the PMC decides.
> > >
> > > But for now we have ne official liaison to ther ASF board. PMC
> > > members need to elect a chair, whether we choose to continue or
> > > head to the attic.
> > >
> > > I would lean toward lowering the bar for PMC membership to anyone
> > > interested in ongoing maintenance.
> > >
> > > I’m still willing to serve as the board liaison (PMC Chair) in
> > > either case of retirement or continuation. I’d also support anyone
> > > else willing to take the position.  The position needs to be filled
> > > either way.
> > >
> > > This is a decision that affects both developers and users. Thank
> > > you Aaron for bringing it to the attention of the  broader
> > > community.
> > >
> > > -Taylor
> > >
> > > > On Feb 1, 2023, at 9:26 PM, sunil yadav 
> > > > wrote:
> > > >
> > > > One side note looking to see what folks are migrating or moved to
> > > > if not using Storm actively. Any cloud native DAG platform to
> > > > recommend?
> > > >
> > > > -Sunil
> > > >
> > > > On Wed, Feb 1, 2023 at 6:12 PM Stephen Powis via user <
> > > > u...@storm.apache.org> wrote:
> > > > > Ah Yea, also not opposed to the move to the attic if it's
> > > > > determined to be most appropriate, but I had a similar
> > > > > experience as Richard where I submitted some PRs, asked for
> > > > > comments/thoughts/review, and even offered to take over
> > > > > ownership of one of the sub modules and got no responses.
> > > > > Perhaps I just submitted under the wrong forum to get an
> > > > > answer, but definitely a bummer, and would love to see the
> > > > > project/community get revived.
> > > > >
> > > > > - Stephen
> > > > >
> > > > > On Wed, Feb 1, 2023 at 10:09 PM Richard Zowalla <
> > > > > r...@apache.org> wrote:
> > > > > > Hi Aaron,
> > > > > >
> > > > > > I am CC the users@ list as they weren't contained in the
> > > > > > initial
> > > > > > proposal. Perhaps, there are people or institutions in the
> > > > > > wild, who
> > > > > > want to volunteer or give it a try (still at the ASF level).
> > > > > >
> > > > > > Of course it shouldn't prevent a VOTE for moving to the
> > > > > > attic, but an
> > > > > > additional try to get some more attention (if needed).
> > > > > >
> > > > > > Gruß
> > > > > > Richard
> > > > > >
> > > > > >
> > > > > > Am Dienstag, dem 31.01.2023 um 08:00 -0600 schrieb Aaron
> > > > > > Niskode-
> > > > > > Dossett:
> > > > > > > Thank you to those who responded. Given that the low number
> > > > > > of
> > > > > > > responses I
> > > > > > > am going to start a vote on moving to the attic tomorrow.
> > > > > > >
> > > > > > > One other note: A vote to move to the attic would not mean
> > > > > > the end of
> > > > > > > Storm, the project can be forked by anyone who wants to
> > > > > > continue the
> > > > > > > work
> > > > > > > (just as it can be forked today), it would just be outside
> > > > > > of the
> > > > > > > ASF.
> > > > > > >
> > > > > > > On Thu, Jan 26, 2023 at 3:12 AM Richard Zowalla <
> > > > > > r...@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > I also 

Re: [DISCUSSION] Apache Storm and moving to the Attic

[Alexandre Vermeerbergen]

Hello,

I'd be happy to contribute to Apache Storm in any kind of way.
Just let me know what kind of help I could provide.
Disclaimer: I not skilled in clojure language, so Storm 1.x
maintenance is probably not something I could do efficiently, for I
can help on Java side and on "management" stuff if that helps the
community.

Alexandre

Le mer. 25 janv. 2023 à 22:50, Aaron Niskode-Dossett
 a écrit :
>
> Hello Storm developer community,
>
> In the past year or so this project has slowed down and the Project
> Management Committee [PMC] has almost no active members.  The PMC chair
> resigned in 2022 with due notice and noone has since volunteered to assume
> those duties.  I myself am an inactive PMC member.
>
> This suggests to me that it's time to consider moving this project to the
> Attic [1].  I would view this as the natural culmination of a very
> successful Apache project and not as a mark of failure.  There are many,
> many successful and influential projects in the attic.
>
> The alternative to moving to the attic would be to reconstitute the PMC
> with *new members* of the development community willing to take on the
> responsibilities of the project and reporting to the Apache Software
> Foundation board.
>
> So, please respond to this thread if you are interested in that role.
> Please also respond if you believe moving to the attic is appropriate.  A
> vote on moving to the attic may be held on this list in the future.
>
> Thanks you for your time,
> Aaron
>
> [1] https://attic.apache.org/

Re: [VOTE] Release Apache Storm 1.2.4 (rc1)

[Alexandre Vermeerbergen]

Hi,

I'm surprised and glad to see a new release on Storm 1.2.x, as I have
still not succeeded in migrating to Storm 2.x.
 Not exactly related to this VOTE, but is there some hope to see a
Storm 1.x release which could run with Java 17 LTS ?

Indeed, we currently run our topologies with Storm 1.2.3 with Java 11
LTS with no issue, my first attempt with Java 17 miserably fail
because of some forbidden access to Java internals...

Should I fill an JIRA for this is there's some hope for Storm 1.x fixes?

Kind regards,
Alex


Le ven. 1 oct. 2021 à 21:19, Ethan Li  a écrit :
>
> This is a call to vote on releasing Apache Storm 1.2.4 (rc1)
>
> Full list of changes in this release:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.4-rc1/RELEASE_NOTES.html
>
> The tag/commit to be voted upon is v1.2.4:
>
> https://gitbox.apache.org/repos/asf?p=storm.git;a=commit;h=1bc944091b727e9a892bb36349231fcc57d9ea30
>
> The source archive being voted upon can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.4-rc1/apache-storm-1.2.4-src.tar.gz
>
> Other release files, signatures and digests can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.4-rc1/
>
> The release artifacts are signed with the following key:
>
> https://www.apache.org/dist/storm/KEYS
>
> The Nexus staging repository for this release is:
>
> https://repository.apache.org/content/repositories/orgapachestorm-1099
>
> Please vote on releasing this package as Apache Storm 1.2.4.
>
> When voting, please list the actions taken to verify the release.
>
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 1.2.4
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.

Re: [ANNOUNCE] Apache Storm 2.1.0 Released

[Alexandre Vermeerbergen]

+1 congratulations with this new Apache Storm release!

Le sam. 2 nov. 2019 à 10:45, Stig Rohde Døssing
 a écrit :
>
> Great job with the release.
>
> Den tor. 31. okt. 2019 kl. 22.48 skrev Ethan Li :
>
> > The Apache Storm community is pleased to announce the release of Apache
> > Storm version 2.1.0.
> >
> > Apache Storm is a distributed, fault-tolerant, and high-performance
> > realtime computation system that provides strong guarantees on the
> > processing of data. You can read more about Storm on the project website:
> >
> > http://storm.apache.org
> >
> > Downloads of source and binary distributions are listed in our download
> > section:
> >
> > http://storm.apache.org/downloads.html
> >
> > You can read more about this release in the following blog post:
> >
> > https://storm.apache.org/2019/10/31/storm210-released.html
> >
> > Distribution artifacts are available in Maven Central at the following
> > coordinates:
> >
> > groupId: org.apache.storm
> > artifactId: storm-{component}
> > version: 2.1.0
> >
> > The full list of changes is available here[1]. Please let us know [2] if
> > you encounter any problems.
> >
> > Regards,
> >
> > The Apache Storm Team
> >
> > [1]:
> > http://www.us.apache.org/dist/storm/apache-storm-2.1.0/RELEASE_NOTES.html
> > [2]: https://issues.apache.org/jira/browse/STORM
> >

Re: Any hope for a fix of STORM-3032 ?

[Alexandre Vermeerbergen]

Hello Stig,

I did not realize that these API exist in Storm 1.2.x, if so then I
can start working on this adaptation to prepare 1.x/2.x transition of
our topologies.
On a side note, I realized that I could only reproduce STORM-3032 only
if running Supervisor processes using AdoptOpenJDK 8 update 222 with
OpenJ9 ; whereas I don't get these exceptions when using AdopOpenJDK 8
update 222 with HotSpot
=> let me update the JIRA with this piece of information

Kind regards,
Alexandre


Le mer. 25 sept. 2019 à 08:21, Stig Rohde Døssing
 a écrit :
>
> I believe those API changes are in Storm 1.2.0+ as well. You can upgrade to
> one of those versions, and update your topologies so they don't use
> deprecated methods. Then the topologies should be compatible with 2.x
> without rebuilding.
>
> Den tir. 24. sep. 2019 kl. 22.34 skrev Alexandre Vermeerbergen <
> avermeerber...@gmail.com>:
>
> > Hello Stig,
> >
> > The API changes are the ones mentionned in this thread:
> >
> > http://mail-archives.apache.org/mod_mbox/storm-dev/201905.mbox/%3cCAG09ER3iDfXY7+SixSsCDo=a+h0z-sd6hjwnfnfp+zfaw-r...@mail.gmail.com%3e
> >
> > My problem is that if I migrate my topologies to 2.x to try it, then I
> > have to duplicate some code to keep our topologies compatible with
> > Storm 1.x until we have 100% tested our migration.
> >
> > Unless I am missing something? Is there a way to run Storm
> > 1.x-compiled topologies on Storm 2.x without rebuilding?
> >
> > Kind regards,
> > Alexandre
> >
> >
> > Le dim. 22 sept. 2019 à 22:56, Stig Rohde Døssing
> >  a écrit :
> > >
> > > I thought the API changes for Storm 2.x were mostly minor. Which API
> > > changes are you being blocked by?
> > >
> > > Den søn. 22. sep. 2019 kl. 21.27 skrev Alexandre Vermeerbergen <
> > > avermeerber...@gmail.com>:
> > >
> > > > Hello Stig,
> > > >
> > > > Thank you very much for your answer.
> > > >
> > > > Upgrading to Storm 2.x is something I would like to try, but I know
> > > > that building with Storm 2.x API requires changing our topologies
> > > > since some API have changes (I used to try, and was kind of blocked by
> > > > the un-reversible changes that were required).
> > > >
> > > > Question: could our Storm 1.x topology work with Storm 2.x ?
> > > >
> > > > Kind regards,
> > > > Alexandre
> > > >
> > > > Le jeu. 19 sept. 2019 à 19:05, Stig Rohde Døssing
> > > >  a écrit :
> > > > >
> > > > > Looking at the stack trace, the issue is not in the Kafka bolt, but
> > in
> > > > some
> > > > > internal Storm statistics code. It seems likely to me that it has
> > been
> > > > > fixed in 2.0.0, as we haven't heard of anyone encounter it there.
> > Are you
> > > > > able to upgrade to 2.x to see if that fixes this?
> > > > >
> > > > > If not, the exception is coming from
> > > > >
> > > >
> > https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131
> > > > .
> > > > > Looks like either "amt" or " (stats-rate stats) " in that line is
> > null.
> > > > My
> > > > > guess would be stats. The equivalent code in 2.x uses primitive
> > ints, so
> > > > > won't be throwing NPEs
> > > > >
> > > >
> > https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63
> > > > >
> > > > > If you have the ability to upgrade to 2.0.0 that would be best.
> > > > >
> > > > > I don't think simply catching the NPE is a good solution, we should
> > > > prevent
> > > > > the NPE from being thrown in the first place. If you want to take a
> > look
> > > > at
> > > > > it, feel free to raise a PR. May want to also lobby to get a 1.2.4
> > > > release
> > > > > out after that.
> > > > >
> > > > > Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
> > > > > avermeerber...@gmail.com>:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I have seen couple of occurrences of the NullPointerException in
> > Storm
> > > > > > Kafka Spout with same stack trace as in
> > > > > > https://issues.apache.org/jira/browse/STORM-3032
> > > > > >
> > > > > > Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected
> > to a
> > > > > > cluster of Kafka Brokers at 2.2.1 version.
> > > > > >
> > > > > > Since the stack traces seems to tell that the NullPointerException
> > > > > > occurs in some code related to statistics computation, wouldn't it
> > be
> > > > > > better to fix the issue by catching the NullPointerException to
> > avoid
> > > > > > blocking Kafka consumption ?
> > > > > >
> > > > > > Kind regards,
> > > > > > Alexandre Vermeerbergen
> > > > > >
> > > >
> >

Re: Any hope for a fix of STORM-3032 ?

[Alexandre Vermeerbergen]

Hello Stig,

The API changes are the ones mentionned in this thread:
http://mail-archives.apache.org/mod_mbox/storm-dev/201905.mbox/%3cCAG09ER3iDfXY7+SixSsCDo=a+h0z-sd6hjwnfnfp+zfaw-r...@mail.gmail.com%3e

My problem is that if I migrate my topologies to 2.x to try it, then I
have to duplicate some code to keep our topologies compatible with
Storm 1.x until we have 100% tested our migration.

Unless I am missing something? Is there a way to run Storm
1.x-compiled topologies on Storm 2.x without rebuilding?

Kind regards,
Alexandre


Le dim. 22 sept. 2019 à 22:56, Stig Rohde Døssing
 a écrit :
>
> I thought the API changes for Storm 2.x were mostly minor. Which API
> changes are you being blocked by?
>
> Den søn. 22. sep. 2019 kl. 21.27 skrev Alexandre Vermeerbergen <
> avermeerber...@gmail.com>:
>
> > Hello Stig,
> >
> > Thank you very much for your answer.
> >
> > Upgrading to Storm 2.x is something I would like to try, but I know
> > that building with Storm 2.x API requires changing our topologies
> > since some API have changes (I used to try, and was kind of blocked by
> > the un-reversible changes that were required).
> >
> > Question: could our Storm 1.x topology work with Storm 2.x ?
> >
> > Kind regards,
> > Alexandre
> >
> > Le jeu. 19 sept. 2019 à 19:05, Stig Rohde Døssing
> >  a écrit :
> > >
> > > Looking at the stack trace, the issue is not in the Kafka bolt, but in
> > some
> > > internal Storm statistics code. It seems likely to me that it has been
> > > fixed in 2.0.0, as we haven't heard of anyone encounter it there. Are you
> > > able to upgrade to 2.x to see if that fixes this?
> > >
> > > If not, the exception is coming from
> > >
> > https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131
> > .
> > > Looks like either "amt" or " (stats-rate stats) " in that line is null.
> > My
> > > guess would be stats. The equivalent code in 2.x uses primitive ints, so
> > > won't be throwing NPEs
> > >
> > https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63
> > >
> > > If you have the ability to upgrade to 2.0.0 that would be best.
> > >
> > > I don't think simply catching the NPE is a good solution, we should
> > prevent
> > > the NPE from being thrown in the first place. If you want to take a look
> > at
> > > it, feel free to raise a PR. May want to also lobby to get a 1.2.4
> > release
> > > out after that.
> > >
> > > Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
> > > avermeerber...@gmail.com>:
> > >
> > > > Hello,
> > > >
> > > > I have seen couple of occurrences of the NullPointerException in Storm
> > > > Kafka Spout with same stack trace as in
> > > > https://issues.apache.org/jira/browse/STORM-3032
> > > >
> > > > Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected to a
> > > > cluster of Kafka Brokers at 2.2.1 version.
> > > >
> > > > Since the stack traces seems to tell that the NullPointerException
> > > > occurs in some code related to statistics computation, wouldn't it be
> > > > better to fix the issue by catching the NullPointerException to avoid
> > > > blocking Kafka consumption ?
> > > >
> > > > Kind regards,
> > > > Alexandre Vermeerbergen
> > > >
> >

Re: Any hope for a fix of STORM-3032 ?

[Alexandre Vermeerbergen]

Hello Stig,

Thank you very much for your answer.

Upgrading to Storm 2.x is something I would like to try, but I know
that building with Storm 2.x API requires changing our topologies
since some API have changes (I used to try, and was kind of blocked by
the un-reversible changes that were required).

Question: could our Storm 1.x topology work with Storm 2.x ?

Kind regards,
Alexandre

Le jeu. 19 sept. 2019 à 19:05, Stig Rohde Døssing
 a écrit :
>
> Looking at the stack trace, the issue is not in the Kafka bolt, but in some
> internal Storm statistics code. It seems likely to me that it has been
> fixed in 2.0.0, as we haven't heard of anyone encounter it there. Are you
> able to upgrade to 2.x to see if that fixes this?
>
> If not, the exception is coming from
> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131.
> Looks like either "amt" or " (stats-rate stats) " in that line is null. My
> guess would be stats. The equivalent code in 2.x uses primitive ints, so
> won't be throwing NPEs
> https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63
>
> If you have the ability to upgrade to 2.0.0 that would be best.
>
> I don't think simply catching the NPE is a good solution, we should prevent
> the NPE from being thrown in the first place. If you want to take a look at
> it, feel free to raise a PR. May want to also lobby to get a 1.2.4 release
> out after that.
>
> Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
> avermeerber...@gmail.com>:
>
> > Hello,
> >
> > I have seen couple of occurrences of the NullPointerException in Storm
> > Kafka Spout with same stack trace as in
> > https://issues.apache.org/jira/browse/STORM-3032
> >
> > Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected to a
> > cluster of Kafka Brokers at 2.2.1 version.
> >
> > Since the stack traces seems to tell that the NullPointerException
> > occurs in some code related to statistics computation, wouldn't it be
> > better to fix the issue by catching the NullPointerException to avoid
> > blocking Kafka consumption ?
> >
> > Kind regards,
> > Alexandre Vermeerbergen
> >

Any hope for a fix of STORM-3032 ?

[Alexandre Vermeerbergen]

Hello,

I have seen couple of occurrences of the NullPointerException in Storm
Kafka Spout with same stack trace as in
https://issues.apache.org/jira/browse/STORM-3032

Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected to a
cluster of Kafka Brokers at 2.2.1 version.

Since the stack traces seems to tell that the NullPointerException
occurs in some code related to statistics computation, wouldn't it be
better to fix the issue by catching the NullPointerException to avoid
blocking Kafka consumption ?

Kind regards,
Alexandre Vermeerbergen

Re: [ANNOUNCE] New Committer/PMC Member Aaron Gresch

[Alexandre Vermeerbergen]

This is great news (the more PMC members, the more healthly the Apache
Storm project is), welcome Aaron!

Alexandre Vermeerbergen

Le sam. 10 août 2019 à 10:47, Stig Rohde Døssing
 a écrit :
>
> I'm happy to announce that Aaron Gresch is now a member of the Storm PMC.
>
> Aaron has been active contributing to Storm for a while now. His
> contributions have a good spread around the codebase, in particular having
> to do with the blob store, metrics system and scheduling. He helped
> implement metrics v2.
>
> Please join me in welcoming Aaron to the PMC.

Re: [VOTE] Release Apache Storm 1.2.3 (rc2)

[Alexandre Vermeerbergen]

Hello Taylor,

Thank you very much for  this Storm 1.2.3 release.
You did a great job to make this process running smoothly for many
releases; and I know that donating one's time on such large project is
a huge contribution.
Whatever you'll do next, may the code with be you!

Alexandre Vermeerbergen

Le sam. 15 juin 2019 à 18:15, P. Taylor Goetz  a écrit :
>
> This vote is now closed and passes with 3 binding +1 votes and 1 non-binding 
> +1 vote.
>
> I will release the artifacts and create a release announcement.
>
> -Taylor
>
> > On May 10, 2019, at 12:20 PM, P. Taylor Goetz  wrote:
> >
> > This is a call to vote on releasing Apache Storm 1.2.3 (rc2)
> >
> > Full list of changes in this release:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc2/RELEASE_NOTES.html
> >
> > The tag/commit to be voted upon is v1.2.3:
> >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=ac0e39d7cad48b3ec863c8a3b711d36511d4daf6;hb=4e162a47c8219546ab9639401363a8f1b5e51119
> >
> > The source archive being voted upon can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc2/apache-storm-1.2.3-src.tar.gz
> >
> > Other release files, signatures and digests can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc2/
> >
> > The release artifacts are signed with the following key:
> >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >
> > The Nexus staging repository for this release is:
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1081
> >
> > Please vote on releasing this package as Apache Storm 1.2.3.
> >
> > When voting, please list the actions taken to verify the release.
> >
> > This vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache Storm 1.2.3
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > -Taylor
>

Re: [VOTE] Release Apache Storm 1.2.3 (rc2)

[Alexandre Vermeerbergen]

Oh and I forgot to mention that all my tests are made using
AdoptOpenJDK11.0.3 with OpenJ9.
It all runs very fine with this Java version & VM type.
We just had to patch our Hadoop dependencies (for HDFS & HBase Bolts)
to fix badly written Java version detection code in these
dependencies.


Le sam. 11 mai 2019 à 19:18, Alexandre Vermeerbergen
 a écrit :
>
> +1 (non binding)
>
> - Downloaded binary distribution & used it to update my Storm cloud service
> - Downloaded source distribution, compiled it and used the generated
> .jar files to update my Storm builder dependencies (storm-core.jar,
> storm-kafa-client.jar, flux*.jar)
> - Run my clusters with ~15 topologies
>
> Disclaimer : I have not yet ran test at big scale with this Storm
> 1.2.3 update, but I noticed from release notes that I was already
> running for several months with a Storm 1.2.3 pre-version updated my
> all commits that sounded useful to us (in particular the ones allowing
> to use recent Kafka brokers), so basically I'm confident to run 1.2.3
> final at scale within very short time after it'll be released.
>
> Kind regards,
> Alexandre Vermeerbergen
>
> Le ven. 10 mai 2019 à 18:28, P. Taylor Goetz  a écrit :
> >
> > This is a call to vote on releasing Apache Storm 1.2.3 (rc2)
> >
> > Full list of changes in this release:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc2/RELEASE_NOTES.html
> >
> > The tag/commit to be voted upon is v1.2.3:
> >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=ac0e39d7cad48b3ec863c8a3b711d36511d4daf6;hb=4e162a47c8219546ab9639401363a8f1b5e51119
> >
> > The source archive being voted upon can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc2/apache-storm-1.2.3-src.tar.gz
> >
> > Other release files, signatures and digests can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc2/
> >
> > The release artifacts are signed with the following key:
> >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >
> > The Nexus staging repository for this release is:
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1081
> >
> > Please vote on releasing this package as Apache Storm 1.2.3.
> >
> > When voting, please list the actions taken to verify the release.
> >
> > This vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache Storm 1.2.3
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > -Taylor

Re: [VOTE] Release Apache Storm 1.2.3 (rc2)

[Alexandre Vermeerbergen]

+1 (non binding)

- Downloaded binary distribution & used it to update my Storm cloud service
- Downloaded source distribution, compiled it and used the generated
.jar files to update my Storm builder dependencies (storm-core.jar,
storm-kafa-client.jar, flux*.jar)
- Run my clusters with ~15 topologies

Disclaimer : I have not yet ran test at big scale with this Storm
1.2.3 update, but I noticed from release notes that I was already
running for several months with a Storm 1.2.3 pre-version updated my
all commits that sounded useful to us (in particular the ones allowing
to use recent Kafka brokers), so basically I'm confident to run 1.2.3
final at scale within very short time after it'll be released.

Kind regards,
Alexandre Vermeerbergen

Le ven. 10 mai 2019 à 18:28, P. Taylor Goetz  a écrit :
>
> This is a call to vote on releasing Apache Storm 1.2.3 (rc2)
>
> Full list of changes in this release:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc2/RELEASE_NOTES.html
>
> The tag/commit to be voted upon is v1.2.3:
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=ac0e39d7cad48b3ec863c8a3b711d36511d4daf6;hb=4e162a47c8219546ab9639401363a8f1b5e51119
>
> The source archive being voted upon can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc2/apache-storm-1.2.3-src.tar.gz
>
> Other release files, signatures and digests can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc2/
>
> The release artifacts are signed with the following key:
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>
> The Nexus staging repository for this release is:
>
> https://repository.apache.org/content/repositories/orgapachestorm-1081
>
> Please vote on releasing this package as Apache Storm 1.2.3.
>
> When voting, please list the actions taken to verify the release.
>
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 1.2.3
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>
> -Taylor

Re: Storm 2.0 blogs ?

[Alexandre Vermeerbergen]

Yes, I am definitely interested first in "Kafka related changes"
because it's the biggest impact I currently have when trying to "port"
my topologies from 1.2.x to 2.x.

The other topics are also very tempting, I think I'll jump to the
"Java port" one as soon it's published..

Storm 2.x's going to such a big change for us, i'm impatient to dig this!

Kind regards,
Alexandre

Le ven. 10 mai 2019 à 18:40, P. Taylor Goetz  a écrit :
>
> I’m going to distill everything mentioned so far into a release announcement 
> that I hope to get out later today, or more likely, Monday.
>
> Then we can add follow up blog posts that go into greater detail.
>
> For reference, I’m reposting the list of topics Roshan came up with:
>
> 1- Re-architecture - [Roshan]
> 2- Windowing enhancements
> 3- SQL enhancements
> 4- Metrics
> 5- Kafka related changes
> 6- Security (nimbus admin groups, delegation tokens, optional impersonation)
> 7- PMML (Machine Learning) support.
> 8- Streams API
> 9- Module restructuring & dependency mitigation
> 10- Java porting
> 11- DRPC cmd line
> 12- Lambda support
> 13- New spouts: Kinesis & Druid ?
> 14- Changes to deployment and cli submission
> 15- RAS changes
> 16- Trident enhancements
> 17- New Admin cmds to debug cluster state
> 18 ... others ?
>
> -Taylor
>
>
>
>
> > On Jan 29, 2019, at 6:16 PM, Arun M  wrote:
> >
> > Heres a short blurb for Streams API & windowing enhancements.
> >
> > 1. Streams API
> >
> > A typed API for expressing streaming computations more easily using
> > functional style operations.
> > It builds on top of the Storm's core spouts and bolt APIs and automatically
> > fuses multiple operations together to optimize the pipeline.
> > For more details and examples see -
> > https://github.com/apache/storm/blob/master/docs/Stream-API.md
> >
> > 2. Windowing enhancements
> >
> > Storm can save/restore the window state to the configured state backend so
> > that larger continuous windows can be supported.
> > The window boundaries can now be accessed via the APIs.
> > For more details see -
> > https://github.com/apache/storm/blob/master/docs/Windowing.md#stateful-windowing
> >
> > Thanks,
> > Arun
>

Re: [CANCELED] [VOTE] Release Apache Storm 1.2.3 (rc1)

[Alexandre Vermeerbergen]

Hi Taylor,

Sorry for having bugged you with this detail.
I see that you posted an annonce for 1.2.3-RC2, let me use this one to
run my tests, I will provide feedback & vote ASAP.

Kind regards,
Alexandre

Le ven. 10 mai 2019 à 18:18, P. Taylor Goetz  a écrit :
>
> Good catch. Off-by-one error on my part.
>
> The release candidate is still valid. I just need to do an `svn mv`.
>
> Canceling vote to do an svn rename.
>
> -Taylor
>
> > On May 9, 2019, at 3:48 AM, Alexandre Vermeerbergen 
> >  wrote:
> >
> > Hello,
> >
> > I am confused : there was a mail calling for voting for 1.2.3-RC1 on
> > 8th of January this year.
> > Is this new call for vote related to exactly the same "code freeze",
> > or is that a newer code freeze, in which case shouldn't be called
> > "1.2.3-rc2" ?
> >
> > Kind regards,
> > Alexandre Vermeerbergen
> >
> > Le mer. 8 mai 2019 à 21:30, P. Taylor Goetz  a écrit :
> >>
> >> This is a call to vote on releasing Apache Storm 1.2.3 (rc1)
> >>
> >> Full list of changes in this release:
> >>
> >> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/RELEASE_NOTES.html
> >>
> >> The tag/commit to be voted upon is v1.2.3:
> >>
> >> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=ac0e39d7cad48b3ec863c8a3b711d36511d4daf6;hb=4e162a47c8219546ab9639401363a8f1b5e51119
> >>
> >> The source archive being voted upon can be found here:
> >>
> >> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/apache-storm-1.2.3-src.tar.gz
> >>
> >> Other release files, signatures and digests can be found here:
> >>
> >> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/
> >>
> >> The release artifacts are signed with the following key:
> >>
> >> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >>
> >> The Nexus staging repository for this release is:
> >>
> >> https://repository.apache.org/content/repositories/orgapachestorm-1081
> >>
> >> Please vote on releasing this package as Apache Storm 1.2.3.
> >>
> >> When voting, please list the actions taken to verify the release.
> >>
> >> This vote will be open for at least 72 hours.
> >>
> >> [ ] +1 Release this package as Apache Storm 1.2.3
> >> [ ]  0 No opinion
> >> [ ] -1 Do not release this package because...
> >>
> >> Thanks to everyone who contributed to this release.
> >>
> >> -Taylor
>

Re: [VOTE] Release Apache Storm 1.2.3 (rc1)

[Alexandre Vermeerbergen]

Hello,

I am confused : there was a mail calling for voting for 1.2.3-RC1 on
8th of January this year.
Is this new call for vote related to exactly the same "code freeze",
or is that a newer code freeze, in which case shouldn't be called
"1.2.3-rc2" ?

Kind regards,
Alexandre Vermeerbergen

Le mer. 8 mai 2019 à 21:30, P. Taylor Goetz  a écrit :
>
> This is a call to vote on releasing Apache Storm 1.2.3 (rc1)
>
> Full list of changes in this release:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/RELEASE_NOTES.html
>
> The tag/commit to be voted upon is v1.2.3:
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=ac0e39d7cad48b3ec863c8a3b711d36511d4daf6;hb=4e162a47c8219546ab9639401363a8f1b5e51119
>
> The source archive being voted upon can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/apache-storm-1.2.3-src.tar.gz
>
> Other release files, signatures and digests can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/
>
> The release artifacts are signed with the following key:
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>
> The Nexus staging repository for this release is:
>
> https://repository.apache.org/content/repositories/orgapachestorm-1081
>
> Please vote on releasing this package as Apache Storm 1.2.3.
>
> When voting, please list the actions taken to verify the release.
>
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 1.2.3
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>
> -Taylor

Re: [VOTE] Release Apache Storm 2.0.0 RC7

[Alexandre Vermeerbergen]

Hello,

Thanks, and my step issue with Storm 2.0.0 is with some Storm Kafka
Client breaking changes in KafkaSpoutConfig class.
See attached source for our homebrewed BasicKafkaSpout, I have at
least the following compilation errors:

./i3DXSupTopologiesCommons/i3DXSupTopologiesCommons.mj/src/com/dassault_systemes/storm/spout/BasicKafkaSpout.java:58:
error: cannot find symbol
logger.debug("topics are: {}", config.getSubscription()

./i3DXSupTopologiesCommons/i3DXSupTopologiesCommons.mj/src/com/dassault_systemes/storm/spout/BasicKafkaSpout.java:77:
error: cannot find symbol
if (config.getKeyDeserializer()==null) {

./i3DXSupTopologiesCommons/i3DXSupTopologiesCommons.mj/src/com/dassault_systemes/storm/spout/BasicKafkaSpout.java:91:
error: cannot find symbol
this.valueDeserializer =
config.getValueDeserializer().getClass();

and this more complex error:

./i3DXSupTopologiesCommons/i3DXSupTopologiesCommons.mj/src/com/dassault_systemes/storm/spout/BasicKafkaSpout.java:91:
error: incompatible types: getClass cannot be converted to Class
this.valueDeserializer =
config.getValueDeserializer().getClass();


What would be the migration path from Storm 1.x to Storm 2.x for this
storm-kafka-client dependent class?

Kind regards,
Alexandre Vermeerbergen

Le mer. 1 mai 2019 à 21:18, Stig Rohde Døssing
 a écrit :
>
> Yes, we already have a check that the Maven version is above 3.0.0 (
> https://github.com/apache/storm/blob/master/pom.xml#L1439), IMO we can just
> bump it to something like 3.5.0. I'll raise an issue for it.
>
> Den ons. 1. maj 2019 kl. 20.59 skrev Alexandre Vermeerbergen <
> avermeerber...@gmail.com>:
>
> > Hello Stig,
> >
> > Good catch: I upgraded Maven to version 3.6.1 and I rebuilt everything
> > from source tar.gz without any trouble.
> >
> > So I delay my vote and I'm no longer blocked since I have
> > storm-kafka-client-2.0.0.jar allowing me to rebuilt my topologies.
> >
> > Just a question: since my experience showed that build Storm 2.0.0
> > break with Maven 3.3.9, wouldn't it be possible to improve Storm's
> > pom.xml to include a Maven version check (see
> > https://maven.apache.org/enforcer/enforcer-rules/versionRanges.html) ?
> >
> > This would improve Storm users' experience a little bit, and sounds
> > very simple to do (not a blocker for 2.0.0 of course).
> >
> > How about that?
> >
> > Kind regards,
> > Alexandre Vermeerbergen
> >
> > Le mer. 1 mai 2019 à 19:42, Stig Rohde Døssing
> >  a écrit :
> > >
> > > I haven't seen the error you're getting before, but it looks like it's a
> > > Maven issue, not a Storm issue. Could you try updating to the latest
> > Maven
> > > version?
> > >
> > > Den ons. 1. maj 2019 kl. 19.30 skrev Alexandre Vermeerbergen <
> > > avermeerber...@gmail.com>:
> > >
> > > > Hello,
> > > >
> > > > I don't know if this is worth a -1 because maybe I did something
> > > > wrong, but I got a failure when trying to build binaries from the
> > > > sources, see attached log (zipped) with 4 errors, captured from the
> > > > output of:
> > > >
> > > > mvn clean install -DskipTests 2>&1 build_Storm_2.0.0_rc7.log
> > > >
> > > > more details on my machine:
> > > >
> > > > $ cat /etc/redhat-release
> > > > CentOS release 6.7 (Final)
> > > >
> > > > $ mvn -version
> > > > Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5;
> > > > 2015-11-10T17:41:47+01:00)
> > > > Maven home: /home/data/ave/maven/src/apache-maven-3.3.9
> > > > Java version: 1.8.0_66, vendor: Oracle Corporation
> > > > Java home: /usr/java/jdk1.8.0_66_x86_64/jdk1.8.0_66/jre
> > > > Default locale: en_US, platform encoding: UTF-8
> > > > OS name: "linux", version: "2.6.32-573.el6.x86_64", arch: "amd64",
> > > > family: "unix"
> > > >
> > > > I got couple of JAR files built, but I'm missing the
> > > > storm-kafka-client one, so am blocked because I need this "external"
> > > > lib to test my topologies rebuilt with Storm 2.0.0 RC7
> > > >
> > > > Did I missed something that break my Storm 2.0.0 RC7 build?
> > > >
> > > > Kind regards,
> > > > Alexandre Vermeerbergen
> > > >
> > > >
> > > > Le mar. 30 avr. 2019 à 00:49, P. Taylor Goetz  a
> > écrit
> > > > :
> > > > >

Re: [VOTE] Release Apache Storm 2.0.0 RC7

[Alexandre Vermeerbergen]

Hello Stig,

Good catch: I upgraded Maven to version 3.6.1 and I rebuilt everything
from source tar.gz without any trouble.

So I delay my vote and I'm no longer blocked since I have
storm-kafka-client-2.0.0.jar allowing me to rebuilt my topologies.

Just a question: since my experience showed that build Storm 2.0.0
break with Maven 3.3.9, wouldn't it be possible to improve Storm's
pom.xml to include a Maven version check (see
https://maven.apache.org/enforcer/enforcer-rules/versionRanges.html) ?

This would improve Storm users' experience a little bit, and sounds
very simple to do (not a blocker for 2.0.0 of course).

How about that?

Kind regards,
Alexandre Vermeerbergen

Le mer. 1 mai 2019 à 19:42, Stig Rohde Døssing
 a écrit :
>
> I haven't seen the error you're getting before, but it looks like it's a
> Maven issue, not a Storm issue. Could you try updating to the latest Maven
> version?
>
> Den ons. 1. maj 2019 kl. 19.30 skrev Alexandre Vermeerbergen <
> avermeerber...@gmail.com>:
>
> > Hello,
> >
> > I don't know if this is worth a -1 because maybe I did something
> > wrong, but I got a failure when trying to build binaries from the
> > sources, see attached log (zipped) with 4 errors, captured from the
> > output of:
> >
> > mvn clean install -DskipTests 2>&1 build_Storm_2.0.0_rc7.log
> >
> > more details on my machine:
> >
> > $ cat /etc/redhat-release
> > CentOS release 6.7 (Final)
> >
> > $ mvn -version
> > Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5;
> > 2015-11-10T17:41:47+01:00)
> > Maven home: /home/data/ave/maven/src/apache-maven-3.3.9
> > Java version: 1.8.0_66, vendor: Oracle Corporation
> > Java home: /usr/java/jdk1.8.0_66_x86_64/jdk1.8.0_66/jre
> > Default locale: en_US, platform encoding: UTF-8
> > OS name: "linux", version: "2.6.32-573.el6.x86_64", arch: "amd64",
> > family: "unix"
> >
> > I got couple of JAR files built, but I'm missing the
> > storm-kafka-client one, so am blocked because I need this "external"
> > lib to test my topologies rebuilt with Storm 2.0.0 RC7
> >
> > Did I missed something that break my Storm 2.0.0 RC7 build?
> >
> > Kind regards,
> > Alexandre Vermeerbergen
> >
> >
> > Le mar. 30 avr. 2019 à 00:49, P. Taylor Goetz  a écrit
> > :
> > >
> > > This is a call to vote on releasing Apache Storm 2.0.0 (rc7)
> > >
> > > Full list of changes in this release:
> > >
> > >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/RELEASE_NOTES.html
> > >
> > > The tag/commit to be voted upon is v2.0.0:
> > >
> > >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=007863edd95e838b3df414928c6fa3f28244ab49;hb=2ba95bbd1c911d4fc6363b1c4b9c4c6d86ac9aae
> > >
> > > The source archive being voted upon can be found here:
> > >
> > >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/apache-storm-2.0.0-src.tar.gz
> > >
> > > Other release files, signatures and digests can be found here:
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/
> > >
> > > The release artifacts are signed with the following key:
> > >
> > >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> > >
> > > The Nexus staging repository for this release is:
> > >
> > > https://repository.apache.org/content/repositories/orgapachestorm-1079
> > >
> > > Please vote on releasing this package as Apache Storm 2.0.0.
> > >
> > > When voting, please list the actions taken to verify the release.
> > >
> > > This vote will be open for at least 72 hours.
> > >
> > > [ ] +1 Release this package as Apache Storm 2.0.0
> > > [ ]  0 No opinion
> > > [ ] -1 Do not release this package because...
> > >
> > > Thanks to everyone who contributed to this release.
> > >
> > > -Taylor
> >

Re: [VOTE] Release Apache Storm 2.0.0 RC7

[Alexandre Vermeerbergen]

Hello,

I tried to rebuild my topologies with 2.0, but I hit a first trivial
issue: I used to rely on storm-core-.jar in our classpath to
resolve Storm API dependencies, but storm-core-2.0.0.jar doesn't seem
to contain anymore classes like org.apache.storm.task.OutputCollector
which is used in our topologies' code.

I did a "grep org.apache.storm.task.OutputCollector " and found that this kind of classes are now in
storm-client-2.0.0.jar

I'm OK to change my dependencies, but then, to avoid other people
falling into this trap, couldn't Storm 2.0.0 documentation tell which
are the build-time dependencies?

I also need to know whether or not my BigJars must embed
storm-client.2.0.0.jar (we use to embed storm-core-.jar in
our BigJar)

Kind regards,
Alexandre Vermeerbergen

Le mar. 30 avr. 2019 à 23:49, Roshan Naik
 a écrit :
>
>  Yes, you need to rebuild your topology jars against 2.0.
> If you have some settings to tweak perf with 1.x,  refer to 
> https://github.com/apache/storm/blob/master/docs/Performance.md for the 2.x 
> configs.
>
> On Tuesday, April 30, 2019, 2:41:49 PM PDT, Alexandre Vermeerbergen 
>  wrote:
>
>  Hello,
>
> I'm eager to test Storm 2.0.0 with my complex topologies, but first of
> all: do I need to rebuild all my topologies' Big Jars with Storm
> 2.0.0, or may I try my existing Storm 1.2.3 (recent snapshot)-based
> Big Jars ?
>
> Kind regards,
> Alexandre Vermeerbergen
>
> Le mar. 30 avr. 2019 à 00:49, P. Taylor Goetz  a écrit :
> >
> > This is a call to vote on releasing Apache Storm 2.0.0 (rc7)
> >
> > Full list of changes in this release:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/RELEASE_NOTES.html
> >
> > The tag/commit to be voted upon is v2.0.0:
> >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=007863edd95e838b3df414928c6fa3f28244ab49;hb=2ba95bbd1c911d4fc6363b1c4b9c4c6d86ac9aae
> >
> > The source archive being voted upon can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/apache-storm-2.0.0-src.tar.gz
> >
> > Other release files, signatures and digests can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/
> >
> > The release artifacts are signed with the following key:
> >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >
> > The Nexus staging repository for this release is:
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1079
> >
> > Please vote on releasing this package as Apache Storm 2.0.0.
> >
> > When voting, please list the actions taken to verify the release.
> >
> > This vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache Storm 2.0.0
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > -Taylor

Re: [VOTE] Release Apache Storm 2.0.0 RC7

[Alexandre Vermeerbergen]

Hello,

I'm eager to test Storm 2.0.0 with my complex topologies, but first of
all: do I need to rebuild all my topologies' Big Jars with Storm
2.0.0, or may I try my existing Storm 1.2.3 (recent snapshot)-based
Big Jars ?

Kind regards,
Alexandre Vermeerbergen

Le mar. 30 avr. 2019 à 00:49, P. Taylor Goetz  a écrit :
>
> This is a call to vote on releasing Apache Storm 2.0.0 (rc7)
>
> Full list of changes in this release:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/RELEASE_NOTES.html
>
> The tag/commit to be voted upon is v2.0.0:
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=007863edd95e838b3df414928c6fa3f28244ab49;hb=2ba95bbd1c911d4fc6363b1c4b9c4c6d86ac9aae
>
> The source archive being voted upon can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/apache-storm-2.0.0-src.tar.gz
>
> Other release files, signatures and digests can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc7/
>
> The release artifacts are signed with the following key:
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>
> The Nexus staging repository for this release is:
>
> https://repository.apache.org/content/repositories/orgapachestorm-1079
>
> Please vote on releasing this package as Apache Storm 2.0.0.
>
> When voting, please list the actions taken to verify the release.
>
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 2.0.0
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>
> -Taylor

Re: [VOTE] Release Apache Storm 1.2.3 (rc1)

[Alexandre Vermeerbergen]

Hello Taylor,

+1

DownloadedStorm1.2.3 RC1 + build 1.2.3 RC1 from sources to get up to
date storm-kafka-client

Upgraded by integration cluster with 1.2.3 RC.
Configuration:
- CentOS7, Java 11 from AdoptOpenJDK11.0.1 with Hotspot  on all VMs
- 1 Nimbus VM, 7 Supervisor,VMs, 3 Zookeeper VMs, 19 topologies with
high dependency on Storm
Kafka Client, using Kafka Client 2.0.1 connected to a distinct Kafka
Brokers Cluster (5 VMs) based on Kafka 1.1.1 (this later one still
runs with Java 8).

Kind regards,
Alexandre

Le mar. 8 janv. 2019 à 21:20, P. Taylor Goetz  a écrit :
>
> This is a call to vote on releasing Apache Storm 1.2.3 (rc1)
>
> Full list of changes in this release:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/RELEASE_NOTES.html
>
> The tag/commit to be voted upon is v1.2.3:
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=5eaaa51591f2e4dc3e31f22bcc581af4a1f39c03;hb=6ba98b215857656e0186887b5d1a6a5aceee10c4
>
> The source archive being voted upon can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/apache-storm-1.2.3-src.tar.gz
>
> Other release files, signatures and digests can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/
>
> The release artifacts are signed with the following key:
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>
> The Nexus staging repository for this release is:
>
> https://repository.apache.org/content/repositories/orgapachestorm-1074
>
> Please vote on releasing this package as Apache Storm 1.2.3.
>
> When voting, please list the actions taken to verify the release.
>
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 1.2.3
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>
> -Taylor

Re: [VOTE] Release Apache Storm 1.2.3 (rc1)

[Alexandre Vermeerbergen]

Hello Taylor,

For your information, I have upgraded my "small test cluster" with
this Storm 1.2.3 RC1 and found no regression.

My setup is: Linux CentOS 7, AdoptOpenJDK with HotSpot 11.0.1, 1
Nimbus VM, 1 Supervisor VM, 1 Zookeeper VM, 19 topologies with high
dependency on Storm Kafka Client, using Kafka Client 2.0.1 connected
to a distinct Kafka Brokers Cluster (5 VMs) based on Kafka 1.1.1 (this
later one still runs with Java 8).

I have observed no  regression on this small cluster vs. the 1.2.3
snapshot taken 6th of November 2018.

Before voting, I want to upgrade my integration cluster with Storm
1.2.3 RC1, its configuration is "larger": 1 Nimbus VM, 7 Supervisor
VMs, 3 Zookeeper VMs, 19 topologies with high dependency on Storm
Kafka Client, using Kafka Client 2.0.1 connected to a distinct Kafka
Brokers Cluster (5 VMs) based on Kafka 1.1.1 (this later one still
runs with Java 8).

I way also try this RC1 with a Kafka Brokers Cluster (still 5 VMs)
based on Kafka 2.1.0 with Java 11.

Stay tuned :)

Kind regards,
Alexandre

Le mar. 8 janv. 2019 à 21:20, P. Taylor Goetz  a écrit :
>
> This is a call to vote on releasing Apache Storm 1.2.3 (rc1)
>
> Full list of changes in this release:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/RELEASE_NOTES.html
>
> The tag/commit to be voted upon is v1.2.3:
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=5eaaa51591f2e4dc3e31f22bcc581af4a1f39c03;hb=6ba98b215857656e0186887b5d1a6a5aceee10c4
>
> The source archive being voted upon can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/apache-storm-1.2.3-src.tar.gz
>
> Other release files, signatures and digests can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/
>
> The release artifacts are signed with the following key:
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>
> The Nexus staging repository for this release is:
>
> https://repository.apache.org/content/repositories/orgapachestorm-1074
>
> Please vote on releasing this package as Apache Storm 1.2.3.
>
> When voting, please list the actions taken to verify the release.
>
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 1.2.3
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>
> -Taylor

Re: Storm 2.0.0 release?

[Alexandre Vermeerbergen]

+1

Le sam. 5 janv. 2019 à 14:12, Kishorkumar Patil
 a écrit :
>
> +1
>
>
> On Sat, Jan 5, 2019 at 5:04 AM Stig Rohde Døssing 
> wrote:
>
> > +1
> >
> > Den lør. 5. jan. 2019 kl. 07.42 skrev Govind Menon <
> > govindappume...@gmail.com>:
> >
> > > +1
> > >
> > > On Jan 4, 2019 23:46, "Roshan Naik" 
> > wrote:
> > >
> > > +1
> > >
> > >
> > > Sent from Yahoo Mail for iPhone
> > >
> > >
> > >
> > > On Friday, January 4, 2019, 9:02 PM, Ethan Li  > >
> > > wrote:
> > >
> > > +1 on this
> > >
> > > Ethan Li
> > >
> > > > On Jan 4, 2019, at 20:19, P. Taylor Goetz  wrote:
> > > >
> > > > If no one objects, I’ll kick off a release candidate.
> > > >
> > > > -Taylor
> > > >
> > > >> On Jan 4, 2019, at 9:06 PM, saurabh mimani 
> > > wrote:
> > > >>
> > > >> Hey, Any approximate date for 2.0 release given there are no blockers?
> > > >>
> > > >>
> > > >>
> > > >> Best Regards
> > > >>
> > > >> Saurabh Kumar Mimani
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Sat, Dec 22, 2018 at 1:13 AM Stig Rohde Døssing <
> > > stigdoess...@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> Looks good to me, the blockers list in JIRA is empty for 2.0.0.
> > > >>>
> > >  Den fre. 21. dec. 2018 kl. 19.52 skrev Bobby Evans <
> > bo...@apache.org
> > > >:
> > > 
> > >  I think all of the blockers are in now.  Please take a look and
> > > >>> hopefully,
> > >  we can get a release out soon.
> > > 
> > >  Thanks,
> > > 
> > >  Bobby
> > > 
> > > 
> > > > On Fri, Dec 14, 2018 at 9:02 AM Bobby Evans 
> > > wrote:
> > > >
> > > > Sorry I was out at a conference for the past week, and have been
> > > heads
> > >  down
> > > > on a different project for a while before that.  I'll respond to
> > the
> > >  JIRA.
> > > > I am happy to let it go in.
> > > >
> > > > Thanks,
> > > >
> > > > Bobby
> > > >
> > > > On Thu, Dec 13, 2018 at 1:07 PM Stig Rohde Døssing <
> > >  stigdoess...@gmail.com
> > > >>
> > > > wrote:
> > > >
> > > >> I think STORM-2990/3279 is ready. Bobby had a question (
> > > >> https://github.com/apache/storm/pull/2907#discussion_r234329136)
> > > > regarding
> > > >> whether Kafka offsets loop, but I wasn't sure where he was going
> > > with
> > >  it,
> > > >> so I didn't want to merge prematurely.
> > > >>
> > > >> I agree that we can postpone STORM-2720. As far as I know it's
> > > >>> waiting
> > > > for
> > > >> STORM-2990 to go in, since it's going to be touching the same
> > code.
> > > >>
> > > >> Den tor. 13. dec. 2018 kl. 19.54 skrev Roshan Naik
> > > >> :
> > > >>
> > > >>> Sounds like -  https://github.com/apache/storm/pull/2913
> > >  (STORM-3290)
> > > >> is
> > > >>> merged -  https://github.com/apache/storm/pull/2908 (STORM-3276)
> > > >>> is
> > > >>> nearly complete and may need some small tweaks. -
> > > >>> https://github.com/apache/storm/pull/2907  (STORM-2990,
> > > >>> STORM-3279)
> > > >>> appears ready to be committed ?
> > > >>>
> > > >>> and- https://github.com/apache/storm/pull/2911   - (STORM-2720)
> > > > seems a
> > > >>> bit inactive and may not be critical enough to wait on.
> > > >>>
> > > >>> -roshan
> > > >>>  On Monday, November 26, 2018, 9:49:30 AM PST, Stig Rohde
> > > >>> Døssing
> > >  <
> > > >>> stigdoess...@gmail.com> wrote:
> > > >>>
> > > >>> I would like to get at least
> > > > https://github.com/apache/storm/pull/2913
> > > >>> (breaking changes) and https://github.com/apache/storm/pull/2908
> > > >>> (regression) in.
> > > >>>
> > > >>> I think it would be nice to also get
> > > >>> https://github.com/apache/storm/pull/2907 and
> > > >>> https://github.com/apache/storm/pull/2911 in, but if we're in a
> > >  hurry
> > > >> they
> > > >>> could go in the next release.
> > > >>>
> > > >>> Den man. 26. nov. 2018 kl. 17.37 skrev Julien Nioche <
> > > >>> lists.digitalpeb...@gmail.com>:
> > > >>>
> > >  Hi devs,
> > > 
> > >  Is there anything blocking the release of Storm 2.0? Any idea of
> > >  when
> > > >> it
> > >  could happen?
> > > 
> > >  Thanks
> > > 
> > >  Julien
> > > 
> > >  --
> > > 
> > >  *Open Source Solutions for Text Engineering*
> > > 
> > >  http://www.digitalpebble.com
> > >  http://digitalpebble.blogspot.com/
> > >  #digitalpebble 
> > > 
> > > >>>
> > > >>
> > > >
> > > 
> > > >>>
> > >
> >
> --
> -Kishor

Re: NullPointerException in KafkaOffsetMetric.getValueAndReset causing worker to die

[Alexandre Vermeerbergen]

Hello Stig,

Thank you very much for your answer : I have tested our many
topologies using Kafka Client 2.0.1 instead of Kafka Client 2.0.0, let
them run at full charge for couple of days, and I can confirm that
this exception no longer occurs !

May I suggest storm-kafka-client documentation to mention that this
version of Kafka Client is recommended ?

Kind regards,
Alexandre Vermeerbergen

Le lun. 12 nov. 2018 à 23:17, Stig Rohde Døssing
 a écrit :
>
> I don't think beginningOffsets is null. I think it's missing one of the
> partitions, which would mean the right hand side of the line is null, which
> gives an NPE when we try to assign it to a primitive long.
>
> I think this could be due to
> https://issues.apache.org/jira/browse/KAFKA-7044, going by the commit
> message for the fix
> https://github.com/apache/kafka/commit/e2ec2d79c8d5adefc0c764583cec47144dbc5705#diff-b45245913eaae46aa847d2615d62cde0.
> Specifically part 2 sounds a lot like what I think might be happening here.
>
> "
>
> `ConsumerGroupCommand.getLogEndOffsets()` and `getLogStartOffsets()`
> assumed that endOffsets()/beginningOffsets() which eventually call
> Fetcher.fetchOffsetsByTimes(), would return a map with all the topic
> partitions passed to endOffsets()/beginningOffsets() and that values
> are not null. Because of (1), null values were possible if some of the
> topic partitions were already known (in metadata cache) and some not
> (metadata cache did not have entries for some of the topic
> partitions). However, even with fixing (1),
> endOffsets()/beginningOffsets() may return a map with some topic
> partitions missing, when list offset request returns a non-retriable
> error.
>
> "
>
> Basically KafkaOffsetMetric also assumes that when beginningOffsets(topics)
> is called, the returned map will contain a value for all requested topics.
> Could you try upgrading to Kafka 2.0.1?
>
> If necessary we can also work around this on the Storm side by skipping the
> metrics if the requested partition isn't in the return values for
> beginningOffsets/endOffsets. Feel free to raise an issue for this.
>
> Den man. 12. nov. 2018 kl. 21.56 skrev Alexandre Vermeerbergen <
> avermeerber...@gmail.com>:
>
> > Hello,
> >
> > Using Storm 1.2.3-snapshot of the 3rd of November 2018 with all libs
> > (storm-core & storm-kafka-client) taken from same Git, we get the
> > following crash coming from a NullPointerException in
> > KafkaOffsetMetric.getValueAndReset :
> >
> > 2018-11-12 19:31:30.496 o.a.s.util
> > Thread-9-metricsFromKafka-executor[13 13] [ERROR] Async loop died!
> >
> > java.lang.RuntimeException: java.lang.NullPointerException
> >
> >   at
> > org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522)
> > ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
> >
> >   at
> > org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)
> > ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
> >
> >   at
> > org.apache.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:477)
> > ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
> >
> >   at
> > org.apache.storm.disruptor$consume_batch.invoke(disruptor.clj:70)
> > ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
> >
> >   at
> > org.apache.storm.daemon.executor$fn__9620$fn__9635$fn__9666.invoke(executor.clj:634)
> > ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
> >
> >   at org.apache.storm.util$async_loop$fn__561.invoke(util.clj:484)
> > [storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
> >
> >   at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
> >
> >   at java.lang.Thread.run(Thread.java:748) [?:1.8.0_192]
> >
> > Caused by: java.lang.NullPointerException
> >
> >   at
> > org.apache.storm.kafka.spout.metrics.KafkaOffsetMetric.getValueAndReset(KafkaOffsetMetric.java:89)
> > ~[stormjar.jar:?]
> >
> >   at
> > org.apache.storm.daemon.executor$metrics_tick$fn__9544.invoke(executor.clj:345)
> > ~[storm-core-1.2.3-SNAPSHOT.jar:1.2.3-SNAPSHOT]
> >
> >   at clojure.core$map$fn__4553.invoke(core.clj:2622)
> > ~[clojure-1.7.0.jar:?]
> >
> >   at clojure.lang.LazySeq.sval(LazySeq.java:40)
> > ~[clojure-1.7.0.jar:?]
> >
> >   at clojure.lang.LazySeq.seq(LazySeq.java:49)
> > ~[clojure-1.7.0.jar:?]
> >
> >   at clojure.lang.RT.seq(RT.java:507) ~[clojure-1.7.0.jar:?]
> >
> >   at clojure.core$seq__4128.invoke(core.clj:137)
> >