Re: [VOTE] Release Apache Storm 2.0.0 (rc4)

[Roshan Naik]

 Was verifying the performance on 2.0 and noticed that the inter worker 
messaging has fallen off a cliff  With ConstSpoutNullBoltTopo (producer 
batch size=1k, ackers=0, workers=2) it was ~3.2mill/sec down  and now its down 
to  ~1.6mill/sec. Something has impacted this recently. Will try to narrow down 
the issue by tomorrow hopefully. Single worker numbers look good.  
-roshan
On Friday, January 11, 2019, 2:46:14 AM PST, Stig Rohde Døssing 
 wrote:  
 
 Thanks Arun,

I don't have an opinion on it, just wanted to make sure that it wasn't an
oversight.

+1

Set up Storm from the binary zip file
Built storm-kafka-client-examples from the zip while pointing Maven at the
Nexus repo
Ran one of the example Kafka topologies with a single-node cluster against
a local Kafka. Verified there were no errors in the logs, clicked around in
UI.

Den tor. 10. jan. 2019 kl. 22.25 skrev Arun Mahadevan :

> This is for users to use the "auto credentials" mechanism (delegation
> tokens) with HDFS/Hive/Hbase.
>
> We have been shipping it since 1.x (I think since 1.2.0 release) so that
> users can just add that directory to class path rather than building it
> separately to get the right dependencies. We could consider removing it
> from the main binary and ship it separately but it will need changes to the
> build, release and documentation and users will need to download and
> install it separately.
>
>
> Thanks,
> Arun
>
> On Thu, 10 Jan 2019 at 10:28, Stig Rohde Døssing 
> wrote:
>
> > I think this was remarked on by Roshan in the last RC, but the binary
> > distribution has become significantly larger since 1.x. It looks like
> this
> > is down to storm-autocreds not being added to the exclusion list in
> > storm-dist/binary/final-package/src/main/assembly/binary.xml.
> >
> > Since the module isn't excluded, external/storm-autocreds contains the
> > module jar, plus all dependency jars. Is this an accident, or do we want
> to
> > include these jars in the distribution?
> >
> > Den ons. 9. jan. 2019 kl. 19.48 skrev Ethan Li <
> ethanopensou...@gmail.com
> > >:
> >
> > > +1
> > >
> > > - Built from the src, ran all the unit tests and integration tests.
> > > - Set up a single-node cluster and submit ThroughputVsLatency topology.
> > > - Checked the UI.
> > > They look good.
> > >
> > > Thanks
> > > Ethan
> > >
> > > > On Jan 9, 2019, at 8:48 AM, Bobby Evans  wrote:
> > > >
> > > > +1 built from the git tag.  Ran all of the unit tests and ran some
> > manual
> > > > tests they all passed.
> > > >
> > > > Thanks,
> > > >
> > > > Bobby
> > > >
> > > > On Tue, Jan 8, 2019 at 6:30 PM Xin Wang 
> > wrote:
> > > >
> > > >> +1
> > > >>
> > > >> Built it and ran all of the tests.  Everything passed.
> > > >>
> > > >> -Xin
> > > >>
> > > >> Kishorkumar Patil  于2019年1月9日周三 上午5:08写道：
> > > >>
> > > >>> +1
> > > >>>
> > > >>> - built from source code and deployment works.
> > > >>> -  Ran some of the tests for UI, DRPC, ThroughputVsLatency
> > > >>> -  Validated UI bugs reported in the recent past are fixed in this
> > > >> version
> > > >>>
> > > >>> -Kishor
> > > >>>
> > > >>>
> > > >>> On Tue, Jan 8, 2019 at 2:29 PM Arun Mahadevan 
> > > wrote:
> > > >>>
> > >  +1
> > > 
> > >  - Downloaded the binaries and validated signatures.
> > >  - Deployed the binaries, ran some sample topologies and checked
> the
> > > UI.
> > >  - Ran top level build using the source zip.
> > > 
> > >  Thanks,
> > >  Arun
> > > 
> > > 
> > >  On Tue, 8 Jan 2019 at 11:03, P. Taylor Goetz 
> > > >> wrote:
> > > 
> > > > This is a call to vote on releasing Apache Storm 2.0.0 (rc4)
> > > >
> > > > Full list of changes in this release:
> > > >
> > > >
> > > >
> > > 
> > > >>>
> > > >>
> > >
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc4/RELEASE_NOTES.html
> > > >
> > > > The tag/commit to be voted upon is v2.0.0:
> > > >
> > > >
> > > >
> > > 
> > > >>>
> > > >>
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=1eece73e8c9ed7f41d2f20f727bc7f644c499360;hb=ddee8decac57d1a4a0aa23cc76066609a2abc8d2
> > > >
> > > > The source archive being voted upon can be found here:
> > > >
> > > >
> > > >
> > > 
> > > >>>
> > > >>
> > >
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc4/apache-storm-2.0.0-src.tar.gz
> > > >
> > > > Other release files, signatures and digests can be found here:
> > > >
> > > >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.0.0-rc4/
> > > >
> > > > The release artifacts are signed with the following key:
> > > >
> > > >
> > > >
> > > 
> > > >>>
> > > >>
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> > > >
> > > > The Nexus staging repository for this release is:
> > > >
> > > >
> > > >>
> > 

Re: [VOTE] Migration of git repos to gitbox.apache.org

[Govind Menon]

+1

On Fri, 18 Jan 2019 at 13:33, Stig Rohde Døssing 
wrote:

> +1
>
> Den fre. 18. jan. 2019 kl. 16.59 skrev Kishorkumar Patil
> :
>
> > +1
> >
> > On Fri, Jan 18, 2019 at 10:47 AM Ethan Li 
> > wrote:
> >
> > > This is a call to vote on migration of git repos to gitbox.apache.org
> <
> > > http://gitbox.apache.org/>.
> > >
> > > The following repositories on git-wip-us belong to our project:
> > > - storm-site.git
> > > - storm.git
> > >
> > > All repositories not migrated on February 7th will be mass migrated
> > > without warning. It’s better to work with the infrastructure team to
> > avoid
> > > the mass that day.
> > >
> > > The notice email is at
> > >
> >
> https://lists.apache.org/thread.html/2bb953946ee6652a054329df2814f512f9186e863ec841555f9cc26a@%3Cdev.storm.apache.org%3E
> > > <
> > >
> >
> https://lists.apache.org/thread.html/2bb953946ee6652a054329df2814f512f9186e863ec841555f9cc26a@%3Cdev.storm.apache.org%3E
> > > >
> > >
> > > This vote will be open for at least 24 hours.
> > >
> > > Thanks
> > >
> > > - Ethan
> >
> > --
> > -Kishor
> >
>

Re: [VOTE] Migration of git repos to gitbox.apache.org

[Stig Rohde Døssing]

+1

Den fre. 18. jan. 2019 kl. 16.59 skrev Kishorkumar Patil
:

> +1
>
> On Fri, Jan 18, 2019 at 10:47 AM Ethan Li 
> wrote:
>
> > This is a call to vote on migration of git repos to gitbox.apache.org <
> > http://gitbox.apache.org/>.
> >
> > The following repositories on git-wip-us belong to our project:
> > - storm-site.git
> > - storm.git
> >
> > All repositories not migrated on February 7th will be mass migrated
> > without warning. It’s better to work with the infrastructure team to
> avoid
> > the mass that day.
> >
> > The notice email is at
> >
> https://lists.apache.org/thread.html/2bb953946ee6652a054329df2814f512f9186e863ec841555f9cc26a@%3Cdev.storm.apache.org%3E
> > <
> >
> https://lists.apache.org/thread.html/2bb953946ee6652a054329df2814f512f9186e863ec841555f9cc26a@%3Cdev.storm.apache.org%3E
> > >
> >
> > This vote will be open for at least 24 hours.
> >
> > Thanks
> >
> > - Ethan
>
> --
> -Kishor
>

Re: [VOTE] Migration of git repos to gitbox.apache.org

[Kishorkumar Patil]

+1

On Fri, Jan 18, 2019 at 10:47 AM Ethan Li  wrote:

> This is a call to vote on migration of git repos to gitbox.apache.org <
> http://gitbox.apache.org/>.
>
> The following repositories on git-wip-us belong to our project:
> - storm-site.git
> - storm.git
>
> All repositories not migrated on February 7th will be mass migrated
> without warning. It’s better to work with the infrastructure team to avoid
> the mass that day.
>
> The notice email is at
> https://lists.apache.org/thread.html/2bb953946ee6652a054329df2814f512f9186e863ec841555f9cc26a@%3Cdev.storm.apache.org%3E
> <
> https://lists.apache.org/thread.html/2bb953946ee6652a054329df2814f512f9186e863ec841555f9cc26a@%3Cdev.storm.apache.org%3E
> >
>
> This vote will be open for at least 24 hours.
>
> Thanks
>
> - Ethan

-- 
-Kishor

Re: [VOTE] Migration of git repos to gitbox.apache.org

[Ethan Li]

I am +1 on this

> On Jan 18, 2019, at 9:46 AM, Ethan Li  wrote:
> 
> This is a call to vote on migration of git repos to gitbox.apache.org 
> .
> 
> The following repositories on git-wip-us belong to our project:
> - storm-site.git
> - storm.git
> 
> All repositories not migrated on February 7th will be mass migrated without 
> warning. It’s better to work with the infrastructure team to avoid the mass 
> that day.
> 
> The notice email is at 
> https://lists.apache.org/thread.html/2bb953946ee6652a054329df2814f512f9186e863ec841555f9cc26a@%3Cdev.storm.apache.org%3E
>  
> 
> 
> This vote will be open for at least 24 hours.
> 
> Thanks 
> 
> - Ethan

Re: [VOTE] Migration of git repos to gitbox.apache.org

[Derek Dagit]

+1 Thank you for looking into this.

On Fri, Jan 18, 2019 at 9:45 AM Ethan Li  wrote:

> This is a call to vote on migration of git repos to gitbox.apache.org <
> http://gitbox.apache.org/>.
>
> The following repositories on git-wip-us belong to our project:
> - storm-site.git
> - storm.git
>
> All repositories not migrated on February 7th will be mass migrated
> without warning. It’s better to work with the infrastructure team to avoid
> the mass that day.
>
> The notice email is at
> https://lists.apache.org/thread.html/2bb953946ee6652a054329df2814f512f9186e863ec841555f9cc26a@%3Cdev.storm.apache.org%3E
> <
> https://lists.apache.org/thread.html/2bb953946ee6652a054329df2814f512f9186e863ec841555f9cc26a@%3Cdev.storm.apache.org%3E
> >
>
> This vote will be open for at least 24 hours.
>
> Thanks
>
> - Ethan



-- 
Derek

[VOTE] Migration of git repos to gitbox.apache.org

[Ethan Li]

This is a call to vote on migration of git repos to gitbox.apache.org 
.

The following repositories on git-wip-us belong to our project:
- storm-site.git
- storm.git

All repositories not migrated on February 7th will be mass migrated without 
warning. It’s better to work with the infrastructure team to avoid the mass 
that day.

The notice email is at 
https://lists.apache.org/thread.html/2bb953946ee6652a054329df2814f512f9186e863ec841555f9cc26a@%3Cdev.storm.apache.org%3E
 


This vote will be open for at least 24 hours.

Thanks 

- Ethan

Re: [NOTICE] Mandatory migration of git repos to gitbox.apache.org - three weeks left!

[Ethan Li]

Before I create a jira, I will start a consensus thread first.

Thanks,
-Ethan


> On Jan 17, 2019, at 2:51 PM, Ethan Li  wrote:
> 
> If there is no objections, I will create a lira at 
> https://issues.apache.org/jira/browse/INFRA 
>  Thanks
> 
> 
> -Ethan
> 
> 
>> On Jan 15, 2019, at 10:43 AM, Ethan Li > > wrote:
>> 
>> Not that I’m aware of.  Only this 
>> https://github.com/apache/storm/blob/master/DEVELOPER.md#merge-a-pull-request-or-patch
>>  
>> 
>>  needs to be updated after I guess.
>> 
>>> On Jan 15, 2019, at 10:39 AM, Stig Rohde Døssing >> > wrote:
>>> 
>>> +1. As far as I know we don't have anything in our setup we need to change
>>> manually?
>>> 
>>> Den tir. 15. jan. 2019 kl. 17.35 skrev Ethan Li >> >:
>>> 
 Hi team,
 
 Should we go ahead to have our repositories moved?
 
 - Ethan
 
 
> On Jan 15, 2019, at 1:50 AM, Apache Infrastructure Team <
 infrastruct...@apache.org > wrote:
> 
> Hello, storm folks.
> As stated earlier in 2018, and reiterated two weeks ago, all git
> repositories must be migrated from the git-wip-us.apache.org 
>  URL to
> gitbox.apache.org , as the old service is 
> being decommissioned. Your
> project is receiving this email because you still have repositories on
> git-wip-us that needs to be migrated.
> 
> The following repositories on git-wip-us belong to your project:
> - storm-site.git
> - storm.git
> 
> 
> We are now entering the remaining three weeks of the mandated
> (coordinated) move stage of the roadmap, and you are asked to please
> coordinate migration with the Apache Infrastructure Team before February
> 7th. All repositories not migrated on February 7th will be mass migrated
> without warning, and we'd appreciate it if we could work together to
> avoid a big mess that day :-).
> 
> As stated earlier, moving to gitbox means you will get full write access
> on GitHub as well, and be able to close/merge pull requests and much
> more. The move is mandatory for all Apache projects using git.
> 
> To have your repositories moved, please follow these steps:
> 
> - Ensure consensus on the move (a link to a lists.apache.org 
>  thread will
> suffice for us as evidence).
> - Create a JIRA ticket at https://issues.apache.org/jira/browse/INFRA 
> 
> 
> Your migration should only take a few minutes. If you wish to migrate
> at a specific time of day or date, please do let us know in the ticket,
> otherwise we will migrate at the earliest convenient time.
> 
> There will be redirects in place from git-wip to gitbox, so requests
> using the old remote origins should still work (however we encourage
> people to update their remotes once migration has completed).
> 
> As always, we appreciate your understanding and patience as we move
> things around and work to provide better services and features for
> the Apache Family.
> 
> Should you wish to contact us with feedback or questions, please do so
> at: us...@infra.apache.org .
> 
> 
> With regards,
> Apache Infrastructure
> 
 
 
>> 
> 

[GitHub] storm issue #2941: STORM-3318: Complete information in Class NewKafkaSpoutOf...

[MichealShin]

Github user MichealShin commented on the issue:

https://github.com/apache/storm/pull/2941
  
@srdo OK


---

[GitHub] storm issue #2941: STORM-3318: Complete information in Class NewKafkaSpoutOf...

[srdo]

Github user srdo commented on the issue:

https://github.com/apache/storm/pull/2941
  
+1. Please squash, and we can merge once the waiting period is over. I'll 
probably wait to merge this until we are done with the current 2.0.0 RC vote.


---

[GitHub] storm pull request #2941: STORM-3318: Complete information in Class NewKafka...

[srdo]

Github user srdo commented on a diff in the pull request:

https://github.com/apache/storm/pull/2941#discussion_r249026503
  
--- Diff: 
external/storm-kafka-monitor/src/main/java/org/apache/storm/kafka/monitor/NewKafkaSpoutOffsetQuery.java
 ---
@@ -64,27 +64,51 @@ public String toString() {
"topics='" + topics + '\'' +
", consumerGroupId='" + consumerGroupId + '\'' +
", bootStrapBrokers='" + bootStrapBrokers + '\'' +
+   ", securityProtocol='" + securityProtocol + '\'' +
+   ", consumerPropertiesFileName='" + 
consumerPropertiesFileName + '\'' +
'}';
 }
 
 @Override
 public boolean equals(Object o) {
-if (this == o) return true;
-if (o == null || getClass() != o.getClass()) return false;
+if (this == o) {
+return true;
+}
+if (o == null || getClass() != o.getClass()) {
+return false;
+}
 
 NewKafkaSpoutOffsetQuery that = (NewKafkaSpoutOffsetQuery) o;
 
-if (topics != null ? !topics.equals(that.topics) : that.topics != 
null) return false;
-if (consumerGroupId != null ? 
!consumerGroupId.equals(that.consumerGroupId) : that.consumerGroupId != null) 
return false;
-return !(bootStrapBrokers != null ? 
!bootStrapBrokers.equals(that.bootStrapBrokers) : that.bootStrapBrokers != 
null);
-
+if (topics != null ? !topics.equals(that.topics) : that.topics != 
null) {
+return false;
+}
+if (consumerGroupId != null ? 
!consumerGroupId.equals(that.consumerGroupId)
+: that.consumerGroupId != null) {
+return false;
+}
+if (bootStrapBrokers != null ? 
!bootStrapBrokers.equals(that.bootStrapBrokers)
+ : that.bootStrapBrokers != null) {
+return false;
+}
+if (securityProtocol != null ? 
!securityProtocol.equals(that.securityProtocol)
+ : that.securityProtocol != null) {
+return false;
+}
+return consumerPropertiesFileName != null ? 
consumerPropertiesFileName
+.equals(that.consumerPropertiesFileName) : 
that.consumerPropertiesFileName == null;
 }
 
 @Override
 public int hashCode() {
 int result = topics != null ? topics.hashCode() : 0;
 result = 31 * result + (consumerGroupId != null ? 
consumerGroupId.hashCode() : 0);
 result = 31 * result + (bootStrapBrokers != null ? 
bootStrapBrokers.hashCode() : 0);
+result = 31 * result + (securityProtocol != null ? 
securityProtocol.hashCode() : 0);
+result =
+31 * result + (consumerPropertiesFileName != null ? 
consumerPropertiesFileName
+.hashCode()
+  : 0);
--- End diff --

The formatting here is off. 


---

[GitHub] storm issue #2940: STORM-3318: Complete information in Class NewKafkaSpoutOf...

[srdo]

Github user srdo commented on the issue:

https://github.com/apache/storm/pull/2940
  
@MichealShin You don't need to open a new PR. If you push your changes to 
the branch this PR is pointing at, the PR will get updated automatically. 

That said, I think Github gets weird about reopening PRs once they're 
closed and the source branch has changed, so let's use the new PR. But just for 
future reference :)


---

[GitHub] storm issue #2940: STORM-3318: Complete information in Class NewKafkaSpoutOf...

[MichealShin]

Github user MichealShin commented on the issue:

https://github.com/apache/storm/pull/2940
  
@srdo Done. See: https://github.com/apache/storm/pull/2941


---

[GitHub] storm pull request #2940: STORM-3318: Complete information in Class NewKafka...

[MichealShin]

Github user MichealShin closed the pull request at:

https://github.com/apache/storm/pull/2940


---

[GitHub] storm pull request #2941: STORM-3318: Complete information in Class NewKafka...

[MichealShin]

GitHub user MichealShin opened a pull request:

https://github.com/apache/storm/pull/2941

STORM-3318: Complete information in Class NewKafkaSpoutOffsetQuery

Just complete information in three methods(toString , equals, hashCode).

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/MichealShin/storm master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/2941.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2941


commit 3eba918afdc8dbc777e790d71be9dc30d17f125d
Author: MichealShin 
Date:   2019-01-18T12:21:49Z

STORM-3318: Complete information in Class NewKafkaSpoutOffsetQuery

Just complete information in three methods(toString , equals, hashCode).




---

[GitHub] storm pull request #2940: STORM-3318: Complete information in Class NewKafka...

[srdo]

Github user srdo commented on a diff in the pull request:

https://github.com/apache/storm/pull/2940#discussion_r248985594
  
--- Diff: 
external/storm-kafka-monitor/src/main/java/org/apache/storm/kafka/monitor/NewKafkaSpoutOffsetQuery.java
 ---
@@ -75,8 +77,13 @@ public boolean equals(Object o) {
 NewKafkaSpoutOffsetQuery that = (NewKafkaSpoutOffsetQuery) o;
 
 if (topics != null ? !topics.equals(that.topics) : that.topics != 
null) return false;
-if (consumerGroupId != null ? 
!consumerGroupId.equals(that.consumerGroupId) : that.consumerGroupId != null) 
return false;
-return !(bootStrapBrokers != null ? 
!bootStrapBrokers.equals(that.bootStrapBrokers) : that.bootStrapBrokers != 
null);
+if (consumerGroupId != null ? 
!consumerGroupId.equals(that.consumerGroupId) : that.consumerGroupId != null)
--- End diff --

I realize this was an issue before as well, but please add braces.


---

[GitHub] storm issue #2940: STORM-3318: Complete information in Class NewKafkaSpoutOf...

[srdo]

Github user srdo commented on the issue:

https://github.com/apache/storm/pull/2940
  
Looks good. Please squash to one commit.


---