[NOTICE] Migration of storm repos to gitbox is done

2019-01-20 Thread Ethan Li 
Hi Team,

The migration to https://gitbox.apache.org/ is done 
(https://issues.apache.org/jira/browse/INFRA-17705 
). 

Thanks,

- Ethan

Re: [VOTE] Migration of git repos to gitbox.apache.org

2019-01-20 Thread Ethan Li 
With six +1 and zero -1 the vote passes. 

I will create a JIRA ticket at https://issues.apache.org/jira/browse/INFRA 


Thanks

-Ethan

> On Jan 20, 2019, at 12:45 AM, Jungtaek Lim  wrote:
> 
> Thanks Ethan for leading the effort on migration! +1 from me.
> 
> Thanks,
> Jungtaek Lim (HeartSaVioR)
> 
> 2019년 1월 19일 (토) 오전 4:48, Govind Menon 님이 작성:
> 
>> +1
>> 
>> On Fri, 18 Jan 2019 at 13:33, Stig Rohde Døssing 
>> wrote:
>> 
>>> +1
>>> 
>>> Den fre. 18. jan. 2019 kl. 16.59 skrev Kishorkumar Patil
>>> :
>>> 
 +1
 
 On Fri, Jan 18, 2019 at 10:47 AM Ethan Li 
 wrote:
 
> This is a call to vote on migration of git repos to
>> gitbox.apache.org
>>> <
> http://gitbox.apache.org/>.
> 
> The following repositories on git-wip-us belong to our project:
> - storm-site.git
> - storm.git
> 
> All repositories not migrated on February 7th will be mass migrated
> without warning. It’s better to work with the infrastructure team to
 avoid
> the mass that day.
> 
> The notice email is at
> 
 
>>> 
>> https://lists.apache.org/thread.html/2bb953946ee6652a054329df2814f512f9186e863ec841555f9cc26a@%3Cdev.storm.apache.org%3E
> <
> 
 
>>> 
>> https://lists.apache.org/thread.html/2bb953946ee6652a054329df2814f512f9186e863ec841555f9cc26a@%3Cdev.storm.apache.org%3E
>> 
> 
> This vote will be open for at least 24 hours.
> 
> Thanks
> 
> - Ethan
 
 --
 -Kishor
 
>>> 
>>

Re: [VOTE] Release Apache Storm 1.1.4 (rc1)

2019-01-20 Thread Jungtaek Lim 
+1 (binding)

> source

- verify file (MD5, SHA)
-- source, tar.gz : OK
-- source, zip : OK

- extract file
-- source, tar.gz : OK
-- source, zip : OK

- diff-ing extracted files between tar.gz and zip : OK

- build source with JDK 8
-- source, tar.gz : OK

- build source dist
-- source, tar.gz : OK

- build binary dist
-- source, tar.gz : OK

> binary

- verify file (MD5, SHA)
-- binary, tar.gz : OK
-- binary, zip : OK

- extract file
-- binary, tar.gz : OK
-- binary, zip : OK

- diff-ing extracted files between tar.gz and zip : OK

- launch daemons : OK

- run ExclamationTopology (local) : OK

- run RollingTopWords (remote) : OK
  - activate / deactivate / rebalance / kill : OK
  - logviewer (worker dir, daemon dir) : OK
  - change log level : OK
  - thread dump, heap dump, restart worker : OK
  - log search : OK

Thanks,
Jungtaek Lim (HeartSaVioR)

2019년 1월 10일 (목) 오전 8:06, Ethan Li 님이 작성:

> +1
>
> - Built from the src, ran all the unit tests and integration tests.
> - Set up a single-node cluster and submit a word count topology.
> - Checked the UI.
> They look good.
>
> Thanks
> Ethan
>
>
> > On Jan 9, 2019, at 12:37 PM, Bobby Evans  wrote:
> >
> > +1 I built the code from the git tag and all the unit tests passed.  I
> also
> > ran some manual tests.
> >
> > Thanks,
> >
> > Bobby
> >
> > On Wed, Jan 9, 2019 at 11:01 AM P. Taylor Goetz 
> wrote:
> >
> >> This is a call to vote on releasing Apache Storm 1.1.4 (rc1)
> >>
> >> Full list of changes in this release:
> >>
> >>
> >>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.1.4-rc1/RELEASE_NOTES.html
> >>
> >> The tag/commit to be voted upon is v1.1.4:
> >>
> >>
> >>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=dfb1f971c35ca985a35692aed6d28be3d7e707de;hb=babab9648a6deb3d444c45bdd6d0ec30804e91f1
> >>
> >> The source archive being voted upon can be found here:
> >>
> >>
> >>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.1.4-rc1/apache-storm-1.1.4-src.tar.gz
> >>
> >> Other release files, signatures and digests can be found here:
> >>
> >> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.1.4-rc1/
> >>
> >> The release artifacts are signed with the following key:
> >>
> >>
> >>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >>
> >> The Nexus staging repository for this release is:
> >>
> >> https://repository.apache.org/content/repositories/orgapachestorm-1075
> >>
> >> Please vote on releasing this package as Apache Storm 1.1.4.
> >>
> >> When voting, please list the actions taken to verify the release.
> >>
> >> This vote will be open for at least 72 hours.
> >>
> >> [ ] +1 Release this package as Apache Storm 1.1.4
> >> [ ]  0 No opinion
> >> [ ] -1 Do not release this package because...
> >>
> >> Thanks to everyone who contributed to this release.
> >>
> >> -Taylor
>
>

Re: [VOTE] Release Apache Storm 2.0.0 (rc4)

2019-01-20 Thread Roshan Naik 
 +1
Sorry .. it was a false alarm. It come down to a config issue. The default 
configs in Storm 2 are tuned for best latency. 
On bumping up the transfer buffer & batch settings for high throughput :
- topology.transfer.batch.size=5k (default=1)  and - 
topology.transfer.buffer.size=20k (default=1k)
Was able to obtain a very solid  ~4 mill/sec inter worker throughput. 
FYI: Does not matter now, but i noticed that I accidentally quoted throughput 
of 1.6mill/sec but it was actually only ~350 k/sec with default transfer buffer 
settings  .. which was alarmingly low. 
On Saturday, January 19, 2019, 10:45:28 PM PST, Jungtaek Lim 
 wrote:  
 
 Hello all,

+1 (binding)

> source

- verify file (MD5, SHA)
-- source, tar.gz : OK
-- source, zip : OK

- extract file
-- source, tar.gz : OK
-- source, zip : OK

- diff-ing extracted files between tar.gz and zip : OK

- build source with JDK 8 (-Pall-tests && -Pexternals)
-- source, tar.gz : OK

- build source dist
-- source, tar.gz : OK

- build binary dist
-- source, tar.gz : OK

> binary

- verify file (MD5, SHA)
-- binary, tar.gz : OK
-- binary, zip : OK

- extract file
-- binary, tar.gz : OK
-- binary, zip : OK

- diff-ing extracted files between tar.gz and zip : OK

- launch daemons : OK

- run ExclamationTopology (local) : OK

- run RollingTopWords (remote) : OK
  - activate / deactivate / rebalance / kill : OK
  - logviewer (worker dir, daemon dir) : OK
  - change log level : OK
  - thread dump, heap dump, restart worker : OK
  - log search : OK

I don't think perf. degrade is a blocker for this release, since we are
still providing far better performance then 1.x. As Roshan is planning to
address this, that could be fixed in 2.0.x hopefully.

Thanks,
Jungtaek Lim (heartSaVioR)

2019년 1월 19일 (토) 오전 5:23, Roshan Naik 님이 작성:

>  Was verifying the performance on 2.0 and noticed that the inter worker
> messaging has fallen off a cliff  With ConstSpoutNullBoltTopo (producer
> batch size=1k, ackers=0, workers=2) it was ~3.2mill/sec down  and now its
> down to  ~1.6mill/sec. Something has impacted this recently. Will try to
> narrow down the issue by tomorrow hopefully. Single worker numbers look
> good.
> -roshan
>    On Friday, January 11, 2019, 2:46:14 AM PST, Stig Rohde Døssing <
> stigdoess...@gmail.com> wrote:
>
>  Thanks Arun,
>
> I don't have an opinion on it, just wanted to make sure that it wasn't an
> oversight.
>
> +1
>
> Set up Storm from the binary zip file
> Built storm-kafka-client-examples from the zip while pointing Maven at the
> Nexus repo
> Ran one of the example Kafka topologies with a single-node cluster against
> a local Kafka. Verified there were no errors in the logs, clicked around in
> UI.
>
> Den tor. 10. jan. 2019 kl. 22.25 skrev Arun Mahadevan :
>
> > This is for users to use the "auto credentials" mechanism (delegation
> > tokens) with HDFS/Hive/Hbase.
> >
> > We have been shipping it since 1.x (I think since 1.2.0 release) so that
> > users can just add that directory to class path rather than building it
> > separately to get the right dependencies. We could consider removing it
> > from the main binary and ship it separately but it will need changes to
> the
> > build, release and documentation and users will need to download and
> > install it separately.
> >
> >
> > Thanks,
> > Arun
> >
> > On Thu, 10 Jan 2019 at 10:28, Stig Rohde Døssing  >
> > wrote:
> >
> > > I think this was remarked on by Roshan in the last RC, but the binary
> > > distribution has become significantly larger since 1.x. It looks like
> > this
> > > is down to storm-autocreds not being added to the exclusion list in
> > > storm-dist/binary/final-package/src/main/assembly/binary.xml.
> > >
> > > Since the module isn't excluded, external/storm-autocreds contains the
> > > module jar, plus all dependency jars. Is this an accident, or do we
> want
> > to
> > > include these jars in the distribution?
> > >
> > > Den ons. 9. jan. 2019 kl. 19.48 skrev Ethan Li <
> > ethanopensou...@gmail.com
> > > >:
> > >
> > > > +1
> > > >
> > > > - Built from the src, ran all the unit tests and integration tests.
> > > > - Set up a single-node cluster and submit ThroughputVsLatency
> topology.
> > > > - Checked the UI.
> > > > They look good.
> > > >
> > > > Thanks
> > > > Ethan
> > > >
> > > > > On Jan 9, 2019, at 8:48 AM, Bobby Evans  wrote:
> > > > >
> > > > > +1 built from the git tag.  Ran all of the unit tests and ran some
> > > manual
> > > > > tests they all passed.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Bobby
> > > > >
> > > > > On Tue, Jan 8, 2019 at 6:30 PM Xin Wang 
> > > wrote:
> > > > >
> > > > >> +1
> > > > >>
> > > > >> Built it and ran all of the tests.  Everything passed.
> > > > >>
> > > > >> -Xin
> > > > >>
> > > > >> Kishorkumar Patil  于2019年1月9日周三
> 上午5:08写道：
> > > > >>
> > > > >>> +1
> > > > >>>
> > > > >>> - built from source code and deployment works.
> > > > >>> -  Ran some of the tests for UI, DRPC, ThroughputVsLatency
> > > > >>> - 

Re: [VOTE] Release Apache Storm 1.2.3 (rc1)

2019-01-20 Thread Jungtaek Lim 
+1 (binding)

> source

- verify file (MD5, SHA)
-- source, tar.gz : OK
-- source, zip : OK

- extract file
-- source, tar.gz : OK
-- source, zip : OK

- diff-ing extracted files between tar.gz and zip : OK

- build source with JDK 8
-- source, tar.gz : OK

- build source dist
-- source, tar.gz : OK

- build binary dist
-- source, tar.gz : OK

> binary

- verify file (MD5, SHA)
-- binary, tar.gz : OK
-- binary, zip : OK

- extract file
-- binary, tar.gz : OK
-- binary, zip : OK

- diff-ing extracted files between tar.gz and zip : OK

- launch daemons : OK

- run ExclamationTopology (local) : OK

- run RollingTopWords (remote) :
  - activate / deactivate / rebalance / kill : OK
  - logviewer (worker dir, daemon dir) : OK
  - change log level : OK
  - thread dump, heap dump, restart worker : OK
  - log search : OK

Thanks,
Jungtaek Lim (HeartSaVioR)

2019년 1월 15일 (화) 오후 4:55, Alexandre Vermeerbergen 님이
작성:

> Hello Taylor,
>
> +1
>
> DownloadedStorm1.2.3 RC1 + build 1.2.3 RC1 from sources to get up to
> date storm-kafka-client
>
> Upgraded by integration cluster with 1.2.3 RC.
> Configuration:
> - CentOS7, Java 11 from AdoptOpenJDK11.0.1 with Hotspot  on all VMs
> - 1 Nimbus VM, 7 Supervisor,VMs, 3 Zookeeper VMs, 19 topologies with
> high dependency on Storm
> Kafka Client, using Kafka Client 2.0.1 connected to a distinct Kafka
> Brokers Cluster (5 VMs) based on Kafka 1.1.1 (this later one still
> runs with Java 8).
>
> Kind regards,
> Alexandre
>
> Le mar. 8 janv. 2019 à 21:20, P. Taylor Goetz  a écrit
> :
> >
> > This is a call to vote on releasing Apache Storm 1.2.3 (rc1)
> >
> > Full list of changes in this release:
> >
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/RELEASE_NOTES.html
> >
> > The tag/commit to be voted upon is v1.2.3:
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=5eaaa51591f2e4dc3e31f22bcc581af4a1f39c03;hb=6ba98b215857656e0186887b5d1a6a5aceee10c4
> >
> > The source archive being voted upon can be found here:
> >
> >
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/apache-storm-1.2.3-src.tar.gz
> >
> > Other release files, signatures and digests can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.3-rc1/
> >
> > The release artifacts are signed with the following key:
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >
> > The Nexus staging repository for this release is:
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1074
> >
> > Please vote on releasing this package as Apache Storm 1.2.3.
> >
> > When voting, please list the actions taken to verify the release.
> >
> > This vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache Storm 1.2.3
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > -Taylor
>