7/28/2020 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi~

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
- Code reviews and discussions
- Test new release candidate

Regards,
--ning

Re: [VOTE] Heron Release 0.20.3-incubating Release Candidate 6

[Ning Wang]

I am afraid we may need to reprepare it. :(

-1

My tests:
- build ok on Ubuntu.
- tracker/ui and test topology work ok on ubuntu.
- license scan returns some errors. A few new files are missing license
header. Also a new font was added. The issues should be fixed in this PR:
https://github.com/apache/incubator-heron/pull/3590



On Mon, Jul 27, 2020 at 3:27 AM H W  wrote:

> Added the description in the github release page. Thanks @ning for
> reminding me.
>
> On Mon, Jul 27, 2020 at 2:21 AM Ning Wang  wrote:
>
> > Seems like rc6 is not created in
> > https://github.com/apache/incubator-heron/releases yet.
> >
> > I am going to test the build.
> >
> > On Sun, Jul 26, 2020 at 10:23 PM thinker0  wrote:
> >
> > >  +1 approve
> > >
> > > 2020년 7월 27일 (월) 오전 10:10, H W 님이 작성:
> > >
> > > > Hello Heron Community,
> > > >
> > > > This is a call for a vote to the 6th release candidate for Apache
> > Heron,
> > > > version 0.20.3-incubating.
> > > > We request project mentors (binded) as well as all contributors
> > > (unbinded)
> > > > and users to review and vote on this incubator release.
> > > >
> > > > *  The tag to be voted upon: 0.20.3-incubating-rc6
> > > > (0c9b209467f2d80390c605a363829134753da544)
> > > > The full list of changes and release notes are available at:
> > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-heron/releases/tag/0.20.3-incubating-rc6
> > > >
> > > > Files can be found in dist.apache.org
> > > > Source:
> > > >
> > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.3-incubating-candidate-6/
> > > > SHA-512
> > > > <
> > >
> >
> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.3-incubating-candidate-6/SHA-512
> > > >
> > > > checksums:
> > > >
> > > >
> > >
> >
> 88eda83fc223e9a7a1d6087f9a5440012681decff18fefddc52a49579ff0f1e2c653918a490635013e1710337d57535d459d366676a19abce7cf972bf77bb6dd
> > > >  0.20.3-incubating-rc6.tar.gz
> > > >
> > > >
> > > > Please download the source package, and follow the compiling guide
> > > > https://heron.incubator.apache.org/docs/next/compiling-overview/
> > > > to setup the environment, build and run locally.
> > > >
> > > > Note that currently Bazel 3.4.1 is required to build this version.
> > > > After Bazel is set up and works correctly, you can use these release
> > > check
> > > > scripts
> > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-heron/blob/master/scripts/release_check/README.md#run-individual-release-checks
> > > > to verify the build.  The vote will be open for at least 72 hours or
> > > until
> > > > the necessary number of votes are reached.
> > > >
> > > > Please vote accordingly:
> > > > [ ] +1 approve
> > > > [ ] +0 no opinion
> > > > [ ] -1 disapprove with the reason
> > > >
> > > > Thank you,
> > > >
> > > > The Apache Heron (Incubating) Team
> > > >
> > >
> >
>

7/14/2020 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi~

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
- Code reviews and discussions
- I may start to prepare for the new release soon, as I might be a little
less busy in these 2 weeks.

Regards,
--ning

6/30/2020 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi~

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates (not much):
- Code reviews and discussions
- Question: should we remove the java doc target from the IDE integration?
It is the blocker for a new release candidate. Thanks in advance!

Regards,
--ning

6/16 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi~

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
- Prepare release. We have one blocker left: the IDE support for the
java-doc target.
- Code reviews.

Regards,
--ning

Re: Heron Website Layout

[Ning Wang]

Cool thanks!

On Sun, Jun 14, 2020 at 7:58 PM Windham Wong 
wrote:

> Removing the CSS of mainContainer position:fixed; can fix the issue.
>
> *Windham Wong*
> Managing Partner, Technical Director of
> *Stormeye.io, Hong Kong Managed Security Operation Center Limited*
>
> 
> Email // windham.w...@stormeye.io
> 
> Phone // +852_3590_2212_|_+852_9832_0707 <+85235902212>
> Fax // +852_3590_2202
> On 6月 15 2020, at 10:37 上午, Josh Fischer  wrote:
>
> Hi All,
>
> It looks like the website layout is all goofed up somehow. I don't see any
> changes that could cause this to happen. Does anyone know what the issue
> is?
>
> - Josh
>
> [image: Sent from Mailspring]

Re: One more thing to fix before next release candidate

[Ning Wang]

class
> >
> > backtype/storm/utils/DefaultMaxSpoutPendingTuner.class
> >
> > backtype/storm/utils/ListDelegate.class
> >
> > backtype/storm/utils/Utils.class
> >
> > clojure/
> >
> > clojure/lang/
> >
> > clojure/lang/Atom.class
> >
> > org/
> >
> > org/apache/
> >
> > org/apache/storm/
> >
> > org/apache/storm/Config.class
> >
> > org/apache/storm/Constants.class
> >
> > org/apache/storm/ILocalCluster.class
> >
> > org/apache/storm/LocalCluster.class
> >
> > org/apache/storm/StormSubmitter.class
> >
> > org/apache/storm/topology/
> >
> > org/apache/storm/topology/BasicBoltExecutor.class
> >
> > org/apache/storm/topology/BasicOutputCollector.class
> >
> > org/apache/storm/topology/BoltDeclarer.class
> >
> > org/apache/storm/topology/BoltDeclarerImpl.class
> >
> > org/apache/storm/trident/spout/
> >
> > org/apache/storm/trident/spout/ISpoutPart
> >
> >
> >
> >
> > output from the java_doc rule zip is below.  If you look below the
> > packages "backtype", "storm", and "clojure" are under the "java" folder
> > which is incorrect.  They should be structured like the files under the
> > "org" folder.
> >
> > heron-storm-javadoc.zip.dir joshfischer$ tree
> >
> > .
> >
> > ├── allclasses-index.html
> >
> > ├── allclasses.html
> >
> > ├── allpackages-index.html
> >
> > ├── constant-values.html
> >
> > ├── deprecated-list.html
> >
> > ├── element-list
> >
> > ├── help-doc.html
> >
> > ├── index-all.html
> >
> > ├── index.html
> >
> > ├── java
> >
> > │   ├── backtype
> >
> > │   │   └── storm
> >
> > │   │   ├── generated
> >
> > │   │   │   ├── package-summary.html
> >
> > │   │   │   └── package-tree.html
> >
> > │   │   ├── grouping
> >
> > │   │   │   ├── package-summary.html
> >
> > │   │   │   └── package-tree.html
> >
> > │   ├── clojure
> >
> > │   │   └── lang
> >
> > │   │   ├── package-summary.html
> >
> > │   │   └── package-tree.html
> >
> > │   └── storm
> >
> > │   └── trident
> >
> > │   ├── spout
> >
> > │   │   ├── package-summary.html
> >
> > │   │   └── package-tree.html
> >
> > │   └── state
> >
> > │   ├── package-summary.html
> >
> > │   └── package-tree.html
> >
> >
> > ├── org
> >
> > │   └── apache
> >
> > │   └── storm
> >
> > │   ├── Config.html
> >
> > │   ├── Constants.html
> >
> > │   ├── ILocalCluster.html
> >
> > │   ├── LocalCluster.html
> >
> > │   ├── StormSubmitter.html
> >
> > │   ├── clojure
> >
> > │   │   └── lang
> >
> > │   │   ├── Atom.html
> >
> > │   │   ├── package-summary.html
> >
> > │   │   └── package-tree.html
> >
> > On Wed, Jun 10, 2020 at 11:49 AM Ning Wang  wrote:
> >
> >> Thanks!
> >>
> >> On Wed, Jun 10, 2020 at 5:49 AM Josh Fischer 
> wrote:
> >>
> >> > Hi All,
> >> >
> >> > The javadoc issue should be fixed on both the website build and in the
> >> code
> >> > base build.  I believe we have one more issue to fix before we start
> the
> >> > next release candidate vote.  The last issue I can think of is fixing
> >> the
> >> > IDE setup scripts. I have an idea of how to fix them. It seems that
> >> Bazel's
> >> > behavior has changed over time resulting in the setup scripts to fail
> >> while
> >> > doing a reverse lookup on targets when looking at a source file.  Once
> >> we
> >> > have this fixed I think we are good to go.
> >> >
> >> > Just a note, it might be helpful to note in the next vote to note in
> the
> >> > email that it takes anywhere from 30 - 45 minutes on a fresh build to
> >> > setup eclipse or intellij with Heron.  It might save some people some
> >> time
> >> > during the vote if they do not wish to wait that long to view the
> >> codebase
> >> > in an IDE.
> >> >
> >> > - Josh
> >> >
> >>
> >
>

Re: One more thing to fix before next release candidate

[Ning Wang]

e/storm/utils/DefaultMaxSpoutPendingTuner.class
> >
> > backtype/storm/utils/ListDelegate.class
> >
> > backtype/storm/utils/Utils.class
> >
> > clojure/
> >
> > clojure/lang/
> >
> > clojure/lang/Atom.class
> >
> > org/
> >
> > org/apache/
> >
> > org/apache/storm/
> >
> > org/apache/storm/Config.class
> >
> > org/apache/storm/Constants.class
> >
> > org/apache/storm/ILocalCluster.class
> >
> > org/apache/storm/LocalCluster.class
> >
> > org/apache/storm/StormSubmitter.class
> >
> > org/apache/storm/topology/
> >
> > org/apache/storm/topology/BasicBoltExecutor.class
> >
> > org/apache/storm/topology/BasicOutputCollector.class
> >
> > org/apache/storm/topology/BoltDeclarer.class
> >
> > org/apache/storm/topology/BoltDeclarerImpl.class
> >
> > org/apache/storm/trident/spout/
> >
> > org/apache/storm/trident/spout/ISpoutPart
> >
> >
> >
> >
> > output from the java_doc rule zip is below.  If you look below the
> > packages "backtype", "storm", and "clojure" are under the "java" folder
> > which is incorrect.  They should be structured like the files under the
> > "org" folder.
> >
> > heron-storm-javadoc.zip.dir joshfischer$ tree
> >
> > .
> >
> > ├── allclasses-index.html
> >
> > ├── allclasses.html
> >
> > ├── allpackages-index.html
> >
> > ├── constant-values.html
> >
> > ├── deprecated-list.html
> >
> > ├── element-list
> >
> > ├── help-doc.html
> >
> > ├── index-all.html
> >
> > ├── index.html
> >
> > ├── java
> >
> > │   ├── backtype
> >
> > │   │   └── storm
> >
> > │   │   ├── generated
> >
> > │   │   │   ├── package-summary.html
> >
> > │   │   │   └── package-tree.html
> >
> > │   │   ├── grouping
> >
> > │   │   │   ├── package-summary.html
> >
> > │   │   │   └── package-tree.html
> >
> > │   ├── clojure
> >
> > │   │   └── lang
> >
> > │   │   ├── package-summary.html
> >
> > │   │   └── package-tree.html
> >
> > │   └── storm
> >
> > │   └── trident
> >
> > │   ├── spout
> >
> > │   │   ├── package-summary.html
> >
> > │   │   └── package-tree.html
> >
> > │   └── state
> >
> > │   ├── package-summary.html
> >
> > │   └── package-tree.html
> >
> >
> > ├── org
> >
> > │   └── apache
> >
> > │   └── storm
> >
> > │   ├── Config.html
> >
> > │   ├── Constants.html
> >
> > │   ├── ILocalCluster.html
> >
> > │   ├── LocalCluster.html
> >
> > │   ├── StormSubmitter.html
> >
> > │   ├── clojure
> >
> > │   │   └── lang
> >
> > │   │   ├── Atom.html
> >
> > │   │   ├── package-summary.html
> >
> > │   │   └── package-tree.html
> >
> > On Wed, Jun 10, 2020 at 11:49 AM Ning Wang  wrote:
> >
> >> Thanks!
> >>
> >> On Wed, Jun 10, 2020 at 5:49 AM Josh Fischer 
> wrote:
> >>
> >> > Hi All,
> >> >
> >> > The javadoc issue should be fixed on both the website build and in the
> >> code
> >> > base build.  I believe we have one more issue to fix before we start
> the
> >> > next release candidate vote.  The last issue I can think of is fixing
> >> the
> >> > IDE setup scripts. I have an idea of how to fix them. It seems that
> >> Bazel's
> >> > behavior has changed over time resulting in the setup scripts to fail
> >> while
> >> > doing a reverse lookup on targets when looking at a source file.  Once
> >> we
> >> > have this fixed I think we are good to go.
> >> >
> >> > Just a note, it might be helpful to note in the next vote to note in
> the
> >> > email that it takes anywhere from 30 - 45 minutes on a fresh build to
> >> > setup eclipse or intellij with Heron.  It might save some people some
> >> time
> >> > during the vote if they do not wish to wait that long to view the
> >> codebase
> >> > in an IDE.
> >> >
> >> > - Josh
> >> >
> >>
> >
>

Re: One more thing to fix before next release candidate

[Ning Wang]

Thanks!

On Wed, Jun 10, 2020 at 5:49 AM Josh Fischer  wrote:

> Hi All,
>
> The javadoc issue should be fixed on both the website build and in the code
> base build.  I believe we have one more issue to fix before we start the
> next release candidate vote.  The last issue I can think of is fixing the
> IDE setup scripts. I have an idea of how to fix them. It seems that Bazel's
> behavior has changed over time resulting in the setup scripts to fail while
> doing a reverse lookup on targets when looking at a source file.  Once we
> have this fixed I think we are good to go.
>
> Just a note, it might be helpful to note in the next vote to note in the
> email that it takes anywhere from 30 - 45 minutes on a fresh build to
> setup eclipse or intellij with Heron.  It might save some people some time
> during the vote if they do not wish to wait that long to view the codebase
> in an IDE.
>
> - Josh
>

Re: [DISCUSSION] Docker OS support

[Ning Wang]

That is the one I am blocked on. Will need to do another full tests before
starting the vote.

On Fri, Jun 5, 2020 at 2:22 PM Josh Fischer  wrote:

> I just opened a PR that fixes javadoc generation for the website build.  Do
> we have anything else outstanding before we can start a vote on the next
> release candidate?
>
> On Wed, May 27, 2020 at 11:31 AM Ning Wang  wrote:
>
> > Yeah. We have chosen Debian 10 for the binary release and clean up the
> > license requirements for the first binary release.
> >
> > In the source level, we would also like to include a few
> platforms/versions
> > in the main source code to be "supported" and the others will be moved to
> > like a "extra" folder and not actively supported/tested.
> >
> >
> > On Wed, May 27, 2020 at 9:22 AM Dave Fisher  wrote:
> >
> > > I thought that the purpose of this discussion was to find the one or
> two
> > > platforms for Heron to support in order to keep down the complexity of
> > > making the first binary release for the community.
> > >
> > > Other platforms could be on a roadmap for future releases.
> > >
> > > Regards,
> > > Dave
> > >
> > > > On May 26, 2020, at 12:26 PM, Ning Wang 
> wrote:
> > > >
> > > > My concern about "latest" is that I feel most people hold it for some
> > > time
> > > > before migrating to the latest version (if it ever happens). If there
> > are
> > > > numbers, it could make sense to use the most popular version (plus
> the
> > > > other criterias).
> > > >
> > > > On Tue, May 26, 2020 at 12:14 PM Oliver Bristow <
> > > oli...@oliverbristow.co.uk>
> > > > wrote:
> > > >
> > > >> Sorry if I missed the rationale for Ubuntu 18.04 instead of 20.04,
> but
> > > >> would the latest LTS be a better choice? 18.04 was released before
> > > Debian
> > > >> 10, so that may be an argument for 20.04 re. compatibility if Debian
> > 10
> > > is
> > > >> the standard.
> > > >>
> > > >> There's already a 20.04 Dockerfile + I It looks like the vagrant
> setup
> > > will
> > > >> get an upgrade from 14.04 to 20.04 as a part of the python3 upgrade,
> > and
> > > >> seems to run fine so far.
> > > >>
> > > >> On Tue, 26 May 2020 at 19:30, Josh Fischer 
> > wrote:
> > > >>
> > > >>> Thanks for further clarifying. Yes, that is correct.
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Tue, May 26, 2020 at 1:16 PM Ning Wang 
> > > wrote:
> > > >>>
> > > >>>> +1.
> > > >>>>
> > > >>>> So my understanding is:
> > > >>>> binary (in future release): Debian 10.
> > > >>>> source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers
> will
> > be
> > > >>>> updated in the future after more discussions/votes.
> > > >>>>
> > > >>>> others will be moved to a "extra" folder only for reference.
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> On Tue, May 26, 2020 at 6:13 AM Josh Fischer  >
> > > >>> wrote:
> > > >>>>
> > > >>>>> Hi All,
> > > >>>>>
> > > >>>>> Based on what I've read on this thread a combination of
> > CentOS/Debian
> > > >>>> would
> > > >>>>> be the better options for us to support multiple versions for the
> > > >>>>> community.  We will also support one version of Ubuntu.
> > > >>>>>
> > > >>>>> So. I will start a vote in 72 hours that calls on the vote on
> > > >> this.
> > > >>>>> I'm going to write down my understanding of what this thread
> covers
> > > >> to
> > > >>>> get
> > > >>>>> some actionable topics to vote on.  If I've written anything down
> > > >>>>> incorrectly please speak up before we start a vote.
> > > >>>>>
> > > >>>>> Debian:
> > > >>>>>- Debian10 is the official docker image to support and be
> > > >> published
> &g

6/2/2020 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi~

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
- Testing release candidate 3. The latest error I got is a javadoc issue.
It seems like there is a new PR related to javadoc and I am testing the
latest code.

Regards,
--ning

Re: [DISCUSSION] Docker OS support

[Ning Wang]

Yeah. We have chosen Debian 10 for the binary release and clean up the
license requirements for the first binary release.

In the source level, we would also like to include a few platforms/versions
in the main source code to be "supported" and the others will be moved to
like a "extra" folder and not actively supported/tested.


On Wed, May 27, 2020 at 9:22 AM Dave Fisher  wrote:

> I thought that the purpose of this discussion was to find the one or two
> platforms for Heron to support in order to keep down the complexity of
> making the first binary release for the community.
>
> Other platforms could be on a roadmap for future releases.
>
> Regards,
> Dave
>
> > On May 26, 2020, at 12:26 PM, Ning Wang  wrote:
> >
> > My concern about "latest" is that I feel most people hold it for some
> time
> > before migrating to the latest version (if it ever happens). If there are
> > numbers, it could make sense to use the most popular version (plus the
> > other criterias).
> >
> > On Tue, May 26, 2020 at 12:14 PM Oliver Bristow <
> oli...@oliverbristow.co.uk>
> > wrote:
> >
> >> Sorry if I missed the rationale for Ubuntu 18.04 instead of 20.04, but
> >> would the latest LTS be a better choice? 18.04 was released before
> Debian
> >> 10, so that may be an argument for 20.04 re. compatibility if Debian 10
> is
> >> the standard.
> >>
> >> There's already a 20.04 Dockerfile + I It looks like the vagrant setup
> will
> >> get an upgrade from 14.04 to 20.04 as a part of the python3 upgrade, and
> >> seems to run fine so far.
> >>
> >> On Tue, 26 May 2020 at 19:30, Josh Fischer  wrote:
> >>
> >>> Thanks for further clarifying. Yes, that is correct.
> >>>
> >>>
> >>>
> >>> On Tue, May 26, 2020 at 1:16 PM Ning Wang 
> wrote:
> >>>
> >>>> +1.
> >>>>
> >>>> So my understanding is:
> >>>> binary (in future release): Debian 10.
> >>>> source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers will be
> >>>> updated in the future after more discussions/votes.
> >>>>
> >>>> others will be moved to a "extra" folder only for reference.
> >>>>
> >>>>
> >>>>
> >>>> On Tue, May 26, 2020 at 6:13 AM Josh Fischer 
> >>> wrote:
> >>>>
> >>>>> Hi All,
> >>>>>
> >>>>> Based on what I've read on this thread a combination of CentOS/Debian
> >>>> would
> >>>>> be the better options for us to support multiple versions for the
> >>>>> community.  We will also support one version of Ubuntu.
> >>>>>
> >>>>> So. I will start a vote in 72 hours that calls on the vote on
> >> this.
> >>>>> I'm going to write down my understanding of what this thread covers
> >> to
> >>>> get
> >>>>> some actionable topics to vote on.  If I've written anything down
> >>>>> incorrectly please speak up before we start a vote.
> >>>>>
> >>>>> Debian:
> >>>>>- Debian10 is the official docker image to support and be
> >> published
> >>>> to
> >>>>> Docker Hub, we will also support Debian9.
> >>>>> CentOS:
> >>>>>- We will support CentOS7.  Note, I do not think it has native
> >>>>> Python3.7 support, but we are also not there yet.
> >>>>> Ubuntu:
> >>>>>- We will still support one version. At this time the version is
> >>>> 18.04.
> >>>>>
> >>>>> We will upgrade supported versions of each OS as updates/deprecation
> >> of
> >>>>> support are needed.
> >>>>>
> >>>>> Again let's wait 72 hours before starting a vote.  Also please check
> >>> that
> >>>>> the statements I wrote down are what you understood!
> >>>>>
> >>>>> See below for notes on lazy consensus:
> >>>>> https://community.apache.org/committers/lazyConsensus.html
> >>>>>
> >>>>> - Josh
> >>>>>
> >>>>>
> >>>>> On Wed, May 20, 2020 at 9:41 PM Ning Wang 
> >>> wrote:
> >>>>>
> >>>>>> +1
> >>>>>

Re: [DISCUSSION] Docker OS support

[Ning Wang]

My concern about "latest" is that I feel most people hold it for some time
before migrating to the latest version (if it ever happens). If there are
numbers, it could make sense to use the most popular version (plus the
other criterias).

On Tue, May 26, 2020 at 12:14 PM Oliver Bristow 
wrote:

> Sorry if I missed the rationale for Ubuntu 18.04 instead of 20.04, but
> would the latest LTS be a better choice? 18.04 was released before Debian
> 10, so that may be an argument for 20.04 re. compatibility if Debian 10 is
> the standard.
>
> There's already a 20.04 Dockerfile + I It looks like the vagrant setup will
> get an upgrade from 14.04 to 20.04 as a part of the python3 upgrade, and
> seems to run fine so far.
>
> On Tue, 26 May 2020 at 19:30, Josh Fischer  wrote:
>
> > Thanks for further clarifying. Yes, that is correct.
> >
> >
> >
> > On Tue, May 26, 2020 at 1:16 PM Ning Wang  wrote:
> >
> > > +1.
> > >
> > > So my understanding is:
> > > binary (in future release): Debian 10.
> > > source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers will be
> > > updated in the future after more discussions/votes.
> > >
> > > others will be moved to a "extra" folder only for reference.
> > >
> > >
> > >
> > > On Tue, May 26, 2020 at 6:13 AM Josh Fischer 
> > wrote:
> > >
> > > > Hi All,
> > > >
> > > > Based on what I've read on this thread a combination of CentOS/Debian
> > > would
> > > > be the better options for us to support multiple versions for the
> > > > community.  We will also support one version of Ubuntu.
> > > >
> > > > So. I will start a vote in 72 hours that calls on the vote on
> this.
> > > > I'm going to write down my understanding of what this thread covers
> to
> > > get
> > > > some actionable topics to vote on.  If I've written anything down
> > > > incorrectly please speak up before we start a vote.
> > > >
> > > > Debian:
> > > > - Debian10 is the official docker image to support and be
> published
> > > to
> > > > Docker Hub, we will also support Debian9.
> > > > CentOS:
> > > > - We will support CentOS7.  Note, I do not think it has native
> > > > Python3.7 support, but we are also not there yet.
> > > > Ubuntu:
> > > > - We will still support one version. At this time the version is
> > > 18.04.
> > > >
> > > > We will upgrade supported versions of each OS as updates/deprecation
> of
> > > > support are needed.
> > > >
> > > > Again let's wait 72 hours before starting a vote.  Also please check
> > that
> > > > the statements I wrote down are what you understood!
> > > >
> > > > See below for notes on lazy consensus:
> > > > https://community.apache.org/committers/lazyConsensus.html
> > > >
> > > > - Josh
> > > >
> > > >
> > > > On Wed, May 20, 2020 at 9:41 PM Ning Wang 
> > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis <
> > > nicholas.ne...@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > I like those options.
> > > > > >
> > > > > > On Wed, May 20, 2020 at 10:15 PM Josh Fischer <
> j...@joshfischer.io
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Let's wait 3 more days for anyone to give input.  Then we can
> > > start a
> > > > > > vote
> > > > > > > to make a final decision on which containers for us to support.
> > > Keep
> > > > > in
> > > > > > > mind that even though we do need to plan for Python3 support
> > (There
> > > > is
> > > > > > work
> > > > > > > in progress now) we do not need it as of yet.
> > > > > > >
> > > > > > > - Josh
> > > > > > >
> > > > > > > On Wed, May 20, 2020 at 8:34 PM thinker0 
> > > wrote:
> > > > > > >
> > > > > > > > My experience is mainly for CentOS 7, so I think it would be
> > nice
> > > > if
> > > > > it
> > > > > > > was
&

Re: [DISCUSSION] Docker OS support

[Ning Wang]

+1.

So my understanding is:
binary (in future release): Debian 10.
source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers will be
updated in the future after more discussions/votes.

others will be moved to a "extra" folder only for reference.



On Tue, May 26, 2020 at 6:13 AM Josh Fischer  wrote:

> Hi All,
>
> Based on what I've read on this thread a combination of CentOS/Debian would
> be the better options for us to support multiple versions for the
> community.  We will also support one version of Ubuntu.
>
> So. I will start a vote in 72 hours that calls on the vote on this.
> I'm going to write down my understanding of what this thread covers to get
> some actionable topics to vote on.  If I've written anything down
> incorrectly please speak up before we start a vote.
>
> Debian:
> - Debian10 is the official docker image to support and be published to
> Docker Hub, we will also support Debian9.
> CentOS:
> - We will support CentOS7.  Note, I do not think it has native
> Python3.7 support, but we are also not there yet.
> Ubuntu:
> - We will still support one version. At this time the version is 18.04.
>
> We will upgrade supported versions of each OS as updates/deprecation of
> support are needed.
>
> Again let's wait 72 hours before starting a vote.  Also please check that
> the statements I wrote down are what you understood!
>
> See below for notes on lazy consensus:
> https://community.apache.org/committers/lazyConsensus.html
>
> - Josh
>
>
> On Wed, May 20, 2020 at 9:41 PM Ning Wang  wrote:
>
> > +1
> >
> > On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis  >
> > wrote:
> >
> > > I like those options.
> > >
> > > On Wed, May 20, 2020 at 10:15 PM Josh Fischer 
> > wrote:
> > >
> > > > Let's wait 3 more days for anyone to give input.  Then we can start a
> > > vote
> > > > to make a final decision on which containers for us to support.  Keep
> > in
> > > > mind that even though we do need to plan for Python3 support (There
> is
> > > work
> > > > in progress now) we do not need it as of yet.
> > > >
> > > > - Josh
> > > >
> > > > On Wed, May 20, 2020 at 8:34 PM thinker0  wrote:
> > > >
> > > > > My experience is mainly for CentOS 7, so I think it would be nice
> if
> > it
> > > > was
> > > > > included.
> > > > >
> > > > > 2020년 5월 21일 (목) 오전 9:56, Josh Fischer 님이 작성:
> > > > >
> > > > > > Well thought out Windham.  I like how you narrowed the criteria
> > down
> > > to
> > > > > > Debian10/CentOS7(depending on PY3 support) thinking about overall
> > > > support
> > > > > > and what others would need/want.
> > > > > >
> > > > > > +1
> > > > > >
> > > > > > - Josh
> > > > > >
> > > > > > On Mon, May 18, 2020 at 8:39 PM Windham Wong <
> > > windham.w...@stormeye.io
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > My experience is that, people tends to pick their container
> image
> > > OS
> > > > > > based
> > > > > > > on few criterias only:
> > > > > > > 1) image size, alpine always win, then debian/centos and 3rd is
> > > > > > > ubuntu/centos. alpine is always the smallest but not much
> library
> > > > > > provided
> > > > > > > and require to compile manually. debian is always good because
> it
> > > has
> > > > > > > apt-get to provide loads of library. centos is cool as debian
> > > because
> > > > > it
> > > > > > > also has good yum repo. ubuntu is always the last because it is
> > > fat,
> > > > > but
> > > > > > it
> > > > > > > provides very large range of libraries to use.
> > > > > > > 2) kernel feature, as mentioned last email, ubuntu has issue
> with
> > > DNS
> > > > > and
> > > > > > > some sort of network discover issues. I remember alpine has
> some
> > as
> > > > > well,
> > > > > > > because it is very plain, requires many fine tunes be done
> > > manually.
> > > > in
> > > > > > > this case, centos and debian should win.
> > > &g

Re: [DISCUSSION] Docker OS support

[Ning Wang]

+1

On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis 
wrote:

> I like those options.
>
> On Wed, May 20, 2020 at 10:15 PM Josh Fischer  wrote:
>
> > Let's wait 3 more days for anyone to give input.  Then we can start a
> vote
> > to make a final decision on which containers for us to support.  Keep in
> > mind that even though we do need to plan for Python3 support (There is
> work
> > in progress now) we do not need it as of yet.
> >
> > - Josh
> >
> > On Wed, May 20, 2020 at 8:34 PM thinker0  wrote:
> >
> > > My experience is mainly for CentOS 7, so I think it would be nice if it
> > was
> > > included.
> > >
> > > 2020년 5월 21일 (목) 오전 9:56, Josh Fischer 님이 작성:
> > >
> > > > Well thought out Windham.  I like how you narrowed the criteria down
> to
> > > > Debian10/CentOS7(depending on PY3 support) thinking about overall
> > support
> > > > and what others would need/want.
> > > >
> > > > +1
> > > >
> > > > - Josh
> > > >
> > > > On Mon, May 18, 2020 at 8:39 PM Windham Wong <
> windham.w...@stormeye.io
> > >
> > > > wrote:
> > > >
> > > > > My experience is that, people tends to pick their container image
> OS
> > > > based
> > > > > on few criterias only:
> > > > > 1) image size, alpine always win, then debian/centos and 3rd is
> > > > > ubuntu/centos. alpine is always the smallest but not much library
> > > > provided
> > > > > and require to compile manually. debian is always good because it
> has
> > > > > apt-get to provide loads of library. centos is cool as debian
> because
> > > it
> > > > > also has good yum repo. ubuntu is always the last because it is
> fat,
> > > but
> > > > it
> > > > > provides very large range of libraries to use.
> > > > > 2) kernel feature, as mentioned last email, ubuntu has issue with
> DNS
> > > and
> > > > > some sort of network discover issues. I remember alpine has some as
> > > well,
> > > > > because it is very plain, requires many fine tunes be done
> manually.
> > in
> > > > > this case, centos and debian should win.
> > > > > 3) linux stream, obviously, alpine, debian/ubuntu, centos, are 3
> > > > different
> > > > > streams that use different native tools. Poeple just pick their
> base
> > > > image
> > > > > based on what they often use in their life. However, I see a lot
> > debian
> > > > and
> > > > > alpine on docker hub because they are just smaller in size.
> > > > > 4. library support. Python library is a very good example in this
> > case.
> > > > As
> > > > > Heron is turning to Python 3, which specifically Python 3.7+, some
> > > > > discontinued based OS version does not provide the support anymore.
> > We
> > > > have
> > > > > tested out that, debian9 supports only up to Python3.5, ubuntu
> 14.04
> > > does
> > > > > not support python3 and ubuntu 16.04 supports up to Python 3.5
> only.
> > > > >
> > > > > I am not a big fan of using centos for docker. Concluding all 4
> > points
> > > > > above, I believe keeping centos7 (if it supports python37),
> debian10
> > > (not
> > > > > debian9), ubuntu 18.04 (supports python3.7+) would be the best
> > solution
> > > > at
> > > > > the moment. We have to cut off the old images due to our library
> > > upgrade
> > > > > and this is unavoidable. Otherwise, we would have tons of issues
> > > sending
> > > > in
> > > > > about library support in the future and the workload could be a lot
> > > more
> > > > > for just answering questions.
> > > > >
> > > > > Windham Wong
> > > > > Co-Founder, Technical Director of
> > > > > Stormeye.io, Hong Kong Managed Security Operation Center Limited
> > > > > Email // windham.w...@stormeye.io (mailto:windham.w...@stormeye.io
> )
> > > > > Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
> > > > > Fax // +852_3590_2202 (tel:+852_3590_2202)
> > > > >
> > > > > On 5月 19 2020, at 6:09 早上, Josh Fischer 
> wrote:
> > > > > > Hi All, I think usi

5/19/2020 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi~

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
- May Heron meetup.
- Docker discussion.
- Docker image fixes towards release.

Regards,
--ning

Re: [DISCUSSION] Docker OS support

[Ning Wang]

Good reasoning! Thanks!

On Mon, May 18, 2020 at 6:39 PM Windham Wong 
wrote:

> My experience is that, people tends to pick their container image OS based
> on few criterias only:
> 1) image size, alpine always win, then debian/centos and 3rd is
> ubuntu/centos. alpine is always the smallest but not much library provided
> and require to compile manually. debian is always good because it has
> apt-get to provide loads of library. centos is cool as debian because it
> also has good yum repo. ubuntu is always the last because it is fat, but it
> provides very large range of libraries to use.
> 2) kernel feature, as mentioned last email, ubuntu has issue with DNS and
> some sort of network discover issues. I remember alpine has some as well,
> because it is very plain, requires many fine tunes be done manually. in
> this case, centos and debian should win.
> 3) linux stream, obviously, alpine, debian/ubuntu, centos, are 3 different
> streams that use different native tools. Poeple just pick their base image
> based on what they often use in their life. However, I see a lot debian and
> alpine on docker hub because they are just smaller in size.
> 4. library support. Python library is a very good example in this case. As
> Heron is turning to Python 3, which specifically Python 3.7+, some
> discontinued based OS version does not provide the support anymore. We have
> tested out that, debian9 supports only up to Python3.5, ubuntu 14.04 does
> not support python3 and ubuntu 16.04 supports up to Python 3.5 only.
>
> I am not a big fan of using centos for docker. Concluding all 4 points
> above, I believe keeping centos7 (if it supports python37), debian10 (not
> debian9), ubuntu 18.04 (supports python3.7+) would be the best solution at
> the moment. We have to cut off the old images due to our library upgrade
> and this is unavoidable. Otherwise, we would have tons of issues sending in
> about library support in the future and the workload could be a lot more
> for just answering questions.
>
> Windham Wong
> Co-Founder, Technical Director of
> Stormeye.io, Hong Kong Managed Security Operation Center Limited
> Email // windham.w...@stormeye.io (mailto:windham.w...@stormeye.io)
> Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
> Fax // +852_3590_2202 (tel:+852_3590_2202)
>
> On 5月 19 2020, at 6:09 早上, Josh Fischer  wrote:
> > Hi All, I think using CentOS and either Debian or Ubuntu is a good idea.
> I would pick Debian over Ubuntu for the reason that Nick pointed out in a
> previous email of DNS issues in Kubernetes. I"ve copy and pasted a section
> of Nick's previous email. ### Start I've run into issues with Ubuntu images
> having DNS issues in Kubernetes.
> https://github.com/kubernetes/kubeadm/issues/787 ### End As for what ends
> up being the official Docker container I'm fine with either choice Debian
> or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang wrote: > Yeah. For
> binary release, the license is a huge factor. > > On Sun, May 17, 2020 at
> 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it comes to choosing
> the official Apache Docker image for Heron we > > have to think of the
> licensing issues first. Everything that is > installed > > into the
> container at build time with the exception of the "FROM" > > statement in
> the Dockerfile must be ALv2 compatible. OpenJDK is under > GPLv2 > > which
> is not compatible with Apache. However we can get around this if > we > >
> bring the OpenJDK into the container via a FROM statement. I've copied a >
> > reply from Dave in our dev list in a previous email. See below. > > > >
>  Start  > > Regarding OpenJDK and GPL2 - here is what Roman
> the VP, Legal wrote when > > answering Beam’s questions. > > > > Roman
> Shaposhnik commented on LEGAL-503: > >
>  > > > > Hey [~altay] if you would
> like to continue linking to the Docker release > > artifact from the > >
> https://beam.apache.org > > you will have: > > 1. Transition to the
> official ASF dockerhub org: > > https://hub.docker.com/u/apache > > 2.
> Start including that binary convenience artifact into your VOTE > > threads
> on Beam releases > > 3. Make sure that all Cat-X licenses are ONLY brought
> into your > > container via FROM statements > >  End  > > >
> > So at the end of the day, we need to use a container that has Java >
> already > > installed in it so that when we create our official image we
> can inherit > > that java containing image via a FROM statement. > &

Re: [DISCUSSION] Docker OS support

[Ning Wang]

Yeah. For binary release, the license is a huge factor.

On Sun, May 17, 2020 at 2:32 PM Josh Fischer  wrote:

> Hi All,
>
> When it comes to choosing the official Apache Docker image for Heron we
> have to think of the licensing issues first.  Everything that is installed
> into the container at build time with the exception of the "FROM"
> statement in the Dockerfile must be ALv2 compatible. OpenJDK is under GPLv2
> which is not compatible with Apache.  However we can get around this if we
> bring the OpenJDK into the container via a FROM statement.  I've copied a
> reply from Dave in our dev list in a previous email.  See below.
>
>  Start 
> Regarding OpenJDK and GPL2 - here is what Roman the VP, Legal wrote when
> answering Beam’s questions.
>
> Roman Shaposhnik commented on LEGAL-503:
> 
>
> Hey [~altay] if you would like to continue linking to the Docker release
> artifact from the
> https://beam.apache.org
>  you will have:
>1. Transition to the official ASF dockerhub org:
> https://hub.docker.com/u/apache
>2. Start including that binary convenience artifact into your VOTE
> threads on Beam releases
>3. Make sure that all Cat-X licenses are ONLY brought into your
> container via FROM statements
>  End 
>
> So at the end of the day, we need to use a container that has Java already
> installed in it so that when we create our official image we can inherit
> that java containing image via a FROM statement.
>
> - Josh
>
>
> On Sun, May 17, 2020 at 4:12 PM H W  wrote:
>
> > We use Centos 7 as well.
> >
> > On Sun, May 17, 2020 at 10:41 AM Nicholas Nezis <
> nicholas.ne...@gmail.com>
> > wrote:
> >
> > > Some thoughts I'd like to add to the discussion:
> > >
> > > Debian and Ubuntu are somewhat similar. Would it make sense to pick one
> > of
> > > those and pair with Centos?
> > >
> > > I've run into issues with Ubuntu images having DNS issues in
> Kubernetes.
> > > https://github.com/kubernetes/kubeadm/issues/787
> > >
> > > Ubuntu 14.04 should be removed. It was end of life last year. For those
> > of
> > > you using Heron in production, is anyone using Ubuntu 16? My vote would
> > be
> > > to remove it also. For what it's worth, my use case is in K8s and we
> use
> > > the Centos7 image on a Centos 7 OS.
> > >
> > > We should do a test of the official image and helm chart in Kubernetes
> > > (maybe with Kind https://kind.sigs.k8s.io/ ) as part of our release
> > > process. This might be nice to add as an integration test.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Sun, May 17, 2020, 12:37 PM Ning Wang  wrote:
> > >
> > > > That is a good point. We need to adjust the OS version accordingly
> too.
> > > >
> > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong <
> windham.w...@stormeye.io
> > >
> > > > wrote:
> > > >
> > > > >
> > > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no python
> 3.7+
> > > dev
> > > > > library support. hope this helps
> > > > > --
> > > > > Sent from myMail for Android Sunday, 17 May 2020, 11:34AM +08:00
> from
> > > > Ning
> > > > > Wang  wangnin...@gmail.com :
> > > > >
> > > > > >Hi,
> > > > > >
> > > > > >We have talked about it a few times in different places. Let's
> make
> > a
> > > > > >decision here. We don't have enough resources to support many
> docker
> > > > OSes
> > > > > >(technically we only support Debian, Ubuntu, and CentOS but each
> of
> > > them
> > > > > >has multiple versions).
> > > > > >
> > > > > >Things we have agreed on (my understanding)
> > > > > >- for binary release, we are going to choose one OS one version. I
> > > > > >remember that Debian 9 was the decision. An open question is: do
> we
> > > want
> > > > > to
> > > > > >switch to Debian 10 now? We haven't had a binary release yet, so
> > there
> > > > is
> > > > > >still time. As the main target, I think 1. the image should
> compile
> > > > > >successfully. and 2 the image should be tested: the tools and an
> > > example
> > > > > >

Re: [DISCUSSION] Docker OS support

[Ning Wang]

Good to know! Thx.

On Sun, May 17, 2020 at 2:12 PM H W  wrote:

> We use Centos 7 as well.
>
> On Sun, May 17, 2020 at 10:41 AM Nicholas Nezis 
> wrote:
>
> > Some thoughts I'd like to add to the discussion:
> >
> > Debian and Ubuntu are somewhat similar. Would it make sense to pick one
> of
> > those and pair with Centos?
> >
> > I've run into issues with Ubuntu images having DNS issues in Kubernetes.
> > https://github.com/kubernetes/kubeadm/issues/787
> >
> > Ubuntu 14.04 should be removed. It was end of life last year. For those
> of
> > you using Heron in production, is anyone using Ubuntu 16? My vote would
> be
> > to remove it also. For what it's worth, my use case is in K8s and we use
> > the Centos7 image on a Centos 7 OS.
> >
> > We should do a test of the official image and helm chart in Kubernetes
> > (maybe with Kind https://kind.sigs.k8s.io/ ) as part of our release
> > process. This might be nice to add as an integration test.
> >
> >
> >
> >
> >
> >
> >
> > On Sun, May 17, 2020, 12:37 PM Ning Wang  wrote:
> >
> > > That is a good point. We need to adjust the OS version accordingly too.
> > >
> > > On Sun, May 17, 2020 at 2:28 AM Windham Wong  >
> > > wrote:
> > >
> > > >
> > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no python 3.7+
> > dev
> > > > library support. hope this helps
> > > > --
> > > > Sent from myMail for Android Sunday, 17 May 2020, 11:34AM +08:00 from
> > > Ning
> > > > Wang  wangnin...@gmail.com :
> > > >
> > > > >Hi,
> > > > >
> > > > >We have talked about it a few times in different places. Let's make
> a
> > > > >decision here. We don't have enough resources to support many docker
> > > OSes
> > > > >(technically we only support Debian, Ubuntu, and CentOS but each of
> > them
> > > > >has multiple versions).
> > > > >
> > > > >Things we have agreed on (my understanding)
> > > > >- for binary release, we are going to choose one OS one version. I
> > > > >remember that Debian 9 was the decision. An open question is: do we
> > want
> > > > to
> > > > >switch to Debian 10 now? We haven't had a binary release yet, so
> there
> > > is
> > > > >still time. As the main target, I think 1. the image should compile
> > > > >successfully. and 2 the image should be tested: the tools and an
> > example
> > > > >topology.
> > > > >
> > > > >- for source release, we haven't talked about it very much.
> > Personally I
> > > > >feel it is a reasonable expectation that, if we have the Docker file
> > in
> > > > the
> > > > >core folders, it should at least compile. Otherwise, it's a failed
> > build
> > > > >hence a failed release. We are not going to test run the tools and
> > > > examples
> > > > >for each release though.
> > > > >
> > > > >Finally, a question about OS versions. I believe at least two
> releases
> > > > >should be supported because most people don't upgrade to the latest
> > > > version
> > > > >when it is out in more serious scenarios, hence IMO the second
> oldest
> > > > could
> > > > >often be more useful than the latest one from a convenience point of
> > > view.
> > > > >On the other hand, this is too flexible, and we will spend time
> > thinking
> > > > >about it again. I think a clear guideline could be helpful to us.
> For
> > > > >example:
> > > > >- We only include only Debian (popular in the server world) and
> Ubuntu
> > > > >(popular in servers and workstations).
> > > > >- for the main OS (Debian) we choose the latest to compile and test
> > for
> > > > our
> > > > >binary release.
> > > > >- for a given OS, only two versions (most likely the most and the
> > second
> > > > >recent releases) are included in the core source code and they need
> to
> > > be
> > > > >fixed ASAP if they don't compile.
> > > > >- OSs and other versions could be included in a special folder and
> but
> > > > >there is no guarantee that the code will compile. Fixes from the
> > > community
> > > > >are welcome.
> > > > >
> > > > >
> > > > >How do you guys think?
> > > > >--ning
> > > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Ning Wang]

A checklist of things to check could be great. We should create two, one
for source release and one for binary release.

It is agood point that Debian and Ubuntu are similar, so CentOS might give
us more coverage. Between Debian and Ubuntu, my personal vote might be
slightly towards Ubuntu because of the popularity in both desktop and
server world. But I am ok with either one.

Ubuntu16 is a bit tricky to me. If we drop it and go to 18+20, we are
supporting only the releases of the most recent 2 years which is really
short period of time for people to adopt.




On Sun, May 17, 2020 at 10:41 AM Nicholas Nezis 
wrote:

> Some thoughts I'd like to add to the discussion:
>
> Debian and Ubuntu are somewhat similar. Would it make sense to pick one of
> those and pair with Centos?
>
> I've run into issues with Ubuntu images having DNS issues in Kubernetes.
> https://github.com/kubernetes/kubeadm/issues/787
>
> Ubuntu 14.04 should be removed. It was end of life last year. For those of
> you using Heron in production, is anyone using Ubuntu 16? My vote would be
> to remove it also. For what it's worth, my use case is in K8s and we use
> the Centos7 image on a Centos 7 OS.
>
> We should do a test of the official image and helm chart in Kubernetes
> (maybe with Kind https://kind.sigs.k8s.io/ ) as part of our release
> process. This might be nice to add as an integration test.
>
>
>
>
>
>
>
> On Sun, May 17, 2020, 12:37 PM Ning Wang  wrote:
>
> > That is a good point. We need to adjust the OS version accordingly too.
> >
> > On Sun, May 17, 2020 at 2:28 AM Windham Wong 
> > wrote:
> >
> > >
> > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no python 3.7+
> dev
> > > library support. hope this helps
> > > --
> > > Sent from myMail for Android Sunday, 17 May 2020, 11:34AM +08:00 from
> > Ning
> > > Wang  wangnin...@gmail.com :
> > >
> > > >Hi,
> > > >
> > > >We have talked about it a few times in different places. Let's make a
> > > >decision here. We don't have enough resources to support many docker
> > OSes
> > > >(technically we only support Debian, Ubuntu, and CentOS but each of
> them
> > > >has multiple versions).
> > > >
> > > >Things we have agreed on (my understanding)
> > > >- for binary release, we are going to choose one OS one version. I
> > > >remember that Debian 9 was the decision. An open question is: do we
> want
> > > to
> > > >switch to Debian 10 now? We haven't had a binary release yet, so there
> > is
> > > >still time. As the main target, I think 1. the image should compile
> > > >successfully. and 2 the image should be tested: the tools and an
> example
> > > >topology.
> > > >
> > > >- for source release, we haven't talked about it very much.
> Personally I
> > > >feel it is a reasonable expectation that, if we have the Docker file
> in
> > > the
> > > >core folders, it should at least compile. Otherwise, it's a failed
> build
> > > >hence a failed release. We are not going to test run the tools and
> > > examples
> > > >for each release though.
> > > >
> > > >Finally, a question about OS versions. I believe at least two releases
> > > >should be supported because most people don't upgrade to the latest
> > > version
> > > >when it is out in more serious scenarios, hence IMO the second oldest
> > > could
> > > >often be more useful than the latest one from a convenience point of
> > view.
> > > >On the other hand, this is too flexible, and we will spend time
> thinking
> > > >about it again. I think a clear guideline could be helpful to us. For
> > > >example:
> > > >- We only include only Debian (popular in the server world) and Ubuntu
> > > >(popular in servers and workstations).
> > > >- for the main OS (Debian) we choose the latest to compile and test
> for
> > > our
> > > >binary release.
> > > >- for a given OS, only two versions (most likely the most and the
> second
> > > >recent releases) are included in the core source code and they need to
> > be
> > > >fixed ASAP if they don't compile.
> > > >- OSs and other versions could be included in a special folder and but
> > > >there is no guarantee that the code will compile. Fixes from the
> > community
> > > >are welcome.
> > > >
> > > >
> > > >How do you guys think?
> > > >--ning
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Ning Wang]

That is a good point. We need to adjust the OS version accordingly too.

On Sun, May 17, 2020 at 2:28 AM Windham Wong 
wrote:

>
> debian 10 has no python 2.7 support. Ubuntu 16.04 has no python 3.7+ dev
> library support. hope this helps
> --
> Sent from myMail for Android Sunday, 17 May 2020, 11:34AM +08:00 from Ning
> Wang  wangnin...@gmail.com :
>
> >Hi,
> >
> >We have talked about it a few times in different places. Let's make a
> >decision here. We don't have enough resources to support many docker OSes
> >(technically we only support Debian, Ubuntu, and CentOS but each of them
> >has multiple versions).
> >
> >Things we have agreed on (my understanding)
> >- for binary release, we are going to choose one OS one version. I
> >remember that Debian 9 was the decision. An open question is: do we want
> to
> >switch to Debian 10 now? We haven't had a binary release yet, so there is
> >still time. As the main target, I think 1. the image should compile
> >successfully. and 2 the image should be tested: the tools and an example
> >topology.
> >
> >- for source release, we haven't talked about it very much. Personally I
> >feel it is a reasonable expectation that, if we have the Docker file in
> the
> >core folders, it should at least compile. Otherwise, it's a failed build
> >hence a failed release. We are not going to test run the tools and
> examples
> >for each release though.
> >
> >Finally, a question about OS versions. I believe at least two releases
> >should be supported because most people don't upgrade to the latest
> version
> >when it is out in more serious scenarios, hence IMO the second oldest
> could
> >often be more useful than the latest one from a convenience point of view.
> >On the other hand, this is too flexible, and we will spend time thinking
> >about it again. I think a clear guideline could be helpful to us. For
> >example:
> >- We only include only Debian (popular in the server world) and Ubuntu
> >(popular in servers and workstations).
> >- for the main OS (Debian) we choose the latest to compile and test for
> our
> >binary release.
> >- for a given OS, only two versions (most likely the most and the second
> >recent releases) are included in the core source code and they need to be
> >fixed ASAP if they don't compile.
> >- OSs and other versions could be included in a special folder and but
> >there is no guarantee that the code will compile. Fixes from the community
> >are welcome.
> >
> >
> >How do you guys think?
> >--ning
>

Re: Building Zookeeper Source on Ubuntu 16.04 fails

[Ning Wang]

looking at 14.04 now, although we may drop it in the near future.

On Sun, May 17, 2020 at 1:46 AM Ning Wang  wrote:

> Fixed 16.04: https://github.com/apache/incubator-heron/pull/3525
>
> I think it is caused by pkg-config.
>
> On Sat, May 16, 2020 at 8:37 PM Ning Wang  wrote:
>
>> Thanks for the input. We can skip Python 3 support for this release.
>> Currently, it is failing on the zookeeper source step in Ubuntu16.04 and
>> 14.04. This is the last known blocker for me to prepare a new release
>> candidate and start a vote.
>>
>>
>> On Sat, May 16, 2020 at 8:28 PM Windham Wong 
>> wrote:
>>
>>> Recalling the test I made in March, Zookeeper should be fine in Ubuntu
>>> 16.04, just require some tuning of the bazel build file to match the
>>> version in there. However, switching to Python 3 breaks Ubuntu 16.04 and
>>> CentOS 7 due to no official support of the Python library dev in ther later
>>> version. This still breaks the Heron build in the later part.
>>>
>>> Windham Wong
>>> OSWE, OSCP, GCIA, Specialist in Cybersecurity
>>> Co-Founder, Technical Director of
>>> Stormeye.io, Hong Kong Managed Security Operation Center Limited
>>> Email // windham.w...@stormeye.io (
>>> https://link.getmailspring.com/link/0d144bf6-b9da-4ceb-ab62-f77dea9ae...@getmailspring.com/1?redirect=mailto%3Awindham.wong%40stormeye.io=ZGV2QGhlcm9uLmluY3ViYXRvci5hcGFjaGUub3Jn
>>> )
>>> Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
>>> Fax // +852_3590_2202 (tel:+852_3590_2202)
>>>
>>> On 5月 17 2020, at 5:31 凌晨, Josh Fischer  wrote:
>>> > Hi All, I'm checking for OS compatibility with git tag
>>> 0.20.3-incubating-rc3 and the zookeeper-srcs target is failing to build on
>>> Ubuntu 16.04. The zookeeper-srcs target builds successfully on Mac. So far
>>> the only show stopping error I can see is that an expected file does not
>>> exist in the logs. Has anyone run into this issue before? See below for my
>>> terminal output. bazel build --config=ubuntu_nostyle
>>> third_party/zookeeper:zookeeper-srcs INFO: Options provided by the client:
>>> Inherited 'common' options: --isatty=1 --terminal_columns=141 INFO: Reading
>>> rc options for 'build' from /root/incubator-heron/tools/bazel.rc: 'build'
>>> options: --genrule_strategy=standalone --host_force_python=PY2
>>> --ignore_unsupported_sandboxing --spawn_strategy=standalone
>>> --workspace_status_command scripts/release/status.sh INFO: Reading rc
>>> options for 'build' from /root/.bazelrc: 'build' options:
>>> --verbose_failures --spawn_strategy=standalone
>>> --genrule_strategy=standalone --local_ram_resources=4096
>>> --local_cpu_resources=2 --announce_rc INFO: Found applicable config
>>> definition build:ubuntu_nostyle in file
>>> /root/incubator-heron/tools/bazel.rc: --genrule_strategy=standalone
>>> --ignore_unsupported_sandboxing --linkopt -lm --linkopt -lpthread --linkopt
>>> -lrt --spawn_strategy=standalone --workspace_status_command
>>> scripts/release/status.sh --copt=-O3 INFO: Analyzed target
>>> //third_party/zookeeper:zookeeper-srcs (1 packages loaded, 2 targets
>>> configured). INFO: Found 1 target... ERROR:
>>> /root/incubator-heron/third_party/zookeeper/BUILD:7:1: Executing genrule
>>> //third_party/zookeeper:zookeeper-srcs failed (Exit 1): bash failed: error
>>> executing command (cd
>>> /root/.cache/bazel/_bazel_root/88837ab595738499a4ef3075862ee807/execroot/org_apache_heron
>>> && \ exec env - \
>>> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
>>> /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh;
>>> export INSTALL_DIR=$(pwd)/bazel-out/k8-fastbuild/bin/third_party/zookeeper
>>> export TMP_DIR=$(mktemp -d -t zookeeper.X) export CXXFLAGS=-Wno-error
>>> export CFLAGS=-Wno-error mkdir -p $TMP_DIR cp -R
>>> $(pwd)/external/org_apache_zookeeper/* $TMP_DIR cd
>>> $TMP_DIR/zookeeper-client/zookeeper-client-c autoreconf -if ./configure
>>> --prefix=$INSTALL_DIR --enable-shared=no make install rm -rf $TMP_DIR')
>>> Execution platform: @local_config_platform//:host cp: cannot stat
>>> '/root/.cache/bazel/_bazel_root/88837ab595738499a4ef3075862ee807/execroot/org_apache_heron/external/org_apache_zookeeper/*':
>>> No such file or directory Target //third_party/zookeeper:zookeeper-srcs
>>> failed to build INFO: Elapsed time: 0.815s, Critical Path: 0.06s INFO: 0
>>> processes. FAILED: Build did NOT complete successfully
>>>
>>>

Re: Building Zookeeper Source on Ubuntu 16.04 fails

[Ning Wang]

Fixed 16.04: https://github.com/apache/incubator-heron/pull/3525

I think it is caused by pkg-config.

On Sat, May 16, 2020 at 8:37 PM Ning Wang  wrote:

> Thanks for the input. We can skip Python 3 support for this release.
> Currently, it is failing on the zookeeper source step in Ubuntu16.04 and
> 14.04. This is the last known blocker for me to prepare a new release
> candidate and start a vote.
>
>
> On Sat, May 16, 2020 at 8:28 PM Windham Wong 
> wrote:
>
>> Recalling the test I made in March, Zookeeper should be fine in Ubuntu
>> 16.04, just require some tuning of the bazel build file to match the
>> version in there. However, switching to Python 3 breaks Ubuntu 16.04 and
>> CentOS 7 due to no official support of the Python library dev in ther later
>> version. This still breaks the Heron build in the later part.
>>
>> Windham Wong
>> OSWE, OSCP, GCIA, Specialist in Cybersecurity
>> Co-Founder, Technical Director of
>> Stormeye.io, Hong Kong Managed Security Operation Center Limited
>> Email // windham.w...@stormeye.io (
>> https://link.getmailspring.com/link/0d144bf6-b9da-4ceb-ab62-f77dea9ae...@getmailspring.com/1?redirect=mailto%3Awindham.wong%40stormeye.io=ZGV2QGhlcm9uLmluY3ViYXRvci5hcGFjaGUub3Jn
>> )
>> Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
>> Fax // +852_3590_2202 (tel:+852_3590_2202)
>>
>> On 5月 17 2020, at 5:31 凌晨, Josh Fischer  wrote:
>> > Hi All, I'm checking for OS compatibility with git tag
>> 0.20.3-incubating-rc3 and the zookeeper-srcs target is failing to build on
>> Ubuntu 16.04. The zookeeper-srcs target builds successfully on Mac. So far
>> the only show stopping error I can see is that an expected file does not
>> exist in the logs. Has anyone run into this issue before? See below for my
>> terminal output. bazel build --config=ubuntu_nostyle
>> third_party/zookeeper:zookeeper-srcs INFO: Options provided by the client:
>> Inherited 'common' options: --isatty=1 --terminal_columns=141 INFO: Reading
>> rc options for 'build' from /root/incubator-heron/tools/bazel.rc: 'build'
>> options: --genrule_strategy=standalone --host_force_python=PY2
>> --ignore_unsupported_sandboxing --spawn_strategy=standalone
>> --workspace_status_command scripts/release/status.sh INFO: Reading rc
>> options for 'build' from /root/.bazelrc: 'build' options:
>> --verbose_failures --spawn_strategy=standalone
>> --genrule_strategy=standalone --local_ram_resources=4096
>> --local_cpu_resources=2 --announce_rc INFO: Found applicable config
>> definition build:ubuntu_nostyle in file
>> /root/incubator-heron/tools/bazel.rc: --genrule_strategy=standalone
>> --ignore_unsupported_sandboxing --linkopt -lm --linkopt -lpthread --linkopt
>> -lrt --spawn_strategy=standalone --workspace_status_command
>> scripts/release/status.sh --copt=-O3 INFO: Analyzed target
>> //third_party/zookeeper:zookeeper-srcs (1 packages loaded, 2 targets
>> configured). INFO: Found 1 target... ERROR:
>> /root/incubator-heron/third_party/zookeeper/BUILD:7:1: Executing genrule
>> //third_party/zookeeper:zookeeper-srcs failed (Exit 1): bash failed: error
>> executing command (cd
>> /root/.cache/bazel/_bazel_root/88837ab595738499a4ef3075862ee807/execroot/org_apache_heron
>> && \ exec env - \
>> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
>> /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh;
>> export INSTALL_DIR=$(pwd)/bazel-out/k8-fastbuild/bin/third_party/zookeeper
>> export TMP_DIR=$(mktemp -d -t zookeeper.X) export CXXFLAGS=-Wno-error
>> export CFLAGS=-Wno-error mkdir -p $TMP_DIR cp -R
>> $(pwd)/external/org_apache_zookeeper/* $TMP_DIR cd
>> $TMP_DIR/zookeeper-client/zookeeper-client-c autoreconf -if ./configure
>> --prefix=$INSTALL_DIR --enable-shared=no make install rm -rf $TMP_DIR')
>> Execution platform: @local_config_platform//:host cp: cannot stat
>> '/root/.cache/bazel/_bazel_root/88837ab595738499a4ef3075862ee807/execroot/org_apache_heron/external/org_apache_zookeeper/*':
>> No such file or directory Target //third_party/zookeeper:zookeeper-srcs
>> failed to build INFO: Elapsed time: 0.815s, Critical Path: 0.06s INFO: 0
>> processes. FAILED: Build did NOT complete successfully
>>
>>

Re: Building Zookeeper Source on Ubuntu 16.04 fails

[Ning Wang]

My concern is that if we have the file there, it should at least compile. I
am open to remove the file or move it to somewhere we don't directly
support. But right now, it is there in the docker dir so I think it is a
reasonable expectation for people to believe it shouldn't throw error just
compiling it.

On Sat, May 16, 2020 at 5:21 PM Josh Fischer  wrote:

> Would it be that bad if we made a release knowing that we couldn't support
> Centos7 or Ubuntu16.04?  This would still allow us to get binaries/an
> official Docker image/ and install scripts for a few supported operating
> systems (Ubuntu 18.x, Debian9, Debian 10, OSX).  I remember hearing  that
> we couldn't release install scripts under Apache, but never really got a
> clear answer yet as to why.
>
> In my opinion the toughest part about building a community around Heron is
> because it is not easy to use.  Doing a release with binaries I think would
> greatly help the community and the current user base (what's left of it).
>  I believe making a small release like this is a step in the right
> direction for the project.
>
> - Josh
>
>
>
> On Sat, May 16, 2020 at 5:03 PM Josh Fischer  wrote:
>
> > If I were to take a guess I would think it's the line below:  But why
> this
> > folder wouldn't exist I'm unsure of..
> >
> > cp -R $(pwd)/external/org_apache_zookeeper/* $TMP_DIR
> >
> > I've also added the github permalink to the line of code below:
> >
> >
> https://github.com/apache/incubator-heron/blob/2a4f112fc567897711ee7117689e636534cb6fb4/third_party/zookeeper/BUILD#L41
> >
> > There is some new conditional logic in that zookeeper rule that may be
> > part of the issue.
> >
> > The link to the conditional in the zookeeper rule is below:
> >
> >
> https://github.com/apache/incubator-heron/blob/2a4f112fc567897711ee7117689e636534cb6fb4/third_party/zookeeper/BUILD#L23
> >
> > On Sat, May 16, 2020 at 4:47 PM Ning Wang  wrote:
> >
> >> Yeah. It's the error when I tried to build on Ubuntu16. The error is in
> >> the
> >> list of commands which is tricky to debug. There was a similar error
> >> message in centos 7 which was caused by cppunit I remember. But it is
> hard
> >> to tell if it is the same here.
> >>
> >> On Sat, May 16, 2020 at 2:31 PM Josh Fischer 
> wrote:
> >>
> >> > Hi All,
> >> >
> >> > I'm checking for OS compatibility with git tag 0.20.3-incubating-rc3
> and
> >> > the zookeeper-srcs target is failing to build on Ubuntu 16.04.  The
> >> > zookeeper-srcs target builds successfully on Mac.  So far the only
> show
> >> > stopping error I can see is that an expected file does not exist in
> the
> >> > logs. Has anyone run into this issue before?  See below for my
> >> > terminal output.
> >> >
> >> > bazel build --config=ubuntu_nostyle
> third_party/zookeeper:zookeeper-srcs
> >> >
> >> > INFO: Options provided by the client:
> >> >
> >> >   Inherited 'common' options: --isatty=1 --terminal_columns=141
> >> >
> >> > INFO: Reading rc options for 'build' from
> >> > /root/incubator-heron/tools/bazel.rc:
> >> >
> >> >   'build' options: --genrule_strategy=standalone
> --host_force_python=PY2
> >> > --ignore_unsupported_sandboxing --spawn_strategy=standalone
> >> > --workspace_status_command scripts/release/status.sh
> >> >
> >> > INFO: Reading rc options for 'build' from /root/.bazelrc:
> >> >
> >> >   'build' options: --verbose_failures --spawn_strategy=standalone
> >> > --genrule_strategy=standalone --local_ram_resources=4096
> >> > --local_cpu_resources=2 --announce_rc
> >> >
> >> > INFO: Found applicable config definition build:ubuntu_nostyle in file
> >> > /root/incubator-heron/tools/bazel.rc: --genrule_strategy=standalone
> >> > --ignore_unsupported_sandboxing --linkopt -lm --linkopt -lpthread
> >> --linkopt
> >> > -lrt --spawn_strategy=standalone --workspace_status_command
> >> > scripts/release/status.sh --copt=-O3
> >> >
> >> > INFO: Analyzed target //third_party/zookeeper:zookeeper-srcs (1
> packages
> >> > loaded, 2 targets configured).
> >> >
> >> > INFO: Found 1 target...
> >> >
> >> > ERROR: /root/incubator-heron/third_party/zookeeper/BUILD:7:1:
> Executing
> >> > genrule //third_party/zookeeper:zookeeper-srcs failed (Exit 1): bash
> >> > failed: error e

Re: Building Zookeeper Source on Ubuntu 16.04 fails

[Ning Wang]

Yeah. It's the error when I tried to build on Ubuntu16. The error is in the
list of commands which is tricky to debug. There was a similar error
message in centos 7 which was caused by cppunit I remember. But it is hard
to tell if it is the same here.

On Sat, May 16, 2020 at 2:31 PM Josh Fischer  wrote:

> Hi All,
>
> I'm checking for OS compatibility with git tag 0.20.3-incubating-rc3 and
> the zookeeper-srcs target is failing to build on Ubuntu 16.04.  The
> zookeeper-srcs target builds successfully on Mac.  So far the only show
> stopping error I can see is that an expected file does not exist in the
> logs. Has anyone run into this issue before?  See below for my
> terminal output.
>
> bazel build --config=ubuntu_nostyle third_party/zookeeper:zookeeper-srcs
>
> INFO: Options provided by the client:
>
>   Inherited 'common' options: --isatty=1 --terminal_columns=141
>
> INFO: Reading rc options for 'build' from
> /root/incubator-heron/tools/bazel.rc:
>
>   'build' options: --genrule_strategy=standalone --host_force_python=PY2
> --ignore_unsupported_sandboxing --spawn_strategy=standalone
> --workspace_status_command scripts/release/status.sh
>
> INFO: Reading rc options for 'build' from /root/.bazelrc:
>
>   'build' options: --verbose_failures --spawn_strategy=standalone
> --genrule_strategy=standalone --local_ram_resources=4096
> --local_cpu_resources=2 --announce_rc
>
> INFO: Found applicable config definition build:ubuntu_nostyle in file
> /root/incubator-heron/tools/bazel.rc: --genrule_strategy=standalone
> --ignore_unsupported_sandboxing --linkopt -lm --linkopt -lpthread --linkopt
> -lrt --spawn_strategy=standalone --workspace_status_command
> scripts/release/status.sh --copt=-O3
>
> INFO: Analyzed target //third_party/zookeeper:zookeeper-srcs (1 packages
> loaded, 2 targets configured).
>
> INFO: Found 1 target...
>
> ERROR: /root/incubator-heron/third_party/zookeeper/BUILD:7:1: Executing
> genrule //third_party/zookeeper:zookeeper-srcs failed (Exit 1): bash
> failed: error executing command
>
>   (cd
>
> /root/.cache/bazel/_bazel_root/88837ab595738499a4ef3075862ee807/execroot/org_apache_heron
> && \
>
>   exec env - \
>
> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
>
>   /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh;
> export INSTALL_DIR=$(pwd)/bazel-out/k8-fastbuild/bin/third_party/zookeeper
>
> export TMP_DIR=$(mktemp -d -t zookeeper.X)
>
> export CXXFLAGS=-Wno-error
>
> export CFLAGS=-Wno-error
>
> mkdir -p $TMP_DIR
>
> cp -R $(pwd)/external/org_apache_zookeeper/* $TMP_DIR
>
> cd $TMP_DIR/zookeeper-client/zookeeper-client-c
>
> autoreconf -if
>
> ./configure --prefix=$INSTALL_DIR --enable-shared=no
>
> make install
>
> rm -rf $TMP_DIR')
>
> Execution platform: @local_config_platform//:host
>
> cp: cannot stat
>
> '/root/.cache/bazel/_bazel_root/88837ab595738499a4ef3075862ee807/execroot/org_apache_heron/external/org_apache_zookeeper/*':
> No such file or directory
>
> Target //third_party/zookeeper:zookeeper-srcs failed to build
>
> INFO: Elapsed time: 0.815s, Critical Path: 0.06s
>
> INFO: 0 processes.
>
> FAILED: Build did NOT complete successfully
>

Re: Podling Heron Report Reminder - May 2020

[Ning Wang]

Thanks, Josh!

On Mon, May 4, 2020 at 1:26 PM Josh Fischer  wrote:

> Thank you to everyone who helped fill out the podling report.  All notes
> have been transferred and submitted.  Let me know if you have any
> questions.
>
> - Josh
>
> On Thu, Apr 16, 2020 at 3:06 AM  wrote:
>
> > Dear podling,
> >
> > This email was sent by an automated system on behalf of the Apache
> > Incubator PMC. It is an initial reminder to give you plenty of time to
> > prepare your quarterly board report.
> >
> > The board meeting is scheduled for Wed, 20 May 2020.
> > The report for your podling will form a part of the Incubator PMC
> > report. The Incubator PMC requires your report to be submitted 2 weeks
> > before the board meeting, to allow sufficient time for review and
> > submission (Wed, May 06).
> >
> > Please submit your report with sufficient time to allow the Incubator
> > PMC, and subsequently board members to review and digest. Again, the
> > very latest you should submit your report is 2 weeks prior to the board
> > meeting.
> >
> > Candidate names should not be made public before people are actually
> > elected, so please do not include the names of potential committers or
> > PPMC members in your report.
> >
> > Thanks,
> >
> > The Apache Incubator PMC
> >
> > Submitting your Report
> >
> > --
> >
> > Your report should contain the following:
> >
> > *   Your project name
> > *   A brief description of your project, which assumes no knowledge of
> > the project or necessarily of its field
> > *   A list of the three most important issues to address in the move
> > towards graduation.
> > *   Any issues that the Incubator PMC or ASF Board might wish/need to be
> > aware of
> > *   How has the community developed since the last report
> > *   How has the project developed since the last report.
> > *   How does the podling rate their own maturity.
> >
> > This should be appended to the Incubator Wiki page at:
> >
> > https://cwiki.apache.org/confluence/display/INCUBATOR/May2020
> >
> > Note: This is manually populated. You may need to wait a little before
> > this page is created from a template.
> >
> > Note: The format of the report has changed to use markdown.
> >
> > Mentors
> > ---
> >
> > Mentors should review reports for their project(s) and sign them off on
> > the Incubator wiki page. Signing off reports shows that you are
> > following the project - projects that are not signed may raise alarms
> > for the Incubator PMC.
> >
> > Incubator PMC
> >
>

5/5/2020 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi~

It has been two weeks since our last sync! Let's share our works for the
past two weeks in this thread.

My updates (not much):
- Podling report
- Fix docker images and testing a new release candidate.

Regards,
--ning

Re: Podling Heron Report Reminder - May 2020

[Ning Wang]

Thanks.

On Mon, Apr 20, 2020 at 3:32 PM Josh Fischer  wrote:

> Hi All,
>
> See the link below for the report template that needs to be filled out for
> this quarter.  It needs to be submitted by May  6, 2020.
>
>
> https://docs.google.com/document/d/1YxdPi2tf7mRirXQ86QOTyc7c_N52W_FrTkXyn_Lr_nY/edit?usp=sharing
>
> On Thu, Apr 16, 2020 at 3:06 AM  wrote:
>
> > Dear podling,
> >
> > This email was sent by an automated system on behalf of the Apache
> > Incubator PMC. It is an initial reminder to give you plenty of time to
> > prepare your quarterly board report.
> >
> > The board meeting is scheduled for Wed, 20 May 2020.
> > The report for your podling will form a part of the Incubator PMC
> > report. The Incubator PMC requires your report to be submitted 2 weeks
> > before the board meeting, to allow sufficient time for review and
> > submission (Wed, May 06).
> >
> > Please submit your report with sufficient time to allow the Incubator
> > PMC, and subsequently board members to review and digest. Again, the
> > very latest you should submit your report is 2 weeks prior to the board
> > meeting.
> >
> > Candidate names should not be made public before people are actually
> > elected, so please do not include the names of potential committers or
> > PPMC members in your report.
> >
> > Thanks,
> >
> > The Apache Incubator PMC
> >
> > Submitting your Report
> >
> > --
> >
> > Your report should contain the following:
> >
> > *   Your project name
> > *   A brief description of your project, which assumes no knowledge of
> > the project or necessarily of its field
> > *   A list of the three most important issues to address in the move
> > towards graduation.
> > *   Any issues that the Incubator PMC or ASF Board might wish/need to be
> > aware of
> > *   How has the community developed since the last report
> > *   How has the project developed since the last report.
> > *   How does the podling rate their own maturity.
> >
> > This should be appended to the Incubator Wiki page at:
> >
> > https://cwiki.apache.org/confluence/display/INCUBATOR/May2020
> >
> > Note: This is manually populated. You may need to wait a little before
> > this page is created from a template.
> >
> > Note: The format of the report has changed to use markdown.
> >
> > Mentors
> > ---
> >
> > Mentors should review reports for their project(s) and sign them off on
> > the Incubator wiki page. Signing off reports shows that you are
> > following the project - projects that are not signed may raise alarms
> > for the Incubator PMC.
> >
> > Incubator PMC
> >
>

4/21/2020 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi~

It has been two weeks since our last sync! Let's share our works for the
past two weeks in this thread.

My updates (not much):
- Code reviews
- Test the new Heron release.

Regards,
--ning

Re: [Result][Vote] Heron Release 0.20.3-incubating Release Candidate 2

[Ning Wang]

No worries~

On Mon, Apr 20, 2020 at 9:50 AM Josh Fischer  wrote:

> I think you are right.  My bad.
>
> On Mon, Apr 20, 2020 at 11:47 AM Ning Wang  wrote:
>
> > Thanks.
> >
> > BTW, I don't think our votes are "binding" votes. Only votes from mentors
> > are "binding" from my understanding.
> >
> > On Mon, Apr 20, 2020 at 5:44 AM Josh Fischer 
> wrote:
> >
> > > The vote has now closed.
> > >
> > > Results are:
> > >
> > > +1 [ 3 BINDING +1 VOTES]
> > >  0 [ 0 BINDING +0/-0 VOTES]
> > > -1 [ 1 BINDING -1 VOTES]
> > >
> > > The vote is unsuccessful
> > >
> > > - Josh
> > >
> >
> --
> Sent from A Mobile Device
>

Re: [Result][Vote] Heron Release 0.20.3-incubating Release Candidate 2

[Ning Wang]

Thanks.

BTW, I don't think our votes are "binding" votes. Only votes from mentors
are "binding" from my understanding.

On Mon, Apr 20, 2020 at 5:44 AM Josh Fischer  wrote:

> The vote has now closed.
>
> Results are:
>
> +1 [ 3 BINDING +1 VOTES]
>  0 [ 0 BINDING +0/-0 VOTES]
> -1 [ 1 BINDING -1 VOTES]
>
> The vote is unsuccessful
>
> - Josh
>

Re: [VOTE] Heron Release 0.20.3-incubating Release Candidate 2

[Ning Wang]

Ok. Thanks.

On Mon, Apr 20, 2020 at 5:41 AM Josh Fischer  wrote:

> Thanks for helping Ning
>
> I don't think there was ever a clear decision on which container to use.
> When we were talking about Debian9 as our official container we didn't have
> a Debian10 container.  Now that we have a Debian10 container we should use
> that I would think.  Plus all of the work that Nick put in with Bazel 3.0.0
> used Ubuntu 18.04 and Debian10 as a test bed.  I'll close this vote in a
> few minutes.
>
> - Josh
>
>
> On Mon, Apr 20, 2020 at 1:38 AM Ning Wang  wrote:
>
> > As we have decided before, Debian 9 is our main docker env. I think
> > successfully building the docker image should be required for our release
> > so I feel we should include the new docker fix in this release. Sorry for
> > the inconvenience.
> >
> > If it sounds good to you guys, I will prepare a new RC for voting after
> > verifying the new change (Josh is very busy these days).
> >
> > On Sun, Apr 19, 2020 at 10:22 AM Simon Weng 
> wrote:
> >
> > > +1
> > >
> > > On Sun, Apr 19, 2020 at 12:03 PM thinker0  wrote:
> > >
> > > > +1
> > > >
> > > > 2020년 4월 20일 (월) 오전 12:23, Josh Fischer 님이 작성:
> > > >
> > > > > I looked at the other files and you are right. This one is named
> > > > > differently.  The other ones have an "incubator" in the beginning
> of
> > > the
> > > > > file name.   I can change this and create a new release candidate.
> > > > >
> > > > > As for the binaries.  I figured we could vote on source first, once
> > > that
> > > > > passes then we could have another vote for convenience binaries. It
> > > will
> > > > > require an additional call for votes, but it's smaller chunks of
> work
> > > to
> > > > be
> > > > > done at one time.
> > > > >
> > > > > Let me know!
> > > > >
> > > > > On Sat, Apr 18, 2020 at 8:37 PM Ning Wang 
> > > wrote:
> > > > >
> > > > > > P <http://dist.apache.org/>lease rename the tar.gz file in the
> > > > > > dist.apache.org to
> > > > apache_heron-0.2.3-incubating-candidate-2-src.tar.gz.
> > > > > >
> > > > > > So we won't have binary this time? I am fine with it.
> > > > > >
> > > > > >
> > > > > > On Sat, Apr 18, 2020 at 9:43 AM Nicholas Nezis <
> > > > nicholas.ne...@gmail.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > +1 approve
> > > > > > >
> > > > > > > On Sat, Apr 18, 2020 at 10:42 AM Josh Fischer <
> > j...@joshfischer.io
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > Hello Heron Community,
> > > > > > > >
> > > > > > > > This is a call for a vote to the 2cnd release candidate for
> > > Apache
> > > > > > Heron,
> > > > > > > > version 0.20.3-incubating. We request project mentors
> (binded)
> > as
> > > > > well
> > > > > > as
> > > > > > > > all contributors (unbinded) and users to review and vote on
> > this
> > > > > > > incubator
> > > > > > > > release.
> > > > > > > >
> > > > > > > > *  The tag to be voted upon:
> > > > > > > > 0.20.3-incubating-rc2
> > (2a4f112fc567897711ee7117689e636534cb6fb4)
> > > > > > > > The full list of changes and release notes are available
> > > > > > > > at:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-heron/releases/tag/0.20.3-incubating-rc2
> > > > > > > >
> > > > > > > > Files can be found in dist.apache.org
> > > > > > > > Source:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.3-i

Re: [VOTE] Heron Release 0.20.3-incubating Release Candidate 2

[Ning Wang]

As we have decided before, Debian 9 is our main docker env. I think
successfully building the docker image should be required for our release
so I feel we should include the new docker fix in this release. Sorry for
the inconvenience.

If it sounds good to you guys, I will prepare a new RC for voting after
verifying the new change (Josh is very busy these days).

On Sun, Apr 19, 2020 at 10:22 AM Simon Weng  wrote:

> +1
>
> On Sun, Apr 19, 2020 at 12:03 PM thinker0  wrote:
>
> > +1
> >
> > 2020년 4월 20일 (월) 오전 12:23, Josh Fischer 님이 작성:
> >
> > > I looked at the other files and you are right. This one is named
> > > differently.  The other ones have an "incubator" in the beginning of
> the
> > > file name.   I can change this and create a new release candidate.
> > >
> > > As for the binaries.  I figured we could vote on source first, once
> that
> > > passes then we could have another vote for convenience binaries. It
> will
> > > require an additional call for votes, but it's smaller chunks of work
> to
> > be
> > > done at one time.
> > >
> > > Let me know!
> > >
> > > On Sat, Apr 18, 2020 at 8:37 PM Ning Wang 
> wrote:
> > >
> > > > P <http://dist.apache.org/>lease rename the tar.gz file in the
> > > > dist.apache.org to
> > apache_heron-0.2.3-incubating-candidate-2-src.tar.gz.
> > > >
> > > > So we won't have binary this time? I am fine with it.
> > > >
> > > >
> > > > On Sat, Apr 18, 2020 at 9:43 AM Nicholas Nezis <
> > nicholas.ne...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > +1 approve
> > > > >
> > > > > On Sat, Apr 18, 2020 at 10:42 AM Josh Fischer  >
> > > > wrote:
> > > > >
> > > > > > Hello Heron Community,
> > > > > >
> > > > > > This is a call for a vote to the 2cnd release candidate for
> Apache
> > > > Heron,
> > > > > > version 0.20.3-incubating. We request project mentors (binded) as
> > > well
> > > > as
> > > > > > all contributors (unbinded) and users to review and vote on this
> > > > > incubator
> > > > > > release.
> > > > > >
> > > > > > *  The tag to be voted upon:
> > > > > > 0.20.3-incubating-rc2 (2a4f112fc567897711ee7117689e636534cb6fb4)
> > > > > > The full list of changes and release notes are available
> > > > > > at:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-heron/releases/tag/0.20.3-incubating-rc2
> > > > > >
> > > > > > Files can be found in dist.apache.org
> > > > > > Source:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.3-incubating-candidate-2/
> > > > > >
> > > > > > SHA-512
> > > > > > checksums:
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 84a94d0c68581e230033a04c4539e8dabbd14461f75319c9caaa1bcbec288599fffc98c2928b520eb320deb9184b594d0e23eeb0521452cd014f21dda1776413
> > > > > >  heron-0.20.3-incubating-rc2.tar.gz
> > > > > >
> > > > > >
> > > > > > Please download the source package, and follow the compiling
> guide
> > > > > > https://heron.incubator.apache.org/docs/next/compiling-overview/
> > > > > > to setup the environment, build and run locally.
> > > > > >
> > > > > > Note that currently Bazel 3.0 is required to build this version.
> > > After
> > > > > > Bazel is set up and works correctly, you can use these release
> > check
> > > > > > scripts
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-heron/blob/master/scripts/release_check/README.md#run-individual-release-checks
> > > > > > to
> > > > > > verify the build.  The vote will be open for at least 72 hours or
> > > until
> > > > > the
> > > > > > necessary number of votes are reached.
> > > > > >
> > > > > > Please vote accordingly:
> > > > > > [ ] +1 approve
> > > > > > [ ] +0 no opinion
> > > > > > [ ] -1 disapprove with the reason
> > > > > >
> > > > > > Thank you,
> > > > > >
> > > > > > The Apache Heron (Incubating) Team
> > > > > >
> > > > >
> > > >
> > >
> >
> --
> Sent from Gmail Mobile
>

Re: [VOTE] Heron Release 0.20.3-incubating Release Candidate 2

[Ning Wang]

-1.

I am afraid that I need to vote -1 because the docker image build fails.

My steps:
$ sh scripts/release_check/full_release_check.sh
~/Downloads/apache-rat-0.12/apache-rat-0.12.jar
source code license check passed.
compiling done and installer works.

$ sh ./scripts/ci/build_release_packages.sh 0.20.3-incubating-rc2 debian9
./artifacts
building debian 9 docker image failed.

Test running an example topology is NOT done.

I believe thinker0's PR:
https://github.com/apache/incubator-heron/pull/3513 fixes
the issue. I am going to merge and test.





On Sat, Apr 18, 2020 at 7:42 AM Josh Fischer  wrote:

> Hello Heron Community,
>
> This is a call for a vote to the 2cnd release candidate for Apache Heron,
> version 0.20.3-incubating. We request project mentors (binded) as well as
> all contributors (unbinded) and users to review and vote on this incubator
> release.
>
> *  The tag to be voted upon:
> 0.20.3-incubating-rc2 (2a4f112fc567897711ee7117689e636534cb6fb4)
> The full list of changes and release notes are available
> at:
>
> https://github.com/apache/incubator-heron/releases/tag/0.20.3-incubating-rc2
>
> Files can be found in dist.apache.org
> Source:
>
> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.3-incubating-candidate-2/
>
> SHA-512
> checksums:
>
>
> 84a94d0c68581e230033a04c4539e8dabbd14461f75319c9caaa1bcbec288599fffc98c2928b520eb320deb9184b594d0e23eeb0521452cd014f21dda1776413
>  heron-0.20.3-incubating-rc2.tar.gz
>
>
> Please download the source package, and follow the compiling guide
> https://heron.incubator.apache.org/docs/next/compiling-overview/
> to setup the environment, build and run locally.
>
> Note that currently Bazel 3.0 is required to build this version.  After
> Bazel is set up and works correctly, you can use these release check
> scripts
>
> https://github.com/apache/incubator-heron/blob/master/scripts/release_check/README.md#run-individual-release-checks
> to
> verify the build.  The vote will be open for at least 72 hours or until the
> necessary number of votes are reached.
>
> Please vote accordingly:
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove with the reason
>
> Thank you,
>
> The Apache Heron (Incubating) Team
>

Re: [VOTE] Heron Release 0.20.3-incubating Release Candidate 2

[Ning Wang]

Ok. Thanks.

It may not be ideal to vote for the source and binary separately I feel,
since they belong to the same release. So if the vote for the binary fails,
we can't have the release.

That being said, I am fine if we keep this release source code only since
it has been a while and we need a release.


On Sun, Apr 19, 2020 at 8:23 AM Josh Fischer  wrote:

> I looked at the other files and you are right. This one is named
> differently.  The other ones have an "incubator" in the beginning of the
> file name.   I can change this and create a new release candidate.
>
> As for the binaries.  I figured we could vote on source first, once that
> passes then we could have another vote for convenience binaries. It will
> require an additional call for votes, but it's smaller chunks of work to be
> done at one time.
>
> Let me know!
>
> On Sat, Apr 18, 2020 at 8:37 PM Ning Wang  wrote:
>
> > P <http://dist.apache.org/>lease rename the tar.gz file in the
> > dist.apache.org to apache_heron-0.2.3-incubating-candidate-2-src.tar.gz.
> >
> > So we won't have binary this time? I am fine with it.
> >
> >
> > On Sat, Apr 18, 2020 at 9:43 AM Nicholas Nezis  >
> > wrote:
> >
> > > +1 approve
> > >
> > > On Sat, Apr 18, 2020 at 10:42 AM Josh Fischer 
> > wrote:
> > >
> > > > Hello Heron Community,
> > > >
> > > > This is a call for a vote to the 2cnd release candidate for Apache
> > Heron,
> > > > version 0.20.3-incubating. We request project mentors (binded) as
> well
> > as
> > > > all contributors (unbinded) and users to review and vote on this
> > > incubator
> > > > release.
> > > >
> > > > *  The tag to be voted upon:
> > > > 0.20.3-incubating-rc2 (2a4f112fc567897711ee7117689e636534cb6fb4)
> > > > The full list of changes and release notes are available
> > > > at:
> > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-heron/releases/tag/0.20.3-incubating-rc2
> > > >
> > > > Files can be found in dist.apache.org
> > > > Source:
> > > >
> > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.3-incubating-candidate-2/
> > > >
> > > > SHA-512
> > > > checksums:
> > > >
> > > >
> > > >
> > >
> >
> 84a94d0c68581e230033a04c4539e8dabbd14461f75319c9caaa1bcbec288599fffc98c2928b520eb320deb9184b594d0e23eeb0521452cd014f21dda1776413
> > > >  heron-0.20.3-incubating-rc2.tar.gz
> > > >
> > > >
> > > > Please download the source package, and follow the compiling guide
> > > > https://heron.incubator.apache.org/docs/next/compiling-overview/
> > > > to setup the environment, build and run locally.
> > > >
> > > > Note that currently Bazel 3.0 is required to build this version.
> After
> > > > Bazel is set up and works correctly, you can use these release check
> > > > scripts
> > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-heron/blob/master/scripts/release_check/README.md#run-individual-release-checks
> > > > to
> > > > verify the build.  The vote will be open for at least 72 hours or
> until
> > > the
> > > > necessary number of votes are reached.
> > > >
> > > > Please vote accordingly:
> > > > [ ] +1 approve
> > > > [ ] +0 no opinion
> > > > [ ] -1 disapprove with the reason
> > > >
> > > > Thank you,
> > > >
> > > > The Apache Heron (Incubating) Team
> > > >
> > >
> >
>

Re: [VOTE] Heron Release 0.20.3-incubating Release Candidate 2

[Ning Wang]

P lease rename the tar.gz file in the
dist.apache.org to apache_heron-0.2.3-incubating-candidate-2-src.tar.gz.

So we won't have binary this time? I am fine with it.


On Sat, Apr 18, 2020 at 9:43 AM Nicholas Nezis 
wrote:

> +1 approve
>
> On Sat, Apr 18, 2020 at 10:42 AM Josh Fischer  wrote:
>
> > Hello Heron Community,
> >
> > This is a call for a vote to the 2cnd release candidate for Apache Heron,
> > version 0.20.3-incubating. We request project mentors (binded) as well as
> > all contributors (unbinded) and users to review and vote on this
> incubator
> > release.
> >
> > *  The tag to be voted upon:
> > 0.20.3-incubating-rc2 (2a4f112fc567897711ee7117689e636534cb6fb4)
> > The full list of changes and release notes are available
> > at:
> >
> >
> https://github.com/apache/incubator-heron/releases/tag/0.20.3-incubating-rc2
> >
> > Files can be found in dist.apache.org
> > Source:
> >
> >
> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.3-incubating-candidate-2/
> >
> > SHA-512
> > checksums:
> >
> >
> >
> 84a94d0c68581e230033a04c4539e8dabbd14461f75319c9caaa1bcbec288599fffc98c2928b520eb320deb9184b594d0e23eeb0521452cd014f21dda1776413
> >  heron-0.20.3-incubating-rc2.tar.gz
> >
> >
> > Please download the source package, and follow the compiling guide
> > https://heron.incubator.apache.org/docs/next/compiling-overview/
> > to setup the environment, build and run locally.
> >
> > Note that currently Bazel 3.0 is required to build this version.  After
> > Bazel is set up and works correctly, you can use these release check
> > scripts
> >
> >
> https://github.com/apache/incubator-heron/blob/master/scripts/release_check/README.md#run-individual-release-checks
> > to
> > verify the build.  The vote will be open for at least 72 hours or until
> the
> > necessary number of votes are reached.
> >
> > Please vote accordingly:
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove with the reason
> >
> > Thank you,
> >
> > The Apache Heron (Incubating) Team
> >
>

Re: Recent workshop publication to appear with Heron related benchmarks

[Ning Wang]

Sounds interesting.

I would like to have a copy. And please let me know if there is any
question.

On Tue, Apr 14, 2020 at 1:40 PM Josh Fischer  wrote:

> Hi Aaron,
>
> I think your best resources would be:
>
> - Karthik Ramasamy
> - Ning Wang
> - Nick Nezis
> - Huijun Wu
>
> All should be subscribed to this list.  Hopefully you will hear back from a
> few of them soon.  If you don't hear from any of them please let me know,
> I'll see what I can do to help.
>
> - Josh
>
>
>
> On Tue, Apr 14, 2020 at 3:12 PM Aaron Harwood 
> wrote:
>
> >
> > Hi Heron Dev Team,
> >
> > I wasn't sure who to contact directly. We recently have a workshop paper
> > to appear that has comparison results between Heron 0.20.2-incubating,
> > Storm 2.1.0, and a new open source system of ours, also based on the
> > Storm API, called Dragon.
> >
> > We are wondering if there is someone we can consult regarding Heron, who
> > can look at the experimental setup for Heron to ensure that everything
> > is appropriately set, as we revise the experiments to make a journal
> > article. We would like to ensure that Heron is properly represented.
> >
> > If there is someone available for this purpose we will send a copy of
> > the paper above for perusal (not for wide distribution at this point)
> > and go from there. This will give a good idea of what we are doing.
> >
> > Many thanks,
> >
> > --aaron
> >
> > --
> >
> > Dr Aaron Harwood
> > School of Computing and Information Systems
> >
> > Melbourne School of Engineering
> > The University of Melbourne, Victoria 3010
> > T:+613 8344 1351 | E:aharw...@unimelb.edu.au
> > <mailto:aharw...@unimelb.edu.au>| W:people.eng.unimelb.edu.au/aharwood
> > <http://people.eng.unimelb.edu.au/aharwood>
> >
> > This email and any attachments may contain personal information or
> > information that is otherwise confidential or the subject of copyright.
> > Any use, disclosure or copying of any part of it is prohibited. The
> > University does not warrant that this email or any attachments are free
> > from viruses or defects. Please check any attachments for viruses and
> > defects before opening them. If this email is received in error please
> > delete it and notify us by return email.
> >
> >
>

Re: [DISCUSS] Next Heron release (0.20.3-incubating)

[Ning Wang]

Also I am hoping we can try to have binary packages included this time:
installers for Mac and Linux, and a docker image with a chosen linux
distribution.

On Fri, Apr 10, 2020 at 12:28 AM Ning Wang  wrote:

> Hi,
>
> It has been 5 months since the last Heron release and it is time for us to
> think about 0.20.3-incubating. Anything  we would like to finish before
> preparing the next release?
>
> --ning
>

[DISCUSS] Next Heron release (0.20.3-incubating)

[Ning Wang]

Hi,

It has been 5 months since the last Heron release and it is time for us to
think about 0.20.3-incubating. Anything  we would like to finish before
preparing the next release?

--ning

Re: [MENTORS] Create a Heron Space in Confluence

[Ning Wang]

Thanks!

On Fri, Apr 3, 2020 at 9:10 AM Dave Fisher  wrote:

> Done!
>
> > On Apr 3, 2020, at 9:09 AM, Ning Wang  wrote:
> >
> > Hi, Dave,
> >
> > Could you please add me as well? Thanks in advance!
> >
> > On Fri, Apr 3, 2020 at 8:39 AM Dave Fisher  wrote:
> >
> >> Hi -
> >>
> >> Apologies for the delay in responding. Both Josh and Nicholas have full
> >> permissions to the Confluence space.
> >>
> >> If others would like permission from the community then please confirm
> you
> >> login and respond to this dev list!
> >>
> >> Regards,
> >> Dave
> >>
> >>> On Mar 14, 2020, at 6:55 PM, Josh Fischer  wrote:
> >>>
> >>> It looks like the Heron space in Confluence is unrestricted but I do
> not
> >>> see a way to edit or create content in the space.  Anyone have an idea
> of
> >>> what needs to be done?
> >>>
> >>> - Josh
> >>>
> >>> On Mon, Feb 24, 2020 at 11:24 AM Josh Fischer 
> >> wrote:
> >>>
> >>>> I can see the Heron space in Confluence but I cannot see it as one of
> >> the
> >>>> spaces I can add documents to.  Is there something else that needs to
> be
> >>>> done?
> >>>>
> >>>> - Josh
> >>>>
> >>>> On Fri, Feb 14, 2020 at 2:14 PM Nicholas Nezis <
> >> nicholas.ne...@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> Just tried logging in. I was able to log in, but then got "Not
> >> Permitted"
> >>>>> when viewing the Heron space.
> >>>>>
> >>>>> On Fri, Feb 14, 2020 at 2:09 PM Josh Fischer 
> >> wrote:
> >>>>>
> >>>>>> Fantastic.  Thank you for the help Dave.
> >>>>>>
> >>>>>> On Fri, Feb 14, 2020 at 12:53 PM Dave Fisher 
> wrote:
> >>>>>>
> >>>>>>> The Space is created -
> >>>>> https://cwiki.apache.org/confluence/display/HERON
> >>>>>>> with Nick as admin.
> >>>>>>>
> >>>>>>>> On Feb 12, 2020, at 1:27 PM, Ning Wang 
> >>>>> wrote:
> >>>>>>>>
> >>>>>>>> sgtm either way.
> >>>>>>>>
> >>>>>>>> On Wed, Feb 12, 2020 at 1:19 PM Josh Fischer  >
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> If Nick has the experience, I'd say let him do it.
> >>>>>>>>>
> >>>>>>>>> - Josh
> >>>>>>>>>
> >>>>>>>>> On Wed, Feb 12, 2020 at 3:16 PM Nicholas Nezis <
> >>>>>>> nicholas.ne...@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> I also can help. Have a bit of experience from using Confluence
> at
> >>>>>>> work.
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Feb 12, 2020 at 4:01 PM Josh Fischer <
> j...@joshfischer.io
> >>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> I did some digging around there today to understand enough that
> >>>>> we
> >>>>>>>>>> should
> >>>>>>>>>>> have our own space.
> >>>>>>>>>>>
> >>>>>>>>>>> The rest I can figure out as we go.
> >>>>>>>>>>>
> >>>>>>>>>>> I hope that works for you.
> >>>>>>>>>>>
> >>>>>>>>>>> - Josh
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Feb 12, 2020 at 2:52 PM Dave Fisher 
> >>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi -
> >>>>>>>>>>>>
> >>>>>>>>>>>> I can make this request, but first who understands Confluence
> >>>>>> enough
> >>>>>>>>> to
> >>>>>>>>>>> be
> >>>>>>>>>>>> the Space Admin?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Regards,
> >>>>>>>>>>>> Dave
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On Feb 12, 2020, at 12:50 PM, SiMing Weng <
> >>>>> siming.w...@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> +1
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Feb 12, 2020, at 2:15 PM, Ning Wang <
> wangnin...@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sounds like a great idea to me~
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, Feb 12, 2020 at 11:14 AM Josh Fischer <
> >>>>>>>>> j...@joshfischer.io>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Could we get a space created in confluence for the Heron
> >>>>>> project?
> >>>>>>>>>>> I'd
> >>>>>>>>>>>> like
> >>>>>>>>>>>>>>> to add some material there for the community.  I think it
> >>>>> can be
> >>>>>>>>>> done
> >>>>>>>>>>>> here:
> >>>>>>>>>>>>>>> https://selfserve.apache.org/
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> - Josh
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>> Sent from A Mobile Device
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> >>
>
>

Re: our first virtual event this evening

[Ning Wang]

It went well. Huijun presented and we had some discussions afterwards. I
remember around 7 or 8 people was there?

On Fri, Apr 3, 2020 at 8:39 AM Dave Fisher  wrote:

> Hi Sree,
>
> How did this meeting go?
>
> Regards,
> Dave
>
> > On Mar 23, 2020, at 12:08 PM, Sree Vaddi 
> wrote:
> >
> > Hi Herons,
> > I request you to join our first virtual event and help make it
> successful.
> > https://www.meetup.com/Apache-Heron-Bay-Area/events/269361401/
> >
> >
> > Thank you./Sree
> >
>
>

Re: [MENTORS] Create a Heron Space in Confluence

[Ning Wang]

Hi, Dave,

Could you please add me as well? Thanks in advance!

On Fri, Apr 3, 2020 at 8:39 AM Dave Fisher  wrote:

> Hi -
>
> Apologies for the delay in responding. Both Josh and Nicholas have full
> permissions to the Confluence space.
>
> If others would like permission from the community then please confirm you
> login and respond to this dev list!
>
> Regards,
> Dave
>
> > On Mar 14, 2020, at 6:55 PM, Josh Fischer  wrote:
> >
> > It looks like the Heron space in Confluence is unrestricted but I do not
> > see a way to edit or create content in the space.  Anyone have an idea of
> > what needs to be done?
> >
> > - Josh
> >
> > On Mon, Feb 24, 2020 at 11:24 AM Josh Fischer 
> wrote:
> >
> >> I can see the Heron space in Confluence but I cannot see it as one of
> the
> >> spaces I can add documents to.  Is there something else that needs to be
> >> done?
> >>
> >> - Josh
> >>
> >> On Fri, Feb 14, 2020 at 2:14 PM Nicholas Nezis <
> nicholas.ne...@gmail.com>
> >> wrote:
> >>
> >>> Just tried logging in. I was able to log in, but then got "Not
> Permitted"
> >>> when viewing the Heron space.
> >>>
> >>> On Fri, Feb 14, 2020 at 2:09 PM Josh Fischer 
> wrote:
> >>>
> >>>> Fantastic.  Thank you for the help Dave.
> >>>>
> >>>> On Fri, Feb 14, 2020 at 12:53 PM Dave Fisher  wrote:
> >>>>
> >>>>> The Space is created -
> >>> https://cwiki.apache.org/confluence/display/HERON
> >>>>> with Nick as admin.
> >>>>>
> >>>>>> On Feb 12, 2020, at 1:27 PM, Ning Wang 
> >>> wrote:
> >>>>>>
> >>>>>> sgtm either way.
> >>>>>>
> >>>>>> On Wed, Feb 12, 2020 at 1:19 PM Josh Fischer 
> >>>>> wrote:
> >>>>>>
> >>>>>>> If Nick has the experience, I'd say let him do it.
> >>>>>>>
> >>>>>>> - Josh
> >>>>>>>
> >>>>>>> On Wed, Feb 12, 2020 at 3:16 PM Nicholas Nezis <
> >>>>> nicholas.ne...@gmail.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> I also can help. Have a bit of experience from using Confluence at
> >>>>> work.
> >>>>>>>>
> >>>>>>>> On Wed, Feb 12, 2020 at 4:01 PM Josh Fischer  >>>>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> I did some digging around there today to understand enough that
> >>> we
> >>>>>>>> should
> >>>>>>>>> have our own space.
> >>>>>>>>>
> >>>>>>>>> The rest I can figure out as we go.
> >>>>>>>>>
> >>>>>>>>> I hope that works for you.
> >>>>>>>>>
> >>>>>>>>> - Josh
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, Feb 12, 2020 at 2:52 PM Dave Fisher 
> >>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi -
> >>>>>>>>>>
> >>>>>>>>>> I can make this request, but first who understands Confluence
> >>>> enough
> >>>>>>> to
> >>>>>>>>> be
> >>>>>>>>>> the Space Admin?
> >>>>>>>>>>
> >>>>>>>>>> Regards,
> >>>>>>>>>> Dave
> >>>>>>>>>>
> >>>>>>>>>>> On Feb 12, 2020, at 12:50 PM, SiMing Weng <
> >>> siming.w...@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> +1
> >>>>>>>>>>>
> >>>>>>>>>>>> On Feb 12, 2020, at 2:15 PM, Ning Wang 
> >>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Sounds like a great idea to me~
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Feb 12, 2020 at 11:14 AM Josh Fischer <
> >>>>>>> j...@joshfischer.io>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Could we get a space created in confluence for the Heron
> >>>> project?
> >>>>>>>>> I'd
> >>>>>>>>>> like
> >>>>>>>>>>>>> to add some material there for the community.  I think it
> >>> can be
> >>>>>>>> done
> >>>>>>>>>> here:
> >>>>>>>>>>>>> https://selfserve.apache.org/
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> - Josh
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>> Sent from A Mobile Device
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>

Re: 3/24/2020 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

And everyone stays safe in this crazy time!

On Mon, Mar 23, 2020 at 2:25 PM Ning Wang  wrote:

> Hi~
>
> It has been two weeks since our last sync! Let's share our works for the
> last two weeks in this thread.
>
> My updates (not much):
> - Code reviews and discussions
>
> Regards,
> --ning
>

3/24/2020 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi~

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates (not much):
- Code reviews and discussions

Regards,
--ning

3/10/2020 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi~

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
- Add documentation of docker based dev environment.
- Code reviews

Regards,
--ning

Re: stmgr coredump

[Ning Wang]

Ok. The log shows that the connection is being closed. So it just looks
like a network issue causing the connection to be unstable.

Looks like a racing condition to me if this theory is correct. You may need
to review the object and add a mutex around the critical sections.

On Tue, Mar 3, 2020 at 10:07 AM Xiaoyao Qian  wrote:

> During normal running state. And it happens only to one data center..
>
> On Tue, Mar 3, 2020 at 9:33 AM Ning Wang  wrote:
>
>> Yeah. Segfault is more likely in this case. But maybe not for the object
>> itself but for the vtable pointer.
>>
>> Pure function call might be possible though I think. It all depends on
>> the 4 bytes in the address. It could happen If a new object of a different
>> type has been created in the place or an object is moved to the memory
>> block.
>>
>> More investigation is necessary.
>>
>> When does it happen? normal running state or when topo or instance is
>> shutting down?
>>
>>
>> On Tue, Mar 3, 2020 at 8:34 AM Xiaoyao Qian  wrote:
>>
>>> I agreed with the second possibility, but wouldn’t it cause segfault if
>>> the object has been deleted but the method is still invoked?
>>>
>>> On Tue, Mar 3, 2020 at 01:16 Ning Wang  wrote:
>>>
>>>> Looks like the exception is caused by pure virtual function calling.
>>>>
>>>> Both exceptions are from BaseClient. It seems like the Client object
>>>> doesn't have the virtual functions implemented, which is not expected.
>>>>
>>>> Another possibility is that the client object has been deleted hence
>>>> the vtable is not valid any more. This could be something you can check
>>>> given the last log shows "Stmgr stmgr-12 closed connection".
>>>>
>>>>
>>>> On Mon, Mar 2, 2020 at 12:20 PM Huijun Wu 
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> We observed the Stmgr coredump, see the below logs. Anybody has any
>>>>> idea on
>>>>> this? Thanks.
>>>>>
>>>>> Best,
>>>>> Huijun
>>>>>
>>>>> 
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: pure virtual
>>>>> method
>>>>> called
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: terminate called
>>>>> without an active exception
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: *** Aborted at
>>>>> 1582847383 (unix time) try "date -d @1582847383" if you are using GNU
>>>>> date
>>>>> ***
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: PC: @
>>>>> 0x7f5bc42da277 __GI_raise
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: *** SIGABRT
>>>>> (@0xbcbb338) received by PID 45880 (TID 0x7f5bc54c1780) from PID
>>>>> 45880;
>>>>> stack trace: ***
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>>>> 0x7f5bc50a76d0 (unknown)
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>>>> 0x7f5bc42da277 __GI_raise
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>>>> 0x7f5bc42db968 __GI_abort
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>>>> 0x7f5bc48e77d5 __gnu_cxx::__verbose_terminate_handler()
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>>>> 0x7f5bc48e5746 (unknown)
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>>>> 0x7f5bc48e5773 std::terminate()
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>>>> 0x7f5bc48e62df __cxa_pure_virtual
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>>>> 0x4dfe1d BaseClient::OnConnect()
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>>>> 0x4de404 EventLoopImpl::handleWriteCallback()
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>>>> 0x4f0a9f event_process_active_single_queue
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>>>> 0x4f115f event_base_loop
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>>>> 0x40c81f main
>>>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>>>> 0x7

Re: stmgr coredump

[Ning Wang]

Yeah. Segfault is more likely in this case. But maybe not for the object
itself but for the vtable pointer.

Pure function call might be possible though I think. It all depends on the
4 bytes in the address. It could happen If a new object of a different type
has been created in the place or an object is moved to the memory block.

More investigation is necessary.

When does it happen? normal running state or when topo or instance is
shutting down?


On Tue, Mar 3, 2020 at 8:34 AM Xiaoyao Qian  wrote:

> I agreed with the second possibility, but wouldn’t it cause segfault if
> the object has been deleted but the method is still invoked?
>
> On Tue, Mar 3, 2020 at 01:16 Ning Wang  wrote:
>
>> Looks like the exception is caused by pure virtual function calling.
>>
>> Both exceptions are from BaseClient. It seems like the Client object
>> doesn't have the virtual functions implemented, which is not expected.
>>
>> Another possibility is that the client object has been deleted hence the
>> vtable is not valid any more. This could be something you can check given
>> the last log shows "Stmgr stmgr-12 closed connection".
>>
>>
>> On Mon, Mar 2, 2020 at 12:20 PM Huijun Wu 
>> wrote:
>>
>>> Hi,
>>>
>>> We observed the Stmgr coredump, see the below logs. Anybody has any idea
>>> on
>>> this? Thanks.
>>>
>>> Best,
>>> Huijun
>>>
>>> 
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: pure virtual method
>>> called
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: terminate called
>>> without an active exception
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: *** Aborted at
>>> 1582847383 (unix time) try "date -d @1582847383" if you are using GNU
>>> date
>>> ***
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: PC: @
>>> 0x7f5bc42da277 __GI_raise
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: *** SIGABRT
>>> (@0xbcbb338) received by PID 45880 (TID 0x7f5bc54c1780) from PID
>>> 45880;
>>> stack trace: ***
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>> 0x7f5bc50a76d0 (unknown)
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>> 0x7f5bc42da277 __GI_raise
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>> 0x7f5bc42db968 __GI_abort
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>> 0x7f5bc48e77d5 __gnu_cxx::__verbose_terminate_handler()
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>> 0x7f5bc48e5746 (unknown)
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>> 0x7f5bc48e5773 std::terminate()
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>> 0x7f5bc48e62df __cxa_pure_virtual
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>> 0x4dfe1d BaseClient::OnConnect()
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>> 0x4de404 EventLoopImpl::handleWriteCallback()
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>> 0x4f0a9f event_process_active_single_queue
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>> 0x4f115f event_base_loop
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>> 0x40c81f main
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>> 0x7f5bc42c6445 __libc_start_main
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>> 0x41257c (unknown)
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>>>  0x0 (unknown)
>>> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout:
>>> 
>>> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: pure virtual method
>>> called
>>> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: terminate called
>>> without an active exception
>>> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: *** Aborted at
>>> 1582837876 (unix time) try "date -d @1582837876" if you are using GNU
>>> date
>>> ***
>>> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: PC: @
>>> 0x7fe9d8935277 __GI_raise
>>> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: *** SIGABRT
>>> (@0xbcb0005176a) received by PID 333674 (TID 0x7fe9d9b1c780) from PID
>>> 333674; stack trace: ***
>>> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
>>> 0x

Re: stmgr coredump

[Ning Wang]

Looks like the exception is caused by pure virtual function calling.

Both exceptions are from BaseClient. It seems like the Client object
doesn't have the virtual functions implemented, which is not expected.

Another possibility is that the client object has been deleted hence the
vtable is not valid any more. This could be something you can check given
the last log shows "Stmgr stmgr-12 closed connection".


On Mon, Mar 2, 2020 at 12:20 PM Huijun Wu  wrote:

> Hi,
>
> We observed the Stmgr coredump, see the below logs. Anybody has any idea on
> this? Thanks.
>
> Best,
> Huijun
>
> 
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: pure virtual method
> called
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: terminate called
> without an active exception
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: *** Aborted at
> 1582847383 (unix time) try "date -d @1582847383" if you are using GNU date
> ***
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: PC: @
> 0x7f5bc42da277 __GI_raise
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: *** SIGABRT
> (@0xbcbb338) received by PID 45880 (TID 0x7f5bc54c1780) from PID 45880;
> stack trace: ***
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
> 0x7f5bc50a76d0 (unknown)
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
> 0x7f5bc42da277 __GI_raise
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
> 0x7f5bc42db968 __GI_abort
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
> 0x7f5bc48e77d5 __gnu_cxx::__verbose_terminate_handler()
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
> 0x7f5bc48e5746 (unknown)
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
> 0x7f5bc48e5773 std::terminate()
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
> 0x7f5bc48e62df __cxa_pure_virtual
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
> 0x4dfe1d BaseClient::OnConnect()
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
> 0x4de404 EventLoopImpl::handleWriteCallback()
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
> 0x4f0a9f event_process_active_single_queue
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
> 0x4f115f event_base_loop
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
> 0x40c81f main
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
> 0x7f5bc42c6445 __libc_start_main
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
> 0x41257c (unknown)
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout: @
>  0x0 (unknown)
> [2020-02-27 23:49:43 +] [INFO]: stmgr-233 stdout:
> 
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: pure virtual method
> called
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: terminate called
> without an active exception
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: *** Aborted at
> 1582837876 (unix time) try "date -d @1582837876" if you are using GNU date
> ***
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: PC: @
> 0x7fe9d8935277 __GI_raise
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: *** SIGABRT
> (@0xbcb0005176a) received by PID 333674 (TID 0x7fe9d9b1c780) from PID
> 333674; stack trace: ***
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x7fe9d97026d0 (unknown)
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x7fe9d8935277 __GI_raise
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x7fe9d8936968 __GI_abort
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x7fe9d8f427d5 __gnu_cxx::__verbose_terminate_handler()
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x7fe9d8f40746 (unknown)
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x7fe9d8f40773 std::terminate()
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x7fe9d8f412df __cxa_pure_virtual
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x4df98a BaseClient::Start_Base()
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x443dfa heron::stmgr::StMgrClient::OnReConnectTimer()
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x4df6ed
>
> _ZNSt17_Function_handlerIFvN9EventLoop6StatusEEZN10BaseClient13AddTimer_BaseESt8functionIFvvEElEUlS1_E_E9_M_invokeERKSt9_Any_dataS1_
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x4ded81 EventLoopImpl::handleTimerCallback()
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x4dc840 EventLoopImpl::eventLoopImplTimerCallback()
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x4f0a9f event_process_active_single_queue
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x4f115f event_base_loop
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x40c81f main
> [2020-02-27 21:11:16 +] [INFO]: stmgr-233 stdout: @
> 0x7fe9d8921445 __libc_start_main
> [2020-02-27 

2/25/2020 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi~

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
- Prepare Heron meetup events and slides.
- Improve Streamlet to allow sink objects to set name and partition.
- Update documents.

Regards,
--ning

Re: [MENTORS] Create a Heron Space in Confluence

[Ning Wang]

sgtm either way.

On Wed, Feb 12, 2020 at 1:19 PM Josh Fischer  wrote:

> If Nick has the experience, I'd say let him do it.
>
> - Josh
>
> On Wed, Feb 12, 2020 at 3:16 PM Nicholas Nezis 
> wrote:
>
> > I also can help. Have a bit of experience from using Confluence at work.
> >
> > On Wed, Feb 12, 2020 at 4:01 PM Josh Fischer 
> wrote:
> >
> > >  I did some digging around there today to understand enough that we
> > should
> > > have our own space.
> > >
> > > The rest I can figure out as we go.
> > >
> > > I hope that works for you.
> > >
> > > - Josh
> > >
> > >
> > > On Wed, Feb 12, 2020 at 2:52 PM Dave Fisher  wrote:
> > >
> > > > Hi -
> > > >
> > > > I can make this request, but first who understands Confluence enough
> to
> > > be
> > > > the Space Admin?
> > > >
> > > > Regards,
> > > > Dave
> > > >
> > > > > On Feb 12, 2020, at 12:50 PM, SiMing Weng 
> > > wrote:
> > > > >
> > > > > +1
> > > > >
> > > > >> On Feb 12, 2020, at 2:15 PM, Ning Wang 
> > wrote:
> > > > >>
> > > > >> Sounds like a great idea to me~
> > > > >>
> > > > >> On Wed, Feb 12, 2020 at 11:14 AM Josh Fischer <
> j...@joshfischer.io>
> > > > wrote:
> > > > >>
> > > > >>> Hi,
> > > > >>>
> > > > >>> Could we get a space created in confluence for the Heron project?
> > > I'd
> > > > like
> > > > >>> to add some material there for the community.  I think it can be
> > done
> > > > here:
> > > > >>> https://selfserve.apache.org/
> > > > >>>
> > > > >>> - Josh
> > > > >>>
> > > > >
> > > >
> > > > --
> > > Sent from A Mobile Device
> > >
> >
>

2/11/2020 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi~

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
- Prepare for the Feb meetup in San Francisco downtown (Amplitude office).
- Replace netcat in Debian docker.
- Improve docker based dev environment.
- Update JSXTransformer.js related licenses.

Regards,
--ning

Re: [DISCUSS] Binary Release Questions

[Ning Wang]

I don't think vim is needed in the dist container. netcat is used in a few
ZK related scripts checking if zk is running.

On Tue, Feb 4, 2020 at 6:56 PM Josh Fischer  wrote:

> Hi All,
>
> After doing some research into the Debian container.. I've listed the
> packages that are installed.  I've also included links to licenses or
> copyrights and my notes based on what I could find.
>
> Installed packages are:
>
> netcat
>
> vim
>
> python
>
> supervisor
>
> curl
>
> unzip
>
>
> My notes below
>
>
>-
>
>Netcat - GPL (Probably a no go)
>-
>
>Vim - GPL  (Probably a no go)
>-
>
>Python - I think we are good here
>-
>
>Supervisor
>
> https://raw.githubusercontent.com/Supervisor/supervisor/master/COPYRIGHT.txt
>(I’m not sure. The link in the file to the license is broken
>-
>
>Curl - https://raw.githubusercontent.com/curl/curl/master/COPYING ( I
>think we are good)
>-
>
>Unzip: http://infozip.sourceforge.net/license.html
>
>
>-
>
>Notes from the page It's basically BSD-like, but note that there may
>still be a few remaining files in some of the packages that are covered
> by
>different licenses. ( I think we are good?)
>
>
> My assumption is that we will have to remove vim (no big deal) and netcat
> at least before we can create binaries and send out a vote. I'm not sure
> about supervisor, curl, or unzip. Questions, comments, concerns?
>
> - Josh
>
> On Mon, Feb 3, 2020 at 5:55 PM Ning Wang  wrote:
>
> > Very helpful!
> >
> > On Mon, Feb 3, 2020 at 12:44 PM Dave Fisher  wrote:
> >
> > > Hi -
> > >
> > > Regarding OpenJDK and GPL2 - here is what Roman the VP, Legal wrote
> when
> > > answering Beam’s questions.
> > >
> > > Roman Shaposhnik commented on LEGAL-503:
> > > 
> > >
> > > Hey [~altay] if you would like to continue linking to the Docker
> release
> > > artifact from the
> > > https://beam.apache.org
> > >  you will have:
> > >1. Transition to the official ASF dockerhub org:
> > > https://hub.docker.com/u/apache
> > >2. Start including that binary convenience artifact into your VOTE
> > > threads on Beam releases
> > >3. Make sure that all Cat-X licenses are ONLY brought into your
> > > container via FROM statements
> > >
> > > Regards,
> > > Dave
> > >
> > > > On Feb 3, 2020, at 12:34 PM, Josh Fischer 
> wrote:
> > > >
> > > > I think what Nick is talking about is Travis CI, not Jenkins.  It
> > seems
> > > > that ubuntu LTS may have some issues with creating the container.
> >  This
> > > > may put a lot of work ahead of us.  My first thought is to use Debian
> > as
> > > > the "official" Heron container for apache.  It is built from the
> > > > openjdk:8-jdk-slim  docker image.  I know that Tomcat uses a
> > > > similar container from the openjdk org.  It might be the path of
> least
> > > > resistance when it comes to making sure the licenses are ok.  But a
> > > > possible hang up with the openjdk container is that is uses a GPL2
> > > license
> > > > which is not compatible with Apache (this is my understanding).  So
> I'm
> > > > thinking of running the questions about the Debian container to
> legal.
> > > > Any thoughts?
> > > >
> > > > On Mon, Feb 3, 2020 at 12:48 PM Ning Wang 
> > wrote:
> > > >
> > > >> I am fine with keeping the docker files.
> > > >>
> > > >> It is a good point that Jenkins machine is a factor.
> > > >>
> > > >> On Mon, Feb 3, 2020 at 9:27 AM Nicholas Nezis <
> > nicholas.ne...@gmail.com
> > > >
> > > >> wrote:
> > > >>
> > > >>> Linux and MacOS installer definitely. For the MacOS users, it would
> > be
> > > >>> awesome to maintain being able to `brew install heron`
> > > >>>
> > > >>> As to the images, I definitely think it would be better to keep a
> > > smaller
> > > >>> set of Dockerfiles. With my move to Bazel 2.0 it has been painful
> > > working
> > > >>> through the various build issues related to the different OS
> builds.
> > If
> > > >> the
> > > >>> Dockerfiles are meant to provid

Re: [DISCUSS] Binary Release Questions

[Ning Wang]

Very helpful!

On Mon, Feb 3, 2020 at 12:44 PM Dave Fisher  wrote:

> Hi -
>
> Regarding OpenJDK and GPL2 - here is what Roman the VP, Legal wrote when
> answering Beam’s questions.
>
> Roman Shaposhnik commented on LEGAL-503:
> 
>
> Hey [~altay] if you would like to continue linking to the Docker release
> artifact from the
> https://beam.apache.org
>  you will have:
>1. Transition to the official ASF dockerhub org:
> https://hub.docker.com/u/apache
>2. Start including that binary convenience artifact into your VOTE
> threads on Beam releases
>3. Make sure that all Cat-X licenses are ONLY brought into your
> container via FROM statements
>
> Regards,
> Dave
>
> > On Feb 3, 2020, at 12:34 PM, Josh Fischer  wrote:
> >
> > I think what Nick is talking about is Travis CI, not Jenkins.  It  seems
> > that ubuntu LTS may have some issues with creating the container.   This
> > may put a lot of work ahead of us.  My first thought is to use Debian as
> > the "official" Heron container for apache.  It is built from the
> > openjdk:8-jdk-slim  docker image.  I know that Tomcat uses a
> > similar container from the openjdk org.  It might be the path of least
> > resistance when it comes to making sure the licenses are ok.  But a
> > possible hang up with the openjdk container is that is uses a GPL2
> license
> > which is not compatible with Apache (this is my understanding).  So I'm
> > thinking of running the questions about the Debian container to legal.
> > Any thoughts?
> >
> > On Mon, Feb 3, 2020 at 12:48 PM Ning Wang  wrote:
> >
> >> I am fine with keeping the docker files.
> >>
> >> It is a good point that Jenkins machine is a factor.
> >>
> >> On Mon, Feb 3, 2020 at 9:27 AM Nicholas Nezis  >
> >> wrote:
> >>
> >>> Linux and MacOS installer definitely. For the MacOS users, it would be
> >>> awesome to maintain being able to `brew install heron`
> >>>
> >>> As to the images, I definitely think it would be better to keep a
> smaller
> >>> set of Dockerfiles. With my move to Bazel 2.0 it has been painful
> working
> >>> through the various build issues related to the different OS builds. If
> >> the
> >>> Dockerfiles are meant to provide people with the list of packages for
> >> their
> >>> local install, perhaps maintaining them is ok. If the goal is to have
> an
> >>> isolated build container and runtime container, then having a single
> >> option
> >>> makes more sense.
> >>>
> >>> Short term:
> >>> Ubuntu 14.04 is used in the Travis CI build so having them be
> consistent
> >>> makes sense to me. This is the image I would focus on.
> >>>
> >>> Long term:
> >>> We should update things to use a newer Ubuntu LTS version if possible.
> >>> There are some issues that might be blockers:
> >>> - cppcheck doesn't compile on Ubuntu 18.04 (
> >>> https://github.com/apache/incubator-heron/issues/3440)
> >>> - TravisCI expects JDK 9+ on Ubuntu 16+
> >>> - DNS issue with Ubuntu in Kubernetes (
> >>>
> >>>
> >>
> https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/#known-issues
> >>> )
> >>>
> >>>
> >>> On Mon, Feb 3, 2020 at 11:58 AM Ning Wang 
> wrote:
> >>>
> >>>> For installer, I feel that MacOS should be included.
> >>>> For docker images, we may choose one to release. I don't really have a
> >>>> preference. Maybe market share is a good indicator. I think Ubuntu was
> >>> #1 a
> >>>> few years ago, but I am not sure what is the current case.
> >>>>
> >>>> So overall my vote would be,
> >>>> docker image: ubuntu or current #1 market share wise if we can find
> the
> >>>> information.
> >>>> installer: MacOS + the same OS as the docker image.
> >>>>
> >>>>
> >>>> On Mon, Feb 3, 2020 at 5:02 AM Josh Fischer 
> >> wrote:
> >>>>
> >>>>> Any thoughts on this email?
> >>>>>
> >>>>> Hi All,
> >>>>>
> >>>>> After  several conversations with people across the Heron repo we
> >> keep
> >>>>> hearing that a Heron convenience binary release would be appreciated.
> >>>>&g

Re: [DISCUSS] Removal of old website

[Ning Wang]

Sounds good to me. Thanks!

On Mon, Feb 3, 2020 at 6:49 AM Josh Fischer  wrote:

> Hi All,
>
> I'm thinking of removing the old Heron website folder from the repo.  Does
> anyone have concerns with this?
>
> - Josh
>

Re: [DISCUSS] Binary Release Questions

[Ning Wang]

I am fine with keeping the docker files.

It is a good point that Jenkins machine is a factor.

On Mon, Feb 3, 2020 at 9:27 AM Nicholas Nezis 
wrote:

> Linux and MacOS installer definitely. For the MacOS users, it would be
> awesome to maintain being able to `brew install heron`
>
> As to the images, I definitely think it would be better to keep a smaller
> set of Dockerfiles. With my move to Bazel 2.0 it has been painful working
> through the various build issues related to the different OS builds. If the
> Dockerfiles are meant to provide people with the list of packages for their
> local install, perhaps maintaining them is ok. If the goal is to have an
> isolated build container and runtime container, then having a single option
> makes more sense.
>
> Short term:
> Ubuntu 14.04 is used in the Travis CI build so having them be consistent
> makes sense to me. This is the image I would focus on.
>
> Long term:
> We should update things to use a newer Ubuntu LTS version if possible.
> There are some issues that might be blockers:
> - cppcheck doesn't compile on Ubuntu 18.04 (
> https://github.com/apache/incubator-heron/issues/3440)
> - TravisCI expects JDK 9+ on Ubuntu 16+
> - DNS issue with Ubuntu in Kubernetes (
>
> https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/#known-issues
> )
>
>
> On Mon, Feb 3, 2020 at 11:58 AM Ning Wang  wrote:
>
> > For installer, I feel that MacOS should be included.
> > For docker images, we may choose one to release. I don't really have a
> > preference. Maybe market share is a good indicator. I think Ubuntu was
> #1 a
> > few years ago, but I am not sure what is the current case.
> >
> > So overall my vote would be,
> > docker image: ubuntu or current #1 market share wise if we can find the
> > information.
> > installer: MacOS + the same OS as the docker image.
> >
> >
> > On Mon, Feb 3, 2020 at 5:02 AM Josh Fischer  wrote:
> >
> > > Any thoughts on this email?
> > >
> > > Hi All,
> > >
> > > After  several conversations with people across the Heron repo we keep
> > > hearing that a Heron convenience binary release would be appreciated.
> > > Based on some feedback from Dave we need to decide on what type of
> > > packaging is helpful to Heron users as the first step to getting people
> > > what they want/need.
> > >
> > > Right now we have, but not released in a while:
> > >
> > > Heron Docker Containers:
> > > - CentOS
> > > - Ubuntu
> > > - Debian
> > > Heron install scripts
> > > - CentOS
> > > - Darwin (MacOs)
> > > - Ubuntu
> > >
> > > Does anyone have a preference on which package and distro they would
> like
> > > us to start with?  If possible, I would like us to scope down to one
> > > supported docker image to use for Heron.  Maintaining 3 separate images
> > is
> > > quite a task.
> > >
> > > On Tue, Jan 28, 2020 at 4:51 PM Josh Fischer 
> > wrote:
> > >
> > > > Hi All,
> > > >
> > > > After  several conversations with people across the Heron repo we
> keep
> > > > hearing that a Heron convenience binary release would be appreciated.
> > > > Based on some feedback from Dave we need to decide on what type of
> > > > packaging is helpful to Heron users as the first step to getting
> people
> > > > what they want/need.
> > > >
> > > > Right now we have, but not released in a while:
> > > >
> > > > Heron Docker Containers:
> > > > - CentOS
> > > > - Ubuntu
> > > > - Debian
> > > > Heron install scripts
> > > > - CentOS
> > > > - Darwin (MacOs)
> > > > - Ubuntu
> > > >
> > > > Does anyone have a preference on which package and distro they would
> > like
> > > > us to start with?  If possible, I would like us to scope down to one
> > > > supported docker image to use for Heron.  Maintaining 3 separate
> images
> > > is
> > > > quite a task.
> > > >
> > > > - Josh
> > > >
> > >
> >
>

Re: [DISCUSS] Binary Release Questions

[Ning Wang]

For installer, I feel that MacOS should be included.
For docker images, we may choose one to release. I don't really have a
preference. Maybe market share is a good indicator. I think Ubuntu was #1 a
few years ago, but I am not sure what is the current case.

So overall my vote would be,
docker image: ubuntu or current #1 market share wise if we can find the
information.
installer: MacOS + the same OS as the docker image.


On Mon, Feb 3, 2020 at 5:02 AM Josh Fischer  wrote:

> Any thoughts on this email?
>
> Hi All,
>
> After  several conversations with people across the Heron repo we keep
> hearing that a Heron convenience binary release would be appreciated.
> Based on some feedback from Dave we need to decide on what type of
> packaging is helpful to Heron users as the first step to getting people
> what they want/need.
>
> Right now we have, but not released in a while:
>
> Heron Docker Containers:
> - CentOS
> - Ubuntu
> - Debian
> Heron install scripts
> - CentOS
> - Darwin (MacOs)
> - Ubuntu
>
> Does anyone have a preference on which package and distro they would like
> us to start with?  If possible, I would like us to scope down to one
> supported docker image to use for Heron.  Maintaining 3 separate images is
> quite a task.
>
> On Tue, Jan 28, 2020 at 4:51 PM Josh Fischer  wrote:
>
> > Hi All,
> >
> > After  several conversations with people across the Heron repo we keep
> > hearing that a Heron convenience binary release would be appreciated.
> > Based on some feedback from Dave we need to decide on what type of
> > packaging is helpful to Heron users as the first step to getting people
> > what they want/need.
> >
> > Right now we have, but not released in a while:
> >
> > Heron Docker Containers:
> > - CentOS
> > - Ubuntu
> > - Debian
> > Heron install scripts
> > - CentOS
> > - Darwin (MacOs)
> > - Ubuntu
> >
> > Does anyone have a preference on which package and distro they would like
> > us to start with?  If possible, I would like us to scope down to one
> > supported docker image to use for Heron.  Maintaining 3 separate images
> is
> > quite a task.
> >
> > - Josh
> >
>

Re: Podling Heron Report Reminder - February 2020

[Ning Wang]

Yeah. Binary releases are for convenience only and not the requirements of
Apache Foundation. I don't know how many releases are qualified as "several
releases". :D

Otherwise, looks good to me. We may need help from Mentors to review/sign
off before submitting now.



On Fri, Jan 31, 2020 at 6:01 PM Josh Fischer  wrote:

> I've updated the report.  Please check and make any changes that you see
> fit.  I'm not sure if we need to list "Making several releases" as a
> requirement to graduate anymore.  We've demonstrated we can make source
> releases.   Binaries/executables are considered unofficial conveniences and
> I would say that we do not need to include this in our top three list.  If
> we don't include this in our top three things to do before graduation, is
> there anything else that should go?  See the report here ->
>
> https://docs.google.com/document/d/1_8ckuEuL7pdo-eOCHaXs32ymD2_qRKzEybL2Zyr6gXU/edit#
>
> - Josh
>
> On Fri, Jan 31, 2020 at 3:02 PM  wrote:
>
> > Dear podling,
> >
> > This email was sent by an automated system on behalf of the Apache
> > Incubator PMC. It is an initial reminder to give you plenty of time to
> > prepare your quarterly board report.
> >
> > The board meeting is scheduled for Wed, 19 February 2020, 10:30 am PDT.
> > The report for your podling will form a part of the Incubator PMC
> > report. The Incubator PMC requires your report to be submitted 2 weeks
> > before the board meeting, to allow sufficient time for review and
> > submission (Wed, February 05).
> >
> > Please submit your report with sufficient time to allow the Incubator
> > PMC, and subsequently board members to review and digest. Again, the
> > very latest you should submit your report is 2 weeks prior to the board
> > meeting.
> >
> > Candidate names should not be made public before people are actually
> > elected, so please do not include the names of potential committers or
> > PPMC members in your report.
> >
> > Thanks,
> >
> > The Apache Incubator PMC
> >
> > Submitting your Report
> >
> > --
> >
> > Your report should contain the following:
> >
> > *   Your project name
> > *   A brief description of your project, which assumes no knowledge of
> > the project or necessarily of its field
> > *   A list of the three most important issues to address in the move
> > towards graduation.
> > *   Any issues that the Incubator PMC or ASF Board might wish/need to be
> > aware of
> > *   How has the community developed since the last report
> > *   How has the project developed since the last report.
> > *   How does the podling rate their own maturity.
> >
> > This should be appended to the Incubator Wiki page at:
> >
> > https://cwiki.apache.org/confluence/display/INCUBATOR/February2020
> >
> > Note: This is manually populated. You may need to wait a little before
> > this page is created from a template.
> >
> > Note: The format of the report has changed to use markdown.
> >
> > Mentors
> > ---
> >
> > Mentors should review reports for their project(s) and sign them off on
> > the Incubator wiki page. Signing off reports shows that you are
> > following the project - projects that are not signed may raise alarms
> > for the Incubator PMC.
> >
> > Incubator PMC
> >
>

1/28/2020 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi~

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
- Jan Heron meetup. And prepare for the Feb meetup in San Francisco
downtown.
- Continue on Java 11 migration. 29 unit tests left to fix.
- POD report.
- License discussion.

Regards,
--ning

Re: [DISCUSS][MENTORS] 0.20.2-incubating binary release

[Ning Wang]

Thanks Dave. This is super helpful.



On Mon, Jan 27, 2020 at 8:13 PM Dave Fisher  wrote:

> Infra had three concerns which roughly are:
>
> (1) The aggregate size of the binary release. There are rules about
> releases larger than 1GB which put pressure on the mirror providers.
> Warning to Infra is required prior to putting these packages on Apache
> Distribution servers.
>
> (2) Apache binary releases are termed unofficial conveniences provided by
> the community which follow normal Apache Release Policy. The question is
> what packaging is helpful for Heron’s users? The tar.gz question is the
> correct one.
>
> (3) Since the binary packages are large have we checked that all the
> included licenses are correct?
>
> Let’s work out (2) and (3) before rechecking about (1)
>
> Regards,
> Dave
>
> Sent from my iPhone
>
> > On Jan 27, 2020, at 6:48 PM, Josh Fischer  wrote:
> >
> > Hi,
> >
> > Part of the Heron binary release is more than just the client artifacts
> > that would be loaded into Maven Central.  For example if you look at an
> old
> > release prior to Heron being donated to Apache we have install scripts
> and
> > tarred binaries for use in different operating systems. You can view them
> > here -> https://github.com/apache/incubator-heron/releases/tag/0.17.8.
> > Where would be go about hosting these install scripts and tar.gz files?
> I
> > think each of them are ~500MB in size today.  My initial thoughts (for
> > simplicity) are that we could ignore the tar.gz files containing binaries
> > and just offer install scripts and one supported docker image.  If
> binaries
> > are needed for manual installation I think people could just compile
> parts
> > of Heron at that point.  I'll look into the licensing/policy issues on
> the
> > Heron docker container in the meantime.
> >
> > - Josh
> >
> >> On Fri, Jan 17, 2020 at 7:24 PM Dave Fisher 
> wrote:
> >>
> >> The ASF as a whole is rather conservative in the sense of being slow to
> >> change.
> >>
> >> Maven Central for JAR packaging has been around for over a decade AND is
> >> supported by Apache projects.
> >>
> >> Docker is comparatively new and there is less policy around it. There is
> >> some, but perhaps not well documented.
> >>
> >> I think we should evaluate the docker file and see what licensing/policy
> >> questions there are. We can then ask the VP, Legal Affairs on
> >> legal-disc...@apache.org
> >>
> >> Regards,
> >> Dave
> >>
> >> Sent from my iPhone
> >>
> >>>> On Jan 17, 2020, at 2:41 PM, Josh Fischer 
> wrote:
> >>>
> >>> Gotcha.  I understand the difference in the two now, but I'm not sure
> >> why
> >>> one would be allowed over the other.  At the end of the day they are
> both
> >>> compiled binaries released under the Apache foundation.  I'll take some
> >>> time to read through apache docs to see if I can get a better
> >> understanding
> >>> of the differences in the two.
> >>>
> >>>> On Fri, Jan 17, 2020 at 4:28 PM Ning Wang 
> wrote:
> >>>>
> >>>> That's true. Maven artifacts are binaries too. My bad.
> >>>>
> >>>> The "binary" I was referring to are the executables, such as
> installer,
> >>>> which contains quite some executables like CLI, exeample jobs, UI,
> >> tracker,
> >>>> etc. The docker image is also part of my "binary".
> >>>>
> >>>> Basically users can just install and run their jobs without compiling
> >> all
> >>>> the tools from source code. Maven artifacts allow users to build their
> >>>> jobs. They still need to compile all the other things in order to run
> >> the
> >>>> jobs.
> >>>>
> >>>> I hope it is more clear this time.
> >>>>
> >>>>
> >>>>
> >>>>> On Fri, Jan 17, 2020 at 1:59 PM Josh Fischer 
> >> wrote:
> >>>>>
> >>>>> I'm not sure I understand how you are defining the differences
> between
> >>>> the
> >>>>> terms "maven artifacts" and "binary releases".  Wouldn't a maven
> >>>> artifact (
> >>>>> java jar ) be considered a binary release as well?  Am I missing
> >>>> something?
> >>>>>
> >>>>>
> >>>>>> On Fri, Jan 17, 2020 at 1:02 AM Ning Wang 
> >> wrote:
> >>>>>
> >>>>>> I think we can not make an official binary release yet before clear
> >> all
> >>>>>> license issues. Therefore, 0.20.0.2 can have maven artifacts but it
> >>>> can't
> >>>>>> have binary release. :(
> >>>>>>
> >>>>>> That's just my understanding of Apache requirements.
> >>>>>>
> >>>>>> On Thu, Jan 16, 2020 at 2:24 PM Josh Fischer 
> >>>>> wrote:
> >>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I'd like to start the process to vote for the 0.20.2-incubating
> >>>> binary
> >>>>>>> release.  Before we start I remember there were concerns about the
> >>>> size
> >>>>>> of
> >>>>>>> the Heron binary and where it would be hosted on Apache Infra.
> Does
> >>>>>> anyone
> >>>>>>> have any knowledge on this potential issue?
> >>>>>>>
> >>>>>>> - Josh
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> >>
>
>

Re: [Meetup] Call for presentation in Feb meetup

[Ning Wang]

No worries. There will definitely be changes in the future meetups.

On Mon, Jan 27, 2020 at 4:42 PM Josh Fischer  wrote:

> I’d be happy to help.  Unfortunately I have too much going on right now to
> work on a presentation at this time.
>
> On Mon, Jan 27, 2020 at 3:09 PM Ning Wang  wrote:
>
>> @Josh Fischer   is there anything you would like to
>> present?
>>
>> On Sun, Jan 26, 2020 at 8:33 PM Ning Wang  wrote:
>>
>>> I need to confirm with @sree vaddi . It will
>>> be on either 17th or 24th.
>>>
>>
>>> So far all the presentations have been in person. Hopefully, I can
>>> figure out how remote presentation works this time.
>>>
>>> On Sun, Jan 26, 2020 at 8:03 PM Josh Fischer 
>>> wrote:
>>>
>>>> Is there a date/time for this event yet?  Is it in person only?
>>>>
>>>> - Josh
>>>>
>>>> On Sun, Jan 26, 2020 at 8:11 PM Ning Wang  wrote:
>>>>
>>>> > Hi,
>>>> >
>>>> > We are trying to host a Heron meetup in the Amplitude office in San
>>>> > Francisco and we are looking for speakers. Anyone would like to
>>>> present?
>>>> >
>>>> > Regards,
>>>> > --ning
>>>> >
>>>>
>>> --
> Sent from A Mobile Device
>

Re: [Meetup] Call for presentation in Feb meetup

[Ning Wang]

@Josh Fischer   is there anything you would like to
present?

On Sun, Jan 26, 2020 at 8:33 PM Ning Wang  wrote:

> I need to confirm with @sree vaddi . It will
> be on either 17th or 24th.
>
> So far all the presentations have been in person. Hopefully, I can figure
> out how remote presentation works this time.
>
> On Sun, Jan 26, 2020 at 8:03 PM Josh Fischer  wrote:
>
>> Is there a date/time for this event yet?  Is it in person only?
>>
>> - Josh
>>
>> On Sun, Jan 26, 2020 at 8:11 PM Ning Wang  wrote:
>>
>> > Hi,
>> >
>> > We are trying to host a Heron meetup in the Amplitude office in San
>> > Francisco and we are looking for speakers. Anyone would like to present?
>> >
>> > Regards,
>> > --ning
>> >
>>
>

Re: [Meetup] Call for presentation in Feb meetup

[Ning Wang]

I need to confirm with @sree vaddi . It will be
on either 17th or 24th.

So far all the presentations have been in person. Hopefully, I can figure
out how remote presentation works this time.

On Sun, Jan 26, 2020 at 8:03 PM Josh Fischer  wrote:

> Is there a date/time for this event yet?  Is it in person only?
>
> - Josh
>
> On Sun, Jan 26, 2020 at 8:11 PM Ning Wang  wrote:
>
> > Hi,
> >
> > We are trying to host a Heron meetup in the Amplitude office in San
> > Francisco and we are looking for speakers. Anyone would like to present?
> >
> > Regards,
> > --ning
> >
>

[Meetup] Call for presentation in Feb meetup

[Ning Wang]

Hi,

We are trying to host a Heron meetup in the Amplitude office in San
Francisco and we are looking for speakers. Anyone would like to present?

Regards,
--ning

Re: Bazel custom rules (tools/rules)

[Ning Wang]

agreed 2

On Fri, Jan 17, 2020 at 2:49 PM SiMing Weng  wrote:

> One thing I really would like to see happening is a better/simpler IDE
> support, maybe via the the Google Jetbrains Bazel Plugin <
> https://ij.bazel.build/>. I knew we had a shell script to generate
> IntelliJ “.iml” file, but it seems no longer working.
>
> I know from CI point of view, IDE support doesn’t seem important, but a
> better developer experience/productivity is going to help drive the
> interest from the community.
>
> > On Jan 17, 2020, at 5:22 PM, Ning Wang  wrote:
> >
> > agreed
> >
> > On Fri, Jan 17, 2020 at 11:45 AM Josh Fischer 
> wrote:
> >
> >> I think that most of the rules that Bazel comes with now did not exist
> when
> >> Heron was under its core development cycles. Now when it comes to
> upgrading
> >> Bazel I would ask the dev@ for feedback if you have to make a decision
> on
> >> what is the best way to go, offering your perspective as well.  If you
> feel
> >> that it is simpler to remove some custom rule implementations then I
> would
> >> make your case write a proposal and send it off to dev@ to get feedback
> >> and
> >> support on the issue.  I think it's safe to say we are all looking for a
> >> simpler  way to manage the Bazel build rules within the Heron code base
> and
> >> we would be happy to get suggestions or feedback on any of our current
> >> Bazel usage/implementation today.
> >>
> >> - Josh
> >>
> >> On Fri, Jan 17, 2020 at 12:19 PM Ning Wang 
> wrote:
> >>
> >>> My feeling is that, Bazel moves fast. When Heron was started, Bazel was
> >>> still young (4+ years ago) and many tasks have to be done via custom
> >> rules.
> >>> And when upgrading Bazel, it is often not backward compatible . so
> >> the
> >>> effort is to fix the build, not clean it up.
> >>>
> >>> However I was not in the team when the project started. Karthik,
> Sanjeev,
> >>> Maosong, Neng, Huijun should have more information.
> >>>
> >>> On Fri, Jan 17, 2020 at 6:37 AM Nicholas Nezis <
> nicholas.ne...@gmail.com
> >>>
> >>> wrote:
> >>>
> >>>> Can I start a thread for a quick discussion about Bazel? I apologize
> if
> >>>> this is a silly question. I'm new to Bazel.
> >>>>
> >>>> I'm curious why there are so many custom rules defined in the Heron
> >> repo
> >>>> (tools/rules). I see Bazel provided rules that we could leverage for
> >>> things
> >>>> like JarJar and Javadoc and Python, etc. Are there specific reasons
> why
> >>>> there is custom logic in the Heron repo? Or was it maybe historical
> >>> because
> >>>> the Bazel provided resources didn't exist at the time? I was looking
> at
> >>>> upgrading our use of Bazel to 1.X. Upgrading our custom logic seem
> >> harder
> >>>> than upgrading a version of a Bazel provided rules dependency.
> >>>>
> >>>> Thanks,
> >>>> Nick
> >>>>
> >>>
> >>
>
>

Re: [DISCUSS][MENTORS] 0.20.2-incubating binary release

[Ning Wang]

Yeah. Technically artifacts contains the dependency libraries tool. Maybe
artifacts are more urgent and necessary than the executables. Afterall
executables are convenient to users but not required in the Apache release
process.

On Fri, Jan 17, 2020 at 2:41 PM Josh Fischer  wrote:

> Gotcha.  I understand the difference in the two now, but I'm not sure why
> one would be allowed over the other.  At the end of the day they are both
> compiled binaries released under the Apache foundation.  I'll take some
> time to read through apache docs to see if I can get a better understanding
> of the differences in the two.
>
> On Fri, Jan 17, 2020 at 4:28 PM Ning Wang  wrote:
>
> > That's true. Maven artifacts are binaries too. My bad.
> >
> > The "binary" I was referring to are the executables, such as installer,
> > which contains quite some executables like CLI, exeample jobs, UI,
> tracker,
> > etc. The docker image is also part of my "binary".
> >
> > Basically users can just install and run their jobs without compiling all
> > the tools from source code. Maven artifacts allow users to build their
> > jobs. They still need to compile all the other things in order to run the
> > jobs.
> >
> > I hope it is more clear this time.
> >
> >
> >
> > On Fri, Jan 17, 2020 at 1:59 PM Josh Fischer 
> wrote:
> >
> > > I'm not sure I understand how you are defining the differences between
> > the
> > > terms "maven artifacts" and "binary releases".  Wouldn't a maven
> > artifact (
> > > java jar ) be considered a binary release as well?  Am I missing
> > something?
> > >
> > >
> > > On Fri, Jan 17, 2020 at 1:02 AM Ning Wang 
> wrote:
> > >
> > > > I think we can not make an official binary release yet before clear
> all
> > > > license issues. Therefore, 0.20.0.2 can have maven artifacts but it
> > can't
> > > > have binary release. :(
> > > >
> > > > That's just my understanding of Apache requirements.
> > > >
> > > > On Thu, Jan 16, 2020 at 2:24 PM Josh Fischer 
> > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I'd like to start the process to vote for the 0.20.2-incubating
> > binary
> > > > > release.  Before we start I remember there were concerns about the
> > size
> > > > of
> > > > > the Heron binary and where it would be hosted on Apache Infra.
> Does
> > > > anyone
> > > > > have any knowledge on this potential issue?
> > > > >
> > > > > - Josh
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS][MENTORS] 0.20.2-incubating binary release

[Ning Wang]

That's true. Maven artifacts are binaries too. My bad.

The "binary" I was referring to are the executables, such as installer,
which contains quite some executables like CLI, exeample jobs, UI, tracker,
etc. The docker image is also part of my "binary".

Basically users can just install and run their jobs without compiling all
the tools from source code. Maven artifacts allow users to build their
jobs. They still need to compile all the other things in order to run the
jobs.

I hope it is more clear this time.



On Fri, Jan 17, 2020 at 1:59 PM Josh Fischer  wrote:

> I'm not sure I understand how you are defining the differences between the
> terms "maven artifacts" and "binary releases".  Wouldn't a maven artifact (
> java jar ) be considered a binary release as well?  Am I missing something?
>
>
> On Fri, Jan 17, 2020 at 1:02 AM Ning Wang  wrote:
>
> > I think we can not make an official binary release yet before clear all
> > license issues. Therefore, 0.20.0.2 can have maven artifacts but it can't
> > have binary release. :(
> >
> > That's just my understanding of Apache requirements.
> >
> > On Thu, Jan 16, 2020 at 2:24 PM Josh Fischer 
> wrote:
> >
> > > Hi,
> > >
> > > I'd like to start the process to vote for the 0.20.2-incubating binary
> > > release.  Before we start I remember there were concerns about the size
> > of
> > > the Heron binary and where it would be hosted on Apache Infra.  Does
> > anyone
> > > have any knowledge on this potential issue?
> > >
> > > - Josh
> > >
> >
>

Re: Bazel custom rules (tools/rules)

[Ning Wang]

agreed

On Fri, Jan 17, 2020 at 11:45 AM Josh Fischer  wrote:

> I think that most of the rules that Bazel comes with now did not exist when
> Heron was under its core development cycles. Now when it comes to upgrading
> Bazel I would ask the dev@ for feedback if you have to make a decision on
> what is the best way to go, offering your perspective as well.  If you feel
> that it is simpler to remove some custom rule implementations then I would
> make your case write a proposal and send it off to dev@ to get feedback
> and
> support on the issue.  I think it's safe to say we are all looking for a
> simpler  way to manage the Bazel build rules within the Heron code base and
> we would be happy to get suggestions or feedback on any of our current
> Bazel usage/implementation today.
>
> - Josh
>
> On Fri, Jan 17, 2020 at 12:19 PM Ning Wang  wrote:
>
> > My feeling is that, Bazel moves fast. When Heron was started, Bazel was
> > still young (4+ years ago) and many tasks have to be done via custom
> rules.
> > And when upgrading Bazel, it is often not backward compatible . so
> the
> > effort is to fix the build, not clean it up.
> >
> > However I was not in the team when the project started. Karthik, Sanjeev,
> > Maosong, Neng, Huijun should have more information.
> >
> > On Fri, Jan 17, 2020 at 6:37 AM Nicholas Nezis  >
> > wrote:
> >
> > > Can I start a thread for a quick discussion about Bazel? I apologize if
> > > this is a silly question. I'm new to Bazel.
> > >
> > > I'm curious why there are so many custom rules defined in the Heron
> repo
> > > (tools/rules). I see Bazel provided rules that we could leverage for
> > things
> > > like JarJar and Javadoc and Python, etc. Are there specific reasons why
> > > there is custom logic in the Heron repo? Or was it maybe historical
> > because
> > > the Bazel provided resources didn't exist at the time? I was looking at
> > > upgrading our use of Bazel to 1.X. Upgrading our custom logic seem
> harder
> > > than upgrading a version of a Bazel provided rules dependency.
> > >
> > > Thanks,
> > > Nick
> > >
> >
>

Re: Bazel custom rules (tools/rules)

[Ning Wang]

My feeling is that, Bazel moves fast. When Heron was started, Bazel was
still young (4+ years ago) and many tasks have to be done via custom rules.
And when upgrading Bazel, it is often not backward compatible . so the
effort is to fix the build, not clean it up.

However I was not in the team when the project started. Karthik, Sanjeev,
Maosong, Neng, Huijun should have more information.

On Fri, Jan 17, 2020 at 6:37 AM Nicholas Nezis 
wrote:

> Can I start a thread for a quick discussion about Bazel? I apologize if
> this is a silly question. I'm new to Bazel.
>
> I'm curious why there are so many custom rules defined in the Heron repo
> (tools/rules). I see Bazel provided rules that we could leverage for things
> like JarJar and Javadoc and Python, etc. Are there specific reasons why
> there is custom logic in the Heron repo? Or was it maybe historical because
> the Bazel provided resources didn't exist at the time? I was looking at
> upgrading our use of Bazel to 1.X. Upgrading our custom logic seem harder
> than upgrading a version of a Bazel provided rules dependency.
>
> Thanks,
> Nick
>

[DISCUSSION][MENTORS] License of JSXTransformer.js

[Ning Wang]

Hi,

The last known license issue in Heron is the JSXTransformer.js file. The
issue is tracked here:
https://github.com/apache/incubator-heron/pull/3437/files

The JS file contains many functions and each function has different
copyrights and licenses, including Apache, BSD (2/3) and MIT.

I prepared a PR (https://github.com/apache/incubator-heron/pull/3437/files)
to solve the problem to meet the requirements. Basically, JSXTransformer is
added under multiple licenses in the LICENSE file, and corresponding
license text files are added in the licenses folder.

The questions are:
- Is this the right way?
- Is there anything we need to do?

This license issue is a major blocker for graduation. Any suggestions would
be greatly appreciated!

--ning

Re: [DISCUSS][MENTORS] 0.20.2-incubating binary release

[Ning Wang]

I think we can not make an official binary release yet before clear all
license issues. Therefore, 0.20.0.2 can have maven artifacts but it can't
have binary release. :(

That's just my understanding of Apache requirements.

On Thu, Jan 16, 2020 at 2:24 PM Josh Fischer  wrote:

> Hi,
>
> I'd like to start the process to vote for the 0.20.2-incubating binary
> release.  Before we start I remember there were concerns about the size of
> the Heron binary and where it would be hosted on Apache Infra.  Does anyone
> have any knowledge on this potential issue?
>
> - Josh
>

Re: Podling Heron Report Reminder - February 2020

[Ning Wang]

Therefore, 0.20.0.2 can have maven artifacts but it can't have binary
release. :(

On Thu, Jan 16, 2020 at 10:58 PM Ning Wang  wrote:

> I think we can not make an official binary release yet before clear all
> license issues.
>
> On Thu, Jan 16, 2020 at 2:20 PM Josh Fischer  wrote:
>
>> Hi,
>>
>> I've been meaning to get a vote out for the 0.20.2-incubating binaries, I
>> just haven't had the time to get around to it yet.  I know some people are
>> needing them.  I'm going to start a discussion email today to discuss the
>> binary release for this version, I think there were concerns from the
>> Apache infra side of things because of the size of the binary.
>>
>> On Thu, Jan 16, 2020 at 4:14 PM Ning Wang  wrote:
>>
>> > I am hoping to have at least 1 binary release before marking #2 as
>> > complete.
>> >
>> > On Thu, Jan 16, 2020 at 10:20 AM Josh Fischer 
>> wrote:
>> >
>> > > Hi,
>> > > inlined
>> > >
>> > >
>> > >
>> > > On Thu, Jan 16, 2020 at 10:11 AM Dave Fisher  wrote:
>> > >
>> > > > Hi -
>> > > >
>> > > > The list of three actions towards graduation should include current
>> > > > actions and not ones that are completed.
>> > > >
>> > > > I have questions on #1. What are the licensing issues that remain?
>> How
>> > > are
>> > > > these being tracked?
>> > > >
>> > > ** Github issues at the moment.  The licensing issue is mentioned here
>> > > https://github.com/apache/incubator-heron/issues/3406 .
>> > >
>> > > > ** I don't think we need to list making several releases anymore.  I
>> > > think
>> > > we've demonstrated that we can create source releases.
>> > >
>> > > >
>> > > > #3 should be improving the amount of community discussion and
>> decision
>> > > > making on the dev@ mailing list. Monthly Meetups are only in
>> Sunnyvale
>> > > > and exclude most of the world. It is a critical part of the Apache
>> Way
>> > to
>> > > > make sure discussion and decision making happens on the mailing
>> list.
>> > > >
>> > > ** agreed
>> > >
>> > > >
>> > > > On last committers and PPMC members please provide the date of the
>> > > > elections.
>> > > >
>> > >
>> > > ** will do
>> > >
>> > > >
>> > > > Regards,
>> > > > Dave
>> > > >
>> > > > > On Jan 16, 2020, at 12:27 AM, Ning Wang 
>> > wrote:
>> > > > >
>> > > > > Hi,
>> > > > >
>> > > > > I prepared a draft of the Feb podling report:
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://docs.google.com/document/d/1_8ckuEuL7pdo-eOCHaXs32ymD2_qRKzEybL2Zyr6gXU/edit#
>> > > > >
>> > > > > Please help to update if anything is missing. Thanks!
>> > > > >
>> > > > > On Wed, Jan 15, 2020 at 4:26 PM  wrote:
>> > > > >
>> > > > >> Dear podling,
>> > > > >>
>> > > > >> This email was sent by an automated system on behalf of the
>> Apache
>> > > > >> Incubator PMC. It is an initial reminder to give you plenty of
>> time
>> > to
>> > > > >> prepare your quarterly board report.
>> > > > >>
>> > > > >> The board meeting is scheduled for Wed, 19 February 2020, 10:30
>> am
>> > > PDT.
>> > > > >> The report for your podling will form a part of the Incubator PMC
>> > > > >> report. The Incubator PMC requires your report to be submitted 2
>> > weeks
>> > > > >> before the board meeting, to allow sufficient time for review and
>> > > > >> submission (Wed, February 05).
>> > > > >>
>> > > > >> Please submit your report with sufficient time to allow the
>> > Incubator
>> > > > >> PMC, and subsequently board members to review and digest. Again,
>> the
>> > > > >> very latest you should submit your report is 2 weeks prior to the
>> > > board
>> > > > >> meeting.
>> &

Re: Podling Heron Report Reminder - February 2020

[Ning Wang]

I think we can not make an official binary release yet before clear all
license issues.

On Thu, Jan 16, 2020 at 2:20 PM Josh Fischer  wrote:

> Hi,
>
> I've been meaning to get a vote out for the 0.20.2-incubating binaries, I
> just haven't had the time to get around to it yet.  I know some people are
> needing them.  I'm going to start a discussion email today to discuss the
> binary release for this version, I think there were concerns from the
> Apache infra side of things because of the size of the binary.
>
> On Thu, Jan 16, 2020 at 4:14 PM Ning Wang  wrote:
>
> > I am hoping to have at least 1 binary release before marking #2 as
> > complete.
> >
> > On Thu, Jan 16, 2020 at 10:20 AM Josh Fischer 
> wrote:
> >
> > > Hi,
> > > inlined
> > >
> > >
> > >
> > > On Thu, Jan 16, 2020 at 10:11 AM Dave Fisher  wrote:
> > >
> > > > Hi -
> > > >
> > > > The list of three actions towards graduation should include current
> > > > actions and not ones that are completed.
> > > >
> > > > I have questions on #1. What are the licensing issues that remain?
> How
> > > are
> > > > these being tracked?
> > > >
> > > ** Github issues at the moment.  The licensing issue is mentioned here
> > > https://github.com/apache/incubator-heron/issues/3406 .
> > >
> > > > ** I don't think we need to list making several releases anymore.  I
> > > think
> > > we've demonstrated that we can create source releases.
> > >
> > > >
> > > > #3 should be improving the amount of community discussion and
> decision
> > > > making on the dev@ mailing list. Monthly Meetups are only in
> Sunnyvale
> > > > and exclude most of the world. It is a critical part of the Apache
> Way
> > to
> > > > make sure discussion and decision making happens on the mailing list.
> > > >
> > > ** agreed
> > >
> > > >
> > > > On last committers and PPMC members please provide the date of the
> > > > elections.
> > > >
> > >
> > > ** will do
> > >
> > > >
> > > > Regards,
> > > > Dave
> > > >
> > > > > On Jan 16, 2020, at 12:27 AM, Ning Wang 
> > wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I prepared a draft of the Feb podling report:
> > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1_8ckuEuL7pdo-eOCHaXs32ymD2_qRKzEybL2Zyr6gXU/edit#
> > > > >
> > > > > Please help to update if anything is missing. Thanks!
> > > > >
> > > > > On Wed, Jan 15, 2020 at 4:26 PM  wrote:
> > > > >
> > > > >> Dear podling,
> > > > >>
> > > > >> This email was sent by an automated system on behalf of the Apache
> > > > >> Incubator PMC. It is an initial reminder to give you plenty of
> time
> > to
> > > > >> prepare your quarterly board report.
> > > > >>
> > > > >> The board meeting is scheduled for Wed, 19 February 2020, 10:30 am
> > > PDT.
> > > > >> The report for your podling will form a part of the Incubator PMC
> > > > >> report. The Incubator PMC requires your report to be submitted 2
> > weeks
> > > > >> before the board meeting, to allow sufficient time for review and
> > > > >> submission (Wed, February 05).
> > > > >>
> > > > >> Please submit your report with sufficient time to allow the
> > Incubator
> > > > >> PMC, and subsequently board members to review and digest. Again,
> the
> > > > >> very latest you should submit your report is 2 weeks prior to the
> > > board
> > > > >> meeting.
> > > > >>
> > > > >> Candidate names should not be made public before people are
> actually
> > > > >> elected, so please do not include the names of potential
> committers
> > or
> > > > >> PPMC members in your report.
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >> The Apache Incubator PMC
> > > > >>
> > > > >> Submitting your Report
> > > > >>
> > > > >> ---

Re: Podling Heron Report Reminder - February 2020

[Ning Wang]

I am hoping to have at least 1 binary release before marking #2 as complete.

On Thu, Jan 16, 2020 at 10:20 AM Josh Fischer  wrote:

> Hi,
> inlined
>
>
>
> On Thu, Jan 16, 2020 at 10:11 AM Dave Fisher  wrote:
>
> > Hi -
> >
> > The list of three actions towards graduation should include current
> > actions and not ones that are completed.
> >
> > I have questions on #1. What are the licensing issues that remain? How
> are
> > these being tracked?
> >
> ** Github issues at the moment.  The licensing issue is mentioned here
> https://github.com/apache/incubator-heron/issues/3406 .
>
> > ** I don't think we need to list making several releases anymore.  I
> think
> we've demonstrated that we can create source releases.
>
> >
> > #3 should be improving the amount of community discussion and decision
> > making on the dev@ mailing list. Monthly Meetups are only in Sunnyvale
> > and exclude most of the world. It is a critical part of the Apache Way to
> > make sure discussion and decision making happens on the mailing list.
> >
> ** agreed
>
> >
> > On last committers and PPMC members please provide the date of the
> > elections.
> >
>
> ** will do
>
> >
> > Regards,
> > Dave
> >
> > > On Jan 16, 2020, at 12:27 AM, Ning Wang  wrote:
> > >
> > > Hi,
> > >
> > > I prepared a draft of the Feb podling report:
> > >
> > >
> >
> https://docs.google.com/document/d/1_8ckuEuL7pdo-eOCHaXs32ymD2_qRKzEybL2Zyr6gXU/edit#
> > >
> > > Please help to update if anything is missing. Thanks!
> > >
> > > On Wed, Jan 15, 2020 at 4:26 PM  wrote:
> > >
> > >> Dear podling,
> > >>
> > >> This email was sent by an automated system on behalf of the Apache
> > >> Incubator PMC. It is an initial reminder to give you plenty of time to
> > >> prepare your quarterly board report.
> > >>
> > >> The board meeting is scheduled for Wed, 19 February 2020, 10:30 am
> PDT.
> > >> The report for your podling will form a part of the Incubator PMC
> > >> report. The Incubator PMC requires your report to be submitted 2 weeks
> > >> before the board meeting, to allow sufficient time for review and
> > >> submission (Wed, February 05).
> > >>
> > >> Please submit your report with sufficient time to allow the Incubator
> > >> PMC, and subsequently board members to review and digest. Again, the
> > >> very latest you should submit your report is 2 weeks prior to the
> board
> > >> meeting.
> > >>
> > >> Candidate names should not be made public before people are actually
> > >> elected, so please do not include the names of potential committers or
> > >> PPMC members in your report.
> > >>
> > >> Thanks,
> > >>
> > >> The Apache Incubator PMC
> > >>
> > >> Submitting your Report
> > >>
> > >> --
> > >>
> > >> Your report should contain the following:
> > >>
> > >> *   Your project name
> > >> *   A brief description of your project, which assumes no knowledge of
> > >>the project or necessarily of its field
> > >> *   A list of the three most important issues to address in the move
> > >>towards graduation.
> > >> *   Any issues that the Incubator PMC or ASF Board might wish/need to
> be
> > >>aware of
> > >> *   How has the community developed since the last report
> > >> *   How has the project developed since the last report.
> > >> *   How does the podling rate their own maturity.
> > >>
> > >> This should be appended to the Incubator Wiki page at:
> > >>
> > >> https://cwiki.apache.org/confluence/display/INCUBATOR/February2020
> > >>
> > >> Note: This is manually populated. You may need to wait a little before
> > >> this page is created from a template.
> > >>
> > >> Note: The format of the report has changed to use markdown.
> > >>
> > >> Mentors
> > >> ---
> > >>
> > >> Mentors should review reports for their project(s) and sign them off
> on
> > >> the Incubator wiki page. Signing off reports shows that you are
> > >> following the project - projects that are not signed may raise alarms
> > >> for the Incubator PMC.
> > >>
> > >> Incubator PMC
> > >>
> >
> >
>

Re: Podling Heron Report Reminder - February 2020

[Ning Wang]

#1: The issue is tracked here:
https://github.com/apache/incubator-heron/issues/3406. I don't know what we
should do yet regarding JSXTransformer for now. It includes many functions
with many different licenses. :(  Any idea would be welcome.

Roger for #3 and election dates.

Thanks Dave.

On Thu, Jan 16, 2020 at 8:11 AM Dave Fisher  wrote:

> Hi -
>
> The list of three actions towards graduation should include current
> actions and not ones that are completed.
>
> I have questions on #1. What are the licensing issues that remain? How are
> these being tracked?
>
> #3 should be improving the amount of community discussion and decision
> making on the dev@ mailing list. Monthly Meetups are only in Sunnyvale
> and exclude most of the world. It is a critical part of the Apache Way to
> make sure discussion and decision making happens on the mailing list.
>
> On last committers and PPMC members please provide the date of the
> elections.
>
> Regards,
> Dave
>
> > On Jan 16, 2020, at 12:27 AM, Ning Wang  wrote:
> >
> > Hi,
> >
> > I prepared a draft of the Feb podling report:
> >
> >
> https://docs.google.com/document/d/1_8ckuEuL7pdo-eOCHaXs32ymD2_qRKzEybL2Zyr6gXU/edit#
> >
> > Please help to update if anything is missing. Thanks!
> >
> > On Wed, Jan 15, 2020 at 4:26 PM  wrote:
> >
> >> Dear podling,
> >>
> >> This email was sent by an automated system on behalf of the Apache
> >> Incubator PMC. It is an initial reminder to give you plenty of time to
> >> prepare your quarterly board report.
> >>
> >> The board meeting is scheduled for Wed, 19 February 2020, 10:30 am PDT.
> >> The report for your podling will form a part of the Incubator PMC
> >> report. The Incubator PMC requires your report to be submitted 2 weeks
> >> before the board meeting, to allow sufficient time for review and
> >> submission (Wed, February 05).
> >>
> >> Please submit your report with sufficient time to allow the Incubator
> >> PMC, and subsequently board members to review and digest. Again, the
> >> very latest you should submit your report is 2 weeks prior to the board
> >> meeting.
> >>
> >> Candidate names should not be made public before people are actually
> >> elected, so please do not include the names of potential committers or
> >> PPMC members in your report.
> >>
> >> Thanks,
> >>
> >> The Apache Incubator PMC
> >>
> >> Submitting your Report
> >>
> >> --
> >>
> >> Your report should contain the following:
> >>
> >> *   Your project name
> >> *   A brief description of your project, which assumes no knowledge of
> >>the project or necessarily of its field
> >> *   A list of the three most important issues to address in the move
> >>towards graduation.
> >> *   Any issues that the Incubator PMC or ASF Board might wish/need to be
> >>aware of
> >> *   How has the community developed since the last report
> >> *   How has the project developed since the last report.
> >> *   How does the podling rate their own maturity.
> >>
> >> This should be appended to the Incubator Wiki page at:
> >>
> >> https://cwiki.apache.org/confluence/display/INCUBATOR/February2020
> >>
> >> Note: This is manually populated. You may need to wait a little before
> >> this page is created from a template.
> >>
> >> Note: The format of the report has changed to use markdown.
> >>
> >> Mentors
> >> ---
> >>
> >> Mentors should review reports for their project(s) and sign them off on
> >> the Incubator wiki page. Signing off reports shows that you are
> >> following the project - projects that are not signed may raise alarms
> >> for the Incubator PMC.
> >>
> >> Incubator PMC
> >>
>
>

Re: Podling Heron Report Reminder - February 2020

[Ning Wang]

Hi,

I prepared a draft of the Feb podling report:

https://docs.google.com/document/d/1_8ckuEuL7pdo-eOCHaXs32ymD2_qRKzEybL2Zyr6gXU/edit#

Please help to update if anything is missing. Thanks!

On Wed, Jan 15, 2020 at 4:26 PM  wrote:

> Dear podling,
>
> This email was sent by an automated system on behalf of the Apache
> Incubator PMC. It is an initial reminder to give you plenty of time to
> prepare your quarterly board report.
>
> The board meeting is scheduled for Wed, 19 February 2020, 10:30 am PDT.
> The report for your podling will form a part of the Incubator PMC
> report. The Incubator PMC requires your report to be submitted 2 weeks
> before the board meeting, to allow sufficient time for review and
> submission (Wed, February 05).
>
> Please submit your report with sufficient time to allow the Incubator
> PMC, and subsequently board members to review and digest. Again, the
> very latest you should submit your report is 2 weeks prior to the board
> meeting.
>
> Candidate names should not be made public before people are actually
> elected, so please do not include the names of potential committers or
> PPMC members in your report.
>
> Thanks,
>
> The Apache Incubator PMC
>
> Submitting your Report
>
> --
>
> Your report should contain the following:
>
> *   Your project name
> *   A brief description of your project, which assumes no knowledge of
> the project or necessarily of its field
> *   A list of the three most important issues to address in the move
> towards graduation.
> *   Any issues that the Incubator PMC or ASF Board might wish/need to be
> aware of
> *   How has the community developed since the last report
> *   How has the project developed since the last report.
> *   How does the podling rate their own maturity.
>
> This should be appended to the Incubator Wiki page at:
>
> https://cwiki.apache.org/confluence/display/INCUBATOR/February2020
>
> Note: This is manually populated. You may need to wait a little before
> this page is created from a template.
>
> Note: The format of the report has changed to use markdown.
>
> Mentors
> ---
>
> Mentors should review reports for their project(s) and sign them off on
> the Incubator wiki page. Signing off reports shows that you are
> following the project - projects that are not signed may raise alarms
> for the Incubator PMC.
>
> Incubator PMC
>

Fwd: [MENTORS] Podling reporting timeline for February

[Ning Wang]

I prepared a draft for the Feb podling report:

https://docs.google.com/document/d/1_8ckuEuL7pdo-eOCHaXs32ymD2_qRKzEybL2Zyr6gXU/edit#

Please feel free to update. Thanks in advance!


-- Forwarded message -
From: Justin Mclean 
Date: Wed, Jan 15, 2020 at 4:23 PM
Subject: [MENTORS] Podling reporting timeline for February
To: 


Hi,

Draft report can be found:
https://cwiki.apache.org/confluence/display/INCUBATOR/February2020

Dates Due:
Wed February 05 - Podling reports due by end of day
Sun February 09 - Shepherd reviews due by end of day
Sun February 09 - Summary due by end of day
Tue February 11 - Mentor signoff due by end of day
Wed February 12 - Report submitted to Board
Wed February 19 - Board meeting

Podlings expected to report:
Annotator*
DataSketches
DolphinScheduler
Doris
ECharts
Heron
Milagro*
Myriad*
NuttX
PageSpeed
Pinot
Ratis
S2Graph
SDAP
StreamPipes
Tamaya
Taverna*
Toree
Training
TubeMQ
Tuweni

It’s likely that YuniKorn will be added to that list.

Thanks,
Justin

* Did not report last month so need to report this month
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

1/14/2020 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Welcome to 2020!

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
- Code reviews.
- Prepare an intro slides for the next Heron meetup. Hopefully it could be
helpful to beginning audience.
- Continue on Java 11 migration. Made some progress but not much. :(

Regards,
--ning

12/31/2019 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Happy holidays everyone!

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
- Code reviews.
- Working on Java 11 support.

Regards,
--ning

12/17/2019 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi, all~,

I forgot this email early this week. Sorry. Here it comes!

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
- Code reviews.
- Update bazel config script for Mac OS 10.14

Regards,
--ning

Re: [DISCUSSION] Questions about docker images support

[Ning Wang]

Sounds good to me.

On Tue, Dec 10, 2019 at 3:47 PM Simon Weng  wrote:

> System process, in the form of container, can still work side by side with
> the spout and bolt container, because they are in the same pod, sharing the
> same Linux kernel and networking space, they’re seeing each other on
> localhost. They can even share the same file system as long as they share
> the same volume.
>
> On Tue, Dec 10, 2019 at 6:37 PM Ning Wang  wrote:
>
> > One image sounds good to me.
> >
> > The two ideas are interesting. Heron system processes are designed to
> work
> > in the same container of worker instances so is might be tricky to
> > separate. That being said, there are definitely things to improve.
> >
> > On Tue, Dec 10, 2019 at 1:07 PM SiMing Weng 
> wrote:
> >
> > > Hi, guys:
> > >
> > > I think it indeed makes sense to narrow down to a single base image.
> > > Regarding which variant to choose, it boils down to any
> > platform-dependent
> > > component in Heron, I can only think of native StreamManager as far as
> I
> > > know. I’m sure others on the team knows better than I.
> > >
> > > Also, it may sounds a long shot, but I’d like to throw the idea here.
> > > Currently, we containerize Heron as an all-in-one image, which is huge
> > and
> > > serves as something like a VM image. Ideally, we should containerize
> each
> > > component and make them slim, then it becomes easier to contribute
> > > to/maintain/release.
> > >
> > > Then even more ambitiously, on k8s cluster, what if we make it possible
> > to
> > > deploy individual Spout/Bolt also as container inside each pod, with a
> > > bunch of side-car container, such as StreamManger, etc, rather than
> > > managing the individual process manually in our Executor. This k8s
> native
> > > approach would open a whole lot possibility, such as easier log
> > aggregation
> > > by existing open source tools, auto scaling/self-tuning with k8s
> operator
> > > pattern. I think the process-based stream processing nature of Heron
> > gives
> > > itself a unique advantage to be used in containerized environment,
> > compared
> > > to thread-based peers.
> > >
> > > Cheers,
> > >
> > > SiMing Weng
> > >
> > > > On Dec 10, 2019, at 2:11 PM, Josh Fischer 
> wrote:
> > > >
> > > > I think we should settle on one "official" version for the community
> to
> > > > support.  I would like to hear from other users on the  list as to
> > which
> > > > image it is we keep supporting moving forward.  I don't think we have
> > > > enough bandwidth or resources to support 5 different versions of
> Heron
> > > in
> > > > a container.
> > > >
> > > > On Tue, Dec 10, 2019 at 1:53 AM Ning Wang 
> > wrote:
> > > >
> > > >> Hi, everyone,
> > > >>
> > > >> When I was working on this PR (
> > > >> https://github.com/apache/incubator-heron/pull/3411), Josh raised
> up
> > a
> > > >> question about the docker images supported by Heron. There are 5
> > images
> > > >> configured: centos7, debian9, ubuntu 14.04, 16.04 and 18.04.
> > > >>
> > > >> Questions to discuss:
> > > >> - It could be a burden to maintain all of them. Do you feel it is
> > > necessary
> > > >> to support all of them? Or is there another one we should have?
> > > >> - For ubuntu, do we really need 3 versions?
> > > >>
> > > >> Regards,
> > > >> --ning
> > > >>
> > >
> > >
> >
> --
> Sent from Gmail Mobile
>

Re: [DISCUSSION] Questions about docker images support

[Ning Wang]

One image sounds good to me.

The two ideas are interesting. Heron system processes are designed to work
in the same container of worker instances so is might be tricky to
separate. That being said, there are definitely things to improve.

On Tue, Dec 10, 2019 at 1:07 PM SiMing Weng  wrote:

> Hi, guys:
>
> I think it indeed makes sense to narrow down to a single base image.
> Regarding which variant to choose, it boils down to any platform-dependent
> component in Heron, I can only think of native StreamManager as far as I
> know. I’m sure others on the team knows better than I.
>
> Also, it may sounds a long shot, but I’d like to throw the idea here.
> Currently, we containerize Heron as an all-in-one image, which is huge and
> serves as something like a VM image. Ideally, we should containerize each
> component and make them slim, then it becomes easier to contribute
> to/maintain/release.
>
> Then even more ambitiously, on k8s cluster, what if we make it possible to
> deploy individual Spout/Bolt also as container inside each pod, with a
> bunch of side-car container, such as StreamManger, etc, rather than
> managing the individual process manually in our Executor. This k8s native
> approach would open a whole lot possibility, such as easier log aggregation
> by existing open source tools, auto scaling/self-tuning with k8s operator
> pattern. I think the process-based stream processing nature of Heron gives
> itself a unique advantage to be used in containerized environment, compared
> to thread-based peers.
>
> Cheers,
>
> SiMing Weng
>
> > On Dec 10, 2019, at 2:11 PM, Josh Fischer  wrote:
> >
> > I think we should settle on one "official" version for the community to
> > support.  I would like to hear from other users on the  list as to which
> > image it is we keep supporting moving forward.  I don't think we have
> > enough bandwidth or resources to support 5 different versions of  Heron
> in
> > a container.
> >
> > On Tue, Dec 10, 2019 at 1:53 AM Ning Wang  wrote:
> >
> >> Hi, everyone,
> >>
> >> When I was working on this PR (
> >> https://github.com/apache/incubator-heron/pull/3411), Josh raised up a
> >> question about the docker images supported by Heron. There are 5 images
> >> configured: centos7, debian9, ubuntu 14.04, 16.04 and 18.04.
> >>
> >> Questions to discuss:
> >> - It could be a burden to maintain all of them. Do you feel it is
> necessary
> >> to support all of them? Or is there another one we should have?
> >> - For ubuntu, do we really need 3 versions?
> >>
> >> Regards,
> >> --ning
> >>
>
>

Re: [Mentors] Heron Docs Update

[Ning Wang]

Great!

On Sun, Dec 8, 2019 at 6:17 AM Simon Weng  wrote:

> Nice! Great work!
>
> On Sun, Dec 8, 2019 at 12:11 AM Josh Fischer  wrote:
>
> > Thanks Huijun.  I went ahead and published the site.  You can view it at
> > https://heron.incubator.apache.org/.
> >
> > On Sat, Dec 7, 2019 at 7:12 PM Huijun Wu 
> wrote:
> >
> > > Good job. The staging website looks good
> > >
> > > Josh Fischer  于 2019年12月7日周六 12:57写道：
> > >
> > > > Hey All,
> > > >
> > > > I have a few more tweaks to the CI script now that I understand the
> > > website
> > > > publishing process.  You can view the new site in the staging
> > environment
> > > > here.  https://heron.staged.apache.org/.
> > > >
> > > >
> > > >
> > > > On Fri, Dec 6, 2019 at 2:31 PM Josh Fischer 
> > wrote:
> > > >
> > > > > Hey All,
> > > > >
> > > > > I filed a Jira issue for this.  You can view the details here ->
> > > > > https://issues.apache.org/jira/browse/INFRA-19537
> > > > >
> > > > > On Fri, Dec 6, 2019 at 9:52 AM Josh Fischer 
> > > wrote:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >> The current site is hosted on Github pages and it needs to be
> > > replaced.
> > > > I
> > > > >> recently managed to get a CI job to run, build the docs correctly
> > and
> > > > >> commit them to the asf-site branch under the folder "content" in
> the
> > > > root
> > > > >> directory.
> > > > >>
> > > > >> Does anyone know what the next steps to have these docs served by
> > > Apache
> > > > >> infrastructure are?
> > > > >>
> > > > >> I've looked at both of the below pages and haven't seen any
> details
> > > that
> > > > >> stick out to me on what to do next.
> > > > >> https://www.apache.org/dev/infra-site.html
> > > > >> https://www.apache.org/dev/project-site.html
> > > > >>
> > > > >> - Josh
> > > > >>
> > > > >> On Tue, Dec 3, 2019 at 9:39 PM Josh Fischer 
> > > > wrote:
> > > > >>
> > > > >>> Hi,
> > > > >>>
> > > > >>> The current site is hosted on Github pages and it needs to be
> > > replaced.
> > > > >>> I recently managed to get a CI job to run, build the docs
> correctly
> > > and
> > > > >>> commit them to the asf-site branch under the folder "content" in
> > the
> > > > root
> > > > >>> directory.
> > > > >>>
> > > > >>> I think we will  need to use is the gitpubsub to fully
> > publish/serve
> > > > our
> > > > >>> new documentation.  Does anyone know what the next steps to have
> > > these
> > > > docs
> > > > >>> served by Apache infrastructure are?
> > > > >>>
> > > > >>> I've looked at both of the below pages and haven't seen any
> details
> > > > that
> > > > >>> stick out to me on what to do next.
> > > > >>> https://www.apache.org/dev/infra-site.html
> > > > >>> https://www.apache.org/dev/project-site.html
> > > > >>>
> > > > >>> - Josh
> > > > >>>
> > > > >>
> > > >
> > >
> >
> --
> Sent from Gmail Mobile
>

11/19/2019 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi, all~,

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
1. Heron UI: add component running info and a capacity based parallelism
calculator.
2.  meetup.

Regards,
--ning

Re: Monthly meetup @Red Hat in Sunnyvale, CA

[Ning Wang]

Thanks. Sree~

On Tue, Nov 12, 2019 at 12:05 PM Sree Vaddi 
wrote:

> Hi Team,
> It's time for our monthly meetup.
> https://www.meetup.com/Apache-Heron-Bay-Area/events/265092166/
>
> Please plan to attend in person.
>
>
> Thank you./Sree
>
>

Re: [VOTE] Heron Release 0.20.2-incubating Release Candidate 1

[Ning Wang]

Thanks!

Yeah. The MacOS environment relies on XCode tool chain which seems to have
some changes in the latest version. :(



On Fri, Nov 8, 2019 at 10:39 AM Dave Fisher  wrote:

> +1 (binding)
>
> Incubating in name
> Signature and checksum are correct
> Rat Check passes
> LICENSE present
> NOTICE present - only the last year of a copyright actually applies, but a
> range is ok
> DISCLAIMER present
>
> Could not build on Catalina 10.15.1 with Xcode 11.2 - Bazel_configure.py
> fails.
>
> Code is :
> # on mac, /usr/bin/cpp --version returns this:
>   #   Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
>   mac_line = re.search('^(Apple LLVM version\s+[\d\.]+)\s+\(clang.*',
> first_line)
>   if mac_line:
> version = get_trailing_version(mac_line.group(1))
> if version:
>   return version
>
> But error message is:
> FAILED:  Could not determine the version of /usr/bin/clang from the
> following output
> /usr/bin/clang --version
> Apple clang version 11.0.0 (clang-1100.0.33.12)
> Target: x86_64-apple-darwin19.0.0
> Thread model: posix
> InstalledDir: /Library/Developer/CommandLineTools/usr/bin
>
> Regards,
> Dave
> > On Nov 6, 2019, at 10:15 AM, Josh Fischer  wrote:
> >
> > Hello Heron Community,
> >
> > This is a call for a vote to the 1st release candidate for Apache Heron,
> > version v0.20.2-incubating. We request project mentors (binded) as well
> as
> > all contributors (unbinded) and users to review and vote on this
> incubator
> > release.
> >
> > *  The tag to be voted upon:0.20.2-incubating-rc1
> > (1dbf1ce1860725827004c7047bf714b87c9df6d7)
> > The full list of changes and release notes are available
> > at:
> >
> https://github.com/apache/incubator-heron/releases/tag/0.20.2-incubating-rc1
> >
> >
> > Files can be found in dist.apache.org
> > Source:
> >
> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.2-incubating-candidate-1/
> >
> > SHA-512
> > checksums:
> >
> >
> 30d1e5e20bfadb448607678d8d754ed95fb5960caa45744a161da62f34ceaa812ded7ed0052cd2663a58f841f94de01344709809f711e7afedc4b00cd9ffd865
> > incubator-heron-0.20.2-incubating-rc1.tar.gz
> >
> >
> > Please download the source package, and follow the compiling guide
> >
> https://apache.github.io/incubator-heron/docs/developers/compiling/compiling/
> > to
> > setup the environment, build and run locally.
> >
> > Note that currently Bazel 0.26.0 is required to build this version.
> After
> > Bazel is set up and works correctly, you can use these release check
> scripts
> >
> https://github.com/apache/incubator-heron/blob/master/scripts/release_check/README.md#run-individual-release-checks
> > to
> > verify the build.  The vote will be open for at least 72 hours or until
> the
> > necessary number of votes are reached.
> >
> > Please vote accordingly:
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove with the reason
> >
> > Thank you,
> >
> > The Apache Heron (Incubating) Team
>
>

Re: [Discuss][Report] Heron November Podling Report

[Ning Wang]

Thanks a lot Dave! This is important information.

In fact one discussion we were having last month is at what stage we should
start thinking about graduation. :)

On Tue, Nov 5, 2019 at 5:57 PM Dave Fisher  wrote:

> Exactly. The binary release is very large. A dependency licensing analysis
> is required.
>
> Once this is done then the podling is nearing graduation.
>
> Regards,
> Dave
>
> Sent from my iPhone
>
> > On Nov 5, 2019, at 5:39 PM, Ning Wang  wrote:
> >
> > There are a few things to worry about during a release, including the
> > binary release, and license check, etc. With a few releases we hope to
> > understand better about all the requirements I think.
> >
> > Is there any suggestion about it?
> >
> >> On Tue, Nov 5, 2019 at 5:23 PM Justin Mclean 
> wrote:
> >>
> >> Hi,
> >>
> >> Thanks for submitting your report. Could you clarify what you mean by
> >> "Making several releases." as a barrier to graduation when you have
> already
> >> made source releases. Are you referring to binary releases or something
> >> else?
> >>
> >> Thanks,
> >> Justin
> >>
> >
> >
> > --
> > Regards,
> > Ning Wang
>
>

Re: [Discuss][Report] Heron November Podling Report

[Ning Wang]

There are a few things to worry about during a release, including the
binary release, and license check, etc. With a few releases we hope to
understand better about all the requirements I think.

Is there any suggestion about it?

On Tue, Nov 5, 2019 at 5:23 PM Justin Mclean  wrote:

> Hi,
>
> Thanks for submitting your report. Could you clarify what you mean by
> "Making several releases." as a barrier to graduation when you have already
> made source releases. Are you referring to binary releases or something
> else?
>
> Thanks,
> Justin
>


-- 
Regards,
Ning Wang

11/05/2019 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi, all~,

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
1. Bug fix, improvements and new feature (instance resource) in Heron UI.
2. License cleaning up for jarjar and health handler.
3. Code reviews.

Regards,
--ning

Re: [Mentors] Looking for feedback on NOTICE file

[Ning Wang]

Thanks!

On Mon, Oct 21, 2019 at 12:53 PM P. Taylor Goetz  wrote:

> Yes, that is accurate.
>
> -Taylor
>
> > On Oct 21, 2019, at 3:26 PM, SiMing Weng  wrote:
> >
> > Hi, guys:
> >
> > I’m looking for some feedback on what work is required for the NOTICE <
> https://github.com/apache/incubator-heron/blob/master/NOTICE> file.
> >
> > According to the Licensing How-To <
> http://www.apache.org/dev/licensing-howto.html> guide on ASF, at least we
> knew:
> >
> > The minimum to start with is the “boilerplate” content as given in the
> guide, which we already have
> > The NOTICE file is meant for legally required notifications which are
> not satisfied by the LICENSE file or license copies under license folder
> > If any bundled dependency has its NOTICE file, we need to examine
> whether it has anything more than the ASF “boilerplate” content; if yes, we
> need to copy that into our NOTICE file
> > We only need to traverse BUNDLED dependencies
> > We need to traverse dependencies recursively
> >
> > Please feedback whether my understanding is correct or anything is
> missing.
> >
> > Cheers,
> >
> > SiMing
>
>

Re: [Discuss][Report] Heron November Podling Report

[Ning Wang]

Thanks Dave for the feedback! I am going to update it. Time flies and it's
good to start early. :)

On Mon, Oct 21, 2019 at 10:45 AM Dave Fisher  wrote:

> Hi -
>
> The report should list the names and dates of all Committers added since
> the last report. If none then name and date of the last committer. The same
> should be reported regarding PPMC members.
>
> There are still 10 days to go in the reporting period. ….
>
> Regards,
> Dave
>
> > On Oct 21, 2019, at 9:05 AM, Ning Wang  wrote:
> >
> > Hi,
> >
> > Thanks for the input. Totally agree that the big rocks are blockers.
> Let's
> > keep the status to be the same as last quarter then.
> >
> >
> > On Mon, Oct 21, 2019 at 9:00 AM Josh Fischer 
> wrote:
> >
> >> Thanks for sending out Ning.  I think you've covered most of the
> progress
> >> that has been made since the last report.  I've added a few additions as
> >> well.   I disagree on the podling's maturity to be marked as nearing
> >> graduation.  Once we have  a few of the big rocks left to move (the new
> >> static site up, licenses fixed, and we are releasing binaries) I feel we
> >> can then say we are nearing graduation.  There is still a fair amount of
> >> work to be done to get there.  Much less work than before, but still a
> ways
> >> to go.
> >>
> >> - Josh
> >>
> >> On Sun, Oct 20, 2019 at 1:09 PM Ning Wang  wrote:
> >>
> >>> Hey All,
> >>>
> >>> It's time to create our podling report for November.  I've copied the
> >>> document and added it to a google sheets document here:
> >>>
> >>>
> >>
> https://docs.google.com/document/d/15ynoG-v75xyxcczZjZgHuEtM7lHF9WeDEz_suq90xNk/edit#
> >>>
> >>> Please add any relevant information you can.  If you have any trouble
> >>> accessing or editing the document let me know.
> >>>
> >>> I feel we have made a lot of progress on the pre-graduation issues so I
> >>> marked the "Nearing graduation" line. Please feel free to comment on
> it.
> >>>
> >>> Thanks in advance.
> >>>
> >>> Regards,
> >>> --ning
> >>>
> >>
>
>

[Proposal][Discussion] Job cluster management

[Ning Wang]

Hi,

Here is the summary of another proposal regarding job cluster management a
few of us have been discussing.

By design, Heron topologies are totally independent from each other for
isolation and maintainability. However, as the result, the deployment (and
other topology management tasks) could be slow which is another operational
burden. The proposal is about how to improve topology and job cluster
management to ease the burden.

Any thoughts could be super helpful to make the proposal better since there
are different use cases. Please definitely feel free to reply. Thanks in
advance!

https://docs.google.com/document/d/1Oq8saQzrcJmUBaFYlVhY4HyevTOrkqzFZryAyVrkaFM/edit

Regards,
--ning

10/22/2019 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Hi, all~,

It has been two weeks since our last sync! Let's share our works for the
last two weeks in this thread.

My updates:
1. Prepared a November podling report
2. New code changes about metrics, UI and docker based dev environment.
3. Code reviews.

Regards,
--ning

Re: [Discuss][Report] Heron November Podling Report

[Ning Wang]

Hi,

Thanks for the input. Totally agree that the big rocks are blockers. Let's
keep the status to be the same as last quarter then.


On Mon, Oct 21, 2019 at 9:00 AM Josh Fischer  wrote:

> Thanks for sending out Ning.  I think you've covered most of the progress
> that has been made since the last report.  I've added a few additions as
> well.   I disagree on the podling's maturity to be marked as nearing
> graduation.  Once we have  a few of the big rocks left to move (the new
> static site up, licenses fixed, and we are releasing binaries) I feel we
> can then say we are nearing graduation.  There is still a fair amount of
> work to be done to get there.  Much less work than before, but still a ways
> to go.
>
> - Josh
>
> On Sun, Oct 20, 2019 at 1:09 PM Ning Wang  wrote:
>
> > Hey All,
> >
> > It's time to create our podling report for November.  I've copied the
> > document and added it to a google sheets document here:
> >
> >
> https://docs.google.com/document/d/15ynoG-v75xyxcczZjZgHuEtM7lHF9WeDEz_suq90xNk/edit#
> >
> > Please add any relevant information you can.  If you have any trouble
> > accessing or editing the document let me know.
> >
> > I feel we have made a lot of progress on the pre-graduation issues so I
> > marked the "Nearing graduation" line. Please feel free to comment on it.
> >
> > Thanks in advance.
> >
> > Regards,
> > --ning
> >
>

[Discuss][Report] Heron November Podling Report

[Ning Wang]

Hey All,

It's time to create our podling report for November.  I've copied the
document and added it to a google sheets document here:
https://docs.google.com/document/d/15ynoG-v75xyxcczZjZgHuEtM7lHF9WeDEz_suq90xNk/edit#

Please add any relevant information you can.  If you have any trouble
accessing or editing the document let me know.

I feel we have made a lot of progress on the pre-graduation issues so I
marked the "Nearing graduation" line. Please feel free to comment on it.

Thanks in advance.

Regards,
--ning

Re: New committer: Siming Weng

[Ning Wang]

Great!

On Fri, Oct 18, 2019 at 1:55 PM Karthik Ramasamy  wrote:

> +1
>
> On Fri, Oct 18, 2019 at 1:23 PM Neng Lu  wrote:
>
> > Hi all,
> >
> > The Project Management Committee (PMC) for Apache Heron-Incubator has
> > invited Siming Weng to become a committer and we are pleased to announce
> > that he has accepted.
> >
> > Siming Weng has been involved in the community for a while. He is active
> in
> > slack discussions, code contributions and code reviews.
> > His contributions include a wide variety of improvements and addition of
> > features into Heron
> >
> > https://github.com/apache/incubator-heron/commits?author=simingweng
> >
> > It will be good to bring him as a committer based on his contributions
> and
> > his positive impact on the project.
> >
> > Being a committer enables easier contribution to the project since there
> is
> > no need to go via the patch submission process. This should enable better
> > productivity. Being a PMC member enables assistance with the management
> and
> > to guide the direction of the project.
> >
> > Best regards,
> > Neng Lu
> >
>

Re: New committer: Xiaoyao Qian

[Ning Wang]

Great!

On Fri, Oct 18, 2019 at 1:55 PM Karthik Ramasamy  wrote:

> +1
>
> On Fri, Oct 18, 2019 at 1:47 PM Josh Fischer  wrote:
>
> > +1!
> >
> > On Fri, Oct 18, 2019 at 3:21 PM Neng Lu  wrote:
> >
> > > Hi all,
> > >
> > > The Project Management Committee (PMC) for Apache Heron-Incubator has
> > > invited Xiaoyao Qian to become a committer and we are pleased to
> announce
> > > that he has accepted.
> > >
> > > Xiaoyao Qian has been involved in the community for a while. He is
> active
> > > in slack discussions, code contributions and code reviews.
> > > His contributions include a wide variety of improvements and addition
> of
> > > features into Heron
> > >
> > > https://github.com/apache/incubator-heron/commits?author=xiaoyao1991
> > >
> > > Xiaoyao works in the Heron team at Twitter. It will be good to bring
> him
> > as
> > > a committer based on his contributions and his positive impact on the
> > > project.
> > >
> > > Being a committer enables easier contribution to the project since
> there
> > is
> > > no need to go via the patch submission process. This should enable
> better
> > > productivity. Being a PMC member enables assistance with the management
> > and
> > > to guide the direction of the project.
> > >
> > > Best regards,
> > > Neng Lu
> > >
> >
>

Re: [DISCUSSION] Heron UI improvement proposal

[Ning Wang]

It seems there are no objections~

I have created a few issues in project github page.

Also, if there are any other ideas about making UI better, let's discuss in
this thread.

Thanks in advance!



On Tue, Sep 24, 2019 at 5:09 PM Ning Wang  wrote:

> Hi, dear fellow Heron devs,
>
> Heron UI was built a few years ago using Storm UI as template. There
> haven't been many changes since then.
>
> We (Realtime compute team in Twitter) have been thinking of the
> information that can be useful for topology owners in their day to day jobs
> based on user feedback. Here is a proposal for discussion in a wider scope
> about the features we feel that should be added in Heron UI.
>
> Any comments and feedback are welcome! Thanks in advance!
>
>
> https://docs.google.com/document/d/1CaDVVifcWqViRO96tmePK5dwhOhYnhwjhMRuImg-aOs/edit
>
>
>
>
>

Re: 10/08/2019 Bi-Weekly OSS Heron Sync-up

[Ning Wang]

Google doc is convenient for collaboration and commenting, such as
highlighting a word and ask questions. This is tricky in email.

I agree that it could be helpful for general discussions to happen in the
mailing list though.

On Mon, Oct 7, 2019 at 8:02 PM Dave Fisher  wrote:

> Sorry I do see the start of a discussion thread ... I just don’t see any
> feedback on the list.
>
> We can’t count on a google doc in a few years.
>
> Sent from my iPhone
>
> > On Oct 7, 2019, at 8:00 PM, Dave Fisher  wrote:
> >
> > What’s this UI proposal? Is there a link?
> >
> > Sent from my iPhone
> >
> >> On Oct 7, 2019, at 7:51 PM, Josh Fischer  wrote:
> >>
> >> Hi All,
> >>
> >> Reviewed the UI Proposal..  Had further discussion about the new site.
> >>
> >> - Josh
> >>
> >>> On Mon, Oct 7, 2019 at 1:05 PM SiMing Weng 
> wrote:
> >>>
> >>> Hi, all:
> >>>
> >>> Review the UI proposal document
> >>> Contribute a PR to generate maven artifacts of Kafka Bolt
> >>>
> >>> Cheers,
> >>>
> >>> Simon Weng
> >>>
> >>>> On Oct 7, 2019, at 1:47 PM, Ning Wang  wrote:
> >>>>
> >>>> Hi, all~,
> >>>>
> >>>> It has been two weeks since our last sync! Let's share our works for
> the
> >>>> last two weeks in this thread.
> >>>>
> >>>> My updates:
> >>>> 1. created and shared a proposal for Heron UI improvements
> >>>> <
> >>>
> https://docs.google.com/document/d/1CaDVVifcWqViRO96tmePK5dwhOhYnhwjhMRuImg-aOs/edit#heading=h.rfm62inflf7w
> >>>> .
> >>>>
> >>>> 2. a few code changes in UI(parallelism, tooltip), metrics sink
> >>>> communicator, gc log files, and rate limiter example.
> >>>> 3. discussion about load topology config from yaml file.
> >>>>
> >>>>
> >>>> Regards,
> >>>> --ning
> >>>
> >>>
> >
>
>

[DISCUSSION] Heron UI improvement proposal

[Ning Wang]

Hi, dear fellow Heron devs,

Heron UI was built a few years ago using Storm UI as template. There
haven't been many changes since then.

We (Realtime compute team in Twitter) have been thinking of the information
that can be useful for topology owners in their day to day jobs based on
user feedback. Here is a proposal for discussion in a wider scope about the
features we feel that should be added in Heron UI.

Any comments and feedback are welcome! Thanks in advance!

https://docs.google.com/document/d/1CaDVVifcWqViRO96tmePK5dwhOhYnhwjhMRuImg-aOs/edit

Re: October Monthly Meetup invitation (#15)

[Ning Wang]

Thanks for the suggestions!

On Mon, Sep 23, 2019 at 4:18 PM Dave Fisher  wrote:

> Hi Sree,
>
> I plan on attending. There are a couple of points I’d like to make.
>
> It would be very helpful to Heron community growth if the Meetup text had
> a pointer to the podling website and included the Disclaimer.
>
> It would also be helpful for the PPMC to be involved in the selection of
> the speakers and topics. This is one of the ways that an Apache project
> should operate.
>
> Follow ups directed to the mailing lists and GitHub issues.
>
> Is there any video for any of the past talks? If so, then that ought to be
> shared from the Heron site.
>
> Perhaps for the November Meetup a topic could be how to contribute to
> Apache Heron directly.
>
> Best Regards,
> Dave
>
> > On Sep 22, 2019, at 10:47 PM, Sree Vaddi 
> wrote:
> >
> > Hi Team,
> > You are cordially invited to our monthly meetup in October.Please note
> the day is 1st Monday instead 3rd Monday for a change.
> >
> > https://www.meetup.com/Apache-Heron-Bay-Area/events/nglzdryznbcc/
> >
> >
> > Thank you./Sree
> >
>
>