Re: [Mentors] Releasing install scripts

[Dave Fisher]

Hi -

> On Jun 7, 2020, at 8:27 AM, Nicholas Nezis  wrote:
> 
> Mentors,
> 
> Could this be non-official convenience binaries as found here in the Apache
> Bookkeeper project?
> https://hub.docker.com/r/apache/bookkeeper/dockerfile

That script only pulls in the official Bookkeeper release.

A Heron build script like this would still need to conform to release policy. 
Class A and class B licensed dependencies are allowed, but class X dependencies 
must be optional, not required, and clearly marked. Users of the build script 
must be able to choose. Any docker image provided must conform.

Exceptions exist for operating system tools and build tools.

Let’s discuss the scripts.

Regards,
Dave

> 
> - Nick
> 
>> On Fri, Jun 5, 2020 at 10:14 PM Josh Fischer  wrote:
>> 
>> Hi,
>> 
>> I remember reading an email that said we could not release heron's install
>> scripts under Apache.  I've looked through my email, but cannot find the
>> email (so very possible I'm remembering this wrong).
>> 
>> Is it possible as part of  a release to create and distribute a select
>> number of operating system specific install scripts for a release?  They
>> are rather large in size  ~500MB each.
>> 
>> - Josh
>> 

Re: [DISCUSSION] Docker OS support

[Dave Fisher]

I thought that the purpose of this discussion was to find the one or two 
platforms for Heron to support in order to keep down the complexity of making 
the first binary release for the community.

Other platforms could be on a roadmap for future releases.

Regards,
Dave

> On May 26, 2020, at 12:26 PM, Ning Wang  wrote:
> 
> My concern about "latest" is that I feel most people hold it for some time
> before migrating to the latest version (if it ever happens). If there are
> numbers, it could make sense to use the most popular version (plus the
> other criterias).
> 
> On Tue, May 26, 2020 at 12:14 PM Oliver Bristow 
> wrote:
> 
>> Sorry if I missed the rationale for Ubuntu 18.04 instead of 20.04, but
>> would the latest LTS be a better choice? 18.04 was released before Debian
>> 10, so that may be an argument for 20.04 re. compatibility if Debian 10 is
>> the standard.
>> 
>> There's already a 20.04 Dockerfile + I It looks like the vagrant setup will
>> get an upgrade from 14.04 to 20.04 as a part of the python3 upgrade, and
>> seems to run fine so far.
>> 
>> On Tue, 26 May 2020 at 19:30, Josh Fischer  wrote:
>> 
>>> Thanks for further clarifying. Yes, that is correct.
>>> 
>>> 
>>> 
>>> On Tue, May 26, 2020 at 1:16 PM Ning Wang  wrote:
>>> 
 +1.
 
 So my understanding is:
 binary (in future release): Debian 10.
 source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers will be
 updated in the future after more discussions/votes.
 
 others will be moved to a "extra" folder only for reference.
 
 
 
 On Tue, May 26, 2020 at 6:13 AM Josh Fischer 
>>> wrote:
 
> Hi All,
> 
> Based on what I've read on this thread a combination of CentOS/Debian
 would
> be the better options for us to support multiple versions for the
> community.  We will also support one version of Ubuntu.
> 
> So. I will start a vote in 72 hours that calls on the vote on
>> this.
> I'm going to write down my understanding of what this thread covers
>> to
 get
> some actionable topics to vote on.  If I've written anything down
> incorrectly please speak up before we start a vote.
> 
> Debian:
>- Debian10 is the official docker image to support and be
>> published
 to
> Docker Hub, we will also support Debian9.
> CentOS:
>- We will support CentOS7.  Note, I do not think it has native
> Python3.7 support, but we are also not there yet.
> Ubuntu:
>- We will still support one version. At this time the version is
 18.04.
> 
> We will upgrade supported versions of each OS as updates/deprecation
>> of
> support are needed.
> 
> Again let's wait 72 hours before starting a vote.  Also please check
>>> that
> the statements I wrote down are what you understood!
> 
> See below for notes on lazy consensus:
> https://community.apache.org/committers/lazyConsensus.html
> 
> - Josh
> 
> 
> On Wed, May 20, 2020 at 9:41 PM Ning Wang 
>>> wrote:
> 
>> +1
>> 
>> On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis <
 nicholas.ne...@gmail.com
>> 
>> wrote:
>> 
>>> I like those options.
>>> 
>>> On Wed, May 20, 2020 at 10:15 PM Josh Fischer <
>> j...@joshfischer.io
 
>> wrote:
>>> 
 Let's wait 3 more days for anyone to give input.  Then we can
 start a
>>> vote
 to make a final decision on which containers for us to support.
 Keep
>> in
 mind that even though we do need to plan for Python3 support
>>> (There
> is
>>> work
 in progress now) we do not need it as of yet.
 
 - Josh
 
 On Wed, May 20, 2020 at 8:34 PM thinker0 
 wrote:
 
> My experience is mainly for CentOS 7, so I think it would be
>>> nice
> if
>> it
 was
> included.
> 
> 2020년 5월 21일 (목) 오전 9:56, Josh Fischer >> 님이
 작성:
> 
>> Well thought out Windham.  I like how you narrowed the
>>> criteria
>> down
>>> to
>> Debian10/CentOS7(depending on PY3 support) thinking about
 overall
 support
>> and what others would need/want.
>> 
>> +1
>> 
>> - Josh
>> 
>> On Mon, May 18, 2020 at 8:39 PM Windham Wong <
>>> windham.w...@stormeye.io
> 
>> wrote:
>> 
>>> My experience is that, people tends to pick their
>> container
> image
>>> OS
>> based
>>> on few criterias only:
>>> 1) image size, alpine always win, then debian/centos and
>>> 3rd
 is
>>> ubuntu/centos. alpine is always the smallest but not much
> library
>> provided
>>> and require to compile manually. debian is always good
 because
> it
>>> has
>>> apt-get to provide loads of library. centos is cool as
>>> debian

Re: [MENTORS] Create a Heron Space in Confluence

[Dave Fisher]

Done!

> On Apr 3, 2020, at 9:09 AM, Ning Wang  wrote:
> 
> Hi, Dave,
> 
> Could you please add me as well? Thanks in advance!
> 
> On Fri, Apr 3, 2020 at 8:39 AM Dave Fisher  wrote:
> 
>> Hi -
>> 
>> Apologies for the delay in responding. Both Josh and Nicholas have full
>> permissions to the Confluence space.
>> 
>> If others would like permission from the community then please confirm you
>> login and respond to this dev list!
>> 
>> Regards,
>> Dave
>> 
>>> On Mar 14, 2020, at 6:55 PM, Josh Fischer  wrote:
>>> 
>>> It looks like the Heron space in Confluence is unrestricted but I do not
>>> see a way to edit or create content in the space.  Anyone have an idea of
>>> what needs to be done?
>>> 
>>> - Josh
>>> 
>>> On Mon, Feb 24, 2020 at 11:24 AM Josh Fischer 
>> wrote:
>>> 
>>>> I can see the Heron space in Confluence but I cannot see it as one of
>> the
>>>> spaces I can add documents to.  Is there something else that needs to be
>>>> done?
>>>> 
>>>> - Josh
>>>> 
>>>> On Fri, Feb 14, 2020 at 2:14 PM Nicholas Nezis <
>> nicholas.ne...@gmail.com>
>>>> wrote:
>>>> 
>>>>> Just tried logging in. I was able to log in, but then got "Not
>> Permitted"
>>>>> when viewing the Heron space.
>>>>> 
>>>>> On Fri, Feb 14, 2020 at 2:09 PM Josh Fischer 
>> wrote:
>>>>> 
>>>>>> Fantastic.  Thank you for the help Dave.
>>>>>> 
>>>>>> On Fri, Feb 14, 2020 at 12:53 PM Dave Fisher  wrote:
>>>>>> 
>>>>>>> The Space is created -
>>>>> https://cwiki.apache.org/confluence/display/HERON
>>>>>>> with Nick as admin.
>>>>>>> 
>>>>>>>> On Feb 12, 2020, at 1:27 PM, Ning Wang 
>>>>> wrote:
>>>>>>>> 
>>>>>>>> sgtm either way.
>>>>>>>> 
>>>>>>>> On Wed, Feb 12, 2020 at 1:19 PM Josh Fischer 
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> If Nick has the experience, I'd say let him do it.
>>>>>>>>> 
>>>>>>>>> - Josh
>>>>>>>>> 
>>>>>>>>> On Wed, Feb 12, 2020 at 3:16 PM Nicholas Nezis <
>>>>>>> nicholas.ne...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> I also can help. Have a bit of experience from using Confluence at
>>>>>>> work.
>>>>>>>>>> 
>>>>>>>>>> On Wed, Feb 12, 2020 at 4:01 PM Josh Fischer >>>>> 
>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> I did some digging around there today to understand enough that
>>>>> we
>>>>>>>>>> should
>>>>>>>>>>> have our own space.
>>>>>>>>>>> 
>>>>>>>>>>> The rest I can figure out as we go.
>>>>>>>>>>> 
>>>>>>>>>>> I hope that works for you.
>>>>>>>>>>> 
>>>>>>>>>>> - Josh
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Feb 12, 2020 at 2:52 PM Dave Fisher 
>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Hi -
>>>>>>>>>>>> 
>>>>>>>>>>>> I can make this request, but first who understands Confluence
>>>>>> enough
>>>>>>>>> to
>>>>>>>>>>> be
>>>>>>>>>>>> the Space Admin?
>>>>>>>>>>>> 
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Dave
>>>>>>>>>>>> 
>>>>>>>>>>>>> On Feb 12, 2020, at 12:50 PM, SiMing Weng <
>>>>> siming.w...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> +1
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Feb 12, 2020, at 2:15 PM, Ning Wang 
>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Sounds like a great idea to me~
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Wed, Feb 12, 2020 at 11:14 AM Josh Fischer <
>>>>>>>>> j...@joshfischer.io>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Could we get a space created in confluence for the Heron
>>>>>> project?
>>>>>>>>>>> I'd
>>>>>>>>>>>> like
>>>>>>>>>>>>>>> to add some material there for the community.  I think it
>>>>> can be
>>>>>>>>>> done
>>>>>>>>>>>> here:
>>>>>>>>>>>>>>> https://selfserve.apache.org/
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> - Josh
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> --
>>>>>>>>>>> Sent from A Mobile Device
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> 
>> 

Re: [MENTORS] Create a Heron Space in Confluence

[Dave Fisher]

Hi -

Apologies for the delay in responding. Both Josh and Nicholas have full 
permissions to the Confluence space.

If others would like permission from the community then please confirm you 
login and respond to this dev list!

Regards,
Dave

> On Mar 14, 2020, at 6:55 PM, Josh Fischer  wrote:
> 
> It looks like the Heron space in Confluence is unrestricted but I do not
> see a way to edit or create content in the space.  Anyone have an idea of
> what needs to be done?
> 
> - Josh
> 
> On Mon, Feb 24, 2020 at 11:24 AM Josh Fischer  wrote:
> 
>> I can see the Heron space in Confluence but I cannot see it as one of the
>> spaces I can add documents to.  Is there something else that needs to be
>> done?
>> 
>> - Josh
>> 
>> On Fri, Feb 14, 2020 at 2:14 PM Nicholas Nezis 
>> wrote:
>> 
>>> Just tried logging in. I was able to log in, but then got "Not Permitted"
>>> when viewing the Heron space.
>>> 
>>> On Fri, Feb 14, 2020 at 2:09 PM Josh Fischer  wrote:
>>> 
>>>> Fantastic.  Thank you for the help Dave.
>>>> 
>>>> On Fri, Feb 14, 2020 at 12:53 PM Dave Fisher  wrote:
>>>> 
>>>>> The Space is created -
>>> https://cwiki.apache.org/confluence/display/HERON
>>>>> with Nick as admin.
>>>>> 
>>>>>> On Feb 12, 2020, at 1:27 PM, Ning Wang 
>>> wrote:
>>>>>> 
>>>>>> sgtm either way.
>>>>>> 
>>>>>> On Wed, Feb 12, 2020 at 1:19 PM Josh Fischer 
>>>>> wrote:
>>>>>> 
>>>>>>> If Nick has the experience, I'd say let him do it.
>>>>>>> 
>>>>>>> - Josh
>>>>>>> 
>>>>>>> On Wed, Feb 12, 2020 at 3:16 PM Nicholas Nezis <
>>>>> nicholas.ne...@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> I also can help. Have a bit of experience from using Confluence at
>>>>> work.
>>>>>>>> 
>>>>>>>> On Wed, Feb 12, 2020 at 4:01 PM Josh Fischer >>> 
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> I did some digging around there today to understand enough that
>>> we
>>>>>>>> should
>>>>>>>>> have our own space.
>>>>>>>>> 
>>>>>>>>> The rest I can figure out as we go.
>>>>>>>>> 
>>>>>>>>> I hope that works for you.
>>>>>>>>> 
>>>>>>>>> - Josh
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Wed, Feb 12, 2020 at 2:52 PM Dave Fisher 
>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Hi -
>>>>>>>>>> 
>>>>>>>>>> I can make this request, but first who understands Confluence
>>>> enough
>>>>>>> to
>>>>>>>>> be
>>>>>>>>>> the Space Admin?
>>>>>>>>>> 
>>>>>>>>>> Regards,
>>>>>>>>>> Dave
>>>>>>>>>> 
>>>>>>>>>>> On Feb 12, 2020, at 12:50 PM, SiMing Weng <
>>> siming.w...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> +1
>>>>>>>>>>> 
>>>>>>>>>>>> On Feb 12, 2020, at 2:15 PM, Ning Wang 
>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Sounds like a great idea to me~
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Feb 12, 2020 at 11:14 AM Josh Fischer <
>>>>>>> j...@joshfischer.io>
>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Could we get a space created in confluence for the Heron
>>>> project?
>>>>>>>>> I'd
>>>>>>>>>> like
>>>>>>>>>>>>> to add some material there for the community.  I think it
>>> can be
>>>>>>>> done
>>>>>>>>>> here:
>>>>>>>>>>>>> https://selfserve.apache.org/
>>>>>>>>>>>>> 
>>>>>>>>>>>>> - Josh
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>> Sent from A Mobile Device
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>> 

Re: our first virtual event this evening

[Dave Fisher]

Hi Sree,

How did this meeting go?

Regards,
Dave

> On Mar 23, 2020, at 12:08 PM, Sree Vaddi  
> wrote:
> 
> Hi Herons,
> I request you to join our first virtual event and help make it successful.
> https://www.meetup.com/Apache-Heron-Bay-Area/events/269361401/
> 
> 
> Thank you./Sree
> 

Re: [MENTORS] Create a Heron Space in Confluence

[Dave Fisher]

The Space is created - https://cwiki.apache.org/confluence/display/HERON with 
Nick as admin.

> On Feb 12, 2020, at 1:27 PM, Ning Wang  wrote:
> 
> sgtm either way.
> 
> On Wed, Feb 12, 2020 at 1:19 PM Josh Fischer  wrote:
> 
>> If Nick has the experience, I'd say let him do it.
>> 
>> - Josh
>> 
>> On Wed, Feb 12, 2020 at 3:16 PM Nicholas Nezis 
>> wrote:
>> 
>>> I also can help. Have a bit of experience from using Confluence at work.
>>> 
>>> On Wed, Feb 12, 2020 at 4:01 PM Josh Fischer 
>> wrote:
>>> 
>>>> I did some digging around there today to understand enough that we
>>> should
>>>> have our own space.
>>>> 
>>>> The rest I can figure out as we go.
>>>> 
>>>> I hope that works for you.
>>>> 
>>>> - Josh
>>>> 
>>>> 
>>>> On Wed, Feb 12, 2020 at 2:52 PM Dave Fisher  wrote:
>>>> 
>>>>> Hi -
>>>>> 
>>>>> I can make this request, but first who understands Confluence enough
>> to
>>>> be
>>>>> the Space Admin?
>>>>> 
>>>>> Regards,
>>>>> Dave
>>>>> 
>>>>>> On Feb 12, 2020, at 12:50 PM, SiMing Weng 
>>>> wrote:
>>>>>> 
>>>>>> +1
>>>>>> 
>>>>>>> On Feb 12, 2020, at 2:15 PM, Ning Wang 
>>> wrote:
>>>>>>> 
>>>>>>> Sounds like a great idea to me~
>>>>>>> 
>>>>>>> On Wed, Feb 12, 2020 at 11:14 AM Josh Fischer <
>> j...@joshfischer.io>
>>>>> wrote:
>>>>>>> 
>>>>>>>> Hi,
>>>>>>>> 
>>>>>>>> Could we get a space created in confluence for the Heron project?
>>>> I'd
>>>>> like
>>>>>>>> to add some material there for the community.  I think it can be
>>> done
>>>>> here:
>>>>>>>> https://selfserve.apache.org/
>>>>>>>> 
>>>>>>>> - Josh
>>>>>>>> 
>>>>>> 
>>>>> 
>>>>> --
>>>> Sent from A Mobile Device
>>>> 
>>> 
>> 

Re: [MENTORS] Create a Heron Space in Confluence

[Dave Fisher]

Hi -

I can make this request, but first who understands Confluence enough to be the 
Space Admin?

Regards,
Dave

> On Feb 12, 2020, at 12:50 PM, SiMing Weng  wrote:
> 
> +1
> 
>> On Feb 12, 2020, at 2:15 PM, Ning Wang  wrote:
>> 
>> Sounds like a great idea to me~
>> 
>> On Wed, Feb 12, 2020 at 11:14 AM Josh Fischer  wrote:
>> 
>>> Hi,
>>> 
>>> Could we get a space created in confluence for the Heron project?  I'd like
>>> to add some material there for the community.  I think it can be done here:
>>> https://selfserve.apache.org/
>>> 
>>> - Josh
>>> 
> 

Re: [DISCUSS] Binary Release Questions

[Dave Fisher]

Sent from my iPhone

> On Feb 3, 2020, at 12:55 PM, Josh Fischer  wrote:
> 
> Nice!! Thank you Dave.  I'm thinking we should research this path of using
> the this openjdk (Debian as we call it)  container and present our findings
> to legal.

Only if there are questions.
> I'm also assuming we will have to look at each package that is
> installed in the container during build time and check those licenses as
> well.

Yes!

Regards,
Dave
> 
> - Josh
> 
>> On Mon, Feb 3, 2020 at 2:44 PM Dave Fisher  wrote:
>> 
>> Hi -
>> 
>> Regarding OpenJDK and GPL2 - here is what Roman the VP, Legal wrote when
>> answering Beam’s questions.
>> 
>> Roman Shaposhnik commented on LEGAL-503:
>> 
>> 
>> Hey [~altay] if you would like to continue linking to the Docker release
>> artifact from the
>> https://beam.apache.org
>> you will have:
>>   1. Transition to the official ASF dockerhub org:
>> https://hub.docker.com/u/apache
>>   2. Start including that binary convenience artifact into your VOTE
>> threads on Beam releases
>>   3. Make sure that all Cat-X licenses are ONLY brought into your
>> container via FROM statements
>> 
>> Regards,
>> Dave
>> 
>>>> On Feb 3, 2020, at 12:34 PM, Josh Fischer  wrote:
>>> 
>>> I think what Nick is talking about is Travis CI, not Jenkins.  It  seems
>>> that ubuntu LTS may have some issues with creating the container.   This
>>> may put a lot of work ahead of us.  My first thought is to use Debian as
>>> the "official" Heron container for apache.  It is built from the
>>> openjdk:8-jdk-slim  docker image.  I know that Tomcat uses a
>>> similar container from the openjdk org.  It might be the path of least
>>> resistance when it comes to making sure the licenses are ok.  But a
>>> possible hang up with the openjdk container is that is uses a GPL2
>> license
>>> which is not compatible with Apache (this is my understanding).  So I'm
>>> thinking of running the questions about the Debian container to legal.
>>> Any thoughts?
>>> 
>>>> On Mon, Feb 3, 2020 at 12:48 PM Ning Wang  wrote:
>>> 
>>>> I am fine with keeping the docker files.
>>>> 
>>>> It is a good point that Jenkins machine is a factor.
>>>> 
>>>> On Mon, Feb 3, 2020 at 9:27 AM Nicholas Nezis >> 
>>>> wrote:
>>>> 
>>>>> Linux and MacOS installer definitely. For the MacOS users, it would be
>>>>> awesome to maintain being able to `brew install heron`
>>>>> 
>>>>> As to the images, I definitely think it would be better to keep a
>> smaller
>>>>> set of Dockerfiles. With my move to Bazel 2.0 it has been painful
>> working
>>>>> through the various build issues related to the different OS builds. If
>>>> the
>>>>> Dockerfiles are meant to provide people with the list of packages for
>>>> their
>>>>> local install, perhaps maintaining them is ok. If the goal is to have
>> an
>>>>> isolated build container and runtime container, then having a single
>>>> option
>>>>> makes more sense.
>>>>> 
>>>>> Short term:
>>>>> Ubuntu 14.04 is used in the Travis CI build so having them be
>> consistent
>>>>> makes sense to me. This is the image I would focus on.
>>>>> 
>>>>> Long term:
>>>>> We should update things to use a newer Ubuntu LTS version if possible.
>>>>> There are some issues that might be blockers:
>>>>> - cppcheck doesn't compile on Ubuntu 18.04 (
>>>>> https://github.com/apache/incubator-heron/issues/3440)
>>>>> - TravisCI expects JDK 9+ on Ubuntu 16+
>>>>> - DNS issue with Ubuntu in Kubernetes (
>>>>> 
>>>>> 
>>>> 
>> https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/#known-issues
>>>>> )
>>>>> 
>>>>> 
>>>>> On Mon, Feb 3, 2020 at 11:58 AM Ning Wang 
>> wrote:
>>>>> 
>>>>>> For installer, I feel that MacOS should be included.
>>>>>> For docker images, we may choose one to release. I don't really have a
>>>>>> preference. Maybe market share is a good indicator. I think Ubuntu was
>>>>> #1 a
>>>>>> few years ago, but I am not sure what is the

Re: [DISCUSS] Binary Release Questions

[Dave Fisher]

Hi -

Regarding OpenJDK and GPL2 - here is what Roman the VP, Legal wrote when 
answering Beam’s questions.

Roman Shaposhnik commented on LEGAL-503:


Hey [~altay] if you would like to continue linking to the Docker release 
artifact from the 
https://beam.apache.org
 you will have:
   1. Transition to the official ASF dockerhub org:  
https://hub.docker.com/u/apache
   2. Start including that binary convenience artifact into your VOTE threads 
on Beam releases
   3. Make sure that all Cat-X licenses are ONLY brought into your container 
via FROM statements

Regards,
Dave

> On Feb 3, 2020, at 12:34 PM, Josh Fischer  wrote:
> 
> I think what Nick is talking about is Travis CI, not Jenkins.  It  seems
> that ubuntu LTS may have some issues with creating the container.   This
> may put a lot of work ahead of us.  My first thought is to use Debian as
> the "official" Heron container for apache.  It is built from the
> openjdk:8-jdk-slim  docker image.  I know that Tomcat uses a
> similar container from the openjdk org.  It might be the path of least
> resistance when it comes to making sure the licenses are ok.  But a
> possible hang up with the openjdk container is that is uses a GPL2 license
> which is not compatible with Apache (this is my understanding).  So I'm
> thinking of running the questions about the Debian container to legal.
> Any thoughts?
> 
> On Mon, Feb 3, 2020 at 12:48 PM Ning Wang  wrote:
> 
>> I am fine with keeping the docker files.
>> 
>> It is a good point that Jenkins machine is a factor.
>> 
>> On Mon, Feb 3, 2020 at 9:27 AM Nicholas Nezis 
>> wrote:
>> 
>>> Linux and MacOS installer definitely. For the MacOS users, it would be
>>> awesome to maintain being able to `brew install heron`
>>> 
>>> As to the images, I definitely think it would be better to keep a smaller
>>> set of Dockerfiles. With my move to Bazel 2.0 it has been painful working
>>> through the various build issues related to the different OS builds. If
>> the
>>> Dockerfiles are meant to provide people with the list of packages for
>> their
>>> local install, perhaps maintaining them is ok. If the goal is to have an
>>> isolated build container and runtime container, then having a single
>> option
>>> makes more sense.
>>> 
>>> Short term:
>>> Ubuntu 14.04 is used in the Travis CI build so having them be consistent
>>> makes sense to me. This is the image I would focus on.
>>> 
>>> Long term:
>>> We should update things to use a newer Ubuntu LTS version if possible.
>>> There are some issues that might be blockers:
>>> - cppcheck doesn't compile on Ubuntu 18.04 (
>>> https://github.com/apache/incubator-heron/issues/3440)
>>> - TravisCI expects JDK 9+ on Ubuntu 16+
>>> - DNS issue with Ubuntu in Kubernetes (
>>> 
>>> 
>> https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/#known-issues
>>> )
>>> 
>>> 
>>> On Mon, Feb 3, 2020 at 11:58 AM Ning Wang  wrote:
>>> 
 For installer, I feel that MacOS should be included.
 For docker images, we may choose one to release. I don't really have a
 preference. Maybe market share is a good indicator. I think Ubuntu was
>>> #1 a
 few years ago, but I am not sure what is the current case.
 
 So overall my vote would be,
 docker image: ubuntu or current #1 market share wise if we can find the
 information.
 installer: MacOS + the same OS as the docker image.
 
 
 On Mon, Feb 3, 2020 at 5:02 AM Josh Fischer 
>> wrote:
 
> Any thoughts on this email?
> 
> Hi All,
> 
> After  several conversations with people across the Heron repo we
>> keep
> hearing that a Heron convenience binary release would be appreciated.
> Based on some feedback from Dave we need to decide on what type of
> packaging is helpful to Heron users as the first step to getting
>> people
> what they want/need.
> 
> Right now we have, but not released in a while:
> 
> Heron Docker Containers:
> - CentOS
> - Ubuntu
> - Debian
> Heron install scripts
> - CentOS
> - Darwin (MacOs)
> - Ubuntu
> 
> Does anyone have a preference on which package and distro they would
>>> like
> us to start with?  If possible, I would like us to scope down to one
> supported docker image to use for Heron.  Maintaining 3 separate
>> images
 is
> quite a task.
> 
> On Tue, Jan 28, 2020 at 4:51 PM Josh Fischer 
 wrote:
> 
>> Hi All,
>> 
>> After  several conversations with people across the Heron repo we
>>> keep
>> hearing that a Heron convenience binary release would be
>> appreciated.
>> Based on some feedback from Dave we need to decide on what type of
>> packaging is helpful to Heron users as the first step to getting
>>> people
>> what they want/need.
>> 
>> Right now we have, but not released in a while:
>> 
>> Heron Docker Containers:
>> - CentOS
>> - Ubuntu

Re: [DISCUSS][MENTORS] 0.20.2-incubating binary release

[Dave Fisher]

Infra had three concerns which roughly are:

(1) The aggregate size of the binary release. There are rules about releases 
larger than 1GB which put pressure on the mirror providers. Warning to Infra is 
required prior to putting these packages on Apache Distribution servers.

(2) Apache binary releases are termed unofficial conveniences provided by the 
community which follow normal Apache Release Policy. The question is what 
packaging is helpful for Heron’s users? The tar.gz question is the correct one.

(3) Since the binary packages are large have we checked that all the included 
licenses are correct?

Let’s work out (2) and (3) before rechecking about (1)

Regards,
Dave

Sent from my iPhone

> On Jan 27, 2020, at 6:48 PM, Josh Fischer  wrote:
> 
> Hi,
> 
> Part of the Heron binary release is more than just the client artifacts
> that would be loaded into Maven Central.  For example if you look at an old
> release prior to Heron being donated to Apache we have install scripts and
> tarred binaries for use in different operating systems. You can view them
> here -> https://github.com/apache/incubator-heron/releases/tag/0.17.8.
> Where would be go about hosting these install scripts and tar.gz files?  I
> think each of them are ~500MB in size today.  My initial thoughts (for
> simplicity) are that we could ignore the tar.gz files containing binaries
> and just offer install scripts and one supported docker image.  If binaries
> are needed for manual installation I think people could just compile parts
> of Heron at that point.  I'll look into the licensing/policy issues on the
> Heron docker container in the meantime.
> 
> - Josh
> 
>> On Fri, Jan 17, 2020 at 7:24 PM Dave Fisher  wrote:
>> 
>> The ASF as a whole is rather conservative in the sense of being slow to
>> change.
>> 
>> Maven Central for JAR packaging has been around for over a decade AND is
>> supported by Apache projects.
>> 
>> Docker is comparatively new and there is less policy around it. There is
>> some, but perhaps not well documented.
>> 
>> I think we should evaluate the docker file and see what licensing/policy
>> questions there are. We can then ask the VP, Legal Affairs on
>> legal-disc...@apache.org
>> 
>> Regards,
>> Dave
>> 
>> Sent from my iPhone
>> 
>>>> On Jan 17, 2020, at 2:41 PM, Josh Fischer  wrote:
>>> 
>>> Gotcha.  I understand the difference in the two now, but I'm not sure
>> why
>>> one would be allowed over the other.  At the end of the day they are both
>>> compiled binaries released under the Apache foundation.  I'll take some
>>> time to read through apache docs to see if I can get a better
>> understanding
>>> of the differences in the two.
>>> 
>>>> On Fri, Jan 17, 2020 at 4:28 PM Ning Wang  wrote:
>>>> 
>>>> That's true. Maven artifacts are binaries too. My bad.
>>>> 
>>>> The "binary" I was referring to are the executables, such as installer,
>>>> which contains quite some executables like CLI, exeample jobs, UI,
>> tracker,
>>>> etc. The docker image is also part of my "binary".
>>>> 
>>>> Basically users can just install and run their jobs without compiling
>> all
>>>> the tools from source code. Maven artifacts allow users to build their
>>>> jobs. They still need to compile all the other things in order to run
>> the
>>>> jobs.
>>>> 
>>>> I hope it is more clear this time.
>>>> 
>>>> 
>>>> 
>>>>> On Fri, Jan 17, 2020 at 1:59 PM Josh Fischer 
>> wrote:
>>>>> 
>>>>> I'm not sure I understand how you are defining the differences between
>>>> the
>>>>> terms "maven artifacts" and "binary releases".  Wouldn't a maven
>>>> artifact (
>>>>> java jar ) be considered a binary release as well?  Am I missing
>>>> something?
>>>>> 
>>>>> 
>>>>>> On Fri, Jan 17, 2020 at 1:02 AM Ning Wang 
>> wrote:
>>>>> 
>>>>>> I think we can not make an official binary release yet before clear
>> all
>>>>>> license issues. Therefore, 0.20.0.2 can have maven artifacts but it
>>>> can't
>>>>>> have binary release. :(
>>>>>> 
>>>>>> That's just my understanding of Apache requirements.
>>>>>> 
>>>>>> On Thu, Jan 16, 2020 at 2:24 PM Josh Fischer 
>>>>> wrote:
>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> I'd like to start the process to vote for the 0.20.2-incubating
>>>> binary
>>>>>>> release.  Before we start I remember there were concerns about the
>>>> size
>>>>>> of
>>>>>>> the Heron binary and where it would be hosted on Apache Infra.  Does
>>>>>> anyone
>>>>>>> have any knowledge on this potential issue?
>>>>>>> 
>>>>>>> - Josh
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> 
>> 

Re: [DISCUSS][MENTORS] 0.20.2-incubating binary release

[Dave Fisher]

The ASF as a whole is rather conservative in the sense of being slow to change.

Maven Central for JAR packaging has been around for over a decade AND is 
supported by Apache projects.

Docker is comparatively new and there is less policy around it. There is some, 
but perhaps not well documented.

I think we should evaluate the docker file and see what licensing/policy 
questions there are. We can then ask the VP, Legal Affairs on 
legal-disc...@apache.org

Regards,
Dave

Sent from my iPhone

> On Jan 17, 2020, at 2:41 PM, Josh Fischer  wrote:
> 
> Gotcha.  I understand the difference in the two now, but I'm not sure why
> one would be allowed over the other.  At the end of the day they are both
> compiled binaries released under the Apache foundation.  I'll take some
> time to read through apache docs to see if I can get a better understanding
> of the differences in the two.
> 
>> On Fri, Jan 17, 2020 at 4:28 PM Ning Wang  wrote:
>> 
>> That's true. Maven artifacts are binaries too. My bad.
>> 
>> The "binary" I was referring to are the executables, such as installer,
>> which contains quite some executables like CLI, exeample jobs, UI, tracker,
>> etc. The docker image is also part of my "binary".
>> 
>> Basically users can just install and run their jobs without compiling all
>> the tools from source code. Maven artifacts allow users to build their
>> jobs. They still need to compile all the other things in order to run the
>> jobs.
>> 
>> I hope it is more clear this time.
>> 
>> 
>> 
>>> On Fri, Jan 17, 2020 at 1:59 PM Josh Fischer  wrote:
>>> 
>>> I'm not sure I understand how you are defining the differences between
>> the
>>> terms "maven artifacts" and "binary releases".  Wouldn't a maven
>> artifact (
>>> java jar ) be considered a binary release as well?  Am I missing
>> something?
>>> 
>>> 
 On Fri, Jan 17, 2020 at 1:02 AM Ning Wang  wrote:
>>> 
 I think we can not make an official binary release yet before clear all
 license issues. Therefore, 0.20.0.2 can have maven artifacts but it
>> can't
 have binary release. :(
 
 That's just my understanding of Apache requirements.
 
 On Thu, Jan 16, 2020 at 2:24 PM Josh Fischer 
>>> wrote:
 
> Hi,
> 
> I'd like to start the process to vote for the 0.20.2-incubating
>> binary
> release.  Before we start I remember there were concerns about the
>> size
 of
> the Heron binary and where it would be hosted on Apache Infra.  Does
 anyone
> have any knowledge on this potential issue?
> 
> - Josh
> 
 
>>> 
>> 

Re: Podling Heron Report Reminder - February 2020

[Dave Fisher]

The next step is to discuss each binary dependency and if it is compatible.

Sent from my iPhone

> On Jan 16, 2020, at 10:59 PM, Ning Wang  wrote:
> 
> Therefore, 0.20.0.2 can have maven artifacts but it can't have binary
> release. :(
> 
>> On Thu, Jan 16, 2020 at 10:58 PM Ning Wang  wrote:
>> 
>> I think we can not make an official binary release yet before clear all
>> license issues.
>> 
>>> On Thu, Jan 16, 2020 at 2:20 PM Josh Fischer  wrote:
>>> 
>>> Hi,
>>> 
>>> I've been meaning to get a vote out for the 0.20.2-incubating binaries, I
>>> just haven't had the time to get around to it yet.  I know some people are
>>> needing them.  I'm going to start a discussion email today to discuss the
>>> binary release for this version, I think there were concerns from the
>>> Apache infra side of things because of the size of the binary.
>>> 
>>>> On Thu, Jan 16, 2020 at 4:14 PM Ning Wang  wrote:
>>> 
>>>> I am hoping to have at least 1 binary release before marking #2 as
>>>> complete.
>>>> 
>>>> On Thu, Jan 16, 2020 at 10:20 AM Josh Fischer 
>>> wrote:
>>>> 
>>>>> Hi,
>>>>> inlined
>>>>> 
>>>>> 
>>>>> 
>>>>> On Thu, Jan 16, 2020 at 10:11 AM Dave Fisher  wrote:
>>>>> 
>>>>>> Hi -
>>>>>> 
>>>>>> The list of three actions towards graduation should include current
>>>>>> actions and not ones that are completed.
>>>>>> 
>>>>>> I have questions on #1. What are the licensing issues that remain?
>>> How
>>>>> are
>>>>>> these being tracked?
>>>>>> 
>>>>> ** Github issues at the moment.  The licensing issue is mentioned here
>>>>> https://github.com/apache/incubator-heron/issues/3406 .
>>>>> 
>>>>>> ** I don't think we need to list making several releases anymore.  I
>>>>> think
>>>>> we've demonstrated that we can create source releases.
>>>>> 
>>>>>> 
>>>>>> #3 should be improving the amount of community discussion and
>>> decision
>>>>>> making on the dev@ mailing list. Monthly Meetups are only in
>>> Sunnyvale
>>>>>> and exclude most of the world. It is a critical part of the Apache
>>> Way
>>>> to
>>>>>> make sure discussion and decision making happens on the mailing
>>> list.
>>>>>> 
>>>>> ** agreed
>>>>> 
>>>>>> 
>>>>>> On last committers and PPMC members please provide the date of the
>>>>>> elections.
>>>>>> 
>>>>> 
>>>>> ** will do
>>>>> 
>>>>>> 
>>>>>> Regards,
>>>>>> Dave
>>>>>> 
>>>>>>> On Jan 16, 2020, at 12:27 AM, Ning Wang 
>>>> wrote:
>>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> I prepared a draft of the Feb podling report:
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> https://docs.google.com/document/d/1_8ckuEuL7pdo-eOCHaXs32ymD2_qRKzEybL2Zyr6gXU/edit#
>>>>>>> 
>>>>>>> Please help to update if anything is missing. Thanks!
>>>>>>> 
>>>>>>> On Wed, Jan 15, 2020 at 4:26 PM  wrote:
>>>>>>> 
>>>>>>>> Dear podling,
>>>>>>>> 
>>>>>>>> This email was sent by an automated system on behalf of the
>>> Apache
>>>>>>>> Incubator PMC. It is an initial reminder to give you plenty of
>>> time
>>>> to
>>>>>>>> prepare your quarterly board report.
>>>>>>>> 
>>>>>>>> The board meeting is scheduled for Wed, 19 February 2020, 10:30
>>> am
>>>>> PDT.
>>>>>>>> The report for your podling will form a part of the Incubator PMC
>>>>>>>> report. The Incubator PMC requires your report to be submitted 2
>>>> weeks
>>>>>>>> before the board meeting, to allow sufficient time for review and
>>>>>>>> submission (Wed, February 

Re: Podling Heron Report Reminder - February 2020

[Dave Fisher]

Hi -

The list of three actions towards graduation should include current actions and 
not ones that are completed.

I have questions on #1. What are the licensing issues that remain? How are 
these being tracked?

#3 should be improving the amount of community discussion and decision making 
on the dev@ mailing list. Monthly Meetups are only in Sunnyvale and exclude 
most of the world. It is a critical part of the Apache Way to make sure 
discussion and decision making happens on the mailing list.

On last committers and PPMC members please provide the date of the elections.

Regards,
Dave

> On Jan 16, 2020, at 12:27 AM, Ning Wang  wrote:
> 
> Hi,
> 
> I prepared a draft of the Feb podling report:
> 
> https://docs.google.com/document/d/1_8ckuEuL7pdo-eOCHaXs32ymD2_qRKzEybL2Zyr6gXU/edit#
> 
> Please help to update if anything is missing. Thanks!
> 
> On Wed, Jan 15, 2020 at 4:26 PM  wrote:
> 
>> Dear podling,
>> 
>> This email was sent by an automated system on behalf of the Apache
>> Incubator PMC. It is an initial reminder to give you plenty of time to
>> prepare your quarterly board report.
>> 
>> The board meeting is scheduled for Wed, 19 February 2020, 10:30 am PDT.
>> The report for your podling will form a part of the Incubator PMC
>> report. The Incubator PMC requires your report to be submitted 2 weeks
>> before the board meeting, to allow sufficient time for review and
>> submission (Wed, February 05).
>> 
>> Please submit your report with sufficient time to allow the Incubator
>> PMC, and subsequently board members to review and digest. Again, the
>> very latest you should submit your report is 2 weeks prior to the board
>> meeting.
>> 
>> Candidate names should not be made public before people are actually
>> elected, so please do not include the names of potential committers or
>> PPMC members in your report.
>> 
>> Thanks,
>> 
>> The Apache Incubator PMC
>> 
>> Submitting your Report
>> 
>> --
>> 
>> Your report should contain the following:
>> 
>> *   Your project name
>> *   A brief description of your project, which assumes no knowledge of
>>the project or necessarily of its field
>> *   A list of the three most important issues to address in the move
>>towards graduation.
>> *   Any issues that the Incubator PMC or ASF Board might wish/need to be
>>aware of
>> *   How has the community developed since the last report
>> *   How has the project developed since the last report.
>> *   How does the podling rate their own maturity.
>> 
>> This should be appended to the Incubator Wiki page at:
>> 
>> https://cwiki.apache.org/confluence/display/INCUBATOR/February2020
>> 
>> Note: This is manually populated. You may need to wait a little before
>> this page is created from a template.
>> 
>> Note: The format of the report has changed to use markdown.
>> 
>> Mentors
>> ---
>> 
>> Mentors should review reports for their project(s) and sign them off on
>> the Incubator wiki page. Signing off reports shows that you are
>> following the project - projects that are not signed may raise alarms
>> for the Incubator PMC.
>> 
>> Incubator PMC
>> 

Re: [Discuss][Report] Heron November Podling Report

[Dave Fisher]

Exactly. The binary release is very large. A dependency licensing analysis is 
required.

Once this is done then the podling is nearing graduation.

Regards,
Dave

Sent from my iPhone

> On Nov 5, 2019, at 5:39 PM, Ning Wang  wrote:
> 
> There are a few things to worry about during a release, including the
> binary release, and license check, etc. With a few releases we hope to
> understand better about all the requirements I think.
> 
> Is there any suggestion about it?
> 
>> On Tue, Nov 5, 2019 at 5:23 PM Justin Mclean  wrote:
>> 
>> Hi,
>> 
>> Thanks for submitting your report. Could you clarify what you mean by
>> "Making several releases." as a barrier to graduation when you have already
>> made source releases. Are you referring to binary releases or something
>> else?
>> 
>> Thanks,
>> Justin
>> 
> 
> 
> -- 
> Regards,
> Ning Wang

Re: Heron Code Base Licenses

[Dave Fisher]

Hi Josh,

For this case:


Third party GNU General Public License, Version 3.0 licenses


The following components are provided under the GNU General Public License, 
Version 3.0.
See project link for details.

Sizzle(https://github.com/jquery/sizzle/blob/master/LICENSE.txt)
  -> heron/shell/assets/jquery.js

Take a look at the source file and choose the MIT and NOT GPL


/*!
 * jQuery JavaScript Library v1.7.2
 * http://jquery.com/
 *
 * Copyright 2011, John Resig
 * Dual licensed under the MIT or GPL Version 2 licenses.
 * http://jquery.org/license
 *
 * Includes Sizzle.js
 * http://sizzlejs.com/
 * Copyright 2011, The Dojo Foundation
 * Released under the MIT, BSD, and GPL Licenses.
 *
 * Date: Wed Mar 21 12:46:34 2012 -0700
 */

Have a look at the current version of Sizzle:

https://github.com/jquery/sizzle/blob/master/src/sizzle.js

It is no longer dual licensed and has an MIT license.

I think you know what to do.

Regards,
Dave


> On Oct 19, 2019, at 7:46 PM, Josh Fischer  wrote:
> 
> Hi All,
> 
> In the last release we made there were several comments about work that
> needed to be done to address issues with the licenses in the Heron code
> base.  I've opened a PR at this link ->
> https://github.com/apache/incubator-heron/pull/3368 to hopefully address
> most of the outstaning issues.  If anyone has any time to review it would
> be very helpful.  I **think** once the license issues are taken care of we
> can start releasing binaries for the project.
> 
> Thanks,
> 
> Josh

Re: [Mentors] Looking for feedback on NOTICE file

[Dave Fisher]

Hi -

The only change required is that the first line should be: “Apache Heron 
(Incubating)”

See http://www.apache.org/dev/licensing-howto.html#step-by-step

Please ask if you have further questions.

Regards,
Dave

> On Oct 21, 2019, at 12:26 PM, SiMing Weng  wrote:
> 
> Hi, guys:
> 
> I’m looking for some feedback on what work is required for the NOTICE 
>  file.
> 
> According to the Licensing How-To 
>  guide on ASF, at least we 
> knew:
> 
> The minimum to start with is the “boilerplate” content as given in the guide, 
> which we already have
> The NOTICE file is meant for legally required notifications which are not 
> satisfied by the LICENSE file or license copies under license folder
> If any bundled dependency has its NOTICE file, we need to examine whether it 
> has anything more than the ASF “boilerplate” content; if yes, we need to copy 
> that into our NOTICE file
> We only need to traverse BUNDLED dependencies
> We need to traverse dependencies recursively
> 
> Please feedback whether my understanding is correct or anything is missing.
> 
> Cheers,
> 
> SiMing

Re: [Discuss][Report] Heron November Podling Report

[Dave Fisher]

Hi -

The report should list the names and dates of all Committers added since the 
last report. If none then name and date of the last committer. The same should 
be reported regarding PPMC members.

There are still 10 days to go in the reporting period. ….

Regards,
Dave

> On Oct 21, 2019, at 9:05 AM, Ning Wang  wrote:
> 
> Hi,
> 
> Thanks for the input. Totally agree that the big rocks are blockers. Let's
> keep the status to be the same as last quarter then.
> 
> 
> On Mon, Oct 21, 2019 at 9:00 AM Josh Fischer  wrote:
> 
>> Thanks for sending out Ning.  I think you've covered most of the progress
>> that has been made since the last report.  I've added a few additions as
>> well.   I disagree on the podling's maturity to be marked as nearing
>> graduation.  Once we have  a few of the big rocks left to move (the new
>> static site up, licenses fixed, and we are releasing binaries) I feel we
>> can then say we are nearing graduation.  There is still a fair amount of
>> work to be done to get there.  Much less work than before, but still a ways
>> to go.
>> 
>> - Josh
>> 
>> On Sun, Oct 20, 2019 at 1:09 PM Ning Wang  wrote:
>> 
>>> Hey All,
>>> 
>>> It's time to create our podling report for November.  I've copied the
>>> document and added it to a google sheets document here:
>>> 
>>> 
>> https://docs.google.com/document/d/15ynoG-v75xyxcczZjZgHuEtM7lHF9WeDEz_suq90xNk/edit#
>>> 
>>> Please add any relevant information you can.  If you have any trouble
>>> accessing or editing the document let me know.
>>> 
>>> I feel we have made a lot of progress on the pre-graduation issues so I
>>> marked the "Nearing graduation" line. Please feel free to comment on it.
>>> 
>>> Thanks in advance.
>>> 
>>> Regards,
>>> --ning
>>> 
>> 

Re: 10/08/2019 Bi-Weekly OSS Heron Sync-up

[Dave Fisher]

Sorry I do see the start of a discussion thread ... I just don’t see any 
feedback on the list.

We can’t count on a google doc in a few years.

Sent from my iPhone

> On Oct 7, 2019, at 8:00 PM, Dave Fisher  wrote:
> 
> What’s this UI proposal? Is there a link?
> 
> Sent from my iPhone
> 
>> On Oct 7, 2019, at 7:51 PM, Josh Fischer  wrote:
>> 
>> Hi All,
>> 
>> Reviewed the UI Proposal..  Had further discussion about the new site.
>> 
>> - Josh
>> 
>>> On Mon, Oct 7, 2019 at 1:05 PM SiMing Weng  wrote:
>>> 
>>> Hi, all:
>>> 
>>> Review the UI proposal document
>>> Contribute a PR to generate maven artifacts of Kafka Bolt
>>> 
>>> Cheers,
>>> 
>>> Simon Weng
>>> 
>>>> On Oct 7, 2019, at 1:47 PM, Ning Wang  wrote:
>>>> 
>>>> Hi, all~,
>>>> 
>>>> It has been two weeks since our last sync! Let's share our works for the
>>>> last two weeks in this thread.
>>>> 
>>>> My updates:
>>>> 1. created and shared a proposal for Heron UI improvements
>>>> <
>>> https://docs.google.com/document/d/1CaDVVifcWqViRO96tmePK5dwhOhYnhwjhMRuImg-aOs/edit#heading=h.rfm62inflf7w
>>>> .
>>>> 
>>>> 2. a few code changes in UI(parallelism, tooltip), metrics sink
>>>> communicator, gc log files, and rate limiter example.
>>>> 3. discussion about load topology config from yaml file.
>>>> 
>>>> 
>>>> Regards,
>>>> --ning
>>> 
>>> 
> 

Re: 10/08/2019 Bi-Weekly OSS Heron Sync-up

[Dave Fisher]

What’s this UI proposal? Is there a link?

Sent from my iPhone

> On Oct 7, 2019, at 7:51 PM, Josh Fischer  wrote:
> 
> Hi All,
> 
> Reviewed the UI Proposal..  Had further discussion about the new site.
> 
> - Josh
> 
>> On Mon, Oct 7, 2019 at 1:05 PM SiMing Weng  wrote:
>> 
>> Hi, all:
>> 
>> Review the UI proposal document
>> Contribute a PR to generate maven artifacts of Kafka Bolt
>> 
>> Cheers,
>> 
>> Simon Weng
>> 
>>> On Oct 7, 2019, at 1:47 PM, Ning Wang  wrote:
>>> 
>>> Hi, all~,
>>> 
>>> It has been two weeks since our last sync! Let's share our works for the
>>> last two weeks in this thread.
>>> 
>>> My updates:
>>> 1. created and shared a proposal for Heron UI improvements
>>> <
>> https://docs.google.com/document/d/1CaDVVifcWqViRO96tmePK5dwhOhYnhwjhMRuImg-aOs/edit#heading=h.rfm62inflf7w
>>> .
>>> 
>>> 2. a few code changes in UI(parallelism, tooltip), metrics sink
>>> communicator, gc log files, and rate limiter example.
>>> 3. discussion about load topology config from yaml file.
>>> 
>>> 
>>> Regards,
>>> --ning
>> 
>> 

Re: October Monthly Meetup invitation (#15)

[Dave Fisher]

Hi Sree,

I plan on attending. There are a couple of points I’d like to make.

It would be very helpful to Heron community growth if the Meetup text had a 
pointer to the podling website and included the Disclaimer.

It would also be helpful for the PPMC to be involved in the selection of the 
speakers and topics. This is one of the ways that an Apache project should 
operate.

Follow ups directed to the mailing lists and GitHub issues.

Is there any video for any of the past talks? If so, then that ought to be 
shared from the Heron site.

Perhaps for the November Meetup a topic could be how to contribute to Apache 
Heron directly.

Best Regards,
Dave

> On Sep 22, 2019, at 10:47 PM, Sree Vaddi  
> wrote:
> 
> Hi Team,
> You are cordially invited to our monthly meetup in October.Please note the 
> day is 1st Monday instead 3rd Monday for a change.
> 
> https://www.meetup.com/Apache-Heron-Bay-Area/events/nglzdryznbcc/
> 
> 
> Thank you./Sree
> 

Re: 09/24/2019 Bi-Weekly OSS Heron Sync-up

[Dave Fisher]

Sent from my iPhone

> On Sep 23, 2019, at 10:05 AM, Ning Wang  wrote:
> 
> Hi, all~,
> 
> It has been two weeks since our last sync! Let's share our works for the
> last two weeks in this thread.
> 
> My updates:
> 1. upgrade zk to 3.4.14 per request from the-schott-hand:
> https://github.com/apache/incubator-heron/issues/3338
> 2. considering about/planning a few Heron UI improvements.

These would make a good thread here on dev@

Regards,
Dave

> 
> 
> Regards,
> --ning

Re: Monthly Meetup for September 2019.

[Dave Fisher]

Hi Sree,

How was the Meetup?

Regards,
Dave

> On Sep 15, 2019, at 2:08 PM, Sree Vaddi  
> wrote:
> 
> Hi Team,
> I cordially invite you to attend our 14th monthly meetup for the month of 
> september 2019.
> https://www.meetup.com/Apache-Heron-Bay-Area/events/nglzdryzmbvb/
> 
> 
> We are planning to record and publish all monthly meetups, very soon.Great 
> efforts underway, so presenters from remote locations will be able to speak 
> as well.
> 
> Please join #meetups slack channel, to keep up to date.
> 
> 
> 
> Thank you./Sree
> 

Fwd: [MENTORS] IPMC Policy change work in progress disclaimer

[Dave Fisher]

FYI

> Begin forwarded message:
> 
> From: Justin Mclean 
> Subject: [MENTORS] IPMC Policy change work in progress disclaimer
> Date: August 1, 2019 at 7:25:00 PM PDT
> To: gene...@incubator.apache.org
> Reply-To: gene...@incubator.apache.org
> 
> Hi,
> 
> The work in progress progress disclaimer (DISCLAIMER-WIP) has been added to 
> the IPMC policy page. [1]
> 
> Podlings can now select which disclaimer they want to use. If they want to 
> use the standard disclaimer then their releases must comply with all ASF 
> policy. If they want the incubator to be more lenient in voting on their 
> release or know that the release has some issues not in line with ASF policy 
> then the work in progress disclaimer can be used.
> 
> Unless the release is simple and straightforward, I would recommend that a 
> podling uses the work in progress disclaimer for the first couple of releases.
> 
> For what is allowed under the work in progress disclaimer please see this 
> legal JIRA [2]. This allows a lot more in a release but not everything. 
> Certain things are required like having a LICENSE, NOTICE and DISCLAIMER-WIP, 
> and while it would be OK to include compiled code you still need to comply 
> with any licensing for that code.
> 
> By the time a podling graduates it's expected that they are making releases 
> with the standard disclaimer.
> 
> Here is the text of the DISCLAIMER-WIP where the Incubator is the sponsor:
> 
> Apache  #Podling-Name# is an effort
> undergoing incubation at The Apache Software Foundation (ASF),
> sponsored by the Apache Incubator. Incubation is required of all
> newly accepted projects until a further review indicates that the
> infrastructure, communications, and decision making process have
> stabilized in a manner consistent with other successful ASF projects.
> While incubation status is not necessarily a reflection of the
> completeness or stability of the code, it does indicate that the
> project has yet to be fully endorsed by the ASF.
> 
> Some of the incubating project's releases may not be fully compliant
> with ASF policy. For example, releases may have incomplete or
> un-reviewed licensing conditions. What follows is a list of known
> issues the project is currently aware of (note that this list, by
> definition, is likely to be incomplete): 
> #List of known issues go here#
> 
> If you are planning to incorporate this work into your
> product/project, please be aware that you will need to conduct a
> thorough licensing review to determine the overall implications of
> including this work. For the current status of this project through the Apache
> Incubator visit: 
> http://incubator.apache.org/project/#Podling-Name#.html
> 
> 
> Just fill in #Podling-Name# with your podling name and list the known issues 
> in the correct place.
> 
> Thanks,
> Justin
> 
> 1. https://incubator.apache.org/policy/incubation.html#disclaimers
> 2. https://issues.apache.org/jira/projects/LEGAL/issues/LEGAL-469
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
> 

Re: [Mentors] [VOTE] Heron Release 0.20.1-incubating Release Candidate 3

[Dave Fisher]

Hi -

Reply to thread with [RESULT][VOTE] in the subject.

Make the release.

Make a list of known issues to be fixed in the next release. After we agree on 
the list then publish on the website. (This will keep in spirit with rules 
being discussed by the IPMC on general@incubator.)

Regards,
Dave

Sent from my iPhone

> On Jul 29, 2019, at 11:47 AM, Josh Fischer  wrote:
> 
> Hi All,
> 
> I sent out the rc-3 vote to general and we've received two more +1 binded
> votes on general@.   This leaves us with total of 3 binded votes for this
> release.  We have received some feedback on issues that will need to be
> addressed for the next release.
> 
> The start to the thread in general is here:
> https://mail-archives.apache.org/mod_mbox/incubator-general/201907.mbox/%3CCAFkuAo0h95piqvJ-B1JG6kU-%2B9rgR%3D5heec8Aosgt42n%3DBWTFA%40mail.gmail.com%3E
> 
> Dave,
> 
> Could you give any guidance on the next steps please?
> 
> - Josh
> 
>> On Sat, Jul 13, 2019 at 4:06 PM Josh Fischer  wrote:
>> 
>> Good Afternoon Heron Community,
>> 
>> The results of the Release Candidate 3 votes are below:
>> 
>> Binded Approvals: 1
>> 
>> Unbinded Approvals: 3
>> 
>> There were no disapproving votes or votes with no opinion.  I'll be moving
>> the vote to general@incubator.
>> 
>> - Josh
>> 
>>> On Thu, Jul 11, 2019 at 10:07 AM Josh Fischer  wrote:
>>> 
>>> Ping...
>>> 
>>>> On Wed, Jul 10, 2019 at 10:54 AM Ning Wang  wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> We need help for two more binding votes from mentors please.
>>>> 
>>>> Thanks in advance!
>>>> --ning
>>>> 
>>>>> 
>>>>>>> On Wed, Jul 3, 2019 at 9:36 AM Dave Fisher 
>>>>>> wrote:
>>>>>>> 
>>>>>>>> +1 (binding)
>>>>>>>> 
>>>>>>>> Signature and checksum is good.
>>>>>>>> DISCLAIMER present
>>>>>>>> LICENSE and NOTICE are there. The third party licenses could use a
>>>>>> little
>>>>>>>> work. The License text should be included in LICENSE or pulled
>>>> into the
>>>>>>>> licenses directory. Any required copyright attribution should be
>>>> in the
>>>>>>>> NOTICE.
>>>>>>>> —> Please create an issue to fix these for the next release.
>>>>>>>> RAT Check. The *.md files should have Apache License Headers and
>>>>>>>> CONTRIBUTING.md is link where the target is missing in the release.
>>>>>>>> —> Please create an issue to fix .rat-excludes and fix the md files
>>>>>> license
>>>>>>>> —> Please create an issue to annotate .rat-excludes to explain any
>>>> that
>>>>>>>> are excludes related to other licensing.
>>>>>>>> 
>>>>>>>> Thanks!
>>>>>>>> 
>>>>>>>> HTH,
>>>>>>>> Dave
>>>>>>>> 
>>>>>>>>> On Jul 2, 2019, at 2:35 PM, Ning Wang 
>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Thanks all for voting! So far we have got 3 +1s.
>>>>>>>>> 
>>>>>>>>> I am pinging mentors to give us some binding votes.
>>>>>>>>> 
>>>>>>>>> On Tue, Jul 2, 2019 at 1:24 PM Simon Weng 
>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> +1
>>>>>>>>>> 
>>>>>>>>>> Tested the locally built Heron Docker image.
>>>>>>>>>> 
>>>>>>>>>> On Sun, Jun 30, 2019 at 1:45 PM Josh Fischer <
>>>> j...@joshfischer.io>
>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> +1
>>>>>>>>>>> 
>>>>>>>>>>> Rat checks passed
>>>>>>>>>>> Tag built successfully
>>>>>>>>>>> Topology launched successfully
>>>>>>>>>>> Topology killed successfully
>>>>>>>>>>> 
>>>>>>>>>>> On Fri, Jun 28, 2019 at 5:32 PM thinker0 
>>>>>> wrote:
>>>>

Re: [VOTE] Heron Release 0.20.1-incubating Release Candidate 3

[Dave Fisher]

Hi -

> On Jul 3, 2019, at 10:09 AM, Ning Wang  wrote:
> 
> Thanks so much Dave!
> 
> Issues are created here:
> https://github.com/apache/incubator-heron/issues/3303
> https://github.com/apache/incubator-heron/issues/3304
> https://github.com/apache/incubator-heron/issues/3305
> 
> Quick question, is there an example of the rat-excludes annotation. I read
> Rat source code before and it seems like the excludes file doesn't support
> comments. :(

It does:

# Files start with .
\..*
.*\.out

# Data/doc files
.*\.conf
.*\.json
.*\.md
#.*\.txt
.*\.BUILD
.*\.patch

_helpers\.tpl
autogen\.sh
description
heron\.iml

See the # comment lines above.

Regards,
Dave

> 
> 
> On Wed, Jul 3, 2019 at 9:36 AM Dave Fisher  wrote:
> 
>> +1 (binding)
>> 
>> Signature and checksum is good.
>> DISCLAIMER present
>> LICENSE and NOTICE are there. The third party licenses could use a little
>> work. The License text should be included in LICENSE or pulled into the
>> licenses directory. Any required copyright attribution should be in the
>> NOTICE.
>> —> Please create an issue to fix these for the next release.
>> RAT Check. The *.md files should have Apache License Headers and
>> CONTRIBUTING.md is link where the target is missing in the release.
>> —> Please create an issue to fix .rat-excludes and fix the md files license
>> —> Please create an issue to annotate .rat-excludes to explain any that
>> are excludes related to other licensing.
>> 
>> Thanks!
>> 
>> HTH,
>> Dave
>> 
>>> On Jul 2, 2019, at 2:35 PM, Ning Wang  wrote:
>>> 
>>> Thanks all for voting! So far we have got 3 +1s.
>>> 
>>> I am pinging mentors to give us some binding votes.
>>> 
>>> On Tue, Jul 2, 2019 at 1:24 PM Simon Weng  wrote:
>>> 
>>>> +1
>>>> 
>>>> Tested the locally built Heron Docker image.
>>>> 
>>>> On Sun, Jun 30, 2019 at 1:45 PM Josh Fischer 
>> wrote:
>>>> 
>>>>> +1
>>>>> 
>>>>> Rat checks passed
>>>>> Tag built successfully
>>>>> Topology launched successfully
>>>>> Topology killed successfully
>>>>> 
>>>>> On Fri, Jun 28, 2019 at 5:32 PM thinker0  wrote:
>>>>> 
>>>>>> +1
>>>>>> 
>>>>>> 2019년 6월 29일 (토) 07:31, Ning Wang 님이 작성:
>>>>>> 
>>>>>>> *Hello, Heron Community,This is a call for a vote to the 3rd release
>>>>>>> candidate for Apache Heron, version v0.20.1-incubating. We kindly
>>>>> request
>>>>>>> project mentors (binded) as well as all contributors (unbinded) and
>>>>> users
>>>>>>> to review and vote on this incubator release.*
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> *The tag to be voted upon:0.20.1-incubating-rc3
>>>>>>> (40ccec635b45191593ff984a9ae81fd4e0a5dadd)The full list of changes
>>>> and
>>>>>>> release notes are available
>>>>>>> at:
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> https://github.com/apache/incubator-heron/releases/tag/0.20.1-incubating-rc3
>>>>>>> <
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> https://github.com/apache/incubator-heron/releases/tag/0.20.1-incubating-rc3
>>>>>>>> Source
>>>>>>> files can be found in dist.apache.org <http://dist.apache.org>
>>>>>>> site:
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.1-incubating-candidate-3/
>>>>>>> <
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.1-incubating-candidate-3/
>>>>>>>> Source
>>>>>>> SHA-512
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> checksums:68e2d4db598d73dc69e474dc82940402dc4689cc14dda14ad114cbaec79b1b3a8926625083e901f8234631f6345c6aa9e58fc0fec3176fce3e91e041e724d3b8
>>>>>>> incubator-heron-0.20.1-incubating-rc3.tar.gzPlease download the
>>>> source
>>>>>>> package, and follow the compiling
>>>>>>> guide(
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> https://apache.github.io/incubator-heron/docs/developers/compiling/compiling/
>>>>>>> <
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> https://apache.github.io/incubator-heron/docs/developers/compiling/compiling/
>>>>>>>> )to
>>>>>>> setup environment, build and run locally. Note that currently Bazel
>>>>>> 0.26.0
>>>>>>> is required to build this version.After Bazel is set up and works
>>>>>>> correctly, you can use these release check scripts
>>>>>>> (
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> https://github.com/apache/incubator-heron/blob/master/scripts/release_check/README.md#run-individual-release-checks
>>>>>>> <
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> https://github.com/apache/incubator-heron/blob/master/scripts/release_check/README.md#run-individual-release-checks
>>>>>>>> )
>>>>>>> to verify the build.The vote will be open for at least 72 hours or
>>>>> until
>>>>>>> the necessary number of votes are reached.Please vote accordingly:[ ]
>>>>> +1
>>>>>>> approve[ ] +0 no opinion[ ] -1 disapprove with the reasonThanks,The
>>>>>> Apache
>>>>>>> Heron (Incubating) Team*
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> 
>> 

Re: [VOTE] Heron Release 0.20.1-incubating Release Candidate 3

[Dave Fisher]

+1 (binding)

Signature and checksum is good.
DISCLAIMER present
LICENSE and NOTICE are there. The third party licenses could use a little work. 
The License text should be included in LICENSE or pulled into the licenses 
directory. Any required copyright attribution should be in the NOTICE.
—> Please create an issue to fix these for the next release.
RAT Check. The *.md files should have Apache License Headers and 
CONTRIBUTING.md is link where the target is missing in the release.
—> Please create an issue to fix .rat-excludes and fix the md files license
—> Please create an issue to annotate .rat-excludes to explain any that are 
excludes related to other licensing.

Thanks!

HTH,
Dave

> On Jul 2, 2019, at 2:35 PM, Ning Wang  wrote:
> 
> Thanks all for voting! So far we have got 3 +1s.
> 
> I am pinging mentors to give us some binding votes.
> 
> On Tue, Jul 2, 2019 at 1:24 PM Simon Weng  wrote:
> 
>> +1
>> 
>> Tested the locally built Heron Docker image.
>> 
>> On Sun, Jun 30, 2019 at 1:45 PM Josh Fischer  wrote:
>> 
>>> +1
>>> 
>>> Rat checks passed
>>> Tag built successfully
>>> Topology launched successfully
>>> Topology killed successfully
>>> 
>>> On Fri, Jun 28, 2019 at 5:32 PM thinker0  wrote:
>>> 
 +1
 
 2019년 6월 29일 (토) 07:31, Ning Wang 님이 작성:
 
> *Hello, Heron Community,This is a call for a vote to the 3rd release
> candidate for Apache Heron, version v0.20.1-incubating. We kindly
>>> request
> project mentors (binded) as well as all contributors (unbinded) and
>>> users
> to review and vote on this incubator release.*
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> *The tag to be voted upon:0.20.1-incubating-rc3
> (40ccec635b45191593ff984a9ae81fd4e0a5dadd)The full list of changes
>> and
> release notes are available
> at:
> 
 
>>> 
>> https://github.com/apache/incubator-heron/releases/tag/0.20.1-incubating-rc3
> <
> 
 
>>> 
>> https://github.com/apache/incubator-heron/releases/tag/0.20.1-incubating-rc3
>> Source
> files can be found in dist.apache.org 
> site:
> 
 
>>> 
>> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.1-incubating-candidate-3/
> <
> 
 
>>> 
>> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.1-incubating-candidate-3/
>> Source
> SHA-512
> 
> 
 
>>> 
>> checksums:68e2d4db598d73dc69e474dc82940402dc4689cc14dda14ad114cbaec79b1b3a8926625083e901f8234631f6345c6aa9e58fc0fec3176fce3e91e041e724d3b8
> incubator-heron-0.20.1-incubating-rc3.tar.gzPlease download the
>> source
> package, and follow the compiling
> guide(
> 
 
>>> 
>> https://apache.github.io/incubator-heron/docs/developers/compiling/compiling/
> <
> 
 
>>> 
>> https://apache.github.io/incubator-heron/docs/developers/compiling/compiling/
>> )to
> setup environment, build and run locally. Note that currently Bazel
 0.26.0
> is required to build this version.After Bazel is set up and works
> correctly, you can use these release check scripts
> (
> 
 
>>> 
>> https://github.com/apache/incubator-heron/blob/master/scripts/release_check/README.md#run-individual-release-checks
> <
> 
 
>>> 
>> https://github.com/apache/incubator-heron/blob/master/scripts/release_check/README.md#run-individual-release-checks
>> )
> to verify the build.The vote will be open for at least 72 hours or
>>> until
> the necessary number of votes are reached.Please vote accordingly:[ ]
>>> +1
> approve[ ] +0 no opinion[ ] -1 disapprove with the reasonThanks,The
 Apache
> Heron (Incubating) Team*
> 
 
>>> 
>> 

Re: ApacheCon

[Dave Fisher]

I have a talk too!

I’ll see you there!

Regards,
Dave

Sent from my iPhone

> On Jun 10, 2019, at 9:56 AM, Josh Fischer  wrote:
> 
> Hey All,
> 
> I had a speaking proposal accepted for ApacheCon in Las Vegas this year
> about Heron.  If anyone is in the area, it would be cool to meet up.
> 
> - Josh

Re: 6/4/2019 Bi-Weekly OSS Heron Sync-up

[Dave Fisher]

Hi -

> On Jun 3, 2019, at 11:56 AM, Dmitry Rusakov  
> wrote:
> 
> My updates:
> - Last week I released the new Heron 0.20.0.3 with the following release
> notes:

What do you mean by “release”?

Regards,
Dave

> 
>   1. UI improvements (for example, now MonViz button is available in heron
>   ui).
>   2. Memory leak in TMaster fixed.
>   3. KRYO serializer support added into Heron API.
> 
> 
> On Mon, Jun 3, 2019 at 11:20 AM Josh Fischer  wrote:
> 
>> Hi all,
>> 
>> My updates
>> - Submitted a new PR for the Apache website
>> 
>> On Mon, Jun 3, 2019 at 12:05 PM Ning Wang  wrote:
>> 
>>> Hi, all~,
>>> 
>>> It has been two weeks since our last sync and there has been quite a few
>>> works done! Let's share our works for the last two weeks in this thread.
>>> 
>>> My updates:
>>> - Continue working on Apache Beam support
>>> - Added release check scripts
>>> - Make RoundRobin packing algo more forgivable when container resource is
>>> not set.
>>> 
>>> 
>>> Regards,
>>> --ning
>>> 
>> --
>> Sent from A Mobile Device
>> 
> 
> 
> -- 
> Dmitry Rusakov,
> Senior Software Engineer,
> Twitter CDM team

Fwd: Incubator (git) website published

[Dave Fisher]

I just noticed these.Sent from my iPhoneBegin forwarded message:From: Apache Jenkins Server Date: May 8, 2019 at 2:56:18 PM PDTTo: nw...@twitter.com, c...@incubator.apache.orgSubject: Incubator (git) website publishedReply-To: gene...@incubator.apache.orgReply-To: gene...@incubator.apache.orgThe Apache Jenkins build system has built heron-asf-site (build #4)Status: Still FailingCheck console output at https://builds.apache.org/job/heron-asf-site/4/ to view the results.

build.log
Description: Binary data
-To unsubscribe, e-mail: cvs-unsubscr...@incubator.apache.orgFor additional commands, e-mail: cvs-h...@incubator.apache.org

Re: [Mentors] Fixing Heron Docs

[Dave Fisher]

Hi Josh,

I responded to your more recent email. In order to create and manage Jenkins 
jobs ids need to be added to the  hudson-jobadmin LDAP group by a PMC Chair.

Looking for that now.

Regards,
Dave

> On Apr 24, 2019, at 4:09 PM, Josh Fischer  wrote:
> 
> Sending this again.  I think the apache servers stripped out the image on
> the last email I sent.
> 
> Here is my original message:
> 
> I was looking around the site https://builds.apache.org/view/Heron%20Jobs/ .
> I've logged in, but I don't see a way to create a freestyle
> project/pipeline/anything in the UI.   Am I missing something obvious?
> 
> Please see the google drive link below for the image I was trying to share
> before.
> 
> https://drive.google.com/file/d/1BGjltTiRNZRfBaT5d2heTKEfKfUe0Irz/view?usp=sharing
> 
> On Tue, Apr 23, 2019 at 9:34 PM Josh Fischer  wrote:
> 
>> Hey All,
>> 
>> I was looking around the site https://builds.apache.org/view/Heron%20Jobs/ .
>> I've logged in, but I don't see a way to create a freestyle
>> project/pipeline/anything in the UI.. See attached image.   Am I missing
>> something?
>> 
>> 
>> 
>> On Mon, Apr 22, 2019 at 1:09 PM Ning Wang  wrote:
>> 
>>> Cool.
>>> 
>>> On Mon, Apr 22, 2019 at 10:31 AM Josh Fischer 
>>> wrote:
>>> 
>>>> I do too.  I’ll resend the email and try to move forward this week.
>>> Ning,
>>>> If I get stuck Ill Let you know.
>>>> 
>>>> On Mon, Apr 22, 2019 at 12:02 PM Ning Wang 
>>> wrote:
>>>> 
>>>>> Nice! I have access to Jenkins. Please let me know if you need helps
>>> from
>>>>> me.
>>>>> 
>>>>> On Mon, Apr 22, 2019 at 9:40 AM Josh Fischer 
>>>> wrote:
>>>>> 
>>>>>> Just an update on the heron doc website. The actual website build
>>> has
>>>>> been
>>>>>> fixed. I sent an email out to bui...@apache.org asking about how to
>>>>>> implement the process and describing what we would need in the
>>> server
>>>> to
>>>>>> build about a week ago.  I didn’t get a reply back, I will try again
>>>> this
>>>>>> week.
>>>>>> 
>>>>>> 
>>>>>> On Wed, Apr 3, 2019 at 9:08 PM Ning Wang 
>>> wrote:
>>>>>> 
>>>>>>> Great!
>>>>>>> 
>>>>>>> On Wed, Apr 3, 2019 at 4:25 PM Josh Fischer 
>>>>> wrote:
>>>>>>> 
>>>>>>>> Yes I think I am all set up now. Thank you for your help.
>>>>>>>> 
>>>>>>>> - Josh
>>>>>>>> 
>>>>>>>> On Wed, Apr 3, 2019 at 3:48 PM Dave Fisher <
>>> dave2w...@comcast.net>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hi Josh,
>>>>>>>>> 
>>>>>>>>> I see you in the group. Are you all set?
>>>>>>>>> 
>>>>>>>>> Regards,
>>>>>>>>> Dave
>>>>>>>>> 
>>>>>>>>>> On Apr 3, 2019, at 12:02 PM, Dave Fisher <
>>>> dave2w...@comcast.net>
>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Email stripped the image out. Please send it directly.
>>>>>>>>>> 
>>>>>>>>>>> On Apr 3, 2019, at 11:54 AM, Josh Fischer <
>>>> j...@joshfischer.io>
>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Ok went to that link and nothing and authed with apache,
>>> GH,
>>>> and
>>>>>> saw
>>>>>>>>> confirmation of repos.  Still no access via UI or CLI.
>>>>>>>>>>> Image attached.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Apr 3, 2019 at 1:46 PM Dave Fisher <
>>>>> dave2w...@comcast.net
>>>>>>>>> <mailto:dave2w...@comcast.net>> wrote:
>>>>>>>>>>> Infra has a question:
>>>>>>>>>>> 
>>>>>>>>>>> Could you please have him submit a screenshot of what
>>>>>>>>> https://git

Re: Podling Report Reminder - May 2019

[Dave Fisher]

Hi -

The MOIN wiki is being retired and podling reports will now be in Confluence.

The link provided is off: 
https://cwiki.apache.org/confluence/display/INCUBATOR/May2019

Regards,
Dave

> On Apr 24, 2019, at 7:12 PM, Josh Fischer  wrote:
> 
> Hey All,
> 
> The link to incubator in this email is broken (seems to point to
> confluence?) and the link in the previous email sent the reports have not
> been populated.  I'm not sure what to do here.  I can guess we can just
> wait until the templates are created.
> 
> Any other suggestions?
> 
> On Wed, Apr 24, 2019 at 8:32 PM  wrote:
> 
>> Dear podling,
>> 
>> This email was sent by an automated system on behalf of the Apache
>> Incubator PMC. It is an initial reminder to give you plenty of time to
>> prepare your quarterly board report.
>> 
>> The board meeting is scheduled for Wed, 15 May 2019, 10:30 am PDT.
>> The report for your podling will form a part of the Incubator PMC
>> report. The Incubator PMC requires your report to be submitted 2 weeks
>> before the board meeting, to allow sufficient time for review and
>> submission (Wed, May 01).
>> 
>> Please submit your report with sufficient time to allow the Incubator
>> PMC, and subsequently board members to review and digest. Again, the
>> very latest you should submit your report is 2 weeks prior to the board
>> meeting.
>> 
>> Candidate names should not be made public before people are actually
>> elected, so please do not include the names of potential committers or
>> PPMC members in your report.
>> 
>> Thanks,
>> 
>> The Apache Incubator PMC
>> 
>> Submitting your Report
>> 
>> --
>> 
>> Your report should contain the following:
>> 
>> *   Your project name
>> *   A brief description of your project, which assumes no knowledge of
>>the project or necessarily of its field
>> *   A list of the three most important issues to address in the move
>>towards graduation.
>> *   Any issues that the Incubator PMC or ASF Board might wish/need to be
>>aware of
>> *   How has the community developed since the last report
>> *   How has the project developed since the last report.
>> *   How does the podling rate their own maturity.
>> 
>> This should be appended to the Incubator Wiki page at:
>> 
>> https://cwiki.apache.org/confluence/INCUBATOR/May2019
>> 
>> Note: This is manually populated. You may need to wait a little before
>> this page is created from a template.
>> 
>> Mentors
>> ---
>> 
>> Mentors should review reports for their project(s) and sign them off on
>> the Incubator wiki page. Signing off reports shows that you are
>> following the project - projects that are not signed may raise alarms
>> for the Incubator PMC.
>> 
>> Incubator PMC
>> 

Re: [Mentors][VOTE] Heron Release 0.20.1-incubating Release Candidate 2 (Second Pass)

[Dave Fisher]

See https://docs.bazel.build/versions/master/install-os-x.html

> On Apr 24, 2019, at 3:15 PM, Julien Le Dem  
> wrote:
> 
> how does one install bazel 0.14?
> 
> On Wed, Apr 24, 2019 at 2:59 PM Ning Wang  wrote:
> 
>> Thanks Julien!
>> 
>> Yeah. This RC was still with Bazel 0.14.1 since it was created in February.
>> Latest master works with Bazel 0.23.1.
>> 
>> On Wed, Apr 24, 2019 at 2:47 PM Julien Le Dem
>>  wrote:
>> 
>>> I was able to build from master but not from the release tarbal.
>>> Is Bazel 0.14.1 actually required? Current release is 0.24.1.
>>> 
>>> On Wed, Apr 24, 2019 at 2:25 PM Julien Le Dem 
>>> wrote:
>>> 
 I'm looking into it.
 I recently got a new laptop so doing the setup from scratch.
 I had to follow:
 
>>> 
>> https://stackoverflow.com/questions/45276830/xcode-version-must-be-specified-to-use-an-apple-crosstool
 I hit that error at step 6 here:
 
>> https://apache.github.io/incubator-heron/docs/developers/compiling/mac/
 
 On Sun, Apr 21, 2019 at 11:53 PM Ning Wang 
>> wrote:
 
> On Fri, Apr 19, 2019 at 11:52 AM Ning Wang 
>>> wrote:
> 
>> Hi, dear Mentors,
>> 
>> Please help if you have time. :)  Thanks in advance.
>> 
>> --ning
>> 
>> 
>> On Fri, Apr 19, 2019 at 11:50 AM Ning Wang 
> wrote:
>> 
>>> *Hello, Heron Community,*
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> *This is a call for a vote to the 2nd release candidate for Apache
> Heron,
>>> version v0.20.1-incubating. This is a second pass since Dave
>> pointed
> out a
>>> few things that we weren’t clear before. The git tag is the same as
>>> the
>>> last time and the previous votes (four +1s from thinker0, Karthik
> Ramasamy,
>>> Josh Fischer, Simong Weng) will be carried over.The release will be
> source
>>> only and I am going to remove the binary packages. We need to
>> reduce
> the
>>> size and another review before including them in the release.The
>> tag
> to be
>>> voted upon:0.20.1-incubating-rc2
>>> (e6134da336fa290fa1b40972bc747a7507948d8a)The full list of changes
>>> and
>>> release notes are available
>>> at:
> 
>>> 
>> https://github.com/apache/incubator-heron/releases/tag/0.20.1-incubating-rc2
>>> <
> 
>>> 
>> https://github.com/apache/incubator-heron/releases/tag/0.20.1-incubating-rc2
>> Release
>>> files can be found in dist.apache.org 
>>> site:
> 
>>> 
>> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.1-incubating-candidate-2/
>>> <
> 
>>> 
>> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.1-incubating-candidate-2/
>> Source:
>>> 
> 
>>> 
>> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.1-incubating-candidate-2/incubator-heron-0.20.1-incubating-rc2.tar.gz
>>> <
> 
>>> 
>> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.1-incubating-candidate-2/incubator-heron-0.20.1-incubating-rc2.tar.gz
>> ASC
>>> signature:
>>> 
> 
>>> 
>> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.1-incubating-candidate-2/incubator-heron-0.20.1-incubating-rc2.tar.gz.asc
>>> <
> 
>>> 
>> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.1-incubating-candidate-2/incubator-heron-0.20.1-incubating-rc2.tar.gz.asc
>> SHA512:
>>> 
> 
>>> 
>> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.1-incubating-candidate-2/incubator-heron-0.20.1-incubating-rc2.tar.gz.sha512
>>> <
> 
>>> 
>> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.1-incubating-candidate-2/incubator-heron-0.20.1-incubating-rc2.tar.gz.sha512
> 
>>> 
>>> c47fc8c228b5543f94dcf8fb5eb0f8083e84602be4f3b5ca52402b6e3e0f893434f971c317f44c3a69e78e597b96642fd69b5bec63e9a8eb7456c816f8e118f3
>>> incubator-heron-0.20.1-incubating-rc2.tar.gzThe artifacts are
>> signed
> with
>>> PGP key 293DB72F865688D1, corresponding to nw...@apache.org
>>> , that can be found in keys
>>> file:
>> https://dist.apache.org/repos/dist/release/incubator/heron/KEYS
>>> > Please
>>> download the source package, and follow the compiling
>>> guide(
> 
>>> 
>> https://apache.github.io/incubator-heron/docs/developers/compiling/compiling/
>>> <
> 
>>> 
>> https://apache.github.io/incubator-heron/docs/developers/compiling/compiling/
>> )to
>>> build and run the Heron locally. Note that currently Bazel 0.14.1
>> is
>>> required to build this version.For your convenience, here are a few
>>> commands I used:License check:$ java -jar
> ~/Downloads/apache-rat-0.12.jar .
>>> -E .rat-excludesBuild the binary package for Mac OS:$ sh
>>> 

Re: Jenkins & Binaries - Re: [DISCUSS][VOTE] Heron Release 0.20.1-incubating (Release Candidate 2)

[Dave Fisher]

Hi -

> On Apr 3, 2019, at 3:20 PM, Ning Wang  wrote:
> 
> Got it. Thanks.
> 
> My bad. I meant for this version (before the review).

Look into the License of all the dependent code and packages included in the 
built binary.

There will then need to be procedures to understand when new dependencies are 
added so that those can be tested.

Also, for the source itself, I took a look at the rat-excludes and they seem to 
be generous. Let’s review these too.

> 
> One more question, what exactly we need to do to review license?

Inspect every dependency ….

> 
> On Wed, Apr 3, 2019 at 12:13 PM Dave Fisher  wrote:
> 
>> Hi -
>> 
>>> On Apr 3, 2019, at 11:50 AM, Ning Wang  wrote:
>>> 
>>> *Inline.
>>> 
>>> On Wed, Apr 3, 2019 at 11:36 AM Dave Fisher 
>> wrote:
>>> 
>>>> Hi -
>>>> 
>>>> See inline:
>>>> 
>>>>>>> 
>>>>>>> For docker hub image, I published it to
>>>>>>> *https://hub.docker.com/r/apacheheron/heron/tags
>>>>>>> <https://hub.docker.com/r/apacheheron/heron/tags>. It is not the
>> right
>>>>>> one?
>>>>>>> I got an invite after becoming a committer then I thought this is the
>>>>>>> official one. But maybe it isn't?*
>>>>>> 
>>>>>> That one is created and managed by this project, and as such it
>> remains
>>>> an
>>>>>> unofficial location - we’ll need to be careful how it is advertised.
>>>>>> 
>>>>>> The official Apache Docker Hub is https://hub.docker.com/u/apache <
>>>>>> https://hub.docker.com/u/apache>
>>>>>> 
>>>>>> https://reference.apache.org/pmc/docker <
>>>>>> https://reference.apache.org/pmc/docker> for an example.
>>>>>> 
>>>>>> https://issues.apache.org/jira/browse/LEGAL-270 <
>>>>>> https://issues.apache.org/jira/browse/LEGAL-270> for a discussion.
>>>>>> 
>>>>>> Let’s discuss via how the apacheheron docker file is produced.
>>>>>> 
>>>>>> 
>>>>> Ok. Another permission to request.
>>>>> 
>>>>> The docker image is built with this jenkins job:
>>>>> https://builds.apache.org/job/apache-heron-github-docker-image-debian/
>>>>> 
>>>>> I then downloaded it and load/publish.
>>>> 
>>>> OK, I see this and I see several build jobs.
>>>> 
>>>> And wow - that’s a 1GB archive!
>>>> 
>>>> Yeah. It is 1G~ With --squash flag it is smaller (500m to 600m), but the
>>> flag is not available in Apache Jenkins machines.
>>> 
>>> 
>>> 
>>>> I wonder how much of these Jenkins scripts should be in Jenkins as
>> opposed
>>>> to all in the Git repository and then invoked as 1-3 scripts from
>> Jenkins
>>>> w/ environment variable pick up. This would ultimately benefit those who
>>>> are trying to understand how to build Heron and what artifacts are
>> brought
>>>> into the binary.
>>>> 
>>>> 
>>> Agreed. The scripts could be refactored/simplifed further.
>>> 
>>> 
>>> 
>>>> We must do a careful license review of everything included in a Binary.
>>>> If some of the binary artifacts going to the maven repository are much
>>>> smaller then we should discuss these separately.
>>>> 
>>>> Until then just make Source releases.
>>>> 
>>> 
>>> So it means docker images and those convenience binary packages are not
>>> allowed on github and dockerhub?
>> 
>> NO! You are missing the point. I have not been clear. We have to know what
>> is inside of these packages before they are allowed! We have to go through
>> a process to confirm that there are no disallowed licenses and that proper
>> notice for certain licenses are followed. It is tedious, but it is required.
>> 
>> http://www.apache.org/legal/release-policy.html
>> 
>> Here is the guide to understand if a license is allowed.
>> 
>> http://www.apache.org/legal/resolved.html
>> 
>> For example, Apache Releases must not include GPL!
>> 
>>> 
>>> So the action items for the binary packages are:
>>> 1. remove them from github, resume the 0.20.1 rc2 vote with updated info.
>>> 2. optimize the bin

Re: [Mentors] Fixing Heron Docs

[Dave Fisher]

Hi Josh,

I see you in the group. Are you all set?

Regards,
Dave

> On Apr 3, 2019, at 12:02 PM, Dave Fisher  wrote:
> 
> Email stripped the image out. Please send it directly.
> 
>> On Apr 3, 2019, at 11:54 AM, Josh Fischer  wrote:
>> 
>> Ok went to that link and nothing and authed with apache, GH, and saw 
>> confirmation of repos.  Still no access via UI or CLI.
>> Image attached.
>> 
>> 
>> On Wed, Apr 3, 2019 at 1:46 PM Dave Fisher > <mailto:dave2w...@comcast.net>> wrote:
>> Infra has a question:
>> 
>> Could you please have him submit a screenshot of what 
>> https://gitbox.apache.org/setup/ <https://gitbox.apache.org/setup/> says 
>> after he has authed? He may need to reset his github linking (reset link in 
>> the middle box of the setup page.) 
>> 
>> Regards,
>> Dave
>> 
>>> On Apr 3, 2019, at 11:19 AM, Dave Fisher >> <mailto:dave2w...@comcast.net>> wrote:
>>> 
>>> It looks like you are missing from the heron committers group on Git Hub. 
>>> I’m asking Infra why on Slack. If no answer soon then I’ll file a JIRA.
>>> 
>>> Anyone else having issues?
>>> 
>>>> On Apr 3, 2019, at 11:03 AM, Josh Fischer >>> <mailto:j...@joshfischer.io>> wrote:
>>>> 
>>>> I do have 2FA enabled.  I've also tried to log in/ log out.  No success.
>>>> Any suggestions on who I should contact to help diagnose and fix?
>>>> 
>>>> On Wed, Apr 3, 2019 at 12:58 PM Dave Fisher >>> <mailto:dave2w...@comcast.net>> wrote:
>>>> 
>>>>> 
>>>>> 
>>>>>> On Apr 3, 2019, at 10:49 AM, Josh Fischer >>>>> <mailto:j...@joshfischer.io>> wrote:
>>>>>> 
>>>>>> Thanks for the info Dave.  I have a couple of responses to your
>>>>> questions.
>>>>>> 
>>>>>> (1) Does the project wish to create documentation in a new repository -
>>>>> say
>>>>>> apache/incubator-heron-site
>>>>>> Or should the build use an asf-site branch in
>>>>>> apache/incubator-heron which just keeps the deployed content.
>>>>>> 
>>>>>> ** The site build is currently dependent on Bazel to generate Javadocs.
>>>>> I
>>>>>> think we should just have a "asf-site" branch that is used within the
>>>>>> current repo to serve static files from as it would require less moving
>>>>>> parts.  Not sure if that would cause any issues with the current setup
>>>>>> though.  I was asking Ning a similar question on how to set this up
>>>>> since I
>>>>>> don't have write access to the repo yet.Hi -
>>>>> 
>>>>> Really? - Have you enabled 2FA on GitHub for your joshfischer1108 
>>>>> username?
>>>>> 
>>>>> I recommend that you get access fixed and then work on doing the build
>>>>> locally. Once you have that we can work on the Jenkins build with 
>>>>> triggers.
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> (2) Does the website Jenkins box already support the build requirements.
>>>>>> 
>>>>>> ** I will look into this.
>>>>> 
>>>>> Ask on the-asf.slack.com <http://the-asf.slack.com/> #asfinfra - tell 
>>>>> them it is for a gitpubsub
>>>>> website build.
>>>>> 
>>>>> Regards,
>>>>> Dave
>>>>> 
>>>>> 
>>>>>> 
>>>>>> On Wed, Apr 3, 2019 at 10:43 AM Dave Fisher >>>>> <mailto:dave2w...@comcast.net>>
>>>>> wrote:
>>>>>> 
>>>>>>> Hi -
>>>>>>> 
>>>>>>> TL:DR - The project should definitely use gitpubsub approach.
>>>>>>> 
>>>>>>>> On Mar 28, 2019, at 7:08 PM, Josh Fischer >>>>>>> <mailto:j...@joshfischer.io>> wrote:
>>>>>>>> 
>>>>>>>> How does everyone want to manage publishing the site? After looking at
>>>>>>> the
>>>>>>>> docs it seems there are 3 options.  Being that we are using git, I
>>>>> would
>>>>>>>> thing that gitsubpub would be the path of least resistance?...  But I
>>>

Re: Jenkins & Binaries - Re: [DISCUSS][VOTE] Heron Release 0.20.1-incubating (Release Candidate 2)

[Dave Fisher]

Hi -

> On Apr 3, 2019, at 11:50 AM, Ning Wang  wrote:
> 
> *Inline.
> 
> On Wed, Apr 3, 2019 at 11:36 AM Dave Fisher  wrote:
> 
>> Hi -
>> 
>> See inline:
>> 
>>>>> 
>>>>> For docker hub image, I published it to
>>>>> *https://hub.docker.com/r/apacheheron/heron/tags
>>>>> <https://hub.docker.com/r/apacheheron/heron/tags>. It is not the right
>>>> one?
>>>>> I got an invite after becoming a committer then I thought this is the
>>>>> official one. But maybe it isn't?*
>>>> 
>>>> That one is created and managed by this project, and as such it remains
>> an
>>>> unofficial location - we’ll need to be careful how it is advertised.
>>>> 
>>>> The official Apache Docker Hub is https://hub.docker.com/u/apache <
>>>> https://hub.docker.com/u/apache>
>>>> 
>>>> https://reference.apache.org/pmc/docker <
>>>> https://reference.apache.org/pmc/docker> for an example.
>>>> 
>>>> https://issues.apache.org/jira/browse/LEGAL-270 <
>>>> https://issues.apache.org/jira/browse/LEGAL-270> for a discussion.
>>>> 
>>>> Let’s discuss via how the apacheheron docker file is produced.
>>>> 
>>>> 
>>> Ok. Another permission to request.
>>> 
>>> The docker image is built with this jenkins job:
>>> https://builds.apache.org/job/apache-heron-github-docker-image-debian/
>>> 
>>> I then downloaded it and load/publish.
>> 
>> OK, I see this and I see several build jobs.
>> 
>> And wow - that’s a 1GB archive!
>> 
>> Yeah. It is 1G~ With --squash flag it is smaller (500m to 600m), but the
> flag is not available in Apache Jenkins machines.
> 
> 
> 
>> I wonder how much of these Jenkins scripts should be in Jenkins as opposed
>> to all in the Git repository and then invoked as 1-3 scripts from Jenkins
>> w/ environment variable pick up. This would ultimately benefit those who
>> are trying to understand how to build Heron and what artifacts are brought
>> into the binary.
>> 
>> 
> Agreed. The scripts could be refactored/simplifed further.
> 
> 
> 
>> We must do a careful license review of everything included in a Binary.
>> If some of the binary artifacts going to the maven repository are much
>> smaller then we should discuss these separately.
>> 
>> Until then just make Source releases.
>> 
> 
> So it means docker images and those convenience binary packages are not
> allowed on github and dockerhub?

NO! You are missing the point. I have not been clear. We have to know what is 
inside of these packages before they are allowed! We have to go through a 
process to confirm that there are no disallowed licenses and that proper notice 
for certain licenses are followed. It is tedious, but it is required.

http://www.apache.org/legal/release-policy.html

Here is the guide to understand if a license is allowed.

http://www.apache.org/legal/resolved.html

For example, Apache Releases must not include GPL!

> 
> So the action items for the binary packages are:
> 1. remove them from github, resume the 0.20.1 rc2 vote with updated info.
> 2. optimize the binary package sizes.
> 3. request permission to uploaded them to Apache dist repo.
> 4. license review

4. is 1.5

> 
> 
> 
>> 
>> BTW - The apacheheron Docker Hub still appears to be from the project and
>> Apache and that means it is not allowed unless it can be VOTED on.
>> 
> 
> My understanding is:
> - remove the docker image from apacheheron
> - wait for the works on the binary packages are done.
> - build and publish to apache docker hub.

That would be preferred.

Regards,
Dave

> 
> 
> 
>> 
>> Please see the VP, Legal and VP, Brand comments on
>> https://issues.apache.org/jira/browse/LEGAL-427
>> 
>> (This is a better and more direct answer than on
>> https://issues.apache.org/jira/browse/LEGAL-270 )
>> 
>> 
> 
> 
>> Does this make sense?
>> 
> 
> Yeah. Thanks.
> 
> 
>> Regards,
>> Dave
>>> 
>>> 
>>>> 
>>>>> 
>>>>> I thought maven artifacts are on repository.apache.org and source
>>>> release
>>>>> (may include binary release as well in future) should be in
>>>> dist.apache.org.
>>>>> Seems I am wrong. I can add the artifacts to dist.apache.org.
>>>> 
>>>> All released artifacts should be on dist.apache.org <
>>>> h

Re: [Mentors] Fixing Heron Docs

[Dave Fisher]

Email stripped the image out. Please send it directly.

> On Apr 3, 2019, at 11:54 AM, Josh Fischer  wrote:
> 
> Ok went to that link and nothing and authed with apache, GH, and saw 
> confirmation of repos.  Still no access via UI or CLI.
> Image attached.
> 
> 
> On Wed, Apr 3, 2019 at 1:46 PM Dave Fisher  <mailto:dave2w...@comcast.net>> wrote:
> Infra has a question:
> 
> Could you please have him submit a screenshot of what 
> https://gitbox.apache.org/setup/ <https://gitbox.apache.org/setup/> says 
> after he has authed? He may need to reset his github linking (reset link in 
> the middle box of the setup page.) 
> 
> Regards,
> Dave
> 
> > On Apr 3, 2019, at 11:19 AM, Dave Fisher  > <mailto:dave2w...@comcast.net>> wrote:
> > 
> > It looks like you are missing from the heron committers group on Git Hub. 
> > I’m asking Infra why on Slack. If no answer soon then I’ll file a JIRA.
> > 
> > Anyone else having issues?
> > 
> >> On Apr 3, 2019, at 11:03 AM, Josh Fischer  >> <mailto:j...@joshfischer.io>> wrote:
> >> 
> >> I do have 2FA enabled.  I've also tried to log in/ log out.  No success.
> >> Any suggestions on who I should contact to help diagnose and fix?
> >> 
> >> On Wed, Apr 3, 2019 at 12:58 PM Dave Fisher  >> <mailto:dave2w...@comcast.net>> wrote:
> >> 
> >>> 
> >>> 
> >>>> On Apr 3, 2019, at 10:49 AM, Josh Fischer  >>>> <mailto:j...@joshfischer.io>> wrote:
> >>>> 
> >>>> Thanks for the info Dave.  I have a couple of responses to your
> >>> questions.
> >>>> 
> >>>> (1) Does the project wish to create documentation in a new repository -
> >>> say
> >>>> apache/incubator-heron-site
> >>>>  Or should the build use an asf-site branch in
> >>>> apache/incubator-heron which just keeps the deployed content.
> >>>> 
> >>>> ** The site build is currently dependent on Bazel to generate Javadocs.
> >>> I
> >>>> think we should just have a "asf-site" branch that is used within the
> >>>> current repo to serve static files from as it would require less moving
> >>>> parts.  Not sure if that would cause any issues with the current setup
> >>>> though.  I was asking Ning a similar question on how to set this up
> >>> since I
> >>>> don't have write access to the repo yet.Hi -
> >>> 
> >>> Really? - Have you enabled 2FA on GitHub for your joshfischer1108 
> >>> username?
> >>> 
> >>> I recommend that you get access fixed and then work on doing the build
> >>> locally. Once you have that we can work on the Jenkins build with 
> >>> triggers.
> >>> 
> >>>> 
> >>>> 
> >>>> (2) Does the website Jenkins box already support the build requirements.
> >>>> 
> >>>> ** I will look into this.
> >>> 
> >>> Ask on the-asf.slack.com <http://the-asf.slack.com/> #asfinfra - tell 
> >>> them it is for a gitpubsub
> >>> website build.
> >>> 
> >>> Regards,
> >>> Dave
> >>> 
> >>> 
> >>>> 
> >>>> On Wed, Apr 3, 2019 at 10:43 AM Dave Fisher  >>>> <mailto:dave2w...@comcast.net>>
> >>> wrote:
> >>>> 
> >>>>> Hi -
> >>>>> 
> >>>>> TL:DR - The project should definitely use gitpubsub approach.
> >>>>> 
> >>>>>> On Mar 28, 2019, at 7:08 PM, Josh Fischer  >>>>>> <mailto:j...@joshfischer.io>> wrote:
> >>>>>> 
> >>>>>> How does everyone want to manage publishing the site? After looking at
> >>>>> the
> >>>>>> docs it seems there are 3 options.  Being that we are using git, I
> >>> would
> >>>>>> thing that gitsubpub would be the path of least resistance?...  But I
> >>>>> can't
> >>>>>> find much on how to create the jenkins job.  Maybe this is listed
> >>>>> somewhere
> >>>>>> else in the documentation?  Does anyone have other suggestions or
> >>> ideas?
> >>>>>> 
> >>>>>> Here is link to the guidelines ->
> >>>>>> http://www.apache.org

Re: [Mentors] Fixing Heron Docs

[Dave Fisher]

Infra has a question:

Could you please have him submit a screenshot of what 
https://gitbox.apache.org/setup/ says after he has authed? He may need to reset 
his github linking (reset link in the middle box of the setup page.) 

Regards,
Dave

> On Apr 3, 2019, at 11:19 AM, Dave Fisher  wrote:
> 
> It looks like you are missing from the heron committers group on Git Hub. I’m 
> asking Infra why on Slack. If no answer soon then I’ll file a JIRA.
> 
> Anyone else having issues?
> 
>> On Apr 3, 2019, at 11:03 AM, Josh Fischer  wrote:
>> 
>> I do have 2FA enabled.  I've also tried to log in/ log out.  No success.
>> Any suggestions on who I should contact to help diagnose and fix?
>> 
>> On Wed, Apr 3, 2019 at 12:58 PM Dave Fisher  wrote:
>> 
>>> 
>>> 
>>>> On Apr 3, 2019, at 10:49 AM, Josh Fischer  wrote:
>>>> 
>>>> Thanks for the info Dave.  I have a couple of responses to your
>>> questions.
>>>> 
>>>> (1) Does the project wish to create documentation in a new repository -
>>> say
>>>> apache/incubator-heron-site
>>>>  Or should the build use an asf-site branch in
>>>> apache/incubator-heron which just keeps the deployed content.
>>>> 
>>>> ** The site build is currently dependent on Bazel to generate Javadocs.
>>> I
>>>> think we should just have a "asf-site" branch that is used within the
>>>> current repo to serve static files from as it would require less moving
>>>> parts.  Not sure if that would cause any issues with the current setup
>>>> though.  I was asking Ning a similar question on how to set this up
>>> since I
>>>> don't have write access to the repo yet.Hi -
>>> 
>>> Really? - Have you enabled 2FA on GitHub for your joshfischer1108 username?
>>> 
>>> I recommend that you get access fixed and then work on doing the build
>>> locally. Once you have that we can work on the Jenkins build with triggers.
>>> 
>>>> 
>>>> 
>>>> (2) Does the website Jenkins box already support the build requirements.
>>>> 
>>>> ** I will look into this.
>>> 
>>> Ask on the-asf.slack.com #asfinfra - tell them it is for a gitpubsub
>>> website build.
>>> 
>>> Regards,
>>> Dave
>>> 
>>> 
>>>> 
>>>> On Wed, Apr 3, 2019 at 10:43 AM Dave Fisher 
>>> wrote:
>>>> 
>>>>> Hi -
>>>>> 
>>>>> TL:DR - The project should definitely use gitpubsub approach.
>>>>> 
>>>>>> On Mar 28, 2019, at 7:08 PM, Josh Fischer  wrote:
>>>>>> 
>>>>>> How does everyone want to manage publishing the site? After looking at
>>>>> the
>>>>>> docs it seems there are 3 options.  Being that we are using git, I
>>> would
>>>>>> thing that gitsubpub would be the path of least resistance?...  But I
>>>>> can't
>>>>>> find much on how to create the jenkins job.  Maybe this is listed
>>>>> somewhere
>>>>>> else in the documentation?  Does anyone have other suggestions or
>>> ideas?
>>>>>> 
>>>>>> Here is link to the guidelines ->
>>>>>> http://www.apache.org/dev/project-site.html. I copy and pasted some
>>> info
>>>>>> below on the different methods of website publishing for quick
>>> reference.
>>>>>> 
>>>>>> 
>>>>>> - Apache CMS, which provides a simple browser-based user interface for
>>>>>> editing, staging and publishing site content in Markdown, HTML or any
>>>>> other
>>>>>> source format for which support has been added. See the CMS reference
>>>>>> <https://www.apache.org/dev/cmsref> and adoption
>>>>>> <https://www.apache.org/dev/cmsadoption> for more details. The Apache
>>>>>> CMS uses svnpubsub as the underlying mechanism for publishing a site.
>>>>> 
>>>>> While this was home grown in Apache and all the rage 7-8 years ago when
>>> I
>>>>> used this to port over OpenOffice.org with a reskin. This should not be
>>>>> used. Infra wants people to stop and there are occasional publishing
>>> issues.
>>>>> 
>>>>>> - svnpubsub, which allows the static contents of a designated svn
>>>>> fo

Jenkins & Binaries - Re: [DISCUSS][VOTE] Heron Release 0.20.1-incubating (Release Candidate 2)

[Dave Fisher]

Hi -

See inline:

>>> 
>>> For docker hub image, I published it to
>>> *https://hub.docker.com/r/apacheheron/heron/tags
>>> <https://hub.docker.com/r/apacheheron/heron/tags>. It is not the right
>> one?
>>> I got an invite after becoming a committer then I thought this is the
>>> official one. But maybe it isn't?*
>> 
>> That one is created and managed by this project, and as such it remains an
>> unofficial location - we’ll need to be careful how it is advertised.
>> 
>> The official Apache Docker Hub is https://hub.docker.com/u/apache <
>> https://hub.docker.com/u/apache>
>> 
>> https://reference.apache.org/pmc/docker <
>> https://reference.apache.org/pmc/docker> for an example.
>> 
>> https://issues.apache.org/jira/browse/LEGAL-270 <
>> https://issues.apache.org/jira/browse/LEGAL-270> for a discussion.
>> 
>> Let’s discuss via how the apacheheron docker file is produced.
>> 
>> 
> Ok. Another permission to request.
> 
> The docker image is built with this jenkins job:
> https://builds.apache.org/job/apache-heron-github-docker-image-debian/
> 
> I then downloaded it and load/publish.

OK, I see this and I see several build jobs.

And wow - that’s a 1GB archive!

I wonder how much of these Jenkins scripts should be in Jenkins as opposed to 
all in the Git repository and then invoked as 1-3 scripts from Jenkins w/ 
environment variable pick up. This would ultimately benefit those who are 
trying to understand how to build Heron and what artifacts are brought into the 
binary.

We must do a careful license review of everything included in a Binary.  If 
some of the binary artifacts going to the maven repository are much smaller 
then we should discuss these separately.

Until then just make Source releases.

BTW - The apacheheron Docker Hub still appears to be from the project and 
Apache and that means it is not allowed unless it can be VOTED on.

Please see the VP, Legal and VP, Brand comments on 
https://issues.apache.org/jira/browse/LEGAL-427

(This is a better and more direct answer than on 
https://issues.apache.org/jira/browse/LEGAL-270 )

Does this make sense?

Regards,
Dave
> 
> 
>> 
>>> 
>>> I thought maven artifacts are on repository.apache.org and source
>> release
>>> (may include binary release as well in future) should be in
>> dist.apache.org.
>>> Seems I am wrong. I can add the artifacts to dist.apache.org.
>> 
>> All released artifacts should be on dist.apache.org <
>> http://dist.apache.org/> first in dev and once the vote is approved then
>> they can be moved.
>> 
>> For repository.apache.org <http://repository.apache.org/> there is an
>> ability to stage, but it may be that you burn a release version if the vote
>> fails.
>> 
>> My suggestion is that we wait to put deploy packages to maven /
>> repository.apache.org <http://repository.apache.org/> until the vote is
>> completed.

This is just a suggestion on my part.

>> 
> 
> Ok. Sounds good. Thanks.
> 
> Also, how about the convenience binary and docker packages? Just to confirm
> that they should or should not be built before the vote?
> 
> 
>> The VOTE thread should be:
>> (0) KEYS path - can already update the release location.
>> (1) For each artifact on dist.
>> - URL for artifact
>> - URL for asc signature
>> - URL for SHA512 hash
>> 
>> It should be very clear and in plain text.
>> 
>> It would be helpful on the binary artifacts to make sure there are clear
>> build instructions.
>> 
> 
> Got it.
> 
> 
>> We never discussed the large binary release.
>> 
>> 
> Right. Currently we are keeping these files in github for now and we will
> try to move them to Apache dist after reducing the file sizes in future.
> 
> 
> 
>>> 
>>> Is there anything else we are missing?
>>> 
>>> Thanks in advance.
>> 
>> Regards,
>> Dave
>> 
>>> 
>>> On Mon, Apr 1, 2019 at 2:17 PM Ning Wang  wrote:
>>> 
>>>> And - general@incubator mailing list.
>>>> 
>>>> On Mon, Apr 1, 2019 at 1:53 PM Ning Wang  wrote:
>>>> 
>>>>> Ok. Thanks!
>>>>> 
>>>>> On Mon, Apr 1, 2019 at 11:55 AM Dave Fisher 
>>>>> wrote:
>>>>> 
>>>>>> -1 - we need to carefully discuss this on dev@heron.
>>>>>> 
>>>>>> I seem to be the only Heron Mentor paying attention. We need more than
>>>>>> me!
>>>>>> 
&g

Re: [Mentors] Fixing Heron Docs

[Dave Fisher]

It looks like you are missing from the heron committers group on Git Hub. I’m 
asking Infra why on Slack. If no answer soon then I’ll file a JIRA.

Anyone else having issues?

> On Apr 3, 2019, at 11:03 AM, Josh Fischer  wrote:
> 
> I do have 2FA enabled.  I've also tried to log in/ log out.  No success.
> Any suggestions on who I should contact to help diagnose and fix?
> 
> On Wed, Apr 3, 2019 at 12:58 PM Dave Fisher  wrote:
> 
>> 
>> 
>>> On Apr 3, 2019, at 10:49 AM, Josh Fischer  wrote:
>>> 
>>> Thanks for the info Dave.  I have a couple of responses to your
>> questions.
>>> 
>>> (1) Does the project wish to create documentation in a new repository -
>> say
>>> apache/incubator-heron-site
>>>   Or should the build use an asf-site branch in
>>> apache/incubator-heron which just keeps the deployed content.
>>> 
>>> ** The site build is currently dependent on Bazel to generate Javadocs.
>> I
>>> think we should just have a "asf-site" branch that is used within the
>>> current repo to serve static files from as it would require less moving
>>> parts.  Not sure if that would cause any issues with the current setup
>>> though.  I was asking Ning a similar question on how to set this up
>> since I
>>> don't have write access to the repo yet.Hi -
>> 
>> Really? - Have you enabled 2FA on GitHub for your joshfischer1108 username?
>> 
>> I recommend that you get access fixed and then work on doing the build
>> locally. Once you have that we can work on the Jenkins build with triggers.
>> 
>>> 
>>> 
>>> (2) Does the website Jenkins box already support the build requirements.
>>> 
>>> ** I will look into this.
>> 
>> Ask on the-asf.slack.com #asfinfra - tell them it is for a gitpubsub
>> website build.
>> 
>> Regards,
>> Dave
>> 
>> 
>>> 
>>> On Wed, Apr 3, 2019 at 10:43 AM Dave Fisher 
>> wrote:
>>> 
>>>> Hi -
>>>> 
>>>> TL:DR - The project should definitely use gitpubsub approach.
>>>> 
>>>>> On Mar 28, 2019, at 7:08 PM, Josh Fischer  wrote:
>>>>> 
>>>>> How does everyone want to manage publishing the site? After looking at
>>>> the
>>>>> docs it seems there are 3 options.  Being that we are using git, I
>> would
>>>>> thing that gitsubpub would be the path of least resistance?...  But I
>>>> can't
>>>>> find much on how to create the jenkins job.  Maybe this is listed
>>>> somewhere
>>>>> else in the documentation?  Does anyone have other suggestions or
>> ideas?
>>>>> 
>>>>> Here is link to the guidelines ->
>>>>> http://www.apache.org/dev/project-site.html. I copy and pasted some
>> info
>>>>> below on the different methods of website publishing for quick
>> reference.
>>>>> 
>>>>> 
>>>>> - Apache CMS, which provides a simple browser-based user interface for
>>>>> editing, staging and publishing site content in Markdown, HTML or any
>>>> other
>>>>> source format for which support has been added. See the CMS reference
>>>>> <https://www.apache.org/dev/cmsref> and adoption
>>>>> <https://www.apache.org/dev/cmsadoption> for more details. The Apache
>>>>> CMS uses svnpubsub as the underlying mechanism for publishing a site.
>>>> 
>>>> While this was home grown in Apache and all the rage 7-8 years ago when
>> I
>>>> used this to port over OpenOffice.org with a reskin. This should not be
>>>> used. Infra wants people to stop and there are occasional publishing
>> issues.
>>>> 
>>>>> - svnpubsub, which allows the static contents of a designated svn
>>>> folder
>>>>> (example <http://svn.apache.org/repos/asf/ant/site/ant/production/>)
>>>> to
>>>>> automatically published as the project web site at
>>>>> http://project.apache.org/. The project team can use any site build
>>>>> mechanism it wants as long as the above requirements are met.
>>>> 
>>>> The CMS also uses svnpubsub. Not many new projects or podlings are using
>>>> Subversion other than as required for releases.
>>>> 
>>>>> - gitpubsub, which allows the static contents of a designated git
>>>>> repository to be served as the web

Re: [Mentors] Fixing Heron Docs

[Dave Fisher]

> On Apr 3, 2019, at 10:49 AM, Josh Fischer  wrote:
> 
> Thanks for the info Dave.  I have a couple of responses to your questions.
> 
> (1) Does the project wish to create documentation in a new repository - say
> apache/incubator-heron-site
>Or should the build use an asf-site branch in
> apache/incubator-heron which just keeps the deployed content.
> 
> ** The site build is currently dependent on Bazel to generate Javadocs.  I
> think we should just have a "asf-site" branch that is used within the
> current repo to serve static files from as it would require less moving
> parts.  Not sure if that would cause any issues with the current setup
> though.  I was asking Ning a similar question on how to set this up since I
> don't have write access to the repo yet.Hi -

Really? - Have you enabled 2FA on GitHub for your joshfischer1108 username?

I recommend that you get access fixed and then work on doing the build locally. 
Once you have that we can work on the Jenkins build with triggers.

> 
> 
> (2) Does the website Jenkins box already support the build requirements.
> 
> ** I will look into this.

Ask on the-asf.slack.com #asfinfra - tell them it is for a gitpubsub website 
build.

Regards,
Dave


> 
> On Wed, Apr 3, 2019 at 10:43 AM Dave Fisher  wrote:
> 
>> Hi -
>> 
>> TL:DR - The project should definitely use gitpubsub approach.
>> 
>>> On Mar 28, 2019, at 7:08 PM, Josh Fischer  wrote:
>>> 
>>> How does everyone want to manage publishing the site? After looking at
>> the
>>> docs it seems there are 3 options.  Being that we are using git, I would
>>> thing that gitsubpub would be the path of least resistance?...  But I
>> can't
>>> find much on how to create the jenkins job.  Maybe this is listed
>> somewhere
>>> else in the documentation?  Does anyone have other suggestions or ideas?
>>> 
>>> Here is link to the guidelines ->
>>> http://www.apache.org/dev/project-site.html. I copy and pasted some info
>>> below on the different methods of website publishing for quick reference.
>>> 
>>> 
>>>  - Apache CMS, which provides a simple browser-based user interface for
>>>  editing, staging and publishing site content in Markdown, HTML or any
>> other
>>>  source format for which support has been added. See the CMS reference
>>>  <https://www.apache.org/dev/cmsref> and adoption
>>>  <https://www.apache.org/dev/cmsadoption> for more details. The Apache
>>>  CMS uses svnpubsub as the underlying mechanism for publishing a site.
>> 
>> While this was home grown in Apache and all the rage 7-8 years ago when I
>> used this to port over OpenOffice.org with a reskin. This should not be
>> used. Infra wants people to stop and there are occasional publishing issues.
>> 
>>>  - svnpubsub, which allows the static contents of a designated svn
>> folder
>>>  (example <http://svn.apache.org/repos/asf/ant/site/ant/production/>)
>> to
>>>  automatically published as the project web site at
>>>  http://project.apache.org/. The project team can use any site build
>>>  mechanism it wants as long as the above requirements are met.
>> 
>> The CMS also uses svnpubsub. Not many new projects or podlings are using
>> Subversion other than as required for releases.
>> 
>>>  - gitpubsub, which allows the static contents of a designated git
>>>  repository to be served as the website for a project. git based
>> websites
>>>  are typically maintained in a asf-site branch to be published as
>>>  https://project.apache.org. They can be hosted from your primary
>> project
>>>  repo. Typically these will be built as a jenkins job or a buildbot
>> job. It
>>>  is recommended that you only have a single writer to the asf-site
>> branch to
>>>  avoid potential conflicts.
>> 
>> I’ve just revised the Incubator’s site build since we have a mix of svn
>> legacy, current analysis and git static content.
>> 
>> See https://github.com/apache/incubator and the README.md. This is a
>> jBake site with two Jenkins build steps.
>> 
>> This should be the approach. Two areas for further discussion.
>> 
>> (1) Does the project wish to create documentation in a new repository -
>> say apache/incubator-heron-site
>>Or should the build use an asf-site branch in
>> apache/incubator-heron which just keeps the deployed content.
>> (2) Does the website Jenkins box already support the build requirements.
>> 
>>>> Hugo --- Static site ge

Re: [Mentors] Fixing Heron Docs

[Dave Fisher]

> On Apr 3, 2019, at 10:49 AM, Josh Fischer  wrote:
> 
> Thanks for the info Dave.  I have a couple of responses to your questions.
> 
> (1) Does the project wish to create documentation in a new repository - say
> apache/incubator-heron-site
>Or should the build use an asf-site branch in
> apache/incubator-heron which just keeps the deployed content.
> 
> ** The site build is currently dependent on Bazel to generate Javadocs.  I
> think we should just have a "asf-site" branch that is used within the
> current repo to serve static files from as it would require less moving
> parts.  Not sure if that would cause any issues with the current setup
> though.  I was asking Ning a similar question on how to set this up since I
> don't have write access to the repo yet.
> 
> 
> (2) Does the website Jenkins box already support the build requirements.
> 
> ** I will look into this.
> 
> On Wed, Apr 3, 2019 at 10:43 AM Dave Fisher  wrote:
> 
>> Hi -
>> 
>> TL:DR - The project should definitely use gitpubsub approach.
>> 
>>> On Mar 28, 2019, at 7:08 PM, Josh Fischer  wrote:
>>> 
>>> How does everyone want to manage publishing the site? After looking at
>> the
>>> docs it seems there are 3 options.  Being that we are using git, I would
>>> thing that gitsubpub would be the path of least resistance?...  But I
>> can't
>>> find much on how to create the jenkins job.  Maybe this is listed
>> somewhere
>>> else in the documentation?  Does anyone have other suggestions or ideas?
>>> 
>>> Here is link to the guidelines ->
>>> http://www.apache.org/dev/project-site.html. I copy and pasted some info
>>> below on the different methods of website publishing for quick reference.
>>> 
>>> 
>>>  - Apache CMS, which provides a simple browser-based user interface for
>>>  editing, staging and publishing site content in Markdown, HTML or any
>> other
>>>  source format for which support has been added. See the CMS reference
>>>  <https://www.apache.org/dev/cmsref> and adoption
>>>  <https://www.apache.org/dev/cmsadoption> for more details. The Apache
>>>  CMS uses svnpubsub as the underlying mechanism for publishing a site.
>> 
>> While this was home grown in Apache and all the rage 7-8 years ago when I
>> used this to port over OpenOffice.org with a reskin. This should not be
>> used. Infra wants people to stop and there are occasional publishing issues.
>> 
>>>  - svnpubsub, which allows the static contents of a designated svn
>> folder
>>>  (example <http://svn.apache.org/repos/asf/ant/site/ant/production/>)
>> to
>>>  automatically published as the project web site at
>>>  http://project.apache.org/. The project team can use any site build
>>>  mechanism it wants as long as the above requirements are met.
>> 
>> The CMS also uses svnpubsub. Not many new projects or podlings are using
>> Subversion other than as required for releases.
>> 
>>>  - gitpubsub, which allows the static contents of a designated git
>>>  repository to be served as the website for a project. git based
>> websites
>>>  are typically maintained in a asf-site branch to be published as
>>>  https://project.apache.org. They can be hosted from your primary
>> project
>>>  repo. Typically these will be built as a jenkins job or a buildbot
>> job. It
>>>  is recommended that you only have a single writer to the asf-site
>> branch to
>>>  avoid potential conflicts.
>> 
>> I’ve just revised the Incubator’s site build since we have a mix of svn
>> legacy, current analysis and git static content.
>> 
>> See https://github.com/apache/incubator and the README.md. This is a
>> jBake site with two Jenkins build steps.
>> 
>> This should be the approach. Two areas for further discussion.
>> 
>> (1) Does the project wish to create documentation in a new repository -
>> say apache/incubator-heron-site
>>Or should the build use an asf-site branch in
>> apache/incubator-heron which just keeps the deployed content.
>> (2) Does the website Jenkins box already support the build requirements.
>> 
>>>> Hugo --- Static site generator
>>>> GulpJS --- Build tool for static assets
>>>> Twitter Bootstrap --- Sass/CSS and JavaScript
>>>> Documentation Setup
>>>> Running the Heron documentation locally requires that you have the
>>>> following installed:
>>>> 
>>>> Make
>>

Re: [Mentors] Fixing Heron Docs

[Dave Fisher]

Hi -

TL:DR - The project should definitely use gitpubsub approach.

> On Mar 28, 2019, at 7:08 PM, Josh Fischer  wrote:
> 
> How does everyone want to manage publishing the site? After looking at the
> docs it seems there are 3 options.  Being that we are using git, I would
> thing that gitsubpub would be the path of least resistance?...  But I can't
> find much on how to create the jenkins job.  Maybe this is listed somewhere
> else in the documentation?  Does anyone have other suggestions or ideas?
> 
> Here is link to the guidelines ->
> http://www.apache.org/dev/project-site.html. I copy and pasted some info
> below on the different methods of website publishing for quick reference.
> 
> 
>   - Apache CMS, which provides a simple browser-based user interface for
>   editing, staging and publishing site content in Markdown, HTML or any other
>   source format for which support has been added. See the CMS reference
>   <https://www.apache.org/dev/cmsref> and adoption
>   <https://www.apache.org/dev/cmsadoption> for more details. The Apache
>   CMS uses svnpubsub as the underlying mechanism for publishing a site.

While this was home grown in Apache and all the rage 7-8 years ago when I used 
this to port over OpenOffice.org with a reskin. This should not be used. Infra 
wants people to stop and there are occasional publishing issues.

>   - svnpubsub, which allows the static contents of a designated svn folder
>   (example <http://svn.apache.org/repos/asf/ant/site/ant/production/>) to
>   automatically published as the project web site at
>   http://project.apache.org/. The project team can use any site build
>   mechanism it wants as long as the above requirements are met.

The CMS also uses svnpubsub. Not many new projects or podlings are using 
Subversion other than as required for releases.

>   - gitpubsub, which allows the static contents of a designated git
>   repository to be served as the website for a project. git based websites
>   are typically maintained in a asf-site branch to be published as
>   https://project.apache.org. They can be hosted from your primary project
>   repo. Typically these will be built as a jenkins job or a buildbot job. It
>   is recommended that you only have a single writer to the asf-site branch to
>   avoid potential conflicts.

I’ve just revised the Incubator’s site build since we have a mix of svn legacy, 
current analysis and git static content.

See https://github.com/apache/incubator and the README.md. This is a jBake site 
with two Jenkins build steps.

This should be the approach. Two areas for further discussion.

(1) Does the project wish to create documentation in a new repository - say 
apache/incubator-heron-site
Or should the build use an asf-site branch in apache/incubator-heron 
which just keeps the deployed content.
(2) Does the website Jenkins box already support the build requirements.

>> Hugo --- Static site generator
>> GulpJS --- Build tool for static assets
>> Twitter Bootstrap --- Sass/CSS and JavaScript
>> Documentation Setup
>> Running the Heron documentation locally requires that you have the
>> following installed:
>> 
>> Make
>> Node.js
>> npm
>> pip - install PyYAML>=3.12
>> Go (make sure that your GOPATH and GOROOT are set)

What versions of the above and we can check with Infra through the 
bui...@apache.org email.

Regards,
Dave


>> 
>> On Fri, Mar 22, 2019 at 10:50 AM Josh Fischer  wrote:
>> 
>>> I have some time free over the next week, I can look into fixing the
>>> build.. However, I think we should figure out the repo situation first
>> with
>>> the submodule that is tied to the documentation.
>>> 
>>> My responses are in *bold.  *
>>> 
>>> On Fri, Mar 22, 2019 at 12:26 PM Dave Fisher 
>>> wrote:
>>> 
>>>> Hi -
>>>> 
>>>> Thanks for asking. This has been on my list to discuss with the Heron
>>>> community.
>>>> 
>>>> The website is seriously non-compliant.
>>>> 
>>>>> On Mar 22, 2019, at 9:38 AM, Josh Fischer 
>> wrote:
>>>>> 
>>>>> Hey All,
>>>>> 
>>>>> With the heavy lifting behind us on the Bazel upgrade, I think the
>> next
>>>>> task we should tackle is fixing the public docs. I have a few
>> questions
>>>>> about them.
>>>>> 
>>>>>  1. Are they still to be served from GH pages?
>>>> 
>>>> No.
>>>> 
>>>>>  2. If not where do they need to live?
>>>> 
>>>> It should be on Apache servers as heron.apache.org 

[DISCUSS][VOTE] Heron Release 0.20.1-incubating (Release Candidate 2)

[Dave Fisher]

Hi -

> On Apr 1, 2019, at 2:23 PM, Ning Wang  wrote:
> 
> Hi,
> 
> For the 0.20.1-incubating-rc2 release:
> 
> So we should require more buy-ins from mentors for the release before
> moving to general@ mailing list? Maybe let's have a rule like minimal two
> Mentors? Sounds good to me but let's make a decision.

Ideally we should have 3 binding VOTEs from Mentors and then we don’t have to 
go to the IPMC unless we want additional review.

If we can’t get that from the current Mentors then on the podling report we say 
so and we also look for mentors who will VOTE.

> 
> For docker hub image, I published it to
> *https://hub.docker.com/r/apacheheron/heron/tags
> <https://hub.docker.com/r/apacheheron/heron/tags>. It is not the right one?
> I got an invite after becoming a committer then I thought this is the
> official one. But maybe it isn't?*

That one is created and managed by this project, and as such it remains an 
unofficial location - we’ll need to be careful how it is advertised.

The official Apache Docker Hub is https://hub.docker.com/u/apache 
<https://hub.docker.com/u/apache>

https://reference.apache.org/pmc/docker 
<https://reference.apache.org/pmc/docker> for an example.

https://issues.apache.org/jira/browse/LEGAL-270 
<https://issues.apache.org/jira/browse/LEGAL-270> for a discussion.

Let’s discuss via how the apacheheron docker file is produced. 


> 
> I thought maven artifacts are on repository.apache.org and source release
> (may include binary release as well in future) should be in dist.apache.org.
> Seems I am wrong. I can add the artifacts to dist.apache.org.

All released artifacts should be on dist.apache.org <http://dist.apache.org/> 
first in dev and once the vote is approved then they can be moved.

For repository.apache.org <http://repository.apache.org/> there is an ability 
to stage, but it may be that you burn a release version if the vote fails.

My suggestion is that we wait to put deploy packages to maven / 
repository.apache.org <http://repository.apache.org/> until the vote is 
completed.

The VOTE thread should be:
(0) KEYS path - can already update the release location.
(1) For each artifact on dist.
- URL for artifact
- URL for asc signature
- URL for SHA512 hash

It should be very clear and in plain text.

It would be helpful on the binary artifacts to make sure there are clear build 
instructions.

We never discussed the large binary release.

> 
> Is there anything else we are missing?
> 
> Thanks in advance.

Regards,
Dave

> 
> On Mon, Apr 1, 2019 at 2:17 PM Ning Wang  wrote:
> 
>> And - general@incubator mailing list.
>> 
>> On Mon, Apr 1, 2019 at 1:53 PM Ning Wang  wrote:
>> 
>>> Ok. Thanks!
>>> 
>>> On Mon, Apr 1, 2019 at 11:55 AM Dave Fisher 
>>> wrote:
>>> 
>>>> -1 - we need to carefully discuss this on dev@heron.
>>>> 
>>>> I seem to be the only Heron Mentor paying attention. We need more than
>>>> me!
>>>> 
>>>> (1) I cannot really follow this email
>>>> (2) We still need to discuss the docker hub image. (I suggest that this
>>>> be unofficial for this round.)
>>>> (3) The binaries on repository.apache.org <http://repository.apache.org/>
>>>> are not on dist.apache.org <http://dist.apache.org/>.
>>>> 
>>>> Apologies, let’s rewind what is included for this release. It needs to
>>>> be discussed on dev@ in advance.
>>>> 
>>>> Regards,
>>>> Dave
>>>> 
>>>>> On Apr 1, 2019, at 11:08 AM, Ning Wang  wrote:
>>>>> 
>>>>> *Hello, dear IPMC members,This is a call for a vote to release Apache
>>>> Heron
>>>>> (Incubating) version 0.20.1.The Apache Heron Community has voted to
>>>> make
>>>>> the Heron Release 0.20.1-incubating release. We kindly request the
>>>>> Incubator PMC members review and vote on this incubator release.The dev
>>>>> voting thread is
>>>>> here:
>>>> https://lists.apache.org/thread.html/7c61de9884bda8f95b798b40ce0bb90b7c768e05f1a90d45e164a7cf@%3Cdev.heron.apache.org%3E
>>>>> <
>>>> https://lists.apache.org/thread.html/7c61de9884bda8f95b798b40ce0bb90b7c768e05f1a90d45e164a7cf@%3Cdev.heron.apache.org%3E
>>>>> Apache
>>>>> Heron(incubating) is a realtime, distributed, fault-tolerant stream
>>>>> processing engine. This release include source code, maven artifacts.
>>>>> Convenience binary packages are also included but not relevant for
>>>> voting
>>>>> purposes.The 

Re: 3/26/2019 Bi-Weekly OSS Heron Sync-up

[Dave Fisher]

Sent from my iPhone

> On Mar 25, 2019, at 10:51 AM, Saikat Kanjilal  wrote:
> 
> My updates
> --Bazel computational upgrade complete
> --Design doc for neo4j spout for heron out for review awaiting community 
> feedback

Where is this document? A link would help!

Thanks,
Dave

> --Design doc for spark spout for heron in the works
> 
> Regards
> 
> 
> From: Ning Wang 
> Sent: Monday, March 25, 2019 10:38 AM
> To: dev@heron.incubator.apache.org
> Subject: 3/26/2019 Bi-Weekly OSS Heron Sync-up
> 
> Hi, all~,
> 
> It has been two weeks since our last sync and there has been quite a few
> works done! Let's share our works for the last two weeks in this thread.
> 
> My updates (mostly internal works, not much on Apache side):
> - Release 0.20.1-incubating-rc1 was sent out for vote and a license issue
> was found. Preparing 0.20.1-incubating-rc2 currently.
> - Small update in Heron UI.
> - Make cppcheck independent of pcre.
> 
> Regards,
> --ning

Re: [Mentors] Fixing Heron Docs

[Dave Fisher]

Hi -

Thanks for asking. This has been on my list to discuss with the Heron community.

The website is seriously non-compliant.

> On Mar 22, 2019, at 9:38 AM, Josh Fischer  wrote:
> 
> Hey All,
> 
> With the heavy lifting behind us on the Bazel upgrade, I think the next
> task we should tackle is fixing the public docs. I have a few questions
> about them.
> 
>   1. Are they still to be served from GH pages?

No.

>   2. If not where do they need to live?

It should be on Apache servers as heron.apache.org 

Many projects build through Jenkins. There are requirements for the site which 
are scanned.

There is a page here (needs some updates)

https://incubator.apache.org/guides/sites.html 


Some projects keep a separate GitHub repository for their website.

There is a lot wrong with the current website.

This page was put onto the Incubator website this week. (It’s been my 
“distraction” the last few weeks.)
https://incubator.apache.org/clutch/heron.html#errata 


Check out that almost all ASF links, trademarks, disclaimer, and copyrights are 
not correct on a site scan that is periodically performed.

https://whimsy.apache.org/pods/project/heron 



>   3. If I remember correctly, some steps in the Makefile are currently
>   broken for building the static assets.  Once those issues are resolved is
>   there anything else we need to do to release new set of docs?

Follow what I listed on 2.

What technology is used to build assets currently?

>   4. Does updating the docs require a vote?

Not normally. The Podling can decide if you want to use RTC or CTR. Since 
everything is in version control I would lean to CTR!

Regards,
Dave


> 
> - Josh

Re: [DISCUSS] Hosting Heron binary release packages

[Dave Fisher]

Hi -

> On Mar 14, 2019, at 10:31 AM, Ning Wang  wrote:
> 
> Thanks Dave! Let me start a vote~
> 
> No complain to you guys. :)  It seems like Apache infra doesn't have a clear 
> guideline about this case. Otherwise it could be a lot more straightforward. 
> Hopefully this issue will be improved soon.

The ASF has had continual evolution over 20 + years. What Infrastructure 
provides now is way more obvious than in the past. You mentors are here to 
guide you through things.

The Docker capability is new and that is certainly not well understood. The 
best thing to do with Infra if your mentors cannot help is to ask asfinfra on 
Slack (it was hip chat last year) - https://theasf.slack.com 
<https://theasf.slack.com/>
> 
> I don't think our jars contain LICENSE files, this is required? All 
> dependencies are built from source code so hopefully they are ok.

The first time is the hardest with binaries and it will take time. The key is 
listing each dependency that gets included and reviewing its license. If any 
are “category X” then they need to be replaced. The Release Policy is the 
responsibility of Legal Affairs.

Here is the reference about which licenses can be included. [1] This is 
frequently updates. If you have any doubts then we ask through a LEGAL jira or 
legal-disc...@apache.org <mailto:legal-disc...@apache.org> ( a public list)

Regards,
Dave

[1] http://www.apache.org/legal/resolved.html


> 
> 
> 
> On Thu, Mar 14, 2019 at 10:24 AM Dave Fisher  <mailto:dave2w...@comcast.net>> wrote:
> Hi -
> 
> Agreed. Let’s just make sure we review how this convenience is shown as some 
> on the IPMC are concerned about how the Github release tab appears.
> 
> For binary packages we really need to know and carefully review the licensing 
> for all the included dependencies. It is more than just the Jars as a whole 
> but what the builds have included. I apologize for not having enough time to 
> push this part of the review.
> 
> On the Dev list we can discuss where users might find these prospective 
> binary packages which will have no guarantees about licensing.
> 
> Let’s do an Official Apache Source Release.
> 
> Regards,
> Dave
> 
>> On Mar 14, 2019, at 10:17 AM, Ning Wang > <mailto:wangnin...@gmail.com>> wrote:
>> 
>> I am thinking that we can leave the "convenience community binary packages" 
>> in github for this release (because of the file size) and then try to reduce 
>> the package size for future releases so that they are more friendly to 
>> Apache infrastructure. It has been blocked by this for a while and I am 
>> hoping to start the vote sooner than later.
>> 
>> Is this ok? @Dave Fisher <mailto:dave2w...@comcast.net> @P. Taylor Goetz 
>> <mailto:ptgo...@gmail.com> ^^
>> 
>> 
>> 
>> On Tue, Mar 12, 2019 at 5:37 PM Ning Wang > <mailto:wangnin...@gmail.com>> wrote:
>> For the content in the packages:
>> 
>> Currently we have binary packages for three platforms: osx, centos and 
>> ubuntu. The file sizes are fairly close among them. I am going to use ubuntu 
>> as example:
>> 
>> - there is a tar.gz file and an sh file. I am thinking we may keep the sh 
>> file only since it is easier to use and contains the same data as the tar.gz 
>> file. This can save about 50% in size (from 470 * 6 -> 470 * 3).
>> 
>> The file size is 470m.
>> -rw-r--r--@ 1 nwang  staff  470508875 Feb 27 10:57 
>> heron-v0.20.1-incubating-rc1-ubuntu14.04.tar.gz
>> 
>> The package is trim-able I believe, but there isn't many obvious ones:
>> drwxr-xr-x@ 14 nwang  staff  448 Dec 31  1969 bin
>> drwxr-xr-x@ 16 nwang  staff  512 Dec 31  1969 conf
>> drwxr-xr-x@  5 nwang  staff  160 Mar  6 15:10 dist
>> drwxr-xr-x@ 16 nwang  staff  512 Dec 31  1969 examples
>> drwxr-xr-x@ 11 nwang  staff  352 Dec 31  1969 include
>> drwxr-xr-x@ 12 nwang  staff  384 Mar  6 15:04 lib
>> -r-xr-xr-x@  1 nwang  staff  285 Dec 31  1969 release.yaml
>> 
>> It is used to install a local environment that contains all necessary 
>> components:
>> - bin: binaries for tools like api server, tracker, ui, etc
>> - conf: configurations for different environments
>> - dist: heron core and its tar.gz file (194m, might be trim-able).
>> - examples: example topology jars
>> - include: c++ header files
>> - lib: c++ and java lib files
>> 
>> More detailed information is attached at the end of this email.
>> 
>> /dist/core/core and /lib are the biggest dirs and they might have a few 
>> duplicated big files we can try to get rid of.
>> 
>> heron-downloader.jar 16m * 2
>> heron-metrics

Re: [DISCUSS] Hosting Heron binary release packages

[Dave Fisher]

Hi -

Agreed. Let’s just make sure we review how this convenience is shown as some on 
the IPMC are concerned about how the Github release tab appears.

For binary packages we really need to know and carefully review the licensing 
for all the included dependencies. It is more than just the Jars as a whole but 
what the builds have included. I apologize for not having enough time to push 
this part of the review.

On the Dev list we can discuss where users might find these prospective binary 
packages which will have no guarantees about licensing.

Let’s do an Official Apache Source Release.

Regards,
Dave

> On Mar 14, 2019, at 10:17 AM, Ning Wang  wrote:
> 
> I am thinking that we can leave the "convenience community binary packages" 
> in github for this release (because of the file size) and then try to reduce 
> the package size for future releases so that they are more friendly to Apache 
> infrastructure. It has been blocked by this for a while and I am hoping to 
> start the vote sooner than later.
> 
> Is this ok? @Dave Fisher <mailto:dave2w...@comcast.net> @P. Taylor Goetz 
> <mailto:ptgo...@gmail.com> ^^
> 
> 
> 
> On Tue, Mar 12, 2019 at 5:37 PM Ning Wang  <mailto:wangnin...@gmail.com>> wrote:
> For the content in the packages:
> 
> Currently we have binary packages for three platforms: osx, centos and 
> ubuntu. The file sizes are fairly close among them. I am going to use ubuntu 
> as example:
> 
> - there is a tar.gz file and an sh file. I am thinking we may keep the sh 
> file only since it is easier to use and contains the same data as the tar.gz 
> file. This can save about 50% in size (from 470 * 6 -> 470 * 3).
> 
> The file size is 470m.
> -rw-r--r--@ 1 nwang  staff  470508875 Feb 27 10:57 
> heron-v0.20.1-incubating-rc1-ubuntu14.04.tar.gz
> 
> The package is trim-able I believe, but there isn't many obvious ones:
> drwxr-xr-x@ 14 nwang  staff  448 Dec 31  1969 bin
> drwxr-xr-x@ 16 nwang  staff  512 Dec 31  1969 conf
> drwxr-xr-x@  5 nwang  staff  160 Mar  6 15:10 dist
> drwxr-xr-x@ 16 nwang  staff  512 Dec 31  1969 examples
> drwxr-xr-x@ 11 nwang  staff  352 Dec 31  1969 include
> drwxr-xr-x@ 12 nwang  staff  384 Mar  6 15:04 lib
> -r-xr-xr-x@  1 nwang  staff  285 Dec 31  1969 release.yaml
> 
> It is used to install a local environment that contains all necessary 
> components:
> - bin: binaries for tools like api server, tracker, ui, etc
> - conf: configurations for different environments
> - dist: heron core and its tar.gz file (194m, might be trim-able).
> - examples: example topology jars
> - include: c++ header files
> - lib: c++ and java lib files
> 
> More detailed information is attached at the end of this email.
> 
> /dist/core/core and /lib are the biggest dirs and they might have a few 
> duplicated big files we can try to get rid of.
> 
> heron-downloader.jar 16m * 2
> heron-metricscachemgr.jar  8.8m * 2
> heron-binpacking-packing.jar 5.3m * 3
> heron-roundrobin-packing.jar 5.3m * 2
> scheduler jars 60m * 2
> heron-localfs-statemgr.jar 5.3m * 2
> heron-zookeeper-statemgr.jar 6.6m * 2
> 
> In case we can get rid of the core.tar.gz and the duplicated files, we might 
> be able to reduce the size to 480m - 190m - 100m = 190m. Then these binary 
> packages will be 190 * 3 = 570m, after some works.
> 
> 
> 
> heron-v0.20.1-incubating-rc1-ubuntu14.04
> ├── [ 448]  bin
> │   ├── [2.1M]  heron
> │   ├── [2.1M]  heron-admin
> │   ├── [1.5K]  heron-apiserver
> │   ├── [1.5K]  heron-apiserver.sh
> │   ├── [1.2K]  heron-downloader
> │   ├── [ 931]  heron-downloader-config
> │   ├── [ 931]  heron-downloader-config.sh
> │   ├── [1.2K]  heron-downloader.sh
> │   ├── [2.2M]  heron-explorer
> │   ├── [ 42M]  heron-nomad
> │   ├── [2.5M]  heron-tracker
> │   └── [3.5M]  heron-ui
> ├── [ 512]  conf
> │   ├── [ 384]  aurora
> │   │   ├── [1.1K]  client.yaml
> │   │   ├── [1.1K]  downloader.yaml
> │   │   ├── [3.3K]  heron.aurora
> │   │   ├── [ 12K]  heron_internals.yaml
> │   │   ├── [6.4K]  metrics_sinks.yaml
> │   │   ├── [1.0K]  packing.yaml
> │   │   ├── [2.7K]  scheduler.yaml
> │   │   ├── [1.6K]  stateful.yaml
> │   │   ├── [2.3K]  statemgr.yaml
> │   │   └── [1.1K]  uploader.yaml
> │   ├── [ 352]  examples
> │   │   ├── [1.3K]  aurora_scheduler.yaml
> │   │   ├── [1.1K]  downloader.yaml
> │   │   ├── [ 12K]  heron_internals.yaml
> │   │   ├── [1.3K]  local_scheduler.yaml
> │   │   ├── [1.5K]  local_stateful.yaml
> │   │   ├── [ 951]  localfs_statemgr.yaml
> │   │   ├── [1001]  localfs_uploader.yaml
> │   │   ├── [7.3K]  metrics_sinks.yaml
> │   │   └── [ 800]  roundrobin_packing.yaml
> │   ├── [2.3K]  heron_tracker.yaml
> │   ├── [ 352]  kub

Re: 3/12/2019 Bi-Weekly OSS Heron Sync-up

[Dave Fisher]

Hi -

> On Mar 12, 2019, at 4:22 PM, Josh Fischer  wrote:
> 
> Not too much for me this week.
> 
> I participated in a few conversations on the Bazel upgrade process and the
> script that sets up IntelliJ.
> 
> A thought/question  on my mind: How do we move conversations that happen in
> slack to the mailing list?  I know this has been discussed a few times.
> Maybe some time of job that pulls conversations from each channel and mails
> them to the list on a nightly basis?

Ask the Apache Pulsar team about slack jobs - Matteo has one.

> 
> I can understand the ease of use of slack and why we have it.  However I
> also understand the importance of managing this project’s discussions
> through the mailing list.

Dev list is preferred for long thoughtful asynchronous threaded discussions 
that you may want to find 15 years from now. I have gone that far back on 
projects where I’ve been on the PMC for 11 years. Slack my be gone in 10 years, 
but the Apache Archives will live.

Regards,
Dave

> 
> - Josh
> 
> On Tue, Mar 12, 2019 at 4:55 PM Xiaoyao Qian  wrote:
> 
>> Hi all,
>> 
>> My updates:
>> 1. Modified the FirstFitDecreasingPacking algorithm so that now it takes
>> CPU and RAM constraints into consideration when generating a packing plan.
>> 2. Added static (cppcheck) and runtime (gperftool heapcheck) checks to
>> native cpp modules.
>> 
>> Xiaoyao Qian
>> 
>> 
>> On Tue, Mar 12, 2019 at 2:48 PM Ning Wang  wrote:
>> 
>>> Friendly ping~
>>> 
>>> On Mon, Mar 11, 2019 at 11:42 AM Neng Lu  wrote:
>>> 
 Hi All,
 
 My updates:
 - I'm finishing the process of a security vulnerability issue.
 
 On Mon, Mar 11, 2019 at 11:19 AM FatJ Love 
 wrote:
 
> My update
> - Worked with OSS team about the March meetup
> 
> On Mon, Mar 11, 2019 at 10:39 AM Ning Wang 
>>> wrote:
> 
>> Hi, all~,
>> 
>> It has been two weeks since our last sync and it is time to share
>> our
>> progress again. Let's share our works for the last two weeks in
>> this
>> thread.
>> 
>> My updates (mostly internal works, not much on Apache side):
>> - Worked with OSS team about the April meetup
>> - Maven artifacts for Apache release 0.20.1 rc1 is uploaded to
>> Apache
> maven
>> repo:
>> 
>> 
> 
 
>>> 
>> https://repository.apache.org/content/repositories/staging/org/apache/heron/
>> .
>> Working on the binary packages now. The current discussion about
>> file
>> hosting can be found here:
>> 
>> 
> 
 
>>> 
>> http://mail-archives.apache.org/mod_mbox/heron-dev/201903.mbox/%3CCAHZ_pm6Dh0mtRdRZL07jMPKZitr%3DNmQLFq7ZuDzL_jGRR%2Btxxw%40mail.gmail.com%3E
>> 
>> 
>> Regards,
>> --ning
>> 
> 
 
 
 --
 Best Regards,
 Neng
 
>>> 
>> 
> -- 
> Sent from A Mobile Device

Re: [DISCUSS] Hosting Heron binary release packages

[Dave Fisher]

Hi -

> On Mar 8, 2019, at 1:58 PM, Ning Wang  wrote:
> 
> Hi,
> 
> I have been trying to release Heron 0.20.1 (being distracted time by time
> though) and the most recent question I am having is where to put the binary
> packages.
> 
> The binary packages are (when we were doing releases on github):
> - tar.gz packages for osx, centos and ubuntu, each one includes all modules
> like core, lib, tools, etc.
> - .sh packages for the three platforms. which is an installer for the
> modules in the tar.gz packages.
> - docker image (dockerhub, not github most of the times)

Let’s discuss the components in each binary package and how big that they 
really are and need to be.

> 
> Currently each package is more than 400MB.

When packages of this size are released from dev to the release area it 
requires replication to the Apache Mirror system. When the size exceeds an 
aggregate of 1GB then Infra needs to manually handle things to avoid impacting 
the mirrors. (There are 250 projects using the mirrors.)



> 
> I was trying to understand the Apache rules and my impression was that
> these package should be on dist.apache.org like the src packages (I might
> be wrong about the rules though) and it looks like Apache Storm has a
> binary package in their release.

Make the case for Heron without comparison to other projects.

There is a place to make Apache Docker releases on docker hub. Let’s figure out 
this if it is a valid distribution that Heron could make.

> 
> However it seems Apache infra has a byte limit of 500MB for each release. I
> guess it means that the binary packages are not "required" to be on Apache
> infra?

Yes and no. Let’s discuss the packages first.

> 
> The binary packages are convenient for users. So I think they should be
> included in release. The question is where should we put them?
> 
> So far it looks like the options are:
> - ask for an exception and publish them to dist.apache.org. It seems like
> Apache infra guys don't suggest this solution.
> - publish only src package to dist.apache.org and publish the binary
> packages on github (or is there any other suggestion?). This is convenient
> for us and there is no problem so far (we have binary packages for all the
> previous releases and github hasn't complained). The question about this
> option is more like if this is acceptable (or ever better suggested) by
> Apache?
> 
> What do you think about the two options above and any other options we
> should consider?

Let’s discuss the packages.

Next we will also need to discuss the website.

> 
> Thanks.
> --ning

Re: Incubator Podling Report (Due 6th February)

[Dave Fisher]

> On Feb 4, 2019, at 12:50 PM, Saikat Kanjilal  wrote:
> 
> Should we mention something around the progress over the bazel upgrade 
> initiative or is that too specific?
> Otherwise looks fine.

"* Current work in progress to support the newest version of Bazel"

> 
> Sent from my iPhone
> 
>> On Feb 4, 2019, at 12:12 PM, Ning Wang  wrote:
>> 
>> Thanks!
>> 
>>> On Mon, Feb 4, 2019 at 12:02 PM Karthik Ramasamy  wrote:
>>> 
>>> Looks good to me the podling report
>>> 
>>>> On Mon, Feb 4, 2019 at 11:06 AM Ning Wang  wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> Ok. Sounds good. Thanks!
>>>> 
>>>> @Karthik Ramasamy  Added Ali, Jerry and me to the
>>>> report. Is there any others?
>>>> 
>>>> @Sree V  We should send the notification to dev@
>>>> as well next time.
>>>> 
>>>> 
>>>> On Mon, Feb 4, 2019 at 10:14 AM Dave Fisher 
>>>> wrote:
>>>> 
>>>>> Hi -
>>>>> 
>>>>> Please post this into the wiki and we can finish discussion.
>>>>> 
>>>>> One point is that the monthly meetups are seldom discussed on the dev
>>>>> list. There should be discussion there reminding interested parties that
>>>>> these meetups are happening and what the discussion topics and talks are
>>>>> going to be.
>>>>> 
>>>>> Also the names of the committers who accepted should be provided.
>>>>> 
>>>>> Regards,
>>>>> Dave
>>>>> 
>>>>>> On Feb 4, 2019, at 10:06 AM, Ning Wang  wrote:
>>>>>> 
>>>>>> Ping for review~
>>>>>> 
>>>>>> Deadline is approaching.
>>>>>> 
>>>>>> On Thu, Jan 31, 2019 at 10:38 PM Ning Wang 
>>>>> wrote:
>>>>>> 
>>>>>>> Put the content of community development. There is no TOADD any more.
>>>>>>> 
>>>>>>> Everyone, please feel free to update if anything is missing.
>>>>>>> 
>>>>>>> On Thu, Jan 31, 2019 at 4:56 PM Josh Fischer 
>>>>> wrote:
>>>>>>> 
>>>>>>>> Thanks for the reminder..  I have attached the link to the google
>>>>> doc that
>>>>>>>> Ning put together..  Does it need anything additional?  Anything
>>>>> taken
>>>>>>>> away?  Once we are ready I can submit it for us (unless someone else
>>>>> would
>>>>>>>> like to do it).
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>> https://docs.google.com/document/d/18-fn-m87lIafnKjueKQ89l_E6GXHMSb47P_hhCtmvw0/edit
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Thu, Jan 31, 2019 at 2:43 PM Justin Mclean 
>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> The incubator PMC would appreciated if you could complete the
>>>>> podling
>>>>>>>>> report on time it's due on 6th February in a few days.
>>>>>>>>> 
>>>>>>>>> It's best if you discuss the contents of the report on the list a
>>>>> week
>>>>>>>>> before it is due and work collaboratively on it before submitting
>>>>> it.
>>>>>>>>> 
>>>>>>>>> It takes time to prepare the incubator report, have your mentors
>>>>> sign
>>>>>>>> off
>>>>>>>>> the report and for the board to review it, so it's best if you can
>>>>> get
>>>>>>>> it
>>>>>>>>> in early.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Justin
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 

Re: Incubator Podling Report (Due 6th February)

[Dave Fisher]

Hi -

Please post this into the wiki and we can finish discussion.

One point is that the monthly meetups are seldom discussed on the dev list. 
There should be discussion there reminding interested parties that these 
meetups are happening and what the discussion topics and talks are going to be.

Also the names of the committers who accepted should be provided.

Regards,
Dave

> On Feb 4, 2019, at 10:06 AM, Ning Wang  wrote:
> 
> Ping for review~
> 
> Deadline is approaching.
> 
> On Thu, Jan 31, 2019 at 10:38 PM Ning Wang  wrote:
> 
>> Put the content of community development. There is no TOADD any more.
>> 
>> Everyone, please feel free to update if anything is missing.
>> 
>> On Thu, Jan 31, 2019 at 4:56 PM Josh Fischer  wrote:
>> 
>>> Thanks for the reminder..  I have attached the link to the google doc that
>>> Ning put together..  Does it need anything additional?  Anything taken
>>> away?  Once we are ready I can submit it for us (unless someone else would
>>> like to do it).
>>> 
>>> 
>>> https://docs.google.com/document/d/18-fn-m87lIafnKjueKQ89l_E6GXHMSb47P_hhCtmvw0/edit
>>> 
>>> 
>>> On Thu, Jan 31, 2019 at 2:43 PM Justin Mclean  wrote:
>>> 
 Hi,
 
 The incubator PMC would appreciated if you could complete the podling
 report on time it's due on 6th February in a few days.
 
 It's best if you discuss the contents of the report on the list a week
 before it is due and work collaboratively on it before submitting it.
 
 It takes time to prepare the incubator report, have your mentors sign
>>> off
 the report and for the board to review it, so it's best if you can get
>>> it
 in early.
 
 Thanks,
 Justin
 
>>> 
>> 

Re: Podling Report Reminder - February 2019

[Dave Fisher]

You are missing me from the Mentor list.



Sent from my iPhone

> On Jan 22, 2019, at 8:59 PM, Ning Wang  wrote:
> 
> Let's use this gdoc to collaborate.
> 
> https://docs.google.com/document/d/18-fn-m87lIafnKjueKQ89l_E6GXHMSb47P_hhCtmvw0/edit
> 
>> On Tue, Jan 22, 2019 at 11:05 AM Josh Fischer  wrote:
>> 
>> Does anyone want to fill out and submit the incubator report?  I can help
>> along the way if any questions come up.
>> 
>> - Josh
>> 
>>> On Tue, Jan 22, 2019 at 1:02 PM  wrote:
>>> 
>>> Dear podling,
>>> 
>>> This email was sent by an automated system on behalf of the Apache
>>> Incubator PMC. It is an initial reminder to give you plenty of time to
>>> prepare your quarterly board report.
>>> 
>>> The board meeting is scheduled for Wed, 20 February 2019, 10:30 am PDT.
>>> The report for your podling will form a part of the Incubator PMC
>>> report. The Incubator PMC requires your report to be submitted 2 weeks
>>> before the board meeting, to allow sufficient time for review and
>>> submission (Wed, February 06).
>>> 
>>> Please submit your report with sufficient time to allow the Incubator
>>> PMC, and subsequently board members to review and digest. Again, the
>>> very latest you should submit your report is 2 weeks prior to the board
>>> meeting.
>>> 
>>> Candidate names should not be made public before people are actually
>>> elected, so please do not include the names of potential committers or
>>> PPMC members in your report.
>>> 
>>> Thanks,
>>> 
>>> The Apache Incubator PMC
>>> 
>>> Submitting your Report
>>> 
>>> --
>>> 
>>> Your report should contain the following:
>>> 
>>> *   Your project name
>>> *   A brief description of your project, which assumes no knowledge of
>>>the project or necessarily of its field
>>> *   A list of the three most important issues to address in the move
>>>towards graduation.
>>> *   Any issues that the Incubator PMC or ASF Board might wish/need to be
>>>aware of
>>> *   How has the community developed since the last report
>>> *   How has the project developed since the last report.
>>> *   How does the podling rate their own maturity.
>>> 
>>> This should be appended to the Incubator Wiki page at:
>>> 
>>> https://wiki.apache.org/incubator/February2019
>>> 
>>> Note: This is manually populated. You may need to wait a little before
>>> this page is created from a template.
>>> 
>>> Mentors
>>> ---
>>> 
>>> Mentors should review reports for their project(s) and sign them off on
>>> the Incubator wiki page. Signing off reports shows that you are
>>> following the project - projects that are not signed may raise alarms
>>> for the Incubator PMC.
>>> 
>>> Incubator PMC
>>> 
>> --
>> Sent from A Mobile Device
>> 

Re: Binaries

[Dave Fisher]

Hi -

This is not quite correct. Since Heron is in the incubator the repositories are 
here:

https://dist.apache.org/repos/dist/dev/incubator/heron/ 

https://dist.apache.org/repos/dist/release/incubator/heron/ 


And archived automatically here:

https://archive.apache.org/dist/incubator/heron/ 


Also - just like the source release, there needs to be a VOTE on binary 
artifacts.

Regards,
Dave

> On Dec 18, 2018, at 10:14 AM, Ning Wang  wrote:
> 
> dist.apache.org might be the repo we need to upload to.
> 
> Created a ticket for requesting a folder in the repo so that we can and
> KEYS file and upload binaries.
> https://issues.apache.org/jira/browse/INFRA-17467
> 
> Not 100% sure if this is the right process
> 
> 
> 
> On Fri, Dec 14, 2018 at 7:50 PM Ning  wrote:
> 
>> Yeah. Just I don’t know were is the repo and the instructions.
>> 
>> Sent from my iPhone
>> 
>>> On Dec 14, 2018, at 6:33 PM, Josh Fischer  wrote:
>>> 
>>> i think the releases are just marked/tagged in the Heron repo with the
>>> corresponding artifacts.
>>> 
>>> It’s think it’s the install scripts and targz files (off the top of my
>>> head).  Michael’s artifacts would work, we just need someone with write
>>> access to save them in the repo.
>>> 
 On Fri, Dec 14, 2018 at 8:24 PM Ning Wang  wrote:
 
 Yes, I think we should.
 
 I think Michael Schmidt have made some binary builds but hasn't
>> uploaded it
 yet. I did a search in the heron tickets but couldn't find the URL to
>> the
 repo.
 
> On Fri, Dec 14, 2018 at 12:36 PM Josh Fischer 
>> wrote:
> 
> Can we put out binaries for the incubating 0.20.0 release?  If so, can
> someone do this?  I can do it if I'm granted access to the repo.
> 
 
>>> --
>>> Sent from A Mobile Device
>> 

Re: Building on MacOS 10.14

[Dave Fisher]

That worked!

INFO: Elapsed time: 758.221s, Critical Path: 168.57s
INFO: 2565 processes: 2235 local, 330 worker.
INFO: Build completed successfully, 4380 total actions


> On Nov 15, 2018, at 8:11 PM, Ning Wang  wrote:
> 
> Agreed.
> 
> For bazel, I think I manually download the specific version from
> https://github.com/bazelbuild/bazel/releases and then install it. I
> remember I upgraded and downgraded a few times but not very often.
> 
> 
> 
> On Thu, Nov 15, 2018 at 2:26 PM Dave Fisher  wrote:
> 
>> I am not sure how to get brew to give me the old version. I looked and do
>> not see a way.
>> 
>> So - more careful instructions next time.
>> 
>> Regards,
>> Dave
>> 
>>> On Nov 15, 2018, at 2:13 PM, Ning Wang  wrote:
>>> 
>>> bazel is not backward compatible. :(
>>> 
>>> Plesae use 0.14.1 instead.
>>> 
>>> This information is updated in the website source files a while ago but I
>>> think it hasn't been deployed yet. It could be helpful to include this
>>> information in the voting message next time.
>>> 
>>> On Thu, Nov 15, 2018 at 11:26 AM Dave Fisher 
>> wrote:
>>> 
>>>> $ bazel version
>>>> WARNING: Processed legacy workspace file
>>>> 
>> /Users/davewave/Development/heron/incubator-heron-v-0.20.0-incubating-candidate-5/tools/bazel.rc.
>>>> This file will not be processed in the next release of Bazel. Please
>> read
>>>> https://github.com/bazelbuild/bazel/issues/6319 for further
>> information,
>>>> including how to upgrade.
>>>> Build label: 0.18.1-homebrew
>>>> Build target:
>>>> 
>> bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
>>>> Build time: Fri Nov 2 11:16:42 2018 (1541157402)
>>>> Build timestamp: 1541157402
>>>> Build timestamp as int: 1541157402
>>>> 
>>>> 
>>>>> On Nov 15, 2018, at 11:23 AM, Ning Wang  wrote:
>>>>> 
>>>>> Interesting.
>>>>> 
>>>>> I am using 10.13 for my corp laptop. Will try at home.
>>>>> 
>>>>> Which bazel version are you using? We are using 0.14.1.
>>>>> 
>>>>> 
>>>>> 
>>>>> On Thu, Nov 15, 2018 at 11:07 AM Dave Fisher 
>>>> wrote:
>>>>> 
>>>>>> Hi -
>>>>>> 
>>>>>> In testing and building the RC5 on macOS 10.14 I encountered some
>>>> issues:
>>>>>> 
>>>>>> $ bazel build --config=darwin heron/...
>>>>>> WARNING: Processed legacy workspace file
>>>>>> 
>>>> 
>> /Users/davewave/Development/heron/incubator-heron-v-0.20.0-incubating-candidate-5/tools/bazel.rc.
>>>>>> This file will not be processed in the next release of Bazel. Please
>>>> read
>>>>>> https://github.com/bazelbuild/bazel/issues/6319 for further
>>>> information,
>>>>>> including how to upgrade.
>>>>>> Starting local Bazel server and connecting to it...
>>>>>> DEBUG:
>>>>>> 
>>>> 
>> /private/var/tmp/_bazel_davewave/f2432422fcc701440a82a59536536f46/external/bazel_tools/tools/osx/xcode_configure.bzl:87:9:
>>>>>> Invoking xcodebuild failed, developer dir:
>>>>>> /Applications/Xcode8.app/Contents/Developer ,return code 256, stderr:
>>>>>> Process terminated by signal 6, stdout:
>>>>>> ERROR:
>>>>>> 
>>>> 
>> /Users/davewave/Development/heron/incubator-heron-v-0.20.0-incubating-candidate-5/heron/api/src/java/BUILD:8:1:
>>>>>> every rule of type java_doc implicitly depends upon the target
>>>>>> '@local_jdk//:jdk-default', but this target could not be found because
>>>> of:
>>>>>> no such target '@local_jdk//:jdk-default': target 'jdk-default' not
>>>>>> declared in package '' (did you mean 'jre-default'?) defined by
>>>>>> 
>>>> 
>> /private/var/tmp/_bazel_davewave/f2432422fcc701440a82a59536536f46/external/local_jdk/BUILD.bazel
>>>>>> ERROR: Analysis of target '//heron/api/src/java:heron-api-javadoc'
>>>> failed;
>>>>>> build aborted: Analysis failed
>>>>>> INFO: Elapsed time: 9.642s
>>>>>> INFO: 0 processes.
>>>>>> FAILED: Build did NOT complete successfully (287 packages loaded)
>>>>>> 
>>>>>> I have several versions of Xcode on my system and I am not sure why
>>>> Bazel
>>>>>> is choosing the older Xcode 8?
>>>>>> 
>>>>>> I did
>>>>>> $ bazel clean --expunge
>>>>>> 
>>>>>> And now I’m getting different results. Any suggestions?
>>>>>> 
>>>>>> Regards,
>>>>>> Dave:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>> 
>> 

Re: Building on MacOS 10.14

[Dave Fisher]

I am not sure how to get brew to give me the old version. I looked and do not 
see a way.

So - more careful instructions next time.

Regards,
Dave

> On Nov 15, 2018, at 2:13 PM, Ning Wang  wrote:
> 
> bazel is not backward compatible. :(
> 
> Plesae use 0.14.1 instead.
> 
> This information is updated in the website source files a while ago but I
> think it hasn't been deployed yet. It could be helpful to include this
> information in the voting message next time.
> 
> On Thu, Nov 15, 2018 at 11:26 AM Dave Fisher  wrote:
> 
>> $ bazel version
>> WARNING: Processed legacy workspace file
>> /Users/davewave/Development/heron/incubator-heron-v-0.20.0-incubating-candidate-5/tools/bazel.rc.
>> This file will not be processed in the next release of Bazel. Please read
>> https://github.com/bazelbuild/bazel/issues/6319 for further information,
>> including how to upgrade.
>> Build label: 0.18.1-homebrew
>> Build target:
>> bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
>> Build time: Fri Nov 2 11:16:42 2018 (1541157402)
>> Build timestamp: 1541157402
>> Build timestamp as int: 1541157402
>> 
>> 
>>> On Nov 15, 2018, at 11:23 AM, Ning Wang  wrote:
>>> 
>>> Interesting.
>>> 
>>> I am using 10.13 for my corp laptop. Will try at home.
>>> 
>>> Which bazel version are you using? We are using 0.14.1.
>>> 
>>> 
>>> 
>>> On Thu, Nov 15, 2018 at 11:07 AM Dave Fisher 
>> wrote:
>>> 
>>>> Hi -
>>>> 
>>>> In testing and building the RC5 on macOS 10.14 I encountered some
>> issues:
>>>> 
>>>> $ bazel build --config=darwin heron/...
>>>> WARNING: Processed legacy workspace file
>>>> 
>> /Users/davewave/Development/heron/incubator-heron-v-0.20.0-incubating-candidate-5/tools/bazel.rc.
>>>> This file will not be processed in the next release of Bazel. Please
>> read
>>>> https://github.com/bazelbuild/bazel/issues/6319 for further
>> information,
>>>> including how to upgrade.
>>>> Starting local Bazel server and connecting to it...
>>>> DEBUG:
>>>> 
>> /private/var/tmp/_bazel_davewave/f2432422fcc701440a82a59536536f46/external/bazel_tools/tools/osx/xcode_configure.bzl:87:9:
>>>> Invoking xcodebuild failed, developer dir:
>>>> /Applications/Xcode8.app/Contents/Developer ,return code 256, stderr:
>>>> Process terminated by signal 6, stdout:
>>>> ERROR:
>>>> 
>> /Users/davewave/Development/heron/incubator-heron-v-0.20.0-incubating-candidate-5/heron/api/src/java/BUILD:8:1:
>>>> every rule of type java_doc implicitly depends upon the target
>>>> '@local_jdk//:jdk-default', but this target could not be found because
>> of:
>>>> no such target '@local_jdk//:jdk-default': target 'jdk-default' not
>>>> declared in package '' (did you mean 'jre-default'?) defined by
>>>> 
>> /private/var/tmp/_bazel_davewave/f2432422fcc701440a82a59536536f46/external/local_jdk/BUILD.bazel
>>>> ERROR: Analysis of target '//heron/api/src/java:heron-api-javadoc'
>> failed;
>>>> build aborted: Analysis failed
>>>> INFO: Elapsed time: 9.642s
>>>> INFO: 0 processes.
>>>> FAILED: Build did NOT complete successfully (287 packages loaded)
>>>> 
>>>> I have several versions of Xcode on my system and I am not sure why
>> Bazel
>>>> is choosing the older Xcode 8?
>>>> 
>>>> I did
>>>> $ bazel clean --expunge
>>>> 
>>>> And now I’m getting different results. Any suggestions?
>>>> 
>>>> Regards,
>>>> Dave:
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>> 
>> 

Re: Building on MacOS 10.14

[Dave Fisher]

$ xcrun --sdk macosx --show-sdk-version
10.14


> On Nov 15, 2018, at 11:23 AM, Ali Ahmed  wrote:
> 
> What’s the output for this.
> 
> xcrun --sdk macosx --show-sdk-version
> 
> 
> 
>> On Nov 15, 2018, at 11:06 AM, Dave Fisher  wrote:
>> 
>> Hi -
>> 
>> In testing and building the RC5 on macOS 10.14 I encountered some issues:
>> 
>> $ bazel build --config=darwin heron/...
>> WARNING: Processed legacy workspace file 
>> /Users/davewave/Development/heron/incubator-heron-v-0.20.0-incubating-candidate-5/tools/bazel.rc.
>>  This file will not be processed in the next release of Bazel. Please read 
>> https://github.com/bazelbuild/bazel/issues/6319 for further information, 
>> including how to upgrade.
>> Starting local Bazel server and connecting to it...
>> DEBUG: 
>> /private/var/tmp/_bazel_davewave/f2432422fcc701440a82a59536536f46/external/bazel_tools/tools/osx/xcode_configure.bzl:87:9:
>>  Invoking xcodebuild failed, developer dir: 
>> /Applications/Xcode8.app/Contents/Developer ,return code 256, stderr: 
>> Process terminated by signal 6, stdout: 
>> ERROR: 
>> /Users/davewave/Development/heron/incubator-heron-v-0.20.0-incubating-candidate-5/heron/api/src/java/BUILD:8:1:
>>  every rule of type java_doc implicitly depends upon the target 
>> '@local_jdk//:jdk-default', but this target could not be found because of: 
>> no such target '@local_jdk//:jdk-default': target 'jdk-default' not declared 
>> in package '' (did you mean 'jre-default'?) defined by 
>> /private/var/tmp/_bazel_davewave/f2432422fcc701440a82a59536536f46/external/local_jdk/BUILD.bazel
>> ERROR: Analysis of target '//heron/api/src/java:heron-api-javadoc' failed; 
>> build aborted: Analysis failed
>> INFO: Elapsed time: 9.642s
>> INFO: 0 processes.
>> FAILED: Build did NOT complete successfully (287 packages loaded)
>> 
>> I have several versions of Xcode on my system and I am not sure why Bazel is 
>> choosing the older Xcode 8?
>> 
>> I did
>> $ bazel clean --expunge
>> 
>> And now I’m getting different results. Any suggestions?
>> 
>> Regards,
>> Dave:
>> 
>> 
>> 
>> 
>> 
>> 
> 

Building on MacOS 10.14

[Dave Fisher]

Hi -

In testing and building the RC5 on macOS 10.14 I encountered some issues:

$ bazel build --config=darwin heron/...
WARNING: Processed legacy workspace file 
/Users/davewave/Development/heron/incubator-heron-v-0.20.0-incubating-candidate-5/tools/bazel.rc.
 This file will not be processed in the next release of Bazel. Please read 
https://github.com/bazelbuild/bazel/issues/6319 for further information, 
including how to upgrade.
Starting local Bazel server and connecting to it...
DEBUG: 
/private/var/tmp/_bazel_davewave/f2432422fcc701440a82a59536536f46/external/bazel_tools/tools/osx/xcode_configure.bzl:87:9:
 Invoking xcodebuild failed, developer dir: 
/Applications/Xcode8.app/Contents/Developer ,return code 256, stderr: Process 
terminated by signal 6, stdout: 
ERROR: 
/Users/davewave/Development/heron/incubator-heron-v-0.20.0-incubating-candidate-5/heron/api/src/java/BUILD:8:1:
 every rule of type java_doc implicitly depends upon the target 
'@local_jdk//:jdk-default', but this target could not be found because of: no 
such target '@local_jdk//:jdk-default': target 'jdk-default' not declared in 
package '' (did you mean 'jre-default'?) defined by 
/private/var/tmp/_bazel_davewave/f2432422fcc701440a82a59536536f46/external/local_jdk/BUILD.bazel
ERROR: Analysis of target '//heron/api/src/java:heron-api-javadoc' failed; 
build aborted: Analysis failed
INFO: Elapsed time: 9.642s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (287 packages loaded)

I have several versions of Xcode on my system and I am not sure why Bazel is 
choosing the older Xcode 8?

I did
$ bazel clean --expunge

And now I’m getting different results. Any suggestions?

Regards,
Dave:

Re: [Mentors] Podling Report Reminder - November 2018

[Dave Fisher]

I signed off on the report.

I did add that Ning Wang was elected as a committer and accepted on 11/1/2018.

Regards,
Dave

> On Nov 5, 2018, at 5:00 PM, Josh Fischer  wrote:
> 
> The podling report has been submitted.
> 
> - Josh
> 
> On Mon, Nov 5, 2018 at 6:30 PM Ning Wang  wrote:
> 
>> SGTM. Thanks!
>> 
>> On Mon, Nov 5, 2018 at 4:12 PM Dave Fisher  wrote:
>> 
>>> No. Please go ahead and file.
>>> 
>>> Sent from my iPhone
>>> 
>>>> On Nov 5, 2018, at 4:01 PM, Ning Wang  wrote:
>>>> 
>>>> LGTM~
>>>> 
>>>> We need an approval from mentors before submission?
>>>> 
>>>>> On Mon, Nov 5, 2018 at 6:58 AM Josh Fischer 
>>> wrote:
>>>>> 
>>>>> Hi All,
>>>>> 
>>>>> Please review the updated podling report. Answers are in *bold*
>>>>> 
>>>>> 
>>>>> Heron
>>>>> 
>>>>> A real-time, distributed, fault-tolerant stream processing engine.
>>>>> 
>>>>> Heron has been incubating since 2017-06-23.
>>>>> 
>>>>> Three most important issues to address in the move towards graduation:
>>>>> 
>>>>> *1. Making the fist Apache Release.  Vote for RC5 is currently on
>>>>> general@incubator*
>>>>> *  2. Making several Releases*
>>>>> *  3. Continuing to grow the community*
>>>>> 
>>>>> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
>>>>> aware of?
>>>>> *Not at this time*
>>>>> How has the community developed since the last report?
>>>>> *The community has been growing.  Two more monthly meetups have been
>>>>> successfully organized so far after the initial one in April 2018.  We
>>> have
>>>>> also seen more interests in the Heron from different channels and
>>> getting
>>>>> more ideas from the community.  In October 2018 there was a
>> presentation
>>>>> given at BazelCon in New York city on building Apache Heron.*
>>>>> How has the project developed since the last report?
>>>>> * There have been mainly bug fixes and improvements to existing
>>> features.*
>>>>> *Some to note are*
>>>>> ** Works towards Apache release and five release candidates have been
>>>>> created.*
>>>>> ** Updates for the licenses*
>>>>> ** Updates to the documentation*
>>>>> 
>>>>> ** New integration tests*
>>>>> 
>>>>> ** New designs/works for the Streamlet API(Heron’s high level DSL)*
>>>>> 
>>>>> 
>>>>> How would you assess the podling's maturity?
>>>>> Please feel free to add your own commentary.
>>>>> 
>>>>> [ ] Initial setup
>>>>> [*x*] Working towards first release
>>>>> [*x*] Community building
>>>>> [ ] Nearing graduation
>>>>> [ ] Other:
>>>>> 
>>>>> Date of last release:
>>>>> 
>>>>> * No Apache releases as of yet. Latest RC was done on Oct 15th. [this
>>> might
>>>>> change if our Apache release succeeds]*
>>>>> 
>>>>> When were the last committers or PPMC members elected
>>>>> *  None as of yet.*
>>>>> Have your mentors been helpful and responsive or are things falling
>>>>> through the cracks? In the latter case, please list any open issues
>>>>> that need to be addressed.
>>>>> *Mentors have been very responsive and helpful.*
>>>>> 
>>>>> 
>>>>>> On Mon, Nov 5, 2018 at 1:11 AM Ning Wang 
>> wrote:
>>>>>> 
>>>>>> Thanks~ :D
>>>>>> 
>>>>>>> On Sun, Nov 4, 2018 at 5:11 AM Josh Fischer 
>>> wrote:
>>>>>>> 
>>>>>>> Thanks Justin.  I didn’t realize that the forms change.  I’ll get a
>>> new
>>>>>>> draft sent out here in a bit.
>>>>>>> 
>>>>>>> Ning,
>>>>>>> 
>>>>>>> Good answer I’ll add it in!
>>>>>>> 
>>>>>>> 
>>>>>>>> On Sun, Nov 4, 2018 at 1:25 AM Ning Wang 
>>> wrote:
>>>>>>>> 
>>>>>>>> For "Have your mentors been helpful and responsive or are things
>>>>>> falling
>>>>>>>> through the cracks? " I think the answer is:
>>>>>>>> 
>>>>>>>> Definitely helpful and responsive.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Sun, Nov 4, 2018 at 12:24 AM Ning Wang 
>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Thanks!
>>>>>>>>> 
>>>>>>>>> On Sat, Nov 3, 2018 at 3:42 PM Justin Mclean 
>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> HI,
>>>>>>>>>> 
>>>>>>>>>> JFYI the temple for what goes into a report sometime changes, so
>>>>>>> looking
>>>>>>>>>> at previous reports, modifying them, and coping and pasting into
>>>>> the
>>>>>>>> report
>>>>>>>>>> document [1] may miss some things, For instance this question has
>>>>>> been
>>>>>>>>>> added:
>>>>>>>>>> 
>>>>>>>>>> "Have your mentors been helpful and responsive or are things
>>>>> falling
>>>>>>>>>> through the cracks? In the latter case, please list any open
>>>>> issues
>>>>>>>>>> that need to be addressed."
>>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> Justin
>>>>>>>>>> 
>>>>>>>>>> 1. https://wiki.apache.org/incubator/November2018
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> --
>>>>>>> Sent from A Mobile Device
>>>>>>> 
>>>>>> 
>>>>> 
>>> 
>>> 
>> 

Re: [Mentors] Podling Report Reminder - November 2018

[Dave Fisher]

No. Please go ahead and file.

Sent from my iPhone

> On Nov 5, 2018, at 4:01 PM, Ning Wang  wrote:
> 
> LGTM~
> 
> We need an approval from mentors before submission?
> 
>> On Mon, Nov 5, 2018 at 6:58 AM Josh Fischer  wrote:
>> 
>> Hi All,
>> 
>> Please review the updated podling report. Answers are in *bold*
>> 
>> 
>> Heron
>> 
>> A real-time, distributed, fault-tolerant stream processing engine.
>> 
>> Heron has been incubating since 2017-06-23.
>> 
>> Three most important issues to address in the move towards graduation:
>> 
>>  *1. Making the fist Apache Release.  Vote for RC5 is currently on
>> general@incubator*
>> *  2. Making several Releases*
>> *  3. Continuing to grow the community*
>> 
>> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
>> aware of?
>>  *Not at this time*
>> How has the community developed since the last report?
>>  *The community has been growing.  Two more monthly meetups have been
>> successfully organized so far after the initial one in April 2018.  We have
>> also seen more interests in the Heron from different channels and getting
>> more ideas from the community.  In October 2018 there was a presentation
>> given at BazelCon in New York city on building Apache Heron.*
>> How has the project developed since the last report?
>> * There have been mainly bug fixes and improvements to existing features.*
>> *Some to note are*
>> ** Works towards Apache release and five release candidates have been
>> created.*
>> ** Updates for the licenses*
>> ** Updates to the documentation*
>> 
>> ** New integration tests*
>> 
>> ** New designs/works for the Streamlet API(Heron’s high level DSL)*
>> 
>> 
>> How would you assess the podling's maturity?
>> Please feel free to add your own commentary.
>> 
>>  [ ] Initial setup
>>  [*x*] Working towards first release
>>  [*x*] Community building
>>  [ ] Nearing graduation
>>  [ ] Other:
>> 
>> Date of last release:
>> 
>> * No Apache releases as of yet. Latest RC was done on Oct 15th. [this might
>> change if our Apache release succeeds]*
>> 
>> When were the last committers or PPMC members elected
>> *  None as of yet.*
>> Have your mentors been helpful and responsive or are things falling
>> through the cracks? In the latter case, please list any open issues
>> that need to be addressed.
>>  *Mentors have been very responsive and helpful.*
>> 
>> 
>>> On Mon, Nov 5, 2018 at 1:11 AM Ning Wang  wrote:
>>> 
>>> Thanks~ :D
>>> 
 On Sun, Nov 4, 2018 at 5:11 AM Josh Fischer  wrote:
 
 Thanks Justin.  I didn’t realize that the forms change.  I’ll get a new
 draft sent out here in a bit.
 
 Ning,
 
 Good answer I’ll add it in!
 
 
> On Sun, Nov 4, 2018 at 1:25 AM Ning Wang  wrote:
> 
> For "Have your mentors been helpful and responsive or are things
>>> falling
> through the cracks? " I think the answer is:
> 
> Definitely helpful and responsive.
> 
> 
> On Sun, Nov 4, 2018 at 12:24 AM Ning Wang 
>>> wrote:
> 
>> Thanks!
>> 
>> On Sat, Nov 3, 2018 at 3:42 PM Justin Mclean 
 wrote:
>> 
>>> HI,
>>> 
>>> JFYI the temple for what goes into a report sometime changes, so
 looking
>>> at previous reports, modifying them, and coping and pasting into
>> the
> report
>>> document [1] may miss some things, For instance this question has
>>> been
>>> added:
>>> 
>>> "Have your mentors been helpful and responsive or are things
>> falling
>>> through the cracks? In the latter case, please list any open
>> issues
>>> that need to be addressed."
>>> 
>>> Thanks,
>>> Justin
>>> 
>>> 1. https://wiki.apache.org/incubator/November2018
>>> 
>> 
> 
 --
 Sent from A Mobile Device
 
>>> 
>> 

Re: 10/22/2018 Bi-Weekly OSS Heron Sync-up

[Dave Fisher]

Hi -

This is better in that it is on the list. Some principles of The Apache Way:

1. Vendor Neutral.
2. Merit is given to individuals.

With that in mind for the next reports:

A. There should not be a “Twitter side”. There is what contributors are doing.
B. It helps to read on this mailing list a report from each individual.

Regards,
Dave

Sent from my iPhone

> On Oct 22, 2018, at 2:28 PM, Neng Lu  wrote:
> 
> Hi All,
> 
> It has been two weeks since our last sync and now is time to share our
> progress again. Let's share our works done for the last two weeks in this
> thread. And see if we need some online discussion for difficult problems.
> 
> From Twitter's side:
> 1. Ning Drafted the streamlet custom operator design doc and created
> several RPs which are under review.
> 2. Huijun worked on Caladrius system, evaluated the system and submitted a
> paper to ICDE.
> 3. Yao also worked on the Caladrius paper and recovered long-missing
> stmgr metrics.
> 4. I initiated the apache heron RC5 release voting process. And helped Ning
> with the custom operator feature.
> 
> Please feel free to add any comments if I missed anything accidentally. And
> also please share your progress or questions you have with us.
> 
> -- 
> Best Regards,
> Neng

Re: how to enable travis ci hook

[Dave Fisher]

Hi -

You can either submit a JIRA issue for INFRA at 
https://issues.apache.org/jira/projects/INFRA 


Or ask infra on HipChat at https://apache.hipchat.com/rooms/show/669587 


Regards,
Dave

> On Aug 23, 2018, at 11:36 AM, Neng Lu  wrote:
> 
> I tried to disable and enable the heron repo but found I don't have the 
> permission to do so.
> 
> Anyone has this permission?
> 
> 
> 
> 
> 
> On Tue, Aug 21, 2018 at 10:44 PM Karthik Ramasamy  > wrote:
> Not that I am aware of
> 
> On Tue, Aug 21, 2018 at 10:39 PM, FatJ Love  >
> wrote:
> 
> > Hi,
> >
> > The Travis-CI failed to trigger build on pull request. It seems the GitHub
> > hook is disabled. Travis-CI support suggested `disable and re-enable it on
> > the organization profile page`. Anybody know how to do it ?
> >
> > Best,
> > Huijun
> >



signature.asc
Description: Message signed with OpenPGP

Re: [VOTE] Heron Release 0.20.0-incubating Candidate 1

[Dave Fisher]

Hi -

Thanks for providing this. Some key items are missing. I have to VOTE -1 (While 
I am not on this PPMC I am giving you a head start with an IPMC binding vote.)

(1) A detached signature is required in the directory that has the release 
package.
(2) A KEYS file needs to be present which contains the public key of the 
release manager who signed this release.
(3) A brand new policy is that SHA-1 is compromised and new releases need an 
SHA-256 or SHA-512.

See https://www.apache.org/dev/release-distribution#sigs-and-sums 
 and 
http://www.apache.org/legal/release-policy.html#what-must-every-release-contain 


I looked at the following:
DISCLAIMER
LICENSE
NOTICE
These look OK.

Many files are missing license headers in the source.
Please provide a way to run a release audit tool to check on licenses in the 
source files.
See https://creadur.apache.org/rat/ 

Binaries cannot be included in source packages:
A discussion is needed about the third_party directory. Binary files cannot be 
included. E.G. cereal-1.2.1.tar.gz.

In the website directory there are binaries for the logo source. I wonder why 
you need ai, eps, psd and pdf versions of the logo in your source release. This 
is branding information and the project should control when these are given out.
For branding policy see http://www.apache.org/foundation/marks/#guidelines 
 , 
http://www.apache.org/foundation/marks/faq/#poweredby 
 , and 
https://www.apache.org/foundation/press/kit/ 


The icomoon font in the tools/ui has licenses that are incompatible with Apache 
releases. I would not block this release for this, but these will need to be 
replaced.

I did not do any builds ….

Regards,
Dave


> On Aug 15, 2018, at 3:14 PM, Neng Lu  wrote:
> 
> Hi All,
> 
> This is the first release candidate for Apache Heron, version
> 0.20.0-incubating.
> 
> It is the starting poiont of Heron and contains heron's main features, such
> as streaming
> processing, stateful processing, streamlet api, api server, eco support,
> etc.
> 
> Full list of changes and fixes are available:
> https://github.com/apache/incubator-heron/compare/0.17.5.1-rc...release/v-0.20.0-incubating
> 
> *** Please download, test and vote on this release. This vote will stay open
> for at least 72 hours ***
> 
> Source files:
> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.0-incubating-candidate-1/
> 
> SHA-1 checksums:
> 9a42c828f2264eb6c0e49ae52c8ba525f0e1c4ee
> ./incubator-heron-v-0.20.0-incubating-candidate-1.tar.gz
> 
> The tag to be voted upon:
> v0.20.0-incubating-candidate-1 (d2946ce0cfb3a6fe230a93d9f16550d7f46d2cf3)
> https://github.com/apache/incubator-heron/releases/tag/v-0.20.0-incubating-candidate-1
> 
> Please download the the source package, and follow the compiling guide(
> https://apache.github.io/incubator-heron/docs/developers/compiling/compiling/)
> to build
> and run the Heron locally.
> 
> Best Regards,
> Neng Lu



signature.asc
Description: Message signed with OpenPGP

Re: ML in Heron weekly meeting

[Dave Fisher]

Hi -

Has anyone reached out to the SAMOA podling? Or is their architecture inverted 
from that being proposed I’m not sure how well the SAMOA community is doing as 
they have had low activity since early this year.

Regards,
Dave

> On Jun 29, 2018, at 11:01 PM, Ning Wang  wrote:
> 
> Brief notes for the meeting on June 29:
> 
> - We need to hook up heron with Apache samoa. Saikat to create new issues
> in github.
> - Create a slack channel: #machine-learning
> - Let's add potential use cases in the design doc:
> https://docs.google.com/document/d/1LrO7XRcMxJoMM83wjRd-Ov74VAaomA_mXOAhCStgGng/edit
> 
> 
> On Sat, Jun 23, 2018 at 3:44 PM, Ning Wang  wrote:
> 
>> Brief notes for the meeting on June 22th:
>> 
>> - still studying the documents.
>>--- https://mapr.com/blog/monitoring-real-time-uber-data-using-
>> spark-machine-learning-streaming-and-kafka-api-part-2/
>>--- https://databricks.com/blog/2018/06/05/introducing-mlflow-an
>> -open-source-machine-learning-platform.html
>>--- https://eng.uber.com/michelangelo/
>> - stateful storage might need to be improved (data size) to support big
>> state object which could be required by ML jobs.
>> 



signature.asc
Description: Message signed with OpenPGP

Apache Releases

[Dave Fisher]

Hi -

Here are some resources and notes about Apache Releases:

Special rules for incubating projects.
https://incubator.apache.org/guides/releasemanagement.html 


General policies.
http://www.apache.org/dev/#releases 

Construction of LICENSE and NOTICE are important. Your Mentors can help, but 
you have to ask on list.
Source release is critical. This is OSS. The only official releases from Apache 
are completely source code.
Download page will be required where the source release is available to users.
The Apache process is very much different from how Heron is distributed now.  
https://apache.github.io/incubator-heron/docs/getting-started/ 

I recommend learning how to do an Apache Release before understanding how to 
properly do several binary distributions.

No press releases, but you can announce. Sally and the Marketing team at 
pr...@apache.org  can be used to get help with the 
rules about announcements.

About Governance: 
https://apache.github.io/incubator-heron/docs/contributors/governance/ 

Please update this page on priority as this is NOT Apache governance. Twitter’s 
CLA does not matter. Apache’s ICLA does. An ICLA is not required for 
contribution, but is for Committers and PPMC members.

Regards,
Dave


signature.asc
Description: Message signed with OpenPGP

Re: ML in Heron weekly meeting

[Dave Fisher]

Sent from my iPhone

> On Jun 8, 2018, at 5:08 PM, Ning Wang  wrote:
> 
> Brief notes for today's meeting:
> 
> - Review DD:
> https://docs.google.com/document/d/1LrO7XRcMxJoMM83wjRd-Ov74VAaomA_mXOAhCStgGng/edit

The document says copying Samoa. Heron should be working with the Samoa team 
and being careful not to fork.


> - We want to understand better about the bigger picture of ML in stream
> processing systems.
>  -- talk to ML users
>  -- doc of related systems to read:
>---
> https://mapr.com/blog/monitoring-real-time-uber-data-using-spark-machine-learning-streaming-and-kafka-api-part-2/
>---
> https://databricks.com/blog/2018/06/05/introducing-mlflow-an-open-source-machine-learning-platform.html
>--- https://eng.uber.com/michelangelo/

In addition to talking to the vendors who are powered by Apache Spark, directly 
talk to Apache Spark and Apache Kafka.

My 2cents.

Regards,
Dave

Re: Heron OSS Sync

[Dave Fisher]

Hi Herons,

Your Mentors should be available to answer these type of release questions. You 
need to ask these straight up on the dev@heron mailing list and not go to 
people within one of the companies working on related projects. The Incubator 
has additional rules beyond Apache’s Release Policy [1]

Typically you would put [MENTORS] at the beginning of the subject. Justin 
Mclean gave a talk at Apachecon NA last year. [2]

Do you all understand? If it doesn’t happen on the mailing list it didn’t 
happen is for a reason.

I also suggest that if you are going to continue these sync ups that you 
provide an agenda at least 24 hours in advance. This can actually serve as 
calls for discussion. It is important to do this. You might find that your 
questions will be answered before the “sync up” by participating in the global 
asynchronous advantage of emails!

Please think about this!

Regards,
Dave

[1] https://incubator.apache.org/guides/releasemanagement.html 

[2] https://www.youtube.com/watch?v=I0-lp1t9ee0 



> On May 22, 2018, at 11:22 PM, Ning Wang  wrote:
> 
> Regarding the Apache release. Sijie responded with the following valuable
> information:
> 
> The general guideline of release:
> http://www.apache.org/dev/release-publishing.html
> An example release process (bookkeeper):
> http://bookkeeper.apache.org/community/release_guide/
> 
> For heron release, I think you need to take care of following 3 parts:
>- For java artifacts (e.g heron api), you need to publish the java
> artifacts to apache artifactory. So you need to enable Nexus access for
> heron. File a JIRA to INFRA - example: [INFRA-14694 Enable Nexus Access For
> Pulsar - ASF JIRA](https://issues.apache.org/jira/browse/INFRA-14694)
>- You need to have a location on apache dist to host those src/binary
> packages. File a JIRA to INFA - example: [INFRA-13024 Setup dists for
> Apache DistributedLog - ASF JIRA](
> https://issues.apache.org/jira/browse/INFRA-13024)
>- website/documentation. That’s not related to releases, but you
> eventually need to put the content under http://heron.incubator.apache.org/
> , which you need to enable gitpubsub. So when you put the generated website
> content under `asf-site` branch, the website is automatically published to
> http://heron.incubator.apache.org/ . File a JIRA to INFRA - example:
> [INFRA-14587 Enable Gitpubsub for Apache Pulsar (incubating) - ASF JIRA](
> https://issues.apache.org/jira/browse/INFRA-14587)
> 
> 
> Here are the Apache Infra tickets I just created accordingly:
> Nexus access: https://issues.apache.org/jira/browse/INFRA-16560
> Dists: https://issues.apache.org/jira/browse/INFRA-16561
> gitpubsub: https://issues.apache.org/jira/browse/INFRA-16562
> 
> Please feel free to comment if I missed anything or there is any mistakes.
> 
> 
> 
> 
> 
> 
> 
> 
> On Tue, May 22, 2018 at 3:20 PM, Ning Wang  wrote:
> 
>> Brief notes:
>> 
>> 
>> 
>> 
>> 
>> * - Not much progress in the new release so far. Twitter forks will reach
>> out to Bill/Sijie for suggestions.- Productionize stateful process in
>> Twitter. Found an issue with local checkpoint expiration. - Downloader has
>> been refactored to be more flexible. - Discussed zombie aurora container
>> (causing duplicated stmgr issue).- Async ack/emit/fail PR is green to
>> merge. To document the requirements/limitations.- Traffic prediction model
>> is on going.- There are many old issues/PRs to clean up.*
>> 
>> On Tue, May 22, 2018 at 1:21 PM, Ning Wang  wrote:
>> 
>>> Hi,
>>> 
>>> The heron OSS sync meeting will be happening today at 2.00 pm PDT.
>>> Please use the following hangout link:
>>> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync
>>> ?authuser=0
>>> 
>>> 
>>> See you all then.
>>> 
>>> 
>> 



signature.asc
Description: Message signed with OpenPGP

Re: Heron github emails

[Dave Fisher]

Hi Josh,

Reply to this thread with [VOTE RESULTS] in the message subject.

You would then fill out an INFRA JIRA requesting the change. Use 
lists.apache.org to get permalinks to the first email and the results email. 
Include those in the JIRA.

Regards,
Dave


Sent from my iPhone

> On May 18, 2018, at 11:21 AM, Josh Fischer  wrote:
> 
> To All,
> 
> What needs to be done to prevent the duplicate gitbox emails sent every
> time there is activity on github?  I would be happy to complete this task.
> 
> -Josh
> 
> On Wed, May 2, 2018 at 12:53 PM, Eren Avsarogullari <
> erenavsarogull...@gmail.com> wrote:
> 
>> +1
>> 
>>> On Wed, May 2, 2018, 18:50 Neng Lu  wrote:
>>> 
>>> +1
>>> 
>>> On Tue, May 1, 2018 at 7:41 PM, Jake Farrell 
>> wrote:
>>> 
 +1
 
 -Jake
 
 On Tue, May 1, 2018 at 10:00 PM, Chris Kellogg 
 wrote:
 
> +1
> 
> On Tue, May 1, 2018 at 4:45 PM, Karthik Ramasamy >> 
> wrote:
> 
>> +1 as well.
>> 
>> On Tue, May 1, 2018 at 3:53 PM Josh Fischer 
 wrote:
>> 
>>> I’m on board with sending git commits to commits@.
>>> 
>>> +1
>>> 
>>> On Tue, May 1, 2018 at 5:20 PM Bill Graham >> 
> wrote:
>>> 
 Hi,
 
 Ever since migrating to Apache's gitbox, the dev list has been
>> dominated
>>> by
 gitbox comment emails. See
 https://lists.apache.org/list.html?d...@heron.apache.org
 
 It seems a bit heavy handed (i.e. spammy) to have every comment
> emailed
>>> to
 the dev lis, especially when git has features to let people
>>> control
>> their
 notifications. What do people think about disabling the git
 comments
> to
>>> the
 dev list? Or we could have the git comments sent to commits@
> instead.
 
 Thoughts?
 
 Bill
 
>>> --
>>> Sent from A Mobile Device
>>> 
>> 
> 
 
>>> 
>> 

Re: Heron OSS Sync

[Dave Fisher]

Thanks for the quick update!

> On May 8, 2018, at 4:08 PM, Ning Wang  wrote:
> 
> And here is a brief notes:
> 
> 
> 
> * - Our current focus is to have our first apache release by the end of
> this week (we should be pretty much ready for it)

If you mean have the first release candidate ready for voting by the end of the 
week this is achievable. What’s not achievable is completing the VOTE process 
and making the first release. You should consider this an indeterminate process 
until the community gets things correct in the Apache Way.

(1) The project will need to take at least 72 hours to review the candidate.
(2) Then the IPMC needs to VOTE on general@. That takes another minimum of 72 
hours. Often more.

To pass requires 3 +1 IPMC votes. We review the source and binaries to make 
sure that all files have license text and that the NOTICE and LICENSE is 
correct for the source and binary.

Regards,
Dave

> .- Heron webpage needs
> some update and reorg- Oracle can’t host the meet up next week.
> Rescheduling.- We need more blogs. Karthik will send out some ideas.-
> Please review Saikat’s machine learning support proposal.
> https://docs.google.com/document/d/1LrO7XRcMxJoMM83wjRd-Ov74VAaomA_mXOAhCStgGng/edit
> -
> Stateful processing is in progress. Found two issues and working on them
> (state data removal and hadoop config)- Investigating stuck stmgr issue-
> Working on the model to predict BP when traffic increases- Got a streamlet
> bug report from user about missing acks.- New ubunton 18.04 has python 3
> only. We need to migrate- A hands-on session will be scheduled by the end
> of June (sree)- Security concern in downloader/extractor java code. Sree
> and karthik to sync up.- CDCI *
> The google doc is here:
> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l90xXiY_HssVo8mE/edit?ts=5aa84932#heading=h.nq0bjo3oqwfy.
> Please feel free to reply/comment if anything is mission.
> 
> 
> 
> 
> On Tue, May 8, 2018 at 1:15 PM, Ning Wang  wrote:
> 
>> Hi,
>> 
>> The heron OSS sync meeting will be happening today at 2.00 pm PST. Please
>> use the following hangout link:
>> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-
>> sync?authuser=0
>> 
>> 
>> See you all then.
>> 
>> 



signature.asc
Description: Message signed with OpenPGP

Re: Copyright Violations

[Dave Fisher]

See https://www.apache.org/legal/src-headers.html#faq-moveothercopyright

Copyrights go in the NOTICE and not the source.

Regards,
Dave

Sent from my iPhone

> On May 1, 2018, at 6:47 PM, Karthik Ramasamy  wrote:
> 
> Taylor -
> 
> As Ali pointed out these are pending task items. One of the major task that
> we finished is converting the namespace from com.twitter.heron to
> org.apache.heron.
> 
> Sree is working on converting the copyrights to use Apache copyrights. One
> of the question he had is whether the copyright can use one line or it
> should be multi line.
> 
> It will be great if you could provide advice on this - so that we can
> change the copyright accordingly.
> 
> cheers
> /karthik
> 
> 
>> On Tue, May 1, 2018 at 5:42 PM, Ali Ahmed  wrote:
>> 
>> Hi Taylor ,
>> 
>> There are some tasks pending in this regard , the goal is to remove all
>> twitter copyright headers soon , some of the commits are in and some are
>> remaining .
>> 
>> -Ali
>> 
>>> On May 1, 2018, at 5:37 PM, P. Taylor Goetz  wrote:
>>> 
>>> Heron PPMC,
>>> 
>>> I’ve mentioned this before, but IMO, this practice needs to stop.
>>> 
>>> The community default seems to be to apply the Apache license header
>> with a twitter copyright assertion on all files, even ones copied from
>> other ALv2-licensed projects.
>>> 
>>> Twitter can’t assert copyright on code their employees didn’t create,
>> though, at least in this project, they continue to do so. I find this
>> practice unacceptable.
>>> 
>>> Please reconsider even including “Copyright Twitter $date” at all in
>> license headers. That’s more suitable for the NOTICE file, and removed from
>> the source header.
>>> 
>>> Aside from having a twitter account, I have no affiliation with twitter.
>> As a mentor I was surprised and disappointed to see a Twitter copyright
>> applied to my own (implicitly copyrighted) work. That’s not cool, nor
>> really (IANAL) legal.
>>> 
>>> I would appreciate if this could be corrected. This kind of thing is
>> something podlings need to know how to address proactively.
>>> 
>>> -Taylor
>> 
>> 

Re: Copyright Violations

[Dave Fisher]

Hi Ali,

Where are pending tasks and plans discussed and recorded so that folks can find 
out and participate.

What I am saying is you are having sync ups and not publishing minutes. You 
also announce the sync up 5 minutes beforehand. Decisions need to come to this 
mailing list.

How was the Meetup last week?

Apache projects are global which actually means that asynchronous communication 
and letting the world turn for three days is a common rule to the Apache Way.

Regards,
Dave

Sent from my iPhone

> On May 1, 2018, at 5:42 PM, Ali Ahmed  wrote:
> 
> Hi Taylor , 
> 
> There are some tasks pending in this regard , the goal is to remove all 
> twitter copyright headers soon , some of the commits are in and some are 
> remaining .
> 
> -Ali
> 
>> On May 1, 2018, at 5:37 PM, P. Taylor Goetz  wrote:
>> 
>> Heron PPMC,
>> 
>> I’ve mentioned this before, but IMO, this practice needs to stop.
>> 
>> The community default seems to be to apply the Apache license header with a 
>> twitter copyright assertion on all files, even ones copied from other 
>> ALv2-licensed projects.
>> 
>> Twitter can’t assert copyright on code their employees didn’t create, 
>> though, at least in this project, they continue to do so. I find this 
>> practice unacceptable.
>> 
>> Please reconsider even including “Copyright Twitter $date” at all in license 
>> headers. That’s more suitable for the NOTICE file, and removed from the 
>> source header.
>> 
>> Aside from having a twitter account, I have no affiliation with twitter. As 
>> a mentor I was surprised and disappointed to see a Twitter copyright applied 
>> to my own (implicitly copyrighted) work. That’s not cool, nor really (IANAL) 
>> legal.
>> 
>> I would appreciate if this could be corrected. This kind of thing is 
>> something podlings need to know how to address proactively.
>> 
>> -Taylor
> 

Re: Regarding package renaming PR#2840

[Dave Fisher]

Hi Karthik,

If the purpose of “owners” is to make this change that is fine. Otherwise 
having official owners of code is not really the Apache Way. Identifying people 
who know a portion best is fine, but the whole project and Apache “own” the 
code now for the public good.

A lot of the time projects actually remove Author tags …

Perhaps Reviewers would be better than Owners.

Regards,
Dave

> On Apr 5, 2018, at 3:02 PM, Karthik Ramasamy  wrote:
> 
> Ashvin -
> 
> It could be good to designate owners for different areas - let me come up
> with a list by the end of the today tonight.
> 
> cheers
> /karthik
> 
> 
> On Thu, Apr 5, 2018 at 11:42 AM, Ning Wang  wrote:
> 
>> Make sense to me.
>> 
>> 
>> 
>> On Thu, Apr 5, 2018 at 9:19 AM, Ashvin A  wrote:
>> 
>>> Hi Devs,
>>> 
>>> PR 2840 renames com.twitter package to org.apache. This change touches
>> more
>>> than *2,127* files. Is there a test strategy for this change which
>> updates
>>> everything? I believe just depending on unit and integration tests may be
>>> insufficient.
>>> 
>>> Also I am hoping git history will be preserved.
>>> 
>>> Should we create a coarse checklist and take ownership of manual
>>> verification of individual components?
>>> 
>>>   1. Examples
>>>   2. Heron UI
>>>  1. Metrics
>>>  2. Logs
>>>   3. API server
>>>   4. Heron client
>>>   5. Docker
>>>   6. Schedulers
>>>   1. Aurora
>>>  2. Kubernetes
>>>  3. Yarn
>>>  4. ..
>>>   7. Python
>>>   8. Heron Tracker
>>>   9. Heron metrics cache
>>>   10. Heron Health manager
>>>   11. ...
>>> 
>>> 
>>> Thanks,
>>> Ashvin
>>> 
>> 



signature.asc
Description: Message signed with OpenPGP

Re: CI job for heron

[Dave Fisher]

Hi -

> On Mar 29, 2018, at 11:37 PM, Ning Wang  wrote:
> 
> Hi, Dave,
> 
> It seems that we need to update heron CI job for unit tests and integration 
> tests. We used Travis previously. I am wondering if there is a CI engine in 
> Apache infra? Or is there any suggestion?

There is a Jenkins and buildbots based system. Subscribe to bui...@apache.org 
and ask. (Or ask Infra on Hipchat. I think I sent you that link before.)

I think that with GitHub you can use Travis as before.

Regards,
Dave

> 
> Thanks in advance.
> 

[MENTORS] Re: Heron Non Apache Release 0.17.8

[Dave Fisher]

Hi -

Adding the tag that Taylor mentioned to help signal when the project has 
questions.

While I on the IPMC I am not one of your mentors.

A few questions to answer about this non-Apache release.

(1) where will the resulting artifacts be published?

(2) How is it different from the prior non-Apache release 0.17.7?

(3) The main step for an Apache release is reviewing the dependencies and those 
licenses in order to build correct LICENSE and NOTICE files. Does someone have 
a list handy - link in the wiki?

Regards,
Dave

Sent from my iPhone

> On Mar 30, 2018, at 10:05 AM, Karthik Ramasamy  wrote:
> 
> All -
> 
> Since we have been planning to release 0.17.8 till last week, I would
> suggest that we can go ahead with this release. However this release will
> be a non Apache release since there are several task items that needs to be
> done before making an Apache release.
> 
> I would suggest the following -
> 
> * Move forward with 0.17.8 as a non Apache Release
> 
> * Target 0.18.0 as a full Apache Release
> 
> Since the task items could be long - there might be a need for some interim
> releases between 0.17.8 and 0.18.0 and these releases might be potentially
> non Apache release as well.
> 
> Let me know if this plan sounds good.
> 
> In the Slack most of committers have said yes to go ahead with 0.17.8
> release.
> 
> cheers
> /karthik
> 
> ps: Mentors please let us know if this looks ok

Re: Heron OSS Sync Meeting notes

[Dave Fisher]

Hi Ning,

If matters including that the hangout is happening were discussed on this 
mailing list then the following benefits occur.

(1) Every person interested and subscribed can know the agenda.
(2) Anyone can share any updates to the meeting url in real time.
(3) Post Meeting Notes are kept in the open.
(4) Apache projects are Open Source for the public good.

I’m here to help Heron become an Apache project.

Regards,
Dave

Sent from my iPhone

> On Mar 27, 2018, at 4:51 PM, Ning Wang <wangnin...@gmail.com> wrote:
> 
> A little more details about the hangout issue:
> 
> Owner of hangout meeting is responsible to accept incoming requests. Since
> Karthik is sick today so we had to create a new temporary meeting, which is
> not ideal.
> 
> 
> 
>> On Tue, Mar 27, 2018 at 4:09 PM, Ning Wang <wangnin...@gmail.com> wrote:
>> 
>> Thanks for the suggestions.
>> 
>> On Tue, Mar 27, 2018 at 4:02 PM, Dave Fisher <dave2w...@comcast.net>
>> wrote:
>> 
>>>  - Looking for another host for our Mailing list
>>> 
>>> 
>>> The Mailing List must be dev@heron.incubator.apache.org using any other
>>> mailing list is not the Apache Way.
>>> 
>>> 
>>>  - Hangout is killing us today! We need a better solution.
>>> 
>>> 
>>> Discuss items in plain sight of the whole community on this list. This
>>> allows other developers to participate asynchronously.
>>> 
>>> Regards,
>>> Dave
>>> 
>>> 
>>> On Mar 27, 2018, at 3:47 PM, Ning Wang <wangnin...@gmail.com> wrote:
>>> 
>>> Hi,
>>> 
>>> Here is a super brief notes for today's sync meeting:
>>> 
>>> 
>>>  - Hangout is killing us today! We need a better solution.
>>>  - Per component cpu/disk config is added but not used yet.
>>>  - Looking into failing integration tests. This has been affecting us
>>>  quite a big
>>>  - New works on nomad scheduler (docker support)
>>>  - New padding configs (to improve padding calculation and avoid hard
>>>  coded configs) are documented and shared:
>>>  https://docs.google.com/document/d/10JNZNYtcUIlsEyXcWWQUyh
>>> cSoQoJZMvYrTYT8UFo4dQ/edit
>>>  - Clean up and k8s scheduler related works ongoing
>>>  - New scala examples have been added. Next step is to document them and
>>>  integration tests
>>>  - Looking for another host for our Mailing list
>>>  - Code change ongoing to make the website apache ready. ETA next
>>>  weekend. Apache Infra setup is needed for starting the website.
>>>  - Apache transfer is on going by Twitter’s OSS team.
>>>  - We need to finalize the meetup date @maosong.
>>> 
>>> 
>>> 
>>> 
>>> Meeting notes are tracked here:
>>> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l9
>>> 0xXiY_HssVo8mE/edit?ts=5aa84932#
>>> 
>>> 
>>> 
>> 

Re: Heron OSS Sync Meeting notes

[Dave Fisher]

>   - Looking for another host for our Mailing list

The Mailing List must be dev@heron.incubator.apache.org 
 using any other mailing list is not the 
Apache Way.


>   - Hangout is killing us today! We need a better solution.

Discuss items in plain sight of the whole community on this list. This allows 
other developers to participate asynchronously.

Regards,
Dave


> On Mar 27, 2018, at 3:47 PM, Ning Wang  wrote:
> 
> Hi,
> 
> Here is a super brief notes for today's sync meeting:
> 
> 
>   - Hangout is killing us today! We need a better solution.
>   - Per component cpu/disk config is added but not used yet.
>   - Looking into failing integration tests. This has been affecting us
>   quite a big
>   - New works on nomad scheduler (docker support)
>   - New padding configs (to improve padding calculation and avoid hard
>   coded configs) are documented and shared:
>   
> https://docs.google.com/document/d/10JNZNYtcUIlsEyXcWWQUyhcSoQoJZMvYrTYT8UFo4dQ/edit
>   - Clean up and k8s scheduler related works ongoing
>   - New scala examples have been added. Next step is to document them and
>   integration tests
>   - Looking for another host for our Mailing list
>   - Code change ongoing to make the website apache ready. ETA next
>   weekend. Apache Infra setup is needed for starting the website.
>   - Apache transfer is on going by Twitter’s OSS team.
>   - We need to finalize the meetup date @maosong.
> 
> 
> 
> 
> Meeting notes are tracked here:
> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l9
> 0xXiY_HssVo8mE/edit?ts=5aa84932#



signature.asc
Description: Message signed with OpenPGP

Re: Heron OSS Sync

[Dave Fisher]

Hi Ning Wang,

You can reach out to Infra on their HipChat at 
https://www.hipchat.com/gh6DJhKrN <https://www.hipchat.com/gh6DJhKrN>

Regards,
Dave


> On Mar 20, 2018, at 2:55 PM, Fu Maosong <maoson...@gmail.com> wrote:
> 
> Hi Dave -
> 
> I added Ning Wang from Twitter with Admin permission and he has started
> working on it.
> 
> 2018-03-16 10:31 GMT-07:00 Dave Fisher <dave2w...@comcast.net>:
> 
>> Hi Maosong -
>> 
>> Infra is waiting.
>> 
>> Daniel Takamori commented on INFRA-16117:
>> -
>> 
>> [~sreevaddi] [~wave]  Ping on adding someone from infra to the Github org
>> so we can move repos into the apache organization.
>> 
>> 
>> Let us know if you need any help or advice.
>> 
>> Regards,
>> Dave
>> 
>> 
>> On Mar 13, 2018, at 3:20 PM, Ning Wang <wangnin...@gmail.com> wrote:
>> 
>> And here are the brief notes for today's meeting:
>> 
>> 
>> * - Maosong needs to contact with Apache admin about github to
>> apache/github.- After code migration is done, we are going to follow apache
>> release process- Sree is following up on the site and documentations- A few
>> bug fixes on k8s support were merged and will be included in the new
>> release.- Dhalion testing is ongoing in Microsoft- SQL API is ongoing,
>> https://docs.google.com/document/d/1PxLCyR_H-
>> mOgPjyFj3DhWXryKW21CH2zFWwzTnqjfEA/edit
>> <https://docs.google.com/document/d/1PxLCyR_H-
>> mOgPjyFj3DhWXryKW21CH2zFWwzTnqjfEA/edit>-
>> More Dhalion metrics are being added (Twitter internal feedback/requests)-
>> Data rate limiting is on going. Will be hooked up with runtime config. In
>> future some automation might be possible. It could also be related to auto
>> scaling. To document after it is stabilized.- First meetup date/time. Late
>> April, Sree is talking to some meeting groups such as
>> http://www.sfbayacm.org/ <http://www.sfbayacm.org/>- Slack/email
>> integration is ongoing (almost ready)*
>> Meeting notes are tracked here:
>> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l9
>> 0xXiY_HssVo8mE/edit?ts=5aa84932#
>> 
>> 
>> On Tue, Mar 13, 2018 at 11:00 AM, Karthik Ramasamy <kart...@streaml.io>
>> wrote:
>> 
>> Will be happening today at 2.00 pm PST. Please use the following hangout
>> link
>> 
>> https://hangouts.google.com/hangouts/_/streaml.io/oss-
>> heron-sync?authuser=0
>> 
>> See you all then.
>> 
>> cheers
>> /karthik
>> 
>> 
>> 
> 
> 
> --
> With my best Regards
> --
> Fu Maosong
> Twitter Inc.
> Mobile: +001-415-244-7520



signature.asc
Description: Message signed with OpenPGP

Re: Heron OSS Sync

[Dave Fisher]

Hi Maosong -

Infra is waiting.

Daniel Takamori commented on INFRA-16117:
-

[~sreevaddi] [~wave]  Ping on adding someone from infra to the Github org so we 
can move repos into the apache organization.

Let us know if you need any help or advice.

Regards,
Dave


> On Mar 13, 2018, at 3:20 PM, Ning Wang  wrote:
> 
> And here are the brief notes for today's meeting:
> 
> 
> * - Maosong needs to contact with Apache admin about github to
> apache/github.- After code migration is done, we are going to follow apache
> release process- Sree is following up on the site and documentations- A few
> bug fixes on k8s support were merged and will be included in the new
> release.- Dhalion testing is ongoing in Microsoft- SQL API is ongoing,
> https://docs.google.com/document/d/1PxLCyR_H-mOgPjyFj3DhWXryKW21CH2zFWwzTnqjfEA/edit
> -
> More Dhalion metrics are being added (Twitter internal feedback/requests)-
> Data rate limiting is on going. Will be hooked up with runtime config. In
> future some automation might be possible. It could also be related to auto
> scaling. To document after it is stabilized.- First meetup date/time. Late
> April, Sree is talking to some meeting groups such as
> http://www.sfbayacm.org/ - Slack/email
> integration is ongoing (almost ready)*
> Meeting notes are tracked here:
> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l90xXiY_HssVo8mE/edit?ts=5aa84932#
> 
> 
> On Tue, Mar 13, 2018 at 11:00 AM, Karthik Ramasamy 
> wrote:
> 
>> Will be happening today at 2.00 pm PST. Please use the following hangout
>> link
>> 
>> https://hangouts.google.com/hangouts/_/streaml.io/oss-
>> heron-sync?authuser=0
>> 
>> See you all then.
>> 
>> cheers
>> /karthik
>> 



signature.asc
Description: Message signed with OpenPGP

Re: Slides of exactly once/effectively once processing in Heron

[Dave Fisher]

Hi Karthick,

Nice. I like slide 37.

Regards,
Dave

> On Mar 9, 2018, at 8:22 AM, Karthik Ramasamy  wrote:
> 
> Sanjeev and I presented at Strata Data Conference on how exactly once is
> implemented in Heron San Jose. Slides are available at
> 
> https://www.slideshare.net/KarthikRamasamy3/exactly-once-effectively-once-in-apache-heron
> 
> cheers
> /karthik



signature.asc
Description: Message signed with OpenPGP

Re: github request

[Dave Fisher]

Hi -

Chris from Infra has replied. The next steps for GitHub integration are:

(1) Someone with admin rights from Twitter will need to grant one of the Infra 
sysadmins temporary access to twitter/heron to do the transfer.
Ask who from Infra will do the work and connect the two sysadmins.

Once Infra completes:

(2) Each team member will need to go to id.apache.org <http://id.apache.org/> 
and make sure that there GitHub id is entered. There are additional steps - 
here is OpenWhisk’s documentation [1] and ServiceComb’s [2]

HTH,
Dave


[1] 
https://cwiki.apache.org/confluence/display/OPENWHISK/Accessing+Apache+GitHub+as+a+Committer
 
<https://cwiki.apache.org/confluence/display/OPENWHISK/Accessing+Apache+GitHub+as+a+Committer>
[2] https://servicecomb.incubator.apache.org/developers/setup-committer-rights/ 
<https://servicecomb.incubator.apache.org/developers/setup-committer-rights/>



> On Mar 2, 2018, at 12:13 AM, Karthik Ramasamy <kart...@streaml.io> wrote:
> 
> Thanks Dave.
> 
> On Thu, Mar 1, 2018 at 9:00 PM, Sree V <sree_at_ch...@yahoo.com.invalid>
> wrote:
> 
>> Thank you very much, Dave.
>> 
>> 
>> Thanking you.
>> With Regards
>> Sree
>> 
>>On Thursday, March 1, 2018, 8:03:49 PM PST, Dave Fisher <
>> dave2w...@comcast.net> wrote:
>> 
>> Hi -
>> 
>> I have made the requested confirmations on the issue.
>> 
>> Regards,
>> Dave
>> 
>>> On Mar 1, 2018, at 6:28 PM, Sree V <sree_at_ch...@yahoo.com.INVALID>
>> wrote:
>>> 
>>> Hi Herons,
>>> Would any IPMC member validate/comment this ticket, so infra can move
>> forward, please ?https://issues.apache.org/jira/browse/INFRA-16117
>>> 
>>> 
>>> 
>>> Thanking you.
>>> With Regards
>>> Sree
>> 
>> 



signature.asc
Description: Message signed with OpenPGP

Re: github request

[Dave Fisher]

Hi -

I have made the requested confirmations on the issue.

Regards,
Dave

> On Mar 1, 2018, at 6:28 PM, Sree V  wrote:
> 
> Hi Herons,
> Would any IPMC member validate/comment this ticket, so infra can move 
> forward, please ?https://issues.apache.org/jira/browse/INFRA-16117
> 
> 
> 
> Thanking you.
> With Regards
> Sree



signature.asc
Description: Message signed with OpenPGP

Re: Moving to Apache

[Dave Fisher]

I think it is good to have a document to edit and discuss. A typical way to do 
this can also be a wiki or an issue tracker. Github offers this. The project 
can use this approach either now or after moving to Apache Gitbox. 
Alternatively, Confluence and MoinMoin wikis are available. Bugzilla and JIRA 
issue trackers are also options.

Regards,
Dave

Sent from my iPhone

> On Feb 27, 2018, at 4:37 PM, Ning Wang <wangnin...@gmail.com> wrote:
> 
> I created a google doc for us to track questions and plans:
> 
> https://docs.google.com/document/d/1-G5qbFN1ftDRf42Dee_BjlEZ75C-gVdGJ_uaGIL8qx8/edit#
> 
> 
>> On Tue, Feb 27, 2018 at 4:23 PM, Josh Fischer <j...@joshfischer.io> wrote:
>> 
>> Dave,
>> 
>> Thank you for the suggestions.  This is fantastic advice.
>> 
>> 
>>> On Tue, Feb 27, 2018 at 6:19 PM Dave Fisher <w...@apache.org> wrote:
>>> 
>>> Hi -
>>> 
>>> I will answer with two hats - (1) IPMC member and mentor to other
>>> projects, and (2) Brand committee member.
>>> 
>>> On Feb 27, 2018, at 3:20 PM, Jerry Peng <jerry.boyang.p...@gmail.com>
>>> wrote:
>>> 
>>> Hello all,
>>> 
>>> I just want to start an email thread discussing moving Heron to
>>> Apache.  There are some items we need to figure out for this:
>>> 
>>> 1. Moving the code to Apache github
>>> 
>>> I was told that an repo can be transferred to another account and
>>> people have done this in the past to move to the Apache github
>>> account.  This is the best way to move the code to be under apache
>>> since with this method heron will keep all its stars and forks.
>>> 
>>> We need to start converting heron packages from com.twitter ->
>> org.apache.
>>> 
>>> Ideally this whole process of migrating to Apache will not be a
>>> blocker for development and releases.
>>> 
>>> Thus, if mentors or people with experience in this area want to chime
>>> in on the exact details (step by step) of what needs to be do for
>>> heron to be completely migrated to Apache that would be great!
>>> 
>>> 
>>> (1) Transfer GitHub. Create an Infrastructure JIRA issue to convert the
>>> twitter repository to apache/incubator-heron/. You will need to have an
>>> admin for twitter give an ASF Infra admin rights to move.
>>> (2) Once moved then your apache-id and your GitHub id are associated
>>> through id.apache.org. You setup 2FA.
>>> (3) Once IDs and the repository are moved then you can begin.
>>> (4) Someone from Twitter in the project should make the changes.
>>> (5) As an Incubator podling you then begin making releases. You aren’t
>>> expected to get it correct the first time, but the closer you are the
>>> sooner you can graduate. The policy [1] and additional constraints for
>>> podlings. [2]
>>> 
>>> 
>>> [1] http://www.apache.org/legal/release-policy.html
>>> [2] https://incubator.apache.org/guides/releasemanagement.html
>>> 
>>> 
>>> 2. Moving website to heron.apache.org
>>> 
>>> What do we want to do here?  Migrate the whole website to
>>> heron.apache.org? And Have heron.io forward to heron.apache.org?
>>> 
>>> 
>>> Yes. And Yes. There are other branding and incubator policies for the
>>> website. [3]
>>> 
>>> [3] https://www.apache.org/foundation/marks/pmcs
>>> 
>>> For those of you using the Heron brand in your site please see [4] for
>> the
>>> policies.
>>> 
>>> [4] https://www.apache.org/foundation/marks/
>>> 
>>> For special branding rules during incubation. [5]
>>> 
>>> [5] https://incubator.apache.org/guides/branding.html
>>> 
>>> 
>>> 3.  How can we use apache infra
>>> 
>>> I think committers/mentors need to file some tickets to apache infra for
>>> this.
>>> 
>>> 
>>> A ticket for Github/GitBox is required.
>>> 
>>> 
>>> How can we use the apache infra to do apache release for heron?
>>> 
>>> 
>>> Follow the rules above and ask questions as you go.
>>> 
>>> Best Regards,
>>> Dave
>>> 
>>> 
>>> Lets get the discussion going!
>>> 
>>> Thanks!
>>> 
>>> Jerry
>>> 
>>> --
>> Sent from A Mobile Device
>> 

Re: Heron Sync Up 02/27/2017

[Dave Fisher]

Please post the notes into an email so that we can see what was discussed 
through Apache hosted resources. This is an important part of the Apache Way.

Thanks!

Regards,
Dave

Sent from my iPhone

> On Feb 27, 2018, at 4:33 PM, Ning Wang  wrote:
> 
> All -
> 
> We had the sync up meeting today and here is the brief note.
> 
> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l90xXiY_HssVo8mE/edit?ts=5a15c620#
> 
> Please feel free to comment or reply if you have any question.

Re: Moving to Apache

[Dave Fisher]

Hi -

I will answer with two hats - (1) IPMC member and mentor to other projects, and 
(2) Brand committee member.

> On Feb 27, 2018, at 3:20 PM, Jerry Peng  > wrote:
> 
> Hello all,
> 
> I just want to start an email thread discussing moving Heron to
> Apache.  There are some items we need to figure out for this:
> 
> 1. Moving the code to Apache github
> 
> I was told that an repo can be transferred to another account and
> people have done this in the past to move to the Apache github
> account.  This is the best way to move the code to be under apache
> since with this method heron will keep all its stars and forks.
> 
> We need to start converting heron packages from com.twitter -> org.apache.
> 
> Ideally this whole process of migrating to Apache will not be a
> blocker for development and releases.
> 
> Thus, if mentors or people with experience in this area want to chime
> in on the exact details (step by step) of what needs to be do for
> heron to be completely migrated to Apache that would be great!

(1) Transfer GitHub. Create an Infrastructure JIRA issue to convert the twitter 
repository to apache/incubator-heron/. You will need to have an admin for 
twitter give an ASF Infra admin rights to move.
(2) Once moved then your apache-id and your GitHub id are associated through 
id.apache.org . You setup 2FA.
(3) Once IDs and the repository are moved then you can begin.
(4) Someone from Twitter in the project should make the changes.
(5) As an Incubator podling you then begin making releases. You aren’t expected 
to get it correct the first time, but the closer you are the sooner you can 
graduate. The policy [1] and additional constraints for podlings. [2]


[1] http://www.apache.org/legal/release-policy.html 

[2] https://incubator.apache.org/guides/releasemanagement.html 


> 
> 2. Moving website to heron.apache.org 
> 
> What do we want to do here?  Migrate the whole website to
> heron.apache.org ? And Have heron.io 
>  forward to heron.apache.org ?

Yes. And Yes. There are other branding and incubator policies for the website. 
[3]

[3] https://www.apache.org/foundation/marks/pmcs 


For those of you using the Heron brand in your site please see [4] for the 
policies.

[4] https://www.apache.org/foundation/marks/ 


For special branding rules during incubation. [5]

[5] https://incubator.apache.org/guides/branding.html 



> 3.  How can we use apache infra
> 
> I think committers/mentors need to file some tickets to apache infra for this.

A ticket for Github/GitBox is required.

> 
> How can we use the apache infra to do apache release for heron?

Follow the rules above and ask questions as you go.

Best Regards,
Dave

> 
> Lets get the discussion going!
> 
> Thanks!
> 
> Jerry



signature.asc
Description: Message signed with OpenPGP

Re: Our Meetup group

[Dave Fisher]

Hi Sree,

That’s good news. It looks like something was in your email that somehow got 
mangled in the transmission.

Is there a URL to join the group?

Regards,
Dave

> On Feb 13, 2018, at 9:07 PM, Sree V  wrote:
> 
> Hi Herons,
> I am excited to share that we have a meetup group, now.Please join and stay 
> tuned for our very first meeting.
> 
> 
> Apache Heron - Bay Area (Sunnyvale, CA)
> 
> 
> |
> |
> |
> |  |  |
> 
> |
> 
> |
> |
> |  |
> Apache Heron - Bay Area (Sunnyvale, CA)
> 
> A realtime, distributed, fault-tolerant stream processing 
> engine.https://twitter.github.io/heron/http://heron.in...
> |
> 
> |
> 
> |
> 
> 
> 
> 
> Thanking you.
> With Regards
> Sree



signature.asc
Description: Message signed with OpenPGP

Re: Heron Sync Up

[Dave Fisher]

Hi -

Is it here? 
https://plus.google.com/hangouts/_/streaml.io/oss-heron-sync?hceid=a2FydGhpa0BzdHJlYW1sLmlv.218m8hgr4ekqsf1o8of2olo1a0=0
 
<https://plus.google.com/hangouts/_/streaml.io/oss-heron-sync?hceid=a2FydGhpa0BzdHJlYW1sLmlv.218m8hgr4ekqsf1o8of2olo1a0=0>

Regards,
Dave

> On Feb 13, 2018, at 12:15 PM, Karthik Ramasamy <kart...@streaml.io> wrote:
> 
> It is today
> 
> On Tue, Feb 13, 2018 at 12:08 PM Dave Fisher <dave2w...@comcast.net> wrote:
> 
>> Hi Karthick,
>> 
>> Is this today or yesterday? Can you share the URL with the list again.
>> 
>> Regards,
>> Dave
>> 
>>> On Feb 12, 2018, at 5:57 PM, Karthik Ramasamy <kart...@streaml.io>
>> wrote:
>>> 
>>> All -
>>> 
>>> Heron sync up will be happening tomorrow - 02/12 at 2.00 pm PST.
>>> 
>>> cheers
>>> /karthik
>> 
>> 



signature.asc
Description: Message signed with OpenPGP

Re: Heron Sync Up

[Dave Fisher]

Hi Karthick,

Is this today or yesterday? Can you share the URL with the list again.

Regards,
Dave

> On Feb 12, 2018, at 5:57 PM, Karthik Ramasamy  wrote:
> 
> All -
> 
> Heron sync up will be happening tomorrow - 02/12 at 2.00 pm PST.
> 
> cheers
> /karthik



signature.asc
Description: Message signed with OpenPGP

Re: Introduction & more

[Dave Fisher]

Hi -

A couple of points inline.

Sent from my iPhone

> On Jan 5, 2018, at 11:14 PM, Karthik Ramasamy  wrote:
> 
> Welcome, Sree to the project. 
> 
> We already have a slack channel at heronstreaming.slack.com and you can get 
> an invitation to the slack at
> 
> http://heronstreaming.herokuapp.com/ 
> 

Pulsar just setup a slack digest to Apache mailing list script. Matteo has the 
code. I suggest that Heron use it too.

> Regarding JIRA Apache allows new projects to be in the github.com 
>  and can use the issue tracking in GitHub.
> For example, checkout the project Apache Pulsar which uses GitHub entirely 
> for their needs - 
> 
> https://pulsar.apache.org/ 
> 
> Things that we could use your help are 
> 
> * Please update the incubator web page and there are some links that are 
> broken which needs to be fixed

If you are referring to the status page that is a mentor/PPMC page to fix.

> * com.twitter.* to org.apache.* is a good idea - we are waiting for the code 
> to transfer before making it 
> * Heron documentation could be improved a lot and could be hosted in Apache 
> incubator website

It can still be maintained in github and then published into the correct Apache 
location.

> * We have every two week OSS sync up - added you to the invite.

Is this a google hangout or face to face meeting? Just a reminder that all 
decisions do need to copied to this mailing list.

> * We could use your help in developing a lot of new features and add 
> additional tests.

Regards,
Dave

> 
> cheers
> /karthik
> 
> A few of us live in Sunnyvale as well.
> 
>> On Jan 5, 2018, at 7:46 PM, Sree V  wrote:
>> 
>> Hi Heron Team,
>> My name is Sree Vaddi.  I have been on apache.org from its birth.My previous 
>> projects were apache gossip, ranger, calcite, drill, ...
>> I am based in Sunnyvale, CA, USA.
>> From the page, http://incubator.apache.org/projects/heron.html, I have been 
>> subscribing to dev@ instead dev-subscribe@.My bad, I didn't infer 
>> (-subscribe) even though I have been on the apache from its birth.  Thank 
>> you, Sebb.
>> 
>> It looks like JIRA and other basic setup, still to do.  This is main reason, 
>> I submitted ICLA.
>> Apache conventions incubator-heron.git and incubator-heron-site.git.In the 
>> recent projects, we deviated from it and created a git branch 'site', for 
>> site pages.
>> 
>> I created slack channel.  Should I update the incubator main page ?
>> 
>> I am planning to do the following. Checking with you, before proceeding to 
>> avoid duplicate effort.Code migration for com.twitter.* to org.apache.* ?
>> Apache license replacements in each file ?
>> I did the apache gossip website. Should I go ahead and use the same template 
>> for Heron website ?
>> What is the plan for current github pages (documentation) ? Move them to 
>> apache github or move to new apache Heron website ?
>> Setup google hangout or othere video conferencing ?Setup up meetup.com and 
>> etc... ? (may be find a food & venue sponsor)
>> Open to any other in your priority list.
>> I look forward to a wonderful time working with you all.
>> 
>> Thanking you.
>> With Regards
>> Sree650.213.2707 (prefer text)
> 

Re: Heron & Flux Style Topologies

[Dave Fisher]

Hi -

Thank you both.

Regards,
Dave

Sent from my iPhone

> On Jan 6, 2018, at 6:24 AM, Josh Fischer <j...@joshfischer.io> wrote:
> 
> Karthik,
> 
> I apologize, I explained that wrong. Thanks for the follow up on the correct 
> way to move over. 
> 
> 
>> On Sat, Jan 6, 2018 at 1:22 AM Karthik Ramasamy <kart...@streaml.io> wrote:
>> Dave -
>> 
>> In order to launch the Storm topology, you need to recompile your Storm 
>> topology using Heron which is pretty straightforward.
>> 
>> cheers
>> /karthik
>> 
>> > On Jan 5, 2018, at 7:43 PM, Josh Fischer <j...@joshfischer.io> wrote:
>> >
>> > Dave, sorry I missed your last inquiry:
>> >
>> > You said "I was wondering about the YAML format to define a topology."
>> >
>> > *At this time the yaml definition should be the exact same as Storm Flux
>> > Topology.  This way you shouldn't have to make any changes to migrate to
>> > Heron with a Storm Flux topology*
>> >
>> > On Fri, Jan 5, 2018 at 9:39 PM, Josh Fischer <j...@joshfischer.io> wrote:
>> >
>> >> My responses are inlined in *blue*
>> >>
>> >> On Fri, Jan 5, 2018 at 9:08 PM, Dave Fisher <dave2w...@comcast.net> wrote:
>> >>
>> >>> Hi Josh,
>> >>>
>> >>> Sent from my iPhone
>> >>>
>> >>>> On Jan 5, 2018, at 6:30 PM, Josh Fischer <j...@joshfischer.io> wrote:
>> >>>>
>> >>>> Hi Dave,
>> >>>>
>> >>>> I am currently working on a Flux like way to deploy topologies for
>> >>> Heron.  We've decided to call it ECO.  It's based off of the already
>> >>> working code that Flux is built on.  Within the next week I plan to open 
>> >>> a
>> >>> pull request to the core committers.
>> >>>
>> >>> Please let the email list know. I’d like to take a look.
>> >>> *will do*
>> >>>>
>> >>>> So the idea is (hopefully) that you can take any Storm Flux topology
>> >>> with spouts and bolts defined in Java and run it directly in Heron 
>> >>> without
>> >>> any code changes.  So far I have this working with several topologies.
>> >>> However, I would like the community to test it more as I am sure there 
>> >>> are
>> >>> edge cases I haven't accounted for yet.
>> >>>
>> >>> Does this handle Python based bolts?
>> >>>
>> >> *I have ran a topology that uses the FluxShellBolt and FluxShellSpout,
>> >> each of them worked as expected.  If you run the Flux Topologies from the
>> >> Flux examples they should all work.  I haven't extensively tested the
>> >> multi-lang feature yet.*
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>>
>> >>> Does this mean I take a topology jar built with Maven for Storm and it
>> >>> runs fine in Heron? That would be cool.
>> >>>
>> >>
>> >>
>> >>> *Yes, you should be able to take the same jar that was build with maven
>> >>> and run it without the need to recompile or change anything.  *
>> >>
>> >>
>> >> *   An example of how submit an ECO topology is below*
>> >> heron submit local ~/path/to/jar-with-dependencies.jar \
>> >> com.twitter.heron.eco.Eco \
>> >>  --eco-config-file /path/to/yaml/simple_wordcount.yaml
>> >>
>> >>>
>> >>>>
>> >>>> As of now, this will only work with Storm components.  If you have any
>> >>> questions, please let me know!
>> >>>
>> >>> I was wondering about the YAML format to define a topology.
>> >>>
>> >>
>> >>
>> >>>
>> >>> Regards,
>> >>> Dave
>> >>>
>> >>>
>> >>>>
>> >>>> -Josh
>> >>>
>> >>>
>> >>
>> 
> 
> -- 
> Sent from A Mobile Device

Re: Release 0.17.3

[Dave Fisher]

> On Jan 5, 2018, at 1:41 PM, Karthik Ramasamy <kart...@streaml.io> wrote:
> 
> Dave -
> 
> Even though the code is in github, we are trying to follow the Apache way
> for making a release since we are officially voted to be incubator.

Then you will need to prepare packages of source and binary distributions and 
begin the start of doing the usual review and vote.

> Moving into Apache infrastructure lies with Twitter - I don't have any
> visibility into it.

Who is working with Twitter on the SGA? (You should respond on private@ if you 
wish to keep this part confidential)

Regards,
Dave

> 
> Regards
> /karthik
> 
> On Fri, Jan 5, 2018 at 12:11 PM, Dave Fisher <dave2w...@comcast.net> wrote:
> 
>> 
>>> On Jan 5, 2018, at 12:00 PM, Karthik Ramasamy <kart...@streaml.io>
>> wrote:
>>> 
>>> We are interested in making a new release 0.17.3. Let me know if you have
>>> any concerns.
>> 
>> As I understand it these releases are coming from the Twitter Github and
>> are not Apache Releases.
>> 
>> When do you think the podling will be moving into the proper Apache
>> process and Infrastructure?
>> 
>> Regards,
>> Dave
>> 



signature.asc
Description: Message signed with OpenPGP

Re: Release 0.17.3

[Dave Fisher]

> On Jan 5, 2018, at 12:00 PM, Karthik Ramasamy  wrote:
> 
> We are interested in making a new release 0.17.3. Let me know if you have
> any concerns.

As I understand it these releases are coming from the Twitter Github and are 
not Apache Releases.

When do you think the podling will be moving into the proper Apache process and 
Infrastructure?

Regards,
Dave


signature.asc
Description: Message signed with OpenPGP

Re: Heron Grant Status & Monthly Reporting

[Dave Fisher]

Hi -

Isn't the question of copyright between Twitter and the author. Isn't it likely 
that the author was working for hire which means that Twitter does have 
copyright? If there is a question here then isn't this a question for 
legal-discuss@?

Regards,
Dave

Sent from my iPhone

> On Sep 15, 2017, at 9:28 AM, John D. Ament  wrote:
> 
> On Fri, Sep 15, 2017 at 11:25 AM Bertrand Delacretaz <
> bdelacre...@codeconsult.ch> wrote:
> 
>> On Fri, Sep 15, 2017 at 3:09 AM, John D. Ament 
>> wrote:
>>> ...I'd like to understand why Heron is inclined to do a SGA rather
>>> than ICLAs
>> 
>> IANAL but my understanding is that the software is donated by its
>> copyright owner.
>> 
>> Java files found under https://github.com/twitter/heron have a
>> "Copyright 2017 Twitter" header.
>> 
>> My conclusion is that a software grant needs to be provided by Twitter.
>> 
> 
> The key word here is "donated."  Right now there are licensing issues
> within the Heron code base that need to be resolved.  However, all of the
> originally developed code is Apache License, v2.  Twitter doesn't actually
> own the copyright on all of the code (see [1] as an example), because of
> that I'm not sure a grant would actually be valid at this time.  When
> they're ready to remove the twitter headers in favor of the ASF standard
> headers that would be when I expect a grant to be processed.
> 
> 
> [1]:
> https://lists.apache.org/thread.html/64bbb16137225217a5626108801a4fc2ed940382000204f8633aa378@%3Cdev.heron.apache.org%3E
> 
> 
> 
>> 
>> -Bertrand
>> 
>> -
>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
>> For additional commands, e-mail: general-h...@incubator.apache.org
>> 
>>