Re: Heron OSS Sync Meeting

[Ning Wang]

No objection so far.

Here are my updates:

- Discussed with Saikat about Machine Learning with Heron. We are planning
to create a new target in Apache Samoa project similar to the existing
Storm support.
- Working on user Bolt/Spout support in Streamlet API. I am stepping back a
bit and reconsider user Bolt as a Custom Operator way and see if this can
make the support more straightforward. I would like to keep the type safety
feature but it seems to be tricky.





On Mon, Sep 24, 2018 at 2:37 PM Neng Lu  wrote:

> +1 for this idea. We should utilize more of the mailing list and use it as
> the main discussion place.
>
> The sync meet should only play as a supplementary role.
>
> On Mon, Sep 24, 2018 at 2:12 PM Ning Wang  wrote:
>
> > Hi,
> >
> >
> > The heron OSS sync meeting will be happening tomorrow at 2.00 pm PDT.
> > Please use the following hangout link:
> >
> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync?authuser=0
> >
> > There has been suggestions about communicating regular biweekly updates
> via
> > this mailing list. It could be easier for some of us to arrange time, and
> > it might be easier for discussions.
> >
> > I am thinking how about we send updates in this email thread by the end
> of
> > tomorrow and leave the sync meeting to discussions if there is any? What
> do
> > you think?
> >
>
>
> --
> Best Regards,
> Neng
>

Re: Heron OSS Sync Meeting

[Neng Lu]

+1 for this idea. We should utilize more of the mailing list and use it as
the main discussion place.

The sync meet should only play as a supplementary role.

On Mon, Sep 24, 2018 at 2:12 PM Ning Wang  wrote:

> Hi,
>
>
> The heron OSS sync meeting will be happening tomorrow at 2.00 pm PDT.
> Please use the following hangout link:
> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync?authuser=0
>
> There has been suggestions about communicating regular biweekly updates via
> this mailing list. It could be easier for some of us to arrange time, and
> it might be easier for discussions.
>
> I am thinking how about we send updates in this email thread by the end of
> tomorrow and leave the sync meeting to discussions if there is any? What do
> you think?
>


-- 
Best Regards,
Neng

Re: Heron OSS Sync Meeting

[Josh Fischer]

+1

On Mon, Sep 10, 2018 at 12:20 PM Ning Wang  wrote:

> One thing I think we may need to discuss tomorrow is to find a replacement
> for Google hangout. We have seen some issues(cant accept requests) in the
> past a few weeks.
>
>
>
> On Mon, Sep 10, 2018 at 10:02 AM Ning Wang  wrote:
>
> > Hi,
> >
> > The heron OSS sync meeting will be happening tomorrow at 2.00 pm PDT.
> > Please use the following hangout link:
> > https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync
> > ?authuser=0
> >
> >
> > See you all then.
> >
>
-- 
Sent from A Mobile Device

Re: Heron OSS Sync Meeting

[Ning Wang]

One thing I think we may need to discuss tomorrow is to find a replacement
for Google hangout. We have seen some issues(cant accept requests) in the
past a few weeks.



On Mon, Sep 10, 2018 at 10:02 AM Ning Wang  wrote:

> Hi,
>
> The heron OSS sync meeting will be happening tomorrow at 2.00 pm PDT.
> Please use the following hangout link:
> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync
> ?authuser=0
>
>
> See you all then.
>

Re: Heron OSS Sync

[Ning Wang]

Brief notes for today's sync up meeting:


   - Apache release is in progress (Neng & Ning). Licenses for 3rdparty
   libraries have been created and heron folders have been created in apache
   dist.
   - K8s deployment improvement is on going (Karthik).
   - A Samoa integration branch is created for devs to discuss and work on
   it (Saikat).
   - Example topology running on Normad uses a lot of resources which is
   not good for new users. Jerry taking a look at topology and configs.
   - Room is found (Palo Alto) for monthly meetups (Sree).


On Tue, Aug 14, 2018 at 11:51 AM, Josh Fischer  wrote:

> I will be on for the first 30 minutes.
>
> Josh
>
> On Tue, Aug 14, 2018 at 1:50 PM Jerry Peng 
> wrote:
>
> > Sounds good.  I will attend the OSS sync meeting today
> >
> > On Tue, Aug 14, 2018 at 11:22 AM Ning Wang  wrote:
> >
> > > Hi,
> > >
> > > Sorry for the late notice The heron OSS sync meeting will be
> > happening
> > > today at 2.00 pm PDT. Please use the following hangout link:
> > >
> > https://hangouts.google.com/hangouts/_/streaml.io/oss-
> heron-sync?authuser=0
> > >
> > >
> > > See you all then.
> > >
> >
> --
> Sent from A Mobile Device
>

Re: Heron OSS Sync

[Josh Fischer]

I will be on for the first 30 minutes.

Josh

On Tue, Aug 14, 2018 at 1:50 PM Jerry Peng 
wrote:

> Sounds good.  I will attend the OSS sync meeting today
>
> On Tue, Aug 14, 2018 at 11:22 AM Ning Wang  wrote:
>
> > Hi,
> >
> > Sorry for the late notice The heron OSS sync meeting will be
> happening
> > today at 2.00 pm PDT. Please use the following hangout link:
> >
> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync?authuser=0
> >
> >
> > See you all then.
> >
>
-- 
Sent from A Mobile Device

Re: Heron OSS Sync

[Jerry Peng]

Sounds good.  I will attend the OSS sync meeting today

On Tue, Aug 14, 2018 at 11:22 AM Ning Wang  wrote:

> Hi,
>
> Sorry for the late notice The heron OSS sync meeting will be happening
> today at 2.00 pm PDT. Please use the following hangout link:
> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync?authuser=0
>
>
> See you all then.
>

Re: Heron OSS Sync

[Ning Wang]

Today's brief notes:

- We are testing DSL API in Twitter. Jerry mentioned some potential
improvement on builder.
- Found a Dhalion issue (2927).
- Shipped new integration tests for topology structures
- The current model in the queuing model (M/M/1) might not be useful.
Trying new ones.
- Added support for transferring stateful data via disk for big stateful
data.
- CkptMgr memory size is hardcoded currently, might add a new config for it.
- Currently windowing process waits for all data first and it might be
improved.
- Heron Lite is under considerations.





On Tue, Jul 3, 2018 at 9:42 AM, Ning Wang  wrote:

> Hi,
>
> Sorry for the late notice The heron OSS sync meeting will be
> happening today at 2.00 pm PDT. Please use the following hangout link:
> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync
> ?authuser=0
>
>
> See you all then.
>
>

Re: Heron OSS Sync

[Karthik Ramasamy]

Definitely - we will discuss those items in the dev mailing list.

cheers
/karthik

On Tue, Jun 19, 2018 at 4:23 PM, P. Taylor Goetz  wrote:

> I would hope that any follow-up discussions happen on-list.
>
> Is that the expectation?
>
> -Taylor
>
> > On Jun 19, 2018, at 6:53 PM, Ning Wang  wrote:
> >
> > Brief notes for today's sync up meeting:
> >
> >
> >   - Apache release is still in progress. Jerry, Ali and Ning will
> sync
> >   up about the previous release process first.
> >   - Summingbird team reviewed streamlet API. Will discuss more about
> >   the window operation and stateful storage.
> >   - Huijun adding toggleable switch into Dhalion.
> >   - Yao working on new integration test for topology structure.
> >   - Fario started to take the traffic modeling project from Thomas.
> >
> >
> >
> >> On Mon, Jun 18, 2018 at 8:58 PM, Ning Wang 
> wrote:
> >>
> >> Thanks for asking.
> >>
> >> Normally we talk about these two items in the meeting:
> >> - Updates
> >> - There could be something we want to discuss briefly or schedule
> further
> >> discussions.
> >>
> >>
> >>
> >> On Mon, Jun 18, 2018 at 5:00 PM, Dave Fisher 
> >> wrote:
> >>
> >>> Hi -
> >>>
> >>> Is there an agenda?
> >>>
> >>> Regards,
> >>> Dave
> >>>
> >>> Sent from my iPhone
> >>>
>  On Jun 18, 2018, at 4:56 PM, Ning Wang  wrote:
> 
>  Hi,
> 
>  The heron OSS sync meeting will be happening tomorrow at 2.00 pm PDT.
>  Please use the following hangout link:
>  https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-
> >>> sync?authuser=0
> 
> 
>  See you all then.
> >>>
> >>
> >>
>

Re: Heron OSS Sync

[P. Taylor Goetz]

I would hope that any follow-up discussions happen on-list.

Is that the expectation?

-Taylor

> On Jun 19, 2018, at 6:53 PM, Ning Wang  wrote:
> 
> Brief notes for today's sync up meeting:
> 
> 
>   - Apache release is still in progress. Jerry, Ali and Ning will sync
>   up about the previous release process first.
>   - Summingbird team reviewed streamlet API. Will discuss more about
>   the window operation and stateful storage.
>   - Huijun adding toggleable switch into Dhalion.
>   - Yao working on new integration test for topology structure.
>   - Fario started to take the traffic modeling project from Thomas.
> 
> 
> 
>> On Mon, Jun 18, 2018 at 8:58 PM, Ning Wang  wrote:
>> 
>> Thanks for asking.
>> 
>> Normally we talk about these two items in the meeting:
>> - Updates
>> - There could be something we want to discuss briefly or schedule further
>> discussions.
>> 
>> 
>> 
>> On Mon, Jun 18, 2018 at 5:00 PM, Dave Fisher 
>> wrote:
>> 
>>> Hi -
>>> 
>>> Is there an agenda?
>>> 
>>> Regards,
>>> Dave
>>> 
>>> Sent from my iPhone
>>> 
 On Jun 18, 2018, at 4:56 PM, Ning Wang  wrote:
 
 Hi,
 
 The heron OSS sync meeting will be happening tomorrow at 2.00 pm PDT.
 Please use the following hangout link:
 https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-
>>> sync?authuser=0
 
 
 See you all then.
>>> 
>> 
>> 

Re: Heron OSS Sync

[Ning Wang]

Brief notes for today's sync up meeting:


   - Apache release is still in progress. Jerry, Ali and Ning will sync
   up about the previous release process first.
   - Summingbird team reviewed streamlet API. Will discuss more about
   the window operation and stateful storage.
   - Huijun adding toggleable switch into Dhalion.
   - Yao working on new integration test for topology structure.
   - Fario started to take the traffic modeling project from Thomas.



On Mon, Jun 18, 2018 at 8:58 PM, Ning Wang  wrote:

> Thanks for asking.
>
> Normally we talk about these two items in the meeting:
> - Updates
> - There could be something we want to discuss briefly or schedule further
> discussions.
>
>
>
> On Mon, Jun 18, 2018 at 5:00 PM, Dave Fisher 
> wrote:
>
>> Hi -
>>
>> Is there an agenda?
>>
>> Regards,
>> Dave
>>
>> Sent from my iPhone
>>
>> > On Jun 18, 2018, at 4:56 PM, Ning Wang  wrote:
>> >
>> > Hi,
>> >
>> > The heron OSS sync meeting will be happening tomorrow at 2.00 pm PDT.
>> > Please use the following hangout link:
>> > https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-
>> sync?authuser=0
>> >
>> >
>> > See you all then.
>>
>
>

Re: Heron OSS Sync

[Ning Wang]

Thanks for asking.

Normally we talk about these two items in the meeting:
- Updates
- There could be something we want to discuss briefly or schedule further
discussions.



On Mon, Jun 18, 2018 at 5:00 PM, Dave Fisher  wrote:

> Hi -
>
> Is there an agenda?
>
> Regards,
> Dave
>
> Sent from my iPhone
>
> > On Jun 18, 2018, at 4:56 PM, Ning Wang  wrote:
> >
> > Hi,
> >
> > The heron OSS sync meeting will be happening tomorrow at 2.00 pm PDT.
> > Please use the following hangout link:
> > https://hangouts.google.com/hangouts/_/streaml.io/oss-
> heron-sync?authuser=0
> >
> >
> > See you all then.
>

Re: Heron OSS Sync

[Ning Wang]

Thanks! You are right!!! Let me send it again.



On Mon, Jun 18, 2018 at 4:55 PM, Jerry Peng 
wrote:

> Ning,
>
> I think you mean tomorrow correct?
>
> On Mon, Jun 18, 2018 at 4:54 PM Ning Wang  wrote:
>
> > Hi,
> >
> > The heron OSS sync meeting will be happening today at 2.00 pm PDT. Please
> > use the following hangout link:
> > https://hangouts.google.com/hangouts/_/streaml.io/oss-
> heron-sync?authuser=0
> >
> >
> > See you all then.
> >
>

Re: Heron OSS Sync

[Ning Wang]

Brief notes for today's meeting:


   - We made some progress on the infra side of heron Apache release. Ning
   will put some time on it this week.
   - Thomas is wrapping up the traffic modeling project and aiming at
   releasing the code. It can do short-time pressure prediction now.
   - Last update would be a useful feature to add.
   - It could be useful for us to do a Heron/Flink benchmark and comparison.
   - windowing depends on user's event time extraction currently. We might
   consider making event time a first class citizen.
   - Jerry working on Nomad scheduler
   - Users asked about Streamlet interface and acking mechanism.



On Mon, Jun 4, 2018 at 5:05 PM, Ning Wang  wrote:

> Hi,
>
> The heron OSS sync meeting will be happening tomorrow at 2.00 pm PDT.
> Please use the following hangout link:
> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync
> ?authuser=0
>
>
> See you all then.
>
>

Re: Heron OSS Sync

[Ning Wang]

Also, thanks a lot for the information Dave! That is very helpful.

We were a bit lost and we were not sure what to ask (the related
information is quite overwhelming and a bit sparse) so we asked for some
info from guys that has the previous experience. Sorry if it is not the
recommended way. We will try to follow the standard procedure.

On Mon, Jun 4, 2018 at 1:31 PM, Dave Fisher  wrote:

> Hi Herons,
>
> Your Mentors should be available to answer these type of release
> questions. You need to ask these straight up on the dev@heron mailing
> list and not go to people within one of the companies working on related
> projects. The Incubator has additional rules beyond Apache’s Release Policy
> [1]
>
> Typically you would put [MENTORS] at the beginning of the subject. Justin
> Mclean gave a talk at Apachecon NA last year. [2]
>
> Do you all understand? If it doesn’t happen on the mailing list it didn’t
> happen is for a reason.
>
> I also suggest that if you are going to continue these sync ups that you
> provide an agenda at least 24 hours in advance. This can actually serve as
> calls for discussion. It is important to do this. You might find that your
> questions will be answered before the “sync up” by participating in the
> global asynchronous advantage of emails!
>
> Please think about this!
>
> Regards,
> Dave
>
> [1] https://incubator.apache.org/guides/releasemanagement.html
> [2] https://www.youtube.com/watch?v=I0-lp1t9ee0
>
>
> On May 22, 2018, at 11:22 PM, Ning Wang  wrote:
>
> Regarding the Apache release. Sijie responded with the following valuable
> information:
>
> The general guideline of release:
> http://www.apache.org/dev/release-publishing.html
> An example release process (bookkeeper):
> http://bookkeeper.apache.org/community/release_guide/
>
> For heron release, I think you need to take care of following 3 parts:
>- For java artifacts (e.g heron api), you need to publish the java
> artifacts to apache artifactory. So you need to enable Nexus access for
> heron. File a JIRA to INFRA - example: [INFRA-14694 Enable Nexus Access For
> Pulsar - ASF JIRA](https://issues.apache.org/jira/browse/INFRA-14694)
>- You need to have a location on apache dist to host those src/binary
> packages. File a JIRA to INFA - example: [INFRA-13024 Setup dists for
> Apache DistributedLog - ASF JIRA](
> https://issues.apache.org/jira/browse/INFRA-13024)
>- website/documentation. That’s not related to releases, but you
> eventually need to put the content under http://heron.incubator.apache.
> org/
> , which you need to enable gitpubsub. So when you put the generated website
> content under `asf-site` branch, the website is automatically published to
> http://heron.incubator.apache.org/ . File a JIRA to INFRA - example:
> [INFRA-14587 Enable Gitpubsub for Apache Pulsar (incubating) - ASF JIRA](
> https://issues.apache.org/jira/browse/INFRA-14587)
>
>
> Here are the Apache Infra tickets I just created accordingly:
> Nexus access: https://issues.apache.org/jira/browse/INFRA-16560
> Dists: https://issues.apache.org/jira/browse/INFRA-16561
> gitpubsub: https://issues.apache.org/jira/browse/INFRA-16562
>
> Please feel free to comment if I missed anything or there is any mistakes.
>
>
>
>
>
>
>
>
> On Tue, May 22, 2018 at 3:20 PM, Ning Wang  wrote:
>
> Brief notes:
>
>
>
>
>
> * - Not much progress in the new release so far. Twitter forks will reach
> out to Bill/Sijie for suggestions.- Productionize stateful process in
> Twitter. Found an issue with local checkpoint expiration. - Downloader has
> been refactored to be more flexible. - Discussed zombie aurora container
> (causing duplicated stmgr issue).- Async ack/emit/fail PR is green to
> merge. To document the requirements/limitations.- Traffic prediction model
> is on going.- There are many old issues/PRs to clean up.*
>
> On Tue, May 22, 2018 at 1:21 PM, Ning Wang  wrote:
>
> Hi,
>
> The heron OSS sync meeting will be happening today at 2.00 pm PDT.
> Please use the following hangout link:
> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync
> ?authuser=0
>
>
> See you all then.
>
>
>
>
>

Re: Heron OSS Sync

[Ning Wang]

Thanks!

On Mon, Jun 4, 2018 at 1:31 PM, Dave Fisher  wrote:

> Hi Herons,
>
> Your Mentors should be available to answer these type of release
> questions. You need to ask these straight up on the dev@heron mailing
> list and not go to people within one of the companies working on related
> projects. The Incubator has additional rules beyond Apache’s Release Policy
> [1]
>
> Typically you would put [MENTORS] at the beginning of the subject. Justin
> Mclean gave a talk at Apachecon NA last year. [2]
>
> Do you all understand? If it doesn’t happen on the mailing list it didn’t
> happen is for a reason.
>
> I also suggest that if you are going to continue these sync ups that you
> provide an agenda at least 24 hours in advance. This can actually serve as
> calls for discussion. It is important to do this. You might find that your
> questions will be answered before the “sync up” by participating in the
> global asynchronous advantage of emails!
>
> Please think about this!
>
> Regards,
> Dave
>
> [1] https://incubator.apache.org/guides/releasemanagement.html
> [2] https://www.youtube.com/watch?v=I0-lp1t9ee0
>
>
> On May 22, 2018, at 11:22 PM, Ning Wang  wrote:
>
> Regarding the Apache release. Sijie responded with the following valuable
> information:
>
> The general guideline of release:
> http://www.apache.org/dev/release-publishing.html
> An example release process (bookkeeper):
> http://bookkeeper.apache.org/community/release_guide/
>
> For heron release, I think you need to take care of following 3 parts:
>- For java artifacts (e.g heron api), you need to publish the java
> artifacts to apache artifactory. So you need to enable Nexus access for
> heron. File a JIRA to INFRA - example: [INFRA-14694 Enable Nexus Access For
> Pulsar - ASF JIRA](https://issues.apache.org/jira/browse/INFRA-14694)
>- You need to have a location on apache dist to host those src/binary
> packages. File a JIRA to INFA - example: [INFRA-13024 Setup dists for
> Apache DistributedLog - ASF JIRA](
> https://issues.apache.org/jira/browse/INFRA-13024)
>- website/documentation. That’s not related to releases, but you
> eventually need to put the content under http://heron.incubator.apache.
> org/
> , which you need to enable gitpubsub. So when you put the generated website
> content under `asf-site` branch, the website is automatically published to
> http://heron.incubator.apache.org/ . File a JIRA to INFRA - example:
> [INFRA-14587 Enable Gitpubsub for Apache Pulsar (incubating) - ASF JIRA](
> https://issues.apache.org/jira/browse/INFRA-14587)
>
>
> Here are the Apache Infra tickets I just created accordingly:
> Nexus access: https://issues.apache.org/jira/browse/INFRA-16560
> Dists: https://issues.apache.org/jira/browse/INFRA-16561
> gitpubsub: https://issues.apache.org/jira/browse/INFRA-16562
>
> Please feel free to comment if I missed anything or there is any mistakes.
>
>
>
>
>
>
>
>
> On Tue, May 22, 2018 at 3:20 PM, Ning Wang  wrote:
>
> Brief notes:
>
>
>
>
>
> * - Not much progress in the new release so far. Twitter forks will reach
> out to Bill/Sijie for suggestions.- Productionize stateful process in
> Twitter. Found an issue with local checkpoint expiration. - Downloader has
> been refactored to be more flexible. - Discussed zombie aurora container
> (causing duplicated stmgr issue).- Async ack/emit/fail PR is green to
> merge. To document the requirements/limitations.- Traffic prediction model
> is on going.- There are many old issues/PRs to clean up.*
>
> On Tue, May 22, 2018 at 1:21 PM, Ning Wang  wrote:
>
> Hi,
>
> The heron OSS sync meeting will be happening today at 2.00 pm PDT.
> Please use the following hangout link:
> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync
> ?authuser=0
>
>
> See you all then.
>
>
>
>
>

Re: Heron OSS Sync

[Dave Fisher]

Hi Herons,

Your Mentors should be available to answer these type of release questions. You 
need to ask these straight up on the dev@heron mailing list and not go to 
people within one of the companies working on related projects. The Incubator 
has additional rules beyond Apache’s Release Policy [1]

Typically you would put [MENTORS] at the beginning of the subject. Justin 
Mclean gave a talk at Apachecon NA last year. [2]

Do you all understand? If it doesn’t happen on the mailing list it didn’t 
happen is for a reason.

I also suggest that if you are going to continue these sync ups that you 
provide an agenda at least 24 hours in advance. This can actually serve as 
calls for discussion. It is important to do this. You might find that your 
questions will be answered before the “sync up” by participating in the global 
asynchronous advantage of emails!

Please think about this!

Regards,
Dave

[1] https://incubator.apache.org/guides/releasemanagement.html 

[2] https://www.youtube.com/watch?v=I0-lp1t9ee0 



> On May 22, 2018, at 11:22 PM, Ning Wang  wrote:
> 
> Regarding the Apache release. Sijie responded with the following valuable
> information:
> 
> The general guideline of release:
> http://www.apache.org/dev/release-publishing.html
> An example release process (bookkeeper):
> http://bookkeeper.apache.org/community/release_guide/
> 
> For heron release, I think you need to take care of following 3 parts:
>- For java artifacts (e.g heron api), you need to publish the java
> artifacts to apache artifactory. So you need to enable Nexus access for
> heron. File a JIRA to INFRA - example: [INFRA-14694 Enable Nexus Access For
> Pulsar - ASF JIRA](https://issues.apache.org/jira/browse/INFRA-14694)
>- You need to have a location on apache dist to host those src/binary
> packages. File a JIRA to INFA - example: [INFRA-13024 Setup dists for
> Apache DistributedLog - ASF JIRA](
> https://issues.apache.org/jira/browse/INFRA-13024)
>- website/documentation. That’s not related to releases, but you
> eventually need to put the content under http://heron.incubator.apache.org/
> , which you need to enable gitpubsub. So when you put the generated website
> content under `asf-site` branch, the website is automatically published to
> http://heron.incubator.apache.org/ . File a JIRA to INFRA - example:
> [INFRA-14587 Enable Gitpubsub for Apache Pulsar (incubating) - ASF JIRA](
> https://issues.apache.org/jira/browse/INFRA-14587)
> 
> 
> Here are the Apache Infra tickets I just created accordingly:
> Nexus access: https://issues.apache.org/jira/browse/INFRA-16560
> Dists: https://issues.apache.org/jira/browse/INFRA-16561
> gitpubsub: https://issues.apache.org/jira/browse/INFRA-16562
> 
> Please feel free to comment if I missed anything or there is any mistakes.
> 
> 
> 
> 
> 
> 
> 
> 
> On Tue, May 22, 2018 at 3:20 PM, Ning Wang  wrote:
> 
>> Brief notes:
>> 
>> 
>> 
>> 
>> 
>> * - Not much progress in the new release so far. Twitter forks will reach
>> out to Bill/Sijie for suggestions.- Productionize stateful process in
>> Twitter. Found an issue with local checkpoint expiration. - Downloader has
>> been refactored to be more flexible. - Discussed zombie aurora container
>> (causing duplicated stmgr issue).- Async ack/emit/fail PR is green to
>> merge. To document the requirements/limitations.- Traffic prediction model
>> is on going.- There are many old issues/PRs to clean up.*
>> 
>> On Tue, May 22, 2018 at 1:21 PM, Ning Wang  wrote:
>> 
>>> Hi,
>>> 
>>> The heron OSS sync meeting will be happening today at 2.00 pm PDT.
>>> Please use the following hangout link:
>>> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync
>>> ?authuser=0
>>> 
>>> 
>>> See you all then.
>>> 
>>> 
>> 



signature.asc
Description: Message signed with OpenPGP

Re: Heron OSS Sync

[Ning Wang]

Regarding the Apache release. Sijie responded with the following valuable
information:

The general guideline of release:
http://www.apache.org/dev/release-publishing.html
An example release process (bookkeeper):
http://bookkeeper.apache.org/community/release_guide/

For heron release, I think you need to take care of following 3 parts:
- For java artifacts (e.g heron api), you need to publish the java
artifacts to apache artifactory. So you need to enable Nexus access for
heron. File a JIRA to INFRA - example: [INFRA-14694 Enable Nexus Access For
Pulsar - ASF JIRA](https://issues.apache.org/jira/browse/INFRA-14694)
- You need to have a location on apache dist to host those src/binary
packages. File a JIRA to INFA - example: [INFRA-13024 Setup dists for
Apache DistributedLog - ASF JIRA](
https://issues.apache.org/jira/browse/INFRA-13024)
- website/documentation. That’s not related to releases, but you
eventually need to put the content under http://heron.incubator.apache.org/
, which you need to enable gitpubsub. So when you put the generated website
content under `asf-site` branch, the website is automatically published to
http://heron.incubator.apache.org/ . File a JIRA to INFRA - example:
[INFRA-14587 Enable Gitpubsub for Apache Pulsar (incubating) - ASF JIRA](
https://issues.apache.org/jira/browse/INFRA-14587)


Here are the Apache Infra tickets I just created accordingly:
Nexus access: https://issues.apache.org/jira/browse/INFRA-16560
Dists: https://issues.apache.org/jira/browse/INFRA-16561
gitpubsub: https://issues.apache.org/jira/browse/INFRA-16562

Please feel free to comment if I missed anything or there is any mistakes.








On Tue, May 22, 2018 at 3:20 PM, Ning Wang  wrote:

> Brief notes:
>
>
>
>
>
> * - Not much progress in the new release so far. Twitter forks will reach
> out to Bill/Sijie for suggestions.- Productionize stateful process in
> Twitter. Found an issue with local checkpoint expiration. - Downloader has
> been refactored to be more flexible. - Discussed zombie aurora container
> (causing duplicated stmgr issue).- Async ack/emit/fail PR is green to
> merge. To document the requirements/limitations.- Traffic prediction model
> is on going.- There are many old issues/PRs to clean up.*
>
> On Tue, May 22, 2018 at 1:21 PM, Ning Wang  wrote:
>
>> Hi,
>>
>> The heron OSS sync meeting will be happening today at 2.00 pm PDT.
>> Please use the following hangout link:
>> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync
>> ?authuser=0
>>
>>
>> See you all then.
>>
>>
>

Re: Heron OSS Sync

[Ning Wang]

Brief notes:





* - Not much progress in the new release so far. Twitter forks will reach
out to Bill/Sijie for suggestions.- Productionize stateful process in
Twitter. Found an issue with local checkpoint expiration. - Downloader has
been refactored to be more flexible. - Discussed zombie aurora container
(causing duplicated stmgr issue).- Async ack/emit/fail PR is green to
merge. To document the requirements/limitations.- Traffic prediction model
is on going.- There are many old issues/PRs to clean up.*

On Tue, May 22, 2018 at 1:21 PM, Ning Wang  wrote:

> Hi,
>
> The heron OSS sync meeting will be happening today at 2.00 pm PDT. Please
> use the following hangout link:
> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync
> ?authuser=0
>
>
> See you all then.
>
>

Re: Heron OSS Sync

[Karthik Ramasamy]

+1

On Tue, May 8, 2018 at 4:50 PM, Ning Wang  wrote:

> Agreed. :)
>
> On Tue, May 8, 2018 at 4:49 PM, P. Taylor Goetz  wrote:
>
> > +1 to everything Dave said.
> >
> > Your first Apache release will likely be the hardest. At times it may
> even
> > seem like hazing. You may get a lot of feedback that seems like
> nitpicking.
> >
> > That is not the case. The goal is to make sure to make sure Apache Heron
> > knows how to make compliant releases and will continue to do so after
> > graduation.
> >
> > Don’t get discouraged. Lean on your mentors/advisors.
> >
> > -Taylor
> >
> > > On May 8, 2018, at 7:21 PM, Dave Fisher  wrote:
> > >
> > > Thanks for the quick update!
> > >
> > >> On May 8, 2018, at 4:08 PM, Ning Wang  wrote:
> > >>
> > >> And here is a brief notes:
> > >>
> > >>
> > >>
> > >> * - Our current focus is to have our first apache release by the end
> of
> > >> this week (we should be pretty much ready for it)
> > >
> > > If you mean have the first release candidate ready for voting by the
> end
> > of the week this is achievable. What’s not achievable is completing the
> > VOTE process and making the first release. You should consider this an
> > indeterminate process until the community gets things correct in the
> Apache
> > Way.
> > >
> > > (1) The project will need to take at least 72 hours to review the
> > candidate.
> > > (2) Then the IPMC needs to VOTE on general@. That takes another
> minimum
> > of 72 hours. Often more.
> > >
> > > To pass requires 3 +1 IPMC votes. We review the source and binaries to
> > make sure that all files have license text and that the NOTICE and
> LICENSE
> > is correct for the source and binary.
> > >
> > > Regards,
> > > Dave
> > >
> > >> .- Heron webpage needs
> > >> some update and reorg- Oracle can’t host the meet up next week.
> > >> Rescheduling.- We need more blogs. Karthik will send out some ideas.-
> > >> Please review Saikat’s machine learning support proposal.
> > >> https://docs.google.com/document/d/1LrO7XRcMxJoMM83wjRd-
> > Ov74VAaomA_mXOAhCStgGng/edit
> > >>  > Ov74VAaomA_mXOAhCStgGng/edit>-
> > >> Stateful processing is in progress. Found two issues and working on
> them
> > >> (state data removal and hadoop config)- Investigating stuck stmgr
> issue-
> > >> Working on the model to predict BP when traffic increases- Got a
> > streamlet
> > >> bug report from user about missing acks.- New ubunton 18.04 has
> python 3
> > >> only. We need to migrate- A hands-on session will be scheduled by the
> > end
> > >> of June (sree)- Security concern in downloader/extractor java code.
> Sree
> > >> and karthik to sync up.- CDCI *
> > >> The google doc is here:
> > >> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l9
> > 0xXiY_HssVo8mE/edit?ts=5aa84932#heading=h.nq0bjo3oqwfy.
> > >> Please feel free to reply/comment if anything is mission.
> > >>
> > >>
> > >>
> > >>
> > >>> On Tue, May 8, 2018 at 1:15 PM, Ning Wang 
> > wrote:
> > >>>
> > >>> Hi,
> > >>>
> > >>> The heron OSS sync meeting will be happening today at 2.00 pm PST.
> > Please
> > >>> use the following hangout link:
> > >>> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-
> > >>> sync?authuser=0
> > >>>
> > >>>
> > >>> See you all then.
> > >>>
> > >>>
> > >
> >
>

Re: Heron OSS Sync

[Ning Wang]

Agreed. :)

On Tue, May 8, 2018 at 4:49 PM, P. Taylor Goetz  wrote:

> +1 to everything Dave said.
>
> Your first Apache release will likely be the hardest. At times it may even
> seem like hazing. You may get a lot of feedback that seems like nitpicking.
>
> That is not the case. The goal is to make sure to make sure Apache Heron
> knows how to make compliant releases and will continue to do so after
> graduation.
>
> Don’t get discouraged. Lean on your mentors/advisors.
>
> -Taylor
>
> > On May 8, 2018, at 7:21 PM, Dave Fisher  wrote:
> >
> > Thanks for the quick update!
> >
> >> On May 8, 2018, at 4:08 PM, Ning Wang  wrote:
> >>
> >> And here is a brief notes:
> >>
> >>
> >>
> >> * - Our current focus is to have our first apache release by the end of
> >> this week (we should be pretty much ready for it)
> >
> > If you mean have the first release candidate ready for voting by the end
> of the week this is achievable. What’s not achievable is completing the
> VOTE process and making the first release. You should consider this an
> indeterminate process until the community gets things correct in the Apache
> Way.
> >
> > (1) The project will need to take at least 72 hours to review the
> candidate.
> > (2) Then the IPMC needs to VOTE on general@. That takes another minimum
> of 72 hours. Often more.
> >
> > To pass requires 3 +1 IPMC votes. We review the source and binaries to
> make sure that all files have license text and that the NOTICE and LICENSE
> is correct for the source and binary.
> >
> > Regards,
> > Dave
> >
> >> .- Heron webpage needs
> >> some update and reorg- Oracle can’t host the meet up next week.
> >> Rescheduling.- We need more blogs. Karthik will send out some ideas.-
> >> Please review Saikat’s machine learning support proposal.
> >> https://docs.google.com/document/d/1LrO7XRcMxJoMM83wjRd-
> Ov74VAaomA_mXOAhCStgGng/edit
> >>  Ov74VAaomA_mXOAhCStgGng/edit>-
> >> Stateful processing is in progress. Found two issues and working on them
> >> (state data removal and hadoop config)- Investigating stuck stmgr issue-
> >> Working on the model to predict BP when traffic increases- Got a
> streamlet
> >> bug report from user about missing acks.- New ubunton 18.04 has python 3
> >> only. We need to migrate- A hands-on session will be scheduled by the
> end
> >> of June (sree)- Security concern in downloader/extractor java code. Sree
> >> and karthik to sync up.- CDCI *
> >> The google doc is here:
> >> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l9
> 0xXiY_HssVo8mE/edit?ts=5aa84932#heading=h.nq0bjo3oqwfy.
> >> Please feel free to reply/comment if anything is mission.
> >>
> >>
> >>
> >>
> >>> On Tue, May 8, 2018 at 1:15 PM, Ning Wang 
> wrote:
> >>>
> >>> Hi,
> >>>
> >>> The heron OSS sync meeting will be happening today at 2.00 pm PST.
> Please
> >>> use the following hangout link:
> >>> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-
> >>> sync?authuser=0
> >>>
> >>>
> >>> See you all then.
> >>>
> >>>
> >
>

Re: Heron OSS Sync

[P. Taylor Goetz]

+1 to everything Dave said.

Your first Apache release will likely be the hardest. At times it may even seem 
like hazing. You may get a lot of feedback that seems like nitpicking.

That is not the case. The goal is to make sure to make sure Apache Heron knows 
how to make compliant releases and will continue to do so after graduation.

Don’t get discouraged. Lean on your mentors/advisors.

-Taylor

> On May 8, 2018, at 7:21 PM, Dave Fisher  wrote:
> 
> Thanks for the quick update!
> 
>> On May 8, 2018, at 4:08 PM, Ning Wang  wrote:
>> 
>> And here is a brief notes:
>> 
>> 
>> 
>> * - Our current focus is to have our first apache release by the end of
>> this week (we should be pretty much ready for it)
> 
> If you mean have the first release candidate ready for voting by the end of 
> the week this is achievable. What’s not achievable is completing the VOTE 
> process and making the first release. You should consider this an 
> indeterminate process until the community gets things correct in the Apache 
> Way.
> 
> (1) The project will need to take at least 72 hours to review the candidate.
> (2) Then the IPMC needs to VOTE on general@. That takes another minimum of 72 
> hours. Often more.
> 
> To pass requires 3 +1 IPMC votes. We review the source and binaries to make 
> sure that all files have license text and that the NOTICE and LICENSE is 
> correct for the source and binary.
> 
> Regards,
> Dave
> 
>> .- Heron webpage needs
>> some update and reorg- Oracle can’t host the meet up next week.
>> Rescheduling.- We need more blogs. Karthik will send out some ideas.-
>> Please review Saikat’s machine learning support proposal.
>> https://docs.google.com/document/d/1LrO7XRcMxJoMM83wjRd-Ov74VAaomA_mXOAhCStgGng/edit
>> -
>> Stateful processing is in progress. Found two issues and working on them
>> (state data removal and hadoop config)- Investigating stuck stmgr issue-
>> Working on the model to predict BP when traffic increases- Got a streamlet
>> bug report from user about missing acks.- New ubunton 18.04 has python 3
>> only. We need to migrate- A hands-on session will be scheduled by the end
>> of June (sree)- Security concern in downloader/extractor java code. Sree
>> and karthik to sync up.- CDCI *
>> The google doc is here:
>> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l90xXiY_HssVo8mE/edit?ts=5aa84932#heading=h.nq0bjo3oqwfy.
>> Please feel free to reply/comment if anything is mission.
>> 
>> 
>> 
>> 
>>> On Tue, May 8, 2018 at 1:15 PM, Ning Wang  wrote:
>>> 
>>> Hi,
>>> 
>>> The heron OSS sync meeting will be happening today at 2.00 pm PST. Please
>>> use the following hangout link:
>>> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-
>>> sync?authuser=0
>>> 
>>> 
>>> See you all then.
>>> 
>>> 
> 

Re: Heron OSS Sync

[Ning Wang]

Got it. Thanks!

Yeah. Cadidate is our goal.


On Tue, May 8, 2018 at 4:21 PM, Dave Fisher  wrote:

> Thanks for the quick update!
>
> > On May 8, 2018, at 4:08 PM, Ning Wang  wrote:
> >
> > And here is a brief notes:
> >
> >
> >
> > * - Our current focus is to have our first apache release by the end of
> > this week (we should be pretty much ready for it)
>
> If you mean have the first release candidate ready for voting by the end
> of the week this is achievable. What’s not achievable is completing the
> VOTE process and making the first release. You should consider this an
> indeterminate process until the community gets things correct in the Apache
> Way.
>
> (1) The project will need to take at least 72 hours to review the
> candidate.
> (2) Then the IPMC needs to VOTE on general@. That takes another minimum
> of 72 hours. Often more.
>
> To pass requires 3 +1 IPMC votes. We review the source and binaries to
> make sure that all files have license text and that the NOTICE and LICENSE
> is correct for the source and binary.
>
> Regards,
> Dave
>
> > .- Heron webpage needs
> > some update and reorg- Oracle can’t host the meet up next week.
> > Rescheduling.- We need more blogs. Karthik will send out some ideas.-
> > Please review Saikat’s machine learning support proposal.
> > https://docs.google.com/document/d/1LrO7XRcMxJoMM83wjRd-
> Ov74VAaomA_mXOAhCStgGng/edit
> >  Ov74VAaomA_mXOAhCStgGng/edit>-
> > Stateful processing is in progress. Found two issues and working on them
> > (state data removal and hadoop config)- Investigating stuck stmgr issue-
> > Working on the model to predict BP when traffic increases- Got a
> streamlet
> > bug report from user about missing acks.- New ubunton 18.04 has python 3
> > only. We need to migrate- A hands-on session will be scheduled by the end
> > of June (sree)- Security concern in downloader/extractor java code. Sree
> > and karthik to sync up.- CDCI *
> > The google doc is here:
> > https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l9
> 0xXiY_HssVo8mE/edit?ts=5aa84932#heading=h.nq0bjo3oqwfy.
> > Please feel free to reply/comment if anything is mission.
> >
> >
> >
> >
> > On Tue, May 8, 2018 at 1:15 PM, Ning Wang  wrote:
> >
> >> Hi,
> >>
> >> The heron OSS sync meeting will be happening today at 2.00 pm PST.
> Please
> >> use the following hangout link:
> >> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-
> >> sync?authuser=0
> >>
> >>
> >> See you all then.
> >>
> >>
>
>

Re: Heron OSS Sync

[Karthik Ramasamy]

Thanks Dave for the feedback. If we can get the release candidate out for
review - it will be a good first step.

It will give mentors and others to review the source, NOTICE, LICENSE and
any other items. Based on the
feedback, we can iterate on a few release candidates.

cheers
/karthik

On Tue, May 8, 2018 at 4:21 PM, Dave Fisher  wrote:

> Thanks for the quick update!
>
> > On May 8, 2018, at 4:08 PM, Ning Wang  wrote:
> >
> > And here is a brief notes:
> >
> >
> >
> > * - Our current focus is to have our first apache release by the end of
> > this week (we should be pretty much ready for it)
>
> If you mean have the first release candidate ready for voting by the end
> of the week this is achievable. What’s not achievable is completing the
> VOTE process and making the first release. You should consider this an
> indeterminate process until the community gets things correct in the Apache
> Way.
>
> (1) The project will need to take at least 72 hours to review the
> candidate.
> (2) Then the IPMC needs to VOTE on general@. That takes another minimum
> of 72 hours. Often more.
>
> To pass requires 3 +1 IPMC votes. We review the source and binaries to
> make sure that all files have license text and that the NOTICE and LICENSE
> is correct for the source and binary.
>
> Regards,
> Dave
>
> > .- Heron webpage needs
> > some update and reorg- Oracle can’t host the meet up next week.
> > Rescheduling.- We need more blogs. Karthik will send out some ideas.-
> > Please review Saikat’s machine learning support proposal.
> > https://docs.google.com/document/d/1LrO7XRcMxJoMM83wjRd-
> Ov74VAaomA_mXOAhCStgGng/edit
> >  Ov74VAaomA_mXOAhCStgGng/edit>-
> > Stateful processing is in progress. Found two issues and working on them
> > (state data removal and hadoop config)- Investigating stuck stmgr issue-
> > Working on the model to predict BP when traffic increases- Got a
> streamlet
> > bug report from user about missing acks.- New ubunton 18.04 has python 3
> > only. We need to migrate- A hands-on session will be scheduled by the end
> > of June (sree)- Security concern in downloader/extractor java code. Sree
> > and karthik to sync up.- CDCI *
> > The google doc is here:
> > https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l9
> 0xXiY_HssVo8mE/edit?ts=5aa84932#heading=h.nq0bjo3oqwfy.
> > Please feel free to reply/comment if anything is mission.
> >
> >
> >
> >
> > On Tue, May 8, 2018 at 1:15 PM, Ning Wang  wrote:
> >
> >> Hi,
> >>
> >> The heron OSS sync meeting will be happening today at 2.00 pm PST.
> Please
> >> use the following hangout link:
> >> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-
> >> sync?authuser=0
> >>
> >>
> >> See you all then.
> >>
> >>
>
>

Re: Heron OSS Sync

[Dave Fisher]

Thanks for the quick update!

> On May 8, 2018, at 4:08 PM, Ning Wang  wrote:
> 
> And here is a brief notes:
> 
> 
> 
> * - Our current focus is to have our first apache release by the end of
> this week (we should be pretty much ready for it)

If you mean have the first release candidate ready for voting by the end of the 
week this is achievable. What’s not achievable is completing the VOTE process 
and making the first release. You should consider this an indeterminate process 
until the community gets things correct in the Apache Way.

(1) The project will need to take at least 72 hours to review the candidate.
(2) Then the IPMC needs to VOTE on general@. That takes another minimum of 72 
hours. Often more.

To pass requires 3 +1 IPMC votes. We review the source and binaries to make 
sure that all files have license text and that the NOTICE and LICENSE is 
correct for the source and binary.

Regards,
Dave

> .- Heron webpage needs
> some update and reorg- Oracle can’t host the meet up next week.
> Rescheduling.- We need more blogs. Karthik will send out some ideas.-
> Please review Saikat’s machine learning support proposal.
> https://docs.google.com/document/d/1LrO7XRcMxJoMM83wjRd-Ov74VAaomA_mXOAhCStgGng/edit
> -
> Stateful processing is in progress. Found two issues and working on them
> (state data removal and hadoop config)- Investigating stuck stmgr issue-
> Working on the model to predict BP when traffic increases- Got a streamlet
> bug report from user about missing acks.- New ubunton 18.04 has python 3
> only. We need to migrate- A hands-on session will be scheduled by the end
> of June (sree)- Security concern in downloader/extractor java code. Sree
> and karthik to sync up.- CDCI *
> The google doc is here:
> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l90xXiY_HssVo8mE/edit?ts=5aa84932#heading=h.nq0bjo3oqwfy.
> Please feel free to reply/comment if anything is mission.
> 
> 
> 
> 
> On Tue, May 8, 2018 at 1:15 PM, Ning Wang  wrote:
> 
>> Hi,
>> 
>> The heron OSS sync meeting will be happening today at 2.00 pm PST. Please
>> use the following hangout link:
>> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-
>> sync?authuser=0
>> 
>> 
>> See you all then.
>> 
>> 



signature.asc
Description: Message signed with OpenPGP

Re: Heron OSS Sync

[Ning Wang]

And here are a brief notes:



* - The first meetup went very well last evening! 60+ attendees. Next
meetup will be in May and could be hackthon format. 200+ followers in the
meetup channel now.- Making an apache release (v0.2.0) is our priority.
Package name changed. Copy right to update. Need to remove binaries. Thrift
0.5.0 is used by Twitter. To remove by Twitter team. Need to sync with
apache infra for the infra part.- Stateful processing in progress. Found
missing functions (stream repartitioning, sink data deduplication).-
Stateful storage layer revisiting- Modeling the traffic evaluation system.-
Integration tests for the Scala streamlet API, as well as the
documentation. Pretty much ready. Scala bazel rule is special and needs to
be doublechecked- New resource config proposal. Could be helpful for some
users.- Container pipeline is in progress.- SQL API in progress too.
Looking into issue with Intellij.*

On Tue, Apr 24, 2018 at 1:55 PM, Ning Wang  wrote:

> Will be happening today at 2.00 pm PST. Please use the following hangou
> link:
> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-
> sync?authuser=0
>
>
> See you all then.
>

Re: Heron OSS Sync Meeting notes

[Ning Wang]

Got it. Thanks.

On Tue, Mar 27, 2018 at 6:35 PM, Dave Fisher  wrote:

> Hi Ning,
>
> If matters including that the hangout is happening were discussed on this
> mailing list then the following benefits occur.
>
> (1) Every person interested and subscribed can know the agenda.
> (2) Anyone can share any updates to the meeting url in real time.
> (3) Post Meeting Notes are kept in the open.
> (4) Apache projects are Open Source for the public good.
>
> I’m here to help Heron become an Apache project.
>
> Regards,
> Dave
>
> Sent from my iPhone
>
> > On Mar 27, 2018, at 4:51 PM, Ning Wang  wrote:
> >
> > A little more details about the hangout issue:
> >
> > Owner of hangout meeting is responsible to accept incoming requests.
> Since
> > Karthik is sick today so we had to create a new temporary meeting, which
> is
> > not ideal.
> >
> >
> >
> >> On Tue, Mar 27, 2018 at 4:09 PM, Ning Wang 
> wrote:
> >>
> >> Thanks for the suggestions.
> >>
> >> On Tue, Mar 27, 2018 at 4:02 PM, Dave Fisher 
> >> wrote:
> >>
> >>>  - Looking for another host for our Mailing list
> >>>
> >>>
> >>> The Mailing List must be dev@heron.incubator.apache.org using any
> other
> >>> mailing list is not the Apache Way.
> >>>
> >>>
> >>>  - Hangout is killing us today! We need a better solution.
> >>>
> >>>
> >>> Discuss items in plain sight of the whole community on this list. This
> >>> allows other developers to participate asynchronously.
> >>>
> >>> Regards,
> >>> Dave
> >>>
> >>>
> >>> On Mar 27, 2018, at 3:47 PM, Ning Wang  wrote:
> >>>
> >>> Hi,
> >>>
> >>> Here is a super brief notes for today's sync meeting:
> >>>
> >>>
> >>>  - Hangout is killing us today! We need a better solution.
> >>>  - Per component cpu/disk config is added but not used yet.
> >>>  - Looking into failing integration tests. This has been affecting us
> >>>  quite a big
> >>>  - New works on nomad scheduler (docker support)
> >>>  - New padding configs (to improve padding calculation and avoid hard
> >>>  coded configs) are documented and shared:
> >>>  https://docs.google.com/document/d/10JNZNYtcUIlsEyXcWWQUyh
> >>> cSoQoJZMvYrTYT8UFo4dQ/edit
> >>>  - Clean up and k8s scheduler related works ongoing
> >>>  - New scala examples have been added. Next step is to document them
> and
> >>>  integration tests
> >>>  - Looking for another host for our Mailing list
> >>>  - Code change ongoing to make the website apache ready. ETA next
> >>>  weekend. Apache Infra setup is needed for starting the website.
> >>>  - Apache transfer is on going by Twitter’s OSS team.
> >>>  - We need to finalize the meetup date @maosong.
> >>>
> >>>
> >>>
> >>>
> >>> Meeting notes are tracked here:
> >>> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l9
> >>> 0xXiY_HssVo8mE/edit?ts=5aa84932#
> >>>
> >>>
> >>>
> >>
>
>

Re: Heron OSS Sync Meeting notes

[Dave Fisher]

Hi Ning,

If matters including that the hangout is happening were discussed on this 
mailing list then the following benefits occur.

(1) Every person interested and subscribed can know the agenda.
(2) Anyone can share any updates to the meeting url in real time.
(3) Post Meeting Notes are kept in the open.
(4) Apache projects are Open Source for the public good.

I’m here to help Heron become an Apache project.

Regards,
Dave

Sent from my iPhone

> On Mar 27, 2018, at 4:51 PM, Ning Wang  wrote:
> 
> A little more details about the hangout issue:
> 
> Owner of hangout meeting is responsible to accept incoming requests. Since
> Karthik is sick today so we had to create a new temporary meeting, which is
> not ideal.
> 
> 
> 
>> On Tue, Mar 27, 2018 at 4:09 PM, Ning Wang  wrote:
>> 
>> Thanks for the suggestions.
>> 
>> On Tue, Mar 27, 2018 at 4:02 PM, Dave Fisher 
>> wrote:
>> 
>>>  - Looking for another host for our Mailing list
>>> 
>>> 
>>> The Mailing List must be dev@heron.incubator.apache.org using any other
>>> mailing list is not the Apache Way.
>>> 
>>> 
>>>  - Hangout is killing us today! We need a better solution.
>>> 
>>> 
>>> Discuss items in plain sight of the whole community on this list. This
>>> allows other developers to participate asynchronously.
>>> 
>>> Regards,
>>> Dave
>>> 
>>> 
>>> On Mar 27, 2018, at 3:47 PM, Ning Wang  wrote:
>>> 
>>> Hi,
>>> 
>>> Here is a super brief notes for today's sync meeting:
>>> 
>>> 
>>>  - Hangout is killing us today! We need a better solution.
>>>  - Per component cpu/disk config is added but not used yet.
>>>  - Looking into failing integration tests. This has been affecting us
>>>  quite a big
>>>  - New works on nomad scheduler (docker support)
>>>  - New padding configs (to improve padding calculation and avoid hard
>>>  coded configs) are documented and shared:
>>>  https://docs.google.com/document/d/10JNZNYtcUIlsEyXcWWQUyh
>>> cSoQoJZMvYrTYT8UFo4dQ/edit
>>>  - Clean up and k8s scheduler related works ongoing
>>>  - New scala examples have been added. Next step is to document them and
>>>  integration tests
>>>  - Looking for another host for our Mailing list
>>>  - Code change ongoing to make the website apache ready. ETA next
>>>  weekend. Apache Infra setup is needed for starting the website.
>>>  - Apache transfer is on going by Twitter’s OSS team.
>>>  - We need to finalize the meetup date @maosong.
>>> 
>>> 
>>> 
>>> 
>>> Meeting notes are tracked here:
>>> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l9
>>> 0xXiY_HssVo8mE/edit?ts=5aa84932#
>>> 
>>> 
>>> 
>> 

Re: Heron OSS Sync Meeting notes

[Ning Wang]

A little more details about the hangout issue:

Owner of hangout meeting is responsible to accept incoming requests. Since
Karthik is sick today so we had to create a new temporary meeting, which is
not ideal.



On Tue, Mar 27, 2018 at 4:09 PM, Ning Wang  wrote:

> Thanks for the suggestions.
>
> On Tue, Mar 27, 2018 at 4:02 PM, Dave Fisher 
> wrote:
>
>>   - Looking for another host for our Mailing list
>>
>>
>> The Mailing List must be dev@heron.incubator.apache.org using any other
>> mailing list is not the Apache Way.
>>
>>
>>   - Hangout is killing us today! We need a better solution.
>>
>>
>> Discuss items in plain sight of the whole community on this list. This
>> allows other developers to participate asynchronously.
>>
>> Regards,
>> Dave
>>
>>
>> On Mar 27, 2018, at 3:47 PM, Ning Wang  wrote:
>>
>> Hi,
>>
>> Here is a super brief notes for today's sync meeting:
>>
>>
>>   - Hangout is killing us today! We need a better solution.
>>   - Per component cpu/disk config is added but not used yet.
>>   - Looking into failing integration tests. This has been affecting us
>>   quite a big
>>   - New works on nomad scheduler (docker support)
>>   - New padding configs (to improve padding calculation and avoid hard
>>   coded configs) are documented and shared:
>>   https://docs.google.com/document/d/10JNZNYtcUIlsEyXcWWQUyh
>> cSoQoJZMvYrTYT8UFo4dQ/edit
>>   - Clean up and k8s scheduler related works ongoing
>>   - New scala examples have been added. Next step is to document them and
>>   integration tests
>>   - Looking for another host for our Mailing list
>>   - Code change ongoing to make the website apache ready. ETA next
>>   weekend. Apache Infra setup is needed for starting the website.
>>   - Apache transfer is on going by Twitter’s OSS team.
>>   - We need to finalize the meetup date @maosong.
>>
>>
>>
>>
>> Meeting notes are tracked here:
>> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l9
>> 0xXiY_HssVo8mE/edit?ts=5aa84932#
>>
>>
>>
>

Re: Heron OSS Sync Meeting notes

[Ning Wang]

Thanks for the suggestions.

On Tue, Mar 27, 2018 at 4:02 PM, Dave Fisher  wrote:

>   - Looking for another host for our Mailing list
>
>
> The Mailing List must be dev@heron.incubator.apache.org using any other
> mailing list is not the Apache Way.
>
>
>   - Hangout is killing us today! We need a better solution.
>
>
> Discuss items in plain sight of the whole community on this list. This
> allows other developers to participate asynchronously.
>
> Regards,
> Dave
>
>
> On Mar 27, 2018, at 3:47 PM, Ning Wang  wrote:
>
> Hi,
>
> Here is a super brief notes for today's sync meeting:
>
>
>   - Hangout is killing us today! We need a better solution.
>   - Per component cpu/disk config is added but not used yet.
>   - Looking into failing integration tests. This has been affecting us
>   quite a big
>   - New works on nomad scheduler (docker support)
>   - New padding configs (to improve padding calculation and avoid hard
>   coded configs) are documented and shared:
>   https://docs.google.com/document/d/10JNZNYtcUIlsEyXcWWQUyhcSoQoJZ
> MvYrTYT8UFo4dQ/edit
>   - Clean up and k8s scheduler related works ongoing
>   - New scala examples have been added. Next step is to document them and
>   integration tests
>   - Looking for another host for our Mailing list
>   - Code change ongoing to make the website apache ready. ETA next
>   weekend. Apache Infra setup is needed for starting the website.
>   - Apache transfer is on going by Twitter’s OSS team.
>   - We need to finalize the meetup date @maosong.
>
>
>
>
> Meeting notes are tracked here:
> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l9
> 0xXiY_HssVo8mE/edit?ts=5aa84932#
>
>
>

Re: Heron OSS Sync Meeting notes

[Dave Fisher]

>   - Looking for another host for our Mailing list

The Mailing List must be dev@heron.incubator.apache.org 
 using any other mailing list is not the 
Apache Way.


>   - Hangout is killing us today! We need a better solution.

Discuss items in plain sight of the whole community on this list. This allows 
other developers to participate asynchronously.

Regards,
Dave


> On Mar 27, 2018, at 3:47 PM, Ning Wang  wrote:
> 
> Hi,
> 
> Here is a super brief notes for today's sync meeting:
> 
> 
>   - Hangout is killing us today! We need a better solution.
>   - Per component cpu/disk config is added but not used yet.
>   - Looking into failing integration tests. This has been affecting us
>   quite a big
>   - New works on nomad scheduler (docker support)
>   - New padding configs (to improve padding calculation and avoid hard
>   coded configs) are documented and shared:
>   
> https://docs.google.com/document/d/10JNZNYtcUIlsEyXcWWQUyhcSoQoJZMvYrTYT8UFo4dQ/edit
>   - Clean up and k8s scheduler related works ongoing
>   - New scala examples have been added. Next step is to document them and
>   integration tests
>   - Looking for another host for our Mailing list
>   - Code change ongoing to make the website apache ready. ETA next
>   weekend. Apache Infra setup is needed for starting the website.
>   - Apache transfer is on going by Twitter’s OSS team.
>   - We need to finalize the meetup date @maosong.
> 
> 
> 
> 
> Meeting notes are tracked here:
> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l9
> 0xXiY_HssVo8mE/edit?ts=5aa84932#



signature.asc
Description: Message signed with OpenPGP

Re: Heron OSS Sync

[Dave Fisher]

Hi Ning Wang,

You can reach out to Infra on their HipChat at 
https://www.hipchat.com/gh6DJhKrN 

Regards,
Dave


> On Mar 20, 2018, at 2:55 PM, Fu Maosong  wrote:
> 
> Hi Dave -
> 
> I added Ning Wang from Twitter with Admin permission and he has started
> working on it.
> 
> 2018-03-16 10:31 GMT-07:00 Dave Fisher :
> 
>> Hi Maosong -
>> 
>> Infra is waiting.
>> 
>> Daniel Takamori commented on INFRA-16117:
>> -
>> 
>> [~sreevaddi] [~wave]  Ping on adding someone from infra to the Github org
>> so we can move repos into the apache organization.
>> 
>> 
>> Let us know if you need any help or advice.
>> 
>> Regards,
>> Dave
>> 
>> 
>> On Mar 13, 2018, at 3:20 PM, Ning Wang  wrote:
>> 
>> And here are the brief notes for today's meeting:
>> 
>> 
>> * - Maosong needs to contact with Apache admin about github to
>> apache/github.- After code migration is done, we are going to follow apache
>> release process- Sree is following up on the site and documentations- A few
>> bug fixes on k8s support were merged and will be included in the new
>> release.- Dhalion testing is ongoing in Microsoft- SQL API is ongoing,
>> https://docs.google.com/document/d/1PxLCyR_H-
>> mOgPjyFj3DhWXryKW21CH2zFWwzTnqjfEA/edit
>> > mOgPjyFj3DhWXryKW21CH2zFWwzTnqjfEA/edit>-
>> More Dhalion metrics are being added (Twitter internal feedback/requests)-
>> Data rate limiting is on going. Will be hooked up with runtime config. In
>> future some automation might be possible. It could also be related to auto
>> scaling. To document after it is stabilized.- First meetup date/time. Late
>> April, Sree is talking to some meeting groups such as
>> http://www.sfbayacm.org/ - Slack/email
>> integration is ongoing (almost ready)*
>> Meeting notes are tracked here:
>> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l9
>> 0xXiY_HssVo8mE/edit?ts=5aa84932#
>> 
>> 
>> On Tue, Mar 13, 2018 at 11:00 AM, Karthik Ramasamy 
>> wrote:
>> 
>> Will be happening today at 2.00 pm PST. Please use the following hangout
>> link
>> 
>> https://hangouts.google.com/hangouts/_/streaml.io/oss-
>> heron-sync?authuser=0
>> 
>> See you all then.
>> 
>> cheers
>> /karthik
>> 
>> 
>> 
> 
> 
> --
> With my best Regards
> --
> Fu Maosong
> Twitter Inc.
> Mobile: +001-415-244-7520



signature.asc
Description: Message signed with OpenPGP

Re: Heron OSS Sync

[Fu Maosong]

Hi Dave -

I added Ning Wang from Twitter with Admin permission and he has started
working on it.

2018-03-16 10:31 GMT-07:00 Dave Fisher :

> Hi Maosong -
>
> Infra is waiting.
>
> Daniel Takamori commented on INFRA-16117:
> -
>
> [~sreevaddi] [~wave]  Ping on adding someone from infra to the Github org
> so we can move repos into the apache organization.
>
>
> Let us know if you need any help or advice.
>
> Regards,
> Dave
>
>
> On Mar 13, 2018, at 3:20 PM, Ning Wang  wrote:
>
> And here are the brief notes for today's meeting:
>
>
> * - Maosong needs to contact with Apache admin about github to
> apache/github.- After code migration is done, we are going to follow apache
> release process- Sree is following up on the site and documentations- A few
> bug fixes on k8s support were merged and will be included in the new
> release.- Dhalion testing is ongoing in Microsoft- SQL API is ongoing,
> https://docs.google.com/document/d/1PxLCyR_H-
> mOgPjyFj3DhWXryKW21CH2zFWwzTnqjfEA/edit
>  mOgPjyFj3DhWXryKW21CH2zFWwzTnqjfEA/edit>-
> More Dhalion metrics are being added (Twitter internal feedback/requests)-
> Data rate limiting is on going. Will be hooked up with runtime config. In
> future some automation might be possible. It could also be related to auto
> scaling. To document after it is stabilized.- First meetup date/time. Late
> April, Sree is talking to some meeting groups such as
> http://www.sfbayacm.org/ - Slack/email
> integration is ongoing (almost ready)*
> Meeting notes are tracked here:
> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l9
> 0xXiY_HssVo8mE/edit?ts=5aa84932#
>
>
> On Tue, Mar 13, 2018 at 11:00 AM, Karthik Ramasamy 
> wrote:
>
> Will be happening today at 2.00 pm PST. Please use the following hangout
> link
>
> https://hangouts.google.com/hangouts/_/streaml.io/oss-
> heron-sync?authuser=0
>
> See you all then.
>
> cheers
> /karthik
>
>
>


-- 
With my best Regards
--
Fu Maosong
Twitter Inc.
Mobile: +001-415-244-7520

Re: Heron OSS Sync

[Dave Fisher]

Hi Maosong -

Infra is waiting.

Daniel Takamori commented on INFRA-16117:
-

[~sreevaddi] [~wave]  Ping on adding someone from infra to the Github org so we 
can move repos into the apache organization.

Let us know if you need any help or advice.

Regards,
Dave


> On Mar 13, 2018, at 3:20 PM, Ning Wang  wrote:
> 
> And here are the brief notes for today's meeting:
> 
> 
> * - Maosong needs to contact with Apache admin about github to
> apache/github.- After code migration is done, we are going to follow apache
> release process- Sree is following up on the site and documentations- A few
> bug fixes on k8s support were merged and will be included in the new
> release.- Dhalion testing is ongoing in Microsoft- SQL API is ongoing,
> https://docs.google.com/document/d/1PxLCyR_H-mOgPjyFj3DhWXryKW21CH2zFWwzTnqjfEA/edit
> -
> More Dhalion metrics are being added (Twitter internal feedback/requests)-
> Data rate limiting is on going. Will be hooked up with runtime config. In
> future some automation might be possible. It could also be related to auto
> scaling. To document after it is stabilized.- First meetup date/time. Late
> April, Sree is talking to some meeting groups such as
> http://www.sfbayacm.org/ - Slack/email
> integration is ongoing (almost ready)*
> Meeting notes are tracked here:
> https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l90xXiY_HssVo8mE/edit?ts=5aa84932#
> 
> 
> On Tue, Mar 13, 2018 at 11:00 AM, Karthik Ramasamy 
> wrote:
> 
>> Will be happening today at 2.00 pm PST. Please use the following hangout
>> link
>> 
>> https://hangouts.google.com/hangouts/_/streaml.io/oss-
>> heron-sync?authuser=0
>> 
>> See you all then.
>> 
>> cheers
>> /karthik
>> 



signature.asc
Description: Message signed with OpenPGP

Re: Heron OSS Sync

[Ning Wang]

And here are the brief notes for today's meeting:


* - Maosong needs to contact with Apache admin about github to
apache/github.- After code migration is done, we are going to follow apache
release process- Sree is following up on the site and documentations- A few
bug fixes on k8s support were merged and will be included in the new
release.- Dhalion testing is ongoing in Microsoft- SQL API is ongoing,
https://docs.google.com/document/d/1PxLCyR_H-mOgPjyFj3DhWXryKW21CH2zFWwzTnqjfEA/edit
-
More Dhalion metrics are being added (Twitter internal feedback/requests)-
Data rate limiting is on going. Will be hooked up with runtime config. In
future some automation might be possible. It could also be related to auto
scaling. To document after it is stabilized.- First meetup date/time. Late
April, Sree is talking to some meeting groups such as
http://www.sfbayacm.org/ - Slack/email
integration is ongoing (almost ready)*
Meeting notes are tracked here:
https://docs.google.com/document/d/1cTIBq3jOVRTSR0Zd5OKK20OqwT2l90xXiY_HssVo8mE/edit?ts=5aa84932#


On Tue, Mar 13, 2018 at 11:00 AM, Karthik Ramasamy 
wrote:

> Will be happening today at 2.00 pm PST. Please use the following hangout
> link
>
> https://hangouts.google.com/hangouts/_/streaml.io/oss-
> heron-sync?authuser=0
>
> See you all then.
>
> cheers
> /karthik
>

Re: Heron OSS Sync - 01/30/18

[Ning Wang]

Please look for another message from Karthik, with subject "Heron Sync up
Meeting Notes". Also please feel free to comment.

On Tue, Jan 30, 2018 at 5:28 PM, Saikat Kanjilal 
wrote:

> All,
> Apologies but I missed this , we’re there any notes or output conversation
> that you guys can share with the community, would be good to be in the loop.
> Thanks
>
> Sent from my iPhone
>
> > On Jan 30, 2018, at 1:59 PM, Karthik Ramasamy 
> wrote:
> >
> > All --
> >
> > Heron OSS Sync will be happening at 2.00 pm PST. Please use the Google
> > Hangout -
> >
> > https://plus.google.com/hangouts/_/streaml.io/oss-heron-sync?hceid=
> a2FydGhpa0BzdHJlYW1sLmlv.218m8hgr4ekqsf1o8of2olo1a0=0
> >
> > cheers
> > /karthik
>

Re: Heron OSS Sync - 01/30/18

[P. Taylor Goetz]

If it’s recurring, it should be advertised to the mailing list well in advance. 
It would be nice to also have a link on the official Apache Heron website [1], 
which doesn’t exist yet, but should by now.

If the community needs help with anything ASF related, your mentors are here to 
help.

-Taylo

> On Jan 30, 2018, at 5:56 PM, Karthik Ramasamy  wrote:
> 
> Taylor -
> 
> This is a OSS sync up that happens every two weeks. We have been doing it
> for a while. If anybody is interested, they can hop on the call. We are
> thinking about adding
> this hangout in a public forum so that whoever is interested can
> participate.  Wondering if others have ideas about hosting a public
> calendar and a hangout link?
> 
> cheers
> /karthik
> 
>> On Tue, Jan 30, 2018 at 2:47 PM, Dave Fisher  wrote:
>> 
>> I just saw it and hopped on. They plan to summarize the notes.  It’s still
>> going if you want ...
>> 
>>> On Jan 30, 2018, at 2:43 PM, P. Taylor Goetz  wrote:
>>> 
>>> It’s probably too late, but I would suggest canceling this. There has
>> been no public, on-list discussion regarding planning this, and announcing
>> it one minute before it takes place hardly gives interested parties enough
>> notice to make plans to attend.
>>> 
>>> This is not how Apache works.
>>> 
>>> -Taylor
>>> 
 On Jan 30, 2018, at 4:59 PM, Karthik Ramasamy 
>> wrote:
 
 All --
 
 Heron OSS Sync will be happening at 2.00 pm PST. Please use the Google
 Hangout -
 
 https://plus.google.com/hangouts/_/streaml.io/oss-heron-sync?hceid=
>> a2FydGhpa0BzdHJlYW1sLmlv.218m8hgr4ekqsf1o8of2olo1a0=0
 
 cheers
 /karthik
>> 
>> 

Re: Heron OSS Sync - 01/30/18

[P. Taylor Goetz]

I can’t at the moment, but would have if given enough notice. 72 hrs. notice 
would be a good start, even 24.

-Taylor

> On Jan 30, 2018, at 5:47 PM, Dave Fisher  wrote:
> 
> I just saw it and hopped on. They plan to summarize the notes.  It’s still 
> going if you want ...
> 
>> On Jan 30, 2018, at 2:43 PM, P. Taylor Goetz  wrote:
>> 
>> It’s probably too late, but I would suggest canceling this. There has been 
>> no public, on-list discussion regarding planning this, and announcing it one 
>> minute before it takes place hardly gives interested parties enough notice 
>> to make plans to attend.
>> 
>> This is not how Apache works.
>> 
>> -Taylor
>> 
>>> On Jan 30, 2018, at 4:59 PM, Karthik Ramasamy  wrote:
>>> 
>>> All --
>>> 
>>> Heron OSS Sync will be happening at 2.00 pm PST. Please use the Google
>>> Hangout -
>>> 
>>> https://plus.google.com/hangouts/_/streaml.io/oss-heron-sync?hceid=a2FydGhpa0BzdHJlYW1sLmlv.218m8hgr4ekqsf1o8of2olo1a0=0
>>> 
>>> cheers
>>> /karthik
> 

Re: Heron OSS Sync - 01/30/18

[Karthik Ramasamy]

Taylor -

This is a OSS sync up that happens every two weeks. We have been doing it
for a while. If anybody is interested, they can hop on the call. We are
thinking about adding
this hangout in a public forum so that whoever is interested can
participate.  Wondering if others have ideas about hosting a public
calendar and a hangout link?

cheers
/karthik

On Tue, Jan 30, 2018 at 2:47 PM, Dave Fisher  wrote:

> I just saw it and hopped on. They plan to summarize the notes.  It’s still
> going if you want ...
>
> > On Jan 30, 2018, at 2:43 PM, P. Taylor Goetz  wrote:
> >
> > It’s probably too late, but I would suggest canceling this. There has
> been no public, on-list discussion regarding planning this, and announcing
> it one minute before it takes place hardly gives interested parties enough
> notice to make plans to attend.
> >
> > This is not how Apache works.
> >
> > -Taylor
> >
> >> On Jan 30, 2018, at 4:59 PM, Karthik Ramasamy 
> wrote:
> >>
> >> All --
> >>
> >> Heron OSS Sync will be happening at 2.00 pm PST. Please use the Google
> >> Hangout -
> >>
> >> https://plus.google.com/hangouts/_/streaml.io/oss-heron-sync?hceid=
> a2FydGhpa0BzdHJlYW1sLmlv.218m8hgr4ekqsf1o8of2olo1a0=0
> >>
> >> cheers
> >> /karthik
>
>