Re: Inter-Container Encryption in Heron

Having separate key pair for each container might have scalability issue for some really large topologies. As far as I know, Twitter has some prod topolog's whose # of containers could be as large as 400+. On Fri, Sep 15, 2017 at 11:29 PM, Sanjeev Kulkarni wrote: > Having a separate keypair per

Re: PEX upgrade to 1.2.11

Currently, Maintaining pex is a pain for us. If we can get rid of its source code. That would be really great. Also, I verified this change with our environment. It passed our building process. On Mon, Sep 18, 2017 at 10:16 AM, Karthik Ramasamy wrote: > All - > > I would like to move the PEX us

Re: Heron Release

Hi all, Bring up the release again, user is requesting the fix: https://github.com/twitter/heron/pull/2291 And I think we already have a bunch of new thing merged since last release. Could we schedule a release by the end of this week or early next week? On Thu, Sep 14, 2017 at 4:57 PM, Fu Maoso

Re: Heron Release

> >> Anything else we want to really go in for this one? > >> Thanks! > >> > >> On Mon, Sep 18, 2017 at 11:57 AM, Sanjeev Kulkarni > > >> wrote: > >> > >>> Neng, I left some comments on 2291. Thanks! > >>> > >>> On M

Re: Specifying Operator Resource in DSL

Just add some thoughts here: for ordinary heron topologies, the definition of a heron job and the request of resources usage for each component are separated: `TopologyBuilder` for job definition, `Config` for resource requirement. In the dsl case, if we could also do something similar that separa

Re: Specifying Operator Resource in DSL

ations can squish the > operators into single physical operator. One possibility would be to add a > name optionally to the operator(like map(mapfn, name), but that seems too > cumbersome/kludgy) > > On Thu, Sep 21, 2017 at 3:57 PM, Neng Lu wrote: > > > Just add some thou

Re: ECO Now Supports Heron API

Nice work! :thumbsup: On Thu, Apr 5, 2018 at 10:00 PM, Ning Wang wrote: > Great! > > On Thu, Apr 5, 2018 at 8:06 PM, Josh Fischer wrote: > > > Based on community feedback we have enabled Heron's ECO to deploy > > topologies that contain spouts and bolts from either the the Heron API or > > the

Re: Heron github emails

+1 On Tue, May 1, 2018 at 7:41 PM, Jake Farrell wrote: > +1 > > -Jake > > On Tue, May 1, 2018 at 10:00 PM, Chris Kellogg > wrote: > > > +1 > > > > On Tue, May 1, 2018 at 4:45 PM, Karthik Ramasamy > > wrote: > > > > > +1 as well. > > > > > > On Tue, May 1, 2018 at 3:53 PM Josh Fischer > wrote:

Re: Stateful updating and deterministic routing

+1 for this idea. As long as the predefined key space is large enough, it should work for most of the cases. Based on my experience with topologies, I never saw one component has more than 1000 instances in a topology. For recovering states from an update, there will be some problems though. Sinc

Proposal of Alternatively Transferring State via Disk

Hi all, During our test at twitter, when a state is very large (GBs), the current checkpointing mechanism doesn't work well. Here's a proposal of transferring large checkpointing state through local disk to solve the problem we've found. https://docs.google.com/document/d/1MiOAV3bZATezuIgpwk8Jkwa

07/17 Heron OSS Sync Notes

Hi all, Here are the notes for today's sync meeting: * - Project Updates- Added integration tests for instance state- Fixing the large state spilling problem- Developing a framework for monitoring backpressure- New OSS heron meetings in the Bay area will be launched soon.- The apache release i

Re: Rough Draft to Podling Report

Thanks for composing the report! On Tue, Aug 7, 2018 at 12:50 PM Karthik Ramasamy wrote: > Thanks Josh > > On Tue, Aug 7, 2018 at 8:37 AM Ning Wang wrote: > > > nice! > > > > On Tue, Aug 7, 2018 at 8:06 AM, Josh Fischer > wrote: > > > > > The Heron Incubator PMC report has been submitted. > >

[VOTE] Heron Release 0.20.0-incubating Candidate 1

.20.0-incubating-candidate-1 Please download the the source package, and follow the compiling guide( https://apache.github.io/incubator-heron/docs/developers/compiling/compiling/) to build and run the Heron locally. Best Regards, Neng Lu

Re: [VOTE] Heron Release 0.20.0-incubating Candidate 1

s/#guidelines > > , http://www.apache.org/foundation/marks/faq/#poweredby , and > > https://www.apache.org/foundation/press/kit/ > > > > The icomoon font in the tools/ui has licenses that are incompatible with > > Apache releases. I would not block this release for th

[VOTE] Heron Release 0.20.0-incubating Candidate 2

ator-heron/docs/developers/compiling/compiling/) to build and run the Heron locally. Best Regards, Neng Lu

Re: [VOTE] Heron Release 0.20.0-incubating Candidate 2

uires git > environment. > > On Wed, Aug 22, 2018 at 11:00 AM Neng Lu wrote: > > > Hi All, > > > > This is the 2nd release candidate for Apache Heron, version > > 0.20.0-incubating. Thanks Dave Fisher for providing various feedback for > > the first relea

Re: how to enable travis ci hook

I tried to disable and enable the heron repo but found I don't have the permission to do so. Anyone has this permission? [image: image (1).png] [image: image (2).png] On Tue, Aug 21, 2018 at 10:44 PM Karthik Ramasamy wrote: > Not that I am aware of > > On Tue, Aug 21, 2018 at 10:39 PM, FatJ L

[VOTE] Heron Release 0.20.0-incubating Candidate 3

the source package, and follow the compiling guide( https://apache.github.io/incubator-heron/docs/developers/compiling/compiling/) to build and run the Heron locally. Best Regards, Neng Lu

Re: Heron OSS Sync Meeting

+1 On Mon, Sep 10, 2018 at 10:23 AM Josh Fischer wrote: > +1 > > On Mon, Sep 10, 2018 at 12:20 PM Ning Wang wrote: > > > One thing I think we may need to discuss tomorrow is to find a > replacement > > for Google hangout. We have seen some issues(cant accept requests) in the > > past a few week

09/11/2018 Heron OSS Sync Notes

Hi all, Here are the notes for today's sync meeting: - Several Release Candidates are done in the last few weeks. There is some work remaining to be done for the next RC: - We need to fix the out of data website - A user reported problems when launching topology with

Re: Need to step down as mentor

Thank you very much for all the help you provided to Heron! On Sun, Sep 16, 2018 at 10:28 PM Karthik Ramasamy wrote: > Thanks Jacques. Understand that you are busy - thanks for the help so far. > > cheers > /karthik > > On Sun, Sep 16, 2018 at 1:09 AM Ning Wang wrote: > > > Thanks for your help

[VOTE] Heron Release 0.20.0-incubating Candidate 3

Hi All, This is the 4th release candidate for Apache Heron, version 0.20.0-incubating. Thank Dave Fisher and Ning Wang for providing various feedback for the previous release candidates. It is the starting point of Heron and contains heron's main features, such as streaming processing, stateful p

[VOTE] Heron Release 0.20.0-incubating Candidate 4

Hi All, This is the 4th release candidate for Apache Heron, version 0.20.0-incubating. Thank Dave Fisher and Ning Wang for providing various feedback for the previous release candidates. It is the starting point of Heron and contains heron's main features, such as streaming processing, stateful p

Re: Heron OSS Sync Meeting

+1 for this idea. We should utilize more of the mailing list and use it as the main discussion place. The sync meet should only play as a supplementary role. On Mon, Sep 24, 2018 at 2:12 PM Ning Wang wrote: > Hi, > > > The heron OSS sync meeting will be happening tomorrow at 2.00 pm PDT. > Plea

Re: Hermetic Build Proposal

This is a great proposal! On Sat, Oct 13, 2018 at 7:33 PM Sree V wrote: > created an issue to implement and track this. > https://github.com/apache/incubator-heron/issues/3045 > > > Thank you.Sree > > On Friday, October 12, 2018, 8:43:40 PM PDT, Karthik Ramasamy < > kart...@streaml.io> wrote

[VOTE] Heron Release 0.20.0-incubating Candidate 5

Hi All, This is the 5th release candidate for Apache Heron, version 0.20.0-incubating. Thanks everyone for providing various feedback for the previous release candidates. It is the starting point of Heron and contains heron's main features, such as streaming processing, stateful processing, strea

Re: [VOTE] Heron Release 0.20.0-incubating Candidate 5

s passed > > > - installed CLI successfully > > > - example topology runs fine locally > > > - ui/tracker work fine and show topology info > > > - license scan with apache-rat (rat couldn't find license in > > cloudpickle.py > > > but it is

Re: Podling Report Reminder - November 2018

Any update on this Podling Report task? On Sat, Oct 20, 2018 at 10:46 AM Ning wrote: > Cool. Thanks! > > Sent from my iPhone > > > On Oct 20, 2018, at 10:13 AM, Josh Fischer wrote: > > > > I’ve pasted in the report from August. You can access where the template > > is to be input here —-> > >

10/22/2018 Bi-Weekly OSS Heron Sync-up

Hi All, It has been two weeks since our last sync and now is time to share our progress again. Let's share our works done for the last two weeks in this thread. And see if we need some online discussion for difficult problems. >From Twitter's side: 1. Ning Drafted the streamlet custom operator de

Re: 10/22/2018 Bi-Weekly OSS Heron Sync-up

tral. > > > 2. Merit is given to individuals. > > > > > > With that in mind for the next reports: > > > > > > A. There should not be a “Twitter side”. There is what contributors are > > > doing. > > > B. It helps to read on this mailing list a r

Re: [VOTE] Heron Release 0.20.0-incubating Candidate 5

9:55 AM Neng Lu wrote: > @Jerry, any update on this issue? > > Should we open a separate thread in @dev to discuss this bazel issue? > > On Thu, Oct 18, 2018 at 11:32 PM Ning Wang wrote: > >> Bazel is often not backward compatible. Please use 0.14.1 >> >> I f

Re: BazelCon

Nice talk! On Mon, Nov 5, 2018 at 1:43 PM Josh Fischer wrote: > Sure. They are available here -> > > https://docs.google.com/presentation/d/1QSJ1o71jFeJyGfn5uRgEgmIVkZ0yP8ZZe4c6I_Ullxg/edit?usp=sharing > > On Mon, Nov 5, 2018 at 3:32 PM Ali Ahmed wrote: > > > Hi Josh, > > > > Could you publis

Re: 11/06/2018 Bi-Weekly OSS Heron Sync-up

Hi All, I mainly worked on the Apache Release Candidate and reviewed a bunch of PRs from the community. On Mon, Nov 5, 2018 at 2:42 PM Eren Avsarogullari < erenavsarogull...@gmail.com> wrote: > Hi All, > > Please find my update as follows: > > PR #3095

Re: 11/20/2018 Bi-Weekly Heron Sync-up

Hi, We passed the voting process for our first Apache release - 0.20.0. And the source code release is available at https://dist.apache.org/repos/dist/release/incubator/heron/ Besides, I prepared a "Heron Stateful Processing" talk at this month's meetup. On Mon, Nov 19, 2018 at 10:05 AM Ning Wa

[RESULT] [VOTE] Apache Heron 0.20.0-incubating (RC5)

Hi all, With three +1 binding votes, three +1 non-binding votes and no -1 or +/-0 votes, this release vote *PASSED*! Heron is now ready for the first Apache release! Thanks to our mentors and contributors for helping us make it happen. Thanks to everyone who checked the release and voted for it.

[CVE-2018-11789] Apache Incubator Heron file access vulnerability

Severity: Important Vendor: The Apache Software Foundation Versions Affected: Heron 0.13.0 to 0.17.8 Description: When accessing the heron-ui webpage, people can modify the file paths outside of the current container to access any file on the host. Mitigation: All Heron users should upgrade to

Re: 3/12/2019 Bi-Weekly OSS Heron Sync-up

Hi All, My updates: - I'm finishing the process of a security vulnerability issue. On Mon, Mar 11, 2019 at 11:19 AM FatJ Love wrote: > My update > - Worked with OSS team about the March meetup > > On Mon, Mar 11, 2019 at 10:39 AM Ning Wang wrote: > > > Hi, all~, > > > > It has been two weeks s

7/16/2019 Bi-Weekly OSS Heron Sync-up

Hi, all~, It has been two weeks since our last sync! Let's share our works for the last two weeks in this thread. -- Best Regards, Neng

07/29/2019 Bi-Weekly OSS Heron Sync-up

Hi, all~, It has been two weeks since our last sync! Let's share our works for the last two weeks in this thread. my updates: 1. we made a new release internally and identified some stmgr performance degradation issue -- Best Regards, Neng

New committer: Xiaoyao Qian

. Best regards, Neng Lu

New committer: Siming Weng

committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. Being a PMC member enables assistance with the management and to guide the direction of the project. Best regards, Neng Lu