Re: ApacheCon

[Jerry Peng]

For those going to apache con, maybe we should pick a time to meet up?

Best,

Jerry

On Sat, Jun 15, 2019 at 11:24 PM FatJ Love  wrote:
>
> Congratulations !
>
> On Fri, Jun 14, 2019 at 11:10 PM Ning Wang  wrote:
>
> > Very nice! Bring some Heron stickers to there~. :D
> >
> > On Fri, Jun 14, 2019 at 7:47 PM Sree V 
> > wrote:
> >
> > > Congratulations, Josh, Dave and Jerry.
> > >
> > >
> > >
> > > Thank you./Sree650.213.2707 M (prefer text)
> > >
> > >
> > > On Friday, June 14, 2019, 4:47:13 PM PDT, Josh Fischer <
> > > j...@joshfischer.io> wrote:
> > >
> > >  Nice.  It would be good to finally meet in person.  See you soon!
> > >
> > > On Fri, Jun 14, 2019 at 5:10 PM Jerry Peng 
> > > wrote:
> > >
> > > > I have a talk accepted at Apache Con as well.  See you guys there!
> > > >
> > > > On Mon, Jun 10, 2019 at 1:31 PM FatJ Love 
> > > > wrote:
> > > > >
> > > > > Congrats!
> > > > >
> > > > > On Mon, Jun 10, 2019 at 11:16 AM Josh Fischer 
> > > > wrote:
> > > > >
> > > > > > Awesome Dave.  See you soon!
> > > > > >
> > > > > > On Mon, Jun 10, 2019 at 12:20 PM Dave Fisher <
> > dave2w...@comcast.net>
> > > > > > wrote:
> > > > > >
> > > > > > > I have a talk too!
> > > > > > >
> > > > > > > I’ll see you there!
> > > > > > >
> > > > > > > Regards,
> > > > > > > Dave
> > > > > > >
> > > > > > > Sent from my iPhone
> > > > > > >
> > > > > > > > On Jun 10, 2019, at 9:56 AM, Josh Fischer  > >
> > > > wrote:
> > > > > > > >
> > > > > > > > Hey All,
> > > > > > > >
> > > > > > > > I had a speaking proposal accepted for ApacheCon in Las Vegas
> > > this
> > > > year
> > > > > > > > about Heron.  If anyone is in the area, it would be cool to
> > meet
> > > > up.
> > > > > > > >
> > > > > > > > - Josh
> > > > > > >
> > > > > > >
> > > > > >
> > > >
> > > --
> > > Sent from A Mobile Device
> >

Re: ApacheCon

[Jerry Peng]

I have a talk accepted at Apache Con as well.  See you guys there!

On Mon, Jun 10, 2019 at 1:31 PM FatJ Love  wrote:
>
> Congrats!
>
> On Mon, Jun 10, 2019 at 11:16 AM Josh Fischer  wrote:
>
> > Awesome Dave.  See you soon!
> >
> > On Mon, Jun 10, 2019 at 12:20 PM Dave Fisher 
> > wrote:
> >
> > > I have a talk too!
> > >
> > > I’ll see you there!
> > >
> > > Regards,
> > > Dave
> > >
> > > Sent from my iPhone
> > >
> > > > On Jun 10, 2019, at 9:56 AM, Josh Fischer  wrote:
> > > >
> > > > Hey All,
> > > >
> > > > I had a speaking proposal accepted for ApacheCon in Las Vegas this year
> > > > about Heron.  If anyone is in the area, it would be cool to meet up.
> > > >
> > > > - Josh
> > >
> > >
> >

Re: [VOTE] Heron Release 0.20.0-incubating Candidate 5

[Jerry Peng]

I had a problem compiling:

Jerrys-MacBook-Pro:incubator-heron-v-0.20.0-incubating-candidate-5
jerrypeng$ bazel  build --config=darwin heron/...

*ERROR: 
*/private/tmp/incubator-heron-v-0.20.0-incubating-candidate-5/heron/spi/src/java/BUILD:6:1:
every rule of type java_doc implicitly depends upon the target
'@local_jdk//:jdk-default', but this target could not be found because of:
no such target '@local_jdk//:jdk-default': target 'jdk-default' not
declared in package '' (did you mean 'jre-default'?) defined by
/private/var/tmp/_bazel_jerrypeng/57e1f3641597b00bc727f301d71cb0d0/external/local_jdk/BUILD.bazel

*ERROR: *Analysis of target '//heron/spi/src/java:heron-spi-javadoc'
failed; build aborted: Analysis failed

INFO: Elapsed time: 0.784s

INFO: 0 processes.

*FAILED:* Build did NOT complete successfully (0 packages loaded)


Though I have a newer version of bazel (I think latest):

Jerrys-MacBook-Pro:incubator-heron-v-0.20.0-incubating-candidate-5
jerrypeng$ bazel version

Build label: 0.17.2-homebrew

Build target:
bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar

Build time: Fri Sep 28 10:42:37 2018 (1538131357)

Build timestamp: 1538131357

Build timestamp as int: 1538131357

On Thu, Oct 18, 2018 at 11:00 AM Ning Wang  wrote:

> +1
>
> Tests I have done:
> - compiled ok, all unit tests passed
> - installed CLI successfully
> - example topology runs fine locally
> - ui/tracker work fine and show topology info
> - license scan with apache-rat (rat couldn't find license in cloudpickle.py
> but it is there)
>
>
>
> On Tue, Oct 16, 2018 at 11:17 PM Ning Wang  wrote:
>
> > Thanks!
> >
> > I am going to try it when I get time.
> >
> > On Tue, Oct 16, 2018 at 10:45 AM Neng Lu  wrote:
> >
> >> Hi All,
> >>
> >> This is the 5th release candidate for Apache Heron, version
> >> 0.20.0-incubating. Thanks everyone for providing various feedback for
> the
> >> previous release candidates.
> >>
> >> It is the starting point of Heron and contains heron's main features,
> such
> >> as streaming
> >> processing, stateful processing, streamlet API, API server, eco support,
> >> etc.
> >>
> >> The full list of changes and fixes are available:
> >>
> >>
> https://github.com/apache/incubator-heron/compare/0.17.8...release/v-0.20.0-incubating
> >>
> >> *** Please download, test and vote on this release. This vote will stay
> >> open
> >> for at least 72 hours ***
> >>
> >> Source files:
> >>
> >>
> https://dist.apache.org/repos/dist/dev/incubator/heron/heron-0.20.0-incubating-candidate-5/
> >>
> >> SHA-512 checksums:
> >>
> >>
> 27890ab30fc3e69b627f47d58d178d1a7dffa9dbe4ebbb5a5aa77caaac882fdc2b6f98b3b76210020db0fa3fd86e294cba214f86072e449837e1b7615cd6124a
> >> incubator-heron-v-0.20.0-incubating-candidate-5.tar.gz
> >>
> >> The tag to be voted upon:
> >> v0.20.0-incubating-candidate-5
> (45043bb6dcef1e8089c0834f17f8be0cc3f451d3)
> >>
> >>
> https://github.com/apache/incubator-heron/releases/tag/v-0.20.0-incubating-candidate-5
> >>
> >> Please download the source package, and follow the compiling guide(
> >>
> >>
> https://apache.github.io/incubator-heron/docs/developers/compiling/compiling/
> >> )
> >> to build and run the Heron locally.
> >>
> >> --
> >> Best Regards,
> >> Neng
> >>
> >
>

Re: Heron OSS Sync

[Jerry Peng]

Sounds good.  I will attend the OSS sync meeting today

On Tue, Aug 14, 2018 at 11:22 AM Ning Wang  wrote:

> Hi,
>
> Sorry for the late notice The heron OSS sync meeting will be happening
> today at 2.00 pm PDT. Please use the following hangout link:
> https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync?authuser=0
>
>
> See you all then.
>

Moving to Apache

[Jerry Peng]

Hello all,

I just want to start an email thread discussing moving Heron to
Apache.  There are some items we need to figure out for this:

1. Moving the code to Apache github

I was told that an repo can be transferred to another account and
people have done this in the past to move to the Apache github
account.  This is the best way to move the code to be under apache
since with this method heron will keep all its stars and forks.

We need to start converting heron packages from com.twitter -> org.apache.

Ideally this whole process of migrating to Apache will not be a
blocker for development and releases.

Thus, if mentors or people with experience in this area want to chime
in on the exact details (step by step) of what needs to be do for
heron to be completely migrated to Apache that would be great!

2. Moving website to heron.apache.org

What do we want to do here?  Migrate the whole website to
heron.apache.org? And Have heron.io forward to heron.apache.org?

3.  How can we use apache infra

I think committers/mentors need to file some tickets to apache infra for this.

How can we use the apache infra to do apache release for heron?

Lets get the discussion going!

Thanks!

Jerry

OSS sync up today

[Jerry Peng]

Hello all,

Just a reminder this is the hangout link to the heron OSS sync up today:

https://plus.google.com/hangouts/_/streaml.io/oss-heron-sync?hceid=a2FydGhpa0BzdHJlYW1sLmlv.218m8hgr4ekqsf1o8of2olo1a0=0

Somethings on agenda for discussion today:

- pointing the heron.apache.org - github pages
- how to automatically do daily digest for slack and push into dev mailing lists
- can we use the apache infrastructure for it?

Best,

Jerry

Release Today/Tomorrow

[Jerry Peng]

We haven't done a release in a while.  Should we do a release today or tomorrow?

Best,

Jerry

Re: Proposing Changes To Heron

[Jerry Peng]

Thanks Josh for taking the initiative to get this start!  SQL on Heron
will be a great feature! The plan sounds great to me.  Lets first get
an initial version of the Heron SQL out and then we can worry about
custom / user defined sources and sinks.  We can even start talking
about UDFs (User defined functions) at that point!

Best,

Jerry

On Sun, Feb 25, 2018 at 9:05 PM, Josh Fischer  wrote:
> Please see this google drive link for adding comments.  I will copy and
> paste the drive doc below as well.
>
> https://docs.google.com/document/d/1PxLCyR_H-mOgPjyFj3DhWXryKW21CH2zFWwzTnqjfEA/edit?usp=sharing
>
>
> Proposal Below
>
>
>
>
>
>
>
> *I am writing this document to propose changes and to start conversations
> on adding functionality similar to Storm SQL to Heron.  We would call it
> Heron SQL.  After reviewing how the code is structured in Storm I have some
> suggestions and questions relating to the implementation into the Heron
> code base. - High Level Overview Of Code Workflow (Keeping Similar to
> Storm)- We would parse the sql with calcite to create the logical and
> physical plans- We would then convert the logical and physical plans to a
> Heron Topology- We would then submit the Heron Topology into the Heron
> System - Some thoughts on code structure and overall functionality- I think
>  we should place the Heron SQL code base as a top level directory in the
> repo. - I will have to add the command “sql” to the Heron command line code
> in python.- As a first pass implementation users  can interact with Heron
> SQL via the following command - heron sql  - We
> will also support the explain command for displaying the query plan, this
> will not deploy the topology- heron sql  --explain- After the
> first pass implementation is working smoothly, we can then add an
> interactive command line interface to accept sql on the fly by omitting the
> sql file argument- Heron sql - We would support all of the
> existing functionality in Storm SQL today with the exception of being
> dependent on trident.  We would use Storm SQL as a way to deploy topologies
> into Heron.  Similar to how you deploy topologies with the Streamlet,
> Topology, and ECO APIs- Questions- Do we see any issue with this plan to
> implement?- I believe we would have to supply an external jar at times to
> connect to external data sources, such as reuse of kafka libraries or
> database drivers.  I see that Storm has few external connectors for mongo,
> kafka, redis and hdfs.  Do we want to limit users to what we decide to
> build as connectors or do we want to give them the ability to load external
> jars at submit time? I don’t think heron offers the ability to pass extra
> jars to via the “--jars” or “--artifacts” flags like Storm does today.
> Would this be the correct way to pull in external jars?  Does anyone have a
> different idea?  I’m thinking that this might be a v2 feature after we get
> Heron sql working well.  Ideas, thoughts or concerns?- Is there anything I
> missed?*

Re: Release 0.17.1

[Jerry Peng]

sounds good

On Fri, Nov 17, 2017 at 4:06 PM, Karthik Ramasamy  wrote:
> Sounds good!
>
> On Fri, Nov 17, 2017 at 12:55 PM, Sanjeev Kulkarni 
> wrote:
>
>> Hi all,
>> In light of the critical bug fix(https://github.com/
>> twitter/heron/pull/2551),
>> might make sense to make a new release today. Any objections?
>>

Re: Migrating Heron code to Apache

[Jerry Peng]

As in terms of the order execution.  I am not a lawyer, but I think
you can still cut releases with packages in com.twitter even after in
it is in apache git.  In the early days of Apache Storm, all the Java
code was under org.backtype, however releases were still cut without a
problem.  If this is the case, I would suggest, we move all the code
to apache git first (step 3) and then we can work on the other changes
such as migrating the package names and etc.  By moving the project,
to apache git, Heron will look more like a legitimate ASF project
sooner.

I also have a question.  Do we need to include the twitter copyright
on existing files?  Many ASF projects that have been donated by
companies do not have a copyright in the header for the respective
companies.

On Sun, Oct 29, 2017 at 9:54 AM, Jerry Peng <jerry.boyang.p...@gmail.com> wrote:
> Hello all,
>
> I strongly agree with Bill on migrating the code base to org.apache.
> This will attract more developers and automatically fix issues
> involving copyrights and headers that we don't need to waste our time
> dealing with.  Heron, as an open source project, is in a state of
> limbo. Developers are confused to whether Heron is indeed in ASF or
> not, and this could dissuade potential developers from investing in
> the project.  There has been talk of Heron joining Apache for a very
> long time, and I think people are starting to wonder if there is
> something wrong with Heron that is still causing it to not officially
> look like an ASF project.  The most important thing to the success of
> an ASF, or any open source project, is the community around the
> project.  The longer we wait on this, more and more potential
> developers and contributors might turn away. Thus, I think we should
> push this through ASAP.
>
> Best,
>
> Jerry
>
> On Sun, Oct 29, 2017 at 7:41 AM, Jacques Nadeau <jacq...@apache.org> wrote:
>> I'm -0 on plan.
>>
>> Why not just import code then do changes 1 and 2 after 3? Just seems like
>> getting 3 done is a key blocking item on forward progress of the community.
>>
>> On Oct 27, 2017 3:16 PM, "Sanjeev Kulkarni" <sanjee...@gmail.com> wrote:
>>
>>> +1
>>>
>>> On Thu, Oct 26, 2017 at 9:15 AM, Bill Graham <billgra...@gmail.com> wrote:
>>>
>>> > Any other comments on this proposal from Heron developers? The next
>>> podling
>>> > report is due on Wednesday so we should address our plan.
>>> >
>>> > On Sat, Oct 21, 2017 at 3:38 PM, John D. Ament <johndam...@apache.org>
>>> > wrote:
>>> >
>>> > > If you do in fact want to use gitbox (which allows you to have github
>>> > > writable repos), infra will need to be made an admin on your
>>> organization
>>> > > temporarily to do the migration.
>>> > >
>>> > > Many new projects are doing this, so it's not uncommon to just use
>>> gitbox
>>> > > since you're already on github.
>>> > >
>>> > > John
>>> > >
>>> > > On 2017-10-19 13:20, Brian Hatfield <bmhatfi...@gmail.com> wrote:
>>> > > > Thank you both for the info :-) I had not realized it would just be
>>> > > > re-homed in a different Github org. Thanks!
>>> > > >
>>> > > > Brian
>>> > > >
>>> > > > On Thu, Oct 19, 2017 at 11:58 AM, Bill Graham <billgra...@gmail.com>
>>> > > wrote:
>>> > > >
>>> > > > > Right, it would still be on github, just at apache/heron instead of
>>> > > > > twitter/heron.
>>> > > > >
>>> > > > > On Thu, Oct 19, 2017 at 8:16 AM Jake Farrell <jfarr...@apache.org>
>>> > > wrote:
>>> > > > >
>>> > > > > > Apache git can also refer to the Github Apache org as a number of
>>> > > > > projects
>>> > > > > > are running in that fashion. They key is that the code has been
>>> > > imported
>>> > > > > > over to the Apache Infra owned/managed infrastructure
>>> > > > > >
>>> > > > > > -Jake
>>> > > > > >
>>> > > > > > On Thu, Oct 19, 2017 at 11:04 AM, Brian Hatfield <
>>> > > bmhatfi...@gmail.com>
>>> > > > > > wrote:
>>> > > > > >
>>> > > > > >> Silly question - and apologies if this has already been
>>>

Re: Migrating Heron code to Apache

[Jerry Peng]

Hello all,

I strongly agree with Bill on migrating the code base to org.apache.
This will attract more developers and automatically fix issues
involving copyrights and headers that we don't need to waste our time
dealing with.  Heron, as an open source project, is in a state of
limbo. Developers are confused to whether Heron is indeed in ASF or
not, and this could dissuade potential developers from investing in
the project.  There has been talk of Heron joining Apache for a very
long time, and I think people are starting to wonder if there is
something wrong with Heron that is still causing it to not officially
look like an ASF project.  The most important thing to the success of
an ASF, or any open source project, is the community around the
project.  The longer we wait on this, more and more potential
developers and contributors might turn away. Thus, I think we should
push this through ASAP.

Best,

Jerry

On Sun, Oct 29, 2017 at 7:41 AM, Jacques Nadeau  wrote:
> I'm -0 on plan.
>
> Why not just import code then do changes 1 and 2 after 3? Just seems like
> getting 3 done is a key blocking item on forward progress of the community.
>
> On Oct 27, 2017 3:16 PM, "Sanjeev Kulkarni"  wrote:
>
>> +1
>>
>> On Thu, Oct 26, 2017 at 9:15 AM, Bill Graham  wrote:
>>
>> > Any other comments on this proposal from Heron developers? The next
>> podling
>> > report is due on Wednesday so we should address our plan.
>> >
>> > On Sat, Oct 21, 2017 at 3:38 PM, John D. Ament 
>> > wrote:
>> >
>> > > If you do in fact want to use gitbox (which allows you to have github
>> > > writable repos), infra will need to be made an admin on your
>> organization
>> > > temporarily to do the migration.
>> > >
>> > > Many new projects are doing this, so it's not uncommon to just use
>> gitbox
>> > > since you're already on github.
>> > >
>> > > John
>> > >
>> > > On 2017-10-19 13:20, Brian Hatfield  wrote:
>> > > > Thank you both for the info :-) I had not realized it would just be
>> > > > re-homed in a different Github org. Thanks!
>> > > >
>> > > > Brian
>> > > >
>> > > > On Thu, Oct 19, 2017 at 11:58 AM, Bill Graham 
>> > > wrote:
>> > > >
>> > > > > Right, it would still be on github, just at apache/heron instead of
>> > > > > twitter/heron.
>> > > > >
>> > > > > On Thu, Oct 19, 2017 at 8:16 AM Jake Farrell 
>> > > wrote:
>> > > > >
>> > > > > > Apache git can also refer to the Github Apache org as a number of
>> > > > > projects
>> > > > > > are running in that fashion. They key is that the code has been
>> > > imported
>> > > > > > over to the Apache Infra owned/managed infrastructure
>> > > > > >
>> > > > > > -Jake
>> > > > > >
>> > > > > > On Thu, Oct 19, 2017 at 11:04 AM, Brian Hatfield <
>> > > bmhatfi...@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > >> Silly question - and apologies if this has already been
>> discussed
>> > -
>> > > but
>> > > > > is
>> > > > > >> #3 (Migrate the code to Apache git) required? From my
>> perspective
>> > > Github
>> > > > > >> is
>> > > > > >> much more preferable as it's where nearly every other open
>> source
>> > > > > codebase
>> > > > > >> I interact with is, and the UI is very friendly to newcomers.
>> > > > > >>
>> > > > > >> On Thu, Oct 19, 2017 at 11:01 AM, Bill Graham <
>> > billgra...@gmail.com
>> > > >
>> > > > > >> wrote:
>> > > > > >>
>> > > > > >> > Hi,
>> > > > > >> >
>> > > > > >> > In LEGAL-339 > >
>> > > it was
>> > > > > >> > concluded that we can in fact move the code to Apache git and
>> > cut
>> > > > > Apache
>> > > > > >> > releases without the SGA. I propose we move forward on that. I
>> > > suggest
>> > > > > >> the
>> > > > > >> > following plan:
>> > > > > >> >
>> > > > > >> > 1.a Refactor all Heron build dependencies (mainly c++ libs) to
>> > be
>> > > > > >> fetched
>> > > > > >> > at build time and not committed in the repo. (#2092
>> > > > > >> > )
>> > > > > >> > 1.b Refactor the bazel checkstyles to support both the Twitter
>> > > > > copyright
>> > > > > >> > (for existing code) and the Apache copyright (for new code
>> after
>> > > the
>> > > > > >> > migration).
>> > > > > >> > 2. Cut the last non-Apache release.
>> > > > > >> > 3. Migrate the code to Apache git
>> > > > > >> > 4. Add incubation disclaimer
>> > > > > >> > 5. Cut the first Apache release.
>> > > > > >> >
>> > > > > >> > What do folks think of that plan? Item's 1a and 1b can happen
>> in
>> > > > > >> parallel,
>> > > > > >> > as could item 2 actually. There will surely be more smaller
>> > > items, but
>> > > > > >> > those are the big ones as I see it. Please chime in if I've
>> > > overlooked
>> > > > > >> > anything major.
>> > > > > >> >
>> > > > > >> > thanks,
>> > > > > >> > Bill
>> > > > > >> >
>> > > > > >>
>> > > > > >
>> > >