Re: Incubator Governance Change

[Joe Schaefer]

Continuing down the road of blaming each other for the problem is stupid.
Look the personnel is already ready, willing, and available to do the real
vetting.
All the IPMC has to do is recognize them and integrate them.

On Wed, Apr 26, 2017 at 1:37 AM, Bertrand Delacretaz <
bdelacre...@codeconsult.ch> wrote:

> On Wed, Apr 26, 2017 at 5:47 AM, P. Taylor Goetz 
> wrote:
> > ...I mostly agree, but would hesitate with the idea that a podling be
> responsible for raising a red flag
> > when mentors are inactive. If the IPMC isn’t doing it’s job, that’s on
> us, not the projects we incubate
>
> The IPMC will typically not know before it's too late, so it's good
> for the podlings to raise those red flags - and ask the IPMC for help.
>
> -Bertrand
>
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>

Re: Incubator Governance Change

[Bertrand Delacretaz]

On Wed, Apr 26, 2017 at 5:47 AM, P. Taylor Goetz  wrote:
> ...I mostly agree, but would hesitate with the idea that a podling be 
> responsible for raising a red flag
> when mentors are inactive. If the IPMC isn’t doing it’s job, that’s on us, 
> not the projects we incubate

The IPMC will typically not know before it's too late, so it's good
for the podlings to raise those red flags - and ask the IPMC for help.

-Bertrand

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Incubator Governance Change

[Roman Shaposhnik]

On Tue, Apr 25, 2017 at 8:47 PM, P. Taylor Goetz  wrote:
>
>> On Apr 25, 2017, at 9:06 PM, Roman Shaposhnik  wrote:
>>
>> I honestly think its time for us to consider being more board like
>> around here at
>> IPMC. May be not exactly as strict as the board, but moving in that 
>> direction.
>>
>> After all, being a podling means being TLP-on-training-wheels. That's the 
>> whole
>> idea. If your PMC shrinks -- you have to know what it means. Same here. If 
>> your
>> mentors are not active -- you've got to raise hell around here on general@
>> before its too late.
>
> I mostly agree, but would hesitate with the idea that a podling be 
> responsible for
> raising a red flag when mentors are inactive. If the IPMC isn’t doing it’s 
> job,
> that’s on us, not the projects we incubate.

I am not saying IPMC is off the hook here, but rather I'm trying to instill
a sense of self-reliance in a future TLP.

While PPMC is mostly an artificial construct from the ASF's point of view,
the idea there is to teach podling how the TLP PMC should behave. One
of the traits of this behaviour is to recognize problems within the community
and try to resolve them instead of semi-passively waiting for either board
or IPMC to intervene.

Again, I'm not advocating for shifting the responsibility as much as
I'm advocating
for shift in perspective. I would still expect the IPMC members to be watchful,
but I would also expect them to use every opportunity like that as a teaching
moment. E.g. "oh, so we've noticed you're struggling with mentors being MIA,
we're sorry about that, but have you considered escalating to IPMC and perhaps
soliciting additional mentors?" instead of "oh, so we've noticed
you're struggling
with mentors being MIA, let us fix this for you" (I'm obviously
exaggerating, but you
catch my drift).

Thanks,
Roman.

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Airflow voting on release artifacts

[Alex Harui]

>
>3. There is no “separate” build script. Pip will just install a binary
>(“wheel”) or uses the source package (as shown above). Both are used
>interchangeable by users. We only distribute source packages at the
>moment.
>
>@Alex: I have to think a little bit more about what you wrote, but it is
>currently confusing the hell out of me :). Furthermore, I am not sure if
>it applies considering the above #3.

It could still apply.  You would just have to add a build script that
renames the package and metadata.

Let's say I wanted to release a single file that reported the version
number.  Forgive me that I don't know Python so I just grabbed what I
think it would look like from the internet.

---MyRelease.py---
print("I am version 1.2.3.")

Let's assume this is what your "customers" want to use.

I am proposing that the Apache Source Package also contain the following
file:

---BuildScript.sh---
# creates Customer Package in out folder.
mkdir out
sed s/1.2.3/1.2.3$1/ < MyRelease.py > out/MyRelease$1.py

Voters would run:

   BuildScript.sh RC1

That would result in:

---out/MyReleaseRC1.py---
print("I am version 1.2.3RC1.")

And this version would be tested by the voters.  The source package being
voted on contains the original MyRelease.py and BuildScript.sh.  The
release manager would also run:


BuildScript.sh

That would result in:

---out/MyRelease.py---
print("I am version 1.2.3.")

In our project, the RM posts the source package in the RC folder and
creates a folder called "binaries" for the compiled source.  You could
call the folder something else, but let's keep the names for now.  The RM
would copy a zip of MyRelease.py and BuildScript.sh (and LICENSE, NOTICE,
README) into the RC folder and out/MyRelease.py to the "binaries" folder.
Along with signatures and checksum files.


Voters would download the zip, expand it, run "BuildScript.sh RC1" and
test with their out/MyReleaseRC1.py.  They would examine the zip to make
sure it is compliant with Apache release policy.  This is what all other
voters on all other projects generally do.  But they would perform one
different step, which is, instead of testing the MyRelease.py in the
"binaries" folder, they would simply diff their MyReleaseRC1.py against
the MyRelease.py in the "binaries" folder.  If the only diffs are the
version, they should feel satisfied that the resulting "customer" packages
is ok for release.

Of course, I could be wrong...

HTH,
-Alex

Re: Incubator Governance Change

[P. Taylor Goetz]

> On Apr 25, 2017, at 9:06 PM, Roman Shaposhnik  wrote:
> 
> I honestly think its time for us to consider being more board like
> around here at
> IPMC. May be not exactly as strict as the board, but moving in that direction.
> 
> After all, being a podling means being TLP-on-training-wheels. That's the 
> whole
> idea. If your PMC shrinks -- you have to know what it means. Same here. If 
> your
> mentors are not active -- you've got to raise hell around here on general@
> before its too late.

I mostly agree, but would hesitate with the idea that a podling be responsible 
for raising a red flag when mentors are inactive. If the IPMC isn’t doing it’s 
job, that’s on us, not the projects we incubate.

-Taylor

Re: Incubator Governance Change

[Roman Shaposhnik]

On Sun, Apr 23, 2017 at 4:50 AM, Greg Stein  wrote:
> On Sat, Apr 22, 2017 at 7:46 PM, Roman Shaposhnik 
> wrote:
>>...
>
>> I'm starting to wonder whether the real solution here should be along the
>> lines
>> of what a board would do to a TLP if its active PMC shrinks to less
>> than 3 people.
>>
>
> Oh, that's easy, and has been done quite a few times. The Board moves them
> to the Attic. ... The Board will give the PMC several chances, and this can
> drag out for a year, but at the end of the day, the Board says "enough" and
> forces the issue. Done. Attic'd.

That's what I was implying.

> Now, I'm not sure if you were trying to draw an analogy between Board/PMC
> and IPMC/PPMC. From a legal/oversight perspective, these are very
> different. I don't think the IPMC should be so drastic. But the Board must,
> and will.

I honestly think its time for us to consider being more board like
around here at
IPMC. May be not exactly as strict as the board, but moving in that direction.

After all, being a podling means being TLP-on-training-wheels. That's the whole
idea. If your PMC shrinks -- you have to know what it means. Same here. If your
mentors are not active -- you've got to raise hell around here on general@
before its too late.

Thanks,
Roman.

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [PROPOSAL] Superset Proposal for Apache Incubator

[P. Taylor Goetz]

Hi Jeff,

With the change to the proposal on the wiki, it now differs from the proposal 
text in the VOTE email. The reason we include the proposal text in the VOTE 
thread is because it represents the state of the proposal (which being a wiki 
page, is inherently mutable) at the time of the VOTE.

It's largely technical/procedural, but I would recommend cancelling the current 
vote and let the DISCUSS thread play out a little longer.

Not that calling a VOTE was wrong or premature, the thread had died down. But 
oftentimes the actual VOTE thread is what kicks IPMC members toward taking a 
closer look at a proposal. (We, the IPMC, are an imperfect group of volunteers. 
:) )

-Taylor



> On Apr 25, 2017, at 3:17 AM, Jeff Feng  wrote:
> 
> Thanks John and Max - I have updated the proposal wiki to reflect this
> update.  It now reads:
> 
> Source and Intellectual Property Submission Plan
> 
> Airbnb will submit a Software Grant Agreement (SGA) as Superset joins the
> incubator. We do not expect any complications for the submission of the
> Superset code base. Our code is already in Github and there is only a
> single code base.
> 
> 
> On Mon, Apr 24, 2017 at 11:32 PM, Maxime Beauchemin <
> maximebeauche...@gmail.com> wrote:
> 
>> "Airbnb will submit a Software Grant Agreement (SGA) as Superset joins the
>> incubator."
>> 
>> Should I add this sentence in the proposal?
>> 
>> Max
>> 
>> On Mon, Apr 24, 2017 at 5:48 AM, John D. Ament 
>> wrote:
>> 
>>> I missed this discussion.  In your IP section, you list out:
>>> 
>>> == Source and Intellectual Property Submission Plan ==
>>> We do not expect any complications for the submission of the Superset
>> code
>>> base.  Our code is already in Github and there is only a single code
>> base.
>>> 
>>> This IMHO not clear.  Does Airbnb plan to submit a SGA for Superset, or
>>> expect that no SGA is required because its Apache licensed?
>>> 
>>> John
>>> 
>>> On Sun, Apr 2, 2017 at 4:09 PM Jeff Feng 
>>> wrote:
>>> 
 Dear Apache Incubator Community,
 
 We are excited to share our proposal for discussion and feedback for
 entering Apache Incubation.  Superset is an enterprise-ready web
 application for data exploration, data visualization and dashboarding.
 
 Our Incubation proposal is at the following Wiki as well as copied in
>> the
 email below:
 
 https://wiki.apache.org/incubator/SupersetProposal
 
 We have an active Superset community including 400+ members and nearly
>>> 200
 topics.  The Google Group can be found below.  We plan to move the
 discussion to the ASF:
 
 https://groups.google.com/forum/#!forum/airbnb_superset
 
 Thank you and look forward to the discussion!
 
 Jeff, Max & Alanna
 
 
 
 
 = Superset =
 
 == Abstract ==
 
 Superset is an enterprise-ready web application for data exploration,
>>> data
 visualization and dashboarding.
 
 == Proposal ==
 
 Superset is business intelligence (BI) software that helps modern
 organizations visualize and interact with their data. Superset enables
 users explore data from a variety of databases, assemble beautiful
 dashboards and share their findings.  Superset works neatly with all
>>> modern
 SQL-speaking databases, and integrates with Druid.io to provide
>>> real-time,
 interactive, blazing fast data access to large datasets.
 
 == Background ==
 
 Data is mission critical. To succeed in this era, organizations need to
 provide low-friction, intuitive and interactive access to data. It is
 paramount for knowledge workers to be capable of answering their own
 questions by querying, exploring and visualizing data.
 
 The entire business intelligence industry has pivoted from a model of
 centralized top-down platforms driven by IT organizations to
>> self-service
 analytics and agile workflows by any user.  This shift unblocks
>>> centralized
 service bottlenecks for creating data visualizations while also
>> creating
>>> an
 environment that is iterative and fast-moving.  This means that
>> business
 intelligence software must also be easy and delightful to use.
 Self-service analytics doesn’t mean that admin and governance features
>>> are
 not needed.
 
 Modern BI tools provide fine-grain access controls and auditing
 capabilities to understand how data is being used.  Superset is a
>>> solution
 that delivers on all of these vectors.
 
 The technology stack is also constantly morphing - vendors are
>> struggling
 to provide cheap, quick and easy solutions to access data.  Business
 intelligence users are finding existing solutions lacking as these
>>> software
 products either disregard or react slowly to recent game-changing
 technologies like Druid.io, PrestoDB, Apache 

Re: [PROPOSAL] Superset Proposal for Apache Incubator

[Maxime Beauchemin]

Hi Raphael, what kind of numbers are you expecting to see?

Max

On Tue, Apr 25, 2017 at 12:59 PM, Raphael Bircher 
wrote:

> Hi all
>
> There is no information about affiliation of the initial committers. The
> only information is, that they are from Airbnb inc and Hortonworks. But
> there are no numbers.
>
> Regards Raphael
>
>
> Am .04.2017, 09:17 Uhr, schrieb Jeff Feng :
>
> Thanks John and Max - I have updated the proposal wiki to reflect this
>> update.  It now reads:
>>
>> Source and Intellectual Property Submission Plan
>>
>> Airbnb will submit a Software Grant Agreement (SGA) as Superset joins the
>> incubator. We do not expect any complications for the submission of the
>> Superset code base. Our code is already in Github and there is only a
>> single code base.
>>
>>
>> On Mon, Apr 24, 2017 at 11:32 PM, Maxime Beauchemin <
>> maximebeauche...@gmail.com> wrote:
>>
>> "Airbnb will submit a Software Grant Agreement (SGA) as Superset joins the
>>> incubator."
>>>
>>> Should I add this sentence in the proposal?
>>>
>>> Max
>>>
>>> On Mon, Apr 24, 2017 at 5:48 AM, John D. Ament 
>>> wrote:
>>>
>>> > I missed this discussion.  In your IP section, you list out:
>>> >
>>> > == Source and Intellectual Property Submission Plan ==
>>> > We do not expect any complications for the submission of the Superset
>>> code
>>> > base.  Our code is already in Github and there is only a single code
>>> base.
>>> >
>>> > This IMHO not clear.  Does Airbnb plan to submit a SGA for Superset, or
>>> > expect that no SGA is required because its Apache licensed?
>>> >
>>> > John
>>> >
>>> > On Sun, Apr 2, 2017 at 4:09 PM Jeff Feng >> >
>>> > wrote:
>>> >
>>> > > Dear Apache Incubator Community,
>>> > >
>>> > > We are excited to share our proposal for discussion and feedback for
>>> > > entering Apache Incubation.  Superset is an enterprise-ready web
>>> > > application for data exploration, data visualization and
>>> dashboarding.
>>> > >
>>> > > Our Incubation proposal is at the following Wiki as well as copied in
>>> the
>>> > > email below:
>>> > >
>>> > > https://wiki.apache.org/incubator/SupersetProposal
>>> > >
>>> > > We have an active Superset community including 400+ members and
>>> nearly
>>> > 200
>>> > > topics.  The Google Group can be found below.  We plan to move the
>>> > > discussion to the ASF:
>>> > >
>>> > > https://groups.google.com/forum/#!forum/airbnb_superset
>>> > >
>>> > > Thank you and look forward to the discussion!
>>> > >
>>> > > Jeff, Max & Alanna
>>> > >
>>> > >
>>> > >
>>> > >
>>> > > = Superset =
>>> > >
>>> > > == Abstract ==
>>> > >
>>> > > Superset is an enterprise-ready web application for data exploration,
>>> > data
>>> > > visualization and dashboarding.
>>> > >
>>> > > == Proposal ==
>>> > >
>>> > > Superset is business intelligence (BI) software that helps modern
>>> > > organizations visualize and interact with their data. Superset
>>> enables
>>> > > users explore data from a variety of databases, assemble beautiful
>>> > > dashboards and share their findings.  Superset works neatly with all
>>> > modern
>>> > > SQL-speaking databases, and integrates with Druid.io to provide
>>> > real-time,
>>> > > interactive, blazing fast data access to large datasets.
>>> > >
>>> > > == Background ==
>>> > >
>>> > > Data is mission critical. To succeed in this era, organizations need
>>> to
>>> > > provide low-friction, intuitive and interactive access to data. It is
>>> > > paramount for knowledge workers to be capable of answering their own
>>> > > questions by querying, exploring and visualizing data.
>>> > >
>>> > > The entire business intelligence industry has pivoted from a model of
>>> > > centralized top-down platforms driven by IT organizations to
>>> self-service
>>> > > analytics and agile workflows by any user.  This shift unblocks
>>> > centralized
>>> > > service bottlenecks for creating data visualizations while also
>>> creating
>>> > an
>>> > > environment that is iterative and fast-moving.  This means that
>>> business
>>> > > intelligence software must also be easy and delightful to use.
>>> > > Self-service analytics doesn’t mean that admin and governance
>>> features
>>> > are
>>> > > not needed.
>>> > >
>>> > > Modern BI tools provide fine-grain access controls and auditing
>>> > > capabilities to understand how data is being used.  Superset is a
>>> > solution
>>> > > that delivers on all of these vectors.
>>> > >
>>> > > The technology stack is also constantly morphing - vendors are
>>> struggling
>>> > > to provide cheap, quick and easy solutions to access data.  Business
>>> > > intelligence users are finding existing solutions lacking as these
>>> > software
>>> > > products either disregard or react slowly to recent game-changing
>>> > > technologies like Druid.io, PrestoDB, Apache Drill, Apache Kylin,
>>> d3.js,
>>> > > React.js and iPython’s Jupyter for instance.

[ANNOUNCE] Apache PredictionIO 0.11.0-incubating Release

[Donald Szeto]

The Apache PredictionIO team would like to announce the release of Apache
PredictionIO 0.11.0-incubating.

Release notes are here:
https://github.com/apache/incubator-predictionio/blob/v0.11.0-incubating/RELEASE.md

Apache PredictionIO (incubating) is an open source Machine Learning Server
built on top of state-of-the-art open source stack, that enables developers
to manage and deploy production-ready predictive services for various kinds
of machine learning tasks.

More details regarding Apache PredictionIO (incubating) can be found here:
http://predictionio.incubator.apache.org/

The release artifacts can be downloaded here:
https://dist.apache.org/repos/dist/release/incubator/predictionio/0.11.0-incubating/

All JIRAs completed for this release are tagged with 'FixVersion =
0.11.0-incubating'; the JIRA release notes can be found here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320420=12338381

Thanks!
The Apache PredictionIO Team

DISCLAIMER
Apache PredictionIO (incubating) is an effort undergoing incubation at the
Apache Software Foundation (ASF), sponsored by the Apache Incubator PMC.
Incubation is required of all newly accepted projects until a further
review indicates that the infrastructure, communications, and decision
making process have stabilized in a manner consistent with other successful
ASF projects. While incubation status is not necessarily a reflection of
the completeness or stability of the code, it does indicate that the
project has yet to be fully endorsed by the ASF.

Re: Airflow voting on release artifacts

[Jim Apple]

Would virtualenv help by avoiding the "Requirement already satisfied"
complaints?

On Tue, Apr 25, 2017 at 2:05 PM, Bolke de Bruin  wrote:

> Thank you thinking with us, that’s really appreciated. Unfortunately, many
> of the assumptions from the Java world do not apply to the Python world:
>
> Version information inside the artefact needs to be in sync with the
> external filename.
>
> Examples:
>
> 1. Plain rename of the tar ball
>
> bolke$ pip install airflow-1.8.0rc11+apache.incubating.tar.gz
> Processing ./airflow-1.8.0rc11+apache.incubating.tar.gz
>   Requirement already satisfied (use --upgrade to upgrade):
> airflow==1.8.0rc5+apache.incubating from file:///Users/bolke/Documents/
> dev/airflow/dist/airflow-1.8.0rc11%2Bapache.incubating.tar.gz in
> /Users/bolke/Documents/dev/airflow_env/lib/python2.7/site-packages
>
> Note that although RC11 is mentioned RC5 is contained in the package and
> is not upgraded.
>
> 2. Rename of the package inside the tar ball
> bolke$ mv airflow-1.8.0rc5+apache.incubating airflow-1.8.0rc11+apache.
> incubating
> bolke$ tar cfz ../airflow-1.8.0rc11+apache.incubating.tar.gz
> airflow-1.8.0rc11+apache.incubating/
>
> bolke$ pip install airflow-1.8.0rc11+apache.incubating.tar.gz
> Processing ./airflow-1.8.0rc11+apache.incubating.tar.gz
>   Requirement already satisfied (use --upgrade to upgrade):
> airflow==1.8.0rc5+apache.incubating from file:///Users/bolke/Documents/
> dev/airflow/dist/airflow-1.8.0rc11%2Bapache.incubating.tar.gz in
> /Users/bolke/Documents/dev/airflow_env/lib/python2.7/site-packages
>
> 3. There is no “separate” build script. Pip will just install a binary
> (“wheel”) or uses the source package (as shown above). Both are used
> interchangeable by users. We only distribute source packages at the moment.
>
> @Alex: I have to think a little bit more about what you wrote, but it is
> currently confusing the hell out of me :). Furthermore, I am not sure if it
> applies considering the above #3.
>
> At the moment we seem stuck between a rock and a hard place. And as we
> would like to release a new version pretty soon, we probably want to vote
> twice for now, and including the IPMC, four times. Hopefully we can have a
> vote in rapid succession :-).
>
> Bolke
>
>
> > On 25 Apr 2017, at 17:33, Alex Harui  wrote:
> >
> >
> >
> > On 4/25/17, 1:43 AM, "Bolke de Bruin" > wrote:
> >
> >> Hi Bertrand,
> >>
> >>> On 25 Apr 2017, at 09:04, Bertrand Delacretaz
> >>>  wrote:
> >>>
> >>> Hi,
> >>>
> >>> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini
> >>>  wrote:
>  ...Hitesh recently raised the issue that the artifact that passes the
>  vote
>  MUST be the one that we actually release...
> >>>
> >>> Yes in terms of having the same binary digests and signatures, but
> >>> renaming the files is fine IMO, especially for removing an -rc suffix
> >>> which makes total sense. I would just add that step to your release
> >>> process documentation to make it clear.
> >>>
>  ...Rename/rebuild after final vote (This is what Airflow is doing, and
>  Beam
>  does this, I believe)...
> >>>
> >>> I'd say rename yes but rebuild no, in order to keep the same digests
> >>> and signatures.
> >>>
> >>
> >> As mentioned earlier, that seems not to be possible. The metadata
> >> (filename) and version information inside the package need to be in
> sync.
> >> This how the python build tools and python ecosystem works.
> >
> > I'm not familiar with Python, but is it possible to have a command line
> > option that adds the "-rc" suffix in the right places?
> >
> > IMO, there are two "audiences".  One is the voters, and one is your
> > customers.  The customers should not be using the RC until after it is
> > approved unless they are volunteering to be a voter.  Voters are
> primarily
> > supposed to examine a source artifact, but if you also release a
> > "convenience binary" artifact, there are some examinations of that
> > "binary" artifact required.  For many projects this "convenience"
> artifact
> > is the one that the vast majority of customers consume.  AIUI, Python
> > doesn't have binaries, but IMO, there is no requirement that this
> > "convenience" artifact actually contains binaries.  A "convenience"
> > artifact is just supposed to be the source artifact processed by a build
> > script since many of your customer may not have, or may not want to
> > configure their machines to run whatever build script engine you choose
> > for your project.  Further, there is no requirement that I know of that
> > voters must test the "convenience" artifact by actually running it.  It
> > just makes sense to do so in most cases.  But if Python is interpreted
> > source, you may be able to use this to your advantage.
> >
> > So, your source package should consist of the source and build script
> > required to build this "customer"/"convenience" 

Re: Airflow voting on release artifacts

[Bolke de Bruin]

Thank you thinking with us, that’s really appreciated. Unfortunately, many of 
the assumptions from the Java world do not apply to the Python world:

Version information inside the artefact needs to be in sync with the external 
filename. 

Examples:

1. Plain rename of the tar ball

bolke$ pip install airflow-1.8.0rc11+apache.incubating.tar.gz
Processing ./airflow-1.8.0rc11+apache.incubating.tar.gz
  Requirement already satisfied (use --upgrade to upgrade): 
airflow==1.8.0rc5+apache.incubating from 
file:///Users/bolke/Documents/dev/airflow/dist/airflow-1.8.0rc11%2Bapache.incubating.tar.gz
 in /Users/bolke/Documents/dev/airflow_env/lib/python2.7/site-packages

Note that although RC11 is mentioned RC5 is contained in the package and is not 
upgraded.

2. Rename of the package inside the tar ball
bolke$ mv airflow-1.8.0rc5+apache.incubating airflow-1.8.0rc11+apache.incubating
bolke$ tar cfz ../airflow-1.8.0rc11+apache.incubating.tar.gz 
airflow-1.8.0rc11+apache.incubating/

bolke$ pip install airflow-1.8.0rc11+apache.incubating.tar.gz
Processing ./airflow-1.8.0rc11+apache.incubating.tar.gz
  Requirement already satisfied (use --upgrade to upgrade): 
airflow==1.8.0rc5+apache.incubating from 
file:///Users/bolke/Documents/dev/airflow/dist/airflow-1.8.0rc11%2Bapache.incubating.tar.gz
 in /Users/bolke/Documents/dev/airflow_env/lib/python2.7/site-packages

3. There is no “separate” build script. Pip will just install a binary 
(“wheel”) or uses the source package (as shown above). Both are used 
interchangeable by users. We only distribute source packages at the moment.

@Alex: I have to think a little bit more about what you wrote, but it is 
currently confusing the hell out of me :). Furthermore, I am not sure if it 
applies considering the above #3.

At the moment we seem stuck between a rock and a hard place. And as we would 
like to release a new version pretty soon, we probably want to vote twice for 
now, and including the IPMC, four times. Hopefully we can have a vote in rapid 
succession :-).

Bolke


> On 25 Apr 2017, at 17:33, Alex Harui  wrote:
> 
> 
> 
> On 4/25/17, 1:43 AM, "Bolke de Bruin"  > wrote:
> 
>> Hi Bertrand,
>> 
>>> On 25 Apr 2017, at 09:04, Bertrand Delacretaz
>>>  wrote:
>>> 
>>> Hi,
>>> 
>>> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini
>>>  wrote:
 ...Hitesh recently raised the issue that the artifact that passes the
 vote
 MUST be the one that we actually release...
>>> 
>>> Yes in terms of having the same binary digests and signatures, but
>>> renaming the files is fine IMO, especially for removing an -rc suffix
>>> which makes total sense. I would just add that step to your release
>>> process documentation to make it clear.
>>> 
 ...Rename/rebuild after final vote (This is what Airflow is doing, and
 Beam
 does this, I believe)...
>>> 
>>> I'd say rename yes but rebuild no, in order to keep the same digests
>>> and signatures.
>>> 
>> 
>> As mentioned earlier, that seems not to be possible. The metadata
>> (filename) and version information inside the package need to be in sync.
>> This how the python build tools and python ecosystem works.
> 
> I'm not familiar with Python, but is it possible to have a command line
> option that adds the "-rc" suffix in the right places?
> 
> IMO, there are two "audiences".  One is the voters, and one is your
> customers.  The customers should not be using the RC until after it is
> approved unless they are volunteering to be a voter.  Voters are primarily
> supposed to examine a source artifact, but if you also release a
> "convenience binary" artifact, there are some examinations of that
> "binary" artifact required.  For many projects this "convenience" artifact
> is the one that the vast majority of customers consume.  AIUI, Python
> doesn't have binaries, but IMO, there is no requirement that this
> "convenience" artifact actually contains binaries.  A "convenience"
> artifact is just supposed to be the source artifact processed by a build
> script since many of your customer may not have, or may not want to
> configure their machines to run whatever build script engine you choose
> for your project.  Further, there is no requirement that I know of that
> voters must test the "convenience" artifact by actually running it.  It
> just makes sense to do so in most cases.  But if Python is interpreted
> source, you may be able to use this to your advantage.
> 
> So, your source package should consist of the source and build script
> required to build this "customer"/"convenience" package, and the build
> script should allow a command line option to add the "-rc" suffix.  Then
> voters would be instructed to download the source package, and to test it,
> build a "customer" package with the "-rc" suffix and test the results.
> And voters would also be instructed to download the "customer" package
> that 

Re: [VOTE] Superset Proposal for Apache Incubator

[Joe Witt]

+1 (binding)

On Tue, Apr 25, 2017 at 4:52 PM, Jitendra Pandey
 wrote:
> +1 (binding)
>
> On 4/25/17, 1:27 PM, "Julian Hyde"  wrote:
>
> +1 binding
>
> > On Apr 25, 2017, at 12:48 PM, moon soo Lee  wrote:
> >
> > +1 (non-binding)
> >
> > On Tue, Apr 25, 2017 at 11:49 AM Ashutosh Chauhan 
> > wrote:
> >
> >> +1 (binding)
> >>
> >> Thanks,
> >> Ashutosh
> >>
> >> On Mon, Apr 24, 2017 at 5:45 AM, Luke Han  wrote:
> >>
> >>> +1 binding
> >>>
> >>> Love to see Superset to be new incubator project.
> >>>
> >>>
> >>> Best Regards!
> >>> -
> >>>
> >>> Luke Han
> >>>
> >>> On Sun, Apr 23, 2017 at 10:53 PM, Jeff Feng  
> wrote:
> >>>
>  Dear Apache Incubator Community,
> 
>  We have updated the Superset proposal
>   (copied below) 
> for
> 
>  Apache Incubation with an additional mentor (Luke Han -
>  luke@apache.org),
>  and would like to start a vote thread for acceptance into the 
> incubator.
> 
>  Our team is excited to share Superset with the Apache community and 
> we
>  hope
>  for the your continued support!
> 
>  Cheers,
>  Jeff & the Superset Team
> 
> 
> 
> 
>  = Superset =
> 
>  == Abstract ==
>  Superset is an enterprise-ready web application for data exploration,
> >> data
>  visualization and dashboarding.
> 
>  == Proposal ==
>  Superset is business intelligence (BI) software that helps modern
>  organizations visualize and interact with their data. Superset 
> enables
>  users explore data from a variety of databases, assemble beautiful
>  dashboards and share their findings.  Superset works neatly with all
>  modern
>  SQL-speaking databases, and integrates with Druid.io to provide
> >> real-time,
>  interactive, blazing fast data access to large datasets.
> 
>  == Background ==
>  Data is mission critical. To succeed in this era, organizations need 
> to
>  provide low-friction, intuitive and interactive access to data. It is
>  paramount for knowledge workers to be capable of answering their own
>  questions by querying, exploring and visualizing data.
> 
>  The entire business intelligence industry has pivoted from a model of
>  centralized top-down platforms driven by IT organizations to
> >> self-service
>  analytics and agile workflows by any user.  This shift unblocks
>  centralized
>  service bottlenecks for creating data visualizations while also 
> creating
>  an
>  environment that is iterative and fast-moving.  This means that 
> business
>  intelligence software must also be easy and delightful to use.
>  Self-service analytics doesn’t mean that admin and governance 
> features
> >> are
>  not needed.
>  Modern BI tools provide fine-grain access controls and auditing
>  capabilities to understand how data is being used.  Superset is a
> >> solution
>  that delivers on all of these vectors.
> 
>  The technology stack is also constantly morphing - vendors are
> >> struggling
>  to provide cheap, quick and easy solutions to access data.  Business
>  intelligence users are finding existing solutions lacking as these
>  software
>  products either disregard or react slowly to recent game-changing
>  technologies like Druid.io, PrestoDB, Apache Drill, Apache Kylin, 
> d3.js,
>  React.js and iPython’s Jupyter for instance.
> 
>  == Rationale ==
>  Business intelligence is more relevant today than at any other point 
> in
>  history.  Organizations are currently very limited in options for 
> open
>  source data visualization solutions, especially solutions that are 
> both
>  self-service and enterprise-ready.  Every company informing their
>  decisions
>  with data needs a BI tool.
> 
>  We believe that Superset will be a strong compliment to existing 
> Apache
>  Software Foundation technologies by offering scalable user 
> interactions
> >> to
>  distributed storage and computation solutions.  Users will often find
> >> that
>  Superset can act as a catalyst for tooling that can visualize the
>  byproduct
>  of data and computation infrastructure.
> 
>  Superset has many key design elements that help fill a gap in current
>  solutions 

Re: [VOTE] Superset Proposal for Apache Incubator

[Jitendra Pandey]

+1 (binding)

On 4/25/17, 1:27 PM, "Julian Hyde"  wrote:

+1 binding

> On Apr 25, 2017, at 12:48 PM, moon soo Lee  wrote:
> 
> +1 (non-binding)
> 
> On Tue, Apr 25, 2017 at 11:49 AM Ashutosh Chauhan 
> wrote:
> 
>> +1 (binding)
>> 
>> Thanks,
>> Ashutosh
>> 
>> On Mon, Apr 24, 2017 at 5:45 AM, Luke Han  wrote:
>> 
>>> +1 binding
>>> 
>>> Love to see Superset to be new incubator project.
>>> 
>>> 
>>> Best Regards!
>>> -
>>> 
>>> Luke Han
>>> 
>>> On Sun, Apr 23, 2017 at 10:53 PM, Jeff Feng  wrote:
>>> 
 Dear Apache Incubator Community,
 
 We have updated the Superset proposal
  (copied below) for
 
 Apache Incubation with an additional mentor (Luke Han -
 luke@apache.org),
 and would like to start a vote thread for acceptance into the 
incubator.
 
 Our team is excited to share Superset with the Apache community and we
 hope
 for the your continued support!
 
 Cheers,
 Jeff & the Superset Team
 
 
 
 
 = Superset =
 
 == Abstract ==
 Superset is an enterprise-ready web application for data exploration,
>> data
 visualization and dashboarding.
 
 == Proposal ==
 Superset is business intelligence (BI) software that helps modern
 organizations visualize and interact with their data. Superset enables
 users explore data from a variety of databases, assemble beautiful
 dashboards and share their findings.  Superset works neatly with all
 modern
 SQL-speaking databases, and integrates with Druid.io to provide
>> real-time,
 interactive, blazing fast data access to large datasets.
 
 == Background ==
 Data is mission critical. To succeed in this era, organizations need to
 provide low-friction, intuitive and interactive access to data. It is
 paramount for knowledge workers to be capable of answering their own
 questions by querying, exploring and visualizing data.
 
 The entire business intelligence industry has pivoted from a model of
 centralized top-down platforms driven by IT organizations to
>> self-service
 analytics and agile workflows by any user.  This shift unblocks
 centralized
 service bottlenecks for creating data visualizations while also 
creating
 an
 environment that is iterative and fast-moving.  This means that 
business
 intelligence software must also be easy and delightful to use.
 Self-service analytics doesn’t mean that admin and governance features
>> are
 not needed.
 Modern BI tools provide fine-grain access controls and auditing
 capabilities to understand how data is being used.  Superset is a
>> solution
 that delivers on all of these vectors.
 
 The technology stack is also constantly morphing - vendors are
>> struggling
 to provide cheap, quick and easy solutions to access data.  Business
 intelligence users are finding existing solutions lacking as these
 software
 products either disregard or react slowly to recent game-changing
 technologies like Druid.io, PrestoDB, Apache Drill, Apache Kylin, 
d3.js,
 React.js and iPython’s Jupyter for instance.
 
 == Rationale ==
 Business intelligence is more relevant today than at any other point in
 history.  Organizations are currently very limited in options for open
 source data visualization solutions, especially solutions that are both
 self-service and enterprise-ready.  Every company informing their
 decisions
 with data needs a BI tool.
 
 We believe that Superset will be a strong compliment to existing Apache
 Software Foundation technologies by offering scalable user interactions
>> to
 distributed storage and computation solutions.  Users will often find
>> that
 Superset can act as a catalyst for tooling that can visualize the
 byproduct
 of data and computation infrastructure.
 
 Superset has many key design elements that help fill a gap in current
 solutions for organizations:
 * Easy, low friction access to data through a simple, web-based data
 exploration interface.  Composing charts and dashboards are intuitive.
 Eliminating the need to write code or SQL empowers anyone to use it.
 * Access to a wide array of rich, interactive data visualization types.

Re: [VOTE] Superset Proposal for Apache Incubator

[Julian Hyde]

+1 binding

> On Apr 25, 2017, at 12:48 PM, moon soo Lee  wrote:
> 
> +1 (non-binding)
> 
> On Tue, Apr 25, 2017 at 11:49 AM Ashutosh Chauhan 
> wrote:
> 
>> +1 (binding)
>> 
>> Thanks,
>> Ashutosh
>> 
>> On Mon, Apr 24, 2017 at 5:45 AM, Luke Han  wrote:
>> 
>>> +1 binding
>>> 
>>> Love to see Superset to be new incubator project.
>>> 
>>> 
>>> Best Regards!
>>> -
>>> 
>>> Luke Han
>>> 
>>> On Sun, Apr 23, 2017 at 10:53 PM, Jeff Feng  wrote:
>>> 
 Dear Apache Incubator Community,
 
 We have updated the Superset proposal
  (copied below) for
 
 Apache Incubation with an additional mentor (Luke Han -
 luke@apache.org),
 and would like to start a vote thread for acceptance into the incubator.
 
 Our team is excited to share Superset with the Apache community and we
 hope
 for the your continued support!
 
 Cheers,
 Jeff & the Superset Team
 
 
 
 
 = Superset =
 
 == Abstract ==
 Superset is an enterprise-ready web application for data exploration,
>> data
 visualization and dashboarding.
 
 == Proposal ==
 Superset is business intelligence (BI) software that helps modern
 organizations visualize and interact with their data. Superset enables
 users explore data from a variety of databases, assemble beautiful
 dashboards and share their findings.  Superset works neatly with all
 modern
 SQL-speaking databases, and integrates with Druid.io to provide
>> real-time,
 interactive, blazing fast data access to large datasets.
 
 == Background ==
 Data is mission critical. To succeed in this era, organizations need to
 provide low-friction, intuitive and interactive access to data. It is
 paramount for knowledge workers to be capable of answering their own
 questions by querying, exploring and visualizing data.
 
 The entire business intelligence industry has pivoted from a model of
 centralized top-down platforms driven by IT organizations to
>> self-service
 analytics and agile workflows by any user.  This shift unblocks
 centralized
 service bottlenecks for creating data visualizations while also creating
 an
 environment that is iterative and fast-moving.  This means that business
 intelligence software must also be easy and delightful to use.
 Self-service analytics doesn’t mean that admin and governance features
>> are
 not needed.
 Modern BI tools provide fine-grain access controls and auditing
 capabilities to understand how data is being used.  Superset is a
>> solution
 that delivers on all of these vectors.
 
 The technology stack is also constantly morphing - vendors are
>> struggling
 to provide cheap, quick and easy solutions to access data.  Business
 intelligence users are finding existing solutions lacking as these
 software
 products either disregard or react slowly to recent game-changing
 technologies like Druid.io, PrestoDB, Apache Drill, Apache Kylin, d3.js,
 React.js and iPython’s Jupyter for instance.
 
 == Rationale ==
 Business intelligence is more relevant today than at any other point in
 history.  Organizations are currently very limited in options for open
 source data visualization solutions, especially solutions that are both
 self-service and enterprise-ready.  Every company informing their
 decisions
 with data needs a BI tool.
 
 We believe that Superset will be a strong compliment to existing Apache
 Software Foundation technologies by offering scalable user interactions
>> to
 distributed storage and computation solutions.  Users will often find
>> that
 Superset can act as a catalyst for tooling that can visualize the
 byproduct
 of data and computation infrastructure.
 
 Superset has many key design elements that help fill a gap in current
 solutions for organizations:
 * Easy, low friction access to data through a simple, web-based data
 exploration interface.  Composing charts and dashboards are intuitive.
 Eliminating the need to write code or SQL empowers anyone to use it.
 * Access to a wide array of rich, interactive data visualization types.
 * Enterprise-ready: Integration with different authentication
>> mechanisms
 and granular permissions centered around actions and data access.
 * Realtime & fast: Superset provides realtime analytics at the speed of
 thought on very large datasets when integrated with Druid.io.
 * Broad data access: Consume data out of any SQL-speaking relational
 database.
 * Extensible: Can be extended to talk to many noSQL databases like
>> Apache
 Drill, Elastic Search, and other popular database engines.
 * Fast loading dashboards 

Re: [PROPOSAL] Superset Proposal for Apache Incubator

[Raphael Bircher]

Hi all

There is no information about affiliation of the initial committers. The  
only information is, that they are from Airbnb inc and Hortonworks. But  
there are no numbers.


Regards Raphael

Am .04.2017, 09:17 Uhr, schrieb Jeff Feng :


Thanks John and Max - I have updated the proposal wiki to reflect this
update.  It now reads:

Source and Intellectual Property Submission Plan

Airbnb will submit a Software Grant Agreement (SGA) as Superset joins the
incubator. We do not expect any complications for the submission of the
Superset code base. Our code is already in Github and there is only a
single code base.


On Mon, Apr 24, 2017 at 11:32 PM, Maxime Beauchemin <
maximebeauche...@gmail.com> wrote:

"Airbnb will submit a Software Grant Agreement (SGA) as Superset joins  
the

incubator."

Should I add this sentence in the proposal?

Max

On Mon, Apr 24, 2017 at 5:48 AM, John D. Ament 
wrote:

> I missed this discussion.  In your IP section, you list out:
>
> == Source and Intellectual Property Submission Plan ==
> We do not expect any complications for the submission of the Superset
code
> base.  Our code is already in Github and there is only a single code
base.
>
> This IMHO not clear.  Does Airbnb plan to submit a SGA for Superset,  
or

> expect that no SGA is required because its Apache licensed?
>
> John
>
> On Sun, Apr 2, 2017 at 4:09 PM Jeff Feng  


> wrote:
>
> > Dear Apache Incubator Community,
> >
> > We are excited to share our proposal for discussion and feedback for
> > entering Apache Incubation.  Superset is an enterprise-ready web
> > application for data exploration, data visualization and  
dashboarding.

> >
> > Our Incubation proposal is at the following Wiki as well as copied  
in

the
> > email below:
> >
> > https://wiki.apache.org/incubator/SupersetProposal
> >
> > We have an active Superset community including 400+ members and  
nearly

> 200
> > topics.  The Google Group can be found below.  We plan to move the
> > discussion to the ASF:
> >
> > https://groups.google.com/forum/#!forum/airbnb_superset
> >
> > Thank you and look forward to the discussion!
> >
> > Jeff, Max & Alanna
> >
> >
> >
> >
> > = Superset =
> >
> > == Abstract ==
> >
> > Superset is an enterprise-ready web application for data  
exploration,

> data
> > visualization and dashboarding.
> >
> > == Proposal ==
> >
> > Superset is business intelligence (BI) software that helps modern
> > organizations visualize and interact with their data. Superset  
enables

> > users explore data from a variety of databases, assemble beautiful
> > dashboards and share their findings.  Superset works neatly with all
> modern
> > SQL-speaking databases, and integrates with Druid.io to provide
> real-time,
> > interactive, blazing fast data access to large datasets.
> >
> > == Background ==
> >
> > Data is mission critical. To succeed in this era, organizations  
need to
> > provide low-friction, intuitive and interactive access to data. It  
is

> > paramount for knowledge workers to be capable of answering their own
> > questions by querying, exploring and visualizing data.
> >
> > The entire business intelligence industry has pivoted from a model  
of

> > centralized top-down platforms driven by IT organizations to
self-service
> > analytics and agile workflows by any user.  This shift unblocks
> centralized
> > service bottlenecks for creating data visualizations while also
creating
> an
> > environment that is iterative and fast-moving.  This means that
business
> > intelligence software must also be easy and delightful to use.
> > Self-service analytics doesn’t mean that admin and governance  
features

> are
> > not needed.
> >
> > Modern BI tools provide fine-grain access controls and auditing
> > capabilities to understand how data is being used.  Superset is a
> solution
> > that delivers on all of these vectors.
> >
> > The technology stack is also constantly morphing - vendors are
struggling
> > to provide cheap, quick and easy solutions to access data.  Business
> > intelligence users are finding existing solutions lacking as these
> software
> > products either disregard or react slowly to recent game-changing
> > technologies like Druid.io, PrestoDB, Apache Drill, Apache Kylin,
d3.js,
> > React.js and iPython’s Jupyter for instance.
> >
> > == Rationale ==
> >
> > Business intelligence is more relevant today than at any other  
point in
> > history.  Organizations are currently very limited in options for  
open
> > source data visualization solutions, especially solutions that are  
both

> > self-service and enterprise-ready.  Every company informing their
> decisions
> > with data needs a BI tool.
> >
> > We believe that Superset will be a strong compliment to existing  
Apache
> > Software Foundation technologies by offering scalable user  
interactions

> to
> > distributed storage and computation solutions.  Users will often  

Re: [VOTE] Superset Proposal for Apache Incubator

[moon soo Lee]

+1 (non-binding)

On Tue, Apr 25, 2017 at 11:49 AM Ashutosh Chauhan 
wrote:

> +1 (binding)
>
> Thanks,
> Ashutosh
>
> On Mon, Apr 24, 2017 at 5:45 AM, Luke Han  wrote:
>
> > +1 binding
> >
> > Love to see Superset to be new incubator project.
> >
> >
> > Best Regards!
> > -
> >
> > Luke Han
> >
> > On Sun, Apr 23, 2017 at 10:53 PM, Jeff Feng  wrote:
> >
> >> Dear Apache Incubator Community,
> >>
> >> We have updated the Superset proposal
> >>  (copied below) for
> >>
> >> Apache Incubation with an additional mentor (Luke Han -
> >> luke@apache.org),
> >> and would like to start a vote thread for acceptance into the incubator.
> >>
> >> Our team is excited to share Superset with the Apache community and we
> >> hope
> >> for the your continued support!
> >>
> >> Cheers,
> >> Jeff & the Superset Team
> >>
> >>
> >>
> >>
> >> = Superset =
> >>
> >> == Abstract ==
> >> Superset is an enterprise-ready web application for data exploration,
> data
> >> visualization and dashboarding.
> >>
> >> == Proposal ==
> >> Superset is business intelligence (BI) software that helps modern
> >> organizations visualize and interact with their data. Superset enables
> >> users explore data from a variety of databases, assemble beautiful
> >> dashboards and share their findings.  Superset works neatly with all
> >> modern
> >> SQL-speaking databases, and integrates with Druid.io to provide
> real-time,
> >> interactive, blazing fast data access to large datasets.
> >>
> >> == Background ==
> >> Data is mission critical. To succeed in this era, organizations need to
> >> provide low-friction, intuitive and interactive access to data. It is
> >> paramount for knowledge workers to be capable of answering their own
> >> questions by querying, exploring and visualizing data.
> >>
> >> The entire business intelligence industry has pivoted from a model of
> >> centralized top-down platforms driven by IT organizations to
> self-service
> >> analytics and agile workflows by any user.  This shift unblocks
> >> centralized
> >> service bottlenecks for creating data visualizations while also creating
> >> an
> >> environment that is iterative and fast-moving.  This means that business
> >> intelligence software must also be easy and delightful to use.
> >> Self-service analytics doesn’t mean that admin and governance features
> are
> >> not needed.
> >> Modern BI tools provide fine-grain access controls and auditing
> >> capabilities to understand how data is being used.  Superset is a
> solution
> >> that delivers on all of these vectors.
> >>
> >> The technology stack is also constantly morphing - vendors are
> struggling
> >> to provide cheap, quick and easy solutions to access data.  Business
> >> intelligence users are finding existing solutions lacking as these
> >> software
> >> products either disregard or react slowly to recent game-changing
> >> technologies like Druid.io, PrestoDB, Apache Drill, Apache Kylin, d3.js,
> >> React.js and iPython’s Jupyter for instance.
> >>
> >> == Rationale ==
> >> Business intelligence is more relevant today than at any other point in
> >> history.  Organizations are currently very limited in options for open
> >> source data visualization solutions, especially solutions that are both
> >> self-service and enterprise-ready.  Every company informing their
> >> decisions
> >> with data needs a BI tool.
> >>
> >> We believe that Superset will be a strong compliment to existing Apache
> >> Software Foundation technologies by offering scalable user interactions
> to
> >> distributed storage and computation solutions.  Users will often find
> that
> >> Superset can act as a catalyst for tooling that can visualize the
> >> byproduct
> >> of data and computation infrastructure.
> >>
> >> Superset has many key design elements that help fill a gap in current
> >> solutions for organizations:
> >>  * Easy, low friction access to data through a simple, web-based data
> >> exploration interface.  Composing charts and dashboards are intuitive.
> >> Eliminating the need to write code or SQL empowers anyone to use it.
> >>  * Access to a wide array of rich, interactive data visualization types.
> >>  * Enterprise-ready: Integration with different authentication
> mechanisms
> >> and granular permissions centered around actions and data access.
> >>  * Realtime & fast: Superset provides realtime analytics at the speed of
> >> thought on very large datasets when integrated with Druid.io.
> >>  * Broad data access: Consume data out of any SQL-speaking relational
> >> database.
> >>  * Extensible: Can be extended to talk to many noSQL databases like
> Apache
> >> Drill, Elastic Search, and other popular database engines.
> >>  * Fast loading dashboards with configurable web-scale caching.
> >>  * Plug-in framework that enables organizations to build custom
> analytical
> >> 

Re: [VOTE] Superset Proposal for Apache Incubator

[Ashutosh Chauhan]

+1 (binding)

Thanks,
Ashutosh

On Mon, Apr 24, 2017 at 5:45 AM, Luke Han  wrote:

> +1 binding
>
> Love to see Superset to be new incubator project.
>
>
> Best Regards!
> -
>
> Luke Han
>
> On Sun, Apr 23, 2017 at 10:53 PM, Jeff Feng  wrote:
>
>> Dear Apache Incubator Community,
>>
>> We have updated the Superset proposal
>>  (copied below) for
>>
>> Apache Incubation with an additional mentor (Luke Han -
>> luke@apache.org),
>> and would like to start a vote thread for acceptance into the incubator.
>>
>> Our team is excited to share Superset with the Apache community and we
>> hope
>> for the your continued support!
>>
>> Cheers,
>> Jeff & the Superset Team
>>
>>
>>
>>
>> = Superset =
>>
>> == Abstract ==
>> Superset is an enterprise-ready web application for data exploration, data
>> visualization and dashboarding.
>>
>> == Proposal ==
>> Superset is business intelligence (BI) software that helps modern
>> organizations visualize and interact with their data. Superset enables
>> users explore data from a variety of databases, assemble beautiful
>> dashboards and share their findings.  Superset works neatly with all
>> modern
>> SQL-speaking databases, and integrates with Druid.io to provide real-time,
>> interactive, blazing fast data access to large datasets.
>>
>> == Background ==
>> Data is mission critical. To succeed in this era, organizations need to
>> provide low-friction, intuitive and interactive access to data. It is
>> paramount for knowledge workers to be capable of answering their own
>> questions by querying, exploring and visualizing data.
>>
>> The entire business intelligence industry has pivoted from a model of
>> centralized top-down platforms driven by IT organizations to self-service
>> analytics and agile workflows by any user.  This shift unblocks
>> centralized
>> service bottlenecks for creating data visualizations while also creating
>> an
>> environment that is iterative and fast-moving.  This means that business
>> intelligence software must also be easy and delightful to use.
>> Self-service analytics doesn’t mean that admin and governance features are
>> not needed.
>> Modern BI tools provide fine-grain access controls and auditing
>> capabilities to understand how data is being used.  Superset is a solution
>> that delivers on all of these vectors.
>>
>> The technology stack is also constantly morphing - vendors are struggling
>> to provide cheap, quick and easy solutions to access data.  Business
>> intelligence users are finding existing solutions lacking as these
>> software
>> products either disregard or react slowly to recent game-changing
>> technologies like Druid.io, PrestoDB, Apache Drill, Apache Kylin, d3.js,
>> React.js and iPython’s Jupyter for instance.
>>
>> == Rationale ==
>> Business intelligence is more relevant today than at any other point in
>> history.  Organizations are currently very limited in options for open
>> source data visualization solutions, especially solutions that are both
>> self-service and enterprise-ready.  Every company informing their
>> decisions
>> with data needs a BI tool.
>>
>> We believe that Superset will be a strong compliment to existing Apache
>> Software Foundation technologies by offering scalable user interactions to
>> distributed storage and computation solutions.  Users will often find that
>> Superset can act as a catalyst for tooling that can visualize the
>> byproduct
>> of data and computation infrastructure.
>>
>> Superset has many key design elements that help fill a gap in current
>> solutions for organizations:
>>  * Easy, low friction access to data through a simple, web-based data
>> exploration interface.  Composing charts and dashboards are intuitive.
>> Eliminating the need to write code or SQL empowers anyone to use it.
>>  * Access to a wide array of rich, interactive data visualization types.
>>  * Enterprise-ready: Integration with different authentication mechanisms
>> and granular permissions centered around actions and data access.
>>  * Realtime & fast: Superset provides realtime analytics at the speed of
>> thought on very large datasets when integrated with Druid.io.
>>  * Broad data access: Consume data out of any SQL-speaking relational
>> database.
>>  * Extensible: Can be extended to talk to many noSQL databases like Apache
>> Drill, Elastic Search, and other popular database engines.
>>  * Fast loading dashboards with configurable web-scale caching.
>>  * Plug-in framework that enables organizations to build custom analytical
>> applications with new UI/UX interfaces.
>>  * SQL Lab, a state-of-the-art SQL IDE that empowers SQL-speaking users
>> with more flexibility.  SQL Lab integrates with the visualization engine
>> seamlessly.
>>
>> == Initial Goals ==
>> The initial goals of the Superset project are several-fold:
>>  * Move the existing codebase to Apache and 

Re: Incubator Governance Change

[Ted Dunning]

On Tue, Apr 25, 2017 at 12:50 AM, Pierre Smits 
wrote:

> Now, we can argue that mentors have an obligation to vote on releases of an
> incubating project first before the vote is taken to this ml. But that
> could lead to unnecessary delays and frustration.
>

Having the mentors vote in the podling mailing list *saves* time in my
experience even if you don't get all three votes.

Re: Airflow voting on release artifacts

[Alex Harui]

On 4/25/17, 1:43 AM, "Bolke de Bruin"  wrote:

>Hi Bertrand,
>
>> On 25 Apr 2017, at 09:04, Bertrand Delacretaz
>> wrote:
>> 
>> Hi,
>> 
>> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini
>> wrote:
>>> ...Hitesh recently raised the issue that the artifact that passes the
>>>vote
>>> MUST be the one that we actually release...
>> 
>> Yes in terms of having the same binary digests and signatures, but
>> renaming the files is fine IMO, especially for removing an -rc suffix
>> which makes total sense. I would just add that step to your release
>> process documentation to make it clear.
>> 
>>> ...Rename/rebuild after final vote (This is what Airflow is doing, and
>>>Beam
>>> does this, I believe)...
>> 
>> I'd say rename yes but rebuild no, in order to keep the same digests
>> and signatures.
>> 
>
>As mentioned earlier, that seems not to be possible. The metadata
>(filename) and version information inside the package need to be in sync.
>This how the python build tools and python ecosystem works.

I'm not familiar with Python, but is it possible to have a command line
option that adds the "-rc" suffix in the right places?

IMO, there are two "audiences".  One is the voters, and one is your
customers.  The customers should not be using the RC until after it is
approved unless they are volunteering to be a voter.  Voters are primarily
supposed to examine a source artifact, but if you also release a
"convenience binary" artifact, there are some examinations of that
"binary" artifact required.  For many projects this "convenience" artifact
is the one that the vast majority of customers consume.  AIUI, Python
doesn't have binaries, but IMO, there is no requirement that this
"convenience" artifact actually contains binaries.  A "convenience"
artifact is just supposed to be the source artifact processed by a build
script since many of your customer may not have, or may not want to
configure their machines to run whatever build script engine you choose
for your project.  Further, there is no requirement that I know of that
voters must test the "convenience" artifact by actually running it.  It
just makes sense to do so in most cases.  But if Python is interpreted
source, you may be able to use this to your advantage.

So, your source package should consist of the source and build script
required to build this "customer"/"convenience" package, and the build
script should allow a command line option to add the "-rc" suffix.  Then
voters would be instructed to download the source package, and to test it,
build a "customer" package with the "-rc" suffix and test the results.
And voters would also be instructed to download the "customer" package
that doesn't have the -rc suffix.  But in order to test it, they only have
to diff that package against the "customer" package they built from the
source package.  It should be the same except for the metadata.

We'll see if others can find a problem with this plan, but IMO, that would
be good enough for me as an PMC voter.

HTH,
-Alex

Re: Incubator Governance Change

[Bertrand Delacretaz]

On Tue, Apr 25, 2017 at 12:46 PM, John D. Ament  wrote:
> On Tue, Apr 25, 2017 at 3:56 AM Bertrand Delacretaz <
> bdelacre...@codeconsult.ch> wrote:
>>... We want all podlings to be actively mentored, and if that doesn't
>> happen they should look for new mentors to make sure they have at
>> least one that's active which includes voting on releases.
>>
> How do we communicate this out more?...

A blog post maybe? We do have some at
https://blogs.apache.org/incubator/ already.

-Bertrand

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: View archive on podling private@ list

[Niclas Hedhman]

ok, cool...

On Tue, Apr 25, 2017 at 6:41 PM, John D. Ament 
wrote:

> Niclas,
>
> All of the access rules are based on LDAP now.  I've updated the guides
> recently to reflect that rosters are maintained via whimsy, and not via the
> XML files.  To setup weex's permissions go to
> https://whimsy.apache.org/roster/ppmc/weex and add additional committers
> to
> the PPMC.
>
> I have a request out to Sam to enhance this further to be committer vs
> PPMC, right now it assumes everyone is PPMC.
>
> John
>
> On Tue, Apr 25, 2017 at 12:31 AM Niclas Hedhman 
> wrote:
>
> > Dave,
> > I am one of the Weex mentors, and I also don't understand this. I used to
> > be pretty sure that the PPMC was a construct in Incubator without LDAP
> > backing, but perhaps that changed during my absence from the Incubator.
> > Wasn't even committership a matter of the Incubator LDAP group ?
> > And wouldn't this mean that only Incubator PMC members have access to
> > private@PODLING.a.o ?
> >
> > Cheers
> >
> > On Tue, Apr 25, 2017 at 11:10 AM, Dave Fisher 
> > wrote:
> >
> > > You must be on the PPMC(podling), a Mentor, or an Apache Member for
> > > private ML access.
> > >
> > > Mentors should be helping podlings with these tasks.
> > >
> > > Regards,
> > > Dave
> > >
> > > Sent from my iPhone
> > >
> > > > On Apr 24, 2017, at 7:02 PM, sospartan  wrote:
> > > >
> > > > Yes, that the RESULT I need. Thank you.
> > > > But It's empty in content when I open this address.
> > > > I want make sure. So, Is there a specific rule that committer can not
> > > view
> > > > these archives of private podling mail list?
> > > >
> > > >> On Tue, Apr 25, 2017 at 2:41 AM, Josh Elser 
> > wrote:
> > > >>
> > > >> To avoid disclosing the name included in the subject, here is a
> > > permalink
> > > >> to the RESULT message for a VOTE for a new committer sent by Adam
> Feng
> > > at
> > > >> 2017-04-24 02:51 (-0400).
> > > >>
> > > >> https://lists.apache.org/thread.html/c26b89e59557ae8793bac1e
> > > >> ddeb9d03bfa3e4203da0e50face55193b@%3Cprivate.weex.apache.org%3E
> > > >>
> > > >> Hopefully this is the one you meant... Pinging your mentors in the
> > > future
> > > >> might be better to avoid any chance of accidentally leaking some
> > private
> > > >> context :)
> > > >>
> > > >> - Josh
> > > >>
> > > >> sospartan wrote:
> > > >>
> > > >>> Hi,
> > > >>> May I ask who has karma to view archives of podling private@ list?
> > I'm
> > > >>> weex-incubating committer, but can't view archive of private.
> When I
> > > try
> > > >>> to open
> > > >>>
> > > >>> https://lists.apache.org/list.html?priv...@weex.apache.org , it's
> > > >>> redirect
> > > >>> to commits@ list.
> > > >>>
> > > >>> I need to get permalink of a vote thread.
> > > >>>
> > > >>>
> > > >> 
> -
> > > >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> > > >> For additional commands, e-mail: general-h...@incubator.apache.org
> > > >>
> > > >>
> > > >
> > > >
> > > > --
> > > > sospartan
> > > > Phone:13588488290
> > > > HangZhou
> > >
> > >
> > > -
> > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> > > For additional commands, e-mail: general-h...@incubator.apache.org
> > >
> > >
> >
> >
> > --
> > Niclas Hedhman, Software Developer
> > http://polygene.apache.org - New Energy for Java
> >
>



-- 
Niclas Hedhman, Software Developer
http://polygene.apache.org - New Energy for Java

Re: Airflow voting on release artifacts

[Niclas Hedhman]

It is Java, but we have version references internally in all kinds of
places.
So, what happens is that the build creates the "expected final release",
say "1.2.3" and sets all internal references to that. But the tarball will
be -RCx, which is voted upon. And then, as Bertrand suggested, a rename of
that tarball if the vote passes, is not a problem.

And as John points out, the release build tags "1.2.3" in git and we can
(re)move that later if the vote fails.


On Tue, Apr 25, 2017 at 5:18 PM, Bolke de Bruin  wrote:

> Hi Niclas,
>
> Is this Java or Python? I can only find Java for Polygene.
>
> Furthermore, how do you manage this you repository? Do you have the
> release already set in one of your files, e.g. something like this:
> https://github.com/apache/incubator-airflow/blob/v1-8-
> test/airflow/version.py  incubator-airflow/blob/v1-8-test/airflow/version.py>
>
> The build system generates the metadata from there, which is used by the
> package installers (e.g. pip).
>
> Cheers
> Bolke
>
>
> > On 25 Apr 2017, at 10:47, Niclas Hedhman  wrote:
> >
> > We have a similar issue in Polygene, but the internal version is simply
> the
> > expected version, say 1.2.3 and the RC has the different file name. No
> > packagers will ever get the -RC named artifact and no confusion is among
> > community members as they are aware of this. IF the RC passes, then the
> > rename can happen. IF the RC doesn't happen, you can rebuild with new
> > content and same internal version.
> >
> > Cheers
> > Niclas
> >
> > On Tue, Apr 25, 2017 at 4:43 PM, Bolke de Bruin 
> wrote:
> >
> >> Hi Bertrand,
> >>
> >>> On 25 Apr 2017, at 09:04, Bertrand Delacretaz <
> >> bdelacre...@codeconsult.ch> wrote:
> >>>
> >>> Hi,
> >>>
> >>> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini <
> criccom...@apache.org>
> >> wrote:
>  ...Hitesh recently raised the issue that the artifact that passes the
> >> vote
>  MUST be the one that we actually release...
> >>>
> >>> Yes in terms of having the same binary digests and signatures, but
> >>> renaming the files is fine IMO, especially for removing an -rc suffix
> >>> which makes total sense. I would just add that step to your release
> >>> process documentation to make it clear.
> >>>
>  ...Rename/rebuild after final vote (This is what Airflow is doing, and
> >> Beam
>  does this, I believe)...
> >>>
> >>> I'd say rename yes but rebuild no, in order to keep the same digests
> >>> and signatures.
> >>>
> >>
> >> As mentioned earlier, that seems not to be possible. The metadata
> >> (filename) and version information inside the package need to be in
> sync.
> >> This how the python build tools and python ecosystem works.
> >>
> >> - Bolke.
> >> -
> >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> >> For additional commands, e-mail: general-h...@incubator.apache.org
> >>
> >>
> >
> >
> > --
> > Niclas Hedhman, Software Developer
> > http://polygene.apache.org - New Energy for Java
>
>


-- 
Niclas Hedhman, Software Developer
http://polygene.apache.org - New Energy for Java

Re: Airflow voting on release artifacts

[John D. Ament]

I'll point out that this is a great conversation and set of questions to
have with your mentors first, the IPMC later if they didn't respond/give
you a clear path forward.  Did you engage with them?

I personally hate seeing rc's being put forward to the incubator.  Most of
us use git these days, branches are cheap.  Even in my $dayjob when rolling
a release we always create a release branch, cut the release from there.
If something broke delete it from our internal artifact repository and drop
the branch - effectively starting over.

Would following an approach like that work for you?

John

On Tue, Apr 25, 2017 at 5:18 AM Bolke de Bruin  wrote:

> Hi Niclas,
>
> Is this Java or Python? I can only find Java for Polygene.
>
> Furthermore, how do you manage this you repository? Do you have the
> release already set in one of your files, e.g. something like this:
>
> https://github.com/apache/incubator-airflow/blob/v1-8-test/airflow/version.py
> <
> https://github.com/apache/incubator-airflow/blob/v1-8-test/airflow/version.py
> >
>
> The build system generates the metadata from there, which is used by the
> package installers (e.g. pip).
>
> Cheers
> Bolke
>
>
> > On 25 Apr 2017, at 10:47, Niclas Hedhman  wrote:
> >
> > We have a similar issue in Polygene, but the internal version is simply
> the
> > expected version, say 1.2.3 and the RC has the different file name. No
> > packagers will ever get the -RC named artifact and no confusion is among
> > community members as they are aware of this. IF the RC passes, then the
> > rename can happen. IF the RC doesn't happen, you can rebuild with new
> > content and same internal version.
> >
> > Cheers
> > Niclas
> >
> > On Tue, Apr 25, 2017 at 4:43 PM, Bolke de Bruin 
> wrote:
> >
> >> Hi Bertrand,
> >>
> >>> On 25 Apr 2017, at 09:04, Bertrand Delacretaz <
> >> bdelacre...@codeconsult.ch> wrote:
> >>>
> >>> Hi,
> >>>
> >>> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini <
> criccom...@apache.org>
> >> wrote:
>  ...Hitesh recently raised the issue that the artifact that passes the
> >> vote
>  MUST be the one that we actually release...
> >>>
> >>> Yes in terms of having the same binary digests and signatures, but
> >>> renaming the files is fine IMO, especially for removing an -rc suffix
> >>> which makes total sense. I would just add that step to your release
> >>> process documentation to make it clear.
> >>>
>  ...Rename/rebuild after final vote (This is what Airflow is doing, and
> >> Beam
>  does this, I believe)...
> >>>
> >>> I'd say rename yes but rebuild no, in order to keep the same digests
> >>> and signatures.
> >>>
> >>
> >> As mentioned earlier, that seems not to be possible. The metadata
> >> (filename) and version information inside the package need to be in
> sync.
> >> This how the python build tools and python ecosystem works.
> >>
> >> - Bolke.
> >> -
> >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> >> For additional commands, e-mail: general-h...@incubator.apache.org
> >>
> >>
> >
> >
> > --
> > Niclas Hedhman, Software Developer
> > http://polygene.apache.org - New Energy for Java
>
>

Re: Incubator Governance Change

[John D. Ament]

On Tue, Apr 25, 2017 at 3:56 AM Bertrand Delacretaz <
bdelacre...@codeconsult.ch> wrote:

> On Tue, Apr 25, 2017 at 9:50 AM, Pierre Smits 
> wrote:
> > ...we can argue that mentors have an obligation to vote on releases of an
> > incubating project first before the vote is taken to this ml...
>
> Mentors not voting on releases gives a very bad signal to me as to
> their engagement in the podling.
>
>
Huge +1 on this.


> We want all podlings to be actively mentored, and if that doesn't
> happen they should look for new mentors to make sure they have at
> least one that's active which includes voting on releases.
>
>
How do we communicate this out more? Several of us have been harping on
this, saw some progress, but not quite enough.


> -Bertrand (who's not perfect either as a mentor, and would accept
> being kicked out of that role as a result for podlings where I'm not
> doing my job)
>
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>

Re: View archive on podling private@ list

[John D. Ament]

Niclas,

All of the access rules are based on LDAP now.  I've updated the guides
recently to reflect that rosters are maintained via whimsy, and not via the
XML files.  To setup weex's permissions go to
https://whimsy.apache.org/roster/ppmc/weex and add additional committers to
the PPMC.

I have a request out to Sam to enhance this further to be committer vs
PPMC, right now it assumes everyone is PPMC.

John

On Tue, Apr 25, 2017 at 12:31 AM Niclas Hedhman  wrote:

> Dave,
> I am one of the Weex mentors, and I also don't understand this. I used to
> be pretty sure that the PPMC was a construct in Incubator without LDAP
> backing, but perhaps that changed during my absence from the Incubator.
> Wasn't even committership a matter of the Incubator LDAP group ?
> And wouldn't this mean that only Incubator PMC members have access to
> private@PODLING.a.o ?
>
> Cheers
>
> On Tue, Apr 25, 2017 at 11:10 AM, Dave Fisher 
> wrote:
>
> > You must be on the PPMC(podling), a Mentor, or an Apache Member for
> > private ML access.
> >
> > Mentors should be helping podlings with these tasks.
> >
> > Regards,
> > Dave
> >
> > Sent from my iPhone
> >
> > > On Apr 24, 2017, at 7:02 PM, sospartan  wrote:
> > >
> > > Yes, that the RESULT I need. Thank you.
> > > But It's empty in content when I open this address.
> > > I want make sure. So, Is there a specific rule that committer can not
> > view
> > > these archives of private podling mail list?
> > >
> > >> On Tue, Apr 25, 2017 at 2:41 AM, Josh Elser 
> wrote:
> > >>
> > >> To avoid disclosing the name included in the subject, here is a
> > permalink
> > >> to the RESULT message for a VOTE for a new committer sent by Adam Feng
> > at
> > >> 2017-04-24 02:51 (-0400).
> > >>
> > >> https://lists.apache.org/thread.html/c26b89e59557ae8793bac1e
> > >> ddeb9d03bfa3e4203da0e50face55193b@%3Cprivate.weex.apache.org%3E
> > >>
> > >> Hopefully this is the one you meant... Pinging your mentors in the
> > future
> > >> might be better to avoid any chance of accidentally leaking some
> private
> > >> context :)
> > >>
> > >> - Josh
> > >>
> > >> sospartan wrote:
> > >>
> > >>> Hi,
> > >>> May I ask who has karma to view archives of podling private@ list?
> I'm
> > >>> weex-incubating committer, but can't view archive of private.  When I
> > try
> > >>> to open
> > >>>
> > >>> https://lists.apache.org/list.html?priv...@weex.apache.org , it's
> > >>> redirect
> > >>> to commits@ list.
> > >>>
> > >>> I need to get permalink of a vote thread.
> > >>>
> > >>>
> > >> -
> > >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> > >> For additional commands, e-mail: general-h...@incubator.apache.org
> > >>
> > >>
> > >
> > >
> > > --
> > > sospartan
> > > Phone:13588488290
> > > HangZhou
> >
> >
> > -
> > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> > For additional commands, e-mail: general-h...@incubator.apache.org
> >
> >
>
>
> --
> Niclas Hedhman, Software Developer
> http://polygene.apache.org - New Energy for Java
>

Re: Airflow voting on release artifacts

[Bolke de Bruin]

Hi Niclas,

Is this Java or Python? I can only find Java for Polygene. 

Furthermore, how do you manage this you repository? Do you have the release 
already set in one of your files, e.g. something like this:
https://github.com/apache/incubator-airflow/blob/v1-8-test/airflow/version.py 


The build system generates the metadata from there, which is used by the 
package installers (e.g. pip).

Cheers
Bolke


> On 25 Apr 2017, at 10:47, Niclas Hedhman  wrote:
> 
> We have a similar issue in Polygene, but the internal version is simply the
> expected version, say 1.2.3 and the RC has the different file name. No
> packagers will ever get the -RC named artifact and no confusion is among
> community members as they are aware of this. IF the RC passes, then the
> rename can happen. IF the RC doesn't happen, you can rebuild with new
> content and same internal version.
> 
> Cheers
> Niclas
> 
> On Tue, Apr 25, 2017 at 4:43 PM, Bolke de Bruin  wrote:
> 
>> Hi Bertrand,
>> 
>>> On 25 Apr 2017, at 09:04, Bertrand Delacretaz <
>> bdelacre...@codeconsult.ch> wrote:
>>> 
>>> Hi,
>>> 
>>> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini 
>> wrote:
 ...Hitesh recently raised the issue that the artifact that passes the
>> vote
 MUST be the one that we actually release...
>>> 
>>> Yes in terms of having the same binary digests and signatures, but
>>> renaming the files is fine IMO, especially for removing an -rc suffix
>>> which makes total sense. I would just add that step to your release
>>> process documentation to make it clear.
>>> 
 ...Rename/rebuild after final vote (This is what Airflow is doing, and
>> Beam
 does this, I believe)...
>>> 
>>> I'd say rename yes but rebuild no, in order to keep the same digests
>>> and signatures.
>>> 
>> 
>> As mentioned earlier, that seems not to be possible. The metadata
>> (filename) and version information inside the package need to be in sync.
>> This how the python build tools and python ecosystem works.
>> 
>> - Bolke.
>> -
>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
>> For additional commands, e-mail: general-h...@incubator.apache.org
>> 
>> 
> 
> 
> -- 
> Niclas Hedhman, Software Developer
> http://polygene.apache.org - New Energy for Java

Re: Airflow voting on release artifacts

[Niclas Hedhman]

We have a similar issue in Polygene, but the internal version is simply the
expected version, say 1.2.3 and the RC has the different file name. No
packagers will ever get the -RC named artifact and no confusion is among
community members as they are aware of this. IF the RC passes, then the
rename can happen. IF the RC doesn't happen, you can rebuild with new
content and same internal version.

Cheers
Niclas

On Tue, Apr 25, 2017 at 4:43 PM, Bolke de Bruin  wrote:

> Hi Bertrand,
>
> > On 25 Apr 2017, at 09:04, Bertrand Delacretaz <
> bdelacre...@codeconsult.ch> wrote:
> >
> > Hi,
> >
> > On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini 
> wrote:
> >> ...Hitesh recently raised the issue that the artifact that passes the
> vote
> >> MUST be the one that we actually release...
> >
> > Yes in terms of having the same binary digests and signatures, but
> > renaming the files is fine IMO, especially for removing an -rc suffix
> > which makes total sense. I would just add that step to your release
> > process documentation to make it clear.
> >
> >> ...Rename/rebuild after final vote (This is what Airflow is doing, and
> Beam
> >> does this, I believe)...
> >
> > I'd say rename yes but rebuild no, in order to keep the same digests
> > and signatures.
> >
>
> As mentioned earlier, that seems not to be possible. The metadata
> (filename) and version information inside the package need to be in sync.
> This how the python build tools and python ecosystem works.
>
> - Bolke.
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>


-- 
Niclas Hedhman, Software Developer
http://polygene.apache.org - New Energy for Java

Re: Airflow voting on release artifacts

[Bolke de Bruin]

Hi Bertrand,

> On 25 Apr 2017, at 09:04, Bertrand Delacretaz  
> wrote:
> 
> Hi,
> 
> On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini  
> wrote:
>> ...Hitesh recently raised the issue that the artifact that passes the vote
>> MUST be the one that we actually release...
> 
> Yes in terms of having the same binary digests and signatures, but
> renaming the files is fine IMO, especially for removing an -rc suffix
> which makes total sense. I would just add that step to your release
> process documentation to make it clear.
> 
>> ...Rename/rebuild after final vote (This is what Airflow is doing, and Beam
>> does this, I believe)...
> 
> I'd say rename yes but rebuild no, in order to keep the same digests
> and signatures.
> 

As mentioned earlier, that seems not to be possible. The metadata (filename) 
and version information inside the package need to be in sync. This how the 
python build tools and python ecosystem works.

- Bolke.
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Incubator Governance Change

[Bertrand Delacretaz]

On Tue, Apr 25, 2017 at 9:50 AM, Pierre Smits  wrote:
> ...we can argue that mentors have an obligation to vote on releases of an
> incubating project first before the vote is taken to this ml...

Mentors not voting on releases gives a very bad signal to me as to
their engagement in the podling.

We want all podlings to be actively mentored, and if that doesn't
happen they should look for new mentors to make sure they have at
least one that's active which includes voting on releases.

-Bertrand (who's not perfect either as a mentor, and would accept
being kicked out of that role as a result for podlings where I'm not
doing my job)

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [PROPOSAL] Superset Proposal for Apache Incubator

[Jeff Feng]

Thanks John and Max - I have updated the proposal wiki to reflect this
update.  It now reads:

Source and Intellectual Property Submission Plan

Airbnb will submit a Software Grant Agreement (SGA) as Superset joins the
incubator. We do not expect any complications for the submission of the
Superset code base. Our code is already in Github and there is only a
single code base.


On Mon, Apr 24, 2017 at 11:32 PM, Maxime Beauchemin <
maximebeauche...@gmail.com> wrote:

> "Airbnb will submit a Software Grant Agreement (SGA) as Superset joins the
> incubator."
>
> Should I add this sentence in the proposal?
>
> Max
>
> On Mon, Apr 24, 2017 at 5:48 AM, John D. Ament 
> wrote:
>
> > I missed this discussion.  In your IP section, you list out:
> >
> > == Source and Intellectual Property Submission Plan ==
> > We do not expect any complications for the submission of the Superset
> code
> > base.  Our code is already in Github and there is only a single code
> base.
> >
> > This IMHO not clear.  Does Airbnb plan to submit a SGA for Superset, or
> > expect that no SGA is required because its Apache licensed?
> >
> > John
> >
> > On Sun, Apr 2, 2017 at 4:09 PM Jeff Feng 
> > wrote:
> >
> > > Dear Apache Incubator Community,
> > >
> > > We are excited to share our proposal for discussion and feedback for
> > > entering Apache Incubation.  Superset is an enterprise-ready web
> > > application for data exploration, data visualization and dashboarding.
> > >
> > > Our Incubation proposal is at the following Wiki as well as copied in
> the
> > > email below:
> > >
> > > https://wiki.apache.org/incubator/SupersetProposal
> > >
> > > We have an active Superset community including 400+ members and nearly
> > 200
> > > topics.  The Google Group can be found below.  We plan to move the
> > > discussion to the ASF:
> > >
> > > https://groups.google.com/forum/#!forum/airbnb_superset
> > >
> > > Thank you and look forward to the discussion!
> > >
> > > Jeff, Max & Alanna
> > >
> > >
> > >
> > >
> > > = Superset =
> > >
> > > == Abstract ==
> > >
> > > Superset is an enterprise-ready web application for data exploration,
> > data
> > > visualization and dashboarding.
> > >
> > > == Proposal ==
> > >
> > > Superset is business intelligence (BI) software that helps modern
> > > organizations visualize and interact with their data. Superset enables
> > > users explore data from a variety of databases, assemble beautiful
> > > dashboards and share their findings.  Superset works neatly with all
> > modern
> > > SQL-speaking databases, and integrates with Druid.io to provide
> > real-time,
> > > interactive, blazing fast data access to large datasets.
> > >
> > > == Background ==
> > >
> > > Data is mission critical. To succeed in this era, organizations need to
> > > provide low-friction, intuitive and interactive access to data. It is
> > > paramount for knowledge workers to be capable of answering their own
> > > questions by querying, exploring and visualizing data.
> > >
> > > The entire business intelligence industry has pivoted from a model of
> > > centralized top-down platforms driven by IT organizations to
> self-service
> > > analytics and agile workflows by any user.  This shift unblocks
> > centralized
> > > service bottlenecks for creating data visualizations while also
> creating
> > an
> > > environment that is iterative and fast-moving.  This means that
> business
> > > intelligence software must also be easy and delightful to use.
> > > Self-service analytics doesn’t mean that admin and governance features
> > are
> > > not needed.
> > >
> > > Modern BI tools provide fine-grain access controls and auditing
> > > capabilities to understand how data is being used.  Superset is a
> > solution
> > > that delivers on all of these vectors.
> > >
> > > The technology stack is also constantly morphing - vendors are
> struggling
> > > to provide cheap, quick and easy solutions to access data.  Business
> > > intelligence users are finding existing solutions lacking as these
> > software
> > > products either disregard or react slowly to recent game-changing
> > > technologies like Druid.io, PrestoDB, Apache Drill, Apache Kylin,
> d3.js,
> > > React.js and iPython’s Jupyter for instance.
> > >
> > > == Rationale ==
> > >
> > > Business intelligence is more relevant today than at any other point in
> > > history.  Organizations are currently very limited in options for open
> > > source data visualization solutions, especially solutions that are both
> > > self-service and enterprise-ready.  Every company informing their
> > decisions
> > > with data needs a BI tool.
> > >
> > > We believe that Superset will be a strong compliment to existing Apache
> > > Software Foundation technologies by offering scalable user interactions
> > to
> > > distributed storage and computation solutions.  Users will often find
> > that
> > > Superset can act as a catalyst for tooling that can 

Re: Airflow voting on release artifacts

[Bertrand Delacretaz]

Hi,

On Mon, Apr 24, 2017 at 10:18 PM, Chris Riccomini  wrote:
> ...Hitesh recently raised the issue that the artifact that passes the vote
> MUST be the one that we actually release...

Yes in terms of having the same binary digests and signatures, but
renaming the files is fine IMO, especially for removing an -rc suffix
which makes total sense. I would just add that step to your release
process documentation to make it clear.

> ...Rename/rebuild after final vote (This is what Airflow is doing, and Beam
> does this, I believe)...

I'd say rename yes but rebuild no, in order to keep the same digests
and signatures.

-Bertrand

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Airflow voting on release artifacts

[Bolke de Bruin]

Hi John,

Typically one handles a “release candidate” equal to a release, except for the 
fact that it hasn’t been fully tested and reviewed yet. In case everyone (all 
entities) is satisfied with the RC it is then just rebranded to Release. 
Unfortunately, in the python ecosystem the version information is contained 
within the (source) package and as metadata by means of the filename. These two 
need to be in sync, otherwise package installers will get confused.

So imagine, if we started voting on “apache-airflow-1.8.1+incubating”, but it 
does *not* pass the vote due to a blocker bug or a license issue. We would need 
to restart the vote with an updated package but with the same name. 
Semantically, this is already confusing as you would need to examine the 
signatures to see if the version has changed. It is also risky for the 
users/testers and therefore for the developers: the new version will not 
trigger an upgrade with package installers and a new installation would need to 
be forced (contrary to how maven handles things for example). This means, we 
could receive bug reports on “1.8.1” which actually are “1.8.1-RC1” or 
“1.8.1-RC2” etc. 

In Apache we vote on artefacts. A rebrand delivers us two different artefacts: 
RC + Release. For us, as a podling, that would mean 4 votes: 2 at the PMC and 2 
at the IPMC. This is very time consuming for a very minor change.

Anecdotical evidence from two other projects shows that Apache Beam voted on 
0.6.0-RC2 as a release and then rebranded and updated packages after the vote 
(please correct me if I am wrong, just did a mailinglist scan and a comparison 
on the trees). Apache Libcloud (a pure python project) votes twice, but they 
seem to handle RCs differently, as there are large time gaps between the votes.

So we are seeking advice how to make efficient use of the Apache process, given 
the limitations of the ecosystem. I prefer what Apache Beam seems to be doing. 
I can also imagine to combine two artefacts in one vote: pointing everyone to 
the RC including it as a tar ball, but also including the tar ball and 
signatures for the *potential* rebranded version to release. Somehow we would 
need to hide the *potential* release as much from plain sight as possible, as 
we would have no way to distinguish between the actual release and the 
potential.

Bolke 

> On 25 Apr 2017, at 02:11, John D. Ament  wrote:
> 
> Do you need to cut a vote with something named rc?  Why can't you just use
> the version #?
> 
> On Mon, Apr 24, 2017 at 4:18 PM Chris Riccomini 
> wrote:
> 
>> Hey all,
>> 
>> We've had a question arise in the Airflow project. We're currently cutting
>> release candidates (RCs), and voting on those. These RCs contain an
>> artifact with the suffix -rc0, -rc1, etc. The problem with this is that the
>> final RC that passes the vote still has an -rcX in its version number. To
>> get around this, the release manager has just been rebuilding the .asc,
>> .sha, etc, and publishing this new artifact as the final release.
>> 
>> Hitesh recently raised the issue that the artifact that passes the vote
>> MUST be the one that we actually release. The problem with this is that it
>> would force us to publish multiple RCs with the exact same version number.
>> This is a terrible experience from a developer and user point of view.
>> Airflow is a Python project, and many users pip install various versions of
>> Airflow to test things out. Having multiple RCs with the same version
>> number is going to make it really difficult to keep things straight, both
>> during the VOTE, and afterwards.
>> 
>> Based on a cursory look, it seems that other projects handle this problem
>> in one of three ways:
>> 
>> * Double vote (the final RC gets voted AGAIN, this time, with proper
>> naming)
>> * Rename/rebuild after final vote (This is what Airflow is doing, and Beam
>> does this, I believe)
>> * All RCs have the exact same version number
>> 
>> Most Java projects are just publishing multiple RCs with the exact same
>> version number. This seems bad from a general software engineering
>> perspective, since it's going to cause build cache issues (e.g. .mvn
>> already contains version 1.2.3, even though a new RC with the same version
>> should be installed). Beam does seem to be using -SNAPSHOT releases in Java
>> for RCs, which is CLOSER to what we've been doing in Airflow (-rc0, -rc1,
>> etc), and they seem to be renaming after the release, just like we were.
>> 
>> What's the guidance here? We seem to have two requirements that are at odds
>> with eachtother:
>> 
>> 1) The RC artifact that we vote on is the one that we release
>> 2) We have real version numbers for RCs which are independent of one
>> another so build systems, dependency managements systems, and package
>> management systems can properly handle them.
>> 
>> Any help or thoughts would be appreciated.
>> 
>> Cheers,
>> Chris
>> 

Re: [PROPOSAL] Superset Proposal for Apache Incubator

[Maxime Beauchemin]

"Airbnb will submit a Software Grant Agreement (SGA) as Superset joins the
incubator."

Should I add this sentence in the proposal?

Max

On Mon, Apr 24, 2017 at 5:48 AM, John D. Ament 
wrote:

> I missed this discussion.  In your IP section, you list out:
>
> == Source and Intellectual Property Submission Plan ==
> We do not expect any complications for the submission of the Superset code
> base.  Our code is already in Github and there is only a single code base.
>
> This IMHO not clear.  Does Airbnb plan to submit a SGA for Superset, or
> expect that no SGA is required because its Apache licensed?
>
> John
>
> On Sun, Apr 2, 2017 at 4:09 PM Jeff Feng 
> wrote:
>
> > Dear Apache Incubator Community,
> >
> > We are excited to share our proposal for discussion and feedback for
> > entering Apache Incubation.  Superset is an enterprise-ready web
> > application for data exploration, data visualization and dashboarding.
> >
> > Our Incubation proposal is at the following Wiki as well as copied in the
> > email below:
> >
> > https://wiki.apache.org/incubator/SupersetProposal
> >
> > We have an active Superset community including 400+ members and nearly
> 200
> > topics.  The Google Group can be found below.  We plan to move the
> > discussion to the ASF:
> >
> > https://groups.google.com/forum/#!forum/airbnb_superset
> >
> > Thank you and look forward to the discussion!
> >
> > Jeff, Max & Alanna
> >
> >
> >
> >
> > = Superset =
> >
> > == Abstract ==
> >
> > Superset is an enterprise-ready web application for data exploration,
> data
> > visualization and dashboarding.
> >
> > == Proposal ==
> >
> > Superset is business intelligence (BI) software that helps modern
> > organizations visualize and interact with their data. Superset enables
> > users explore data from a variety of databases, assemble beautiful
> > dashboards and share their findings.  Superset works neatly with all
> modern
> > SQL-speaking databases, and integrates with Druid.io to provide
> real-time,
> > interactive, blazing fast data access to large datasets.
> >
> > == Background ==
> >
> > Data is mission critical. To succeed in this era, organizations need to
> > provide low-friction, intuitive and interactive access to data. It is
> > paramount for knowledge workers to be capable of answering their own
> > questions by querying, exploring and visualizing data.
> >
> > The entire business intelligence industry has pivoted from a model of
> > centralized top-down platforms driven by IT organizations to self-service
> > analytics and agile workflows by any user.  This shift unblocks
> centralized
> > service bottlenecks for creating data visualizations while also creating
> an
> > environment that is iterative and fast-moving.  This means that business
> > intelligence software must also be easy and delightful to use.
> > Self-service analytics doesn’t mean that admin and governance features
> are
> > not needed.
> >
> > Modern BI tools provide fine-grain access controls and auditing
> > capabilities to understand how data is being used.  Superset is a
> solution
> > that delivers on all of these vectors.
> >
> > The technology stack is also constantly morphing - vendors are struggling
> > to provide cheap, quick and easy solutions to access data.  Business
> > intelligence users are finding existing solutions lacking as these
> software
> > products either disregard or react slowly to recent game-changing
> > technologies like Druid.io, PrestoDB, Apache Drill, Apache Kylin, d3.js,
> > React.js and iPython’s Jupyter for instance.
> >
> > == Rationale ==
> >
> > Business intelligence is more relevant today than at any other point in
> > history.  Organizations are currently very limited in options for open
> > source data visualization solutions, especially solutions that are both
> > self-service and enterprise-ready.  Every company informing their
> decisions
> > with data needs a BI tool.
> >
> > We believe that Superset will be a strong compliment to existing Apache
> > Software Foundation technologies by offering scalable user interactions
> to
> > distributed storage and computation solutions.  Users will often find
> that
> > Superset can act as a catalyst for tooling that can visualize the
> byproduct
> > of data and computation infrastructure.
> >
> > Superset has many key design elements that help fill a gap in current
> > solutions for organizations:
> >
> > * Easy, low friction access to data through a simple, web-based data
> > exploration interface.  Composing charts and dashboards are intuitive.
> > Eliminating the need to write code or SQL empowers anyone to use it.
> >
> > * Access to a wide array of rich, interactive data visualization types.
> >
> > * Enterprise-ready: Integration with different authentication mechanisms
> > and granular permissions centered around actions and data access.
> >
> > * Realtime & fast: Superset provides realtime analytics at the speed of