Re: [DISCUSS] Graduate Apache Gobblin (Incubating) as a TLP

[Owen O'Malley]

Talking to Justin, although you removed the non-Apache logos from the
Gobblin home page, he would still like you to start a discussion with
trademarks to clarify what would be required to use them. That way, we'll
know the rules going forward.

Thanks,
   Owen

On Tue, Jan 5, 2021 at 10:30 PM Abhishek Tiwari  wrote:

> Hi Justin,
>
> Please let us know if this addresses the points that you had brought
> forward, and if we can move forward with the graduation?
>
> Thanks,
> Abhishek
>
> On Sun, Jan 3, 2021 at 11:20 PM Abhishek Tiwari  wrote:
>
> > Thanks for clarifying. So far, we are experimenting with Docker, so not
> > many downloads had happened (except from a few devs). We will tag it
> > clearly that they are not for public consumption if we enable publishing
> > nightly in future.
> >
> > Thanks,
> > Abhishek
> >
> > On Sun, Jan 3, 2021 at 11:10 PM Justin Mclean 
> > wrote:
> >
> >> HI,
> >>
> >> > 4. Regarding docker, per policy [1] I thought unreleased tags are fine
> >> (ie.
> >> > if its nightly or snapshot), where-as the *latest* tag should point to
> >> an
> >> > (approved) released code. Seems like my understanding was incorrect,
> so
> >> I
> >> > will remove the unreleased tags from there as well.
> >>
> >> You're correct that the latest tag should point to the the last approved
> >> release version of the code. I was not sure this was happening in this
> >> case. Re other tags it's generally OK if they are very clearly marked as
> >> not for public consumption/for project internal development use only. If
> >> they get excessive number of downloads you may need to reconsider
> putting
> >> them there as it may be the general public is using them.
> >>
> >> Thanks,
> >> Justin
> >> -
> >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> >> For additional commands, e-mail: general-h...@incubator.apache.org
> >>
> >>
>

Re: [VOTE] Graduate Apache Gobblin (Incubating) as a TLP

[Owen O'Malley]

+1 (binding)

On Thu, Dec 17, 2020 at 4:10 PM Jean-Baptiste Onofre 
wrote:

> +1 (binding)
>
> Regards
> JB
>
> > Le 17 déc. 2020 à 09:54, Abhishek Tiwari  a écrit :
> >
> > Hi all,
> >
> > After a discussion with the community [1], a positive vote
> > result [2], and a discussion with the IPMC [7], we believe
> > Apache Gobblin (Incubating) is ready for graduation to a TLP,
> > and we'd like to call for a Vote.
> >
> > Please vote on the resolution pasted below to graduate
> > Apache Gobblin (Incubating) from the incubator to a TLP:
> >
> > [ ] +1, Graduate Apache Gobblin from the Incubator.
> > [ ] +0, Don't care.
> > [ ] -1, Don't graduate Apache Gobblin from the Incubator (provide
> details)
> >
> > To list a few of the community's achievements:
> >
> > - 1700+ contributions from 149 contributors
> > - 4 releases under 3 different release managers
> > - 5 new committers invited, including 2 new PPMC members (all accepted)
> > - Diverse committers and PPMCs (from 9 companies/institutes)
> > - New Apache website setup [3]
> > - Issue tracking migrated to Jira [4]
> > - Dev conversations moved to d...@gobblin.incubator.org
> > - Apache Gobblin name search has been approved [5]
> > - Assessed ourselves against the Apache Project maturity matrix [6]
> > - We have built a meritocratic and open collaborative process (the Apache
> > way)
> >
> > [1] https://s.apache.org/le17y
> > [2] https://s.apache.org/q4ug6
> > [3] https://gobblin.incubator.apache.org/
> > [4] https://issues.apache.org/jira/projects/GOBBLIN/issues/
> > [5] https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-131
> > [6]
> >
> https://cwiki.apache.org/confluence/display/GOBBLIN/ASF+Maturity+Evaluation
> > [7] https://s.apache.org/1gt3s
> >
> > The vote will remain open for at least 72 hours.
> >
> > If approved by the Apache Incubator PMC members, the proposed
> > resolution will be submitted to the Board of Directors for their
> > consideration.
> >
> > Thanks,
> > Abhishek
> >
> > ---
> >
> > Establish the Apache Gobblin Project
> >
> > WHEREAS, the Board of Directors deems it to be in the best interests of
> > the Foundation and consistent with the Foundation's purpose to establish
> > a Project Management Committee charged with the creation and maintenance
> > of open-source software, for distribution at no charge to the public,
> > related to a distributed data integration framework that simplifies
> > common aspects of big data integration such as data ingestion,
> > replication, organization and lifecycle management for both streaming
> > and batch data ecosystems.
> >
> > NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
> > (PMC), to be known as the "Apache Gobblin Project", be and hereby is
> > established pursuant to Bylaws of the Foundation; and be it further
> >
> > RESOLVED, that the Apache Gobblin Project be and hereby is responsible
> > for the creation and maintenance of software related to a distributed
> > data integration framework that simplifies common aspects of big data
> > integration such as data ingestion, replication, organization and
> > lifecycle management for both streaming and batch data ecosystems; and
> > be it further
> >
> > RESOLVED, that the office of "Vice President, Apache Gobblin" be and
> > hereby is created, the person holding such office to serve at the
> > direction of the Board of Directors as the chair of the Apache Gobblin
> > Project, and to have primary responsibility for management of the
> > projects within the scope of responsibility of the Apache Gobblin
> > Project; and be it further
> >
> > RESOLVED, that the persons listed immediately below be and hereby are
> > appointed to serve as the initial members of the Apache Gobblin Project:
> >
> > * Abhishek Tiwari  
> > * Hung Tran 
> > * Issac Buenrostro 
> > * Jean-Baptiste Onofré 
> > * Kishore G
> > * Lorand Bendig
> > * Olivier Lamy 
> > * Owen O'Malley
> > * Sahil Takiar 
> > * Shirshanka Das   
> > * Sudarshan Vasudevan  
> > * Tamás Németh 
> > * Yinan Li 
> >
> > NOW, THEREFORE, BE IT FURTHER RESOLVED, that Abhishek Tiwari be
> > appointed to the office of Vice President, Apache Gobblin, to serve in
> > accordance with and subject to the direction of the

Re: [VOTE] Recommend Apache Iceberg graduation to top-level project resolution to the board

[Owen O'Malley]

+1 (binding)

.. Owen

> On May 15, 2020, at 17:40, Ryan Blue  wrote:
> 
> Hi everyone,
> 
> With the support of our mentors (as well as helpful ASF members), the
> Apache Iceberg community has voted to graduate to a top-level project.
> 
> I propose a vote to recommend graduation for the Iceberg community to the
> board. Here is the proposed resolution:
> 
> ```
> Establish the Apache Iceberg Project
> 
> WHEREAS, the Board of Directors deems it to be in the best interests of
> the Foundation and consistent with the Foundation's purpose to establish
> a Project Management Committee charged with the creation and maintenance
> of open-source software, for distribution at no charge to the public,
> related to managing huge analytic datasets using a standard at-rest
> table format that is designed for high performance and ease of use..
> 
> NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
> (PMC), to be known as the "Apache Iceberg Project", be and hereby is
> established pursuant to Bylaws of the Foundation; and be it further
> 
> RESOLVED, that the Apache Iceberg Project be and hereby is responsible
> for the creation and maintenance of software related to managing huge
> analytic datasets using a standard at-rest table format that is designed
> for high performance and ease of use; and be it further
> 
> RESOLVED, that the office of "Vice President, Apache Iceberg" be and
> hereby is created, the person holding such office to serve at the
> direction of the Board of Directors as the chair of the Apache Iceberg
> Project, and to have primary responsibility for management of the
> projects within the scope of responsibility of the Apache Iceberg
> Project; and be it further
> 
> RESOLVED, that the persons listed immediately below be and hereby are
> appointed to serve as the initial members of the Apache Iceberg Project:
> 
> * Anton Okolnychyi 
> * Carl Steinbach   
> * Daniel C. Weeks  
> * James R. Taylor  
> * Julien Le Dem
> * Owen O'Malley
> * Parth Brahmbhatt 
> * Ratandeep Ratti  
> * Ryan Blue
> 
> NOW, THEREFORE, BE IT FURTHER RESOLVED, that Ryan Blue be appointed to
> the office of Vice President, Apache Iceberg, to serve in accordance
> with and subject to the direction of the Board of Directors and the
> Bylaws of the Foundation until death, resignation, retirement, removal
> or disqualification, or until a successor is appointed; and be it
> further
> 
> RESOLVED, that the Apache Iceberg Project be and hereby is tasked with
> the migration and rationalization of the Apache Incubator Iceberg
> podling; and be it further
> 
> RESOLVED, that all responsibilities pertaining to the Apache Incubator
> Iceberg podling encumbered upon the Apache Incubator PMC are hereafter
> discharged.
> ```
> 
> The community vote passed with 19 +1 votes:
> https://lists.apache.org/thread.html/r9081df0181768f26490f5a85aed7b5a111a82a741764eb3a0e99621d%40%3Cdev.iceberg.apache.org%3E
> 
> We also addressed concerns on the IPMC thread, which you can read here:
> https://lists.apache.org/thread.html/r3e5795d959feb0a19b233aeaf1121a1d97fd473f5e9b14227de41c54%40%3Cgeneral.incubator.apache.org%3E
> 
> Please vote on whether to recommend graduation for the Apache Iceberg
> community to the board.
> 
> [ ] +1 Apache Iceberg should graduate
> [ ] +0
> [ ] -1 Apache Iceberg should not graduate because . . .
> 
> The vote will be open for at least 72 hours.
> 
> -- 
> Ryan Blue

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Graduate to a top-level project

[Owen O'Malley]

On Tue, May 12, 2020 at 10:44 PM Justin Mclean 
wrote:

> I notice a few minor things that may need to be sorted out/dicussed:

- You have only made two releases and the last one had some issue around
> licensing and NOTICE files
>

Obviously the first release at Apache takes the longest as the code and
processes migrate over. The second release was about 6 months after the
first, which is probably a bit long for a young project. I think that
moving closer to quarterly is probably a good goal.

We'll also keep a watch on bugs that should get released in 0.8.1.

- Several of the proposed PMC members don’t seem to be subscribed to the
> projects private list
>

I have sent emails (cc'ed private) to get the relevant members to
subscribe. Thanks for pointing that out.

.. Owen

Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC2

[Owen O'Malley]

+1 (binding)

On Mon, May 4, 2020 at 8:56 AM Ryan Blue  wrote:

> Thanks, Justin. Can you be more specific about what you think isn't correct
> in the NOTICE file? Our practice is to copy the entire notice of
> third-party projects, if they exist. We don't do this for the boiler-plate
> ASF NOTICE, but any ASF notice that contains other text will be copied in
> (like Yetus in the runtime Jar).
>
> On Sun, May 3, 2020 at 7:49 PM Justin Mclean 
> wrote:
>
> > Hi,
> >
> > +1(binding)
> >
> > I checked:
> > - incubating in name
> > - signature and hashes are fine
> > - DISCLAIMER exits
> > - LICENSE is fine but may mention code that is no longer bundled? (e.g
> mvn)
> > - NOTICE year needs updating
> > - NOTICE content are not correct. There is a number of ALv2 bt of code
> > bundles and the relevant parts of their NOTICE files haven’t been copied
> > into the NOTICE.  [1] Please fix for the next release. There is no need
> for
> > the license header to be in there.
> > - No unexpected binary files
> > - All ASF files have ASF headers
> > -  I had trouble compiling from source as it required a new version of
> > cradle than I had on my machine but is incompatible with the latest
> version
> > off cradle (7.0)
> >
> > Thanks,
> > Justin
> >
> > 1. https://www.apache.org/dev/licensing-howto.html
> > -
> > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> > For additional commands, e-mail: general-h...@incubator.apache.org
> >
> >
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>

Re: [VOTE] Accept the Iceberg project for incubation

[Owen O'Malley]

+1 (binding)

On Tue, Nov 13, 2018 at 12:12 PM Dave Fisher  wrote:

> +1 (binding)
>
> > On Nov 13, 2018, at 9:10 AM, Matt Sicker  wrote:
> >
> > +1 binding
> >
> > On Tue, 13 Nov 2018 at 11:09, Ryan Blue  wrote:
> >
> >> +1 (binding)
> >>
> >> On Tue, Nov 13, 2018 at 9:06 AM Ryan Blue  wrote:
> >>
> >>> The discuss thread seems to have reached consensus, so I propose
> >> accepting
> >>> the Iceberg project for incubation.
> >>>
> >>> The proposal is copied below and in the wiki:
> >>> https://wiki.apache.org/incubator/IcebergProposal
> >>>
> >>> Please vote on whether to accept Iceberg in the next 72 hours:
> >>>
> >>> [ ] +1, accept Iceberg for incubation
> >>> [ ] -1, reject the Iceberg proposal because . . .
> >>>
> >>> Thank you for reviewing the proposal and voting,
> >>>
> >>> rb
> >>> --
> >>> Iceberg Proposal Abstract
> >>>
> >>> Iceberg is a table format for large, slow-moving tabular data.
> >>>
> >>> It is designed to improve on the de-facto standard table layout built
> >> into
> >>> Apache Hive, Presto, and Apache Spark.
> >>> Proposal
> >>>
> >>> The purpose of Iceberg is to provide SQL-like tables that are backed by
> >>> large sets of data files. Iceberg is similar to the Hive table layout,
> >> the
> >>> de-facto standard structure used to track files in a table, but
> provides
> >>> additional guarantees and performance optimizations:
> >>>
> >>>   - Atomicity - Each change to the table is will be complete or will
> >>>   fail. “Do or do not. There is no try.”
> >>>   - Snapshot isolation - Reads use one and only one snapshot of a table
> >>>   at some time without holding a lock.
> >>>   - Safe schema evolution - A table’s schema can change in well-defined
> >>>   ways, without breaking older data files.
> >>>   - Column projection - An engine may request a subset of the available
> >>>   columns, including nested fields.
> >>>   - Predicate pushdown - An engine can push filters into read planning
> >>>   to improve performance using partition data and file-level
> statistics.
> >>>
> >>> Iceberg does NOT define a new file format. All data is stored in Apache
> >>> Avro, Apache ORC, or Apache Parquet files.
> >>>
> >>> Additionally, Iceberg is designed to work well when data files are
> stored
> >>> in cloud blob stores, even when those systems provide weaker guarantees
> >>> than a file system, including:
> >>>
> >>>   - Eventual consistency in the namespace
> >>>   - High latency for directory listings
> >>>   - No renames of objects
> >>>   - No folder hierarchy
> >>>
> >>> Rationale
> >>>
> >>> Initial benchmarks show dramatic improvements in query planning. For
> >>> example, in Netflix’s Atlas use case, which stores time-series metrics
> >> from
> >>> Netflix runtime systems and 1 month is stored across 2.7 million files
> in
> >>> 2,688 partitions:
> >>>
> >>>   - Hive table using Parquet:
> >>>  - 400k+ splits, not combined
> >>>  - Explain query: 9.6 minutes wall time (planning only)
> >>>   - Iceberg table with partition filtering:
> >>>  - 15,218 splits, combined
> >>>  - Planning: 10 seconds
> >>>  - Query wall time: 13 minutes
> >>>   - Iceberg table with partition and min/max filtering:
> >>>  - 412 splits
> >>>  - Planning: 25 seconds
> >>>  - Query wall time: 42 seconds
> >>>
> >>> These performance gains combined with the cross-engine compatibility
> are
> >> a
> >>> very compelling story.
> >>> Initial Goals
> >>>
> >>> The initial goal will be to move the existing codebase to Apache and
> >>> integrate with the Apache development process and infrastructure. A
> >> primary
> >>> goal of incubation will be to grow and diversify the Iceberg community.
> >> We
> >>> are well aware that the project community is largely comprised of
> >>> individuals from a single company. We aim to change that during
> >> incubation.
> >>> Current Status
> >>>
> >>> As previously mentioned, Iceberg is under active development at
> Netflix,
> >>> and is being used in processing large volumes of data in Amazon EC2.
> >>>
> >>> Iceberg license documentation is already based on Apache guidelines for
> >>> LICENSE and NOTICE content.
> >>> Meritocracy
> >>>
> >>> We value meritocracy and we understand that it is the basis for an open
> >>> community that encourages multiple companies and individuals to
> >> contribute
> >>> and be invested in the project’s future. We will encourage and monitor
> >>> participation and make sure to extend privileges and responsibilities
> to
> >>> all contributors.
> >>> Community
> >>>
> >>> Iceberg is currently being used by developers at Netflix and a growing
> >>> number of users are actively using it in production environments.
> Iceberg
> >>> has received contributions from developers working at Hortonworks,
> >> WeWork,
> >>> and Palantir. By bringing Iceberg to Apache we aim to assure current
> and
> >>> future contributors that the Iceberg community is meritocratic and
> open,
> >> in
> >>> 

Re: [DISCUSS] Storage-class memory ecosystem program

[Owen O'Malley]

I can represent ORC within the group.

.. Owen

> On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang  wrote:
> 
> Hi all,
> 
> We can expect more and more projects will take the huge potential
> advantages of storage-class memory for data processing and analytics
> because silicon companies are able to produce high capacity non-volatile
> memory on a large scale, this hardware technology will fundamentally change
> the way to construct high performance applications similar to what happened
> when replacing tape with disk technology since the 1980s. so if possible, I
> advocate establishing an Apache working group to enhance the collaboration
> and synergies mentioned by Patrick Stuedi for storage-class memory
> technology-oriented projects.
> 
> Best.
> Gary.


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Graduate Apache Ranger Project from the Incubator - Resending with additional mail distro

[Owen O'Malley]

+1 (binding)

On Wed, Jan 4, 2017 at 5:15 PM, Balaji Ganesan 
wrote:

> +1 Great to see the progress from the Ranger community.
>
> On Wed, Jan 4, 2017 at 4:54 PM, Suneel Marthi  wrote:
>
> > +1 binding
> >
> > On Wed, Jan 4, 2017 at 5:48 PM, Ramesh Mani 
> wrote:
> >
> > > Dear Incubator members,
> > >
> > > Apache Ranger Project community has successfully released 0.6.2 version
> > > and with it there had been a lot of discussion within Apache Ranger
> > > community to consider graduation to TLP. Apache Ranger entered into
> > > incubation on 24th July 2014 and from this welcoming community had
> done a
> > > tremendous job in resolving various technical hurdles like refactoring
> > the
> > > project core model to  be service based, adding more Apache Hadoop
> > > components like Apache YARN, Apache Storm, Apache Kafka, Apache Nifi,
> > > Apache Ranger KMS into Ranger Authorizing  model for security and
> making
> > it
> > > into a core product in the Apache Hadoop security space. PPMC has
> > exhibited
> > > a clear understanding of this growing apache community by electing  4
> > > individuals as committers  and  inculding 22 individuals as
> contributors
> > to
> > > the Apache Ranger project. PPMC also has done 8 successful releases
> under
> > > the guidance of mentors demonstrating their mastery over AFS’s IP
> > policies.
> > >
> > > An voting was conducted within Apache Ranger Community to graduate
> Apache
> > > Ranger Project to Top Level Project. Vote passed with 16 +1 votes , no
> 0
> > or
> > > –1 votes.
> > > http://mail-archives.apache.org/mod_mbox/incubator-ranger-
> > > dev/201612.mbox/%3CD479D4C8.11E4E%25rmani%40hortonworks.com%3E
> > >
> > > Apache Ranger Project has shown a great perspective to become a true
> TLP.
> > > Following summary on the project reflects its accomplishment.
> > >
> > > Please vote on the Project resolution that is found in bottom to
> graduate
> > > Apache Ranger Project from Incubator to Top Level Project.
> > >
> > > [ ] +1 Graduate Apache Ranger from the Incubator.
> > > [ ] +0 No opinion
> > > [ ] -1 Don't graduate Apache Ranger from the Incubator ( please provide
> > > the reason)
> > >
> > > This VOTE will be opened for next 72 hours.
> > >
> > > Thanks all Mentors and Apache Ranger Project members for their support
> > and
> > > contributions.
> > >
> > > Here is my vote +1 (binding)
> > >
> > > Project Summary:
> > > =
> > >
> > > http://incubator.apache.org/projects/ranger.html
> > >
> > > Project website:
> > > =
> > >
> > > http://ranger.incubator.apache.org >
> > >
> > > Project Documentation:
> > > ===
> > >
> > > http://ranger.incubator.apache.org/index.html
> > > http://ranger.incubator.apache.org/quick_start_guide.html
> > > https://cwiki.apache.org/confluence/display/RANGER/Release+Folders
> > >
> > > Project maturity Assessment:
> > > ===
> > >
> > > https://cwiki.apache.org/confluence/display/RANGER/
> > > Apache+Ranger+Project+Ma
> > > turity+Model
> > >
> > > Proposed PMC size: 17
> > >
> > > Total number of committers   : 14 members
> > > Total number of contributors : 22 members
> > >
> > > PMC affiliation (* indicated chair)
> > >
> > > * Hortonworks (9)
> > >Privacera (2)
> > >BlueTalon (1)
> > >Others(1)
> > >
> > > 1802 commits on develop
> > > 22 contributors across all branches
> > > Dev list averaged ~50 msgs/month in 2016
> > > User list averaged ~40 msgs/month in 2016
> > > 1208 issues created
> > > 997 issues resolved
> > >
> > > Committer’s affiliation:
> > > ===
> > > Active:
> > > Hortonworks
> > > Talend
> > > Freestone infotech
> > > BlueTalon
> > > eBay
> > > Others
> > >
> > >
> > > Apache Ranger Top Level Project Resolution:
> > > 
> > >
> > > Establish the Apache Ranger Project
> > >
> > > WHEREAS, the Board of Directors deems it to be in the best interests of
> > > the Foundation and consistent with the Foundation’s purpose to
> establish
> > a
> > > Project Management Committee charged with the creation and maintenance
> of
> > > open-source software, for distribution at no charge to the public,
> > related
> > > to a data management platform That provides real-time, consistent
> access
> > > to data-intensive applications throughout widely distributed cloud
> > > architectures.
> > >
> > > NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
> > > (PMC), to be known as the "Apache Ranger Project", be and hereby is
> > > established pursuant to Bylaws of the Foundation; and be it further
> > >
> > > RESOLVED,that the Apache Ranger Project be and hereby is responsible
> for
> > > the creation and maintenance of software related to a data management
> > > platform that provides real-time, consistent access to data-intensive
> > > applications 

[VOTE] release of Metron_0.1BETA_rc7

[Owen O'Malley]

+1 on the release.

I tested:
  * The SHA and MD5 checksums are correct.
  * The GPG signature is correct.
  * The metron-streaming module builds and the unit tests pass.
  * The LICENSE and NOTICE file look correct.

Things that should be improved before the next release:
  * The top level pom.xml should build the entire project.
  * There should be build instructions included in the release with a
description of the various
 modules and what they are intended for.
  * The release numbers should have three levels 0.1.0 instead of 0.1 so
that you can do patch
 releases.
  * James' gpg key should be signed by others in the Apache web of trust.
I'll set up a phone call
with James to get that done.

.. Owen

> > This is a call to vote on releasing Apache Metron 0.1BETA-RC7
> > >
> > > Full list of changes in this release:
> > >
> >
> https://dist.apache.org/repos/dist/dev/incubator/metron/0.1BETA-RC7-incubating/CHANGES
> > >
> > > The tag/commit to be voted upon is Metron_0.1BETA_rc7:
> > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=incubator-metron.git;a=commit;h=ad3866bdf4b6233950e7803c3c3141f0f859e994
> > >
> > > The source archive being voted upon can be found here:
> > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/incubator/metron/0.1BETA-RC7-incubating/apache-metron-0.1BETA-RC7-incubating.tar.gz
> > >
> > > Other release files, signatures and digests can be found here:
> > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/incubator/metron/0.1BETA-RC7-incubating/
> > >
> > > The release artifacts are signed with the following key:
> > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=incubator-metron.git;a=blob_plain;f=KEYS;hb=dc59e37e402bd868aeac7ab42a0cc9c51ccae3c2
> > >
> > > The Nexus staging repository for this release will be created after
> this
> > vote has been passed.
> > >
> > > Please vote on releasing this package as Apache Metron 0.1BETA-RC7.
> > >
> > > Reference to the Metron PPMC vote can be found here:
> > >
> >
> http://mail-archives.apache.org/mod_mbox/incubator-metron-dev/201604.mbox/%3C0DFDA611-5C6E-43FB-A64E-291DD2B02413%40hortonworks.com%3E
> > >
> > > When voting, please list the actions taken to verify the release.
> > >
> > > Recommended build validation and verification instructions are posted
> > here:
> > > https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds
> > >
> > > This vote will be open for at least 72 hours.
> > > [ ] +1 Release this package as Apache Metron 0.1BETA-RC7
> > > [ ]  0 No opinion
> > > [ ] -1 Do not release this package because...
>

Re: svn commit: r1718228 - in /incubator/public/trunk/content: podlings.xml projects/metron.xml

[Owen O'Malley]

Thanks, that is a good point.

.. Owen

On Sun, Dec 6, 2015 at 3:48 PM, John D. Ament <johndam...@apache.org> wrote:

> Owen,
>
> I'd strongly advise you to add Metron to reporting group 1 and make the
> first report in January instead of trying to rush a report in for this
> month.  The podling reports are already past due.
>
> John
>
> On Sun, Dec 6, 2015 at 4:50 PM <omal...@apache.org> wrote:
>
> > Author: omalley
> > Date: Sun Dec  6 21:50:16 2015
> > New Revision: 1718228
> >
> > URL: http://svn.apache.org/viewvc?rev=1718228=rev
> > Log:
> > Add Metron as new podling.
> >
> > Added:
> > incubator/public/trunk/content/projects/metron.xml   (with props)
> > Modified:
> > incubator/public/trunk/content/podlings.xml
> >
> > Modified: incubator/public/trunk/content/podlings.xml
> > URL:
> >
> http://svn.apache.org/viewvc/incubator/public/trunk/content/podlings.xml?rev=1718228=1718227=1718228=diff
> >
> >
> ==
> > --- incubator/public/trunk/content/podlings.xml [utf-8] (original)
> > +++ incubator/public/trunk/content/podlings.xml [utf-8] Sun Dec  6
> > 21:50:16 2015
> > @@ -1278,6 +1278,18 @@ and a set of useful extensions for this
> >  Noah Slater
> >  
> >  
> > + > sponsor="Incubator" startdate="2015-12-06">
> > +Metron is a project dedicated to providing an
> > extensible and scalable advanced network security analytics tool. It has
> > strong foundations in the Apache Hadoop ecosystem.
> > +December, January,
> > February
> > +Owen O'Malley
> > +
> > +Billie Rinaldi
> > +Chris Mattmann
> > +Owen O'Malley
> > +P. Taylor Goetz
> > +Vinod Kumar Vavilapalli
> > +
> > +
> >   > sponsor="HTTP Server" startdate="2005-08-06" enddate="2007-02">
> >  FTP protocol module for Apache httpd
> > 2.x
> >  http://httpd.apache.org/mod_ftp/"/>
> >
> > Added: incubator/public/trunk/content/projects/metron.xml
> > URL:
> >
> http://svn.apache.org/viewvc/incubator/public/trunk/content/projects/metron.xml?rev=1718228=auto
> >
> >
> ==
> > --- incubator/public/trunk/content/projects/metron.xml (added)
> > +++ incubator/public/trunk/content/projects/metron.xml [utf-8] Sun Dec  6
> > 21:50:16 2015
> > @@ -0,0 +1,388 @@
> > +
> > +
> > +  
> > +
> > +Metron Incubation Status
> > +http://purl.org/DC/elements/1.0/; rel="schema.DC"/>
> > +  
> > +  
> > +
> > +  Metron Project Incubation Status
> > +  This page tracks the project status, incubator-wise. For more
> > general
> > + project status, look on the project website.
> > +
> > +
> > +  Description
> > +Metron integrates a variety of open source big data technologies in
> > +order to offer a centralized tool for security monitoring and
> > +analysis. Metron provides capabilities for log aggregation, full
> > +packet capture indexing, storage, advanced behavioral analytics and
> > +data enrichment, while applying the most current threat-intelligence
> > +information to security telemetry within a single platform.
> > +
> > +Metron can be divided into 4 areas:
> > +
> > +A mechanism to capture, store, and normalize any type of security
> > +telemetry at extremely high rates. Because security telemetry is
> > +constantly being generated, it requires a method for ingesting the
> > +data at high speeds and pushing it to various processing units for
> > +advanced computation and analytics.
> > +Real time processing and application of enrichments such as threat
> > +intelligence, geolocation, and DNS information to telemetry being
> > +collected. The immediate application of this information to incoming
> > +telemetry provides the context and situational awareness, as well as
> > +the “who” and “where” information that is critical for
> investigation.
> > +Efficient information storage based on how the information will be
> > used:
> > +
> > +  Logs and telemetry are stored such that they can be efficiently
> > mined
> > +  and analyzed for concise security visibility
> > +  The ability to extract and reconstruct full pac

[RESULT][VOTE] Accept Metron into Apache Incubator

[Owen O'Malley]

With 10 binding +1's and 10 non-binding +1's and no -1's, the vote to
accept Metron in the Apache Incubator passes. Thank you everyone.

Binding +1's:
  Owen O'Malley
  Chris Nauroth
  Chris Mattmann
  Seetharan Venkatesh
  P. Taylor Goetz
  Vinod Kumar Vavilapalli
  Brock Noland
  Billie Rinaldi
  Jacques Nadeau
  Julian Hyde

Non-binding +1's:
  Joe Witt
  Larry McCay
  Debo Dutta
  Phillip Rhodes
  Dave Hirko
  Brad Kolarov
  James Sirota
  Ryan Merriman
  Amol Kekre
  Balaji Ganesan

.. Owen

Re: [DISCUSS] Metron incubator proposal

[Owen O'Malley]

Phillip,
   The current committers have voted to add you. I've updated the proposal
to reflect that.

.. Owen

On Tue, Dec 1, 2015 at 2:37 PM, Phillip Rhodes <motley.crue@gmail.com>
wrote:

> I contributed to this project a little bit (as an outside consultant) back
> when Cisco was running it, and would be interested in staying involved with
> it if this happens.
>
>
> Phil
>
>
> This message optimized for indexing by NSA PRISM
>
> On Tue, Dec 1, 2015 at 2:33 PM, Greg Stein <gst...@gmail.com> wrote:
>
> > RIght. That's why I call it "opt-out" rather than opt-in. If there is no
> > active community to opt out, then I see no problem with us picking up the
> > codebase and (re)forming a community around it.
> >
> > With ALv2 licensing on the codebase, I believe we have most of the rights
> > we need, but will continue that over on legal-discuss.
> >
> > Cheers,
> > -g
> >
> > On Tue, Dec 1, 2015 at 9:16 AM, Chris Mattmann <mattm...@apache.org>
> > wrote:
> >
> > > Hi Bertrand,
> > >
> > > Taken in the opposite case, is there someone from OpenSOC
> > > objecting to this code being forked? It doesn’t sound like
> > > there is a community there.
> > >
> > >
> > > Cheers,
> > > Chris
> > >
> > > -Original Message-
> > > From: Bertrand Delacretaz <bdelacre...@apache.org>
> > > Reply-To: "general@incubator.apache.org" <general@incubator.apache.org
> >
> > > Date: Tuesday, December 1, 2015 at 2:02 AM
> > > To: Incubator General <general@incubator.apache.org>
> > > Subject: Re: [DISCUSS] Metron incubator proposal
> > >
> > > >Hi Owen,
> > > >
> > > >On Mon, Nov 30, 2015 at 5:55 PM, Owen O'Malley <omal...@apache.org>
> > > wrote:
> > > >> ...The
> > > >> condensed version is that the original code base (OpenSOC) was
> created
> > > >>by a
> > > >> company (Cisco) that put it on github as ALv2, but then hasn't been
> > > >>working
> > > >> on it. We posted a message
> > > >>
> > > >><
> > >
> https://groups.google.com/d/msg/opensoc-support/rFlW2uSSvmU/Sw_cO-T2AAAJ
> > > >>>
> > > >> to the OpenSOC support group a month ago proposing a move to Apache
> > and
> > > >>got
> > > >> a single positive response...
> > > >
> > > >The response I see there ("this is great news - looking forward to
> > > >working with you") does not look to me like approval from whoever owns
> > > >that code to move it to the ASF.
> > > >
> > > >What's needed IMO is approval from whoever owns the code that's at
> > > >https://github.com/OpenSOC/opensoc . As usual, although the license
> of
> > > >that code would allow us to fork, we only accept voluntary
> > > >contributions.
> > > >
> > > >-Bertrand
> > > >
> > > >-
> > > >To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> > > >For additional commands, e-mail: general-h...@incubator.apache.org
> > > >
> > >
> > >
> > >
> > > -
> > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> > > For additional commands, e-mail: general-h...@incubator.apache.org
> > >
> > >
> >
>

Re: [VOTE] Accept Metron into Apache Incubator

[Owen O'Malley]

+1 (binding)

On Thu, Dec 3, 2015 at 9:33 AM, Owen O'Malley <omal...@apache.org> wrote:

> The [DISCUSS] thread has would down, so I'd like to start a VOTE on
> whether Apache Incubator should accept Metron as a podling. The proposal is
> pasted below and is available on the wiki as well.
>
> https://wiki.apache.org/incubator/MetronProposal
>
> We've added a paragraph in the background section discussing how Apache
> avoids hostile forks of projects, because we don't want to fork
> communities. We've also added Larry McCay, P. Taylor Goetz, and Phillip
> Rhodes to the proposal.
>
> The vote will run until 12pm PST on Sunday.
>
> Thanks,
>Owen
>
> = Apache Metron Proposal =
>
> 
> /!\ '''FINAL''' /!\
>
> This proposal is now complete and has been submitted for a VOTE.
> 
>
> == Abstract ==
>
> The Metron project is an open source project dedicated to providing an
> extensible and scalable advanced security analytics tool. It has strong
> foundations in the Apache Hadoop ecosystem.
>
> == Proposal ==
>
> Metron integrates a variety of open source big data technologies in order
> to offer a centralized tool for security monitoring and analysis. Metron
> provides capabilities for log aggregation, full packet capture indexing,
> storage, advanced behavioral analytics and data enrichment, while applying
> the most current threat-intelligence information to security telemetry
> within a single platform.
>
> Metron can be divided into 4 areas:
>
>   1. '''A mechanism to capture, store, and normalize any type of security
> telemetry at extremely high rates.''' Because security telemetry is
> constantly being generated, it requires a method for ingesting the data at
> high speeds and pushing it to various processing units for advanced
> computation and analytics.
>   1. '''Real time processing and application of enrichments''' such as
> threat intelligence, geolocation, and DNS information to telemetry being
> collected. The immediate application of this information to incoming
> telemetry provides the context and situational awareness, as well as the
> “who” and “where” information that is critical for investigation.
>   1. '''Efficient information storage''' based on how the information will
> be used:
> a. Logs and telemetry are stored such that they can be efficiently
> mined and analyzed for concise security visibility
> a. The ability to extract and reconstruct full packets helps an
> analyst answer questions such as who the true attacker was, what data was
> leaked, and where that data was sent
> a. Long-term storage not only increases visibility over time, but also
> enables advanced analytics such as machine learning techniques to be used
> to create models on the information. Incoming data can then be scored
> against these stored models for advanced anomaly detection.
>   1. '''An interface that gives a security investigator a centralized view
> of data and alerts passed through the system.''' Metron’s interface
> presents alert summaries with threat intelligence and enrichment data
> specific to that alert on one single page. Furthermore, advanced search
> capabilities and full packet extraction tools are presented to the analyst
> for investigation without the need to pivot into additional tools.
>
> Big data is a natural fit for powerful security analytics. The Metron
> framework integrates a number of elements from the Hadoop ecosystem to
> provide a scalable platform for security analytics, incorporating such
> functionality as full-packet capture, stream processing, batch processing,
> real-time search, and telemetry aggregation. With Metron, our goal is to
> tie big data into security analytics and drive towards an extensible
> centralized platform to effectively enable rapid detection and rapid
> response for advanced security threats.
>
> == Background ==
>
> OpenSOC was developed by Cisco over the last two years and pushed out to
> Github (https://github.com/OpenSOC/opensoc) under the ALv2. However, the
> development was mostly closed and has largely stopped. As evidence of the
> inactivity, users have complained that pull requests are not answered for a
> while
> https://groups.google.com/d/msg/opensoc-support/R2W-ZFux8Vk/Y-5tL-EmAAAJ.
> Finally, no public releases of OpenSOC have been made. From an Apache point
> of view, the current community is not viable.
>
> However, some of the developers of the project have left Cisco and have
> found interest from several others that would like to work together to form
> an active and open community at Apache starting from the current OpenSOC
> code base. A message to the current support group proposing moving to
> Apache got a single posit

[VOTE] Accept Metron into Apache Incubator

[Owen O'Malley]

The [DISCUSS] thread has would down, so I'd like to start a VOTE on whether
Apache Incubator should accept Metron as a podling. The proposal is pasted
below and is available on the wiki as well.

https://wiki.apache.org/incubator/MetronProposal

We've added a paragraph in the background section discussing how Apache
avoids hostile forks of projects, because we don't want to fork
communities. We've also added Larry McCay, P. Taylor Goetz, and Phillip
Rhodes to the proposal.

The vote will run until 12pm PST on Sunday.

Thanks,
   Owen

= Apache Metron Proposal =


/!\ '''FINAL''' /!\

This proposal is now complete and has been submitted for a VOTE.


== Abstract ==

The Metron project is an open source project dedicated to providing an
extensible and scalable advanced security analytics tool. It has strong
foundations in the Apache Hadoop ecosystem.

== Proposal ==

Metron integrates a variety of open source big data technologies in order
to offer a centralized tool for security monitoring and analysis. Metron
provides capabilities for log aggregation, full packet capture indexing,
storage, advanced behavioral analytics and data enrichment, while applying
the most current threat-intelligence information to security telemetry
within a single platform.

Metron can be divided into 4 areas:

  1. '''A mechanism to capture, store, and normalize any type of security
telemetry at extremely high rates.''' Because security telemetry is
constantly being generated, it requires a method for ingesting the data at
high speeds and pushing it to various processing units for advanced
computation and analytics.
  1. '''Real time processing and application of enrichments''' such as
threat intelligence, geolocation, and DNS information to telemetry being
collected. The immediate application of this information to incoming
telemetry provides the context and situational awareness, as well as the
“who” and “where” information that is critical for investigation.
  1. '''Efficient information storage''' based on how the information will
be used:
a. Logs and telemetry are stored such that they can be efficiently
mined and analyzed for concise security visibility
a. The ability to extract and reconstruct full packets helps an analyst
answer questions such as who the true attacker was, what data was leaked,
and where that data was sent
a. Long-term storage not only increases visibility over time, but also
enables advanced analytics such as machine learning techniques to be used
to create models on the information. Incoming data can then be scored
against these stored models for advanced anomaly detection.
  1. '''An interface that gives a security investigator a centralized view
of data and alerts passed through the system.''' Metron’s interface
presents alert summaries with threat intelligence and enrichment data
specific to that alert on one single page. Furthermore, advanced search
capabilities and full packet extraction tools are presented to the analyst
for investigation without the need to pivot into additional tools.

Big data is a natural fit for powerful security analytics. The Metron
framework integrates a number of elements from the Hadoop ecosystem to
provide a scalable platform for security analytics, incorporating such
functionality as full-packet capture, stream processing, batch processing,
real-time search, and telemetry aggregation. With Metron, our goal is to
tie big data into security analytics and drive towards an extensible
centralized platform to effectively enable rapid detection and rapid
response for advanced security threats.

== Background ==

OpenSOC was developed by Cisco over the last two years and pushed out to
Github (https://github.com/OpenSOC/opensoc) under the ALv2. However, the
development was mostly closed and has largely stopped. As evidence of the
inactivity, users have complained that pull requests are not answered for a
while
https://groups.google.com/d/msg/opensoc-support/R2W-ZFux8Vk/Y-5tL-EmAAAJ.
Finally, no public releases of OpenSOC have been made. From an Apache point
of view, the current community is not viable.

However, some of the developers of the project have left Cisco and have
found interest from several others that would like to work together to form
an active and open community at Apache starting from the current OpenSOC
code base. A message to the current support group proposing moving to
Apache got a single positive response.
https://groups.google.com/d/msg/opensoc-support/rFlW2uSSvmU/09PIsWL4AAAJ

In general Apache accepts only voluntary contributions and avoids
hostile forks. In this case, given that the community is demonstrably
dead, it seems fair to fork the existing code at Apache to allow a new
community to work on it. Once incubation starts, we will send a
message pointing to the new home to the OpenSOC support group.

Because Cisco is not currently interested in being involved, the project
expects to change their name. The project would like to 

Re: [VOTE] Accept Metron into Apache Incubator

[Owen O'Malley]

On Thu, Dec 3, 2015 at 10:04 AM, Debo Dutta (dedutta) 
wrote:

> Would like to know who in Cisco was asked actually. I am from Cisco and
> can help.


Debo,
   If you can help get an SGA signed that would be great. I don't have any
contacts in Cisco, so I didn't have anywhere to ask.

.. Owen

Re: [VOTE] Accept Metron into Apache Incubator

[Owen O'Malley]

On Thu, Dec 3, 2015 at 9:39 AM, Ted Dunning  wrote:

> I think that there was a still pending question of why not ask Cisco for an
> SGA for cleanliness.


The answer that I gave you on board@apache was that large corporations
don't sign legal documents unless they perceive that they have more to gain
the potential risk of signing the document. Cisco has *nothing* to gain by
signing an SGA.

.. Owen

Re: [RESULT] [VOTE] Accept Impala into the Apache Incubator

[Owen O'Malley]

Not that it changes the result, but I also voted +1.

https://www.mail-archive.com/general@incubator.apache.org/msg52096.html

.. Owen

On Tue, Dec 1, 2015 at 1:51 PM, Henry Robinson  wrote:

> This vote is now closed and passes with 17 binding +1 votes, 7 non-binding
> +1 votes and 4 binding -1 votes.
>
> Thanks to everyone that voted!
>
> Here is my tally of the votes:
>
> Binding +1s (17):
> Todd Lipcon
> Arvind Prabhakar
> Chris Mattmann
> Julien Le Dem
> Carl Steinbach
> Brock Noland
> Tom White
> Alex Karasulu
> Jarek Jarcec Cecho
> Chris Douglas
> Doug Cutting
> Hitesh Shah
> Julian Hyde
> Ted Dunning
> Andrew Bayer
> Jean-Baptiste Onofré
> Andrei Savu
> Michael Stack
>
> Binding -1s (4):
> Greg Stein
> Ralph Goers
> Roman Shaposhnik
> Konstantin Boudnik
>
> Non-binding +1s (7):
> Patrick Angeles
> Ashish Paliwal
> Mike Percy
> Luke Han
> Amol Kekre
> Joe Witt
> Sree V
>
> Best,
> Henry
>

Re: [DISCUSS] Metron incubator proposal

[Owen O'Malley]

Thanks for the good suggestion, Bertrand. I've included your paragraph in
the background section of the proposal.

.. Owen

On Tue, Dec 1, 2015 at 8:17 AM, Bertrand Delacretaz <bdelacre...@apache.org>
wrote:

> On Tue, Dec 1, 2015 at 4:53 PM, Owen O'Malley <omal...@apache.org> wrote:
> > ...To avoid forking communities, you need to either have:
> > * the community wants to move to Apache
> > * the community is dead or inactive
> >
> > There is a pretty compelling evidence that OpenSOC is currently
> inactive...
>
> Ok, fair enough.
>
> I suggest that you add a more explicit statement about this to the
> proposal, that can be useful for future similar cases. Maybe something
> like
>
>  "In general Apache accepts only voluntary contributions and avoids
> hostile forks. In this case, given that the community is demonstrably
> dead, it seems fair to fork the existing code at Apache to allow a new
> community to work on it. Once incubation starts, we will send a
> message pointing to the new home to the OpenSOC support group".
>
> Which might overlap with existing text but looks more reusable to me.
>
> -Bertrand
>
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>

Re: [DISCUSS] Metron incubator proposal

[Owen O'Malley]

On Mon, Nov 30, 2015 at 4:06 PM, P. Taylor Goetz <ptgo...@gmail.com> wrote:

> I'm interested as well, particularly given the ties to Storm.
>
> I'd be happy to volunteer as mentor and/or committer if it would be
> welcome. I have some familiarity with both projects (obviously one more so
> than the other ;) ).
>

I had the project vote off-list on adding Larry and Taylor to the project
and the result of both votes was 12 +1's and no -1's. I've added them to
the proposal.

.. Owen


>
> -Taylor
>
> > On Nov 30, 2015, at 1:15 PM, larry mccay <lmc...@apache.org> wrote:
> >
> > This is an interesting proposal that seems would build a community where
> an
> > open one doesn't really exist at the moment.
> > A project like this needs a healthy community to survive and scale with
> the
> > pace of changes in attacks.
> > I for one would be interested in lending a hand as a contributor or
> > committer - if that would be welcomed.
> >
> >
> >> On Mon, Nov 30, 2015 at 11:55 AM, Owen O'Malley <omal...@apache.org>
> wrote:
> >>
> >> Hi all,
> >>
> >> We'd like to start a discussion proposing creating Metron as an
> incubator
> >> podling. The proposal is on the wiki here:
> >> https://wiki.apache.org/incubator/MetronProposal
> >>
> >> I would call your attention to the background section in particular. The
> >> condensed version is that the original code base (OpenSOC) was created
> by a
> >> company (Cisco) that put it on github as ALv2, but then hasn't been
> working
> >> on it. We posted a message
> >> <
> https://groups.google.com/d/msg/opensoc-support/rFlW2uSSvmU/Sw_cO-T2AAAJ>
> >> to the OpenSOC support group a month ago proposing a move to Apache and
> got
> >> a single positive response.
> >>
> >> The text of the proposal is included below for easy quoting during
> >> discussion.
> >>
> >> Thanks,
> >>   Owen
> >>
> >> = Apache Metron Proposal =
> >>
> >> == Abstract ==
> >>
> >> The Metron project is an open source project dedicated to providing an
> >> extensible and scalable advanced security analytics tool. It has strong
> >> foundations in the Apache Hadoop ecosystem.
> >>
> >> == Proposal ==
> >>
> >> Metron integrates a variety of open source big data technologies in
> order
> >> to offer a centralized tool for security monitoring and analysis. Metron
> >> provides capabilities for log aggregation, full packet capture indexing,
> >> storage, advanced behavioral analytics and data enrichment, while
> applying
> >> the most current threat-intelligence information to security telemetry
> >> within a single platform.
> >>
> >> Metron can be divided into 4 areas:
> >>
> >>  1. '''A mechanism to capture, store, and normalize any type of security
> >> telemetry at extremely high rates.''' Because security telemetry is
> >> constantly being generated, it requires a method for ingesting the data
> at
> >> high speeds and pushing it to various processing units for advanced
> >> computation and analytics.
> >>  1. '''Real time processing and application of enrichments''' such as
> >> threat intelligence, geolocation, and DNS information to telemetry being
> >> collected. The immediate application of this information to incoming
> >> telemetry provides the context and situational awareness, as well as the
> >> “who” and “where” information that is critical for investigation.
> >>  1. '''Efficient information storage''' based on how the information
> will
> >> be used:
> >>a. Logs and telemetry are stored such that they can be efficiently
> >> mined and analyzed for concise security visibility
> >>a. The ability to extract and reconstruct full packets helps an
> analyst
> >> answer questions such as who the true attacker was, what data was
> leaked,
> >> and where that data was sent
> >>a. Long-term storage not only increases visibility over time, but
> also
> >> enables advanced analytics such as machine learning techniques to be
> used
> >> to create models on the information. Incoming data can then be scored
> >> against these stored models for advanced anomaly detection.
> >>  1. '''An interface that gives a security investigator a centralized
> view
> >> of data and alerts passed through the system.''' Metron’s interface
> >> presents alert summaries with threat

Re: [DISCUSS] Metron incubator proposal

[Owen O'Malley]

On Tue, Dec 1, 2015 at 2:02 AM, Bertrand Delacretaz <bdelacre...@apache.org>
wrote:

> Hi Owen,
>
> On Mon, Nov 30, 2015 at 5:55 PM, Owen O'Malley <omal...@apache.org> wrote:
> > ...The
> > condensed version is that the original code base (OpenSOC) was created
> by a
> > company (Cisco) that put it on github as ALv2, but then hasn't been
> working
> > on it. We posted a message
> > <
> https://groups.google.com/d/msg/opensoc-support/rFlW2uSSvmU/Sw_cO-T2AAAJ>
> > to the OpenSOC support group a month ago proposing a move to Apache and
> got
> > a single positive response...
>
> The response I see there ("this is great news - looking forward to
> working with you") does not look to me like approval from whoever owns
> that code to move it to the ASF.
>

The point was that the community around OpenSOC is unresponsive. If there
was an active community, it would generate more than a single response
after a month. Cisco threw the code over the wall, but doesn't seem to be
investing in it.


> What's needed IMO is approval from whoever owns the code that's at
> https://github.com/OpenSOC/opensoc . As usual, although the license of
> that code would allow us to fork, we only accept voluntary
> contributions.
>

My point is that Apache's self-imposed rule is not to do hostile forks,
because we want to play nicely with other open source projects. That rule
keeps us out of a lot of trouble and avoids bad feelings. I've heard it
expressed as "Apache doesn't fork communities," which I think ties nicely
with "Community over code."

To avoid forking communities, you need to either have:
* the community wants to move to Apache
* the community is dead or inactive

There is a pretty compelling evidence that OpenSOC is currently inactive.

.. Owen


>
> -Bertrand
>
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>

[DISCUSS] Metron incubator proposal

[Owen O'Malley]

Hi all,

 We'd like to start a discussion proposing creating Metron as an incubator
podling. The proposal is on the wiki here:
https://wiki.apache.org/incubator/MetronProposal

I would call your attention to the background section in particular. The
condensed version is that the original code base (OpenSOC) was created by a
company (Cisco) that put it on github as ALv2, but then hasn't been working
on it. We posted a message

to the OpenSOC support group a month ago proposing a move to Apache and got
a single positive response.

The text of the proposal is included below for easy quoting during
discussion.

Thanks,
   Owen

= Apache Metron Proposal =

== Abstract ==

The Metron project is an open source project dedicated to providing an
extensible and scalable advanced security analytics tool. It has strong
foundations in the Apache Hadoop ecosystem.

== Proposal ==

Metron integrates a variety of open source big data technologies in order
to offer a centralized tool for security monitoring and analysis. Metron
provides capabilities for log aggregation, full packet capture indexing,
storage, advanced behavioral analytics and data enrichment, while applying
the most current threat-intelligence information to security telemetry
within a single platform.

Metron can be divided into 4 areas:

  1. '''A mechanism to capture, store, and normalize any type of security
telemetry at extremely high rates.''' Because security telemetry is
constantly being generated, it requires a method for ingesting the data at
high speeds and pushing it to various processing units for advanced
computation and analytics.
  1. '''Real time processing and application of enrichments''' such as
threat intelligence, geolocation, and DNS information to telemetry being
collected. The immediate application of this information to incoming
telemetry provides the context and situational awareness, as well as the
“who” and “where” information that is critical for investigation.
  1. '''Efficient information storage''' based on how the information will
be used:
a. Logs and telemetry are stored such that they can be efficiently
mined and analyzed for concise security visibility
a. The ability to extract and reconstruct full packets helps an analyst
answer questions such as who the true attacker was, what data was leaked,
and where that data was sent
a. Long-term storage not only increases visibility over time, but also
enables advanced analytics such as machine learning techniques to be used
to create models on the information. Incoming data can then be scored
against these stored models for advanced anomaly detection.
  1. '''An interface that gives a security investigator a centralized view
of data and alerts passed through the system.''' Metron’s interface
presents alert summaries with threat intelligence and enrichment data
specific to that alert on one single page. Furthermore, advanced search
capabilities and full packet extraction tools are presented to the analyst
for investigation without the need to pivot into additional tools.

Big data is a natural fit for powerful security analytics. The Metron
framework integrates a number of elements from the Hadoop ecosystem to
provide a scalable platform for security analytics, incorporating such
functionality as full-packet capture, stream processing, batch processing,
real-time search, and telemetry aggregation. With Metron, our goal is to
tie big data into security analytics and drive towards an extensible
centralized platform to effectively enable rapid detection and rapid
response for advanced security threats.

== Background ==

OpenSOC was developed by Cisco over the last two years and pushed out to
Github (https://github.com/OpenSOC/opensoc) under the ALv2. However, the
development was mostly closed and has largely stopped. As evidence of the
inactivity, users have complained that pull requests are not answered for a
while
https://groups.google.com/d/msg/opensoc-support/R2W-ZFux8Vk/Y-5tL-EmAAAJ.
Finally, no public releases of OpenSOC have been made. From an Apache point
of view, the current community is not viable.

However, some of the developers of the project have left Cisco and have
found interest from several others that would like to work together to form
an active and open community at Apache starting from the current OpenSOC
code base. A message to the current support group proposing moving to
Apache got a single positive response.
https://groups.google.com/d/msg/opensoc-support/rFlW2uSSvmU/09PIsWL4AAAJ

Because Cisco is not currently interested in being involved, the project
expects to change their name. The project would like to use Metron,
although we will perform a podling name search to check for conflicts.
Metron, meaning measure, is half of the greek root for the word
'telemetry.'  Metron is also a DC Comics character who “... wanders in
search of greater knowledge beyond his own”.


== 

Re: [DISCUSS] Kudu incubator proposal

[Owen O'Malley]

On Tue, Nov 24, 2015 at 7:39 PM, Greg Stein  wrote:

> On Mon, Nov 23, 2015 at 12:46 PM, Alex Harui  wrote:
>
> > On 11/23/15, 8:23 AM, "Mattmann, Chris A (3980)"
> >  wrote:
> >
> > >Alex,
> > >
> > >Please re-read my email. As I stated we don’t take code that
> > >authors don’t want us to have. So far, we haven’t heard from any of
> > >the authors on the incoming Kudu project that that’s the case. If
> > >it’s not the case, we go by the license of the project which stipulates
> > >how code can be copied, modified, reused, etc.
> >
> > Yes, but my interpretation of your words is that folks have to opt out,
> >
>
> Correct: opt-out.
>
> Since this code is under ALv2, we can import it to the ASF under that
> license. We have always done stuff like this, including other permissive
> licenses.
>
> But this isn't simply importing a library, this is saying "the ASF is now
> the primary locus of development for >this< code." And that's where people
> can say, "woah. I hate you guys. don't develop my code there", and so we
> nuke it.
>
> SGA/iCLA is to give us rights that we otherwise wouldn't have (ie. the code
> was under a different license).
>

It is worth looking back at the thread on Bloodhound

.

The important thing is that Apache doesn't fork communities. In this case,
the community wants to move to Apache. That is great and should be allowed.
They shouldn't need to get an explicit permission from each contributor
over the years.

.. Owen

Re: [VOTE] Accept Kudu into the Apache Incubator

[Owen O'Malley]

+1 (binding)

On Tue, Nov 24, 2015 at 9:13 PM, Ralph Goers 
wrote:

> -1 (binding)
> I’d like to see the project start with CTR and use RTC only for specific
> cases (like where tests must be modified, over X (1000 lines?) of code
> added, etc.
>
> I must say I do find the part about achieving quality through automation
> attractive, but following that up with requiring RTC leads me to conclude
> that the project doesn’t really believe that to be true.
>
> Ralph
>
> > On Nov 24, 2015, at 12:32 PM, Todd Lipcon  wrote:
> >
> > Hi all,
> >
> > Discussion on the [DISCUSS] thread seems to have wound down, so I'd like
> to
> > call a VOTE on acceptance of Kudu into the ASF Incubator. The proposal is
> > pasted below and also available on the wiki at:
> > https://wiki.apache.org/incubator/KuduProposal
> >
> > The proposal is unchanged since the original version, except for the
> > addition of Carl Steinbach as a Mentor.
> >
> > Please cast your votes:
> >
> > [] +1, accept Kudu into the Incubator
> > [] +/-0, positive/negative non-counted expression of feelings
> > [] -1, do not accept Kudu into the incubator (please state reasoning)
> >
> > Given the US holiday this week, I imagine many folks are traveling or
> > otherwise offline. So, let's run the vote for a full week rather than the
> > traditional 72 hours. Unless the IPMC objects to the extended voting
> > period, the vote will close on Tues, Dec 1st at noon PST.
> >
> > Thanks
> > -Todd
> > -
> >
> > = Kudu Proposal =
> >
> > == Abstract ==
> >
> > Kudu is a distributed columnar storage engine built for the Apache Hadoop
> > ecosystem.
> >
> > == Proposal ==
> >
> > Kudu is an open source storage engine for structured data which supports
> > low-latency random access together with efficient analytical access
> > patterns. Kudu distributes data using horizontal partitioning and
> > replicates each partition using Raft consensus, providing low
> > mean-time-to-recovery and low tail latencies. Kudu is designed within the
> > context of the Apache Hadoop ecosystem and supports many integrations
> with
> > other data analytics projects both inside and outside of the Apache
> > Software Foundation.
> >
> >
> >
> > We propose to incubate Kudu as a project of the Apache Software
> Foundation.
> >
> > == Background ==
> >
> > In recent years, explosive growth in the amount of data being generated
> and
> > captured by enterprises has resulted in the rapid adoption of open source
> > technology which is able to store massive data sets at scale and at low
> > cost. In particular, the Apache Hadoop ecosystem has become a focal point
> > for such “big data” workloads, because many traditional open source
> > database systems have lagged in offering a scalable alternative.
> >
> >
> >
> > Structured storage in the Hadoop ecosystem has typically been achieved in
> > two ways: for static data sets, data is typically stored on Apache HDFS
> > using binary data formats such as Apache Avro or Apache Parquet. However,
> > neither HDFS nor these formats has any provision for updating individual
> > records, or for efficient random access. Mutable data sets are typically
> > stored in semi-structured stores such as Apache HBase or Apache
> Cassandra.
> > These systems allow for low-latency record-level reads and writes, but
> lag
> > far behind the static file formats in terms of sequential read throughput
> > for applications such as SQL-based analytics or machine learning.
> >
> >
> >
> > Kudu is a new storage system designed and implemented from the ground up
> to
> > fill this gap between high-throughput sequential-access storage systems
> > such as HDFS and low-latency random-access systems such as HBase or
> > Cassandra. While these existing systems continue to hold advantages in
> some
> > situations, Kudu offers a “happy medium” alternative that can
> dramatically
> > simplify the architecture of many common workloads. In particular, Kudu
> > offers a simple API for row-level inserts, updates, and deletes, while
> > providing table scans at throughputs similar to Parquet, a commonly-used
> > columnar format for static data.
> >
> >
> >
> > More information on Kudu can be found at the existing open source project
> > website: http://getkudu.io and in particular in the Kudu white-paper
> PDF:
> > http://getkudu.io/kudu.pdf from which the above was excerpted.
> >
> > == Rationale ==
> >
> > As described above, Kudu fills an important gap in the open source
> storage
> > ecosystem. After our initial open source project release in September
> 2015,
> > we have seen a great amount of interest across a diverse set of users and
> > companies. We believe that, as a storage system, it is critical to build
> an
> > equally diverse set of contributors in the development community. Our
> > experiences as committers and PMC members on other Apache projects have
> > taught us the value of diverse communities in ensuring both longevity 

Re: [VOTE] Accept Impala into the Apache Incubator

[Owen O'Malley]

+1 (binding)

On Tue, Nov 24, 2015 at 9:10 PM, Ralph Goers 
wrote:

> -1 (binding)
> I’d like to see the project start with CTR and use RTC only for specific
> cases (like where tests must be modified, over X (1000 lines?) of code
> added, etc.
>
> Ralph
>
>
> > On Nov 24, 2015, at 2:03 PM, Henry Robinson  wrote:
> >
> > Hi -
> >
> > The [DISCUSS] thread has been quiet for a few days, so I think there's
> been
> > sufficient opportunity for discussion around our proposal to bring Impala
> > to the ASF Incubator.
> >
> > I'd like to call a VOTE on that proposal, which is on the wiki at
> > https://wiki.apache.org/incubator/ImpalaProposal, and which I've pasted
> > below.
> >
> > During the discussion period, the proposal has been amended to add Brock
> > Noland as a new mentor, to add one missed committer from the list and to
> > correct some issues with the dependency list.
> >
> > Please cast your votes as follows:
> >
> > [] +1, accept Impala into the Incubator
> > [] +/-0, non-counted vote to express a disposition
> > [] -1, do not accept Impala into the Incubator (please give your
> reason(s))
> >
> > As with the concurrent Kudu vote, I propose leaving the vote open for a
> > full seven days (to close at Tuesday, December 1st at noon PST), due to
> the
> > upcoming US holiday.
> >
> > Thanks,
> > Henry
> >
> > 
> >
> > = Abstract =
> > Impala is a high-performance C++ and Java SQL query engine for data
> stored
> > in Apache Hadoop-based clusters.
> >
> > = Proposal =
> >
> > We propose to contribute the Impala codebase and associated artifacts
> (e.g.
> > documentation, web-site content etc.) to the Apache Software Foundation
> > with the intent of forming a productive, meritocratic and open community
> > around Impala’s continued development, according to the ‘Apache Way’.
> >
> > Cloudera owns several trademarks regarding Impala, and proposes to
> transfer
> > ownership of those trademarks in full to the ASF.
> >
> > = Background =
> > Engineers at Cloudera developed Impala and released it as an
> > Apache-licensed open-source project in Fall 2012. Impala was written as a
> > brand-new, modern C++ SQL engine targeted from the start for data stored
> in
> > Apache Hadoop clusters.
> >
> > Impala’s most important benefit to users is high-performance, making it
> > extremely appropriate for common enterprise analytic and business
> > intelligence workloads. This is achieved by a number of software
> > techniques, including: native support for data stored in HDFS and related
> > filesystems, just-in-time compilation and optimization of individual
> query
> > plans, high-performance C++ codebase and massively-parallel distributed
> > architecture. In benchmarks, Impala is routinely amongst the very highest
> > performing SQL query engines.
> >
> > = Rationale =
> >
> > Despite the exciting innovation in the so-called ‘big-data’ space, SQL
> > remains by far the most common interface for interacting with data in
> both
> > traditional warehouses and modern ‘big-data’ clusters. There is clearly a
> > need, as evidenced by the eager adoption of Impala and other SQL engines
> in
> > enterprise contexts, for a query engine that offers the familiar SQL
> > interface, but that has been specifically designed to operate in massive,
> > distributed clusters rather than in traditional, fixed-hardware,
> > warehouse-specific deployments. Impala is one such query engine.
> >
> > We believe that the ASF is the right venue to foster an open-source
> > community around Impala’s development. We expect that Impala will benefit
> > from more productive collaboration with related Apache projects, and
> under
> > the auspices of the ASF will attract talented contributors who will push
> > Impala’s development forward at pace.
> >
> > We believe that the timing is right for Impala’s development to move
> > wholesale to the ASF: Impala is well-established, has been
> Apache-licensed
> > open-source for more than three years, and the core project is relatively
> > stable. We are excited to see where an ASF-based community can take
> Impala
> > from this strong starting point.
> >
> > = Initial Goals =
> > Our initial goals are as follows:
> >
> > * Establish ASF-compatible engineering practices and workflows
> > * Refactor and publish existing internal build scripts and test
> > infrastructure, in order to make them usable by any community member.
> > * Transfer source code, documentation and associated artifacts to the
> ASF.
> > * Grow the user and developer communities
> >
> > = Current Status =
> >
> > Impala is developed as an Apache-licensed open-source project. The source
> > code is available at http://github.com/cloudera/Impala, and developer
> > documentation is at https://github.com/cloudera/Impala/wiki. The
> majority
> > of commits to the project have come from Cloudera-employed developers,
> but
> > we have accepted some contributions from individuals from 

Resigning as HAWQ mentor

[Owen O'Malley]

All,
   I'd like to resign as a HAWQ mentor. HAWQ will still have 5 mentors, so
it should still be in good shape.

Thanks,
   Owen

Re: [VOTE] Accept Eagle into Apache Incubation

[Owen O'Malley]

+1 (binding)

On Fri, Oct 23, 2015 at 8:42 AM, wp chun  wrote:

> +1
> wp_c...@hotmail.com
> >
> > On 10/23/15, 11:26 PM, "P. Taylor Goetz"  wrote:
> >
> > >+1 (binding)
> > >
> > >-Taylor
> > >
> > >> On Oct 23, 2015, at 10:11 AM, Manoharan, Arun 
> > >>wrote:
> > >>
> > >> Hello Everyone,
> > >>
> > >> Thanks for all the feedback on the Eagle Proposal.
> > >>
> > >> I would like to call for a [VOTE] on Eagle joining the ASF as an
> > >>incubation project.
> > >>
> > >> The vote is open for 72 hours:
> > >>
> > >> [ ] +1 accept Eagle in the Incubator
> > >> [ ] ±0
> > >> [ ] -1 (please give reason)
> > >>
> > >> Eagle is a Monitoring solution for Hadoop to instantly identify access
> > >>to sensitive data, recognize attacks, malicious activities and take
> > >>actions in real time. Eagle supports a wide variety of policies on HDFS
> > >>data and Hive. Eagle also provides machine learning models for
> detecting
> > >>anomalous user behavior in Hadoop.
> > >>
> > >> The proposal is available on the wiki here:
> > >> https://wiki.apache.org/incubator/EagleProposal
> > >>
> > >> The text of the proposal is also available at the end of this email.
> > >>
> > >> Thanks for your time and help.
> > >>
> > >> Thanks,
> > >> Arun
> > >>
> > >> 
> > >>
> > >> Eagle
> > >>
> > >> Abstract
> > >> Eagle is an Open Source Monitoring solution for Hadoop to instantly
> > >>identify access to sensitive data, recognize attacks, malicious
> > >>activities in hadoop and take actions.
> > >>
> > >> Proposal
> > >> Eagle audits access to HDFS files, Hive and HBase tables in real time,
> > >>enforces policies defined on sensitive data access and alerts or blocks
> > >>user¹s access to that sensitive data in real time. Eagle also creates
> > >>user profiles based on the typical access behaviour for HDFS and Hive
> > >>and sends alerts when anomalous behaviour is detected. Eagle can also
> > >>import sensitive data information classified by external classification
> > >>engines to help define its policies.
> > >>
> > >> Overview of Eagle
> > >> Eagle has 3 main parts.
> > >> 1.Data collection and storage - Eagle collects data from various
> hadoop
> > >>logs in real time using Kafka/Yarn API and uses HDFS and HBase for
> > >>storage.
> > >> 2.Data processing and policy engine - Eagle allows users to create
> > >>policies based on various metadata properties on HDFS, Hive and HBase
> > >>data.
> > >> 3.Eagle services - Eagle services include policy manager, query
> service
> > >>and the visualization component. Eagle provides intuitive user
> interface
> > >>to administer Eagle and an alert dashboard to respond to real time
> > >>alerts.
> > >>
> > >> Data Collection and Storage:
> > >> Eagle provides programming API for extending Eagle to integrate any
> > >>data source into Eagle policy evaluation framework. For example, Eagle
> > >>hdfs audit monitoring collects data from Kafka which is populated from
> > >>namenode log4j appender or from logstash agent. Eagle hive monitoring
> > >>collects hive query logs from running job through YARN API, which is
> > >>designed to be scalable and fault-tolerant. Eagle uses HBase as storage
> > >>for storing metadata and metrics data, and also supports relational
> > >>database through configuration change.
> > >>
> > >> Data Processing and Policy Engine:
> > >> Processing Engine: Eagle provides stream processing API which is an
> > >>abstraction of Apache Storm. It can also be extended to other streaming
> > >>engines. This abstraction allows developers to assemble data
> > >>transformation, filtering, external data join etc. without physically
> > >>bound to a specific streaming platform. Eagle streaming API allows
> > >>developers to easily integrate business logic with Eagle policy engine
> > >>and internally Eagle framework compiles business logic execution DAG
> > >>into program primitives of underlying stream infrastructure e.g. Apache
> > >>Storm. For example, Eagle HDFS monitoring transforms audit log from
> > >>Namenode to object and joins sensitivity metadata, security zone
> > >>metadata which are generated from external programs or configured by
> > >>user. Eagle hive monitoring filters running jobs to get hive query
> > >>string and parses query string into object and then joins sensitivity
> > >>metadata.
> > >> Alerting Framework: Eagle Alert Framework includes stream metadata
> API,
> > >>scalable policy engine framework, extensible policy engine framework.
> > >>Stream metadata API allows developers to declare event schema including
> > >>what attributes constitute an event, what is the type for each
> > >>attribute, and how to dynamically resolve attribute value in runtime
> > >>when user configures policy. Scalable policy engine framework allows
> > >>policies to be executed on different physical nodes in parallel. It is
> > >>also used to define your own policy partitioner class. Policy engine
> > >>framework 

Re: [VOTE] Graduate Apache Kylin from the Apache Incubator

[Owen O'Malley]

> > > >>
> > > >>WHEREAS, the Board of Directors deems it to be in the best
> > > >>interests of the Foundation and consistent with the
> > > >>Foundation's purpose to establish a Project Management
> > > >>Committee charged with the creation and maintenance of
> > > >>open-source software, for distribution at no charge to the
> > > >>public, relative to distributed and scalable OLAP engine
> > > >>
> > > >>NOW, THEREFORE, BE IT RESOLVED, that a Project Management
> > > >>    Committee (PMC), to be known as the "Apache Kylin Project",
> > > >>be and hereby is established pursuant to Bylaws of the
> > > >>Foundation; and be it further
> > > >>
> > > >>RESOLVED, that the Apache Kylin Project be and hereby is
> > > >>responsible for the creation and maintenance of open-source
> > > >>software related to distributed and scalable OLAP engine;
> > > >>and be it further
> > > >>
> > > >>RESOLVED, that the office of "Vice President, Kylin" be and
> > > >>hereby is created, the person holding such office to serve at
> > > >>the direction of the Board of Directors as the chair of the
> > > >>Apache Kylin Project, and to have primary responsibility for
> > > >>management of the projects within the scope of responsibility
> > > >>of the Apache Kylin Project; and be it further
> > > >>
> > > >>RESOLVED, that the persons listed immediately below be and
> > > >>hereby are appointed to serve as the initial members of the
> > > >>Apache Kylin Project:
> > > >>
> > > >> * Dayue Gao 
> > > >> * Jason Zhong 
> > > >> * Julian Hyde 
> > > >> * Luke Han 
> > > >> * Henry Saputra 
> > > >> * Hongbin Ma 
> > > >> * Hua Huang 
> > > >> * Owen O'Malley 
> > > >> * P. Taylor Goetz 
> > > >> * Qianhao Zhou 
> > > >> * Shaofeng Shi 
> > > >> * Song Yi 
> > > >> * Ted Dunning 
> > > >> * Xu Jiang 
> > > >> * Yang Li 
> > > >> * Yerui Sun < sunyerui at apache dot org>
> > > >>
> > > >>
> > > >>NOW, THEREFORE, BE IT FURTHER RESOLVED, that Luke Han
> > > >>be appointed to the office of Vice President,Apache Kylin, to
> > > >>serve
> > > >>in accordance with and subject to the direction of the Board
> of
> > > >>Directors and the Bylaws of the Foundation until death,
> > > >>resignation, retirement, removal or disqualification, or
> until
> > > >>a successor is appointed; and be it further
> > > >>
> > > >>RESOLVED, that the initial Apache Kylin Project be and hereby
> > > >>is tasked with the creation of a set of bylaws intended to
> > > >>encourage open development and increased participation in the
> > > >>Kylin Project; and be it further
> > > >>
> > > >>RESOLVED, that the initial Apache Kylin Project be and hereby
> > > >>is tasked with the migration and rationalization of the
> Apache
> > > >>Incubator Kylin podling; and be it further
> > > >>
> > > >>RESOLVED, that all responsibility pertaining to the Apache
> > > >>Incubator Kylin podling encumbered upon the Apache Incubator
> > > >>PMC are hereafter discharged.
> > > >>
> > > >>
> > > >> - - - end - - -
> > > >>
> > > >>
> > > >>
> -
> > > >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> > > >> <javascript:;>
> > > >> For additional commands, e-mail: general-h...@incubator.apache.org
> > > >> <javascript:;>
> > > >>
> > >
> > >
> >
> > --
> > _
> > The information contained in this communication is intended solely for
> the
> > use of the individual or entity to whom it is addressed and others
> > authorized to receive it. It may contain confidential or legally
> privileged
> > information. If you are not the intended recipient you are hereby
> notified
> > that any disclosure, copying, distribution or taking any action in
> reliance
> > on the contents of this information is strictly prohibited and may be
> > unlawful. If you have received this communication in error, please notify
> > us immediately by responding to this email and then delete it from your
> > system. The firm is neither liable for the proper and complete
> transmission
> > of the information contained in this communication nor for any delay in
> its
> > receipt.
> >
>

Re: [DISCUSS] Eagle incubator proposal

[Owen O'Malley]

On Mon, Oct 19, 2015 at 9:00 AM, Ted Dunning <ted.dunn...@gmail.com> wrote:

> I would suggest that Owen O'Malley has not had enough time to be a viable
> mentor recently and should not be on the list of mentors.
>

I have been helping Kylin out and it is graduating, so I'm down to just
Hawq. I'd like to help Eagle out.

.. Owen


> Henry and Julian are good if their schedules permit.  Henry, I know has
> been mentoring a number of projects lately.
>
>
>
> On Mon, Oct 19, 2015 at 8:40 AM, Jean-Baptiste Onofré <j...@nanthrax.net>
> wrote:
>
> > Hi Arun,
> >
> > very interesting proposal. I may see some possible interaction with
> > Falcon. In Falcon, we have HDFS files (and Hive/HBase) monitoring (with a
> > kind of Change Data Capture), etc.
> >
> > So, I see a different perspective in Eagle, but Eagle could also leverage
> > Falcon somehow.
> >
> > Regards
> > JB
> >
> >
> > On 10/19/2015 05:33 PM, Manoharan, Arun wrote:
> >
> >> Hello Everyone,
> >>
> >> My name is Arun Manoharan. Currently a product manager in the Analytics
> >> platform team at eBay Inc.
> >>
> >> I would like to start a discussion on Eagle and its joining the ASF as
> an
> >> incubation project.
> >>
> >> Eagle is a Monitoring solution for Hadoop to instantly identify access
> to
> >> sensitive data, recognize attacks, malicious activities and take
> actions in
> >> real time. Eagle supports a wide variety of policies on HDFS data and
> Hive.
> >> Eagle also provides machine learning models for detecting anomalous user
> >> behavior in Hadoop.
> >>
> >> The proposal is available on the wiki here:
> >> https://wiki.apache.org/incubator/EagleProposal
> >>
> >> The text of the proposal is also available at the end of this email.
> >>
> >> Thanks for your time and help.
> >>
> >> Thanks,
> >> Arun
> >>
> >> 
> >>
> >> Eagle
> >>
> >> Abstract
> >> Eagle is an Open Source Monitoring solution for Hadoop to instantly
> >> identify access to sensitive data, recognize attacks, malicious
> activities
> >> in hadoop and take actions.
> >>
> >> Proposal
> >> Eagle audits access to HDFS files, Hive and HBase tables in real time,
> >> enforces policies defined on sensitive data access and alerts or blocks
> >> user’s access to that sensitive data in real time. Eagle also creates
> user
> >> profiles based on the typical access behaviour for HDFS and Hive and
> sends
> >> alerts when anomalous behaviour is detected. Eagle can also import
> >> sensitive data information classified by external classification
> engines to
> >> help define its policies.
> >>
> >> Overview of Eagle
> >> Eagle has 3 main parts.
> >> 1.Data collection and storage - Eagle collects data from various hadoop
> >> logs in real time using Kafka/Yarn API and uses HDFS and HBase for
> storage.
> >> 2.Data processing and policy engine - Eagle allows users to create
> >> policies based on various metadata properties on HDFS, Hive and HBase
> data.
> >> 3.Eagle services - Eagle services include policy manager, query service
> >> and the visualization component. Eagle provides intuitive user
> interface to
> >> administer Eagle and an alert dashboard to respond to real time alerts.
> >>
> >> Data Collection and Storage:
> >> Eagle provides programming API for extending Eagle to integrate any data
> >> source into Eagle policy evaluation framework. For example, Eagle hdfs
> >> audit monitoring collects data from Kafka which is populated from
> namenode
> >> log4j appender or from logstash agent. Eagle hive monitoring collects
> hive
> >> query logs from running job through YARN API, which is designed to be
> >> scalable and fault-tolerant. Eagle uses HBase as storage for storing
> >> metadata and metrics data, and also supports relational database through
> >> configuration change.
> >>
> >> Data Processing and Policy Engine:
> >> Processing Engine: Eagle provides stream processing API which is an
> >> abstraction of Apache Storm. It can also be extended to other streaming
> >> engines. This abstraction allows developers to assemble data
> >> transformation, filtering, external data join etc. without physically
> bound
> >> to a specific streaming platform. Eagle streaming API allows developers
> to
> >

Re: [VOTE] Graduate Calcite from the Apache Incubator

[Owen O'Malley]

+1 (binding)

On Tue, Sep 15, 2015 at 9:48 AM, Ted Dunning  wrote:

>
> +1 (binding)
>
> Sent from my iPhone
>
> > On Sep 14, 2015, at 22:56, Balaji Ganesan  wrote:
> >
> > +1
> >
> > On Mon, Sep 14, 2015 at 8:47 PM, Chris Nauroth  >
> > wrote:
> >
> >> +1 (binding)
> >>
> >> --Chris Nauroth
> >>
> >>
> >>
> >>
> >>> On 9/14/15, 6:56 PM, "Julian Hyde"  wrote:
> >>>
> >>> This is a vote for Calcite to become a top-level project.
> >>>
> >>> Since joining the Incubator in May, 2014, the Calcite
> >>> community has:
> >>> * Produced eight IPMC-approved releases under two release
> >>> managers;
> >>> * Added five new committers and one new PPMC member;
> >>> * Collaborated successfully with several other Apache
> >>> projects (Drill, Hive, Kylin, Phoenix, Samza);
> >>> * Grown into an active community (typical monthly activity
> >>> is 100 emails, 30 commits and 20 issues fixed);
> >>> * Conducted a successful community vote to graduate with
> >>> 20 +1 votes, of which 2 were from our mentors, 12 were
> >>> from committers, and 6 were from IPMC members.
> >>>
> >>> Further information: the discussion on the dev list [1],
> >>> vote thread [2] and result [3]. Also relevant are the
> >>> incubation status page [4] and a thread on this list
> >>> requesting review of whether Calcite met the criteria to
> >>> graduate [5].
> >>>
> >>> Below is our proposed resolution for the Board.
> >>>
> >>> Please vote:
> >>>
> >>> [ ] +1 Graduate Apache Calcite as a top-level project
> >>> [ ] +0
> >>> [ ] -1 Do not graduate Apache Calcite becauseŠ
> >>>
> >>> Here is my vote:
> >>> +1 (binding)
> >>>
> >>> Voting will last 72 hours, ending at 19:00 Pacific on
> >>> September 17th.
> >>>
> >>> Julian Hyde, on behalf of Calcite PPMC
> >>>
> >>> [1] http://s.apache.org/ZPC
> >>> [2] http://s.apache.org/rvB
> >>> [3] http://s.apache.org/sv
> >>> [4] http://incubator.apache.org/projects/calcite.html
> >>> [5] http://s.apache.org/itP
> >>>
> >>> - - - snip - - -
> >>>
> >>> WHEREAS, the Board of Directors deems it to be in the best
> >>> interests of the Foundation and consistent with the
> >>> Foundation's purpose to establish a Project Management
> >>> Committee charged with the creation and maintenance of
> >>> open-source software, for distribution at no charge to the
> >>> public, related to parsing and planning queries on data in a
> >>> wide variety of formats.
> >>>
> >>> NOW, THEREFORE, BE IT RESOLVED, that a Project Management
> >>> Committee (PMC), to be known as the "Apache Calcite
> >>> Project", be and hereby is established pursuant to Bylaws of
> >>> the Foundation; and be it further
> >>>
> >>> RESOLVED, that the Apache Calcite Project be and hereby is
> >>> responsible for the creation and maintenance of software
> >>> related to parsing and planning queries on data in a wide
> >>> variety of formats; and be it further
> >>>
> >>> RESOLVED, that the office of "Vice President, Apache
> >>> Calcite" be and hereby is created, the person holding such
> >>> office to serve at the direction of the Board of Directors
> >>> as the chair of the Apache Calcite Project, and to have
> >>> primary responsibility for management of the projects within
> >>> the scope of responsibility of the Apache Calcite Project;
> >>> and be it further
> >>>
> >>> RESOLVED, that the persons listed immediately below be and
> >>> hereby are appointed to serve as the initial members of the
> >>> Apache Calcite Project:
> >>>
> >>> * Alan Gates 
> >>> * Aman Sinha 
> >>> * Ashutosh Chauhan 
> >>> * James R. Taylor 
> >>> * Jacques Nadeau 
> >>> * Jesús Camacho Rodríguez 
> >>> * Jinfeng Ni 
> >>> * John Pullokkaran 
> >>> * Julian Hyde 
> >>> * Nick Dimiduk 
> >>> * Steven Noels 
> >>> * Ted Dunning 
> >>> * Vladimir Sitnikov 
> >>>
> >>> NOW, THEREFORE, BE IT FURTHER RESOLVED, that Julian Hyde be
> >>> appointed to the office of Vice President, Apache Calcite,
> >>> to serve in accordance with and subject to the direction of
> >>> the Board of Directors and the Bylaws of the Foundation
> >>> until death, resignation, retirement, removal or
> >>> disqualification, or until a successor is appointed; and be
> >>> it further
> >>>
> >>> RESOLVED, that the Apache Calcite Project be and hereby is
> >>> tasked with the migration and rationalization of the Apache
> >>> Incubator Calcite podling; and be it further
> >>>
> >>> RESOLVED, that all responsibilities pertaining to the Apache
> >>> Incubator Calcite podling encumbered upon the Apache
> >>> Incubator Project are hereafter discharged.
> >>>
> >>> - - - end - - -
> >>>
> >>>
> >>> -
> >>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> >>> For additional commands, e-mail: general-h...@incubator.apache.org
> >>
> >>
> >> -
> >> To unsubscribe, e-mail: 

Re: [VOTE] Release Apache Kylin-0.7.2-incubating

[Owen O'Malley]

* Checked GPG signature and hashes.
* Built and ran unit tests.
* Checked that all java files had Apache headers.
* All of the java files are in org.apache.kylin package. Only exceptions
were in the
  atopcalcite module:
* net.hydromatic.optiq.runtime
* org.eigenbase.sql2rel
  Are those classes coming from a different package? Do you intend to
repackage them?

+1 (binding)


On Sat, Jul 18, 2015 at 12:43 AM, Henry Saputra henry.sapu...@gmail.com
wrote:

 Signature file looks good
 Hashes are checked (NOTE next time probably use sha512)
 NOTICE file looks good
 LICENSE file looks good
 DISCLAIMER exists
 No 3rd party exes.

 +1 (binding)



 On Thu, Jul 16, 2015 at 9:22 PM, Luke Han luke...@apache.org wrote:
  Hi all,
 
  The Apache Kylin community has voted on and approved a proposal to
 release
  Apache Kylin 0.7.2 (incubating), the first release of Apache Kylin.
 
  We already have applied some suggestion
  in last vote attempt, please refer to below vote mail thread for detail.
 
  Proposal:
  http://s.apache.org/kylin-0.7.2-vote_rc2
 
  Vote result:
  http://s.apache.org/kylin-0.7.2-vote_result
 
  5 binding +1 votes
  2 non-binding +1 votes
  No -1 votes
 
 
  The commit to be voted upon:
 
 https://github.com/apache/incubator-kylin/commit/f7a28422645bc209b2c38bb05b24a340444f9066
 
  Its hash is f7a28422645bc209b2c38bb05b24a340444f9066.
 
  The artifacts to be voted on are located here:
 
 https://dist.apache.org/repos/dist/dev/incubator/kylin/apache-kylin-0.7.2-incubating-rc2/
 
  The hashes of the artifacts are as follows:
  src.tar.gz.md5 334c0981d0a258f2d83402bdb75486dd
  src.tar.gz.sha1 c377bcb7be17e0ad5540ce2a3d1366cc0c451eda
 
  A staged Maven repository is available for review at:
  https://repository.apache.org/content/repositories/orgapachekylin-1008/
 
  Release artifacts are signed with the following key:
  https://people.apache.org/keys/committer/lukehan.asc
 
  Pursuant to the Releases section of the Incubation Policy and with
  the endorsement of our mentors we would now like to request
  the permission of the Incubator PMC to publish the release. The vote
  is open for 72 hours, or until the necessary number of votes (3 +1)
  is reached.
 
  [ ] +1 Release this package
  [ ]  0 I don't feel strongly about it, but I'm okay with the release
  [ ] -1 Do not release this package because...
 
  +1 from Luke
 
  Luke Han, on behalf of Apache Kylin PPMC

 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Release Apache Kylin-0.7.2-incubating

[Owen O'Malley]

On Sat, Jul 18, 2015 at 12:43 AM, Henry Saputra henry.sapu...@gmail.com
wrote:

  (NOTE next time probably use sha512)


I'd like to second the request for using sha512 and not including md5.

Re: [VOTE] Accept REEF into the Apache Incubator

[Owen O'Malley]

 2.0)
 
  To the best of our knowledge, the external dependencies of REEF are
  distributed under Apache compatible licenses:
 
  * guava-libraries (Apache 2.0)
  * protobuf (BSD)
  * asm (BSD)
  * netty (Apache 2.0)
  * mockito (MIT)
  * junit (EPL 1.0)
  * slf4j (MIT)
 
 
  # Cryptography
 
  REEF will depend on secure Hadoop, which can optionally use Kerberos.
 
  # Required Resources
 
  ## Mailing Lists
 
   * reef-private for private PMC discussions
   * reef-dev for technical discussions among contributors and
  notification about commits
 
  ## Subversion Directory
 
  The REEF team uses Git for source version control:
  git://git.apache.org/reef
 
  ## Issue Tracking
 
  JIRA REEF (REEF)
 
  ## Other Resources
 
  Jenkins continuous integration testing
 
  # Initial Committers
 
  * Markus Weimer
  * Sergiy Matusevych
  * Julia Wang
  * Shravan M Narayanamurthy
  * Yingda Chen
  * Tony Majestro
  * Beysim Sezgin
  * Boris Shulman
  * Russell Sears
  * Jung Ryong Lee
  * You Sun Jung
  * Dong Joon Hyun
  * Josh Rosen
  * Tyson Condie
  * Brandon Myers
  * Yunseong Lee
  * Taegeon Um
  * Youngseok Yang
  * Brian Cho
  * Byung-Gon Chun
 
  # Affiliations
 
  * Microsoft:
   * Markus Weimer
   * Sergiy Matusevych
   * Julia Wang
   * Shravan M Narayanamurthy
   * Yingda Chen
   * Tony Majestro
   * Beysim Sezgin
   * Boris Shulman
  * Purestorage:
   * Russell Sears
  * SK Telecom:
   * Jung Ryong Lee
   * You Sun Jung
   * Dong Joon Hyun
  * University of California:
   * Josh Rosen (Berkeley)
   * Tyson Condie (LA)
  * University of Washington:
   * Brandon Myers
  * Seoul National University:
   * Yunseong Lee
   * Taegeon Um
   * Youngseok Yang
   * Brian Cho
   * Byung-Gon Chun
 
 
  # Sponsors
 
  ## Champions
  Chris Douglas cdoug...@apache.org
 
  ## Nominated Mentors
  * Chris Mattmann mattm...@apache.org
  * Ross Gardler rgard...@apache.org
  * Owen O'Malley omal...@apache.org
 
  ## Sponsoring Entity
  The Apache Incubator


 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Argus as a new incubator project

[Owen O'Malley]

With 10 binding +1's and 5 non-binding +1's and no -1's, the vote passes.

Thanks everyone,
   Owen


On Wed, Jul 23, 2014 at 9:34 PM, j...@nanthrax.net wrote:

 +1 (binding)

 Regards
 JB


 On 2014-07-23 22:32, Arun Murthy wrote:

 +1 (binding)

 Arun


  On Jul 21, 2014, at 9:03 AM, Owen O'Malley omal...@apache.org wrote:

 Following the discussion earlier, I'm calling a vote to accept Argus as a
 new Incubator project.

 The proposal draft is available at:
 https://wiki.apache.org/incubator/ArgusProposal, and is also included
 below.

 Vote is open for 72h and closes at 24 July 2014 at 10am PST.

 [ ] +1 accept Argus in the Incubator
 [ ] +/-0
 [ ] -1 because...

 I'm +1.

 .. Owen


 --
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or entity
 to
 which it is addressed and may contain information that is confidential,
 privileged and exempt from disclosure under applicable law. If the reader
 of this message is not the intended recipient, you are hereby notified
 that
 any printing, copying, dissemination, distribution, disclosure or
 forwarding of this communication is strictly prohibited. If you have
 received this communication in error, please contact the sender
 immediately
 and delete it from your system. Thank You.

 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org


 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org

[VOTE] Argus as a new incubator project

[Owen O'Malley]

Following the discussion earlier, I'm calling a vote to accept Argus as a
new Incubator project.

 The proposal draft is available at:
https://wiki.apache.org/incubator/ArgusProposal, and is also included
 below.

 Vote is open for 72h and closes at 24 July 2014 at 10am PST.

 [ ] +1 accept Argus in the Incubator
 [ ] +/-0
 [ ] -1 because...

I'm +1.

.. Owen

Re: [PROPOSAL] Apache Argus Proposal

[Owen O'Malley]

On Fri, Jul 18, 2014 at 11:12 AM, Henry Saputra henry.sapu...@gmail.com
wrote:

 when all PPMCs including mentors and committers are from same organizations


Argus *does* have an external mentor. Jakob Homan is working at LinkedIn
and has never worked at Hortonworks.

I've been looking for a second external mentor, but found one yet.

.. Owen

Re: [PROPOSAL] Apache Argus Proposal

[Owen O'Malley]

Sorry that should be but I have not found one yet.


On Fri, Jul 18, 2014 at 12:33 PM, Owen O'Malley omal...@apache.org wrote:

 On Fri, Jul 18, 2014 at 11:12 AM, Henry Saputra henry.sapu...@gmail.com
 wrote:

 when all PPMCs including mentors and committers are from same
 organizations


 Argus *does* have an external mentor. Jakob Homan is working at LinkedIn
 and has never worked at Hortonworks.

 I've been looking for a second external mentor, but found one yet.

 .. Owen

Re: [PROPOSAL] Apache Argus Proposal

[Owen O'Malley]

Thanks, Daniel. I'll add you to the proposal. 

.. Owen

 On Jul 18, 2014, at 12:36, Daniel Gruno humbed...@apache.org wrote:
 
 On 07/18/2014 09:33 PM, Owen O'Malley wrote:
 Sorry that should be but I have not found one yet.
 
 
 On Fri, Jul 18, 2014 at 12:33 PM, Owen O'Malley omal...@apache.org wrote:
 
 On Fri, Jul 18, 2014 at 11:12 AM, Henry Saputra henry.sapu...@gmail.com
 wrote:
 
 when all PPMCs including mentors and committers are from same
 organizations
 
 Argus *does* have an external mentor. Jakob Homan is working at LinkedIn
 and has never worked at Hortonworks.
 
 I've been looking for a second external mentor, but found one yet.
 
 .. Owen
 So, from the peanut gallery; Do you want someone who knows _something_
 about what your project is/does, or do you just need someone who knows
 about the inner workings of the ASF? If the latter, I'd be happy to
 volunteer. I'm already working 12h a day on ASF related matters, so
 what's another hour ;)
 
 With regards,
 Daniel.
 
 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org
 

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [PROPOSAL] Apache Argus Proposal

[Owen O'Malley]

On Thu, Jul 17, 2014 at 2:42 PM, Andrew Purtell apurt...@apache.org wrote:

 Establishing trust and a track record of inclusion and
 openness will be essential for Argus to achieve your objectives, if I
 understand them correctly. I encourage you to address concerns raised on
 this thread about the complexion of the initial PMC and mentorships.


Andrew, I agree that security project even more than typical open source
projects, trust in the development community is critical. We will do our
best to earn the trust and support of the community.

Obviously, the proposal is very up-front about the lack of diversity in the
committer list. It however, reflects the reality of the current project. We
are very interested in inclusively enlarging the development community for
the project after the initial code drop to Apache gets us off the ground.

.. Owen

Re: [PROPOSAL] Apache Argus Proposal

[Owen O'Malley]

On Wed, Jul 16, 2014 at 11:29 AM, Henry Saputra henry.sapu...@gmail.com
wrote:

 HI,

 Just curious if you could give reference to Apache encourages
 disjoint teams to form independent projects, even when those projects
 overlap in scope statement ?


Henry,
It generally comes from the Community over Code, which is a phrase
discussing how the focus is on how the people working on a project work
together and not the code. The assumption is teams with healthy communities
will build good code.

In terms of documentation, it like most of Apache is learned by doing. In
Nick Burch's talk The Apache Way at ApacheCon this year (
http://events.linuxfoundation.org/sites/events/files/slides/TheApacheWay14.pdf)
there was a slide about this:

Don't pick winners, pick runners
• Board doesn't say “We want X”
• Developers say “X is cool”
• We enable developers to do cool stuff
• Apache developers are at the forefront of innovation
• Not interested in a single runner
• We want relay teams
• Community is critical to the Apache Way
• Apache is about supporting communities

.. Owen


 - Henry

 On Mon, Jul 14, 2014 at 7:16 PM, Selvamohan Neethiraj
 sneethi...@hortonworks.com wrote:
  Apache Argus Proposal (http://wiki.apache.org/incubator/ArgusProposal)
 
  == Abstract ==
 
  Argus is a framework to enable, monitor and manage comprehensive data
 security across the Hadoop platform.
 
  The name “Argus” is derived from Argus Panoptes, a 100-eyed giant in
 Greek mythology, endowed with a role to keep “an eye” open and be an
 effective watchman at all times.
 
  == Background ==
 
  The vision with Argus is to provide comprehensive security across the
 Apache Hadoop ecosystem. With the advent of  Apache YARN, the Hadoop
 platform can now support a true data lake architecture. Enterprises can
 potentially run multiple workloads, in a multi tenant environment. Data
 security within Hadoop needs to evolve to support multiple use cases for
 data access, while also providing a framework for central administration of
 security policies and monitoring of user access.
 
  XA Secure, a Hadoop security focused startup, developed the initial
 technology behind Argus. XA Secure was acquired by Hortonworks, which now
 is contributing the technology to the open source community to extend and
 innovate.
 
  == Rationale ==
 
  Many of the projects in the Hadoop ecosystem have their own
 authentication, authorization, and auditing components. There are no
 central administration and auditing capabilities. We are looking to address
 these enterprises security needs of central administration and
 comprehensive security through the Argus project.
  Our initial focus would be around authorization and auditing, the longer
 term vision would be to tie all aspects around data security within the
 Hadoop platform.
 
  == Proposal Details ==
 
  The vision of Argus is to enable comprehensive data security across the
 Hadoop platform. The goal is provide a single user interface or API to
 manage security policies, monitor user access and policy changes history.
 The framework would work with individual components in enforcing these
 policies and in capturing relevant audit information.
  Initial Goals
  1.  Donate the Argus source code and documentation to the
 Apache Software Foundation
  2.  Setup and standardize the open governance of the Argus
 project
  3.  Build a user and developer community
  4.  Deeper Integration with Hadoop Platform
  a.  Enable integration with Apache Storm, Apache
 Knox and Apache Falcon for authorization and auditing
  5.  Configurable centralized storage of audit data into HDFS
  6.  Enable framework to be run in both Linux and Windows
 environments
  7.  Rationalize install procedure, making it easier for
 enterprises to deploy
 
  == Longer Term Goals ==
 
  In longer term, Argus should provide a comprehensive security framework
 for Hadoop platform components, covering the following
  1.  Centralized security administration to manage all
 security related tasks in a central UI
  2.  Fine grained authorization to do a specific action
 and/or operation with Hadoop component/tool and managed through a central
 administration tool
  a.  Standardize authorization method across all
 Hadoop components
  b.  Enhanced support for different authorization
 methods - Role based access control, attribute based access control etc
  c.  Enable tag based global policies
  3.  Centralize auditing of user access and administrative
 actions (security related) within all the components of Hadoop
 
  == Current Status ==
 
  Argus’ technology is currently being used by enterprises and is under
 active development.
 
  The key components of Argus are:
  •   Enterprise Security Administration Portal
  ◦   A 

Re: [PROPOSAL] Apache Argus Proposal

[Owen O'Malley]

On Wed, Jul 16, 2014 at 1:20 AM, Chris Douglas cdoug...@apache.org wrote:

 The committer and PPMC list should be honest. If there is no
 diversity, that fact must be visible for it to be tracked.


a big +1

Chris, I know you are pretty overloaded at the moment with podlings, but
would
you be willing to be a mentor for Argus?

.. Owen



 Initial committers are a dodge that demonstrates nothing about the
 project's willingness to not only accept, but attract new ideas. The
 viability of an OSS project is not inaccurately *defined* as its
 ability to compete for that attention. Demanding pro forma adherence
 to heuristics is a harmful lesson to podlings, unless the goal is to
 teach them to evade accountability. -C

  On Tue, Jul 15, 2014 at 02:38PM, Henry Saputra wrote:
  Hi Andrew, thanks for chiming in =)
 
  I like your and Owen's opinion about initial members of an incubator
 project.
  And also agree it is responsibility of the polling to achieve better
  diversity during its time under Apache incubator.
 
  For this particular proposal, however, the mentors mostly coming from
  same organization.
  Hopefully with this proposal announcement, the project could solicit
  mentors from different organizations to help add check and balance.
 
  - Henry
 
  On Tue, Jul 15, 2014 at 2:22 PM, Andrew Purtell apurt...@apache.org
 wrote:
   I had started typing up a response to Henry's mail but will discard
 the
   beginning of it to say I agree with Owen. A new project coming into
 the
   incubator quite naturally could have the initial set of committers
 entirely
   from one organization. An organization donating an existing code
 base, for
   example.
  
   However, there has been recent discussion elsewhere that the Incubator
   should more closely consider if an incubating project has succeeded
 to grow
   a community beyond the initially limited group before declaring a
 project
   ready for graduation. (Refer to the discussion on the graduation of
 Apache
   Tez.) This position seems reasonable, and should naturally apply
 here. If a
   project exits graduation with the same lack of PMC/committer
 diversity with
   which it entered, this is in effect Apache-washing, in my opinion. A
   stacked PMC is no more open a community then one controlled by a BDFL
 and
   hosted on GitHub.
  
  
  
   On Tue, Jul 15, 2014 at 2:14 PM, Owen O'Malley omal...@apache.org
 wrote:
  
   On Tue, Jul 15, 2014 at 1:59 PM, Henry Saputra 
 henry.sapu...@gmail.com
   wrote:
  
Maybe we should start asking incubator project to try to build some
kind of momentum or community before going to ASF incubator.
   
  
   Apache incubator is a great place for new projects to grow their
 community
   and diversity. Once a project is established it is much harder to
 change
   the infrastructure and there projects started on github are more
 likely to
   stay on github. I think there is significant value in these projects
 being
   done in the Apache Way.
  
  
   
All but one PPMC members for this proposal would be from
 Hortonworks.
   
  
   Would you like to volunteer to help mentor?
  
  
Just from high level it seems like it has similar goal as Apache
 Knox
[1], what is the differences between the 2?
   
  
   I think Knox and Argus complement each other. Knox provides perimeter
   security via proxy/gateway services, while Argus is providing fine
 grain
   integrated authorization and auditing.
  
   .. Owen
  
  
  
  
   --
   Best regards,
  
  - Andy
  
   Problems worthy of attack prove their worth by hitting back. - Piet
 Hein
   (via Tom White)
 
  -
  To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
  For additional commands, e-mail: general-h...@incubator.apache.org
 
 
  -
  To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
  For additional commands, e-mail: general-h...@incubator.apache.org
 

 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org

Re: [PROPOSAL] Apache Argus Proposal

[Owen O'Malley]

On Wed, Jul 16, 2014 at 2:14 PM, Alejandro Abdelnur t...@cloudera.com
wrote:

 Have you considered reaching out to the Sentry community to see if they
 would be interested in bringing Argus existing code base into Sentry?


We did consider it, but decided that the challenge of integrating the two
distinct
code bases moving in different directions would likely create an umbrella
project.
Such umbrella projects are strongly discouraged by Apache and are
usually broken up into separate projects.

 .. Owen

Re: [PROPOSAL] Apache Argus Proposal

[Owen O'Malley]

On Tue, Jul 15, 2014 at 1:59 PM, Henry Saputra henry.sapu...@gmail.com
wrote:

 Maybe we should start asking incubator project to try to build some
 kind of momentum or community before going to ASF incubator.


Apache incubator is a great place for new projects to grow their community
and diversity. Once a project is established it is much harder to change
the infrastructure and there projects started on github are more likely to
stay on github. I think there is significant value in these projects being
done in the Apache Way.



 All but one PPMC members for this proposal would be from Hortonworks.


Would you like to volunteer to help mentor?


 Just from high level it seems like it has similar goal as Apache Knox
 [1], what is the differences between the 2?


I think Knox and Argus complement each other. Knox provides perimeter
security via proxy/gateway services, while Argus is providing fine grain
integrated authorization and auditing.

.. Owen

Re: [PROPOSAL] Apache Argus Proposal

[Owen O'Malley]

On Tue, Jul 15, 2014 at 2:47 PM, Sean Owen sro...@apache.org wrote:

The proposal just says it overlaps, and that it might be good have a
 different group of people reimplement some of the same things. Is
 there a real gap that people want to build to plug here with overlap
 at the edges, or is the scope of the pre-existing code driving
 overlap? (I honestly do not know, asking.)


There are a fair number of Apache projects that overlap each other's scope:

   - Thrift and Avro
   - Parquet, Hive's ORC, and Avro's Trevni
   - Accumulo, HBase, and Cassandra

This is accepted because Apache isn't trying to dictate a universal
platform. It is about groups of developers learning from each other about
how to grow their respective open source communities.

That said, Sentry and Argus have different approaches, focus, and
technologies. Obviously, open source projects are largely about scratching
an unanswered itch. If Sentry met everyone's needs, no one would waste
their time reimplementing it. You only work on something if you think you
can do it better.

.. Owen

Re: [VOTE] Accept Stratosphere into the incubator

[Owen O'Malley]

 such
  functionality is not yet implemented.
 
  Stratosphere is also related with several projects undergoing incubation
  in the Apache Incubation project, such as Tez, Drill, and Spark
  (graduated). While all these projects target sufficiently different
 spaces
  and have different architectures, it would be interesting to explore code
  reuse possibilities. For example, we are currently basing our design for
  compiling SQL to Stratosphere on the Optiq library, also used by Apache
  Drill.
 
  === An Excessive Fascination with the Apache Brand ===
  We believe that the Apache brand will help us attract contributors to
  Stratosphere, by giving us a well-defined, transparent development
 process
  under a known brand. At the same time, Stratosphere already has a healthy
  community and current funding guarantees the further codebase development
  and growth of the project for the next 3-5 years. The reason for this
  proposal is not to gain publicity, but to further strengthen the
 longevity
  of the project as explained in the Rationale section.
 
  == Documentation ==
   * [[https://stratosphere.eu|Project website]]
   * [[http://stratosphere.eu/docs/0.4/|Documentation]]
   * [[https://github.com/stratosphere/stratosphere|Codebase]]
   * [[
 https://groups.google.com/forum/#!forum/stratosphere-dev|Mailinglist]]
 
  == Initial Source ==
  Stratosphere is hosted on [[
  https://github.com/stratosphere/stratosphere|GitHub]] . This is the
  codebase that we will migrate to the Apache Foundation. The code was
  previously hosted on a TU Berlin's own git infrastructure. It has always
  been Apache 2.0 licensed.
 
  === Source and Intellectual Property Submission Plan ===
  All initial and past committers will sign a CLA with the ASF while the
  incubator proposal for Stratosphere is being discussed. All organizations
  that have employed Stratosphere contributors in the past will sign a SGA.
  Current contributors will sign a CCLA. All major contributors are still
  active in the project.
 
  === External Dependencies ===
  All critical dependencies are, to the extend of our knowledge, from other
  Apache projects. These include Apache Hadoop (for YARN and HDFS) and some
  libraries (log4j, commons codec, junit and more). Our web frontend uses
  some MIT-licensed JavaScript libraries.
 
  == Required Resources ==
  === Mailing list ===
  We will migrate our mailing lists to the following:
 
   * us...@stratosphere.incubator.apache.org
   * d...@stratosphere.incubator.apache.org
   * priv...@stratosphere.incubator.apache.org
   * comm...@stratosphere.incubator.apache.org
 
  === Source control ===
  We would like to use Git for source control and enable GitHib mirroring
  functionality, where code reviews on GitHub are automatically forwarded
 to
  the developer mailing list. (See also:
 
 https://blogs.apache.org/infra/entry/improved_integration_between_apache_and
  )
 
  === Issue tracking ===
  We are currently using GitHub for issue tracking. We request an
  Apache-hosted JIRA, and we will import existing issues there.
 
  == Initial committers ==
   * Stephan Ewen - stephan.e...@tu-berlin.de
   * Fabian Hueske - fabian.hue...@tu-berlin.de
   * Daniel Warneke - warn...@posteo.de
   * Robert Metzger - metrob...@gmail.com
   * Ufuk Celebi - u.cel...@fu-berlin.de
   * Aljoscha Krettek - aljoscha.kret...@gmail.com
   * Kostas Tzoumas - kostas.tzou...@tu-berlin.de
   * Sebastian Schelter  - s...@apache.org
 
  === Affiliations ===
   * Stephan Ewen (TU Berlin)
   * Fabian Hueske (TU Berlin)
   * Daniel Warneke (Amadeus IT Group)
   * Robert Metzger (TU Berlin)
   * Ufuk Celebi (FU Berlin)
   * Aljoscha Krettek (TU Berlin)
   * Kostas Tzoumas (TU Berlin)
   * Sebastian Schelter (TU Berlin)
 
  == Sponsors ==
  === Champion ===
  Alan Gates ( ga...@apache.org )
 
  === Nominated Mentors ===
   * Sean Owen ( sro...@apache.org ) (Note: Sean is an Apache member but
  not currently on the IPC, he will need to request IPMC membership)
   * Ted Dunning ( tdunn...@apache.org )
   * Owen O'Malley ( omal...@apache.org )
   * Henry Saputra ( hsapu...@apache.org )
   * Ashutosh Chauhan (hashut...@apache.org)
 
  === Sponsoring Entity ===
  The Apache Incubator
 
 
  --
  CONFIDENTIALITY NOTICE
  NOTICE: This message is intended for the use of the individual or entity
 to
  which it is addressed and may contain information that is confidential,
  privileged and exempt from disclosure under applicable law. If the reader
  of this message is not the intended recipient, you are hereby notified
 that
  any printing, copying, dissemination, distribution, disclosure or
  forwarding of this communication is strictly prohibited. If you have
  received this communication in error, please contact the sender
 immediately
  and delete it from your system. Thank You.
 
  -
  To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
  For additional commands, e-mail

Re: [VOTE] Release Apache Knox-0.3.1-incubating RC3

[Owen O'Malley]

+1 on the RC. I checked the signatures, looked for license headers, built,
and ran unit tests.

For the next release, please add a license header
to 
gateway-shell/src/main/java/org/apache/hadoop/gateway/shell/BasicResponse.java.


On Wed, Nov 27, 2013 at 4:26 PM, Devaraj Das d...@hortonworks.com wrote:

 +1 on the RC. Downloaded, built and ran unit tests.


 On Mon, Nov 25, 2013 at 8:00 PM, larry mccay lmc...@apache.org wrote:

  Hello All -
 
  Just to follow up on this thread, we will have these NOTICES file issues
  corrected in the next release and assume that this VOTE will continue in
  the absence of a -1 vote.
 
  Thank you again, for the insight here.
 
  --larry
 
 
  On Sat, Nov 23, 2013 at 10:40 AM, larry mccay larry.mc...@gmail.com
  wrote:
 
   Hi Sebb -
  
   Thank you for your review and insight!
  
   We will make these changes immediately and insure that they are correct
   going forward.
  
   Do these issues require a new release candidate - or would that only be
   indicated by a -1?
  
   thanks,
  
   --larry
  
  
   On Sat, Nov 23, 2013 at 10:23 AM, sebb seb...@gmail.com wrote:
  
   On 20 November 2013 16:09, larry mccay larry.mc...@gmail.com wrote:
Hello All,
   
This is a call for a vote on Apache Knox Gateway 0.3.1 incubating.
   
A vote was held on developer mailing list and it passed with 3 +1's,
   and 0
-1's or +0's and now
requires a vote on general@incubator.apache.org.
   
The [VOTE] thread can be found at:
   
  
 
 http://mail-archives.apache.org/mod_mbox/incubator-knox-dev/201311.mbox/%3CCACRbFyjgLrCSahhtWWHK-%3DaeQFM4Oegbe3fQjs-RV2-TAnhdxA%40mail.gmail.com%3E
   
The release candidate is a zip archive of the sources in:
https://git-wip-us.apache.org/repos/asf/incubator-knox.git
Branch v0.3.1 (git checkout -b v0.3.1)
   
Tag:
   
  
 
 https://git-wip-us.apache.org/repos/asf?p=incubator-knox.git;a=tag;h=5a907022dbc2b0a8534de47fe7b8c871c4f075f9
  
   The NOTICE file is wrong; the first 6 lines should be removed (i.e.
   lines with == and following blank line)
  
   Also, the text should be:
  
   This product includes software developed at
   not
   This product includes software developed by
  
Source archive zip file and signature are available from:
   
  
 
 https://dist.apache.org/repos/dist/dev/incubator/knox/knox-incubating-0.3.1/knox-incubating-0.3.1-src.zip
   
  
 
 https://dist.apache.org/repos/dist/dev/incubator/knox/knox-incubating-0.3.1/knox-incubating-0.3.1-src.zip.asc
   
Checksums of the source archive:
  SHA1:   04bb11360f57c0431c30cfb181e3199868fe6053
   
The KEYS file can be found at:
   
   
  
 
 https://dist.apache.org/repos/dist/dev/incubator/knox/knox-incubating-0.3.1/KEYS
  
   The KEYS file should really be at the top level of the release area,
  i.e.
  
   https://dist.apache.org/repos/dist/release/incubator/knox/KEYS
  
The release changes file can be found at:
   
   
  
 
 https://dist.apache.org/repos/dist/dev/incubator/knox/knox-incubating-0.3.1/CHANGES
   
The release has been signed with key (587C089B):
   
  
 http://pgp.mit.edu:11371/pks/lookup?op=vindexsearch=0x82F9C371587C089B
   
Vote will be open for 72 hours.
   
thanks,
   
--larry
   
Larry McCay
  
   -
   To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
   For additional commands, e-mail: general-h...@incubator.apache.org
  
  
  
 

 --
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or entity to
 which it is addressed and may contain information that is confidential,
 privileged and exempt from disclosure under applicable law. If the reader
 of this message is not the intended recipient, you are hereby notified that
 any printing, copying, dissemination, distribution, disclosure or
 forwarding of this communication is strictly prohibited. If you have
 received this communication in error, please contact the sender immediately
 and delete it from your system. Thank You.

Re: [VOTE] Release Apache Knox-0.3.1-incubating RC3

[Owen O'Malley]

On further inspection, I see the header is there, but not at the top of the
file. Please move it in a future release.

-- Owen


On Mon, Dec 2, 2013 at 9:56 AM, Owen O'Malley omal...@apache.org wrote:

 +1 on the RC. I checked the signatures, looked for license headers, built,
 and ran unit tests.

 For the next release, please add a license header
 to 
 gateway-shell/src/main/java/org/apache/hadoop/gateway/shell/BasicResponse.java.


 On Wed, Nov 27, 2013 at 4:26 PM, Devaraj Das d...@hortonworks.com wrote:

 +1 on the RC. Downloaded, built and ran unit tests.


 On Mon, Nov 25, 2013 at 8:00 PM, larry mccay lmc...@apache.org wrote:

  Hello All -
 
  Just to follow up on this thread, we will have these NOTICES file issues
  corrected in the next release and assume that this VOTE will continue in
  the absence of a -1 vote.
 
  Thank you again, for the insight here.
 
  --larry
 
 
  On Sat, Nov 23, 2013 at 10:40 AM, larry mccay larry.mc...@gmail.com
  wrote:
 
   Hi Sebb -
  
   Thank you for your review and insight!
  
   We will make these changes immediately and insure that they are
 correct
   going forward.
  
   Do these issues require a new release candidate - or would that only
 be
   indicated by a -1?
  
   thanks,
  
   --larry
  
  
   On Sat, Nov 23, 2013 at 10:23 AM, sebb seb...@gmail.com wrote:
  
   On 20 November 2013 16:09, larry mccay larry.mc...@gmail.com
 wrote:
Hello All,
   
This is a call for a vote on Apache Knox Gateway 0.3.1 incubating.
   
A vote was held on developer mailing list and it passed with 3
 +1's,
   and 0
-1's or +0's and now
requires a vote on general@incubator.apache.org.
   
The [VOTE] thread can be found at:
   
  
 
 http://mail-archives.apache.org/mod_mbox/incubator-knox-dev/201311.mbox/%3CCACRbFyjgLrCSahhtWWHK-%3DaeQFM4Oegbe3fQjs-RV2-TAnhdxA%40mail.gmail.com%3E
   
The release candidate is a zip archive of the sources in:
https://git-wip-us.apache.org/repos/asf/incubator-knox.git
Branch v0.3.1 (git checkout -b v0.3.1)
   
Tag:
   
  
 
 https://git-wip-us.apache.org/repos/asf?p=incubator-knox.git;a=tag;h=5a907022dbc2b0a8534de47fe7b8c871c4f075f9
  
   The NOTICE file is wrong; the first 6 lines should be removed (i.e.
   lines with == and following blank line)
  
   Also, the text should be:
  
   This product includes software developed at
   not
   This product includes software developed by
  
Source archive zip file and signature are available from:
   
  
 
 https://dist.apache.org/repos/dist/dev/incubator/knox/knox-incubating-0.3.1/knox-incubating-0.3.1-src.zip
   
  
 
 https://dist.apache.org/repos/dist/dev/incubator/knox/knox-incubating-0.3.1/knox-incubating-0.3.1-src.zip.asc
   
Checksums of the source archive:
  SHA1:   04bb11360f57c0431c30cfb181e3199868fe6053
   
The KEYS file can be found at:
   
   
  
 
 https://dist.apache.org/repos/dist/dev/incubator/knox/knox-incubating-0.3.1/KEYS
  
   The KEYS file should really be at the top level of the release area,
  i.e.
  
   https://dist.apache.org/repos/dist/release/incubator/knox/KEYS
  
The release changes file can be found at:
   
   
  
 
 https://dist.apache.org/repos/dist/dev/incubator/knox/knox-incubating-0.3.1/CHANGES
   
The release has been signed with key (587C089B):
   
  
 http://pgp.mit.edu:11371/pks/lookup?op=vindexsearch=0x82F9C371587C089B
   
Vote will be open for 72 hours.
   
thanks,
   
--larry
   
Larry McCay
  
   -
   To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
   For additional commands, e-mail: general-h...@incubator.apache.org
  
  
  
 

 --
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or entity
 to
 which it is addressed and may contain information that is confidential,
 privileged and exempt from disclosure under applicable law. If the reader
 of this message is not the intended recipient, you are hereby notified
 that
 any printing, copying, dissemination, distribution, disclosure or
 forwarding of this communication is strictly prohibited. If you have
 received this communication in error, please contact the sender
 immediately
 and delete it from your system. Thank You.

Re: [VOTE] Graduate Ambari from Apache Incubator

[Owen O'Malley]

+1 for graduation (binding)


On Sun, Nov 17, 2013 at 9:52 AM, Arun C. Murthy a...@hortonworks.com wrote:

 
  [  ] +1 Graduate Ambari from Incubator

 +1 for graduation (binding)

 thanks,
 Arun


  On Nov 15, 2013, at 9:12 AM, Yusaku Sako yus...@hortonworks.com wrote:
 
  This is a call for vote to graduate Ambari from Apache Incubator.
 
  The Apache Ambari project has been incubating since August 2011.
  We have made significant progress with the project during the two years
  of Incubation, adding 27 committers for a total of 36 committers [1], and
  producing 8 releases following ASF policies and guidelines.
  The Apache Ambari community has voted to graduate Ambari as a TLP [2].
  The community vote results can be found at [3].
  The discussion thread for the board resolution can be found at [4].
 
  Please cast your votes:
 
  [  ] +1 Graduate Ambari from Incubator
  [  ] +0 Indifferent to graduation status of Ambari
  [  ] -1 Reject graduation of Ambari from Incubator
 
  This vote will remain open for at least 72 hours from now.
  Please find the proposed board resolution below.
 
  [1] http://people.apache.org/committers-by-project.html#ambari
  [2] http://markmail.org/thread/cp5ajf2uvfl3oj66
  [3] http://markmail.org/message/l6zss4rgfcs3gvpm
  [4] http://markmail.org/thread/36dplr3pwzmazwmg
 
  Regards,
  Yusaku Sako
 
  ###
 
  X. Establish the Apache Ambari Project
 
   WHEREAS, the Board of Directors deems it to be in the best
   interests of the Foundation and consistent with the
   Foundation's purpose to establish a Project Management
   Committee charged with the creation and maintenance of
   open-source software, for distribution at no charge to
   the public, related to Hadoop cluster management.
 
   NOW, THEREFORE, BE IT RESOLVED, that a Project Management
   Committee (PMC), to be known as the Apache Ambari Project,
   be and hereby is established pursuant to Bylaws of the
   Foundation; and be it further
 
   RESOLVED, that the Apache Ambari Project be and hereby is
   responsible for the creation and maintenance of software
   related to Hadoop cluster management;
   and be it further
 
   RESOLVED, that the office of Vice President, Apache Ambari be
   and hereby is created, the person holding such office to
   serve at the direction of the Board of Directors as the chair
   of the Apache Ambari Project, and to have primary responsibility
   for management of the projects within the scope of
   responsibility of the Apache Ambari Project; and be it further
 
   RESOLVED, that the persons listed immediately below be and
   hereby are appointed to serve as the initial members of the
   Apache Ambari Project:
 
 * Babiichuk Andriy (ababiichuk)
 * Arun Murthy (acmurthy)
 * Aleksandr Kovalenko (akovalenko)
 * Antonenko Aleksandr Igorevich (alexantonenko)
 * Andrii Tkach (atkach)
 * Bernd Fondermann (berndf)
 * Billie Rinaldi (billie)
 * Christopher Douglas (cdouglas)
 * Chad Roberts (croberts)
 * Devaraj Das (ddas)
 * Dmitry Lysnichenko (dmitriusan)
 * Dmytro Sen (dsen)
 * Eric Yang (eyang)
 * Hitesh Shah (hitesh)
 * Jagane Sundar (jagane)
 * Jaimin Jetly (jaimin)
 * Jitendra Pandey (jitendra)
 * John Speidel (jspeidel)
 * Kan Zhang (kzhang)
 * Mahadev Konar (mahadev)
 * Papirkovskyy Myroslav (mpapirkovskyy)
 * Nate Cole (ncole)
 * Oleksandr Diachenko (odiachenko)
 * Owen O’Malley (omalley)
 * Oleg Nechiporenko (onechiporenko)
 * Ramya Sunil (ramya)
 * Varun Kapoor (reznor)
 * Sumit Mohanty (smohanty)
 * Srimanth Gunturi (srimanth)
 * Siddharth Wagle (swagle)
 * Thomas Beerbower (tbeerbower)
 * Suhas (vgogate)
 * Vikram Dixit K (vikram)
 * Vinod Kumar Vavilapalli (vinodkv)
 * Xi Wang (xiwang)
 * Yusaku Sako (yusaku)
 
 
   NOW, THEREFORE, BE IT FURTHER RESOLVED, that Yusaku Sako
   be appointed to the office of Vice President, Apache Ambari, to
   serve in accordance with and subject to the direction of the
   Board of Directors and the Bylaws of the Foundation until
   death, resignation, retirement, removal or disqualification,
   or until a successor is appointed; and be it further
 
   RESOLVED, that the initial Apache Ambari PMC be and hereby is
   tasked with the creation of a set of bylaws intended to
   encourage open development and increased participation in the
   Apache Ambari Project; and be it further
 
   RESOLVED, that the Apache Ambari Project be and hereby
   is tasked with the migration and rationalization of the Apache
   Incubator Ambari podling; and be it further
 
   RESOLVED, that all responsibilities pertaining to the Apache
  

Re: [VOTE] Accept Falcon into the Apache Incubator (was originally named Ivory)

[Owen O'Malley]

+1 (binding)


On Mon, Mar 25, 2013 at 8:32 AM, Leif Hedstrom zw...@apache.org wrote:

 On Mar 20, 2013, at 9:54 PM, Srikanth Sundarrajan wrote:

 [ ]  +1 Accept Apache Falcon into the Incubator
 [ ]  +0 Don't care.
 [ ]  -1 Don't accept Apache Falcon into the Incubator because...



 +1. Binding.

 -- Leif



 --**--**-
 To unsubscribe, e-mail: 
 general-unsubscribe@incubator.**apache.orggeneral-unsubscr...@incubator.apache.org
 For additional commands, e-mail: 
 general-help@incubator.apache.**orggeneral-h...@incubator.apache.org

Re: [VOTE] Apache Ambari (incubating) 1.2.1 Release Candidate RC0.

[Owen O'Malley]

On Mon, Mar 11, 2013 at 1:49 PM, Roman Shaposhnik r...@apache.org wrote:

  At least
 in Hadoop ecosystem I know of no other project that provides a combined
 bundle for the checksums.


Have you looked at the Hadoop releases?

http://www.us.apache.org/dist/hadoop/common/hadoop-1.1.2/

-- Owen

Re: [VOTE] Apache Ambari (incubating) 1.2.1 Release Candidate RC0.

[Owen O'Malley]

On Sun, Mar 10, 2013 at 8:13 PM, sebb seb...@gmail.com wrote:

The ambari-1.2.1-incubating.tar.gz.mds file is wrong; there should be
 separate files for each hash type.
 Normally only MD5 and SHA are provided.


In what way are they wrong? It is a perfectly well defined output format
that is the preferred one for Ambari. Depending on MD5 or even SHA1 for
tamper detection is in fact wrong.

-- Owen

Re: [VOTE] Accept Tajo into the Apache Incubator

[Owen O'Malley]

With 11 binding +1's and 5 non-binding +1's the vote passes. Jakob, can you
start the process for setting up the project?

Thanks, all.

-- Owen


On Mon, Mar 4, 2013 at 4:35 PM, Alex Karasulu akaras...@apache.org wrote:

 On Sat, Mar 2, 2013 at 3:48 AM, Alex Karasulu akaras...@apache.org
 wrote:

  +1 (binding)
 
 
 Just as an FYI, I'm also a mentor of this project.

 --
 Best Regards,
 -- Alex

Re: [VOTE] Accept Tajo into the Apache Incubator

[Owen O'Malley]

.
 
 
  = Required Resources =
 
  == Mailling List ==
 
  * tajo-private (with moderated subscriptions)
 
  * tajo-dev
 
  * tajo-commits
 
 
  == Subversion Directory ==
 
  https://git-wip-us.apache.org/repos/asf/tajo.git
 
 
  == Issue Tracking ==
 
  Jira Tajo (TAJO)
 
 
  == Other Resources ==
 
  * Continuous Integration
 
* Jenkins
 
  * Wiki
 
* http://wiki.apache.org/tajo
 
 
  = Initial Committers =
 
  * Eli Reisman ereisman AT apache DOT org
 
  * Henry Saputra hsaputra AT apache DOT org
 
  * Hyunsik Choi hyunsik AT apache DOT org
 
  * Jae Hwa Jung jhjung AT gruter DOT com
 
  * Jihoon Son ghoonson AT gmail DOT com
 
  * Jin Ho Kim jhkim AT gruter DOT com
 
  * Roshan Sumbaly rsumbaly AT gmail DOT com
 
  * Sangwook Kim swkim AT inervit DOT com
 
  * Yi A Liu yi DOT a DOT liu AT intel DOT com
 
 
  = Affiliations =
 
  * Eli Reisman (Hortonworks)
 
  * Henry Saputra (Platfora)
 
  * Hyunsik Choi (Database Lab., Korea University)
 
  * Jae Hwa Jung (Gruter)
 
  * Jihoon Son (Database Lab., Korea University)
 
  * Jin Ho Kim (Gruter)
 
  * Roshan Sumbaly (LinkedIn)
 
  * Sangwook Kim (Inervit)
 
  * Yi A Liu (Intel)
 
 
  The nominated mentors are employees of NASA JPL, LinkedIn, and
 Hortonworks.
 
  * Chris Mattmann - NASA JPL
 
  * Jakob Homan - LinkedIn
 
  * Owen O'Malley - Hortonworks
 
 
  = Sponsors =
 
  == Champion ==
 
  * Jakob Homan ghoman AT apache DOT org
 
 
  == Nominated Mentors ==
 
  * Chris Mattmann chris DOT a DOT mattmann AT jpl DOT nasa DOT gov
 
  * Jakob Homan jghoman AT apache DOT org
 
  * Owen O'Malley omalley AT apache DOT org
 
 
  == Sponsoring Entity ==
 
  Apache Incubator


 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Accept Tez into Incubator

[Owen O'Malley]

 dot org
  * Kevin Wilfong kevinwilfong at apache dot org
  * Mike Liddell mike dot lidell at microsoft dot com
  * Namit Jain namit at apache dot org
  * Nathan Roberts nroberts at yahoo dash inc dot com
  * Owen O'Malley omalley at apache dot org
  * Robert Evans bobby at apache dot org
  * Siddharth Seth sseth at apache dot org
  * Tom White tomwhite at apache dot org
  * Thomas Graves tgraves at apache dot org
  * Vikram Dixit vikram at apache dot org
  * Vinod Kumar Vavilapalli vinodkv at apache dot org
  * William Graham billgraham at apache dot org
 
 We realize that though we have significant employer diversity already,
 additional diversity is always better, and we will work
 aggressively to recruit developers from additional companies.
 
 === Alignment ===
 The initial committers strongly believe that a standard task DAG
 application on Apache Hadoop YARN will gain broader adoption as an open
 source,
 community driven project, where the community can contribute not only to
 the
 core components, but also to a growing collection of applications which
 will
 be based on top of Tez. Our hope is that the Apache Hive, Apache Pig,
 Cascading and other communities will find tremendous value in Tez and
 will adopt
 it en masse.
 
 == Known Risks ==
 
 === Orphaned Products ===
 The contributors are leading users and vendors in the Apache Hadoop
 ecosystem,
 with significant open source experience, so the risk of being orphaned is
 relatively low. The project could be at risk if vendors decided to change
 their strategies in the market. In such an event, the current committers
 plan to continue working on the project on their own time, though the
 progress will likely be slower. We plan to mitigate this risk by
 recruiting additional committers.
 
 === Inexperience with Open Source ===
 The initial committers include veteran Apache members (Committers, PMC
 members
 and Apache Members) and other developers who have varying degrees of
 experience
 with open source projects. All have been involved with source code that
 has
 been released under an open source license, and several also have
 experience
 developing code with an open source development process.
 
 === Homogenous Developers ===
 The initial committers are employed by a number of companies, including
 Cloudera, Facebook, Hortonworks, Microsoft, Twitter and Yahoo. We are
 committed
 to recruiting additional committers from other companies based on their
 contributions to the project even though we do have significant diversity
 already.
 
 === Reliance on Salaried Developers ===
 It is expected that Tez development will occur on both salaried time and
 on
 volunteer time, after hours. The majority of initial committers are paid
 by
 their employer to contribute to this project. However, they are all
 passionate
 about the project, and we are confident that the project will continue
 even if
 no salaried developers contribute to the project. We are committed to
 recruiting
 additional committers including non-salaried developers.
 
 === Relationships with Other Apache Products ===
 As mentioned in the Alignment section, Tez is closely integrated with
 Hadoop,
 Hive and Pig in a numerous ways. We look forward to collaborating with
 those communities, as well as other Apache communities.
 
 === An Excessive Fascination with the Apache Brand ===
 Tez solves a real need for generic task DAG management in the Apache
 Hadoop
 ecosystem, something which has been addressed in a very ad hoc manner so
 far
 by multiple Apache projects. Our rationale for developing Tez as an
 Apache
 project is detailed in the Rationale section. We believe that the Apache
 brand
 and community process will help us attract more contributors to this
 project,
 and help establish ubiquitous APIs.
 
 == Documentation ==
 http://wiki.apache.org/incubator/TezProposal
 
 == Initial Source ==
 Available as a patch.
 
 == Cryptography ==
 Tez will eventually support encryption on the wire. This is not one of
 the initial
 goals, and we do not expect Tez to be a controlled export item due to the
 use
 of encryption.
 
 == Required Resources ==
 
 === Mailing List ===
  * tez-private
  * tez-dev
  * tez-user
 
 === Subversion Directory ===
 Git is the preferred source control system: git://git.apache.org/tez
 
 === Issue Tracking ===
 
 JIRA Tez (TEZ)
 
 == Initial Committers ==
  * Alan Gates gates at apache dot org
  * Arun C Murthy acmurthy at apache dot org
  * Ashutosh Chauhan hashutosh at apache dot org
  * Bikas Saha bikas at apache dot org
  * Chris Douglas cdouglas at apache dot org
  * Daryn Sharp daryn at apache dot org
  * Devaraj Das ddas at apache dot org
  * Gopal Vijayaraghavan gopal at hortonworks dot com
  * Gunther Hagleitner ghagleitner at hortonworks dot com
  * Hitesh Shah hitesh at apache dot org
  * Jason Lowe jlowe at apache dot org
  * Jean Xu jeanxu at facebook dot com
  * Jitendra Pandey jitendra at apache dot org
  * Julien Le Dem julien at apache dot org

Re: [RESULT] [VOTE] Accept Apache Open Climate Workbench into the Incubator

[Owen O'Malley]

+1 (binding)


On Thu, Feb 14, 2013 at 9:06 PM, Mattmann, Chris A (388J) 
chris.a.mattm...@jpl.nasa.gov wrote:

 Hi Everyone,

 This VOTE has passed with the following tallies:

 +1

 Chris Mattmann*
 Andrew Hart*
 Daniel Gruno
 Paul Ramirez*
 Gary Martin
 Ross Gardler*
 Ted Dunning*
 Alexei Fedotov
 Dave Fisher*
 Suresh Marru*
 Tommaso Teofili*
 Andrea Pescetti
 Chris Douglas*
 Emmanuel Lécharny*
 Tsengdar Lee

 * - indicates IPMC

 I'll get started creating the infrastructure tickets, and thanks to
 everyone for VOTE'ing!

 Cheers,
 Chris


 On 2/5/13 8:18 AM, Mattmann, Chris A (388J)
 chris.a.mattm...@jpl.nasa.gov wrote:

 Hi Folks,
 
 OK, now that discussion has settled down, I'd like to call a VOTE for
 acceptance of Apache Open Climate Workbench into the Incubator.
 I'll leave the VOTE open the rest of the week and close it out next
 Monday, February 11th early am PT.
 
 [ ]  +1 Accept Apache Open Climate Workbench into the Incubator
 [ ]  +0 Don't care.
 [ ]  -1 Don't accept Apache Open Climate Workbench into the Incubator
 because...
 
 Full proposal is pasted at the bottom of this email. Only VOTEs from
 Incubator PMC members are binding, but all are welcome to express their
 thoughts.
 
 Thank you!
 
 Cheers,
 Chris
 
 P.S. Here's my +1 (binding)
 
 -
 = Apache Open Climate Workbench, tool for scalable comparison of remote
 sensing observations to climate model outputs, regionally and globally. =
 === Abstract ===
 The Apache Open Climate Workbench proposal desires to contribute an
 existing community of software related to the analysis and evaluation of
 climate models, and related to the use of remote sensing data in that
 process.
 
 Specifically, we will bring a fundamental software toolkit for analysis
 and evaluation of climate model output against remote sensing data. The
 toolkit is called the [[http://rcmes.jpl.nasa.gov|Regional Climate Model
 Evaluation System (RCMES)]]. RCMES provides two fundamental components for
 the easy, intuitive comparison of climate model output against remote
 sensing data. The first component called RCMED (for Regional Climate
 Model Evaluation Database) is a scalable cloud database that decimates
 remote sensing data and renalysis data related to climate using Apache
 OODT extractors, Apache Tika, etc. These transformations make
 traditionally heterogeneous upstream remote sensing data and climate model
 output homogeneous and unify them into a data point model of the form
 (lat, lng, time, value, height) on a per parameter basis. Latitude (lat)
 and Longitude (lng) are in WGS84 format, but can be reformatted on the
 fly. time is in ISO 8601 format, a string sortable format independent of
 underlying store. value carries with it units, related to interpretation
 and height allows for different values for different atmospheric vertical
 levels. All of RCMES is built on Apache OODT, Apache Sqoop/Apache Hadoop
 and Apache Hive, along with hooks to PostGIS and MySQL (traditional
 relational databases). The second component of the system, RCMET (for
 Regional Climate Model Evaluation Toolkit) provides facilities for
 connecting to RCMED, dynamically obtaining remote sensing data for a
 space/time region of interest, grabbing associated model output (that the
 user brings, or from the Earth System Grid Federation) of the same form,
 and then regridding the remote sensing data to be on the model output
 grid, or the model output to be on the remote sensing data grid. The
 regridded data spatially is then temporally regridded using techniques
 including seasonal cycle compositing (e.g., all summer months, all
 Januaries, etc.), or by daily, monthly, etc. The uniform model output and
 remote sensing data are then analyzed using pluggable metrics, e.g.,
 Probability Distribution Functions (PDFs), Root Mean Squared Error (RMSE),
 Bias, and other (possibly user-defined) techniques, computing an analyzed
 comparison or evaluation. This evaluation is then visualized by plugging
 in to the NCAR NCL library for producing static plots (histograms, time
 series, etc.)
 
 We also have performed a great deal of work in packaging RCMES to make the
 system easy to deploy. We have working Virtual Machines (VMWare VMX and
 Virtual Box OVA compatible formats) and we also have an installer built on
 Python Buildout (http://buildout.org/) called Easy RCMET for
 dynamically
 constructing the RCMET toolkit.
 
 RCMES is currently supporting a number of recognized climate projects of
 (inter-)national significance. In particular, RCMES is supporting the
 [[http://www.globalchange.gov/what-we-do/assessment|U.S. National Climate
 Assessment (NCA) activities]] on behalf of NASA's contribution to the NCA;
 is working with the [[http://www.narccap.ucar.edu/|North American
 Regional
 Climate Change Assessment Program (NARCCAP)]]; and is also working with
 the International [[http://wcrp-cordex.ipsl.jussieu.fr/|Coordinated
 Regional Downscaling Experiment (CORDEX)]].
 
 === Proposal 

Re: [VOTE] Accept Apache Knox Hadoop Gateway Project into the Incubator

[Owen O'Malley]

 DOT org

 Subversion Directory

 https://svn.apache.org/repos/asf/incubator/knox

 Issue Tracking

 JIRA Knox (KNOX)

 Initial Committers

 Kevin Minder (kevin DOT minder AT hortonworks DOT com)

 Larry McCay (lmccay AT hortonworks DOT com)

 John Speidel (jspeidel AT hortonworks DOT com)
 Tom Beerbower (tbeerbower AT hortonworks DOT com)
 Sumit Mohanty (smohanty AT hortonworks DOT com)
 Venkatesh Seetharam (venkatesh AT hortonworks DOT com)

 Affiliations

 Kevin Minder (Hortonworks)

 Larry McCay (Hortonworks)

 John Speidel (Hortonworks)
 Tom Beerbower (Hortonworks)
 Sumit Mohanty (Hortonworks)
 Venkatesh Seetharm (Hortonworks)
 Owen O'Malley (Hortonworks)
 Mahadev Konar (Hortonworks)
 Alan Gates (Hortonworks)
 Devaraj Das (Hortwonrks)
 Chris Douglas (Microsoft)
 Chris Mattmann (NASA)
 Tom White (Cloudera)

 Sponsors

 Champion

 Devaraj Das (ddas AT apache DOT org)

 Nominated Mentors

 Owen O’Malley (omalley AT apache DOT org)
 Mahadev Konar (mahadev AT apache DOT org)
 Alan Gates (gates AT apache DOT org)
 Devaraj Das (ddas AT apache DOT org)
 Chris Douglas (cdouglas AT apache DOT org)
 Chris Mattmann (chris DOT a DOT mattmann AT jpl DOT nasa DOT gov)
 Tom White (tom DOT e DOT white AT gmail DOT com)

 Sponsoring Entity

 Incubator PMC

 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Graduate HCatalog from the incubator and become part of Hive

[Owen O'Malley]

+1 (binding)


On Tue, Feb 5, 2013 at 1:26 PM, Jakob Homan jgho...@gmail.com wrote:

 +1 Binding.


 On Tue, Feb 5, 2013 at 1:24 AM, Alexander Alten-Lorenz
 wget.n...@gmail.comwrote:

  +1, non-binding
 
  - Alex
 
  On Feb 5, 2013, at 10:06 AM, Sushanth Sowmyan khorg...@gmail.com
 wrote:
 
   And my axe! Erm... I mean, my +1.
  
  
   On Mon, Feb 4, 2013 at 10:18 PM, Alan Gates ga...@hortonworks.com
  wrote:
   FYI.
  
   Alan.
  
   Begin forwarded message:
  
   From: Alan Gates ga...@hortonworks.com
   Date: February 4, 2013 10:18:09 PM PST
   To: hcatalog-...@incubator.apache.org
   Subject: [VOTE] Graduate HCatalog from the incubator and become part
  of Hive
  
   The Hive PMC has voted to accept HCatalog as a submodule of Hive.
  You
  can see the vote thread at
 
 http://mail-archives.apache.org/mod_mbox/hive-dev/201301.mbox/%3cCACf6RrzktBYD0suZxn3Pfv8XkR=vgwszrzyb_2qvesuj2vh...@mail.gmail.com%3e.
  We now need to vote to graduate from the incubator and become a
  submodule of Hive.  This entails the following:
  
   1) the establishment of an HCatalog submodule in the Apache Hive
  Project;
   2) the adoption of the Apache HCatalog codebase into the Hive
 HCatalog
  submodule; and
   3) adding all currently active HCatalog committers as submodule
  committers on the Hive HCatalog submodule.
  
   Definitions for all these can be found in the (now adopted) Hive
  bylaws at
 
 https://cwiki.apache.org/confluence/display/Hive/Proposed+Changes+to+Hive+Bylaws+for+Submodule+Committer
  .
  
   This vote will stay open for at least 72 hours (thus 23:00 PST on
  2/7/13).  PPMC members votes are binding in this vote, though input from
  all is welcome.
  
   If this vote passes the next step will be to submit the graduation
  motion to the Incubator PMC.
  
   Here's my +1.
  
   Alan.
  
 
  --
  Alexander Alten-Lorenz
  http://mapredit.blogspot.com
  German Hadoop LinkedIn Group: http://goo.gl/N8pCF
 
 
  -
  To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
  For additional commands, e-mail: general-h...@incubator.apache.org
 
 

Re: [VOTE] Release Kafka 0.7.2-incubating (Candidate 5)

[Owen O'Malley]

On Sun, Oct 7, 2012 at 10:17 AM, Joe Stein crypt...@gmail.com wrote:
 I would like to keep the vote open for another few days to give the IPMC 
 members time to review and vote, thanks.

Joe,
   Could you update your gpg key:
* set it in id.apache.org
* get someone who knows you to sign it.

WIthout a signed key that is on id.apache.org, I'm -0 (binding) on the
release. I did verify the md5 checksum.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Release Apache HCatalog 0.4.0-incubating

[Owen O'Malley]

On Fri, May 11, 2012 at 4:33 PM, Alan Gates ga...@hortonworks.com wrote:
 Hi,

 Please vote for releasing Apache HCatalog 0.4.0-incubating.

 rc6 for this release has passed a PPMC vote and is now advancing to an IPMC 
 vote.

+1 (binding)

verified sigs, checked rat, and compiled.

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Ambari Status

[Owen O'Malley]

On Thu, May 10, 2012 at 4:55 AM, Franklin, Matthew B.
mfrank...@mitre.org wrote:

 Also, I noticed that ddas signed off on it as a mentor.  I don't see him 
 listed
as an Apache member or IPMC member.

 He was added to the IPMC. The message from Noel was dated 6 Oct 2011.

Note: he is currently listed in committee-info.txt as a member of the IPMC.

 Missed that yesterday.  Thanks for the clarification.  Can we update the 
 podling status page to reflect his mentorship of Ambari?

That was my mistake in that I misremembered Devaraj as a mentor for
Ambari and so I added him as approving it on the wiki. (He had +1'ed
the report on the ambari-dev mailing list.) Given that Devaraj is on
the IPMC and is following the dev lists already, I think it would make
sense to add him as a mentor. If Chris Mattmann is willing to join as
a mentor too, that would add a non-Hortonworks viewpoint that would be
appreciated.

Thanks,
   Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Ambari Status

[Owen O'Malley]

On Wed, May 9, 2012 at 5:56 PM, Franklin, Matthew B.
mfrank...@mitre.org wrote:
 In reviewing the Amabari report, I have a couple of notes/concerns:

  * The community IMHO is in serious risk of failure to launch.  A recent code 
 change has sparked project interest, but list activity was non existent after 
 January.

Ambari's code base isn't functional yet and until it is and it is
difficult to build a community until a version works. All of the
contributors  are employed by Hortonworks, which exposes the project
to the whims of the company's priorities. There's a new code base that
has been checked in on a branch that is close to working. Once there
is a working version, I hope that community will build.

   *As Ambari went nearly 2 months without a single e-mail to the list, I 
 think an issue for graduation should be added that the community must 
 demonstrate viability for X more reporting periods (where x IMO is  2).

I agree that Ambari is no where close to graduation and was close to
being dead. I hope the new code base will revitalize it.

   * The list of completed JIRA issues is not very telling of anything and 
 might give the impression that the community is more vibrant than it is

It was very active for September to January. I explicitly called out
that the project had been dormant since January.

 Also, I noticed that ddas signed off on it as a mentor.  I don't see him 
 listed as an Apache member or IPMC member.

He was added to the IPMC. The message from Noel was dated 6 Oct 2011.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Ambari Status

[Owen O'Malley]

On Wed, May 9, 2012 at 6:40 PM, Jukka Zitting jukka.zitt...@gmail.com wrote:

 Sounds like the project community still needs to learn how to develop
 code out in the open. The massive revision 1335750 [1] seems to have
 been based on no open discussion on the mailing list or the issue
 tracker.

There is a new code base that was developed in private, which has
momentum. It was checked in on to a development branch and the
discussion started. I was hoping for some response to Suhas' message
from someone outside of Hortonworks, but so far it has been silent.
Unfortunately, the lack of response is another sign that the community
hasn't formed yet.

 Of course learning that stuff is what the Incubator exists for, so
 it's all fine as long as the community is willing to learn and mentors
 are there to help.

I'm trying my best.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0

[Owen O'Malley]

On Mon, May 7, 2012 at 11:08 PM, Greg Stein gst...@gmail.com wrote:
 On Mon, May 7, 2012 at 9:48 PM, Bruno Mahé bm...@apache.org wrote:
...

 It seems that we're talking about this location:
  http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/

 Again, we don't distribute non-Apache software,

 I didn't find any non-Apache software in the location noted above.

They are proposing adding Hue, which is a Apache licensed project on
Github. I raised the concern when they stated the plan, not after the
release.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0

[Owen O'Malley]

On Tue, May 8, 2012 at 12:30 AM, Jukka Zitting jukka.zitt...@gmail.com wrote:
 Hi,

 On Tue, May 8, 2012 at 8:38 AM, Owen O'Malley omal...@apache.org wrote:
 They are proposing adding Hue, which is a Apache licensed project on
 Github.

 Does Hue match the guidelines in http://www.apache.org/legal/resolved.html?

 If yes, then it's fine by ASF policy for an Apache project to include it.

 Is there some other reason why BigTop shouldn't be adding Hue?

Hue is Apache licensed, so that isn't an issue. The issue is that
Bigtop is planning on distributing non-Apache software for the primary
purpose of providing it to their users. It seems to me pretty
fundamental that Apache projects should only distribute Apache
software except as a dependency. Bigtop is only including Hue to
provide it to their users with no dependency at all.

It would be great if Cloudera were to contribute Hue to Apache. Unless
they do, I really don't think it is appropriate for us to distribute
standalone binary packages of their project from Apache's servers.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0

[Owen O'Malley]

On Fri, May 4, 2012 at 2:00 PM, Roman Shaposhnik r...@apache.org wrote:

 I don't understand the BigTop use cases

 Perhaps this preso can help a bit:
    http://people.apache.org/~rvs/apache-bigtop2.pdf

Notes from this presentation:
* 100% Apache Bigdata management distribution
* Bigtop is a distribution for users

When users install Bigtop they get a small rpm of bigtop utilities
and rpms lot of other projects. Apache has never had a Redhat-like
project before, but to claim that publishing non-Apache rpms and debs
for the sole purpose of providing them to users is identical to
project X bundling their required external dependencies is bending the
truth substantially.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0

[Owen O'Malley]

On Sat, May 5, 2012 at 2:13 AM, Jukka Zitting jukka.zitt...@gmail.com wrote:

 Now (again IIUC) the interesting bit is whether it's better for BigTop
 to be repackaging and -distributing upstream components by itself, or
 if it would in fact be better for BigTop to simply provide something
 like bigtop-x.y.rpm and bigtop-x.y.deb packages that just declare
 dependencies to specific, integration-tested versions of upstream
 packages.

It would be much better for Bigtop to work with the Apache dev
communities that already exist for each of the projects rather than
releasing incompatible artifacts. It creates a lot of confusion that
the Bigtop Hadoop rpms are labelled as hadoop-*.rpm and yet are
significantly different from the Hadoop rpms that are produced by the
Hadoop project. Even more troubling is that the Bigtop distro is
starting to change the interaction with the user to the projects.

 To do this, BigTop would need to work with the upstream projects to
 help them produce the appropriate deployment packages as a part of
 their normal release processes. And BigTop could also team up with
 Infrastructure to maintain the kind of repository structure and
 download service expected by deployment tools like yum and apt, a bit
 like what Maven projects have in https://repository.apache.org/.

I asked Roman to do this and he decided not to. Certainly Hadoop is
interested in having the improvements made in their rpms and debs.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0

[Owen O'Malley]

On Mon, May 7, 2012 at 9:05 AM, Roman Shaposhnik r...@apache.org wrote:
 On Mon, May 7, 2012 at 8:46 AM, Owen O'Malley omal...@apache.org wrote:
 On Fri, May 4, 2012 at 2:00 PM, Roman Shaposhnik r...@apache.org wrote:

 I don't understand the BigTop use cases

 Perhaps this preso can help a bit:
    http://people.apache.org/~rvs/apache-bigtop2.pdf

 Notes from this presentation:
 * 100% Apache Bigdata management distribution
 * Bigtop is a distribution for users

 Owen, I'd appreciate if you refrained from quoting out of context.
 Please take a look at slide #11 for a complete list of things
 that Bigtop is trying to accomplish. Saying that it is *just*
 a distribution for users is like saying that Debian GNU Linux
 is just about the Linux kernel.

I never said just a distribution for users, but your presentation
makes it very clear it is first and foremost a distribution for users.
Since we now agree about that, how about getting back to whether an
Apache distro project should be distributing non-Apache software.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Graduate Giraph to a TLP

[Owen O'Malley]

With 4 binding +1's from the IPMC, I'll add the resolution to the board agenda.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0

[Owen O'Malley]

On Fri, May 4, 2012 at 8:58 AM, Patrick Hunt ph...@apache.org wrote:
 It's not the job of the incubator to create new rules, but rather to
 help podlings to graduation while following existing Apache
 guidelines.

We aren't making new rules. We are trying to help the Bigtop project
understand the rules about not releasing non-Apache software. There is
a huge difference between depending on an artifact from another
project and building and distributing non-Apache rpms in the project's
/dist directory.

 It's very clear from
 http://www.apache.org/legal/resolved.html that what has been proposed
 is acceptable under existing Apache rules.

Can you find a single instance other than the disagreement between
Apache Lucene and Apache Commons where one project is distributing
another project's rpms? Are there any other non-Apache rpms in /dist?
Clearly the answer is a resounding NO. It would be a huge violation of
the trust the incubator is putting in me as a mentor if I didn't block
Bigtop's plan to do so.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0

[Owen O'Malley]

On Mon, Apr 30, 2012 at 9:39 PM, Roman Shaposhnik r...@apache.org wrote:
 On Mon, Apr 30, 2012 at 9:13 PM, Eric Baldeschwieler
 eri...@hortonworks.com wrote:
 So you are suggesting expanding the charter to include projects not hosted 
 at Apache?

 I don't think this is what Bruno suggested. Personally I can't find
 any reference in Bigtop
 charter that restricts it to Projects that belong to Apache Software
 Foundation.

As a mentor of the Bigtop project, I don't see it as acceptable for an
Apache project to distribute binaries of non-Apache software. If the
owners of the Hue project decide to donate it to Apache and it had
been released by Apache, then it would be acceptable. I'm strictly -1
on releasing any version of Bigtop with Hue or any other non-Apache
software as part of the release.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0

[Owen O'Malley]

On Thu, May 3, 2012 at 12:01 PM, Bruno Mahé bm...@apache.org wrote:

 As a mentor of the Bigtop project, I don't see it as acceptable for an
 Apache project to distribute binaries of non-Apache software. If the
 owners of the Hue project decide to donate it to Apache and it had
 been released by Apache, then it would be acceptable. I'm strictly -1
 on releasing any version of Bigtop with Hue or any other non-Apache
 software as part of the release.

 -- Owen


 As part of mentoring Apache Bigtop (incubating) project, it would also
 be greatly appreciated if you would explain why this -1.

 Apache Bigtop (incubating) does not and will not include anything that
 does not belong to the Apache Foundation.
 So I am really confused as to why this strong reaction.

The strong reaction is because Roman was proposing a Bigtop release
with rpms and debs for non-Apache projects. That is a non-starter.
Apache will not distribute non-Apache projects. Saying that Bigtop
does not release the projects that it incorporates is not justified
given the fact that Bigtop is putting rpms of each of the incorporated
projects into /dist/incubator/bigtop. The 2.9GB size of the latest
Bigtop release has already caused infrastructure significant
headaches.

 The convenience artefact may pull Hue in, but this is in no way
 different from Apache Hadoop pulling in Google protocol buffer or Google
 guava. So again, how is this different? Is Apache Hadoop going to
 avandon Google Protocolbuffer?

There is a big difference between referencing external projects that
are required for your project's functionality and incorporating
non-Apache projects into your project and publishing releases of them
using independent artifacts. When the user installs a Hadoop rpm, the
protobuf.jar is there under the hood, but is considered an
implementation detail that is required for Hadoop to run.

I'd complain similarly if Hadoop was downloading protobuf tarballs,
making changes to protobuf, making protobuf rpms with those changes,
and publishing those rpms on Apache's servers.

However, it goes deeper than than that. If the user installs Bigtop's
rpms and hits a bug do they contact Hue or Bigtop? Furthermore, I'm
sure the links that are displayed when you run Bigtop's Hue point off
to Cloudera's bug and support system. That kind of branding is not ok
for an Apache project.

Even with Apache projects, Bigtop may become problematic. Look at the
mess that happened when Lucene made a bugfix release of Apache Commons
CSV. Lucene needed a bugfix release of CSV, didn't wait for CSV to
release, and instead released it themselves. Needless to say Apache
Commons didn't like that result. Bigtop is overriding decisions made
by the upstream projects about things like the way the launching
scripts operate and where the configuration directory is. When asked
to correct it, they complain about compatibility for their users
rather than compatibility for Hadoop's users.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Giraph status (Was: [Incubator Wiki] Update of April2012 by OwenOmalley)

[Owen O'Malley]

On Wed, Apr 11, 2012 at 4:30 PM, Jukka Zitting jukka.zitt...@gmail.com wrote:
 Hi,

 Thanks for the report, Giraph!

 On Tue, Apr 3, 2012 at 7:15 AM, Apache Wiki wikidi...@apache.org wrote:
 + Next steps:
 + * Another incubator release
 + * Adding new committers

 Looking at the report and recent project activity it looks like Giraph
 is doing pretty well. In fact I can't spot any obvious graduation
 blockers. Anything I'm missing? If not, have you already started
 preparing for graduation?

We haven't started yet, but I was thinking along similar lines. Should
I start a vote on the dev list?

Thanks,
   Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Other mentors for accumulo?

[Owen O'Malley]

On Fri, Sep 9, 2011 at 12:28 AM, Bernd Fondermann
bernd.fonderm...@googlemail.com wrote:
 I've signed up myself, too.

I'll sign up too.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Accumulo to join the Incubator

[Owen O'Malley]

 is necessary, we would like to work through that process during
 the incubation phase of Accumulo rather than before acceptance as this
 may take time to enter into an agreement.

 == External Dependencies ==
 jetty (Apache and EPL), jline (BSD), jfreechart (LGPL), jcommon (LGPL),
 slf4j (MIT), junit (CPL)

 == Cryptography ==
 none

 == Required Resources ==
  * Mailing Lists
   * accumulo-private
   * accumulo-dev
   * accumulo-commits
   * accumulo-user

  * Subversion Directory
   * https://svn.apache.org/repos/asf/incubator/accumulo

  * Issue Tracking
   * JIRA Accumulo (ACCUMULO)

  * Continuous Integration
   * Jenkins builds on https://builds.apache.org/

  * Web
   * http://incubator.apache.org/accumulo/
   * wiki at http://wiki.apache.org or http://cwiki.apache.org

 == Initial Committers ==
  * Aaron Cordova (aaron at cordovas dot org)
  * Adam Fuchs (adam.p.fuchs at ugov dot gov)
  * Eric Newton (ecn at swcomplete dot com)
  * Billie Rinaldi (billie.j.rinaldi at ugov dot gov)
  * Keith Turner (keith.turner at ptech-llc dot com)
  * John Vines (john.w.vines at ugov dot gov)
  * Chris Waring (christopher.a.waring at ugov dot gov)

 == Affiliations ==
  * Aaron Cordova, The Interllective
  * Adam Fuchs, National Security Agency
  * Eric Newton, SW Complete Incorporated
  * Billie Rinaldi, National Security Agency
  * Keith Turner, Peterson Technology LLC
  * John Vines, National Security Agency
  * Chris Waring, National Security Agency

 == Sponsors ==
  * Champion: Doug Cutting

 == Nominated Mentors ==
  * Benson Margulies
  * Alan Cabrera
  * Bernd Fondermann
  * Owen O'Malley

 == Sponsoring Entity ==
  * Apache Incubator


 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org



-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Change the name of the HMS podling to Ambari.

[Owen O'Malley]

With 8 +1's and no -1's, the name change passes.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [PROPOSAL] Accumulo for the Apache Incubator

[Owen O'Malley]

Is the NSA going to file a code grant for the project? How deeply
embedded are the LGPL dependencies? Are they optional components or
mandatory?

Thanks,
   Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [PROPOSAL] Accumulo for the Apache Incubator

[Owen O'Malley]

On Fri, Sep 2, 2011 at 3:22 PM, Adam P Fuchs adam.p.fu...@ugov.gov wrote:

The project looks interesting.

 I believe the answer is yes regarding the code grant, and I am currently 
 confirming that with our lawyers. We'll get you an official answer early next 
 week.

Great. I know that the US government has its own rules for such
things. I took part in the meetings that created the NASA Open Source
Agreement. (eg. the lawyers wouldn't let us call it an open source
license...) Let us know how it goes.

 The LGPL dependencies are not core to Accumulo, and we're working on 
 substituting other packages. We would have no problem doing this before the 
 initial commit if necessary.

I needs to be cleaned up before release, but the original commit is fine.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

[VOTE] Change the name of the HMS podling to Ambari.

[Owen O'Malley]

When I was doing the initial trademark search for HMS, I found that
there are a lot of projects (including software) named HMS. It also
has the problem of being very bad to search for (42.7m hits on
google).

Before I go through and create the infrastructure for Apache
incubator, I wanted to discuss changing the name. Suhas did some
searches and found that the name for the royal chair on top of
elephants is Ambari, which seems like a very nice name to me.

Toward that end, I'd like to propose changing the name from HMS to Ambari.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Accept HMS as an incubator project

[Owen O'Malley]

On Thu, Aug 25, 2011 at 1:57 PM, Devaraj Das d...@hortonworks.com wrote:

 Hello everyone,
 This is a vote proposing that HMS be accepted as a project in the Apache
 Incubator. HMS is monitoring, administration and lifecycle management
 project for Apache Hadoop clusters. The latest proposal is pasted at the end
 and it could be found in the wiki as well -
 http://wiki.apache.org/incubator/HMSProposal

 The related discussion thread is at:

 http://www.mail-archive.com/general@incubator.apache.org/msg30354.html

 Please cast your votes:
 [  ] +1 Accept HMS for incubation
 [  ] +0 Indifferent to HMS incubation
 [  ] -1 Reject HMS for incubation (and why)


+1

Re: [VOTE] Giraph to join the incubator

[Owen O'Malley]

+1 (binding)

Re: [VOTE] Release HCatalog 0.1-incubating (RC1)

[Owen O'Malley]

Ok, I've verified the key, signature and md5. All of the unit tests passed.

+1

Re: [PROPOSAL] Proposing Giraph for the Apache Incubator

[Owen O'Malley]

On Fri, Jul 15, 2011 at 11:14 AM, Avery Ching ach...@yahoo-inc.com wrote:

 Hi,

 I would like to propose Giraph as an Apache Incubator project.


Obviously, I'm +1 for this. *smile*

-- Owen

Re: [PROPOSAL] Bigtop for the Apache Incubator

[Owen O'Malley]

On Mon, Jun 13, 2011 at 9:26 PM, Tom White tomwh...@apache.org wrote:

 Hi,

 I would like to propose Bigtop to be an Apache Incubator project.
 Bigtop is a project for the development of packaging and tests of the
 Hadoop ecosystem. The goal is to do testing at various levels
 (packaging, platform, runtime, upgrade, etc...) developed by a
 community with a focus on the system as a whole, rather than
 individual projects.


I'd like to join as a committer and mentor.

-- Owen

Re: Hadoop related Incubator projects

[Owen O'Malley]

On Tue, Jun 14, 2011 at 8:35 AM, Noel J. Bergman n...@devtech.com wrote:

 There are some which indicate that their sole impediment to graduation is
 community size.  Has there been any consideration of the Hadoop TLP picking
 them up as sub-projects?


 Wow, the wheels turn. The Hadoop PMC was strongly encouraged by the board
to divest itself of the Hadoop-podlings, because it had become an umbrella
project. I think that over all, most of the projects are doing better are
doing as TLP and incubator projects.

-- Owen

Basing Apache releases on releases from incubating projects

[Owen O'Malley]

In http://bit.ly/fhCxIt , Steve argued that it isn't permitted for non- 
podling releases to depend on the artifacts that come out of podlings.  
I believe that to be false, but I can't find a relevant statement in  
the FAQ. Can I get clarification?


Thanks,
   Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Basing Apache releases on releases from incubating projects

[Owen O'Malley]

On Dec 1, 2010, at 8:51 AM, Bertrand Delacretaz wrote:


It is common for podlings to import their code here and only then
start cleaning up dependencies that do not comply with ASF
requirements. So if another project release contains unreleased code
from a podling, it might not be clean.


In general, TLP releases shouldn't depend on unreleased artifacts of  
podlings or other TLP.  As you point out unreleased podling artifacts  
are even more likely to be problematic. If someone wants to depend on  
it, their effort would probably be best spent helping the podling to  
release.


But consensus seems to agree that TLP can depend on podling releases,  
right?


-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Accept Gora into the Apache Incubator

[Owen O'Malley]

+1 binding

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Thrift 0.3.0 RC6

[Owen O'Malley]

On Jul 28, 2010, at 12:28 PM, Bryan Duxbury wrote:


The GPG signature can be found at
http://people.apache.org/~bryanduxbury/thrift-0.3.0-rc6.tar.gz.aschttp://people.apache.org/%7Ebryanduxbury/thrift-0.3.0-rc6.tar.gz.asc 



Shouldn't you make a KEYS file in subversion somewhere so that people  
can validate the release? For example, we put the Hadoop Common keys  
into http://svn.apache.org/respos/asf/hadoop/common/dist/KEYS

.

-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Thrift 0.3.0 RC6

[Owen O'Malley]

On Aug 2, 2010, at 11:37 AM, Bryan Duxbury wrote:

Owen - I'm not sure I understand what you mean, particularly because  
your

link doesn't work.


*Sigh* You'd think that the 5 billion times that I'd typed the Hadoop  
subversion URL that I wouldn't mess it up any more.


http://svn.apache.org/repos/asf/hadoop/common/dist/KEYS

The point is to post your public keys in a public place that is  
controlled by Apache. Of course it is better if the keys are signed by  
other Apache members. We really should do a key signing at one of the  
Bay Area HUGs.


-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [Result][Vote] Move Chukwa to incubator

[Owen O'Malley]

On Jun 30, 2010, at 10:51 AM, Mattmann, Chris A (388J) wrote:


On 6/30/10 9:42 AM, Eric Yang ey...@yahoo-inc.com wrote:

Original incubator proposal:

http://wiki.apache.org/incubator/ChukwaProposal

Vote put forward to incubator:

1. recommend TLP with guides to help the initial pmc,
2. accept incubating with tlp resource naming, but -incubating release
naming
3. accept incubating requiring all incubator naming conventions,  
that might

help the incubator simplify this decision.

Result of the vote:

Option 1) Ant Elder, Eric Yang, William A. Rowe Jr.
Option 2) Ari Rabkin, Jerome Boulon, Chris Douglas, Greg Reddin
Option 3) Bernd Fondermann

Owen O'Malley +1 on proposal


Sorry, I meant to vote for option 2.  I think that it would be good to  
have more visibility in before they become a TLP. They also need to  
figure out how to add new committers to their project.


-- Owen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Move Chukwa to incubator

[Owen O'Malley]

On Jun 21, 2010, at 10:29 AM, Eric Yang wrote:

Please vote as to whether you think Chukwa should move to Apache  
incubator.


The proposal is posted at:

http://wiki.apache.org/incubator/ChukwaProposal


+1

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Should we accept Pig as a Hadoop subproject?

[Owen O'Malley]

Owen O'Malley wrote:


Should we accept Pig as a subproject?


With six +1's from the Hadoop PMC, the vote passes. Hadoop will have a  
new subproject. *smile*


-- Owen

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]