Re: [VOTE] Graduate Apache Gobblin (Incubating) as a TLP
+1 (binding) On Sun, Dec 20, 2020 at 2:31 PM Olivier Lamy wrote: > +1 > > On Thu, 17 Dec 2020 at 18:55, Abhishek Tiwari wrote: > > > Hi all, > > > > After a discussion with the community [1], a positive vote > > result [2], and a discussion with the IPMC [7], we believe > > Apache Gobblin (Incubating) is ready for graduation to a TLP, > > and we'd like to call for a Vote. > > > > Please vote on the resolution pasted below to graduate > > Apache Gobblin (Incubating) from the incubator to a TLP: > > > > [ ] +1, Graduate Apache Gobblin from the Incubator. > > [ ] +0, Don't care. > > [ ] -1, Don't graduate Apache Gobblin from the Incubator (provide > details) > > > > To list a few of the community's achievements: > > > > - 1700+ contributions from 149 contributors > > - 4 releases under 3 different release managers > > - 5 new committers invited, including 2 new PPMC members (all accepted) > > - Diverse committers and PPMCs (from 9 companies/institutes) > > - New Apache website setup [3] > > - Issue tracking migrated to Jira [4] > > - Dev conversations moved to d...@gobblin.incubator.org > > - Apache Gobblin name search has been approved [5] > > - Assessed ourselves against the Apache Project maturity matrix [6] > > - We have built a meritocratic and open collaborative process (the Apache > > way) > > > > [1] https://s.apache.org/le17y > > [2] https://s.apache.org/q4ug6 > > [3] https://gobblin.incubator.apache.org/ > > [4] https://issues.apache.org/jira/projects/GOBBLIN/issues/ > > [5] https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-131 > > [6] > > > https://cwiki.apache.org/confluence/display/GOBBLIN/ASF+Maturity+Evaluation > > [7] https://s.apache.org/1gt3s > > > > The vote will remain open for at least 72 hours. > > > > If approved by the Apache Incubator PMC members, the proposed > > resolution will be submitted to the Board of Directors for their > > consideration. > > > > Thanks, > > Abhishek > > > > --- > > > > Establish the Apache Gobblin Project > > > > WHEREAS, the Board of Directors deems it to be in the best interests of > > the Foundation and consistent with the Foundation's purpose to establish > > a Project Management Committee charged with the creation and maintenance > > of open-source software, for distribution at no charge to the public, > > related to a distributed data integration framework that simplifies > > common aspects of big data integration such as data ingestion, > > replication, organization and lifecycle management for both streaming > > and batch data ecosystems. > > > > NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee > > (PMC), to be known as the "Apache Gobblin Project", be and hereby is > > established pursuant to Bylaws of the Foundation; and be it further > > > > RESOLVED, that the Apache Gobblin Project be and hereby is responsible > > for the creation and maintenance of software related to a distributed > > data integration framework that simplifies common aspects of big data > > integration such as data ingestion, replication, organization and > > lifecycle management for both streaming and batch data ecosystems; and > > be it further > > > > RESOLVED, that the office of "Vice President, Apache Gobblin" be and > > hereby is created, the person holding such office to serve at the > > direction of the Board of Directors as the chair of the Apache Gobblin > > Project, and to have primary responsibility for management of the > > projects within the scope of responsibility of the Apache Gobblin > > Project; and be it further > > > > RESOLVED, that the persons listed immediately below be and hereby are > > appointed to serve as the initial members of the Apache Gobblin Project: > > > > * Abhishek Tiwari > > * Hung Tran > > * Issac Buenrostro > > * Jean-Baptiste Onofré > > * Kishore G > > * Lorand Bendig > > * Olivier Lamy > > * Owen O'Malley > > * Sahil Takiar > > * Shirshanka Das > > * Sudarshan Vasudevan > > * Tamás Németh > > * Yinan Li > > > > NOW, THEREFORE, BE IT FURTHER RESOLVED, that Abhishek Tiwari be > > appointed to the office of Vice President, Apache Gobblin, to serve in > > accordance with and subject to the direction of the Board of Directors > > and th
Re: [VOTE] Graduation of Apache DataSketches Project to TLP
+1 (binding) On Tue, Nov 24, 2020 at 11:56 AM Kevin Ratnasekera wrote: > +1 ( binding ) > > On Wed, Nov 25, 2020 at 1:25 AM Dave Fisher wrote: > > > +1 (binding) > > > > > On Nov 24, 2020, at 11:54 AM, leerho wrote: > > > > > > Folks, > > > > > > We have completed a discussion and voting on our community list, > > > d...@datasketches.apache.org. as follows: > > > > > > - Discussion Thread: > > > > > > https://lists.apache.org/thread.html/r691d6394e78f7a9d321fddd040b14a8282e7c6f799fb327ee743beae%40%3Cdev.datasketches.apache.org%3E > > > - Vote Thread: > > > > > > https://lists.apache.org/thread.html/r922e2a10e53b4eabdeb089336828c8c256277c9fd5ab80a3a13329d0%40%3Cdev.datasketches.apache.org%3E > > > > > > We have completed a discussion on the Incubator General list, > > > general@incubator.apache.org as follows: > > > > > > - Discussion Thread: > > > > > > https://lists.apache.org/thread.html/r65f2c40c15a35026f4bf2e270051cfffed1b47c78c5685e7dd353d2a%40%3Cgeneral.incubator.apache.org%3E > > > > > > In summary we have 11 (+1) votes to graduate, and no (0) nor (-1) > votes. > > > > > > Of these 11, we have > > > > > > - 4 binding (+1) votes from our Mentors: Dave Fisher, Furkan Kamaci, > > > Liang Chen, and Kenneth Knowles; > > > - 6 PPMC members: Alex Saydakov, Eshcar Hillel, Jon Maklin, Justin > > > Thaler, Lee Rhodes, Roman Leventov; > > > - 1 Contributor: Michael Himes. > > > > > > Please visit our documentation for this graduation so far: > > > > > > - Our Graduation Checklist: > > > > > https://datasketches.apache.org/docs/Graduation/GraduationCheckList.html > > > - Our Maturity Model: > > > https://datasketches.apache.org/docs/Graduation/Maturity.html > > > - Our Resolution: > > > https://datasketches.apache.org/docs/Graduation/Resolution.html > > > > > > If there is anyone in the IPMC that has any questions or concerns that > > > would block our ability to graduate, please bring these issues into > this > > > thread. > > > > > > Unless there are issues that need to be resolved, this vote will end > next > > > Monday, Nov 30th, Midnight Pacific Standard Time (UTC-8). > > > > > > If this vote concludes with no objections that would block our > > graduation, > > > I will proceed to the next step and draft the Proposal Letter to the > > Board. > > > > > > It is not clear to me whether I, as DataSketches PMC Chair Elect, may > > send > > > this Proposal Letter to the Board, or whether it must be sent by one of > > our > > > Mentors or the IPMC Chair. Please advise. > > > > > > Regards, > > > Lee Rhodes > > > DataSketches PMC Chair Elect, Committer > > > > > > - > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > -- Ryan Blue Software Engineer Netflix
Re: [VOTE] Graduate to a top-level project
> I would assume because it's a dependancy and not bundled in the source release. I searched for "copied from" to test that assumption and found code from Apache Commons: https://github.com/apache/tomcat/blob/bb0befeebe18a9b8ff50e686a3d06e84ccdfb108/java/org/apache/tomcat/util/http/fileupload/package-info.java#L20-L22 That lines up with my reading of LEGAL-234. If a copyright line for each project was required in NOTICE, then the question of which dates are required would not be relevant. And the conclusion to that question was that "My [Mark T.] reading of [1] is that you only need the Accumulo dates." On Wed, May 20, 2020 at 5:04 PM Justin Mclean wrote: > Hi, > > > Mark is talking about what they did for the Tomcat NOTICE, with no > mention > > of Commons: https://github.com/apache/tomcat/blob/master/NOTICE > > I would assume because it's a dependancy and not bundled in the source > release. Even if that is the case older projects may not be in line with > ASF policy or do thing in a different way because of other reasons. > > Only things that are bundled in the release need to be mentioned in > LICENSE and NOTICE. > > > And the result was this Accumulo NOTICE: > > https://github.com/apache/accumulo/blob/master/NOTICE > > Which from a casual glance looks fine, the license includes JQuery, Flat > and code form OneLab. None of those are ALv2 or have other required notices > so there’s no effect NOTICE. > > Thanks, > Justin > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > > -- Ryan Blue Software Engineer Netflix
Re: [VOTE] Graduate to a top-level project
> You see in that Mark recommends including the project name/copyright line in the NOTICE file as I’ve suggested to you. Are you sure about that? Mark is talking about what they did for the Tomcat NOTICE, with no mention of Commons: https://github.com/apache/tomcat/blob/master/NOTICE And the result was this Accumulo NOTICE: https://github.com/apache/accumulo/blob/master/NOTICE On Wed, May 20, 2020 at 3:56 PM Justin Mclean wrote: > Hi, > > > 1. Should the entire file header be moved into NOTICE when a > third-party > > file is contributed, or should we edit it > > When a 3rd party file is contributed, it’s header stays were it is on the > file, it’s not moved to NOTICE. It the file was part of a software grant > then the copyright notice is added to NOTICE. > > > 2. When including all or part of another ASF project is the standard > > NOTICE sufficient? > > No it depends on the content of it's NOTICE file (and what parts you have > bundled). > > > I went back through legal-discuss and found that #2 has already been > asked > > and answered on https://issues.apache.org/jira/browse/LEGAL-234. > > You see in that Mark recommends including the project name/copyright line > in the NOTICE file as I’ve suggested to you. > > > If the source file is submitted with a copyright notice included in it, > the > > copyright owner (or owner’s agent) must either: > > a. remove such notices, or > > b. move them to the NOTICE file associated with each applicable project > > release, or > > c. provide written permission for the ASF to make such removal or > > relocation of the notices. > > You note it states to move the copyright notice not the header. > > License information belongs in LICENSE, other notices in NOTICE. > > Thanks, > Justin > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > > -- Ryan Blue Software Engineer Netflix
Re: [VOTE] Graduate to a top-level project
To summarize, it looks like there are two issues: 1. Should the entire file header be moved into NOTICE when a third-party file is contributed, or should we edit it 2. When including all or part of another ASF project is the standard NOTICE sufficient? I went back through legal-discuss and found that #2 has already been asked and answered on https://issues.apache.org/jira/browse/LEGAL-234. The recommendation from Mark Thomas was to not copy the standard NOTICE contents from other projects. Marvin Humphrey also notes that this documentation is canonical: http://www.apache.org/legal/src-headers.html#notice and the other with confusing clause (“must be considered”) is advisory. The same canonical documentation also covers source file headers for code that was contributed. This is the document that we used when we brought in the contribution with the full Cloudera header. It states: If the source file is submitted with a copyright notice included in it, the copyright owner (or owner’s agent) must either: a. remove such notices, or b. move them to the NOTICE file associated with each applicable project release, or c. provide written permission for the ASF to make such removal or relocation of the notices. That’s why we waited to merge the PR until someone from Cloudera had consented to moving the header. We didn’t ask whether we could edit the header at that time. Given that we are careful to document consent to move the header in this situation, I don’t think that we should edit it years after the contribution. Everyone at the time thought that the header was the entire block of text; while I understand where you’re drawing a distinction, I think it is reasonable to continue publishing the entire header as it was found in the original contribution. Are you satisfied with those answers, Justin? On Mon, May 18, 2020 at 6:57 PM Justin Mclean wrote: > Hi, > > > At the time, we considered the whole header to be the relevant "Copyright > > notification". Do you think this practice is incorrect? > > I’ve not seem other projects do it this way, it's generally: > "Portions of this software were developed at > Acme Inc, copyright 20XX.” > > > Which ones are you referring to? > > > > I just went through the ones that you mentioned: > > * Spark's NOTICE doesn't have relevant portions, considering what is used > > is the check-license script and some Parquet vectorization code. > > * Same for Hive's NOTICE because the only portion of Hive that is used > is a > > SQL file with metastore table definitions. > > * The relevant portion of Parquet's notice is the code from Cloudera and > > the Iceberg NOTICE contains the same text. > > * Avro's NOTICE contains mostly entries for the C# project and the only > > code from Avro is the decimal conversion code. > > * The iBATIS NOTICE contains two full relocated copyright headers, with > > Copyright attributed to the HSQL Development Group and the Hypersonic SQL > > group. ScriptRunner that is included in Iceberg was written by Clinton > > Begin, who started the ASF project, so those sections aren't relevant. > > The Spark notice contains: > "Apache Spark > Copyright 2014 and onwards The Apache Software Foundation.” > > The Hive notice contains: > "Apache Hive > Copyright 2008-2018 The Apache Software Foundation” > > etc > > This I believe would apply to your notice file. > > > Can you explain a bit more about this? > > See above. > > Thanks, > Justin > ----- > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > > -- Ryan Blue
Re: [VOTE] Graduate to a top-level project
On Mon, May 18, 2020 at 4:40 PM Justin Mclean wrote: > > Per NOTICE policy > > <https://www.apache.org/dev/licensing-howto.html#mod-notice>: "Copyright > > notifications which have been relocated, rather than removed, from source > > files must be preserved in NOTICE” > > Yes the copyright line needs to be mentioned not the licenser header. > At the time, we considered the whole header to be the relevant "Copyright notification". Do you think this practice is incorrect? > > My understanding is that it is not necessary to modify NOTICE for an ASF > > dependency, if it is the boiler-plate ASF NOTICE because the project's > > notice contains "This product includes software developed at the Apache > > Software Foundation", while the LICENSE contains the copyright > information > > for individual ASF projects. > > Yes but those notice files contain more than that and are not boiler-plate. > Which ones are you referring to? I just went through the ones that you mentioned: * Spark's NOTICE doesn't have relevant portions, considering what is used is the check-license script and some Parquet vectorization code. * Same for Hive's NOTICE because the only portion of Hive that is used is a SQL file with metastore table definitions. * The relevant portion of Parquet's notice is the code from Cloudera and the Iceberg NOTICE contains the same text. * Avro's NOTICE contains mostly entries for the C# project and the only code from Avro is the decimal conversion code. * The iBATIS NOTICE contains two full relocated copyright headers, with Copyright attributed to the HSQL Development Group and the Hypersonic SQL group. ScriptRunner that is included in Iceberg was written by Clinton Begin, who started the ASF project, so those sections aren't relevant. >From double-checking, it appears that the last time we updated license documentation, we did it correctly. > > " though the ASF copyright line and any other portions of NOTICE must be > considered for propagation.” > > I don’t believe your project have done this. You'll note the copyright > line will be different depending on when you incorporated the code. > Can you explain a bit more about this? Do you think that we skipped it or just didn't do it correctly? What about the copyright lines are not correct? And what do you think we need to do to correct it? -- Ryan Blue Software Engineer Netflix
Re: [VOTE] Graduate to a top-level project
On Mon, May 18, 2020 at 3:35 PM Justin Mclean wrote: > Hi, > > > What license information are you referring to? > > " Licensed under the Apache License” by Cloudera > This is not a license, it is the original copyright header for files that were contributed by Cloudera. Here's the commit <https://github.com/apache/parquet-mr/commit/ddbeb4dd17d9c219b99b1e66d8be28efe37e3aa6> . Per NOTICE policy <https://www.apache.org/dev/licensing-howto.html#mod-notice>: "Copyright notifications which have been relocated, rather than removed, from source files must be preserved in NOTICE" > > Which entries in the LICENSE file? Many Apache-licensed projects don't > have > > NOTICE files. > > Yes but all ASF projects do, your license mentions Apache Avro, Apache > Parquet, Apache iBATIS, Apache Hive and Apache Spark. > My understanding is that it is not necessary to modify NOTICE for an ASF dependency, if it is the boiler-plate ASF NOTICE because the project's notice contains "This product includes software developed at the Apache Software Foundation", while the LICENSE contains the copyright information for individual ASF projects. NOTICE policy <https://www.apache.org/dev/licensing-howto.html#bundle-asf-product>: "It is not necessary to duplicate the line 'This product includes software developed at the Apache Software Foundation...', though the ASF copyright line and any other portions of NOTICE must be considered for propagation." And the general guidance: "Do not add anything to NOTICE which is not legally required." Also kitesdk does have a notice file [1] as does presto [2] > Given the guidance, "Do not add anything to NOTICE which is not legally required," and "The LICENSE and NOTICE files must exactly represent the contents of the distribution they reside in," we have been including only sections of third-party NOTICE files that apply to the code distributed by Apache Iceberg. The Kite NOTICE file credits the ASF (already covered) and Saxonica. Because I was a Kite developer, I know that the Saxonica part does not apply to the code that was included in Apache Iceberg. In fact, Iceberg only includes classes from Kite that I wrote while I worked at Cloudera. For the Presto, the code that we include is also clearly not covered by either line of the Presto NOTICE: it is neither t-digest related nor AVLTreeDigest related. Because this clearly doesn't apply to the code we are using, we omitted it. I should note that when it isn't clear which parts of a third-party NOTICE apply to the code that a project uses, our practice is to copy the entire contents of the third-party NOTICE into ours. > What is the correct archive location? > > > https://archive.apache.org/dist/incubator/iceberg/apache-iceberg-0.7.0-incubating/ > Thanks, I'll open a PR to update it. rb -- Ryan Blue
Re: [VOTE] Graduate to a top-level project
Hi Justin, I have more questions for you inline: On Sat, May 16, 2020 at 6:29 PM Justin Mclean wrote: > Hi, > 1. You have included some license information this should be in LICENSE > not NOTICE > What license information are you referring to? > 2. Going on what is listed in the LICENSE file you are missing content > from bundled code NOTICE files [1] (see bundling an apache 2.0 licences > dependancy) > Which entries in the LICENSE file? Many Apache-licensed projects don't have NOTICE files. Maybe you are looking for a NOTICE entry for projects with no NOTICE? Re the download page, the older version 0.7 needs to point to the archive > not the mirrors and not use https://dist.apache.org/repos/dist/ What is the correct archive location? -- Ryan Blue Software Engineer Netflix
Re: [VOTE] Recommend Apache Iceberg graduation to top-level project resolution to the board
+1 (binding) On Fri, May 15, 2020 at 5:39 PM Ryan Blue wrote: > Hi everyone, > > With the support of our mentors (as well as helpful ASF members), the > Apache Iceberg community has voted to graduate to a top-level project. > > I propose a vote to recommend graduation for the Iceberg community to the > board. Here is the proposed resolution: > > ``` > Establish the Apache Iceberg Project > > WHEREAS, the Board of Directors deems it to be in the best interests of > the Foundation and consistent with the Foundation's purpose to establish > a Project Management Committee charged with the creation and maintenance > of open-source software, for distribution at no charge to the public, > related to managing huge analytic datasets using a standard at-rest > table format that is designed for high performance and ease of use.. > > NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee > (PMC), to be known as the "Apache Iceberg Project", be and hereby is > established pursuant to Bylaws of the Foundation; and be it further > > RESOLVED, that the Apache Iceberg Project be and hereby is responsible > for the creation and maintenance of software related to managing huge > analytic datasets using a standard at-rest table format that is designed > for high performance and ease of use; and be it further > > RESOLVED, that the office of "Vice President, Apache Iceberg" be and > hereby is created, the person holding such office to serve at the > direction of the Board of Directors as the chair of the Apache Iceberg > Project, and to have primary responsibility for management of the > projects within the scope of responsibility of the Apache Iceberg > Project; and be it further > > RESOLVED, that the persons listed immediately below be and hereby are > appointed to serve as the initial members of the Apache Iceberg Project: > > * Anton Okolnychyi > * Carl Steinbach > * Daniel C. Weeks > * James R. Taylor > * Julien Le Dem > * Owen O'Malley > * Parth Brahmbhatt > * Ratandeep Ratti > * Ryan Blue > > NOW, THEREFORE, BE IT FURTHER RESOLVED, that Ryan Blue be appointed to > the office of Vice President, Apache Iceberg, to serve in accordance > with and subject to the direction of the Board of Directors and the > Bylaws of the Foundation until death, resignation, retirement, removal > or disqualification, or until a successor is appointed; and be it > further > > RESOLVED, that the Apache Iceberg Project be and hereby is tasked with > the migration and rationalization of the Apache Incubator Iceberg > podling; and be it further > > RESOLVED, that all responsibilities pertaining to the Apache Incubator > Iceberg podling encumbered upon the Apache Incubator PMC are hereafter > discharged. > ``` > > The community vote passed with 19 +1 votes: > https://lists.apache.org/thread.html/r9081df0181768f26490f5a85aed7b5a111a82a741764eb3a0e99621d%40%3Cdev.iceberg.apache.org%3E > > We also addressed concerns on the IPMC thread, which you can read here: > https://lists.apache.org/thread.html/r3e5795d959feb0a19b233aeaf1121a1d97fd473f5e9b14227de41c54%40%3Cgeneral.incubator.apache.org%3E > > Please vote on whether to recommend graduation for the Apache Iceberg > community to the board. > > [ ] +1 Apache Iceberg should graduate > [ ] +0 > [ ] -1 Apache Iceberg should not graduate because . . . > > The vote will be open for at least 72 hours. > > -- > Ryan Blue > -- Ryan Blue
[VOTE] Recommend Apache Iceberg graduation to top-level project resolution to the board
Hi everyone, With the support of our mentors (as well as helpful ASF members), the Apache Iceberg community has voted to graduate to a top-level project. I propose a vote to recommend graduation for the Iceberg community to the board. Here is the proposed resolution: ``` Establish the Apache Iceberg Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software, for distribution at no charge to the public, related to managing huge analytic datasets using a standard at-rest table format that is designed for high performance and ease of use.. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the "Apache Iceberg Project", be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache Iceberg Project be and hereby is responsible for the creation and maintenance of software related to managing huge analytic datasets using a standard at-rest table format that is designed for high performance and ease of use; and be it further RESOLVED, that the office of "Vice President, Apache Iceberg" be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache Iceberg Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache Iceberg Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache Iceberg Project: * Anton Okolnychyi * Carl Steinbach * Daniel C. Weeks * James R. Taylor * Julien Le Dem * Owen O'Malley * Parth Brahmbhatt * Ratandeep Ratti * Ryan Blue NOW, THEREFORE, BE IT FURTHER RESOLVED, that Ryan Blue be appointed to the office of Vice President, Apache Iceberg, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed; and be it further RESOLVED, that the Apache Iceberg Project be and hereby is tasked with the migration and rationalization of the Apache Incubator Iceberg podling; and be it further RESOLVED, that all responsibilities pertaining to the Apache Incubator Iceberg podling encumbered upon the Apache Incubator PMC are hereafter discharged. ``` The community vote passed with 19 +1 votes: https://lists.apache.org/thread.html/r9081df0181768f26490f5a85aed7b5a111a82a741764eb3a0e99621d%40%3Cdev.iceberg.apache.org%3E We also addressed concerns on the IPMC thread, which you can read here: https://lists.apache.org/thread.html/r3e5795d959feb0a19b233aeaf1121a1d97fd473f5e9b14227de41c54%40%3Cgeneral.incubator.apache.org%3E Please vote on whether to recommend graduation for the Apache Iceberg community to the board. [ ] +1 Apache Iceberg should graduate [ ] +0 [ ] -1 Apache Iceberg should not graduate because . . . The vote will be open for at least 72 hours. -- Ryan Blue
Re: [VOTE] Graduate to a top-level project
re the project was donated to Apache. In fact, both presentations occurred before the ASF signed the trademark agreement, on 2019-09-28. > - I note your last report stated "Add more committers and PPMC members” > and ”grow the community” as a graduation step. Has this happened since > last month report? > Growing the community and adding committers/PPMC members is a general aspiration. This was not intended to be a self-imposed requirement. > - The last Committer/PMC addition was over 9 months ago. Do you have any > currently people you might consider for committership? The last report > mentioned "A contributor has built support for Spark streaming, which is > under review”. > The last committer was added in February. Looks like we didn't update the date in our last report because August 2019 is when the previous committer was added. I addressed the broader question of maturity above. Yes, there are contributors on the path to committer but we don't see value in a requirement for some number of committers before graduation. -- Ryan Blue
Fwd: [VOTE] Graduate to a top-level project
Forwarding the graduation vote thread from the Iceberg community to let everyone here know, per [1]. [1]: https://incubator.apache.org/guides/graduation.html#community_graduation_vote -- Forwarded message - From: Ryan Blue Date: Tue, May 12, 2020 at 2:16 PM Subject: [VOTE] Graduate to a top-level project To: Iceberg Dev List Hi everyone, I propose that the Iceberg community should petition to graduate from the Apache Incubator to a top-level project. Here is the draft board resolution: Establish the Apache Iceberg Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software, for distribution at no charge to the public, related to managing huge analytic datasets using a standard at-rest table format that is designed for high performance and ease of use.. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the "Apache Iceberg Project", be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache Iceberg Project be and hereby is responsible for the creation and maintenance of software related to managing huge analytic datasets using a standard at-rest table format that is designed for high performance and ease of use; and be it further RESOLVED, that the office of "Vice President, Apache Iceberg" be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache Iceberg Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache Iceberg Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache Iceberg Project: * Anton Okolnychyi * Carl Steinbach * Daniel C. Weeks * James R. Taylor * Julien Le Dem * Owen O'Malley * Parth Brahmbhatt * Ratandeep Ratti * Ryan Blue NOW, THEREFORE, BE IT FURTHER RESOLVED, that Ryan Blue be appointed to the office of Vice President, Apache Iceberg, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed; and be it further RESOLVED, that the Apache Iceberg Project be and hereby is tasked with the migration and rationalization of the Apache Incubator Iceberg podling; and be it further RESOLVED, that all responsibilities pertaining to the Apache Incubator Iceberg podling encumbered upon the Apache Incubator PMC are hereafter discharged. Please vote in the next 72 hours. [ ] +1 Petition the IPMC to graduate to top-level project [ ] +0 [ ] -1 Wait to graduate because . . . -- Ryan Blue -- Ryan Blue
[RESULT] [VOTE] Release Apache Iceberg 0.8.0-incubating RC2
With 4 +1 votes and no -1 or -0 votes, this passes. Thanks for taking a look at this release, everyone! On Wed, May 6, 2020 at 11:26 AM Owen O'Malley wrote: > +1 (binding) > > On Mon, May 4, 2020 at 8:56 AM Ryan Blue > wrote: > >> Thanks, Justin. Can you be more specific about what you think isn't >> correct >> in the NOTICE file? Our practice is to copy the entire notice of >> third-party projects, if they exist. We don't do this for the boiler-plate >> ASF NOTICE, but any ASF notice that contains other text will be copied in >> (like Yetus in the runtime Jar). >> >> On Sun, May 3, 2020 at 7:49 PM Justin Mclean >> wrote: >> >> > Hi, >> > >> > +1(binding) >> > >> > I checked: >> > - incubating in name >> > - signature and hashes are fine >> > - DISCLAIMER exits >> > - LICENSE is fine but may mention code that is no longer bundled? (e.g >> mvn) >> > - NOTICE year needs updating >> > - NOTICE content are not correct. There is a number of ALv2 bt of code >> > bundles and the relevant parts of their NOTICE files haven’t been copied >> > into the NOTICE. [1] Please fix for the next release. There is no need >> for >> > the license header to be in there. >> > - No unexpected binary files >> > - All ASF files have ASF headers >> > - I had trouble compiling from source as it required a new version of >> > cradle than I had on my machine but is incompatible with the latest >> version >> > off cradle (7.0) >> > >> > Thanks, >> > Justin >> > >> > 1. https://www.apache.org/dev/licensing-howto.html >> > - >> > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >> > For additional commands, e-mail: general-h...@incubator.apache.org >> > >> > >> >> -- >> Ryan Blue >> Software Engineer >> Netflix >> > -- Ryan Blue Software Engineer Netflix
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC2
Thanks, Justin. Can you be more specific about what you think isn't correct in the NOTICE file? Our practice is to copy the entire notice of third-party projects, if they exist. We don't do this for the boiler-plate ASF NOTICE, but any ASF notice that contains other text will be copied in (like Yetus in the runtime Jar). On Sun, May 3, 2020 at 7:49 PM Justin Mclean wrote: > Hi, > > +1(binding) > > I checked: > - incubating in name > - signature and hashes are fine > - DISCLAIMER exits > - LICENSE is fine but may mention code that is no longer bundled? (e.g mvn) > - NOTICE year needs updating > - NOTICE content are not correct. There is a number of ALv2 bt of code > bundles and the relevant parts of their NOTICE files haven’t been copied > into the NOTICE. [1] Please fix for the next release. There is no need for > the license header to be in there. > - No unexpected binary files > - All ASF files have ASF headers > - I had trouble compiling from source as it required a new version of > cradle than I had on my machine but is incompatible with the latest version > off cradle (7.0) > > Thanks, > Justin > > 1. https://www.apache.org/dev/licensing-howto.html > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > > -- Ryan Blue Software Engineer Netflix
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC2
For anyone that would like to vote on this release that is unfamiliar with
Iceberg, here are the community’s recommended checks:
1. Download the source tarball, signature (.asc), and checksum (.sha512)
from
https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
2. Import gpg keys: download KEYS
<https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS> and run gpg
--import /path/to/downloaded/KEYS (optional if this hasn’t changed)
3. Verify the signature by running: gpg --verify
apache-iceberg-0.8.0-incubating.tar.gz.asc
4. Verify the checksum by running: sha512sum -c
apache-iceberg-0.8.0-incubating.tar.gz.sha512
5. Untar the archive and go into the source directory: tar xzf
apache-iceberg-0.8.0-incubating.tar.gz && cd apache-iceberg-0.8.0-incubating
6. Run RAT checks to validate license headers: dev/check-license
7. Build and test the project: ./gradlew build (use Java 8)
To validate the convenience binaries, add the Maven URL from the email
above to a downstream project and update your Iceberg dependency to
0.8.0-incubating, like this:
repositories {
maven {
name 'stagedIceberg'
url
'https://repository.apache.org/content/repositories/orgapacheiceberg-1006/'
}
}
ext {
icebergVersion = '0.8.0-incubating'
}
If you would like to test out Iceberg with Apache Spark 2.4, you can run
Spark with the Apache Iceberg 0.8.0-incubating runtime Jar like this:
spark-2.4.5-bin-hadoop2.7/bin/spark-shell \
--conf
spark.jars.repositories=https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
\
--packages org.apache.iceberg:iceberg-spark-runtime:0.8.0-incubating \
--conf spark.sql.warehouse.dir=/home/blue/tmp/spark-warehouse \
--conf spark.hadoop.hive.metastore.uris=thrift://localhost:45655
To create tables, please refer to the Spark guide
<https://iceberg.apache.org/api-quickstart/> and the API guide
<https://iceberg.apache.org/api/>.
On Sun, May 3, 2020 at 11:51 AM Ryan Blue wrote:
> Hi everyone,
>
> I propose the following candidate to be released as the official Apache
> Iceberg 0.8.0-incubating release.
>
> The Iceberg PPMC vote passed
> <https://lists.apache.org/thread.html/r28e30803aeab81bf822750aa26cb35b378bacd9aa0d68ea463dbcfff%40%3Cdev.iceberg.apache.org%3E>
> with 12 +1 votes, 7 of which were binding PPMC votes, and 3 of which were
> binding IPMC votes from project mentors (blue, omalley, and cws).
>
> The commit id is 8c05a2f5f1c8b111c049d43cf15cd8a51920dda1
> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc2
> *
> https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc2
> * https://github.com/apache/incubator-iceberg/tree/8c05a2f5
>
> The release tarball, signature, and checksums are here:
> *
> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc2/
>
> You can find the KEYS file here:
> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
>
> Convenience binary artifacts are staged in Nexus. The Maven repository URL
> is:
> *
> https://repository.apache.org/content/repositories/orgapacheiceberg-1006/
>
> This release contains many bug fixes and several new features:
> * Actions to remove orphaned files and to optimize metadata for query
> performance
> * Support for ORC data files
> * Snapshot cherry-picking
> * Incremental scan planning based on table history
> * In and notIn expressions
> * An InputFormat for writing MR jobs
>
> Please download, verify, and test.
>
> Please vote in the next 72 hours.
>
> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
> [ ] +0
> [ ] -1 Do not release this because...
>
> --
> Ryan Blue
>
--
Ryan Blue
[VOTE] Release Apache Iceberg 0.8.0-incubating RC2
Hi everyone, I propose the following candidate to be released as the official Apache Iceberg 0.8.0-incubating release. The Iceberg PPMC vote passed <https://lists.apache.org/thread.html/r28e30803aeab81bf822750aa26cb35b378bacd9aa0d68ea463dbcfff%40%3Cdev.iceberg.apache.org%3E> with 12 +1 votes, 7 of which were binding PPMC votes, and 3 of which were binding IPMC votes from project mentors (blue, omalley, and cws). The commit id is 8c05a2f5f1c8b111c049d43cf15cd8a51920dda1 * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc2 * https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc2 * https://github.com/apache/incubator-iceberg/tree/8c05a2f5 The release tarball, signature, and checksums are here: * https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc2/ You can find the KEYS file here: * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS Convenience binary artifacts are staged in Nexus. The Maven repository URL is: * https://repository.apache.org/content/repositories/orgapacheiceberg-1006/ This release contains many bug fixes and several new features: * Actions to remove orphaned files and to optimize metadata for query performance * Support for ORC data files * Snapshot cherry-picking * Incremental scan planning based on table history * In and notIn expressions * An InputFormat for writing MR jobs Please download, verify, and test. Please vote in the next 72 hours. [ ] +1 Release this as Apache Iceberg 0.8.0-incubating [ ] +0 [ ] -1 Do not release this because... -- Ryan Blue
[RESULT] [VOTE] Release Apache Iceberg 0.7.0-incubating RC4
This release vote passes, with 5 +1 votes and no +0 or -1 votes. +1 votes: James Taylor Ryan Blue Jacques Nadeau Julien Le Dem Carl Steinbach Thank you for taking the time to verify this release! On Wed, Oct 23, 2019 at 2:36 PM Carl Steinbach wrote: > +1 (binding) > > On Tue, Oct 22, 2019, 4:42 PM Julien Le Dem .invalid> > wrote: > > > +1 (binding) > > > > Downloaded, verified signature, checked licences, ran the build. > > > > On Tue, Oct 22, 2019 at 4:29 PM Jacques Nadeau > wrote: > > > > > +1 (binding) > > > > > > Downloaded, reviewed licenses. Spot check for bad licenses. Built from > > > source + tests. > > > > > > Nice job on the first release guys! > > > > > > On Tue, Oct 22, 2019 at 4:05 PM James Taylor > > > wrote: > > > > > > > +1 (binding). > > > > > > > > Successfully downloaded, verified license, verified checksum, > verified > > > > signature, built from source, and ran unit tests. > > > > Minor nit (ok to fix in next release IMHO): Copyright in NOTICE for > > > Iceberg > > > > should read 2017-2019 > > > > > > > > > > > > > > > > On Tue, Oct 22, 2019 at 3:21 PM Ryan Blue wrote: > > > > > > > > > Hi everyone, > > > > > > > > > > I propose the following RC to be released as official Apache > Iceberg > > > > > 0.7.0-incubating release. > > > > > > > > > > This candidate has passed a PPMC vote. The result thread is here: > > > > > * > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/fe76494f9feab454837bc2dc540cd9e59d55dac3f7e0b5b335de7725@ > > > > > > > > > > > > > > > The commit id is 9c81babac65351f7aa21dd878f01c5c81ae304af > > > > > * This corresponds to the tag: apache-iceberg-0.7.0-incubating-rc4 > > > > > * > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-iceberg/tree/apache-iceberg-0.7.0-incubating-rc4 > > > > > * > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-iceberg/tree/9c81babac65351f7aa21dd878f01c5c81ae304af > > > > > > > > > > The release tarball, signature, and checksums are here: > > > > > * > > > > > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.7.0-incubating-rc4/ > > > > > > > > > > You can find the KEYS file here: > > > > > * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS > > > > > > > > > > This release includes convenience binary artifacts that are staged > in > > > > > Nexus. The Maven repository URL is: > > > > > * > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapacheiceberg-1004/ > > > > > > > > > > The convenience binaries include a Spark runtime Jar with shaded > > > > > dependencies. The LICENSE and NOTICE content for that Jar are > > viewable > > > in > > > > > github as well as in the Jar: > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-iceberg/tree/apache-iceberg-0.7.0-incubating-rc4/runtime > > > > > > > > > > To build and test, run `./gradlew build`. You can also test by > adding > > > the > > > > > iceberg-spark-runtime Jar to the jars folder of Spark 2.4. > > > > > > > > > > This is the first Apache Iceberg release. > > > > > > > > > > Please download, verify, and test; then vote in the next 72 hours. > > > > > > > > > > [ ] +1 Release this as Apache Iceberg 0.7.0-incubating > > > > > [ ] +0 > > > > > [ ] -1 Do not release this because... > > > > > > > > > > -- > > > > > Ryan Blue > > > > > > > > > > > > > > > -- Ryan Blue
[VOTE] Release Apache Iceberg 0.7.0-incubating RC4
Hi everyone, I propose the following RC to be released as official Apache Iceberg 0.7.0-incubating release. This candidate has passed a PPMC vote. The result thread is here: * https://lists.apache.org/thread.html/fe76494f9feab454837bc2dc540cd9e59d55dac3f7e0b5b335de7725@ The commit id is 9c81babac65351f7aa21dd878f01c5c81ae304af * This corresponds to the tag: apache-iceberg-0.7.0-incubating-rc4 * https://github.com/apache/incubator-iceberg/tree/apache-iceberg-0.7.0-incubating-rc4 * https://github.com/apache/incubator-iceberg/tree/9c81babac65351f7aa21dd878f01c5c81ae304af The release tarball, signature, and checksums are here: * https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.7.0-incubating-rc4/ You can find the KEYS file here: * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS This release includes convenience binary artifacts that are staged in Nexus. The Maven repository URL is: * https://repository.apache.org/content/repositories/orgapacheiceberg-1004/ The convenience binaries include a Spark runtime Jar with shaded dependencies. The LICENSE and NOTICE content for that Jar are viewable in github as well as in the Jar: https://github.com/apache/incubator-iceberg/tree/apache-iceberg-0.7.0-incubating-rc4/runtime To build and test, run `./gradlew build`. You can also test by adding the iceberg-spark-runtime Jar to the jars folder of Spark 2.4. This is the first Apache Iceberg release. Please download, verify, and test; then vote in the next 72 hours. [ ] +1 Release this as Apache Iceberg 0.7.0-incubating [ ] +0 [ ] -1 Do not release this because... -- Ryan Blue
Re: [VOTE] Recommend 'Apache Druid graduation to Top Level Project' resolution to the board
own as the "The Apache Druid Project", > > > > > be and hereby is established pursuant to Bylaws of the > > > > > Foundation; and be it further > > > > > > > > > > RESOLVED, that The Apache Druid Project be and hereby is > > > > > responsible for the creation and maintenance of an analytical > > > > > database software project; and be it further > > > > > > > > > > RESOLVED, that the office of "Vice President, Druid" be and > > > > > hereby is created, the person holding such office to serve at the > > > > > direction of the Board of Directors as the chair of The Apache > > > > > Druid Project, and to have primary responsibility for > > > > > management of the projects within the scope of responsibility of > > > > > The Apache Druid Project; and be it further > > > > > > > > > > RESOLVED, that the persons listed immediately below be and > > > > > hereby are appointed to serve as the initial members of The > > > > > Apache Druid Project: > > > > > > > > > > * Benedict Jin (asdf2...@apache.org) > > > > > * Charles Allen(cral...@apache.org) > > > > > * Clint Wylie (cwy...@apache.org) > > > > > * David Lim(david...@apache.org) > > > > > * Dylan Wylie (dylanwy...@apache.org) > > > > > * Eric Tschetter (ched...@apache.org) > > > > > * Fangjin Yang (f...@apache.org) > > > > > * Gian Merlino (g...@apache.org) > > > > > * Himanshu Gupta (himans...@apache.org) > > > > > * Jihoon Son (jihoon...@apache.org) > > > > > * Jonathan Wei (jon...@apache.org) > > > > > * Julian Hyde (jh...@apache.org) > > > > > * Kurt Young (k...@apache.org) > > > > > * Lijin Bin(binli...@apache.org) > > > > > * Maxime Beauchemin(maximebeauche...@apache.org) > > > > > * Niketh Sabbineni (nik...@apache.org) > > > > > * Nishant Bangarwa (nish...@apache.org) > > > > > * P. Taylor Goetz (ptgo...@apache.org) > > > > > * Parag Jain (pja...@apache.org) > > > > > * Roman Leventov (leven...@apache.org) > > > > > * Slim Bouguerra (bs...@apache.org) > > > > > * Surekha Saharan (sure...@apache.org) > > > > > * Xavier Léauté(x...@apache.org) > > > > > > > > > > NOW, THEREFORE, BE IT FURTHER RESOLVED, that Gian Merlino > > > > > be and hereby is appointed to the office of Vice President, > > > > > Druid, to serve in accordance with and subject to the direction > > > > > of the Board of Directors and the Bylaws of the Foundation until > > > > > death, resignation, retirement, removal or disqualification, or > > > > > until a successor is appointed; and be it further > > > > > > > > > > RESOLVED, that the initial Apache Druid Project be and hereby > > > > > is tasked with the migration and rationalization of the Apache > > > > > Incubator Druid podling; and be it further > > > > > > > > > > RESOLVED, that all responsibility pertaining to the Apache > > > > > Incubator Druid podling encumbered upon the Apache Incubator > > > > > PMC are hereafter discharged. > > > > > > > > > > > - > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > > > > -- Ryan Blue Software Engineer Netflix
Re: [IP CLEARANCE] Apache Arrow Parquet Rust Library
+1 (binding) On Thu, Dec 13, 2018 at 8:24 AM Wes McKinney wrote: > Apache Arrow is receiving a donation of a Rust library to read and > write the Apache Parquet file format, intended to work together with > the Arrow Rust library [1]. > > Please vote to approve this contribution. > > This is a lazy consensus majority vote, per the IP clearance process > [2], open for at least 72 hours. > > Wes > > [1]: http://incubator.apache.org/ip-clearance/arrow-parquet-rust.html > [2] http://incubator.apache.org/ip-clearance/ > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > > -- Ryan Blue Software Engineer Netflix
[RESULT] [VOTE] Accept the Iceberg project for incubation
The vote passes with 13 binding +1 and 5 non-binding +1 votes. Thank you for voting, everyone! I'll get started with the next steps. +1 votes: Ryan Blue* Matt Sicker* Felix Cheung Dave Fisher* Owen O'Malley* Hugo Louro Arthur Wiedmer Julian Hyde* Kevin A. McGrail* Willem Jiang* James Taylor* Uwe Korn Lars Francke* Jean-Baptiste Onofré* Olivier Lamy* Michael Wall* Kenneth Knowles Julien Le Dem* * = binding On Tue, Nov 13, 2018 at 9:06 AM Ryan Blue wrote: > The discuss thread seems to have reached consensus, so I propose accepting > the Iceberg project for incubation. > > The proposal is copied below and in the wiki: > https://wiki.apache.org/incubator/IcebergProposal > > Please vote on whether to accept Iceberg in the next 72 hours: > > [ ] +1, accept Iceberg for incubation > [ ] -1, reject the Iceberg proposal because . . . > > Thank you for reviewing the proposal and voting, > > rb > -- > Iceberg Proposal Abstract > > Iceberg is a table format for large, slow-moving tabular data. > > It is designed to improve on the de-facto standard table layout built into > Apache Hive, Presto, and Apache Spark. > Proposal > > The purpose of Iceberg is to provide SQL-like tables that are backed by > large sets of data files. Iceberg is similar to the Hive table layout, the > de-facto standard structure used to track files in a table, but provides > additional guarantees and performance optimizations: > >- Atomicity - Each change to the table is will be complete or will >fail. “Do or do not. There is no try.” >- Snapshot isolation - Reads use one and only one snapshot of a table >at some time without holding a lock. >- Safe schema evolution - A table’s schema can change in well-defined >ways, without breaking older data files. >- Column projection - An engine may request a subset of the available >columns, including nested fields. >- Predicate pushdown - An engine can push filters into read planning >to improve performance using partition data and file-level statistics. > > Iceberg does NOT define a new file format. All data is stored in Apache > Avro, Apache ORC, or Apache Parquet files. > > Additionally, Iceberg is designed to work well when data files are stored > in cloud blob stores, even when those systems provide weaker guarantees > than a file system, including: > >- Eventual consistency in the namespace >- High latency for directory listings >- No renames of objects >- No folder hierarchy > > Rationale > > Initial benchmarks show dramatic improvements in query planning. For > example, in Netflix’s Atlas use case, which stores time-series metrics from > Netflix runtime systems and 1 month is stored across 2.7 million files in > 2,688 partitions: > >- Hive table using Parquet: > - 400k+ splits, not combined > - Explain query: 9.6 minutes wall time (planning only) >- Iceberg table with partition filtering: > - 15,218 splits, combined > - Planning: 10 seconds > - Query wall time: 13 minutes >- Iceberg table with partition and min/max filtering: > - 412 splits > - Planning: 25 seconds > - Query wall time: 42 seconds > > These performance gains combined with the cross-engine compatibility are a > very compelling story. > Initial Goals > > The initial goal will be to move the existing codebase to Apache and > integrate with the Apache development process and infrastructure. A primary > goal of incubation will be to grow and diversify the Iceberg community. We > are well aware that the project community is largely comprised of > individuals from a single company. We aim to change that during incubation. > Current Status > > As previously mentioned, Iceberg is under active development at Netflix, > and is being used in processing large volumes of data in Amazon EC2. > > Iceberg license documentation is already based on Apache guidelines for > LICENSE and NOTICE content. > Meritocracy > > We value meritocracy and we understand that it is the basis for an open > community that encourages multiple companies and individuals to contribute > and be invested in the project’s future. We will encourage and monitor > participation and make sure to extend privileges and responsibilities to > all contributors. > Community > > Iceberg is currently being used by developers at Netflix and a growing > number of users are actively using it in production environments. Iceberg > has received contributions from developers working at Hortonworks, WeWork, > and Palantir. By bringing Iceberg to Apache we aim to assure current and > future contributors that the Iceberg community is meritocratic and
Re: [VOTE] Accept the Iceberg project for incubation
Quick update: James Taylor has offered to mentor the project as well, so I've added him to the list. Thanks, James! On Tue, Nov 13, 2018 at 9:06 AM Ryan Blue wrote: > The discuss thread seems to have reached consensus, so I propose accepting > the Iceberg project for incubation. > > The proposal is copied below and in the wiki: > https://wiki.apache.org/incubator/IcebergProposal > > Please vote on whether to accept Iceberg in the next 72 hours: > > [ ] +1, accept Iceberg for incubation > [ ] -1, reject the Iceberg proposal because . . . > > Thank you for reviewing the proposal and voting, > > rb > -- > Iceberg Proposal Abstract > > Iceberg is a table format for large, slow-moving tabular data. > > It is designed to improve on the de-facto standard table layout built into > Apache Hive, Presto, and Apache Spark. > Proposal > > The purpose of Iceberg is to provide SQL-like tables that are backed by > large sets of data files. Iceberg is similar to the Hive table layout, the > de-facto standard structure used to track files in a table, but provides > additional guarantees and performance optimizations: > >- Atomicity - Each change to the table is will be complete or will >fail. “Do or do not. There is no try.” >- Snapshot isolation - Reads use one and only one snapshot of a table >at some time without holding a lock. >- Safe schema evolution - A table’s schema can change in well-defined >ways, without breaking older data files. >- Column projection - An engine may request a subset of the available >columns, including nested fields. >- Predicate pushdown - An engine can push filters into read planning >to improve performance using partition data and file-level statistics. > > Iceberg does NOT define a new file format. All data is stored in Apache > Avro, Apache ORC, or Apache Parquet files. > > Additionally, Iceberg is designed to work well when data files are stored > in cloud blob stores, even when those systems provide weaker guarantees > than a file system, including: > >- Eventual consistency in the namespace >- High latency for directory listings >- No renames of objects >- No folder hierarchy > > Rationale > > Initial benchmarks show dramatic improvements in query planning. For > example, in Netflix’s Atlas use case, which stores time-series metrics from > Netflix runtime systems and 1 month is stored across 2.7 million files in > 2,688 partitions: > >- Hive table using Parquet: > - 400k+ splits, not combined > - Explain query: 9.6 minutes wall time (planning only) >- Iceberg table with partition filtering: > - 15,218 splits, combined > - Planning: 10 seconds > - Query wall time: 13 minutes >- Iceberg table with partition and min/max filtering: > - 412 splits > - Planning: 25 seconds > - Query wall time: 42 seconds > > These performance gains combined with the cross-engine compatibility are a > very compelling story. > Initial Goals > > The initial goal will be to move the existing codebase to Apache and > integrate with the Apache development process and infrastructure. A primary > goal of incubation will be to grow and diversify the Iceberg community. We > are well aware that the project community is largely comprised of > individuals from a single company. We aim to change that during incubation. > Current Status > > As previously mentioned, Iceberg is under active development at Netflix, > and is being used in processing large volumes of data in Amazon EC2. > > Iceberg license documentation is already based on Apache guidelines for > LICENSE and NOTICE content. > Meritocracy > > We value meritocracy and we understand that it is the basis for an open > community that encourages multiple companies and individuals to contribute > and be invested in the project’s future. We will encourage and monitor > participation and make sure to extend privileges and responsibilities to > all contributors. > Community > > Iceberg is currently being used by developers at Netflix and a growing > number of users are actively using it in production environments. Iceberg > has received contributions from developers working at Hortonworks, WeWork, > and Palantir. By bringing Iceberg to Apache we aim to assure current and > future contributors that the Iceberg community is meritocratic and open, in > order to broaden and diversity the user and developer community. > Core Developers > > Iceberg was initially developed at Netflix and is under active > development. We believe Netflix will be of interest to a broad range of > users and developers and that incubating th
Re: [VOTE] Accept the Iceberg project for incubation
+1 (binding) On Tue, Nov 13, 2018 at 9:06 AM Ryan Blue wrote: > The discuss thread seems to have reached consensus, so I propose accepting > the Iceberg project for incubation. > > The proposal is copied below and in the wiki: > https://wiki.apache.org/incubator/IcebergProposal > > Please vote on whether to accept Iceberg in the next 72 hours: > > [ ] +1, accept Iceberg for incubation > [ ] -1, reject the Iceberg proposal because . . . > > Thank you for reviewing the proposal and voting, > > rb > -- > Iceberg Proposal Abstract > > Iceberg is a table format for large, slow-moving tabular data. > > It is designed to improve on the de-facto standard table layout built into > Apache Hive, Presto, and Apache Spark. > Proposal > > The purpose of Iceberg is to provide SQL-like tables that are backed by > large sets of data files. Iceberg is similar to the Hive table layout, the > de-facto standard structure used to track files in a table, but provides > additional guarantees and performance optimizations: > >- Atomicity - Each change to the table is will be complete or will >fail. “Do or do not. There is no try.” >- Snapshot isolation - Reads use one and only one snapshot of a table >at some time without holding a lock. >- Safe schema evolution - A table’s schema can change in well-defined >ways, without breaking older data files. >- Column projection - An engine may request a subset of the available >columns, including nested fields. >- Predicate pushdown - An engine can push filters into read planning >to improve performance using partition data and file-level statistics. > > Iceberg does NOT define a new file format. All data is stored in Apache > Avro, Apache ORC, or Apache Parquet files. > > Additionally, Iceberg is designed to work well when data files are stored > in cloud blob stores, even when those systems provide weaker guarantees > than a file system, including: > >- Eventual consistency in the namespace >- High latency for directory listings >- No renames of objects >- No folder hierarchy > > Rationale > > Initial benchmarks show dramatic improvements in query planning. For > example, in Netflix’s Atlas use case, which stores time-series metrics from > Netflix runtime systems and 1 month is stored across 2.7 million files in > 2,688 partitions: > >- Hive table using Parquet: > - 400k+ splits, not combined > - Explain query: 9.6 minutes wall time (planning only) >- Iceberg table with partition filtering: > - 15,218 splits, combined > - Planning: 10 seconds > - Query wall time: 13 minutes >- Iceberg table with partition and min/max filtering: > - 412 splits > - Planning: 25 seconds > - Query wall time: 42 seconds > > These performance gains combined with the cross-engine compatibility are a > very compelling story. > Initial Goals > > The initial goal will be to move the existing codebase to Apache and > integrate with the Apache development process and infrastructure. A primary > goal of incubation will be to grow and diversify the Iceberg community. We > are well aware that the project community is largely comprised of > individuals from a single company. We aim to change that during incubation. > Current Status > > As previously mentioned, Iceberg is under active development at Netflix, > and is being used in processing large volumes of data in Amazon EC2. > > Iceberg license documentation is already based on Apache guidelines for > LICENSE and NOTICE content. > Meritocracy > > We value meritocracy and we understand that it is the basis for an open > community that encourages multiple companies and individuals to contribute > and be invested in the project’s future. We will encourage and monitor > participation and make sure to extend privileges and responsibilities to > all contributors. > Community > > Iceberg is currently being used by developers at Netflix and a growing > number of users are actively using it in production environments. Iceberg > has received contributions from developers working at Hortonworks, WeWork, > and Palantir. By bringing Iceberg to Apache we aim to assure current and > future contributors that the Iceberg community is meritocratic and open, in > order to broaden and diversity the user and developer community. > Core Developers > > Iceberg was initially developed at Netflix and is under active > development. We believe Netflix will be of interest to a broad range of > users and developers and that incubating the project at the ASF will help > us build a diverse, sustainable community. > Alignment > >
[VOTE] Accept the Iceberg project for incubation
Incubator project incurs a certain cost in terms of ASF infrastructure and volunteer time. Inexperience with Open Source Three of the initial committers are Apache members and Incubator PMC members. They will work with the other community members to teach them the Apache Way. Homogenous Developers The majority of the committers work at Netflix, though we are committed to recruiting and developing additional committers from a wide spectrum of industries and backgrounds. Reliance on Salaried Developers It is expected that Iceberg development will occur on both salaried time and on volunteer time, after hours. Most of the initial committers are paid by Netflix to contribute to this project. However, they are all passionate about the project, and we are both confident and hopeful that the project will continue even if no salaried developers contribute to the project. Relationships with Other Apache Products As mentioned in the Rationale section, Iceberg utilizes a number of existing Apache projects (Avro, Hadoop, Hive, ORC, Parquet, Pig, & Spark), and we expect that list to expand as the community grows and diversifies. Any Apache project in the big data space that needs to store or process tabular data would be potentially relevant. An Excessive Fascination with the Apache Brand We are applying to the Incubator process because we think it is the next logical step for the Iceberg project after open-sourcing the code. This proposal is not for the purpose of generating publicity. Rather, we want to make sure to create a very inclusive and meritocratic community, outside the umbrella of a single company. Netflix has a long history of contributing to Apache projects and the Iceberg developers and contributors understand the implication of making it an Apache project. Required Resources Mailing lists - d...@iceberg.incubator.apache.org - comm...@iceberg.incubator.apache.org - priv...@iceberg.incubator.apache.org The podling may also create a user mailing list, if needed. Source Control and Issue Tracking The Iceberg podling would use Apache’s gitbox integration to sync between github and Apache infrastructure. The podling would use github issues and pull requests for community engagement. Current Resources - Initial source: https://github.com/Netflix/iceberg - Java documentation: https://netflix.github.io/iceberg/current/javadoc/index.html?com/netflix/iceberg/package-summary.html - Table specification: https://docs.google.com/document/d/1Q-zL5lSCle6NEEdyfiYsXYzX_Q8Qf0ctMyGBKslOswA/edit Source and Intellectual Property Submission Plan The Iceberg source code in Github is currently licensed under Apache License v2.0 and the copyright is assigned to Netflix. If Iceberg becomes an Incubator project at the ASF, Netflix will transfer the source code and trademark ownership to the Apache Software Foundation via a Software Grant Agreement. External Dependencies External dependencies licensed under Apache License 2.0 - Guava https://github.com/google/guava - Jackson https://github.com/FasterXML/jackson-core - Joda-Time http://www.joda.org/joda-time/ External dependencies licensed under the MIT License - SLF4J https://www.slf4j.org/ - Mockito https://github.com/mockito/mockito ASF Projects - Apache Avro - Apache Hadoop - Apache Hive - Apache ORC - Apache Parquet - Apache Pig - Apache Spark Cryptography We do not expect Iceberg to be a controlled export item due to the use of encryption. Initial Committers and Affiliations - Ryan Blue b...@apache.org (Netflix) - Parth Brahmbhatt pa...@apache.org (Netflix) - Julien Le Dem jul...@apache.org (WeWork) - Owen O’Malley omal...@apache.org (Hortonworks) - Daniel Weeks dwe...@apache.org (Netflix) Sponsors and Nominated Mentors - Champion and mentor: Owen O’Malley omal...@apache.org - Mentor: Ryan Blue b...@apache.org - Mentor: Julien Le Dem jul...@apache.org Sponsoring Entity The Apache Incubator -- Ryan Blue
Re: [DISCUSS] Iceberg proposal for incubation
It sounds like there is consensus, so I'll start a vote thread. Thanks, everyone, for looking at the proposal! rb On Tue, Nov 13, 2018 at 1:27 AM Lars Francke wrote: > I've been following the Iceberg project on and off for a while now and I > think this is desperately needed. I don't claim to understand all of the > details but I'm very happy to hear this being proposed for Apache. I think > it'd be a great fit. Thank you Ryan and everyone else working on this so > far. > > The proposal looks good to me. > > Cheers, > Lars > > On Tue, Nov 13, 2018 at 4:01 AM Dave Fisher wrote: > >> Looks good to me. The challenge here is community development. With such >> mentor/committer engagement I trust that the Apache Way is covered! >> >> Regards, >> Dave >> >> Sent from my iPhone >> >> > On Nov 12, 2018, at 6:46 PM, Kevin A. McGrail >> wrote: >> > >> >> On 11/12/2018 3:17 PM, Ryan Blue wrote: >> >> Thanks for pointing that out, Dave. That section was accidentally not >> >> updated after I copied the markup template. I'll fix it. >> > Looks good to me. That's a lot of data to query! >> > >> > - >> > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >> > For additional commands, e-mail: general-h...@incubator.apache.org >> > >> >> >> - >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >> For additional commands, e-mail: general-h...@incubator.apache.org >> >> -- Ryan Blue Software Engineer Netflix
Re: [DISCUSS] Iceberg proposal for incubation
Thanks for pointing that out, Dave. That section was accidentally not updated after I copied the markup template. I'll fix it. On Mon, Nov 12, 2018 at 12:01 PM Dave Fisher wrote: > Hi - > > Looks very interesting. > > I have questions about the discrepancy between the Initial Committer list > of 5 people vs. the Affliations list with 16 people? > > Is it true that the podling plans to start with just 5 committers? > > Who from this group is affiliated with Netflix? > > Regards, > Dave > > > On Nov 12, 2018, at 11:51 AM, Ryan Blue wrote: > > > > Since it has been a week without substantial comments or changes to this > > proposal, I've posted it to the wiki as Bertrand suggested: > > https://wiki.apache.org/incubator/IcebergProposal > > > > Any further comments or discussion? If not, I can start a vote thread > > tomorrow. > > > > Thanks, everyone! > > > > rb > > > > On Tue, Nov 6, 2018 at 12:10 AM Bertrand Delacretaz < > > bdelacre...@codeconsult.ch> wrote: > > > >> Hi, > >> > >> On Mon, Nov 5, 2018 at 7:04 PM Ryan Blue > >> wrote: > >>> ... I'd like to start a discussion about moving Netflix's Iceberg > project > >> to > >>> the incubator... > >> > >> Looks like a very interesting project! > >> > >>> ...I can post it to the wiki as well if that's needed,... > >> > >> Yes please, it's good to have all proposals there - as soon as it's > >> past the draft stage. > >> > >> -Bertrand > >> > > > > > > -- > > Ryan Blue > > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > > -- Ryan Blue Software Engineer Netflix
Re: [DISCUSS] Iceberg proposal for incubation
Since it has been a week without substantial comments or changes to this proposal, I've posted it to the wiki as Bertrand suggested: https://wiki.apache.org/incubator/IcebergProposal Any further comments or discussion? If not, I can start a vote thread tomorrow. Thanks, everyone! rb On Tue, Nov 6, 2018 at 12:10 AM Bertrand Delacretaz < bdelacre...@codeconsult.ch> wrote: > Hi, > > On Mon, Nov 5, 2018 at 7:04 PM Ryan Blue > wrote: > >... I'd like to start a discussion about moving Netflix's Iceberg project > to > > the incubator... > > Looks like a very interesting project! > > >...I can post it to the wiki as well if that's needed,... > > Yes please, it's good to have all proposals there - as soon as it's > past the draft stage. > > -Bertrand > -- Ryan Blue
Re: [VOTE] Apache Toree 0.3.0-incubating (RC1)
+1 (binding, carried over from the podling vote) On Thu, Nov 8, 2018 at 7:54 PM Justin Mclean wrote: > Hi, > > +1 (binding) > > I checked: > - incubating in name (although I notice the pip package doesn’t - is this > a limitation?) > - signatures and hashes all good > - DISCLAIMER exists > - LICENSE and NOTICE file > - all source files have ASF headers > - No binary files in release > - Can compile from source > > Thanks, > Justin > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > > -- Ryan Blue Software Engineer Netflix
[DISCUSS] Iceberg proposal for incubation
d of 2 years or less, knowing that every Incubator project incurs a certain cost in terms of ASF infrastructure and volunteer time.Inexperience with Open SourceThree of the initial committers are Apache members and Incubator PMC members. They will work with the other community members to teach them the Apache Way.Homogenous DevelopersThe majority of the committers work at Netflix, though we are committed to recruiting and developing additional committers from a wide spectrum of industries and backgrounds.Reliance on Salaried DevelopersIt is expected that Iceberg development will occur on both salaried time and on volunteer time, after hours. Most of the initial committers are paid by Netflix to contribute to this project. However, they are all passionate about the project, and we are both confident and hopeful that the project will continue even if no salaried developers contribute to the project.Relationships with Other Apache ProductsAs mentioned in the Rationale section, Iceberg utilizes a number of existing Apache projects (Avro, Hadoop, Hive, ORC, Parquet, Pig, & Spark), and we expect that list to expand as the community grows and diversifies. Any Apache project in the big data space that needs to store or process tabular data would be potentially relevant.A Excessive Fascination with the Apache BrandWe are applying to the Incubator process because we think it is the next logical step for the Iceberg project after open-sourcing the code. This proposal is not for the purpose of generating publicity. Rather, we want to make sure to create a very inclusive and meritocratic community, outside the umbrella of a single company. Netflix has a long history of contributing to Apache projects and the Iceberg developers and contributors understand the implication of making it an Apache project.Required ResourcesMailing lists - d...@iceberg.incubator.apache.org - comm...@iceberg.incubator.apache.org - priv...@iceberg.incubator.apache.org The podling may also create a user mailing list, if needed.Source Control and Issue TrackingThe Iceberg podling would use Apache’s gitbox integration to sync between github and Apache infrastructure. The podling would use github issues and pull requests for community engagement.Current Resources - Initial source: github.com/Netflix/iceberg <https://github.com/Netflix/iceberg>- Java documentation <https://netflix.github.io/iceberg/current/javadoc/index.html?com/netflix/iceberg/package-summary.html>- Table specification <https://docs.google.com/document/d/1Q-zL5lSCle6NEEdyfiYsXYzX_Q8Qf0ctMyGBKslOswA/edit>Source and Intellectual Property Submission PlanThe Iceberg source code in Github is currently licensed under Apache License v2.0 and the copyright is assigned to Netflix. If Iceberg becomes an Incubator project at the ASF, Netflix will transfer the source code and trademark ownership to the Apache Software Foundation via a Software Grant Agreement.External DependenciesExternal dependencies licensed under Apache License 2.0 - Guava https://github.com/google/guava <https://github.com/google/guava>- Jackson https://github.com/FasterXML/jackson-core <https://github.com/FasterXML/jackson-core>- Joda-Time http://www.joda.org/joda-time/ <http://www.joda.org/joda-time/>External dependencies licensed under the MIT License - SLF4J https://www.slf4j.org/ <https://www.slf4j.org/>- Mockito https://github.com/mockito/mockito <https://github.com/mockito/mockito>ASF Projects - Apache Avro- Apache Hadoop- Apache Hive- Apache ORC- Apache Parquet- Apache Pig- Apache SparkCryptographyNot applicable.Initial Committers - Ryan Blue b...@apache.org - Parth Brahmbhatt pa...@apache.org - Julien Le Dem jul...@apache.org - Owen O’Malley omal...@apache.org - Daniel Weeks dwe...@netflix.com Sponsors - Champion and mentor: Owen O’Malley omal...@apache.org - Mentor: Ryan Blue b...@apache.org - Mentor: Julien Le Dem jul...@apache.org Sponsoring Entity - The Apache Incubator* -- Ryan Blue Software Engineer Netflix
Re: [IP CLEARANCE] Arrow Gandiva library
+1 On Mon, Sep 24, 2018 at 9:24 AM Matt Sicker wrote: > +1 > > On Fri, 21 Sep 2018 at 14:09, Wes McKinney wrote: > > > Apache Arrow is receiving a donation of the Gandiva library, an > > LLVM-based analytical expression compiler framework [1] > > > > Please vote to approve this contribution. > > > > This is a lazy consensus majority vote, per the IP clearance process > > [2], open for at least 72 hours. > > > > Wes > > > > [1]: http://incubator.apache.org/ip-clearance/arrow-gandiva.html > > [2] http://incubator.apache.org/ip-clearance/ > > > > - > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > -- > Matt Sicker > -- Ryan Blue Software Engineer Netflix
Re: Late Toree Podling Report
I added my sign-off on the report. Thanks for putting it together, Luciano! On Sun, Aug 12, 2018 at 4:36 PM Justin Mclean wrote: > Hi, > > Added and I assumed and marked it signed off by you. > > Thanks, > Justin > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > > -- Ryan Blue Software Engineer Netflix
Re: Looking for Champion
Okay, then let me rephrase: I would like to see a plan in the Palo proposal for a licensing scrub to be done before graduation. I'm still a little skeptical about this practice because the Incubator PMC validates the release on behalf of the foundation, but I think that's a separate issue to consider that doesn't need to distract on this Palo thread. Thanks for the explanation, Greg! On Mon, Jun 18, 2018 at 1:00 PM, Greg Stein wrote: > Heya Ryan, > > On Mon, Jun 18, 2018 at 2:39 PM Ryan Blue wrote: > >> > we have allowed (and IMO should continue) podlings to have licensing >> issues during their incubator releases >> >> Thanks for pointing this out, Greg. I wasn't aware of this and have >> always had releases fail when we discover licensing issues. I think there's >> a significant risk of license problems, so I had assumed we would require a >> thorough scrub before the first release. >> >> What's the argument for finishing this work before graduation rather than >> first release? Isn't the release a product for which the ASF is legally >> responsible? Given that we fail releases for known license issues, >> shouldn't we also be more careful when we know there are likely to be >> issues? >> > > This is why incubator releases have a disclaimer. It gives them time to > work through dependency and licensing issues, even while they're testing > their release process with our KEYS and distribution framework. So the > "argument" is simply to allow the podling to multitask, rather than gate > one of their activities. > > When you really want to lift the cover, there isn't a problem if a podling > releases (say) a hard LGPL dependency. That's just a policy choice of the > Foundation, to avoid such dependencies. We don't like it, and maybe some > messed up licensing downstream, possibly, for somebody to tease apart. But > historically, the Incubator has let these issues slide for a while, yet > gate on graduation. > > I also feel that podling releases are in a grey area, that don't truly > have the full backing of the ASF (thus the disclaimer, and them not being a > TLP; although technically the Apache Incubator is the stand-in PMC behind > the release). > > Cheers, > -g > > -- Ryan Blue Software Engineer Netflix
Re: Looking for Champion
> we have allowed (and IMO should continue) podlings to have licensing issues during their incubator releases Thanks for pointing this out, Greg. I wasn't aware of this and have always had releases fail when we discover licensing issues. I think there's a significant risk of license problems, so I had assumed we would require a thorough scrub before the first release. What's the argument for finishing this work before graduation rather than first release? Isn't the release a product for which the ASF is legally responsible? Given that we fail releases for known license issues, shouldn't we also be more careful when we know there are likely to be issues? rb On Mon, Jun 18, 2018 at 12:24 PM, Greg Stein wrote: > On Mon, Jun 18, 2018 at 2:08 PM Ryan Blue > wrote: > >... > >> 2. The license problems so far show that the project has not paid adequate >> attention to licensing up to now, which is a big risk. I'd like to see >> what >> kind of licensing scrub is proposed before the potential podling's first >> release. I don't think that catching all the obvious ones is sufficient. > > > To be clear: we have allowed (and IMO should continue) podlings to have > licensing issues during their incubator releases. For example, while > they're still dealing with Hibernate dependencies. It is understandable and > (IMO) acceptable that such releases will have problems. That is just part > of the process. As long as it gets cleaned up before graduation. > > Not diminishing the need for a good scrub, but I would not want to see > releases gated on that. (it's unclear from your text; maybe just a *plan* > rather than completion of the scrub?) > > Cheers, > -g > > -- Ryan Blue Software Engineer Netflix
Re: Looking for Champion
che > > Palo intends to do this. > > > > > > ###An Excessive Fascination with the Apache Brand > > > > Palo is proposing to enter incubation at Apache in order to help > > efforts > > > > to diversify the committer-base, not so much to capitalize on the > > Apache > > brand. The Palo project is in production use already inside Baidu, > > but is > > not expected to be an Baidu product for external customers. As such, > > the > > Palo project is not seeking to use the Apache brand as a marketing > > tool. > > > > > > ##Documentation > > > > Information about Palo can be found at > > https://github.com/baidu/palo. > > > > The following links provide more information about Palo in open > > source: > > > > > > * Palo wiki site: https://github.com/baidu/palo/wiki > > * Codebase at Github: https://github.com/baidu/palo > > * Issue Tracking: https://github.com/baidu/palo/issues > > * Overview: https://github.com/baidu/palo/wiki/Palo-Overview > > * FAQ: https://github.com/baidu/palo/wiki/Palo-FAQ > > > > ##Initial Source > > > > Palo has been under development since 2017 by a team of engineers at > > > > Baidu Inc. It is currently hosted on Github.com under an Apache > > license at > > https://github.com/baidu/palo. > > > > > > ##External Dependencies > > > > Palo has the following external dependencies. > > > > * Google gflags (BSD) > > * Google glog (BSD) > > * Apache Thrift (Apache Software License v2.0) > > * Apache Commons (Apache Software License v2.0) > > * Boost (Boost Software License) > > * OpenLdap (OpenLDAP Software License) > > * rapidjson (Tencent) > > * Google RE2 (BSD-style) > > * lz4 (BSD) > > * snappy (BSD) > > * cyrus-sasl (CMU License) > > * Twitter Bootstrap (Apache Software License v2.0) > > * d3 (BSD) > > * LLVM (BSD-like) > > > > Build and test dependencies: > > > > * ant (Apache Software License v2.0) > > * Apache Maven (Apache Software License v2.0) > > * cmake (BSD) > > * clang (BSD) > > * Google gtest (Apache Software License v2.0) > > > > ##Required Resources > > > > ###Mailing List > > > > There are currently no mailing lists. The usual mailing lists are > > > > expected to be set up when entering incubation: > > > > > > priv...@palo.incubator.apache.org<mailto:private@palo. > > > > incubator.apache.org> > > > > d...@palo.incubator.apache.org<mailto:d...@palo.incubator.apache.org> > > comm...@palo.incubator.apache.org<mailto:commits@palo. > > > > incubator.apache.org> > > > > > > ###Subversion Directory > > > > Upon entering incubation: https://github.com/baidu/palo. > > After incubation, we want to move the existing repo from > > > > https://github.com/baidu/palo to Apache infrastructure. > > > > > > ###Issue Tracking > > > > Palo currently uses GitHub to track issues. Would like to continue > > to do > > > > so while we discuss migration possibilities with the ASF Infra > > committee. > > > > > > ###Other Resources > > > > The existing code already has unit tests so we will make use of > > existing > > > > Apache continuous testing infrastructure. The resulting load should > > not be > > very large. > > > > > > ##Initial Committers > > > > * Ruyue Ma (https://github.com/maruyue, > > maru...@baidu.com<mailto:maruy > > > > u...@baidu.com>) > > > > * Chun Zhao (https://github.com/imay, buaa.zh...@gmail.com > > > bu > > > > aa.zh...@gmail.com>) > > > > * Mingyu Chen (https://github.com/morningman,chenmin...@baidu.com) > > * De Li（https://github.com/lide-reed, mailtol...@sina.com） > > > ma > > > > iltol...@sina.com%EF%BC%89> > > > > * Hao Chen (https://github.com/chenhao7253886, chenha...@baidu.com > > > > <mailto:chenha...@baidu.com>) > > > > * Chaoyong Li (https://github.com/cyongli, > > lichaoy...@baidu.com > > > lichaoy...@baidu.com>) > > > > * Bin Lin (https://github.com/lingbin, > > lingbi...@gmail.com<mailto:lin > > > > gbi...@gmail.com>) > > > > > > ##Affiliations > > > > The initial committers are employees of Baidu Inc.. The nominated > > > > mentors are employees of TODO. > > > > > > ##Sponsors > > > > ###Champion > > > > TODO > > > > ###Nominated Mentors > > > > * sijie guo, guosi...@gmail.com<mailto:guosi...@gmail.com> > > * Luke Han, luke...@apache.org<mailto:luke...@apache.org> > > * Zheng Shao, zs...@apache.org<mailto:zs...@apache.org> > > > > > > Mentors must be members of the IPMC and almost always Members of the > > ASF. > > > > At this moment only Luke Han is qualified. > > > > Regards, > > Dave > > > > > > ###Sponsoring Entity > > > > We are requesting the Incubator to sponsor this project. > > > > > > > > ?B婯 > > > > > > KKKCB??[ > > > > 溳 > > X溫軞X橩??K[XZ[??賉橽榌 ][溳X溫軞X橮?[樰X榏?軏榎?X?K涇櫭B憶軋?Y??]?[蹣[?? > > > > 圹[X[???K[XZ[??賉橽榌 > > > > Z?[???[樰X榏?軏榎?X?K涇櫭B > > > > > > > > ?B� > > > > CB� > > > > ?�?[��X��ܚX�K??K[XZ[?�?�[�\�[?][��X��ܚX�P?[��X�]?܋�\?X�?K� > > > > ܙ�B��܈?Y??]?[ۘ[? > > > > ?��[X[�?�??K[XZ[?�?�[�\�[?Z?[???[��X�]?܋�\?X�?K�ܙ�B > > > > > > > > > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > > -- Ryan Blue Software Engineer Netflix
Re: [VOTE] Apache Toree 0.2.0-incubating (RC4)
How did you run tests? It worked fine from source for me with `make test` On Fri, Apr 13, 2018 at 1:12 PM, Matt Sicker <boa...@gmail.com> wrote: > I built it from the source tar.gz. > > On 13 April 2018 at 14:05, Luciano Resende <luckbr1...@gmail.com> wrote: > > > On Wed, Apr 11, 2018 at 9:44 AM, Matt Sicker <boa...@gmail.com> wrote: > > > > > * Signatures ok > > > * License and disclaimer are fine > > > * Notice included, but the copyright year needs to be updated to > > 2016-2018 > > > * Rat check ok <https://github.com/jvz/sbt-rat> > > > * I got a test failure locally for > > > integration.interpreter.scala.AddExternalJarMagicSpecForIntegration > > > "should > > > be able to load Java jars". > > > > > > > > Sory for the delay replying, I finally got time to give this a try, and I > > build from a clean > > source distribution a few times successfully. I am trying this on a mac > os > > environment, > > could you please describe your env, and if it works on a second run (note > > that this is > > probably the one test that validates adding something from external > source > > so a download > > is involved and internet connectivity might play a part here) > > > > Thanks > > > > -- > > Luciano Resende > > http://twitter.com/lresende1975 > > http://lresende.blogspot.com/ > > > > > > -- > Matt Sicker <boa...@gmail.com> > -- Ryan Blue Software Engineer Netflix
Re: [VOTE] Apache Toree 0.2.0-incubating (RC4)
For the .md5 file, we can simply not publish that file when copying the RC files. Please don't let that prevent you from checking this release! On Wed, Apr 11, 2018 at 8:41 AM, Luciano Resende <luckbr1...@gmail.com> wrote: > Here is my binding +1, any volunteers to help review the RC, we need one > more IPMC vote. > > On Sat, Apr 7, 2018 at 6:28 PM, Luciano Resende <luckbr1...@gmail.com> > wrote: > > > Please vote to approve the release of Apache Toree 0.2.0-incubating > > (RC4). > > > > The podling dev vote thread: > > https://www.mail-archive.com/dev@toree.incubator.apache. > org/msg01740.html > > > > And the result: > > https://www.mail-archive.com/dev@toree.incubator.apache. > org/msg01748.html > > > > Tag: v0.2.0-incubating-rc4 (8849271e28a4ea4e77c57ef8fb3da77b73469799) > > > > https://github.com/apache/incubator-toree/tree/v0.2.0-incubating-rc4 > > > > All distribution packages, including signatures, digests, etc. can be > > found at: > > > > https://dist.apache.org/repos/dist/dev/incubator/toree/0.2.0 > > -incubating-rc4/ > > > > Staging artifacts can be found at: > > > > https://repository.apache.org/content/repositories/orgapachetoree-1011 > > > > The vote is open for at least 72 hours and passes if a majority of at > > least 3 +1 PMC votes are cast. > > > > [ ] +1 Release this package as Apache Toree 0.2.0-incubating > > [ ] -1 Do not release this package because ... > > > > > > -- > > Luciano Resende > > http://twitter.com/lresende1975 > > http://lresende.blogspot.com/ > > > > > > -- > Luciano Resende > http://twitter.com/lresende1975 > http://lresende.blogspot.com/ > -- Ryan Blue Software Engineer Netflix
[jira] [Commented] (INCUBATOR-210) Servicecomb当前性能统计不支持时延分布的统计，这个对于业务有重要意义
[ https://issues.apache.org/jira/browse/INCUBATOR-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378985#comment-16378985 ] Ryan Blue commented on INCUBATOR-210: - I thought it was, but maybe not. I didn't realize ServiceComb was an incubating project. > Servicecomb当前性能统计不支持时延分布的统计，这个对于业务有重要意义 > --- > > Key: INCUBATOR-210 > URL: https://issues.apache.org/jira/browse/INCUBATOR-210 > Project: Incubator > Issue Type: Task >Reporter: renyingxin >Priority: Major > > Servicecomb当前性能统计不支持时延分布的统计，这个对于业务有重要意义,也就是类似[0,200),[200,400）这种时延分布的统计，平均时延可能会掩盖系统的部分问题，比如部分高延时，但是被正常平均值掩盖了 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
[jira] [Commented] (INCUBATOR-210) Servicecomb当前性能统计不支持时延分布的统计，这个对于业务有重要意义
[ https://issues.apache.org/jira/browse/INCUBATOR-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378959#comment-16378959 ] Ryan Blue commented on INCUBATOR-210: - [~vorburger], you can open an INFRA ticket for that. > Servicecomb当前性能统计不支持时延分布的统计，这个对于业务有重要意义 > --- > > Key: INCUBATOR-210 > URL: https://issues.apache.org/jira/browse/INCUBATOR-210 > Project: Incubator > Issue Type: Task >Reporter: renyingxin >Priority: Major > > Servicecomb当前性能统计不支持时延分布的统计，这个对于业务有重要意义,也就是类似[0,200),[200,400）这种时延分布的统计，平均时延可能会掩盖系统的部分问题，比如部分高延时，但是被正常平均值掩盖了 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Accept Druid into the Apache Incubator
ike to continue using GitHub, if possible, in order to > > preserve the workflows the community has developed around > > GitHub pull requests. > > > > === Issue tracking === > > > > Druid currently uses GitHub issues for issue tracking. We > > would like to migrate to Apache JIRA at > > http://issues.apache.org/jira/browse/DRUID. > > > > == Documentation == > > > > Druid's documentation can be found at > > http://druid.io/docs/latest/. > > > > == Initial Source == > > > > Druid was initially open-sourced by Metamarkets in 2012 and > > has been run in a community-governed fashion since then. The > > code is currently hosted at https://github.com/druid-io/ and > > includes the following repositories: > > > > * druid (primary repository) > > * druid-console (web console for Druid) > > * druid-io.github.io (source for Druid's website at > > http://druid.io/) > > * tranquility (realtime stream push client for Druid) > > * docker-druid (Docker image for Druid) > > * pydruid (Python library) > > * RDruid (R library) > > * oss-parent (Maven POM files) > > > > == Source and Intellectual Property Submission Plan == > > > > A complete set of the open source code needs to be licensed > > from the owning organization to the Foundation. Commercial > > legal counsel for the owning organization will review the > > standard Foundation licensing paperwork and propose any > > updates as needed. This license will enable Apache to > > incubate and manage the Druid project moving forward. > > > > Other Druid paraphernalia to be transferred to Apache > > consists of: > > > > * GitHub organization at https://github.com/druid-io/ > > * Twitter account at https://twitter.com/druidio > > * "druid.io" domain name > > * "Druid" trademark assignment per Foundation standard > > paper. The trademark assignment paperwork shall be > > reviewed by the owning organization's commercial and IP > > counsel > > * CLAs - all rights in the code licensed above should > > encompass the CLAs that existed between developers and > > owning organization > > > > A copyright license to the code, trademark assignment of > > Druid, and transfer of other paraphernalia to Apache should > > be sufficient to cover all rights required by Apache to > > operate the project. > > > > == External Dependencies == > > > > External dependencies distributed with Druid currently all > > have one of the following Category A or B licenses: ASL, > > BSD, CDDL, EPL, MIT, MPL; with one exception: the optional > > Druid MySQL metadata store extension depends on MySQL > > Connector/J, which is GPL licensed. Druid currently packages > > this as a separate download; see our current presentation > > on: http://druid.io/downloads.html. As part of incubation we > > intend to determine the best strategy for handling the MySQL > > extension. > > > > == Cryptography == > > > > Not applicable. > > > > == Initial Committers == > > > > The initial committers for incubation are the current set of > > committers on Druid who have expressed interest in being > > involved in Apache incubation. Affiliations are listed > > where relevant. We may seek to add other committers during > > incubation; for example, we would want to add any current > > Druid committers who express an interest after incubation > > begins. > > > > * Charles Allen (char...@allen-net.com) (Snap) > > * David Lim (david.clarence@gmail.com) (Imply) > > * Eric Tschetter (ched...@apache.org) (Splunk) > > * Fangjin Yang (f...@imply.io) (Imply) > > * Gian Merlino (g...@apache.org) (Imply) > > * Himanshu Gupta (g.himan...@gmail.com) (Oath) > > * Jihoon Son (jihoon...@apache.org) (Imply) > > * Jonathan Wei (jon@imply.io) (Imply) > > * Maxime Beauchemin (maximebeauche...@gmail.com) (Lyft) > > * Mohamed Slim Bouguerra (slim.bougue...@gmail.com) (Hortonworks) > > * Nishant Bangarwa (nish...@apache.org) (Hortonworks) > > * Parag Jain (paragjai...@gmail.com) (Oath) > > * Roman Leventov (leventov...@gmail.com) (Metamarkets) > > * Xavier Léauté (xav...@leaute.com) (Confluent) > > > > == Sponsors == > > > > * Champion: Julian Hyde > > * Nominated mentors: Julian Hyde, P. Taylor Goetz, Jun Rao > > * Sponsoring entity: Apache Incubator > > > > - > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > -- > thanks > ashish > > Blog: http://www.ashishpaliwal.com/blog > My Photo Galleries: http://www.pbase.com/ashishpaliwal > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > > -- Ryan Blue Software Engineer Netflix
Re: [VOTE] Apache Toree 0.2.0-incubating (RC3)
Justin, A bit more context on those Jars: these were created to test the ability to add jars at runtime to the Scala interpreter, and they were contributed to the project as tests. I'm not sure there's a strong case to build these Jars just before using them in tests. That would require much more time and maintenance than just including the binaries. There's nothing special about them besides that they have classes that can be loaded to verify the classpath addition worked. I think this is similar enough to having binary test files included in file format projects that it should not be a reason to fail the release. rb On Sun, Jan 21, 2018 at 10:39 PM, Justin Mclean <justinmcl...@me.com> wrote: > Hi, > > -1 (binding) as there is compiled source in the release > > I checked: > - incubating in name > - signatures and hashes correct > - disclaimer exists > - LICENSE is good > - NOTICE needs year updating > - Unexpected jars in source release. > - can compile from source > > The 3 jars are: > toree-0.2.0-incubating-src/scala-interpreter/src/test/ > resources/ScalaTestJar.jar > toree-0.2.0-incubating-src/scala-interpreter/src/test/ > resources/TestJar.jar > toree-0.2.0-incubating-src/scala-interpreter/src/test/ > resources/TestJar2.jar > > Could these be compiled as part of the build process? It currently doesn’t > look like the source code for those files are included at all but I assume > you know where they come from? > > Thanks, > Justin > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > > -- Ryan Blue Software Engineer Netflix
Re: [VOTE] Apache Toree 0.2.0-incubating (RC1)
-1 (binding) There are a few things that could be improved, but my -1 is because the release tarball doesn’t match the release tag (inclusion of .example-image), some files are missing the license header, and there is no mention in the license file of mesos-protobuf that’s included in the tarball. rb Here are my other notes: The .sha file has a sha512 checksum, which should be in a .sha512 file. Also (but minor), both .md5 and .sha files have a full path instead of a relative path: /Users/lresende/opensource/jupyter/incubator-toree-apache/dist/toree-src/toree-0.2.0-incubating-src.tar.gz I had to import the key using gpg --recv-keys EFB55DF1. Is there a KEYS file published for Toree? The tarball currently unpacks into the current directory, which is unusual for source tarballs. RAT checks fail for some files. Here’s the summary: !? .jvmopts !? .example-image !? sparkr-interpreter/src/main/resources/README.md !? index.ipynb !? README.md !? RELEASE_NOTES.md !? etc/pip_install/MANIFEST.in !? etc/.src-release-ignore We should have license headers in the .md files, and the release process should ideally use git archive to avoid picking up files from the local working directory that aren’t part of the tagged release. This also distributes a few Jars: Archives: + scala-interpreter/src/test/resources/TestJar2.jar + scala-interpreter/src/test/resources/ScalaTestJar.jar + scala-interpreter/src/test/resources/TestJar.jar + kernel/lib/mesos-0.18.1-shaded-protobuf.jar I think the test Jars are fine, but LICENSE and NOTICE don’t mention distributing mesos-protobuf. On Wed, Nov 1, 2017 at 6:37 AM, Atri Sharma <atri.j...@gmail.com> wrote: > +1 > -- Checked Headers > -- Checked License > -- Checked DISCLAIMER and Incubator policies > > On Thu, Oct 26, 2017 at 9:37 PM, Luciano Resende <luckbr1...@gmail.com> > wrote: > > Please vote to approve the release of Apache Toree 0.2.0-incubating > (RC1). > > > > The PPM vote thread: > > https://www.mail-archive.com/dev@toree.incubator.apache. > org/msg01527.html > > > > And the result: > > https://www.mail-archive.com/dev@toree.incubator.apache. > org/msg01539.html > > > > Tag: v0.2.0-incubating-rc1 (01cd97e9bad04878a8014016c154a50e2a00f21d) > > > > https://github.com/apache/incubator-toree/tree/v0.2.0-incubating-rc1 > > > > All distribution packages, including signatures, digests, etc. can be > found > > at: > > > > https://dist.apache.org/repos/dist/dev/incubator/toree/0.2. > 0-incubating-rc1/ > > > > Staging artifacts can be found at: > > > > https://repository.apache.org/content/repositories/orgapachetoree-1007 > > > > The vote is open for at least 72 hours and passes if a majority of at > least > > 3 +1 PMC votes are cast. > > > > [ ] +1 Release this package as Apache Toree 0.2.0-incubating > > [ ] -1 Do not release this package because ... > > > > -- > > Luciano Resende > > http://twitter.com/lresende1975 > > http://lresende.blogspot.com/ > > > > -- > Regards, > > Atri > l'apprenant > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > > -- Ryan Blue Software Engineer Netflix
Editing reports page
Can someone grant me permission to edit the incubator reports? I'd like to submit one for Toree but I don't have permission. My user is RyanBlue. Thanks! -- Ryan Blue
Re: [VOTE] Accept Fineract into Apache Incubator
ject. The platform is built on top of the Spring Framework, utilizing multiple modules, like Spring Data, Spring Web, or Spring Security. === An Excessive Fascination with the Apache Brand === While we respect the reputation of the Apache brand we see a benefit for both sides. We bring a stable and mature community with a deep rooted social mission and are able to give Fineract a solid home as an open source project following an established development model. == Documentation == Information about Mifos X can be found at https://mifosforge.jira.com/wiki/display/MIFOSX/Welcome+to+Mifos+X. == Initial Source == Initial source is available on GitHub under the MPLv2 at https://github.com/openMF/mifosx. == Source and Intellectual Property Submission Plan == The Mifos Initiative owns all copyright to all contributions and has the legal right to sign the ASF Software Grant Agreement. As soon as Fineract is approved to join Apache Incubator, the source code will be transitioned via the Software Grant Agreement onto ASF infrastructure and in turn made available under the Apache License, version 2.0. == External Dependencies == || '''Vendor''' || '''Library''' || '''License''' || '''Compatible''' || || Amazon.com, Inc. || AWS Java SDK || AL v2 || yes || || Apache Software Foundation || Commons || AL v2 || yes || || Apache Software Foundation || Tomcat || AL v2 || yes || || Apache Software Foundation || POI || AL v2 || yes || || Apache Software Foundation || HTTPClient || AL v2 || yes || || Ben Fortuna || iCal4j || BSD 3-clause || yes || || Boxfuse GmbH || Flyway DB || AL v2 || yes || || Bruce Dunwiddie^1^ || JavaCSV || LGPL v2.1 || no || || Google, Inc. || gson || AL v2 || yes || || Google, Inc. || guava || AL v2 || yes || || iText Group NV || iText || MPL v1.1 || yes || || Jayway || JsonPath || AL v2 || yes || || Jayway || RESTassured || AL v2 || yes || || JUnit || JUnit || EPL v1 || yes || || Michael Vorburger || MariaDB4J || AL v2 || yes || || Oracle, Corp. || JAX-RS || CDDL v1 || yes || || Oracle, Corp. || Jersey || CDDL v1.1 || yes || || Oracle, Corp.^2^ || MySQL Connector/J || GPL v2 || no || || Pentaho, Corp.^3^ || Reporting Engine || LGPL v2.1 || no || || Pentaho, Corp.^3^ || Reporting Library || LGPL v2.1 || no || || Pivotal Software, Inc. || Spring Boot || AL v2 || yes || || Pivotal Software, Inc. || Spring Framework || AL v2 || yes || || Pivotal Software, Inc. || Spring Security || AL v2 || yes || || Sam Pullar || Mustache || AL v2 || yes || || Square, Inc. || retrofit || AL v2 || yes || || Square, Inc. || okhttp || AL v2 || yes || || Stephen Colebourne || Joda-Time || AL v2 || yes || || Szczepan Faber || Mockito || MIT || yes || || Terracotta, Inc || Quartz || AL v2 || yes || || Terracotta, Inc || Ehache || AL v2 || yes || * ^1^ = can be removed * ^2^ = can be replaced * ^3^ = need an abstraction on our side to become replaceable == Cryptography == The only cryptography included by the project will be via library inclusion, and will be used to encrypt stored user data on mobile devices and in cloud storages. == Required Resources == === Mailing lists === * priv...@fineract.incubator.apache.org (moderated subscriptions) * comm...@fineract.incubator.apache.org * d...@fineract.incubator.apache.org * u...@fineract.incubator.apache.org === Git Repository === https://git-wip-us.apache.org/repos/asf/incubator-fineract.git === Issue Tracking === JIRA Project Fineract (FINERACT) === Other Resources === * Project website (http://fineract.incubator.apache.org) * Fineract Wiki pages (https://cwiki.apache.org/confluence/display/FINERACT) == Initial Committers == ''The following list comprises the current long term committers and does not contain occasional developers.'' * Vishwas Babu AJ (vishwas at confluxtechnologies dot com) * Edward Cable (edcable at mifos dot org) * Andrew Dzakpasu (andrewdzakpasu at musoni dot eu) * Markus Geiss (mgeiss at mifos dot org) * Sander van der Heijden (sander at musoni dot eu) * Ishan Khanna (ishan1604 at gmail dot com) * Myrle Krantz (mkrantz at mifos dot org) * Terence Monteiro (terence at sanjosesolutions dot in) * Adi Nayaran Raju (adi dot raju at confluxtechnologies dot com) * Gaurav Saini (gsaini at apache dot org) * Nazeer Hussain Shaik (nazeer dot shaik at confluxtechnologies dot com) * Michael Vorburger (mike at vorburger dot ch) == Affiliations == * Vishwas Babu AJ (Conflux Technologies) * Ed Cable (The Mifos Initiative) * Andrew Dzakpasu (Musoni Systems) * Markus Geiss (The Mifos Initiative) * Sander van der Heijden (Musoni Systems) * Myrle Krantz (The Mifos Initiative) * Terence Monteiro (SanJose Foundation) * Adi Nayaran Raju (Conflux Technologies) * Nazeer Hussain Shaik (Conflux Technologies) == Sponsors == === Champion === Ross Gardler === Nominated Mentors === * Ross Gardler * Roman Shaposhnik * Greg Stein === Sponsoring Entity === Incubator PMC -- Ryan Blue Software Engineer Cloudera,
Re: [VOTE] Graduate Apache NiFi from the Incubator
+1 (non-binding) On 06/16/2015 09:05 PM, Joe Witt wrote: Hello Apache Incubator This thread is to call a vote within the Incubator for the graduation of Apache NiFi. Status page: http://incubator.apache.org/projects/nifi.html Incubator graduation discussion: http://s.apache.org/wmy NiFi community graduation vote: http://s.apache.org/GT9 The discussion prompted Benson to join the PMC list to provide apache member experience and we clarified the scope language with discussion in the NiFi community. From the discussion thread Chris Mattmann offered his binding +1 vote. Since joining the Incubator in November, 2014 the NiFi community has: * Produced three IPMC approved releases. * Expanded the PPMC with 4 new members. * Grown the community. Developed increased interoperability with other Apache projects. * Conducted a successful community vote to graduate with 22 +1 votes (5 IPMC binding). Please vote [ ] +1 Graduate Apache NiFi as a TLP [ ] +0 [ ] -1 Do not graduate Apache NiFi as a TLP because… Voting will last 72 hours ending at 05:00 June 20, 2015 UTC. Thanks Joe === Board Resolution === Establish the Apache NiFi Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software, for distribution at no charge to the public, related to an automated and durable data broker between systems providing interactive command and control and detailed chain of custody for data. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache NiFi Project, be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache NiFi Project be and hereby is responsible for the creation and maintenance of software related to an automated and durable data broker between systems providing interactive command and control and detailed chain of custody for data; and be it further RESOLVED, that the office of Vice President, Apache NiFi be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache NiFi Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache NiFi Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache NiFi Project: * Brandon DeVries (devri...@apache.org) * Matt Gilman (mcgil...@apache.org) * Tony Kurc (tk...@apache.org) * Mark Payne (marka...@apache.org) * Aldrin Piri (ald...@apache.org) * Dan Bress (danbr...@apache.org) * Jennifer Barnabee (jbarna...@apache.org) * Joseph L. Witt (joew...@apache.org) * Jason Carey (jca...@apache.org) * Adam Taft (tafts...@apache.org) * Benson Margulies (bimargul...@apache.org) * Bryan Bende (bbe...@apache.org) NOW, THEREFORE, BE IT FURTHER RESOLVED, that Joseph L. Witt be appointed to the office of Vice President, Apache NiFi, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed; and be it further RESOLVED, that the initial Apache NiFi PMC be and hereby is tasked with the creation of a set of bylaws intended to encourage open development and increased participation in the Apache NiFi Project; and be it further RESOLVED, that the Apache NiFi Project be and hereby is tasked with the migration and rationalization of the Apache Incubator NiFi podling; and be it further RESOLVED, that all responsibilities pertaining to the Apache Incubator NiFi podling encumbered upon the Apache Incubator Project are hereafter discharged. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- Ryan Blue - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Accept Trafodion into Apache Incubator
with capabilities more geared towards database administrators. There are alternative open source projects that are providing SQL-on-Hadoop capabilities, such as Apache Hive, Apache Drill, and Apache Phoenix. These are more focused on reporting and analytics across data structures supported on HDFS. In comparison to all of these technologies Trafodion provides a very complete implementation of ANSI SQL, one of the most sophisticated optimizers for such workloads, a completely parallel data flow architecture that does not materialize intermediate results unless necessary, full ACID transactional support, ANSI GRANT/REVOKE security, and other capabilities that would take decades to build in these products. On the other hand currently Trafodion is just focused on HBase and querying Hive, whereas Hive and Drill provide access to other data formats in HDFS. An Excessive Fascination with the Apache Brand We understand the reputation and value of the Apache brand, and no doubt believe that it will help us attract contributors and users. Our primary goal is to follow a proven, open source development and community building model that will make Trafodion successful and enable better collaboration with other Apache projects in the Hadoop ecosystem. We also understand the rules and guidelines about the use of the Apache brand and intend to follow them. Documentation Documentation and technical details on Trafodion can be found at: http://www.trafodion.org/ Initial Source The source is available today in a public github repository: https://github.com/trafodion/trafodion. Source and Intellectual Property Submission Plan The source code has already been released under the Apache License, Version 2. The manuals have been released in Adobe PDF format. As part of the submission process, the source for the manuals will be converted from a proprietary DocBook XML format to AsciiDoc. External Dependencies Two dependencies do not have Apache compatible licenses and will be addressed as we enter incubation. One dependency is log4cpp, which is licensed under the LGPL. A compatible alternative might be Apache incubator project log4cxx. The other dependency is unixodbc, which is used as the ODBC driver manager. We will look into how Apache Hive manages being able to use this incompatible software and do similar. All other dependencies have Apache compatible licenses, including Apache 2.0, MIT/X11, MIT, and BSD. Cryptography Trafodion does not contain any cryptographic code. It does call cryptographic libraries: OpenSSL for C++ code and Java Cryptography Extension (JCE) for Java code. Required Resources Mailing Lists priv...@trafodion.incubator.apache.org d...@trafodion.incubator.apache.org comm...@trafodion.incubator.apache.org Git Repository https://git-wip-us.apache.org/repos/afs/incubator-trafodion.git Issue Tracking JIRA: JIRA Trafodion (Trafodion) Initial Committers and Affiliation Dave Birdsall, Hewlett-Packard Company, Dave.BirdsallAThpDOTcom Matt Brown, Hewlett-Packard Company, mattbrownAThpDOTcom Tharak Capirala, Hewlett-Packard Company, Tharak.CapiralaAThpDOTcom Alice Chen, Hewlett-Packard Company, Alice.ChenAThpDOTcom John DeRoo, Hewlett-Packard Company, John.DerooAThpDOTcom Roberta Marton, Hewlett-Packard Company, Roberta.MartonAThpDOTcom Amanda Moran, Hewlett-Packard Company, Amanda.Kay.MoranAThpDOTcom Suresh Subbiah, Hewlett-Packard Company, Suresh.SubbiahAThpDOTcom Sandyha Sundaresan, Hewlett-Packard Company, Sandhya.SundaresanAThpDOTcom Sponsors Champion Michael Stack, StackATapacheDOTorg Nominated Mentors Andrew Purtell apurtellATapacheDOTorg Devaraj Das, ddasATapacheDOTor Enis Söztutar, EnisATapacheDOTorg Lars Hofhansl, larshATapacheDOTorg Michael Stack, StackATapacheDOTorg Roman Shaposhnik, rshaposhnikATpivotalDOTio Sponsoring Entity Apache Incubator PMC - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- Ryan Blue Software Engineer Cloudera, Inc. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: You know what... Apache is just too complicated.
On 05/18/2015 11:35 AM, Stefan Reich wrote: Lots of rules, Americans in the background... I don't see that it works. Why don't we spend our time just PRODUCING SOMETHING? All you'd have to do is connect programmers to projects. Simple. Why all the rules? Cheers Stefan Hi Stefan, In my experience, the code is the easy part of a project, while building and maintaining a community around it is the hard part. That's what Apache focuses on: building a healthy community around a project so that it lives beyond a single developer. That's quite a bit more than connecting programmers with projects. For example, what happens when the developers disagree? How do we disagree politely and move on? Who says we *should* disagree politely and move on? Apache's answer to this is what is behind the voting and procedure. In addition, there are a number of concerns around ownership and responsibility. There are a lot of rules concerning licensing that are in place to protect contributors and those require some diligence and, unfortunately, rules. rb -- Ryan Blue - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Release Apache Parquet MR (Incubating) 1.6.0 RC2
On 04/16/2015 02:08 PM, Justin Mclean wrote:
Hi,
I'm curious about the problems you had with the install. As far as I know, the
build doesn't depend on PHP. Maybe this is something I had installed by
default, but could you tell us what error you ran into?
It with installing and compiling thrift rather than parquet.
First I got:
/Users/justinmclean/Downloads/ApacheParquet/apache-parquet-mr-1.6.0-incubating/thrift-0.7.0/install-sh:
Permission denied
When was it trying to run install-sh? During ./configure?
Easily fixed via a chmod +x, then I got:
/Users/justinmclean/Downloads/ApacheParquet/apache-parquet-mr-1.6.0-incubating/thrift-0.7.0/lib/php/src/ext/thrift_protocol/php_thrift_protocol.cpp:95:8:
error:
unknown type name 'function_entry'
static function_entry thrift_protocol_functions[] = {
The error is related to the PHP version as far as I can tell. I tried
installing the latest PHP version but had trouble pointing the build script at
it and at that point I gave up.
Sounds like we need to add --without-php to the list of configure flags.
Thanks,
Justin
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org
rb
--
Ryan Blue
Software Engineer
Cloudera, Inc.
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org
[RESULT] [VOTE] Release Apache Parquet MR (Incubating) 1.6.0 RC2
This vote passes, with 3 IPMC +1 votes and no -1 or 0 votes (2 IPMC and 4 PPMC +1s from the project vote). Thanks to everyone that had a look and voted! rb On 04/12/2015 09:04 AM, Ryan Blue wrote: Hi everyone, I propose the following RC to be released as official Apache Parquet 1.6.0 release. This RC passed the vote in the Parquet community with 2 binding IPMC votes. The thread is here: * http://s.apache.org/parquet-1.6.0-ppmc-vote This is the first Apache release of Parquet MR. Like the 2.2.0 release of Parquet Format, this uses the old parquet.* package names and will be immediately followed by a 1.7.0 release that is strictly a move to org.apache.parquet.* packages and artifacts. The commit id is 4f660778f89d4164c72676f6b15bbc74c0d09373. * http://s.apache.org/apache-parquet-mr-1.6.0-incubating-rc2 * https://github.com/apache/incubator-parquet-mr/tree/4f660778 The release tarball, signature, and checksums are here: * https://dist.apache.org/repos/dist/dev/incubator/parquet/apache-parquet-mr-1.6.0-incubating-rc2/ You can find the KEYS file here: * https://dist.apache.org/repos/dist/dev/incubator/parquet/KEYS Full changes for the release can be found here: * https://github.com/apache/incubator-parquet-mr/blob/4f660778/CHANGES.md#version-160 Please download, verify, and test. Building the project requires both protoc and thrift compilers. We recommend versions 2.5.0 and 0.7.0 and there are build instructions in the README. This vote will close in 3 days. [ ] +1 Release this as Apache Parquet Format 1.6.0 [ ] +0 [ ] -1 Do not release this because... -- Ryan Blue Software Engineer Cloudera, Inc. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Graduate Apache Parquet from the Incubator
Thanks for linking to the MR release thread, Jake! This is a good opportunity to remind people that the Parquet MR release vote is on-going and could use a little attention, too. Thanks! rb On 04/14/2015 06:01 AM, Jake Farrell wrote: correction, mr vote is all ready on the IPMC list for vote since passing the PPMC vote http://mail-archives.apache.org/mod_mbox/incubator-general/201504.mbox/%3C552A9797.5070100%40apache.org%3E -Jake On Tue, Apr 14, 2015 at 8:57 AM, Jake Farrell jfarr...@apache.org wrote: Hey Ted The 21 Parquet PPMC members listed on the proposal come from 8 different companies, the new committers added do not all come from the same companies. There are two different components to Parquet, format and mr. Format has been successfully released and mr has just passed a PPMC on Apr 12 and is ready for an IPMC vote, not sure why this was not started yet. The sync up meetings are currently conducted over google hangouts, and notes brought back to the dev list. No official decisions are being carried out without being put to the list and anyone is welcome to attend. -Jake On Tue, Apr 14, 2015 at 3:08 AM, Ted Dunning ted.dunn...@gmail.com wrote: Could we have some discussion on this first? Jumping to a vote may be a bit premature (or not ... thus the need to discuss). I have some questions myself that would prevent me from being able to support the issue right now. Presumably these can all be resolved, but it is hard to do so under a 72 hour clock, especially since I am in an inconvenient time zone relative to some. For instance, this project started with limited diversity. Have the additions to the committership improved that or not? The last incubator report had some language about parquet components that had not seen a release yet. Any word on that? In reviewing the dev list, I see evidence of a sync'up meeting that apparently occurs off-list where it seems decisions about release scope are made. Any comments about that in terms of Apache policy? On Mon, Apr 13, 2015 at 5:46 PM, Jake Farrell jfarr...@apache.org wrote: This thread is to start a vote on the graduation resolution Apache Parquet has approved on the dev list with 14 positive votes (11 PPMC (4 from IPMC members), 3 community) Status page: http://incubator.apache.org/projects/parquet.html Graduation discussion: http://s.apache.org/8qU Graduation resolution: http://s.apache.org/w7 Apache Parquet has been apart of the Incubator since May 2014 and in that time has * Had successful IPMC approved releases * Expanded the PPMC with new members * Worked and submitted fixes in other Apache projects to improve Parquet support * A successful community vote to graduate with 14 positive votes I would like to get the voting started with my own +1 Please vote [ ] +1 Graduate Apache Parquet as a TLP [ ] +0 [ ] -1 Don't graduate Apache Parquet as a TLP because… Voting will end in 72 hours, i.e. 12:00 UTC on 2015-04-16 http://www.timeanddate.com/countdown/to?iso=20150416T12 -Jake === Board Resolution == Establish the Apache Parquet Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software, for distribution at no charge to the public, related to a columnar storage format for Hadoop. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache Parquet Project, be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache Parquet Project be and hereby is responsible for the creation and maintenance of software related to a columnar storage format for Hadoop; and be it further RESOLVED, that the office of Vice President, Apache Parquet be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache Parquet Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache Parquet Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache Parquet Project: * Chris Aniszczyk caniszc...@apache.org * Ryan Blue b...@apache.org * Jonathan Coveney jcove...@apache.org * Tianshuo Deng tians...@apache.org * Jake Farrell jfarr...@apache.org * Marcel Kornacker mar...@apache.org * Mickael Lacour mlac...@apache.org * Julien Le Dem jul...@apache.org * Alex Levenson alexleven...@apache.org * Nong Li n...@apache.org * Todd Lipcon t...@apache.org * Chris Mattmann mattm...@apache.org * Aniket Mokashi aniket...@apache.org * Lukas Nalezenec lu...@apache.org * Brock Noland br...@apache.org * Wesley Graham Peck wesleyp...@apache.org * Remy Pecqueur
Re: [VOTE] Graduate Apache Parquet from the Incubator
+1 (non-binding) rb On 04/13/2015 09:02 AM, jan i wrote: +1 (binding) jan i. On 13 April 2015 at 17:46, Jake Farrell jfarr...@apache.org wrote: This thread is to start a vote on the graduation resolution Apache Parquet has approved on the dev list with 14 positive votes (11 PPMC (4 from IPMC members), 3 community) Status page: http://incubator.apache.org/projects/parquet.html Graduation discussion: http://s.apache.org/8qU Graduation resolution: http://s.apache.org/w7 Apache Parquet has been apart of the Incubator since May 2014 and in that time has * Had successful IPMC approved releases * Expanded the PPMC with new members * Worked and submitted fixes in other Apache projects to improve Parquet support * A successful community vote to graduate with 14 positive votes I would like to get the voting started with my own +1 Please vote [ ] +1 Graduate Apache Parquet as a TLP [ ] +0 [ ] -1 Don't graduate Apache Parquet as a TLP because… Voting will end in 72 hours, i.e. 12:00 UTC on 2015-04-16 http://www.timeanddate.com/countdown/to?iso=20150416T12 -Jake === Board Resolution == Establish the Apache Parquet Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software, for distribution at no charge to the public, related to a columnar storage format for Hadoop. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache Parquet Project, be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache Parquet Project be and hereby is responsible for the creation and maintenance of software related to a columnar storage format for Hadoop; and be it further RESOLVED, that the office of Vice President, Apache Parquet be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache Parquet Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache Parquet Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache Parquet Project: * Chris Aniszczyk caniszc...@apache.org * Ryan Blue b...@apache.org * Jonathan Coveney jcove...@apache.org * Tianshuo Deng tians...@apache.org * Jake Farrell jfarr...@apache.org * Marcel Kornacker mar...@apache.org * Mickael Lacour mlac...@apache.org * Julien Le Dem jul...@apache.org * Alex Levenson alexleven...@apache.org * Nong Li n...@apache.org * Todd Lipcon t...@apache.org * Chris Mattmann mattm...@apache.org * Aniket Mokashi aniket...@apache.org * Lukas Nalezenec lu...@apache.org * Brock Noland br...@apache.org * Wesley Graham Peck wesleyp...@apache.org * Remy Pecqueur rpecqu...@apache.org * Dmitriy Ryaboy dvrya...@apache.org * Roman Shaposhnik r...@apache.org * Daniel Weeks dwe...@apache.org * Tom White tomwh...@apache.org NOW, THEREFORE, BE IT FURTHER RESOLVED, that Julien Le Dem be appointed to the office of Vice President, Apache Parquet, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed; and be it further RESOLVED, that the initial Apache Parquet PMC be and hereby is tasked with the creation of a set of bylaws intended to encourage open development and increased participation in the Apache Parquet Project; and be it further RESOLVED, that the Apache Parquet Project be and hereby is tasked with the migration and rationalization of the Apache Incubator Parquet podling; and be it further RESOLVED, that all responsibilities pertaining to the Apache Incubator Parquet podling encumbered upon the Apache Incubator Project are hereafter discharged. -- Ryan Blue Software Engineer Cloudera, Inc. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
[VOTE] Release Apache Parquet MR (Incubating) 1.6.0 RC2
Hi everyone, I propose the following RC to be released as official Apache Parquet 1.6.0 release. This RC passed the vote in the Parquet community with 2 binding IPMC votes. The thread is here: * http://s.apache.org/parquet-1.6.0-ppmc-vote This is the first Apache release of Parquet MR. Like the 2.2.0 release of Parquet Format, this uses the old parquet.* package names and will be immediately followed by a 1.7.0 release that is strictly a move to org.apache.parquet.* packages and artifacts. The commit id is 4f660778f89d4164c72676f6b15bbc74c0d09373. * http://s.apache.org/apache-parquet-mr-1.6.0-incubating-rc2 * https://github.com/apache/incubator-parquet-mr/tree/4f660778 The release tarball, signature, and checksums are here: * https://dist.apache.org/repos/dist/dev/incubator/parquet/apache-parquet-mr-1.6.0-incubating-rc2/ You can find the KEYS file here: * https://dist.apache.org/repos/dist/dev/incubator/parquet/KEYS Full changes for the release can be found here: * https://github.com/apache/incubator-parquet-mr/blob/4f660778/CHANGES.md#version-160 Please download, verify, and test. Building the project requires both protoc and thrift compilers. We recommend versions 2.5.0 and 0.7.0 and there are build instructions in the README. This vote will close in 3 days. [ ] +1 Release this as Apache Parquet Format 1.6.0 [ ] +0 [ ] -1 Do not release this because... -- Ryan Blue - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Release Apache Parquet Format 2.3.0 RC2
On 02/16/2015 11:16 AM, Ted Dunning wrote: I have downloaded this release and verified the signature and md5 checksum. The SHA checksum is a binary file and it isn't obvious to me how to check it. How was it produced? I have verified that the LICENSE and NOTICE files are present and seem well-formed. I have verified that the DISCLAIMER is present and well-formed. I was unable to compile the code, apparently due to a thrift version mismatch of some kind. The README does not specify pre-requisites to compilation, but simply states that [mvn package] should suffice. It did not suffice for me. Here is the error message: [ERROR] thrift failed output: [ERROR] thrift failed error: !!! Unrecognized option: -out Usage: thrift [options] file Painful as it is to say this, I am -1 on this release unless somebody can explain some simple process to resolve my issues. Those issues are: 1) What format is used for the SHA checksum? How is it expected that I should check it? 2) What are the true pre-requisites for compilation? I would be willing to upgrade my vote on this release if the checksum is corrected and the README is updated for future releases (and I am told how to compile things now). Since the only change that I am requesting to this release is the checksum file, it should be possible to move forward without cutting a new release file. As Julien noted, you need to have thrift 0.7.0 installed. You also need protoc 2.5.0 installed for protobuf. I'll add more information in the README for this. In the mean time, if you take a look at the .travis.yml file, you can see the preparation steps used to run CI jobs. That's what I usually point people at instead of maintaining it in two places, but I agree that shouldn't be the solution. Thanks for taking a look at this! rb -- Ryan Blue Software Engineer Cloudera, Inc. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
[VOTE] Release Apache Parquet Format 2.3.0 RC2
Hi everyone, I propose RC2 to be released as official Apache Parquet 2.3.0 release. A similar vote has passed in the podling, with 4 +1 votes and 0 -1 or +0 votes. (With one binding +1 from an IPMC member.) The commit id is 7a6079ed5ddfa98a59cf8ac8728bcf5b0a1233b9 * This corresponds to the tag: apache-parquet-format-2.3.0-incubating * https://github.com/apache/incubator-parquet-format/tree/7a6079ed * https://git-wip-us.apache.org/repos/asf/incubator/repo?p=incubator-parquet-format.gita=commith=7a6079ed5ddfa98a59cf8ac8728bcf5b0a1233b9 The release tarball, signature, and checksums are here: * https://dist.apache.org/repos/dist/dev/incubator/parquet/apache-parquet-format-2.3.0-incubating-rc2 You can find the KEYS file here: * https://dist.apache.org/repos/dist/dev/incubator/parquet/KEYS Binary artifacts are staged in Nexus here: * https://repository.apache.org/content/groups/staging/org/apache/parquet/parquet-format/ Parquet Format 2.3.0 is functionally identical to the 2.2.0 release, but the classes have been moved to the org.apache.parquet package and the maven groupId is now org.apache.parquet. Please download, verify, and test. This vote will close on 19 Feb. [ ] +1 Release this as Apache Parquet Format 2.3.0 [ ] +0 [ ] -1 Do not release this because... -- Ryan Blue - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Votes for git repos - commit id vs tag
On 12/23/2014 01:27 AM, Bertrand Delacretaz wrote: On Tue, Dec 23, 2014 at 3:54 AM, Marvin Humphrey mar...@rectangular.com wrote: ...Although many consider it best practice for release tarballs to be tied back to a specific version control identifier (including me), Apache release policy does not require it As we tried to say above, the timeline gets into play here. Tying releases to version control is very useful *in the short term* for people to verify the release, but in the long term you cannot count on it. That's why our release policy centers on tarballs, the rest is convenience. I understand wanting to follow general ASF policy, but I wouldn't mind a more strict policy for incubating projects, simply because it is much easier for IPMC members to verify the release when we most need those careful checks. The release checklist appears to allow requirements imposed by Incubator policy: Each review item in this list is either required by Foundation-wide policy and would block a release by any Apache top-level project, or is required by Incubator policy. Either way, I think we should clarify this in the documentation to state whether or not the Incubator requires a git id or svn equivalent, and where that can be linked to. If we don't require a tarball being tied back to git, then it doesn't matter whether a tag or id is given for convenience. rb -- Ryan Blue Software Engineer Cloudera, Inc. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Votes for git repos - commit id vs tag
On 12/20/2014 04:07 AM, Bertrand Delacretaz wrote: On Sat, Dec 20, 2014 at 7:16 AM, Niclas Hedhman nic...@hedhman.org wrote: ...Releases are the tarball(s) prepared by the release manager, not a pointer into the source control system Agreed. I also agree with Brane about the pointer into source code control system being useful for PMC members to check that the released code is what they expect, but as you say long-term it's only the signed release tarball that matters. ...So, to make this clear to the community, I would discourage to publish the commit ID in the vote request, and only provide the URL link to the tarball(s) The way we work in Sling is that the tarball's name points to a well-known svn tag URL. This matches your idea of having the commit ID or equivalent somewhere else, but easily accessible. I like that. OTOH I also like to include the tarball archive's digest (sha1 or equivalent) in the archived vote thread as that's a long term (*) guarantee that what you got is what was voted on. -Bertrand (*) As long as the digest algorithm is not broken, that is. There seems to be some support for release tarballs independent of version control and some support for tarballs that are tied back to a specific version of the repository (whether SVN tag or git ID). I think it is not just great for convenience, but necessary to link back to version control. That makes it easy for PMC members to verify certain aspects of the release that are otherwise difficult. Tasks like verifying source additions were correctly mirrored in NOTICE updates are important, and we want that to be as easy as possible. If I'm verifying an independent tarball, then I can't browse history as easily. If it is best practice to link to version control, then we have to have a way to verify the version control link matches the release. I think policy should be that a release tarball is based on the most reasonable identifier in version control. For svn, that's a tag and revision number. For git, that's a commit id/hash. Projects should have repeatable processes to get the release tarball from the identifier that can be verified against the RM's signature. `git archive` works most of the time. Given that there is confusion on this, I think we should decide whether it is required or not and update the docs to be more clear. Does that require a vote? One last point: if the requirement is for git id and verification is required, then this allows us to use links to preferred mirrors as well, which can be easier to work with. As long as both apache git and the mirror are given, of course. rb -- Ryan Blue Software Engineer Cloudera, Inc. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Votes for git repos - commit id vs tag
On 12/18/2014 05:58 AM, John D. Ament wrote: All, I was looking through the incubator site and I don't see anything definite. Whenever a podling goes for a vote, and they include a git tag in their vote message, it's typically asked to change to a commit id. It seems to me this is done for the reproducible builds concept. Tags are mutable, and therefore could be changed and rebuilding a tag could give you a different result. So, is this the right understanding? Do we want to ask podlings to always submit a git commit id? If so, is there a place in the website we can clarify this? Thanks, John I recently found this confusing with the first parquet-format release. I thought that both commit id and tag were optional, given that the actual release candidate is a signed tarball (actually, the necessary source code to build the project [1]). We can't necessarily recover the commit id from the tarball because the parent information is lost [2], so requiring the commit id is only useful for convenience and validating that a new tarball from git at the commit id matches the vote tarball. Is this validation done? Is it a requirement? If it isn't a requirement for a commit to match what is being voted on, then does it matter whether we use a tag for convenience or a commit id? We could also accept signed tags, though I don't know if there are issues that would prevent it. rb [1]: https://www.apache.org/dev/release-publishing.html#valid [2]: Unless using `git archive`: http://git-scm.com/docs/git-archive -- Ryan Blue - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] accept NiFi into the incubator
+1 (non-binding) rb On 11/21/2014 01:54 PM, Jake Farrell wrote: +1 (binding) -Jake On Fri, Nov 21, 2014 at 1:55 PM, Benson Margulies bimargul...@gmail.com wrote: http://wiki.apache.org/incubator/NiFiProposal has elicited a cheerful and positive conversation, so I offer this vote. Vote will be open for the usual 72 hours ... Here is my [+1] -- Ryan Blue Software Engineer Cloudera, Inc. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
[RESULT] [VOTE] Release Apache Parquet Format (incubating) 2.2.0 RC2
This vote has passed with 3 +1 votes from IPMC members. Here are the votes: +1 (binding): * Tom White (from podling vote) * Stack * Patrick Hunt +1 (non-binding): * Chris Aniszczyk * Julien Le Dem +0: (none) -1: (none) Thank you to everyone for verifying the release! rb On 11/10/2014 02:35 PM, Ryan Blue wrote: Hi everyone, I'd like to propose a vote to release parquet-format-2.2.0-rc2 as the official Parquet Format 2.2.0 release. This release candidate has passed a podling vote, which can be found here: https://mail-archives.apache.org/mod_mbox/incubator-parquet-dev/201411.mbox/%3C54613B48.6060602%40apache.org%3E The release candidate, signature, and checksums are available at: https://dist.apache.org/repos/dist/dev/incubator/parquet/2.2.0-rc2/ The branch used to create the release candidate is: https://git-wip-us.apache.org/repos/asf?p=incubator-parquet-format.git;hb=release-2.2.0-rc2 KEYS to validate the signature are available at: https://dist.apache.org/repos/dist/dev/incubator/parquet/KEYS Please download, verify, and test. [ ] +1: Release this tag as parquet-format-2.2.0 [ ] +0: [ ] -1: This release is not ready because . . . -- Ryan Blue Software Engineer Cloudera, Inc. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Release Apache Parquet Format (incubating) 2.2.0 RC2
On 11/12/2014 07:06 PM, John D. Ament wrote: The use of tags has come up several times on the incubator list in the past. They're not immutable, linking to a tag can be over-written. Please link to the appropriate commit, which in your case is likely https://git-wip-us.apache.org/repos/asf?p=incubator-parquet-format.git;a=commit;h=6979a6d60a319de6aab0b7743ef5b6a50a2f974e - John Thanks for finding the correct reference, John. You're right that the commit is 6979a6d60a319de6aab0b7743ef5b6a50a2f974e, which can be viewed at this URL: https://git-wip-us.apache.org/repos/asf?p=incubator-parquet-format.git;a=commit;h=6979a6d60a319de6aab0b7743ef5b6a50a2f974e This commit corresponds to the contents of the release candidate source tarball, which you can find here (along with checksums and signature): https://dist.apache.org/repos/dist/dev/incubator/parquet/2.2.0-rc2/ For future vote threads, I'll make sure we provide the right commit rather than a tag or branch reference. Thanks! rb On Mon, Nov 10, 2014 at 5:35 PM, Ryan Blue b...@apache.org wrote: Hi everyone, I'd like to propose a vote to release parquet-format-2.2.0-rc2 as the official Parquet Format 2.2.0 release. This release candidate has passed a podling vote, which can be found here: https://mail-archives.apache.org/mod_mbox/incubator- parquet-dev/201411.mbox/%3C54613B48.6060602%40apache.org%3E The release candidate, signature, and checksums are available at: https://dist.apache.org/repos/dist/dev/incubator/parquet/2.2.0-rc2/ The branch used to create the release candidate is: https://git-wip-us.apache.org/repos/asf?p=incubator-parquet- format.git;hb=release-2.2.0-rc2 KEYS to validate the signature are available at: https://dist.apache.org/repos/dist/dev/incubator/parquet/KEYS Please download, verify, and test. [ ] +1: Release this tag as parquet-format-2.2.0 [ ] +0: [ ] -1: This release is not ready because . . . == More details == This will be the first release from the Parquet project. We are releasing parquet-format first because the other projects (parquet-mr and parquet-cpp) depend on it. This release uses the pre-Apache maven coordinate com.twitter:parquet-format and parquet.* package names. To make the transition to org.apache naming as smooth as possible for downstream projects, we are planning to release a com.twitter artifact immediately followed by an org.apache artifact. These artifacts will be identical except for naming changes: * com.twitter:parquet-format = org.apache.parquet:parquet-format * package parquet.* = package org.apache.parquet.* There are three major changes included in this release: * PARQUET-119: Add encoding stats to ColumnMetaData * PARQUET-79: Streaming thrift API * PARQUET-12: New logical types rb -- Ryan Blue - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- Ryan Blue Software Engineer Cloudera, Inc. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Release Apache Parquet Format (incubating) 2.2.0 RC2
On 11/13/2014 02:21 PM, Julien Le Dem wrote: +1 We need votes from the incubator PMC, right? Julien That's right, this vote requires at least 3 +1s from IPMC members to pass. I know that some of our mentors are out of town, so it would be helpful if anyone has time to take a look at the RC. This is our first release vote for the Parquet project, so I'm hoping to get good feedback. rb On Thu, Nov 13, 2014 at 8:51 AM, Ryan Blue b...@cloudera.com wrote: On Mon, Nov 10, 2014 at 5:35 PM, Ryan Blue b...@apache.org wrote: Hi everyone, I'd like to propose a vote to release parquet-format-2.2.0-rc2 as the official Parquet Format 2.2.0 release. This release candidate has passed a podling vote, which can be found here: https://mail-archives.apache.org/mod_mbox/incubator- parquet-dev/201411.mbox/%3C54613B48.6060602%40apache.org%3E The release candidate, signature, and checksums are available at: https://dist.apache.org/repos/dist/dev/incubator/parquet/2.2.0-rc2/ The branch used to create the release candidate is: https://git-wip-us.apache.org/repos/asf?p=incubator-parquet- format.git;hb=release-2.2.0-rc2 KEYS to validate the signature are available at: https://dist.apache.org/repos/dist/dev/incubator/parquet/KEYS Please download, verify, and test. [ ] +1: Release this tag as parquet-format-2.2.0 [ ] +0: [ ] -1: This release is not ready because . . . == More details == This will be the first release from the Parquet project. We are releasing parquet-format first because the other projects (parquet-mr and parquet-cpp) depend on it. This release uses the pre-Apache maven coordinate com.twitter:parquet-format and parquet.* package names. To make the transition to org.apache naming as smooth as possible for downstream projects, we are planning to release a com.twitter artifact immediately followed by an org.apache artifact. These artifacts will be identical except for naming changes: * com.twitter:parquet-format = org.apache.parquet:parquet-format * package parquet.* = package org.apache.parquet.* There are three major changes included in this release: * PARQUET-119: Add encoding stats to ColumnMetaData * PARQUET-79: Streaming thrift API * PARQUET-12: New logical types rb -- Ryan Blue - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- Ryan Blue Software Engineer Cloudera, Inc. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- Ryan Blue Software Engineer Cloudera, Inc. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
[VOTE] Release Apache Parquet Format (incubating) 2.2.0 RC2
Hi everyone, I'd like to propose a vote to release parquet-format-2.2.0-rc2 as the official Parquet Format 2.2.0 release. This release candidate has passed a podling vote, which can be found here: https://mail-archives.apache.org/mod_mbox/incubator-parquet-dev/201411.mbox/%3C54613B48.6060602%40apache.org%3E The release candidate, signature, and checksums are available at: https://dist.apache.org/repos/dist/dev/incubator/parquet/2.2.0-rc2/ The branch used to create the release candidate is: https://git-wip-us.apache.org/repos/asf?p=incubator-parquet-format.git;hb=release-2.2.0-rc2 KEYS to validate the signature are available at: https://dist.apache.org/repos/dist/dev/incubator/parquet/KEYS Please download, verify, and test. [ ] +1: Release this tag as parquet-format-2.2.0 [ ] +0: [ ] -1: This release is not ready because . . . == More details == This will be the first release from the Parquet project. We are releasing parquet-format first because the other projects (parquet-mr and parquet-cpp) depend on it. This release uses the pre-Apache maven coordinate com.twitter:parquet-format and parquet.* package names. To make the transition to org.apache naming as smooth as possible for downstream projects, we are planning to release a com.twitter artifact immediately followed by an org.apache artifact. These artifacts will be identical except for naming changes: * com.twitter:parquet-format = org.apache.parquet:parquet-format * package parquet.* = package org.apache.parquet.* There are three major changes included in this release: * PARQUET-119: Add encoding stats to ColumnMetaData * PARQUET-79: Streaming thrift API * PARQUET-12: New logical types rb -- Ryan Blue - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
