Re: Looking for Champion

[Jim Apple]

Hello! As a contributor to Impala, I’d be interested in hearing thoughts from 
the Palo community about integration between Impala and Palo.

For instance, are there any apparent design goals of Impala that the Palo 
community thinks are fundamentally incompatible with Palo?

Thanks,
Jim

On 2018/06/08 04:45:32, "Li,De(BDG)"  wrote: 
> Hi all,
> 
> I am Reed, as a developer worked with the team for Palo (a MPP-based 
> interactive SQL data warehousing).
> https://github.com/baidu/palo/wiki/Palo-Overview
> 
> We propose to contribute Palo as an Apache Incubator project, and
> we are still looking for possible Champion if anyone would like to volunteer. 
> Thanks a lot.
> 
> Best Regards,
> Reed
> 
> ===
> The draft of the proposal as below:
> 
> #Apache Palo
> 
> ##Abstract
> 
> Palo is a MPP-based interactive SQL data warehousing for reporting and 
> analysis.
> 
> ##Proposal
> 
> We propose to contribute the Palo codebase and associated artifacts (e.g. 
> documentation, web-site content etc.) to the Apache Software Foundation with 
> the intent of forming a productive, meritocratic and open community around 
> Palo’s continued development, according to the ‘Apache Way’.
> 
> Baidu owns several trademarks regarding Palo, and proposes to transfer 
> ownership of those trademarks in full to the ASF.
> 
> ###Overview of Palo
> 
> Palo’s implementation consists of two daemons: Frontend (FE) and Backend (BE).
> 
> **Frontend daemon** consists of query coordinator and catalog manager. Query 
> coordinator is responsible for receiving users’ sql queries, compiling 
> queries and managing queries execution. Catalog manager is responsible for 
> managing metadata such as databases, tables, partitions, replicas and etc. 
> Several frontend daemons could be deployed to guarantee fault-tolerance, and 
> load balancing.
> 
> **Backend daemon** stores the data and executes the query fragments. Many 
> backend daemons could also be deployed to provide scalability and 
> fault-tolerance.
> 
> A typical Palo cluster generally composes of several frontend daemons and 
> dozens to hundreds of backend daemons.
> 
> Users can use MySQL client tools to connect any frontend daemon to submit SQL 
> query. Frontend receives the query and compiles it into query plans 
> executable by the Backend. Then Frontend sends the query plan fragments to 
> Backend. Backend will build a query execution DAG. Data is fetched and 
> pipelined into the DAG. The final result response is sent to client via 
> Frontend. The distribution of query fragment execution takes minimizing data 
> movement and maximizing scan locality as the main goal.
> 
> ##Background
> 
> At Baidu, Prior to Palo, different tools were deployed to solve diverse 
> requirements in many ways. And when a use case requires the simultaneous 
> availability of capabilities that cannot all be provided by a single tool, 
> users were forced to build hybrid architectures that stitch multiple tools 
> together, but we believe that they shouldn’t need to accept such inherent 
> complexity. A storage system built to provide great performance across a 
> broad range of workloads provides a more elegant solution to the problems 
> that hybrid architectures aim to solve. Palo is the solution.
> 
> Palo is designed to be a simple and single tightly coupled system, not 
> depending on other systems. Palo provides high concurrent low latency point 
> query performance, but also provides high throughput queries of ad-hoc 
> analysis. Palo provides bulk-batch data loading, but also provides near 
> real-time mini-batch data loading. Palo also provides high availability, 
> reliability, fault tolerance, and scalability.
> 
> ##Rationale
> 
> Palo mainly integrates the technology of Google Mesa and Apache Impala.
> 
> Mesa is a highly scalable analytic data storage system that stores critical 
> measurement data related to Google's Internet advertising business. Mesa is 
> designed to satisfy complex and challenging set of users’ and systems’ 
> requirements, including near real-time data ingestion and query ability, as 
> well as high availability, reliability, fault tolerance, and scalability for 
> large data and query volumes.
> 
> Impala is a modern, open-source MPP SQL engine architected from the ground up 
> for the Hadoop data processing environment. At present, by virtue of its 
> superior performance and rich functionality， Impala has been comparable to 
> many commercial MPP database query engine. Mesa can satisfy the needs of many 
> of our storage requirements, however Mesa itself does not provide a SQL query 
> engine; Impala is a very good MPP SQL query engine, but the lack of a perfect 
> distributed storage engine. So in the end we chose the combination of these 
> two technologies.
> 
> Learning from Mesa’s data model, we developed a distributed storage engine. 
> Unlike Mesa, this storage engine does not rely on any distributed file 
> system. Then we deeply 

Looking for Champion

[Li,De(BDG)]

Hi all,

I am Reed, as a developer worked with the team for Palo (a MPP-based 
interactive SQL data warehousing).
https://github.com/baidu/palo/wiki/Palo-Overview

We propose to contribute Palo as an Apache Incubator project, and
we are still looking for possible Champion if anyone would like to volunteer. 
Thanks a lot.

Best Regards,
Reed

===
The draft of the proposal as below:

#Apache Palo

##Abstract

Palo is a MPP-based interactive SQL data warehousing for reporting and analysis.

##Proposal

We propose to contribute the Palo codebase and associated artifacts (e.g. 
documentation, web-site content etc.) to the Apache Software Foundation with 
the intent of forming a productive, meritocratic and open community around 
Palo’s continued development, according to the ‘Apache Way’.

Baidu owns several trademarks regarding Palo, and proposes to transfer 
ownership of those trademarks in full to the ASF.

###Overview of Palo

Palo’s implementation consists of two daemons: Frontend (FE) and Backend (BE).

**Frontend daemon** consists of query coordinator and catalog manager. Query 
coordinator is responsible for receiving users’ sql queries, compiling queries 
and managing queries execution. Catalog manager is responsible for managing 
metadata such as databases, tables, partitions, replicas and etc. Several 
frontend daemons could be deployed to guarantee fault-tolerance, and load 
balancing.

**Backend daemon** stores the data and executes the query fragments. Many 
backend daemons could also be deployed to provide scalability and 
fault-tolerance.

A typical Palo cluster generally composes of several frontend daemons and 
dozens to hundreds of backend daemons.

Users can use MySQL client tools to connect any frontend daemon to submit SQL 
query. Frontend receives the query and compiles it into query plans executable 
by the Backend. Then Frontend sends the query plan fragments to Backend. 
Backend will build a query execution DAG. Data is fetched and pipelined into 
the DAG. The final result response is sent to client via Frontend. The 
distribution of query fragment execution takes minimizing data movement and 
maximizing scan locality as the main goal.

##Background

At Baidu, Prior to Palo, different tools were deployed to solve diverse 
requirements in many ways. And when a use case requires the simultaneous 
availability of capabilities that cannot all be provided by a single tool, 
users were forced to build hybrid architectures that stitch multiple tools 
together, but we believe that they shouldn’t need to accept such inherent 
complexity. A storage system built to provide great performance across a broad 
range of workloads provides a more elegant solution to the problems that hybrid 
architectures aim to solve. Palo is the solution.

Palo is designed to be a simple and single tightly coupled system, not 
depending on other systems. Palo provides high concurrent low latency point 
query performance, but also provides high throughput queries of ad-hoc 
analysis. Palo provides bulk-batch data loading, but also provides near 
real-time mini-batch data loading. Palo also provides high availability, 
reliability, fault tolerance, and scalability.

##Rationale

Palo mainly integrates the technology of Google Mesa and Apache Impala.

Mesa is a highly scalable analytic data storage system that stores critical 
measurement data related to Google's Internet advertising business. Mesa is 
designed to satisfy complex and challenging set of users’ and systems’ 
requirements, including near real-time data ingestion and query ability, as 
well as high availability, reliability, fault tolerance, and scalability for 
large data and query volumes.

Impala is a modern, open-source MPP SQL engine architected from the ground up 
for the Hadoop data processing environment. At present, by virtue of its 
superior performance and rich functionality， Impala has been comparable to many 
commercial MPP database query engine. Mesa can satisfy the needs of many of our 
storage requirements, however Mesa itself does not provide a SQL query engine; 
Impala is a very good MPP SQL query engine, but the lack of a perfect 
distributed storage engine. So in the end we chose the combination of these two 
technologies.

Learning from Mesa’s data model, we developed a distributed storage engine. 
Unlike Mesa, this storage engine does not rely on any distributed file system. 
Then we deeply integrate this storage engine with Impala query engine. Query 
compiling, query execution coordination and catalog management of storage 
engine are integrated to be frontend daemon; query execution and data storage 
are integrated to be backend daemon. With this integration, we implemented a 
single, full-featured, high performance state the art of MPP database, as well 
as maintaining the simplicity.

##Current Status

Palo has been an open source project on GitHub (https://github.com/baidu/palo).

###Meritocracy

Palo has been 

Re: [VOTE] Pulsar Release 1.22.1-incubating Candidate 2

[Yang Bo]

Hi,

I checked the source release and found a minor issue:
pulsar-common/src/main/java/org/apache/pulsar/common/util/protobuf/ByteBufCoded{Input,Output}Stream.java
Those files are from google thus should not have ASF header in the source
file.


On Fri, Jun 8, 2018 at 9:51 AM, Sijie Guo  wrote:

> +1 (repeated my +1 from dev@ votes)
>
> On Thu, Jun 7, 2018 at 11:29 AM Jai Asher  wrote:
>
> > This is the second release candidate for Apache Pulsar, patch release
> > version 1.22.1-incubating.
> >
> > It fixes the following issues:
> > https://github.com/apache/incubator-pulsar/milestone/15?closed=1
> >
> > *** Please download, test and vote on this release. This vote will stay
> > open for at least 72 hours ***
> >
> > Note that we are voting upon the source (tag), binaries are provided for
> > convenience.
> >
> > Source and binary files:
> >
> > https://dist.apache.org/repos/dist/dev/incubator/pulsar/
> pulsar-1.22.1-incubating-candidate-2/
> >
> > SHA-1 checksums:
> >
> > f2d29aa5e046c5bdefd8f466bce8e9ead80a2e09
> > apache-pulsar-1.22.1-incubating-src.tar.gz
> > f2704cae22b7fb3c1b72daab6ebe7d484fdfec6b
> > apache-pulsar-1.22.1-incubating-bin.tar.gz
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachepulsar-1018/
> >
> > The tag to be voted upon:
> > v1.22.1-incubating-candidate-2 (c9a369936af3b3ecc663b86ae959a3
> fbfa627aca)
> >
> > https://github.com/apache/incubator-pulsar/releases/tag/
> v1.22.1-incubating-candidate-2
> >
> > Pulsar's KEYS file containing PGP keys we use to sign the release:
> > https://dist.apache.org/repos/dist/release/incubator/pulsar/KEYS
> >
> > Please download the source package, and follow the README to build and
> run
> > the Pulsar standalone service.
> >
>



-- 
Best Regards,
Yang.

Re: [VOTE] Pulsar Release 1.22.1-incubating Candidate 2

[Sijie Guo]

+1 (repeated my +1 from dev@ votes)

On Thu, Jun 7, 2018 at 11:29 AM Jai Asher  wrote:

> This is the second release candidate for Apache Pulsar, patch release
> version 1.22.1-incubating.
>
> It fixes the following issues:
> https://github.com/apache/incubator-pulsar/milestone/15?closed=1
>
> *** Please download, test and vote on this release. This vote will stay
> open for at least 72 hours ***
>
> Note that we are voting upon the source (tag), binaries are provided for
> convenience.
>
> Source and binary files:
>
> https://dist.apache.org/repos/dist/dev/incubator/pulsar/pulsar-1.22.1-incubating-candidate-2/
>
> SHA-1 checksums:
>
> f2d29aa5e046c5bdefd8f466bce8e9ead80a2e09
> apache-pulsar-1.22.1-incubating-src.tar.gz
> f2704cae22b7fb3c1b72daab6ebe7d484fdfec6b
> apache-pulsar-1.22.1-incubating-bin.tar.gz
>
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachepulsar-1018/
>
> The tag to be voted upon:
> v1.22.1-incubating-candidate-2 (c9a369936af3b3ecc663b86ae959a3fbfa627aca)
>
> https://github.com/apache/incubator-pulsar/releases/tag/v1.22.1-incubating-candidate-2
>
> Pulsar's KEYS file containing PGP keys we use to sign the release:
> https://dist.apache.org/repos/dist/release/incubator/pulsar/KEYS
>
> Please download the source package, and follow the README to build and run
> the Pulsar standalone service.
>

Re: [VOTE] Pulsar Release 1.22.1-incubating Candidate 2

[Dave Fisher]

Repeating my +1 (binding) from the dev@pulsar VOTE

Regards,
Dave

Sent from my iPhone

> On Jun 7, 2018, at 11:29 AM, Jai Asher  wrote:
> 
> This is the second release candidate for Apache Pulsar, patch release
> version 1.22.1-incubating.
> 
> It fixes the following issues:
> https://github.com/apache/incubator-pulsar/milestone/15?closed=1
> 
> *** Please download, test and vote on this release. This vote will stay
> open for at least 72 hours ***
> 
> Note that we are voting upon the source (tag), binaries are provided for
> convenience.
> 
> Source and binary files:
> https://dist.apache.org/repos/dist/dev/incubator/pulsar/pulsar-1.22.1-incubating-candidate-2/
> 
> SHA-1 checksums:
> 
> f2d29aa5e046c5bdefd8f466bce8e9ead80a2e09
> apache-pulsar-1.22.1-incubating-src.tar.gz
> f2704cae22b7fb3c1b72daab6ebe7d484fdfec6b
> apache-pulsar-1.22.1-incubating-bin.tar.gz
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachepulsar-1018/
> 
> The tag to be voted upon:
> v1.22.1-incubating-candidate-2 (c9a369936af3b3ecc663b86ae959a3fbfa627aca)
> https://github.com/apache/incubator-pulsar/releases/tag/v1.22.1-incubating-candidate-2
> 
> Pulsar's KEYS file containing PGP keys we use to sign the release:
> https://dist.apache.org/repos/dist/release/incubator/pulsar/KEYS
> 
> Please download the source package, and follow the README to build and run
> the Pulsar standalone service.


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

[VOTE] Pulsar Release 1.22.1-incubating Candidate 2

[Jai Asher]

This is the second release candidate for Apache Pulsar, patch release
version 1.22.1-incubating.

It fixes the following issues:
https://github.com/apache/incubator-pulsar/milestone/15?closed=1

*** Please download, test and vote on this release. This vote will stay
open for at least 72 hours ***

Note that we are voting upon the source (tag), binaries are provided for
convenience.

Source and binary files:
https://dist.apache.org/repos/dist/dev/incubator/pulsar/pulsar-1.22.1-incubating-candidate-2/

SHA-1 checksums:

f2d29aa5e046c5bdefd8f466bce8e9ead80a2e09
apache-pulsar-1.22.1-incubating-src.tar.gz
f2704cae22b7fb3c1b72daab6ebe7d484fdfec6b
apache-pulsar-1.22.1-incubating-bin.tar.gz

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachepulsar-1018/

The tag to be voted upon:
v1.22.1-incubating-candidate-2 (c9a369936af3b3ecc663b86ae959a3fbfa627aca)
https://github.com/apache/incubator-pulsar/releases/tag/v1.22.1-incubating-candidate-2

Pulsar's KEYS file containing PGP keys we use to sign the release:
https://dist.apache.org/repos/dist/release/incubator/pulsar/KEYS

Please download the source package, and follow the README to build and run
the Pulsar standalone service.

Re: Deploying Omid release 0.9.0.0

[Ohad Shacham]

It worked!

Thanks for your help Sijie!!!

Thanks,
Ohad

On Thu, Jun 7, 2018 at 9:56 AM, Sijie Guo  wrote:

> It seems to publish the maven artifacts to bintray.com, not apache
> repository.
>
> just took a quick look at your pom.xml, it seems that your profile doesn't
> inherit from Apache POM, which will help you setup the distribution
> management section to use the correct release / snapshot repositories.
>
> You can checkout the instructions here -
> http://www.apache.org/dev/publishing-maven-artifacts.html
>
> Hope this helps!
>
> - Sijie
>
> On Wed, Jun 6, 2018 at 1:20 AM Ohad Shacham 
> wrote:
>
> > Hi All,
> >
> > I am trying to deploy Omid 0.9.0.0 release and get the following error:
> >
> > [*ERROR*] Failed to execute goal
> > org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy
> *(default-deploy)*
> > on project omid: *Failed to deploy artifacts: Could not transfer artifact
> > org.apache.omid:omid:pom:0.9.0.0 from/to bintray
> > (https://api.bintray.com/maven/yahoo/maven/omid
> > ): Failed to transfer
> > file:
> >
> > https://api.bintray.com/maven/yahoo/maven/omid/org/apache/
> omid/omid/0.9.0.0/omid-0.9.0.0.pom
> > <
> > https://api.bintray.com/maven/yahoo/maven/omid/org/apache/
> omid/omid/0.9.0.0/omid-0.9.0.0.pom
> > >.
> > Return code is: 401, ReasonPhrase: Unauthorized.* -> *[Help 1]*
> >
> > I am using command:
> > mvn deploy -Phbase-1 -Prelease_profile -DskipTests
> >
> > The release tag is:
> > https://git-wip-us.apache.org/repos/asf/incubator-omid.git/?
> > p=incubator-omid.git;a=tag;h=229831bc377e47431cc7cbc211803069158b5711
> >
> > and the repository is:
> >
> > https://git-wip-us.apache.org/repos/asf/incubator-omid.git
> >
> > Can anyone please advise?
> >
> > Thanks,
> > Ohad
> >
>

[RESULT][VOTE] Release Apache Traffic Control 2.2.0-RC6

[Robert Butts]

Thanks to all who voted!

The release has PASSED with the following IPMC votes:

+1 Jeff Elsloo (binding)
+1 Steve Malenfant (binding)
+1 Dan Kirkwood (binding)

I will proceed to publish the release and send ANNOUNCE.

On behalf of Apache Traffic Control, thank you!

Regards,
Robert O Butts r...@apache.org

Re: [ANN] Please welcome Justin Mclean as the new Incubator PMC chair!

[Pierre Smits]

Thank you, John, for all your hard work as the Incubator VP and PMC Chair.

Congratulations, Justin.


Best regards,

Pierre Smits

Apache Trafodion , Vice President
Apache Directory , PMC Member
Apache Incubator , committer
Apache OFBiz , contributor since 2008
Apache Steve , committer

On Mon, Jun 4, 2018 at 12:12 PM, Bertrand Delacretaz  wrote:

> Hi,
>
> John D. Ament recently announced his desire to step down from the
> Incubator PMC chair role. Thank you so much John for your efficient
> service here, "retiring" from that is certainly well deserved!
>
> The Incubator PMC had the difficult task of choosing between a handful
> of excellent candidates and has selected Justin as our new chair. The
> ASF's Board of Directors has ratified this choice at our May meeting.
>
> Please join me in welcoming Jusin as our new PMC chair! Justin's been
> very active in the Incubator in the last few years, it's great to see
> him step up to this role.
>
> -Bertrand
>
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>

Re: graphviz as a potential apache project

[sharan]

Forwarding Stephen's comments to the mailing list.

Thanks
Sharan

 Forwarded Message 
Subject:Re: graphviz as a potential apache project
Date:   Mon, 4 Jun 2018 17:56:09 -0400
From:   Stephen North 
To: sha...@apache.org



Thank you.  I’ll check on the subscription.

Some brief comments:
1) Yes, we might have felt a little too much like a special snowflake.
2) Formally, AT owns most of the code, which was developed before they decided
they didn’t want me to work there any more around the end of 2013. I still have 
a few
colleagues there that can help a little with intellectual property issues but 
the corporate
apathy can seem overwhelming at times.  I don’t know if they would be open to 
changing
the license. Also, we did not make use of a contributor license, though the 
external
contributions were not that significant except for Tim Dwyer’s layout algorithm 
work
and I’m still in touch with Tim quite often.
3) I have to admit it is not a very vibrant developer community any more. Emden
Gansner, John Ellson and I were the main gatekeepers/committers to the source
repository.  I’m still in touch with John and Emden but they are either retired 
or
very close to it and due to personal reasons they are sympathetic toward work on
the code but are not in a position to do a lot on their own these days.
Stephen North



On Jun 4, 2018, at 8:44 AM, sha...@apache.org wrote:

Hi Stephen

I'm not sure if you have been copied on all the responses and comments to your 
email so am including a link to the complete mailing list thread so far:

https://s.apache.org/KDoq

Also please feel free to respond or add your comments to anything mentioned.

Thanks
Sharan

On 2.6.2018 20:52, sha...@apache.org wrote:

Hi Stephen

Thanks for your email. I will follow up and find you who can respond to this 
query for you.

Thanks
Sharan

On 31.5.2018 23:51, Stephen North wrote:

Hi, Sharan.   I hope you’re the right person for this.  (Your org. just sent an 
invitation
to an Apache roadshow event under your name.)

I’m one of the founders of Graphviz, 20+ years ago, and it’s useful 
infrastructure
software with new and old applications in bioinformatics, machine learning, 
software
engineering and other fields, including plugins for R, python, Haskell, 
transpiled
into javascript, who knows what else.  I think for example clang, llvm, maybe Go
use it for debug output.  See www.graphviz.org

We started this in AT Labs but AT discontinued all support for the work,
eliminated some of our jobs or encouraged us to leave, has disclaimed the 
project,
so we’re on our own.  We maintain the software and the website, that we moved 
to gitlab.

We are looking for a more stable institutional home for Graphviz.  Apache seems 
like
a great match. Is this possible?  How is a decision like that made?  I’m not 
sure a mature
project like this fits the incubator model. Are there other ways?

Sorry if this has gone long (I could say a lot more, too, for example how we 
could put
focus effort on new features, or providing better resources to people that need
network visualization as a service). I’m hoping you’re the right person to 
start the conversation.

Thank you for your time, and for your consideration.

Stephen North

Re: Deploying Omid release 0.9.0.0

[Sijie Guo]

It seems to publish the maven artifacts to bintray.com, not apache
repository.

just took a quick look at your pom.xml, it seems that your profile doesn't
inherit from Apache POM, which will help you setup the distribution
management section to use the correct release / snapshot repositories.

You can checkout the instructions here -
http://www.apache.org/dev/publishing-maven-artifacts.html

Hope this helps!

- Sijie

On Wed, Jun 6, 2018 at 1:20 AM Ohad Shacham  wrote:

> Hi All,
>
> I am trying to deploy Omid 0.9.0.0 release and get the following error:
>
> [*ERROR*] Failed to execute goal
> org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy *(default-deploy)*
> on project omid: *Failed to deploy artifacts: Could not transfer artifact
> org.apache.omid:omid:pom:0.9.0.0 from/to bintray
> (https://api.bintray.com/maven/yahoo/maven/omid
> ): Failed to transfer
> file:
>
> https://api.bintray.com/maven/yahoo/maven/omid/org/apache/omid/omid/0.9.0.0/omid-0.9.0.0.pom
> <
> https://api.bintray.com/maven/yahoo/maven/omid/org/apache/omid/omid/0.9.0.0/omid-0.9.0.0.pom
> >.
> Return code is: 401, ReasonPhrase: Unauthorized.* -> *[Help 1]*
>
> I am using command:
> mvn deploy -Phbase-1 -Prelease_profile -DskipTests
>
> The release tag is:
> https://git-wip-us.apache.org/repos/asf/incubator-omid.git/?
> p=incubator-omid.git;a=tag;h=229831bc377e47431cc7cbc211803069158b5711
>
> and the repository is:
>
> https://git-wip-us.apache.org/repos/asf/incubator-omid.git
>
> Can anyone please advise?
>
> Thanks,
> Ohad
>