Re: Unable to commit code on Apache incubator-hawq repo

[Radar Lei]

Hi Lav,

I don't know what goes wrong, but maybe you can try to fill a ticket in:
https://issues.apache.org/jira/projects/INFRA

Regards,
Radar

On Sat, Aug 11, 2018 at 6:32 AM, Lav Jain  wrote:

> Hello Everyone,
>
> I became a HAWQ committer about 3 months ago. However, I am still not able
> to commit directly to the incubator-hawq repo on
> https://git-wp-us.apache.org/repos/asf/incubator-hawq.git
>
> My apache id is *lavjain*. Could someone please point me in the correct
> direction on how to get this resolved? My paperwork was completed 2 years
> ago when I became a committer on another Apache project.
>
>
> Regards,
>
>
> *Lav Jain*
> *Pivotal Data*
>
> lj...@pivotal.io
>

[jira] [Created] (HAWQ-1647) Update HAWQ version from 2.3.0.0 to 2.4.0.0

[Radar Lei (JIRA)]

Radar Lei created HAWQ-1647:
---

 Summary: Update HAWQ version from 2.3.0.0 to 2.4.0.0
 Key: HAWQ-1647
 URL: https://issues.apache.org/jira/browse/HAWQ-1647
 Project: Apache HAWQ
  Issue Type: Task
  Components: Build
Reporter: Radar Lei
Assignee: Radar Lei


Update version number from 2.3.0.0 to 2.4.0.0 for release of HAWQ 
2.4.0.0-incubating.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Share Web-Based HAWQ Admin Tool

[Radar Lei]

Cool! Thanks for sharing.

As Ming mentioned, it would be more helpful to have more contents in README.

Regards,
Radar

On Mon, Aug 6, 2018 at 9:57 AM, Ming Li  wrote:

> Thanks for your contribution, maybe we could create one page on hawq wiki
> for related projects.
>
> On Mon, Aug 6, 2018 at 9:14 AM, Ivan Weng  wrote:
>
> >
> >
> >
> >
> >
> >
> > Great, thanks for sharing to HAWQ community.
> >
> >
> > Regards,Ivan
> > _
> > From: zhouzhan.chen 
> > Sent: 星期一, 八月 6, 2018 4:42 上午
> > Subject: Share Web-Based HAWQ Admin Tool
> > To: dev 
> > Cc: dp_nj 
> >
> >
> >
> >
> > Hi All:
> >
> >
> >
> >
> > In the past month, our team has been adapting pgadmin4 to enable it to be
> > used as a management tool for HAWQ.
> >
> >
> >
> >
> > Now，the pgadmin4 we adapted has been able to meet the basic use. So we
> > plan to share it with the community and hope the uers of HAWQ can use it
> > together.
> >
> >
> >
> >
> > The source code repository can be found here: https://github.com/SkyAI/
> > PGAdmin4HAWQ.git.
> >
> >
> >
> >
> > All the users can post bug reports to us through the email:
> > dp...@iluvatar.ai.
> >
> >
> >
> >
> > We also welcome you to discuss pgAdmin4 adapted for HAWQ, or contribute
> to
> > the project.
> >
> >
> >
> >
> > Best Regards
> >
> >
> >
> >
> >
> >
> > 陈周瞻
> >
> > 平台事业部
> >
> > 南京天数智芯科技有限公司
> >
> > Iluvatar CoreX Inc.
> >
> > Tel:18115151186 web: http://iluvatar.ai
> >
> > 江苏省南京市雨花台区软件大道180号 大数据产基地5栋4层
> >
> > 4F, Building 5, No.180, Ruan Jian Ave, Yuhuatai District, Nanjing,
> Jiangsu
> > Province
> >
> >
> >
> >
>

Apache HAWQ 2.4.0.0-incubating release

[Radar Lei]

Hi All,

As all the planed features/issues are addressed for 2.4.0.0-incubating, I'm
now starting the release processes for Apache HAWQ 2.4.0.0-incubating.

The release branch will be based on the latest master code, including below
new contents:

   - New Feature: Pluggable Vectorized Execution Engine on HAWQ.
   - New Feature: Support Runtime Filter for HAWQ local hash join.
   - Bug fixes.

For release details, please refer to the release wiki page:

   -
   
https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.4.0.0-incubating+Release

Any questions, please feel free to let me know. Thanks.

Regards,
Radar

Re: Re: Re: Revisit plan for Apache HAWQ 2.4.0.0 release with Hive access support in pluggable storage framework

[Radar Lei]

Now seems all the issues in 2.4.0.0-incubating scope are addressed. I plan
to start the release process this week. Thanks all!

Regards,
Radar

On Mon, Jul 9, 2018 at 4:08 PM, Ruilong Huo  wrote:

> Sounds a good idea. Thanks Radar for taking care of that!
>
>
> Best regards,
> Ruilong Huo
> At 2018-07-09 16:02:54, "Radar Lei"  wrote:
> >Hi Ruilong,
> >
> >As there are no further comments on this topic, I will remove the
> >feature 'Support
> >accessing Hive table data by Pluggable Storage Framework' from current
> >Apache HAWQ 2.4.0.0-incubating release.
> >
> >The updated features including in current release are listed as:
> >
> >   - New Feature: Pluggable Vectorized Execution Engine on HAWQ.
> >   - New Feature: Support Runtime Filter for HAWQ local hash join.
> >
> >
> >Since the two list features are all completed, I will start the release
> >process soon.
> >
> >Thanks.
> >
> >
> >Regards,
> >Radar
> >
> >On Tue, Jul 3, 2018 at 9:18 AM, HuoRuilong  wrote:
> >
> >> Exactly, Ed. The plan is to split the work and make them available in
> >> subsequent releases. For now, there are five sub tasks tracking them
> >> (HAWQ-1628 ~ HAWQ-1632). Though we might separate them even more as we
> go
> >> deeper with it.
> >>
> >> In fact, the work for the initial task has been started. We might
> discuss
> >> which releases these features will fit into as we drafting the plan for
> >> future releases.
> >>
> >>
> >> Best regards,
> >> Ruilong
> >>
> >>
> >> At 2018-07-02 22:27:52, "Ed Espino"  wrote:
> >> >Thanks for bring this up Ruilong. Given your input, It is reasonable to
> >> >remove the pluggable storage framework features from 2.4.0.0. To keep
> the
> >> >momentum going, going forward, can we separate the work and
> incrementally
> >> >commit prerequisite work over several releases?
> >> >
> >> >-=e
> >> >
> >> >On Mon, Jul 2, 2018 at 2:34 AM Radar Lei  wrote:
> >> >
> >> >> Hi Ruilong,
> >> >>
> >> >> Thanks for the update, for me, I'm fine with removing this feature
> from
> >> >> 2.4.0.0 release since the scope is bigger than expected.
> >> >>
> >> >> And we still have Vectorized Execution and Runtime Filter features in
> >> this
> >> >> release, they are good enough to make a new release.
> >> >>
> >> >> Regards,
> >> >> Radar
> >> >>
> >> >> On Mon, Jul 2, 2018 at 4:03 PM, Ruilong Huo  wrote:
> >> >>
> >> >> > Hi Radar and All,
> >> >> >
> >> >> >
> >> >> > Had a closer look at the plan for Apache HAWQ 2.4.0.0 release, it
> >> turns
> >> >> > out that the support of Hive data access in pluggable storage
> >> framework
> >> >> has
> >> >> > larger content than expected. In fact, to make Hive protocol
> >> available,
> >> >> the
> >> >> > prerequisite need to be done at first, which including HDFS
> protocol,
> >> >> ORC,
> >> >> > TEXT/CSV format, etc. The corresponding task have been added in
> >> umbrella
> >> >> > feature HAWQ-786 as below:
> >> >> >
> >> >> >
> >> >> > HAWQ-1628:  Support HDFS protocol using pluggable storage framework
> >> >> > HAWQ-1629:  Support ORC format using pluggable storage framework
> >> >> > HAWQ-1630:  Support TEXT/CSV format using pluggable storage
> framework
> >> >> > HAWQ-1631:  Support Hive protocol using pluggable storage framework
> >> >> > HAWQ-1632:  Wrap up pluggable storage framework and HDFS/Hive
> >> >> ORC/TEXT/CSV
> >> >> > support
> >> >> >
> >> >> >
> >> >> > Given that, I think it will be better if we can release 2.4.0.0
> >> without
> >> >> > this feature for now and accommodate it to subsequent releases. Any
> >> >> comment?
> >> >> >
> >> >> >
> >> >> > Best regards,
> >> >> > Ruilong Huo
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >>
> >>
>

Re: Re: Revisit plan for Apache HAWQ 2.4.0.0 release with Hive access support in pluggable storage framework

[Radar Lei]

Hi Ruilong,

As there are no further comments on this topic, I will remove the
feature 'Support
accessing Hive table data by Pluggable Storage Framework' from current
Apache HAWQ 2.4.0.0-incubating release.

The updated features including in current release are listed as:

   - New Feature: Pluggable Vectorized Execution Engine on HAWQ.
   - New Feature: Support Runtime Filter for HAWQ local hash join.


Since the two list features are all completed, I will start the release
process soon.

Thanks.


Regards,
Radar

On Tue, Jul 3, 2018 at 9:18 AM, HuoRuilong  wrote:

> Exactly, Ed. The plan is to split the work and make them available in
> subsequent releases. For now, there are five sub tasks tracking them
> (HAWQ-1628 ~ HAWQ-1632). Though we might separate them even more as we go
> deeper with it.
>
> In fact, the work for the initial task has been started. We might discuss
> which releases these features will fit into as we drafting the plan for
> future releases.
>
>
> Best regards,
> Ruilong
>
>
> At 2018-07-02 22:27:52, "Ed Espino"  wrote:
> >Thanks for bring this up Ruilong. Given your input, It is reasonable to
> >remove the pluggable storage framework features from 2.4.0.0. To keep the
> >momentum going, going forward, can we separate the work and incrementally
> >commit prerequisite work over several releases?
> >
> >-=e
> >
> >On Mon, Jul 2, 2018 at 2:34 AM Radar Lei  wrote:
> >
> >> Hi Ruilong,
> >>
> >> Thanks for the update, for me, I'm fine with removing this feature from
> >> 2.4.0.0 release since the scope is bigger than expected.
> >>
> >> And we still have Vectorized Execution and Runtime Filter features in
> this
> >> release, they are good enough to make a new release.
> >>
> >> Regards,
> >> Radar
> >>
> >> On Mon, Jul 2, 2018 at 4:03 PM, Ruilong Huo  wrote:
> >>
> >> > Hi Radar and All,
> >> >
> >> >
> >> > Had a closer look at the plan for Apache HAWQ 2.4.0.0 release, it
> turns
> >> > out that the support of Hive data access in pluggable storage
> framework
> >> has
> >> > larger content than expected. In fact, to make Hive protocol
> available,
> >> the
> >> > prerequisite need to be done at first, which including HDFS protocol,
> >> ORC,
> >> > TEXT/CSV format, etc. The corresponding task have been added in
> umbrella
> >> > feature HAWQ-786 as below:
> >> >
> >> >
> >> > HAWQ-1628:  Support HDFS protocol using pluggable storage framework
> >> > HAWQ-1629:  Support ORC format using pluggable storage framework
> >> > HAWQ-1630:  Support TEXT/CSV format using pluggable storage framework
> >> > HAWQ-1631:  Support Hive protocol using pluggable storage framework
> >> > HAWQ-1632:  Wrap up pluggable storage framework and HDFS/Hive
> >> ORC/TEXT/CSV
> >> > support
> >> >
> >> >
> >> > Given that, I think it will be better if we can release 2.4.0.0
> without
> >> > this feature for now and accommodate it to subsequent releases. Any
> >> comment?
> >> >
> >> >
> >> > Best regards,
> >> > Ruilong Huo
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
>

Re: BUG:ERROR: Unexpected internal error (appendonlywriter.c:525)

[Radar Lei]

Hi Tony,

Would you fill a issue for better track and it's easier to attach logs and
screenshots? Thanks.

https://issues.apache.org/jira/projects/HAWQ/issues/


Regards,
Radar

On Thu, Jul 5, 2018 at 4:19 PM, tao tony  wrote:

> Sorry for the screenshot not shown in the mail:
>
> hdb=# select version();
>
>   version
>
>
> 
> 
> --
> --
>  PostgreSQL 8.2.15 (Greenplum Database 4.2.0 build 1) (HAWQ
> 2.3.0.0-incubating build dev) on x86_64-unknown-linux-gnu, compiled by GCC
> gcc (GCC) 4.8.5 20150623 (R
> ed Hat 4.8.5-16) compiled on May  4 2018 06:27:27
> (1 row)
>
> hdb=# begin;
> BEGIN
> hdb=# select * from test limit 2;
>  a
> 
>  asdfsdgrtecvxbfgdh
>  asdfsdgrtecvxbfgdh
> (2 rows)
>
> hdb=# truncate table test;
> TRUNCATE TABLE
> hdb=# select * from test limit 2;
>  a
> ---
> (0 rows)
>
> hdb=# alter table test add column b varchar(20) default '';
> ALTER TABLE
> hdb=# commit;
> ERROR:  Unexpected internal error (appendonlywriter.c:525)
> hdb=# rollback;
> WARNING:  there is no transaction in progress
> ROLLBACK
> hdb=#
>
>
> On 07/05/2018 04:17 PM, tao tony wrote:
>
> hi ,
>
> We had found a bug in 2.3.0.0.
>
> It appeared when truncate and alter in the same transaction.
>
> Thanks!
>
> Tao Jin.
>
>
>
>

Re: Revisit plan for Apache HAWQ 2.4.0.0 release with Hive access support in pluggable storage framework

[Radar Lei]

Hi Ruilong,

Thanks for the update, for me, I'm fine with removing this feature from
2.4.0.0 release since the scope is bigger than expected.

And we still have Vectorized Execution and Runtime Filter features in this
release, they are good enough to make a new release.

Regards,
Radar

On Mon, Jul 2, 2018 at 4:03 PM, Ruilong Huo  wrote:

> Hi Radar and All,
>
>
> Had a closer look at the plan for Apache HAWQ 2.4.0.0 release, it turns
> out that the support of Hive data access in pluggable storage framework has
> larger content than expected. In fact, to make Hive protocol available, the
> prerequisite need to be done at first, which including HDFS protocol, ORC,
> TEXT/CSV format, etc. The corresponding task have been added in umbrella
> feature HAWQ-786 as below:
>
>
> HAWQ-1628:  Support HDFS protocol using pluggable storage framework
> HAWQ-1629:  Support ORC format using pluggable storage framework
> HAWQ-1630:  Support TEXT/CSV format using pluggable storage framework
> HAWQ-1631:  Support Hive protocol using pluggable storage framework
> HAWQ-1632:  Wrap up pluggable storage framework and HDFS/Hive ORC/TEXT/CSV
> support
>
>
> Given that, I think it will be better if we can release 2.4.0.0 without
> this feature for now and accommodate it to subsequent releases. Any comment?
>
>
> Best regards,
> Ruilong Huo
>
>
>
>
>
>

Re: [DISCUSS] Graduate Apache HAWQ (incubating) as a TLP

[Radar Lei]

Hi Kuien,

We already see the contributions from Alibaba, and your great presentation
about how Alibaba use and improve HAWQ in last HAWQ meetup. I believe
Alibaba is pretty close to have Apache HAWQ committers.


Regards,
Radar

On Wed, Jun 27, 2018 at 5:13 PM, 刘奎恩(局外)  wrote:

> +1
>
> And will you guys consider to list "Alibaba" into the committer
> affiliations? It may encourage more engineers to join this great project
> community.
>
>
> -——
> Kuien Liu/奎恩
>
>
> --
> 发件人：Ming Li 
> 发送时间：2018年6月27日(星期三) 10:44
> 收件人：dev 
> 主 题：Re: [DISCUSS] Graduate Apache HAWQ (incubating) as a TLP
>
> +1
>
> On Wed, Jun 27, 2018 at 3:05 AM, Don Bosco Durai  wrote:
>
> > +1
> >
> >  On 6/26/18, 6:04 AM, "Hong Wu"  wrote:
> >
> > +1
> >
> > Sent from my iPhone
> >
> > > On Jun 22, 2018, at 11:06 PM, Roman Shaposhnik <
> ro...@shaposhnik.org>
> > wrote:
> > >
> > > +1!
> > >
> > >> On Tue, Jun 19, 2018 at 2:40 AM, Radar Lei 
> wrote:
> > >> Hi All,
> > >>
> > >> With the 2.3.0.0-incubating release officially out, the Apache
> HAWQ
> > >> community and its mentors believe it is time to consider
> graduation
> > to the
> > >> TLP:
> > >>https://lists.apache.org/thread.html/b4a0b5671ce377b3d51c9b7
> > >> ab00496a1eebfcbf1696ce8b67e078c64@%3Cdev.hawq.apache.org%3E
> > >>
> > >> Apache HAWQ entered incubation in September of 2015, since then,
> > the HAWQ
> > >> community learned a lot about how to do things in Apache ways. Now
> > we have
> > >> a healthy and engaged community, ready to help with all questions
> > from the
> > >> HAWQ community. We delivered four releases including two binary
> > releases,
> > >> now we can do self-driving releases in good cadence. The PPMC has
> > >> demonstrated a good understanding of growing the community by
> > electing 12
> > >> individuals as committers and PPMC members. The PPMC addressed the
> > maturity
> > >> issues one by one followed by Apache Project Maturity Model,
> > currently all
> > >> the License and IP issues are resolved. This demonstrated our
> > understanding
> > >> of ASF's IP policies.
> > >>
> > >> All in all, I believe this project is qualified as a true TLP and
> > we should
> > >> recognize this fact by formally awarding it such a status. This
> > thread
> > >> means to open up the very same discussion that we had among the
> > mentors and
> > >> HAWQ community to the rest of the IPMC. It is a DISCUSS thread so
> > feel free
> > >> to ask questions.
> > >>
> > >> To get you all going, here are a few data points which may help:
> > >>
> > >> Project status:
> > >> http://incubator.apache.org/projects/hawq.html
> > >>
> > >> Project website:
> > >>  http://hawq.incubator.apache.org/
> > >>
> > >> Project documentation:
> > >>   http://hawq.incubator.apache.org/docs/userguide/2.3.0.0-inc
> > >> ubating/overview/HAWQOverview.html
> > >>   http://hawq.apache.org/#download
> > >>
> > >> Maturity assessment:
> > >> https://cwiki.apache.org/confluence/display/HAWQ/ASF+
> > Maturity+Evaluation
> > >>
> > >> DRAFT of the board resolution is at the bottom of this email
> > >>
> > >> Proposed PMC size: 45 members
> > >>
> > >> Total number of committers: 45 members
> > >>
> > >> PMC affiliation (* indicated chair):
> > >>   Pivotal (20)
> > >> * Oushu (7)
> > >>   Amazon (3)
> > >>   Hashdata (2)
> > >>   Autonomic (1)
> > >>   Confluent (1)
> > >>   Datometry (1)
> > >>   Hortonworks (1)
> > >>   Microsoft (1)
> > >>   PETUUM (1)
> > >>   Privacera (1)
> > >>   Qubole (1)
> > >>   Snowflake (1)
> > >>   State Street (1)
> > >>   Unifi (1)
> > >>   Visa

[DISCUSS] Graduate Apache HAWQ (incubating) as a TLP

[Radar Lei]

Hi All,

With the 2.3.0.0-incubating release officially out, the Apache HAWQ
community and its mentors believe it is time to consider graduation to the
TLP:
https://lists.apache.org/thread.html/b4a0b5671ce377b3d51c9b7
ab00496a1eebfcbf1696ce8b67e078c64@%3Cdev.hawq.apache.org%3E

Apache HAWQ entered incubation in September of 2015, since then, the HAWQ
community learned a lot about how to do things in Apache ways. Now we have
a healthy and engaged community, ready to help with all questions from the
HAWQ community. We delivered four releases including two binary releases,
now we can do self-driving releases in good cadence. The PPMC has
demonstrated a good understanding of growing the community by electing 12
individuals as committers and PPMC members. The PPMC addressed the maturity
issues one by one followed by Apache Project Maturity Model, currently all
the License and IP issues are resolved. This demonstrated our understanding
of ASF's IP policies.

All in all, I believe this project is qualified as a true TLP and we should
recognize this fact by formally awarding it such a status. This thread
means to open up the very same discussion that we had among the mentors and
HAWQ community to the rest of the IPMC. It is a DISCUSS thread so feel free
to ask questions.

To get you all going, here are a few data points which may help:

Project status:
 http://incubator.apache.org/projects/hawq.html

Project website:
  http://hawq.incubator.apache.org/

Project documentation:
   http://hawq.incubator.apache.org/docs/userguide/2.3.0.0-inc
ubating/overview/HAWQOverview.html
   http://hawq.apache.org/#download

Maturity assessment:
https://cwiki.apache.org/confluence/display/HAWQ/ASF+Maturity+Evaluation

DRAFT of the board resolution is at the bottom of this email

Proposed PMC size: 45 members

Total number of committers: 45 members

PMC affiliation (* indicated chair):
   Pivotal (20)
 * Oushu (7)
   Amazon (3)
   Hashdata (2)
   Autonomic (1)
   Confluent (1)
   Datometry (1)
   Hortonworks (1)
   Microsoft (1)
   PETUUM (1)
   Privacera (1)
   Qubole (1)
   Snowflake (1)
   State Street (1)
   Unifi (1)
   Visa (1)
   ZEDEDA (1)

1549 commits on develop
1375 PR”s on GitHub
63 contributors across all branches

1624 issues created
1350 issues resolved

dev list averaged ~53 msgs/month over last 12 months
user list averaged ~6 msgs/month over last 12 months
129 unique posters


committer affiliations:
active
  pivotal.io
  oushu.io
  hashdata.cn
occasional
  amazon.com
  autonomic.ai
  confluent.io
  datometry.com
  hortonworks.com
  microsoft.com
  petuum.com
  privacera.com
  qubole.com
  snowflake.net
  statestreet.com
  unifisoftware.com
  visa.com
  zededa.com


Thanks,
Radar



## Resolution to create a TLP from graduating Incubator podling

X. Establish the Apache HAWQ Project

   WHEREAS, the Board of Directors deems it to be in the best
   interests of the Foundation and consistent with the
   Foundation's purpose to establish a Project Management
   Committee charged with the creation and maintenance of
   open-source software, for distribution at no charge to
   the public, related to Hadoop native SQL query engine that
   combines the key technological advantages of MPP database
   with the scalability and convenience of Hadoop.

   NOW, THEREFORE, BE IT RESOLVED, that a Project Management
   Committee (PMC), to be known as the "Apache HAWQ Project",
   be and hereby is established pursuant to Bylaws of the
   Foundation; and be it further

   RESOLVED, that the Apache HAWQ Project be and hereby is
   responsible for the creation and maintenance of software
   related to Hadoop native SQL query engine that
   combines the key technological advantages of MPP database
   with the scalability and convenience of Hadoop;
   and be it further

   RESOLVED, that the office of "Vice President, Apache HAWQ" be
   and hereby is created, the person holding such office to
   serve at the direction of the Board of Directors as the chair
   of the Apache HAWQ Project, and to have primary responsibility
   for management of the projects within the scope of
   responsibility of the Apache HAWQ Project; and be it further

   RESOLVED, that the persons listed immediately below be and
   hereby are appointed to serve as the initial members of the
   Apache HAWQ Project:

* Alan Gates   
* Alexander Denissov   
* Amy Bai  
* Atri Sharma  
* Bhuvnesh Chaudhary   
* Bosco
* Chunling Wang
* David Yozie  
* Ed Espino
* Entong Shen  
* Foyzur Rahman
* Goden Yao
* Gregory 

Re: [DISCUSS] Apache HAWQ Graduation from Incubator

[Radar Lei]

Thanks Bosco.

I also did some research by Linkedin, now I have a full picture of the
committer/PMC affiliations.

Regards,
Radar

On Thu, Jun 14, 2018 at 10:36 PM, Don Bosco Durai  wrote:

> Hi Radar
>
> Mine is "Privacera"
>
> Thanks
>
> Bosco
>
>
> On 6/14/18, 1:08 AM, "Radar Lei"  wrote:
>
> Hi Guys,
>
> I will keep the affiliation topic till next Monday, you can also ping
> me
> your affiliation so it's name can be list on the resolution.
>
> BTW, thanks Roman, Alan and Atri's share.
>
>
> Regards,
> Radar
>
>
> On Wed, Jun 13, 2018 at 11:47 AM Alan Gates 
> wrote:
>
> > Alan Gates - Hortonworks.
> >
> > Alan.
> >
> > On Tue, Jun 12, 2018 at 7:37 PM Radar Lei  wrote:
> >
> >> Hi!
> >>
> >> I'm counting the PMC/Committer affiliation, please let me know
> what's
> >> your affiliation if convenience. Thanks.
> >>
> >> E.g.
> >> PMC affiliation
> >>Pivotal (22)
> >>Oushu (7)
> >>Hashdata (2)
> >>Amazon(1)
> >>
> >>
> >> Regards,
> >> Radar
> >>
> >> On Mon, Jun 11, 2018 at 5:12 PM, Radar Lei  wrote:
> >>
> >>> Thanks all. Now I'm pretty close to finish drafting the
> resolution, will
> >>> send it out soon.
> >>>
> >>> Regards,
> >>> Radar
> >>>
> >>> On Thu, Jun 7, 2018 at 11:24 AM, Lili Ma 
> wrote:
> >>>
> >>>> Thanks Radar for your great efforts for preparation for HAWQ
> graduation!
> >>>>
> >>>> I'm very glad to see our product having coming to the phase of
> >>>> graduation
> >>>> discussion, and I believe we are always a team no matter where we
> work
> >>>> at
> >>>> now.  I hope we can soon graduate HAWQ as a TLP :)
> >>>>
> >>>> As for chairman, I support Lei Chang too. He initiated the idea of
> >>>> putting
> >>>> SQL on top of HDFS and lead us through the process of HAWQ Alpha,
> 1.0
> >>>> version, 2.0 version, open source, etc.  And he is still working
> on this
> >>>> project actively :)
> >>>>
> >>>> Best Regards,
> >>>> Lili Ma
> >>>>
> >>>>
> >>>> 2018-06-04 8:35 GMT-04:00 Lei Chang :
> >>>>
> >>>> > Thank you guys for the nomination :-).
> >>>> >
> >>>> > Great to work with the team on the HAWQ graduation and any
> future
> >>>> issues.
> >>>> >
> >>>> > Cheers
> >>>> > Lei
> >>>> >
> >>>> >
> >>>> >
> >>>> > On Fri, Jun 1, 2018 at 9:50 AM, 陶征霖 
> wrote:
> >>>> >
> >>>> > > +1 to Lei. He is worthy of the title of Apache HAWQ PMC
> Chairman.
> >>>> > >
> >>>> > > 2018-06-01 9:44 GMT+08:00 Yi JIN :
> >>>> > >
> >>>> > > > +1 to Lei, I fully support Lei Change as Apache HAWQ
> project PMC
> >>>> > > Chairman,
> >>>> > > > he deserves this role, not only outstanding contributions
> to this
> >>>> > project
> >>>> > > > in a very long period till today also his solid leadership
> and
> >>>> vision.
> >>>> > > >
> >>>> > > > Best,
> >>>> > > > Yi  Jin
> >>>> > > >
> >>>> > > > On Fri, Jun 1, 2018 at 3:17 AM, Ed Espino <
> esp...@apache.org>
> >>>> wrote:
> >>>> > > >
> >>>> > > > > Ruilong,
> >>>> > > > >
> >>>> > > > > I also give my full support for Lei Chang as the Apache
> HAWQ
> >>>> > project's
> >>>> > > > > initial PMC Chairman. His leadership and vision have
> contributed
> >>>> > > > immensely
> >>>> > &g

Re: Vectorized Execution plugin lib has been released

[Radar Lei]

Awesome! 2 - 8X speed up! The performance improvement is impressive, thanks
Weinan and Shujie who delivered this great feature.

With Vectorized Execution and Runtime filter feature finished, we have
completed 2 of 3 features targets for HAWQ 2.4.0.0-incubating release:

   - [Done] New Feature: Pluggable Vectorized Execution Engine on HAWQ.
   - [Done] New Feature: Support Runtime Filter for HAWQ local hash join.
   - New Feature: Support accessing Hive table data by Pluggable Storage
   Framework.



Regards,
Radar

On Tue, Jun 19, 2018 at 2:18 PM, Weinan Wang  wrote:

> Greetings,
>
> I am glad to announce that Vectorized Execution plugin has been completed
> released in HAWQ.
>
>  At present, it supports 7 types of data trigger the vectorized function
> and reach two to eight times
> acceleration comparing with regular execution in different performance
> test queries. Furthermore,
> it has an extended friendly framework, you can patch other types into as
> your need.
>
> You can acquire the introduction document  as follow:
> https://cwiki.apache.org/confluence/display/HAWQ/Vectorized+Execution+
> Introduction  Vectorized+Execution+Introduction>
>
>
>
>
> Regards,
>
> WANG Weinan

Re: Re: [DISCUSS] Apache HAWQ Graduation from Incubator

[Radar Lei]

Hi Guys,

I will keep the affiliation topic till next Monday, you can also ping me
your affiliation so it's name can be list on the resolution.

BTW, thanks Roman, Alan and Atri's share.


Regards,
Radar


On Wed, Jun 13, 2018 at 11:47 AM Alan Gates  wrote:

> Alan Gates - Hortonworks.
>
> Alan.
>
> On Tue, Jun 12, 2018 at 7:37 PM Radar Lei  wrote:
>
>> Hi!
>>
>> I'm counting the PMC/Committer affiliation, please let me know what's
>> your affiliation if convenience. Thanks.
>>
>> E.g.
>> PMC affiliation
>>Pivotal (22)
>>Oushu (7)
>>Hashdata (2)
>>    Amazon(1)
>>
>>
>> Regards,
>> Radar
>>
>> On Mon, Jun 11, 2018 at 5:12 PM, Radar Lei  wrote:
>>
>>> Thanks all. Now I'm pretty close to finish drafting the resolution, will
>>> send it out soon.
>>>
>>> Regards,
>>> Radar
>>>
>>> On Thu, Jun 7, 2018 at 11:24 AM, Lili Ma  wrote:
>>>
>>>> Thanks Radar for your great efforts for preparation for HAWQ graduation!
>>>>
>>>> I'm very glad to see our product having coming to the phase of
>>>> graduation
>>>> discussion, and I believe we are always a team no matter where we work
>>>> at
>>>> now.  I hope we can soon graduate HAWQ as a TLP :)
>>>>
>>>> As for chairman, I support Lei Chang too. He initiated the idea of
>>>> putting
>>>> SQL on top of HDFS and lead us through the process of HAWQ Alpha, 1.0
>>>> version, 2.0 version, open source, etc.  And he is still working on this
>>>> project actively :)
>>>>
>>>> Best Regards,
>>>> Lili Ma
>>>>
>>>>
>>>> 2018-06-04 8:35 GMT-04:00 Lei Chang :
>>>>
>>>> > Thank you guys for the nomination :-).
>>>> >
>>>> > Great to work with the team on the HAWQ graduation and any future
>>>> issues.
>>>> >
>>>> > Cheers
>>>> > Lei
>>>> >
>>>> >
>>>> >
>>>> > On Fri, Jun 1, 2018 at 9:50 AM, 陶征霖  wrote:
>>>> >
>>>> > > +1 to Lei. He is worthy of the title of Apache HAWQ PMC Chairman.
>>>> > >
>>>> > > 2018-06-01 9:44 GMT+08:00 Yi JIN :
>>>> > >
>>>> > > > +1 to Lei, I fully support Lei Change as Apache HAWQ project PMC
>>>> > > Chairman,
>>>> > > > he deserves this role, not only outstanding contributions to this
>>>> > project
>>>> > > > in a very long period till today also his solid leadership and
>>>> vision.
>>>> > > >
>>>> > > > Best,
>>>> > > > Yi  Jin
>>>> > > >
>>>> > > > On Fri, Jun 1, 2018 at 3:17 AM, Ed Espino 
>>>> wrote:
>>>> > > >
>>>> > > > > Ruilong,
>>>> > > > >
>>>> > > > > I also give my full support for Lei Chang as the Apache HAWQ
>>>> > project's
>>>> > > > > initial PMC Chairman. His leadership and vision have contributed
>>>> > > > immensely
>>>> > > > > to the project.
>>>> > > > >
>>>> > > > > Regards,
>>>> > > > > -=e
>>>> > > > >
>>>> > > > > On Thu, May 31, 2018 at 6:32 AM, Ruilong Huo 
>>>> > wrote:
>>>> > > > >
>>>> > > > > > Great progress towards Apache HAWQ graduation! Thanks Radar
>>>> for
>>>> > > pushing
>>>> > > > > > this forward!
>>>> > > > > >
>>>> > > > > > I would like to nominate Lei Chang as the PMC Chair. He
>>>> initiated
>>>> > > HAWQ
>>>> > > > > > project several years ago, led the development, brought it to
>>>> > Apache
>>>> > > > > > incubation, and has always been active in HAWQ community to
>>>> make
>>>> > it a
>>>> > > > > > world-leading big data product as well as a successfully
>>>> Apache
>>>> > > > project.
>>>> > > > > > There is no doubt that he is perfect for the role. And I
>>>> believe he
>

Re: Re: [DISCUSS] Apache HAWQ Graduation from Incubator

[Radar Lei]

Hi!

I'm counting the PMC/Committer affiliation, please let me know what's your
affiliation if convenience. Thanks.

E.g.
PMC affiliation
   Pivotal (22)
   Oushu (7)
   Hashdata (2)
   Amazon(1)


Regards,
Radar

On Mon, Jun 11, 2018 at 5:12 PM, Radar Lei  wrote:

> Thanks all. Now I'm pretty close to finish drafting the resolution, will
> send it out soon.
>
> Regards,
> Radar
>
> On Thu, Jun 7, 2018 at 11:24 AM, Lili Ma  wrote:
>
>> Thanks Radar for your great efforts for preparation for HAWQ graduation!
>>
>> I'm very glad to see our product having coming to the phase of graduation
>> discussion, and I believe we are always a team no matter where we work at
>> now.  I hope we can soon graduate HAWQ as a TLP :)
>>
>> As for chairman, I support Lei Chang too. He initiated the idea of putting
>> SQL on top of HDFS and lead us through the process of HAWQ Alpha, 1.0
>> version, 2.0 version, open source, etc.  And he is still working on this
>> project actively :)
>>
>> Best Regards,
>> Lili Ma
>>
>>
>> 2018-06-04 8:35 GMT-04:00 Lei Chang :
>>
>> > Thank you guys for the nomination :-).
>> >
>> > Great to work with the team on the HAWQ graduation and any future
>> issues.
>> >
>> > Cheers
>> > Lei
>> >
>> >
>> >
>> > On Fri, Jun 1, 2018 at 9:50 AM, 陶征霖  wrote:
>> >
>> > > +1 to Lei. He is worthy of the title of Apache HAWQ PMC Chairman.
>> > >
>> > > 2018-06-01 9:44 GMT+08:00 Yi JIN :
>> > >
>> > > > +1 to Lei, I fully support Lei Change as Apache HAWQ project PMC
>> > > Chairman,
>> > > > he deserves this role, not only outstanding contributions to this
>> > project
>> > > > in a very long period till today also his solid leadership and
>> vision.
>> > > >
>> > > > Best,
>> > > > Yi  Jin
>> > > >
>> > > > On Fri, Jun 1, 2018 at 3:17 AM, Ed Espino 
>> wrote:
>> > > >
>> > > > > Ruilong,
>> > > > >
>> > > > > I also give my full support for Lei Chang as the Apache HAWQ
>> > project's
>> > > > > initial PMC Chairman. His leadership and vision have contributed
>> > > > immensely
>> > > > > to the project.
>> > > > >
>> > > > > Regards,
>> > > > > -=e
>> > > > >
>> > > > > On Thu, May 31, 2018 at 6:32 AM, Ruilong Huo 
>> > wrote:
>> > > > >
>> > > > > > Great progress towards Apache HAWQ graduation! Thanks Radar for
>> > > pushing
>> > > > > > this forward!
>> > > > > >
>> > > > > > I would like to nominate Lei Chang as the PMC Chair. He
>> initiated
>> > > HAWQ
>> > > > > > project several years ago, led the development, brought it to
>> > Apache
>> > > > > > incubation, and has always been active in HAWQ community to make
>> > it a
>> > > > > > world-leading big data product as well as a successfully Apache
>> > > > project.
>> > > > > > There is no doubt that he is perfect for the role. And I
>> believe he
>> > > > will
>> > > > > > continue to share his insights and go even further with HAWQ
>> after
>> > > > > > graduation.
>> > > > > >
>> > > > > >
>> > > > > > Best regards,
>> > > > > > Ruilong Huo
>> > > > > >
>> > > > > >
>> > > > > > At 2018-05-31 15:40:42, "Radar Lei"  wrote:
>> > > > > > >Just find some good material for nominating chair from Roman's
>> > > email,
>> > > > > > >thanks Roman.
>> > > > > > >
>> > > > > > >I think we can follow this to nominate a Chair in this thread
>> too.
>> > > > Guys
>> > > > > > >please help to nominate or self-nominate. Thanks a lot.
>> > > > > > >
>> > > > > > >See:
>> > > > > > >At the very minimum your resolution will contain: 1. A name of
>> the
>> > > > > project
>> > > > > > >2. A list of proposed PMC 3. A proposed PMC chair
>> > > > > > >
>

Re: Re: [DISCUSS] Apache HAWQ Graduation from Incubator

[Radar Lei]

Thanks all. Now I'm pretty close to finish drafting the resolution, will
send it out soon.

Regards,
Radar

On Thu, Jun 7, 2018 at 11:24 AM, Lili Ma  wrote:

> Thanks Radar for your great efforts for preparation for HAWQ graduation!
>
> I'm very glad to see our product having coming to the phase of graduation
> discussion, and I believe we are always a team no matter where we work at
> now.  I hope we can soon graduate HAWQ as a TLP :)
>
> As for chairman, I support Lei Chang too. He initiated the idea of putting
> SQL on top of HDFS and lead us through the process of HAWQ Alpha, 1.0
> version, 2.0 version, open source, etc.  And he is still working on this
> project actively :)
>
> Best Regards,
> Lili Ma
>
>
> 2018-06-04 8:35 GMT-04:00 Lei Chang :
>
> > Thank you guys for the nomination :-).
> >
> > Great to work with the team on the HAWQ graduation and any future issues.
> >
> > Cheers
> > Lei
> >
> >
> >
> > On Fri, Jun 1, 2018 at 9:50 AM, 陶征霖  wrote:
> >
> > > +1 to Lei. He is worthy of the title of Apache HAWQ PMC Chairman.
> > >
> > > 2018-06-01 9:44 GMT+08:00 Yi JIN :
> > >
> > > > +1 to Lei, I fully support Lei Change as Apache HAWQ project PMC
> > > Chairman,
> > > > he deserves this role, not only outstanding contributions to this
> > project
> > > > in a very long period till today also his solid leadership and
> vision.
> > > >
> > > > Best,
> > > > Yi  Jin
> > > >
> > > > On Fri, Jun 1, 2018 at 3:17 AM, Ed Espino  wrote:
> > > >
> > > > > Ruilong,
> > > > >
> > > > > I also give my full support for Lei Chang as the Apache HAWQ
> > project's
> > > > > initial PMC Chairman. His leadership and vision have contributed
> > > > immensely
> > > > > to the project.
> > > > >
> > > > > Regards,
> > > > > -=e
> > > > >
> > > > > On Thu, May 31, 2018 at 6:32 AM, Ruilong Huo 
> > wrote:
> > > > >
> > > > > > Great progress towards Apache HAWQ graduation! Thanks Radar for
> > > pushing
> > > > > > this forward!
> > > > > >
> > > > > > I would like to nominate Lei Chang as the PMC Chair. He initiated
> > > HAWQ
> > > > > > project several years ago, led the development, brought it to
> > Apache
> > > > > > incubation, and has always been active in HAWQ community to make
> > it a
> > > > > > world-leading big data product as well as a successfully Apache
> > > > project.
> > > > > > There is no doubt that he is perfect for the role. And I believe
> he
> > > > will
> > > > > > continue to share his insights and go even further with HAWQ
> after
> > > > > > graduation.
> > > > > >
> > > > > >
> > > > > > Best regards,
> > > > > > Ruilong Huo
> > > > > >
> > > > > >
> > > > > > At 2018-05-31 15:40:42, "Radar Lei"  wrote:
> > > > > > >Just find some good material for nominating chair from Roman's
> > > email,
> > > > > > >thanks Roman.
> > > > > > >
> > > > > > >I think we can follow this to nominate a Chair in this thread
> too.
> > > > Guys
> > > > > > >please help to nominate or self-nominate. Thanks a lot.
> > > > > > >
> > > > > > >See:
> > > > > > >At the very minimum your resolution will contain: 1. A name of
> the
> > > > > project
> > > > > > >2. A list of proposed PMC 3. A proposed PMC chair
> > > > > > >
> > > > > > >On #3 I typically recommend podlings I mento to setup a rotating
> > > chair
> > > > > > >policy. This is, in no way, an ASF requirement so feel free to
> > > ignore
> > > > > it,
> > > > > > >but it worked well before. The chair will be expected up for
> > > rotation
> > > > > > every
> > > > > > >year. It will be more that ok for the same person to
> self-nominate
> > > > once
> > > > > > the
> > > > > > >year is up -- but at the same time it'll be up to the same
> person
> > to
> > > > > &

Re: Remain with HAWQ project or not?

[Radar Lei]

I'd like to add 'Don Bosco Durai ' to the HAWQ PMC
proposal list, he missed this email due to a long vacation. Now he is
replying his interests to continue contributing to HAWQ project, especially
in security side.

I think this should be fine as we have not made the officially Graduation
Resolution yet. Please let me know if you have any questions about that.
Thanks.

Regards,
Radar

On Mon, May 28, 2018 at 8:48 AM, Hongxu Ma  wrote:

> thanks Radar!
>
> 在 26/05/2018 00:26, Ed Espino 写道:
> > Radar, Nice work coordinating this important project task!
> >
> > 谢谢 (Thank you),
> > -=e
> >
> > On Fri, May 25, 2018 at 12:18 AM, Radar Lei  wrote:
> >
> >> Thanks all for your responses! It's really great to see so many people
> >> willing to remain with HAWQ project!
> >>
> >> Now we will end this thread since more than two weeks passed.
> >>
> >> As a summary of all the responses:
> >>
> >> - We have 42 committers are willing to remain with HAWQ project.
> >> - There are 19 committers either respond 'NO' or do not have
> responses.
> >> - We will have two mentors join the PMC: Roman and Alan have
> confirmed
> >> to stay for helping after graduation.
> >>
> >>
> >> Below is the detail list of who will remain in Apache HAWQ project as
> >> committer/PMC members:
> >>
> >> Alan Gates
> >> Alexander Denissov
> >> Amy Bai
> >> Atri Sharma
> >> Bhuvnesh Chaudhary
> >> Chunling Wang
> >> David Yozie
> >> Ed Espino
> >> Entong Shen
> >> Foyzur Rahman
> >> Goden Yao
> >> Gregory Chase
> >> Hong Wu
> >> Hongxu Ma
> >> Hubert Zhang
> >> Ivan Weng
> >> Jesse Zhang
> >> Jiali Yao
> >> Jun Aoki
> >> Kavinder Dhaliwal
> >> Lav Jain
> >> Lei Chang
> >> Lili Ma
> >> Lirong Jian
> >> Lisa Owen
> >> Ming Li
> >> Mohamed Soliman
> >> Newton Alex
> >> Noa Horn
> >> Oleksandr Diachenko
> >> Paul Guo
> >> Radar Da Lei
> >> Roman Shaposhnik
> >> Ruilong Huo
> >> Shivram Mani
> >> Shubham Sharma
> >> Tushar Pednekar
> >> Venkatesh Raghavan
> >> Vineet Goel
> >> Wen Lin
> >> Xiang Sheng
> >> Yi Jin
> >> Zhanwei Wang
> >> Zhenglin Tao
> >>
> >> Please feel free to let me know if you have any questions with this
> result.
> >> Thanks.
> >>
> >> Regards,
> >> Radar
> >>
> >> On Tue, May 22, 2018 at 2:43 PM, Radar Lei  wrote:
> >>
> >>> Forward responds I received in individual mails in recent days:
> >>>
> >>> Yes:
> >>> Vineet Goel
> >>> Jesse Zhang
> >>>
> >>> No:
> >>> Owen O'Malley
> >>>
> >>>
> >>>
> >>> Regards,
> >>> Radar
> >>>
> >>> On Mon, May 21, 2018 at 12:20 PM, Greg Chase  wrote:
> >>>
> >>>> Yes :-)
> >>>>
> >>>> This email encrypted by tiny buttons & fat thumbs, beta voice
> >>>> recognition, and autocorrect on my iPhone.
> >>>>
> >>>>> On May 14, 2018, at 9:41 PM, Radar Lei  wrote:
> >>>>>
> >>>>> FWD responds I received in individual mails:
> >>>>>
> >>>>> Yes:
> >>>>> Goden Yao
> >>>>> Kavinder Dhaliwal
> >>>>> Mohamed Soliman
> >>>>> Newton Alex
> >>>>> Oleksandr Diachenko
> >>>>> Tushar Pednekar
> >>>>> Venkatesh Raghavan
> >>>>> David Yozie
> >>>>>
> >>>>> No:
> >>>>> Mel Kiyama
> >>>>>
> >>>>>
> >>>>> Regards,
> >>>>> Radar
> >>>>>
> >>>>>> On Mon, May 14, 2018 at 6:18 AM, Entong Shen 
> >>>> wrote:
> >>>>>> Yes.
> >>>>>>
> >>>>>>> On 2018/05/07 08:11:53, Radar Lei  wrote:
> >>>>>>> HAWQ committers,
> >>>>>>>
> >>>>>>> Per the discussion in "Apache HAWQ graduation from incubator?" [1],
> >> we
> >>>>>> want
> >>>>>>> to setup the PMC as part of HAWQ graduation resolution.
> >>>>>>>
> >>>>>>> So we'd like to confirm whether you want to remain as a
> >> committer/PMC
> >>>>>>> member of Apache HAWQ project?
> >>>>>>>
> >>>>>>> If you'd like to remain with HAWQ project, it's welcome and please
> >>>>>> *respond**
> >>>>>>> 'Yes'* in this thread, or *respond 'No'* if you are not interested
> >> in
> >>>> any
> >>>>>>> more. Thanks.
> >>>>>>>
> >>>>>>> This thread will be available for at least 72 hours, after that, we
> >>>> will
> >>>>>>> send individual confirm emails.
> >>>>>>>
> >>>>>>> [1]
> >>>>>>> https://lists.apache.org/thread.html/b4a0b5671ce377b3d51c9b7
> ab00496
> >>>>>> a1eebfcbf1696ce8b67e078c64@%3Cdev.hawq.apache.org%3E
> >>>>>>> Regards,
> >>>>>>> Radar
> >>>>>>>
> >>>
>
> --
> Regards,
> Hongxu.
>
>

Re: [DISCUSS] Apache HAWQ Graduation from Incubator

[Radar Lei]

Just find some good material for nominating chair from Roman's email,
thanks Roman.

I think we can follow this to nominate a Chair in this thread too. Guys
please help to nominate or self-nominate. Thanks a lot.

See:
At the very minimum your resolution will contain: 1. A name of the project
2. A list of proposed PMC 3. A proposed PMC chair

On #3 I typically recommend podlings I mento to setup a rotating chair
policy. This is, in no way, an ASF requirement so feel free to ignore it,
but it worked well before. The chair will be expected up for rotation every
year. It will be more that ok for the same person to self-nominate once the
year is up -- but at the same time it'll be up to the same person to
actually kick off a thread asking if anybody else is interested in serving
as a chair for the next year. Of course, if there multiple candidates there
will have to be a vote.


Regards,
Radar

On Tue, May 29, 2018 at 10:05 PM, Radar Lei  wrote:

> Thanks Roman.
>
> This make sense, I will start to draft the resolution. BTW, we would need
> to nominate a chair, I guess it's the last piece we need to draft the
> resolution.
>
> Regards,
> Radar
>
> On Tue, May 29, 2018 at 11:24 AM, Roman Shaposhnik 
> wrote:
>
>> On Sun, May 27, 2018 at 11:37 PM, Radar Lei  wrote:
>> > Hi Roman,
>> >
>> > We have confirmed with each HAWQ committer whether they want to remain
>> with
>> > HAWQ project.  As a summary, 37 PPMC members(including two mentors) and
>> 7
>> > committers confirmed they want to remain with HAWQ. [1] The total
>> committers
>> > number 44 seems pretty close with PPMC member number 37, is it good
>> enough
>> > to make PMC == committers as our graduation resolution?
>>
>> PMC == committers in this case makes perfect sense to me!
>>
>> > Should we update the whimsy and project webpage now or do update after
>> graduation?
>>
>> It really doesn't matter much. Your next step is to draft a resolution
>> similar to:
>> https://www.mail-archive.com/general@incubator.apache.org/ms
>> g56982.html
>>
>> and start a [DISCUSS] thread similar to the above.
>>
>> Makes sense?
>>
>> Thanks,
>> Roman.
>>
>
>

Re: how hawq use HA hdfs

[Radar Lei]

Seems like you set the 'dfs.nameservices' as skydata, but not 'dx' which
you defined in hawq-site.xml.

Regards,
Radar

On Thu, May 31, 2018 at 11:22 AM,  wrote:

> I had, here is related configs in hdfs-client.xml:
>
>
>  
> dfs.nameservices
> skydata
> 
>
> 
> dfs.ha.namenodes.skydata
> nn1,nn2
> 
>
> 
> dfs.namenode.rpc-address.skydata.nn1
> 192.168.60.24:8020
> 
>
> 
> dfs.namenode.rpc-address.skydata.nn2
> 192.168.60.32:8020
> 
>
> 
> dfs.namenode.http-address.skydata.nn1
> 192.168.60.24:50070
> 
> 
> dfs.namenode.http-address.skydata.nn2
>         192.168.60.32:50070
> 
>
>
> --
> *From: *"Radar Lei" 
> *To: *"user" 
> *Cc: *"dev" 
> *Sent: *Thursday, May 31, 2018 10:27:21 AM
>
> *Subject: *Re: how hawq use HA hdfs
>
> Have you made changes to  HAWQ configuration file 'hdfs-client.xml'?
>
> Regards,
> Radar
>
> On Thu, May 31, 2018 at 10:07 AM,  wrote:
>
>> Change the following parameter in the $GPHOME/etc/hawq-site.xml file:
>>
>> 
>> hawq_dfs_url
>> hdpcluster/hawq_default
>> URL for accessing HDFS.
>>
>> In the listing above:
>>
>>- Replace hdpcluster with the actual service ID that is configured in
>>HDFS.
>>- Replace /hawq_default with the directory you want to use for
>>storing data on HDFS. Make sure this directory exists and is writable.
>>
>> In my case, i think that hdpcluster is "dx" which defined in
>> core-site.xml.
>>
>> So i use "dx/hawq_default" but failed.
>>
>> Could anyone help me about this?
>>
>> Thanks very much.
>>
>>
>> --
>> *From: *"Radar Lei" 
>> *To: *"user" , "dev" <
>> dev@hawq.incubator.apache.org>
>> *Sent: *Thursday, May 31, 2018 9:50:39 AM
>>
>> *Subject: *Re: how hawq use HA hdfs
>>
>> If you are installing a new HAWQ, then file space move is not required.
>> I think  HAWQ will treat the host string as an url unless you configured
>> HAWQ hdfs HA correctly. So please verify if you missed any other steps.
>>
>> Regards,
>> Radar
>>
>> On Thu, May 31, 2018 at 9:40 AM,  wrote:
>>
>>> I think that move filespace is the action that hawq has installed before
>>> HA hdfs,
>>> but in my case, i installed HA hdfs then installed hawq.
>>>
>>> In this way, i think there is no need to move filespace.
>>>
>>> Besides, my question is that why URI can not work as expected?
>>> Or which URI i should use?
>>>
>>> --
>>> *From: *"Radar Lei" 
>>> *To: *"user" 
>>> *Sent: *Wednesday, May 30, 2018 8:30:01 PM
>>> *Subject: *Re: how hawq use HA hdfs
>>>
>>> As the steps in the document you mentioned, you need to finish all the
>>> steps to convert your HAWQ cluster to HA mode.
>>> E.g. Do filespace move, make changes in hdfs-client.xml etc.
>>>
>>> Regards,
>>> Radar
>>>
>>> On Wed, May 30, 2018 at 7:44 PM,  wrote:
>>>
>>>> Hi!
>>>>
>>>> I has setting HA hdfs using ZKFC with below config:
>>>>
>>>> 
>>>> 
>>>> fs.defaultFS
>>>> hdfs://dx
>>>> 
>>>>  
>>>>ha.zookeeper.quorum
>>>>192.168.60.24 :2181,192.168.60.32
>>>> :2181,192.168.60.37
>>>> :2181
>>>>  
>>>> 
>>>>
>>>> Then i install hawq in another node(not any node installed hdfs).
>>>>
>>>> I follow http://hawq.incubator.apache.org/docs/userguide/2.3.0.0-
>>>> incubating/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html
>>>> to set `hawq_dfs_url` as "dx/hawq_default", but init failed when check
>>>> if hdfs path is available:
>>>> [WARNING]:-ERROR: Can not connect to 'hdfs://dx:0'
>>>>
>>>> Then i change `hawq_dfs_url` to "dx:8020/hawq_default", still failed:
>>>> ERROR Failed to setup RPC connection to "dx:8020" caused by:
>>>> TcpSocket.cpp: 171: HdfsNetworkConnectException: Failed to resolve
>>>> address "dx:8020" Name or service not known
>>>>
>>>> It consider "dx" as hostname, which is not expected.
>>>>
>>>> I also test "hdfs://dx", "file://dx", all failed.
>>>>
>>>> I know that "192.168.60.24 :8020" can work, but
>>>> it does not use HA hdfs.
>>>>
>>>> How can hawq use HA hdfs?
>>>>
>>>> Thanks
>>>>
>>>
>>>
>>> --
>>> 戴翔
>>> 南京天数信息科技有限公司
>>> 电话: +86 1 3382776490
>>> 公司官网: www.sky-data.cn
>>> 免费使用天数润科智能计算平台 SkyDiscovery
>>>
>>
>>
>> --
>> 戴翔
>> 南京天数信息科技有限公司
>> 电话: +86 1 3382776490
>> 公司官网: www.sky-data.cn
>> 免费使用天数润科智能计算平台 SkyDiscovery
>>
>
>
> --
> 戴翔
> 南京天数信息科技有限公司
> 电话: +86 1 3382776490
> 公司官网: www.sky-data.cn
> 免费使用天数润科智能计算平台 SkyDiscovery
>

Re: how hawq use HA hdfs

[Radar Lei]

Have you made changes to  HAWQ configuration file 'hdfs-client.xml'?

Regards,
Radar

On Thu, May 31, 2018 at 10:07 AM,  wrote:

> Change the following parameter in the $GPHOME/etc/hawq-site.xml file:
>
> 
> hawq_dfs_url
> hdpcluster/hawq_default
> URL for accessing HDFS.
>
> In the listing above:
>
>- Replace hdpcluster with the actual service ID that is configured in
>HDFS.
>- Replace /hawq_default with the directory you want to use for storing
>data on HDFS. Make sure this directory exists and is writable.
>
> In my case, i think that hdpcluster is "dx" which defined in
> core-site.xml.
>
> So i use "dx/hawq_default" but failed.
>
> Could anyone help me about this?
>
> Thanks very much.
>
>
> --
> *From: *"Radar Lei" 
> *To: *"user" , "dev" <
> dev@hawq.incubator.apache.org>
> *Sent: *Thursday, May 31, 2018 9:50:39 AM
>
> *Subject: *Re: how hawq use HA hdfs
>
> If you are installing a new HAWQ, then file space move is not required.
> I think  HAWQ will treat the host string as an url unless you configured
> HAWQ hdfs HA correctly. So please verify if you missed any other steps.
>
> Regards,
> Radar
>
> On Thu, May 31, 2018 at 9:40 AM,  wrote:
>
>> I think that move filespace is the action that hawq has installed before
>> HA hdfs,
>> but in my case, i installed HA hdfs then installed hawq.
>>
>> In this way, i think there is no need to move filespace.
>>
>> Besides, my question is that why URI can not work as expected?
>> Or which URI i should use?
>>
>> --
>> *From: *"Radar Lei" 
>> *To: *"user" 
>> *Sent: *Wednesday, May 30, 2018 8:30:01 PM
>> *Subject: *Re: how hawq use HA hdfs
>>
>> As the steps in the document you mentioned, you need to finish all the
>> steps to convert your HAWQ cluster to HA mode.
>> E.g. Do filespace move, make changes in hdfs-client.xml etc.
>>
>> Regards,
>> Radar
>>
>> On Wed, May 30, 2018 at 7:44 PM,  wrote:
>>
>>> Hi!
>>>
>>> I has setting HA hdfs using ZKFC with below config:
>>>
>>> 
>>> 
>>> fs.defaultFS
>>> hdfs://dx
>>> 
>>>  
>>>ha.zookeeper.quorum
>>>192.168.60.24 :2181,192.168.60.32
>>> :2181,192.168.60.37
>>> :2181
>>>  
>>> 
>>>
>>> Then i install hawq in another node(not any node installed hdfs).
>>>
>>> I follow http://hawq.incubator.apache.org/docs/userguide/2.3.0.0-
>>> incubating/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html
>>> to set `hawq_dfs_url` as "dx/hawq_default", but init failed when check
>>> if hdfs path is available:
>>> [WARNING]:-ERROR: Can not connect to 'hdfs://dx:0'
>>>
>>> Then i change `hawq_dfs_url` to "dx:8020/hawq_default", still failed:
>>> ERROR Failed to setup RPC connection to "dx:8020" caused by:
>>> TcpSocket.cpp: 171: HdfsNetworkConnectException: Failed to resolve
>>> address "dx:8020" Name or service not known
>>>
>>> It consider "dx" as hostname, which is not expected.
>>>
>>> I also test "hdfs://dx", "file://dx", all failed.
>>>
>>> I know that "192.168.60.24 :8020" can work, but
>>> it does not use HA hdfs.
>>>
>>> How can hawq use HA hdfs?
>>>
>>> Thanks
>>>
>>
>>
>> --
>> 戴翔
>> 南京天数信息科技有限公司
>> 电话: +86 1 3382776490
>> 公司官网: www.sky-data.cn
>> 免费使用天数润科智能计算平台 SkyDiscovery
>>
>
>
> --
> 戴翔
> 南京天数信息科技有限公司
> 电话: +86 1 3382776490
> 公司官网: www.sky-data.cn
> 免费使用天数润科智能计算平台 SkyDiscovery
>

Re: how hawq use HA hdfs

[Radar Lei]

If you are installing a new HAWQ, then file space move is not required.

I think  HAWQ will treat the host string as an url unless you configured
HAWQ hdfs HA correctly. So please verify if you missed any other steps.

Regards,
Radar

On Thu, May 31, 2018 at 9:40 AM,  wrote:

> I think that move filespace is the action that hawq has installed before
> HA hdfs,
> but in my case, i installed HA hdfs then installed hawq.
>
> In this way, i think there is no need to move filespace.
>
> Besides, my question is that why URI can not work as expected?
> Or which URI i should use?
>
> ------
> *From: *"Radar Lei" 
> *To: *"user" 
> *Sent: *Wednesday, May 30, 2018 8:30:01 PM
> *Subject: *Re: how hawq use HA hdfs
>
> As the steps in the document you mentioned, you need to finish all the
> steps to convert your HAWQ cluster to HA mode.
> E.g. Do filespace move, make changes in hdfs-client.xml etc.
>
> Regards,
> Radar
>
> On Wed, May 30, 2018 at 7:44 PM,  wrote:
>
>> Hi!
>>
>> I has setting HA hdfs using ZKFC with below config:
>>
>> 
>> 
>> fs.defaultFS
>> hdfs://dx
>> 
>>  
>>ha.zookeeper.quorum
>>192.168.60.24 :2181,192.168.60.32
>> :2181,192.168.60.37
>> :2181
>>  
>> 
>>
>> Then i install hawq in another node(not any node installed hdfs).
>>
>> I follow http://hawq.incubator.apache.org/docs/userguide/2.3.0.0-
>> incubating/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html
>> to set `hawq_dfs_url` as "dx/hawq_default", but init failed when check if
>> hdfs path is available:
>> [WARNING]:-ERROR: Can not connect to 'hdfs://dx:0'
>>
>> Then i change `hawq_dfs_url` to "dx:8020/hawq_default", still failed:
>> ERROR Failed to setup RPC connection to "dx:8020" caused by:
>> TcpSocket.cpp: 171: HdfsNetworkConnectException: Failed to resolve
>> address "dx:8020" Name or service not known
>>
>> It consider "dx" as hostname, which is not expected.
>>
>> I also test "hdfs://dx", "file://dx", all failed.
>>
>> I know that "192.168.60.24 :8020" can work, but it
>> does not use HA hdfs.
>>
>> How can hawq use HA hdfs?
>>
>> Thanks
>>
>
>
> --
> 戴翔
> 南京天数信息科技有限公司
> 电话: +86 1 3382776490
> 公司官网: www.sky-data.cn
> 免费使用天数润科智能计算平台 SkyDiscovery
>

Re: how hawq auto active standby

[Radar Lei]

There is no command line tools to activate standby automatically yet.

Maybe you can write a script to do it by yourself, just need to be more
carefull.


Regards,
Radar

On Wed, May 30, 2018 at 8:22 PM,  wrote:

> Hi！
>
> I has add a standby node for hawq, and it can active to master by hand  if
> master failed.
>
> But i do not find details about how to config hawq auto active standby to
> master.
>
> I google and only find with ambari, hawq can auto active standby to master.
>
> Is there any doc about how to make it with command line?
>
> Thanks
>

Re: [DISCUSS] Apache HAWQ Graduation from Incubator

[Radar Lei]

Thanks Roman.

This make sense, I will start to draft the resolution. BTW, we would need
to nominate a chair, I guess it's the last piece we need to draft the
resolution.

Regards,
Radar

On Tue, May 29, 2018 at 11:24 AM, Roman Shaposhnik 
wrote:

> On Sun, May 27, 2018 at 11:37 PM, Radar Lei  wrote:
> > Hi Roman,
> >
> > We have confirmed with each HAWQ committer whether they want to remain
> with
> > HAWQ project.  As a summary, 37 PPMC members(including two mentors) and 7
> > committers confirmed they want to remain with HAWQ. [1] The total
> committers
> > number 44 seems pretty close with PPMC member number 37, is it good
> enough
> > to make PMC == committers as our graduation resolution?
>
> PMC == committers in this case makes perfect sense to me!
>
> > Should we update the whimsy and project webpage now or do update after
> graduation?
>
> It really doesn't matter much. Your next step is to draft a resolution
> similar to:
> https://www.mail-archive.com/general@incubator.apache.org/
> msg56982.html
>
> and start a [DISCUSS] thread similar to the above.
>
> Makes sense?
>
> Thanks,
> Roman.
>

Re: [DISCUSS] Apache HAWQ Graduation from Incubator

[Radar Lei]

Hi Roman,

We have confirmed with each HAWQ committer whether they want to remain with
HAWQ project.  As a summary, 37 PPMC members(including two mentors) and 7
committers confirmed they want to remain with HAWQ. [1] The total
committers number 44 seems pretty close with PPMC member number 37, is it
good enough to make PMC == committers as our graduation resolution? Should
we update the whimsy and project webpage now or do update after graduation?

BTW, please kindly advice the next steps towards graduation? Thanks.

[1]
https://lists.apache.org/thread.html/4f31f2766ed6963f845ac4034163db542c7895cf7f4b7dfcaeedfb8c@%3Cdev.hawq.apache.org%3E


Regards,
Radar

On Fri, May 4, 2018 at 2:26 PM, Radar Lei <r...@pivotal.io> wrote:

> Thanks Alan, this is very helpful. It would be great we still can have
> mentors to stay as HAWQ PMC members.
>
> I'd volunteer to proceed and collect all the current committers's
> responds. I plan to send the email to dev list after this weekend, and then
> send individual emails to other committers and our mentors one week after.
>
> Regards,
> Radar
>
> On Fri, May 4, 2018 at 2:18 AM, Alan Gates <alanfga...@gmail.com> wrote:
>
>> On Thu, May 3, 2018 at 1:14 AM, Radar Lei <r...@pivotal.io> wrote:
>>
>> > Hi Roman,
>> >
>> > Thanks for your recognition and suggestion.
>> >
>> > Would you please give some guidance on:
>> >
>> > 1. Currently we have more than 50 committers on the list, should we send
>> > email to each committer individually to get their confirmation? Or we
>> can
>> > just send one public email to ask in private/dev list?
>> >
>> ​Start with an email to the dev list.  80% of the people who are going to
>> respond will respond there.  After a few days send individual emails to
>> those who don't respond on the dev list.
>> ​
>>
>>
>> > 2. What if someone want to keep to be a committer but not be a PMC
>> member?
>> >
>> ​That's fine, if it's what they want.  But you'll need to note that
>> clearly
>> when we write up the graduation resolution as there may be questions on
>> why
>> different people are being granted different status.​
>>
>>
>> >
>> > 3. What if we can not get respond from some committers in time? E.g. one
>> > week.
>> >
>> ​I would include in the email a time limit, probably closer to two weeks
>> than one so that anyone who is on vacation has a better chance of seeing
>> it.  That two weeks can be from the time you send the original email on
>> the
>> dev list.  Anyone who does not respond in the given time frame will be
>> assumed to not want to continue with HAWQ.
>>
>> As another note, it is traditional to ask the mentors if they wish to stay
>> on as PMC members.  For new projects I think it is very valuable to have
>> at
>> least one experienced Apache person on the PMC, even if he or she is not
>> contributing code.
>>
>> Alan.​
>>
>>
>> >
>> >
>> >
>> > Regards,
>> > Radar
>> >
>> > On Wed, May 2, 2018 at 5:40 AM, Roman Shaposhnik <ro...@shaposhnik.org>
>> > wrote:
>> >
>> > > I think in general it makes sense to me. Looking at:
>> > >http://incubator.apache.org/projects/hawq.html
>> > > I'd like to suggest that perhaps you may want to
>> > > explicitly ask folks on the committers list whether
>> > > they wish to remain with the project or not (I suspect
>> > > some of them will decline the invitation -- but perhaps
>> > > I'm wrong).
>> > >
>> > > If the list of committers will match the list of PPMC
>> > > closes -- then I'd also suggest considering making
>> > > PMC == committers as part of your graduation
>> > > resolution.
>> > >
>> > > Other than that -- I think you will make a great community.
>> > >
>> > > Thanks,
>> > > Roman.
>> > >
>> > > On Sat, Apr 28, 2018 at 4:02 AM, 刘奎恩(局外) <kuien@alibaba-inc.com>
>> > > wrote:
>> > > > Great news, I have no reason to reject this wonderful idea. One
>> > > consideration is may we have time window to do a bit more efforts on
>> > > execuation engine? because these derivative startup projects (hawq+,
>> > > deepgreen) all claims their superiority on execuation engine. Maybe,
>> Hawq
>> > > may co-plan it with Greenplum, it is one common issue in both data
>> > > warehouse.
>> > >

Re: Remain with HAWQ project or not?

[Radar Lei]

Forward responds I received in individual mails in recent days:

Yes:
Vineet Goel
Jesse Zhang

No:
Owen O'Malley



Regards,
Radar

On Mon, May 21, 2018 at 12:20 PM, Greg Chase <gch...@gmail.com> wrote:

> Yes :-)
>
> This email encrypted by tiny buttons & fat thumbs, beta voice recognition,
> and autocorrect on my iPhone.
>
> > On May 14, 2018, at 9:41 PM, Radar Lei <r...@pivotal.io> wrote:
> >
> > FWD responds I received in individual mails:
> >
> > Yes:
> > Goden Yao
> > Kavinder Dhaliwal
> > Mohamed Soliman
> > Newton Alex
> > Oleksandr Diachenko
> > Tushar Pednekar
> > Venkatesh Raghavan
> > David Yozie
> >
> > No:
> > Mel Kiyama
> >
> >
> > Regards,
> > Radar
> >
> >> On Mon, May 14, 2018 at 6:18 AM, Entong Shen <ent...@apache.org> wrote:
> >>
> >> Yes.
> >>
> >>> On 2018/05/07 08:11:53, Radar Lei <r...@pivotal.io> wrote:
> >>> HAWQ committers,
> >>>
> >>> Per the discussion in "Apache HAWQ graduation from incubator?" [1], we
> >> want
> >>> to setup the PMC as part of HAWQ graduation resolution.
> >>>
> >>> So we'd like to confirm whether you want to remain as a committer/PMC
> >>> member of Apache HAWQ project?
> >>>
> >>> If you'd like to remain with HAWQ project, it's welcome and please
> >> *respond**
> >>> 'Yes'* in this thread, or *respond 'No'* if you are not interested in
> any
> >>> more. Thanks.
> >>>
> >>> This thread will be available for at least 72 hours, after that, we
> will
> >>> send individual confirm emails.
> >>>
> >>> [1]
> >>> https://lists.apache.org/thread.html/b4a0b5671ce377b3d51c9b7ab00496
> >> a1eebfcbf1696ce8b67e078c64@%3Cdev.hawq.apache.org%3E
> >>>
> >>> Regards,
> >>> Radar
> >>>
> >>
>

Re: Remain with HAWQ project or not?

[Radar Lei]

FWD responds I received in individual mails:

Yes:
Goden Yao
Kavinder Dhaliwal
Mohamed Soliman
Newton Alex
Oleksandr Diachenko
Tushar Pednekar
Venkatesh Raghavan
David Yozie

No:
Mel Kiyama


Regards,
Radar

On Mon, May 14, 2018 at 6:18 AM, Entong Shen <ent...@apache.org> wrote:

> Yes.
>
> On 2018/05/07 08:11:53, Radar Lei <r...@pivotal.io> wrote:
> > HAWQ committers,
> >
> > Per the discussion in "Apache HAWQ graduation from incubator?" [1], we
> want
> > to setup the PMC as part of HAWQ graduation resolution.
> >
> > So we'd like to confirm whether you want to remain as a committer/PMC
> > member of Apache HAWQ project?
> >
> > If you'd like to remain with HAWQ project, it's welcome and please
> *respond**
> > 'Yes'* in this thread, or *respond 'No'* if you are not interested in any
> > more. Thanks.
> >
> > This thread will be available for at least 72 hours, after that, we will
> > send individual confirm emails.
> >
> > [1]
> > https://lists.apache.org/thread.html/b4a0b5671ce377b3d51c9b7ab00496
> a1eebfcbf1696ce8b67e078c64@%3Cdev.hawq.apache.org%3E
> >
> > Regards,
> > Radar
> >
>

Re: error (connection pointer is NULL) when using hawq pxf

[Radar Lei]

Hi Gang,

The log is not attached, please provide it again by the email or in jira.
Thanks.

Regards,
Radar

On Tue, May 8, 2018 at 4:24 PM, gang.wang  wrote:

> Hi，hawq team:
>
>
> I have met an error using hawq pxf.
>
>
> Here is my environment：
>
> 10.0.0.11，10.0.0.12，10.0.0.13 for hdfs
>
> 10.0.0.14，10.0.0.15，10.0.0.16 for hawq and pxf.
>
> centos 7.4
>
> hawq version:2.2.0
>
>
> I followed the example on ‘''https://cwiki.apache.org/
> confluence/display/HAWQ/PXF+Build+and+Install''’ for
>
> '''CREATE EXTERNAL TABLE pxf_hdfs_textsimple ...’''
>
>
> But,when I query the external table ,I got the error below:
>
>
> postgres=# select * from pxf_hdfs_textsimple;
>
> ERROR:  Error dispatching to seg0 skyaxe-computing-0.localdomain:4:
> connection pointer is NULL
>
> DETAIL:
>
> Master unable to connect to seg0 skyaxe-computing-0.localdomain:4:
> could not connect to server: Connection refused
>
> Is the server running on host "10.0.0.14" and accepting
>
> TCP/IP connections on port 4?
>
>
> Mail attachments is pg_log.
>
>
> Thanks!
>
>
>
>

Re: Remain with HAWQ project or not?

[Radar Lei]

Thanks all for your responds. Great to see so many people still have
passion to contribute to Apache HAWQ.

For the ones who have not responded yet, I will start sending individual
email to reminder them to give their respond here. I may forward some of
the responds if they are not able to respond in the dev list for some
reasons.

Regards,
Radar

On Thu, May 10, 2018 at 11:48 PM, Shivram Mani <shivram.m...@gmail.com>
wrote:

> +1
>
> On Wed, May 9, 2018 at 2:38 AM, Lei Chang <chang.lei...@gmail.com> wrote:
>
> > yes
> >
> > On Wed, May 9, 2018 at 1:01 PM, Yi JIN <y...@apache.org> wrote:
> >
> > > Yes， I would like to remain a committer/PMC member of Apache HAWQ
> > project.
> > >
> > > Best,
> > > Yi (yjin)
> > >
> > > On Wed, May 9, 2018 at 12:41 AM, Ed Espino <esp...@apache.org> wrote:
> > >
> > > > Yes I would like to remain a committer/PMC member of Apache HAWQ
> > project.
> > > >
> > > > Regards,
> > > > -=e
> > > >
> > > > On Mon, May 7, 2018 at 1:11 AM, Radar Lei <r...@pivotal.io> wrote:
> > > >
> > > > > HAWQ committers,
> > > > >
> > > > > Per the discussion in "Apache HAWQ graduation from incubator?" [1],
> > we
> > > > want
> > > > > to setup the PMC as part of HAWQ graduation resolution.
> > > > >
> > > > > So we'd like to confirm whether you want to remain as a
> committer/PMC
> > > > > member of Apache HAWQ project?
> > > > >
> > > > > If you'd like to remain with HAWQ project, it's welcome and please
> > > > > *respond**
> > > > > 'Yes'* in this thread, or *respond 'No'* if you are not interested
> in
> > > any
> > > > > more. Thanks.
> > > > >
> > > > > This thread will be available for at least 72 hours, after that, we
> > > will
> > > > > send individual confirm emails.
> > > > >
> > > > > [1]
> > > > > https://lists.apache.org/thread.html/
> b4a0b5671ce377b3d51c9b7ab00496
> > > > > a1eebfcbf1696ce8b67e078c64@%3Cdev.hawq.apache.org%3E
> > > > >
> > > > > Regards,
> > > > > Radar
> > > > >
> > > >
> > >
> >
>
>
>
> --
> shivram mani
>

Re: Remain with HAWQ project or not?

[Radar Lei]

For myself: Yes, I want to remain with HAWQ project!

Regards,
Radar

On Mon, May 7, 2018 at 4:11 PM, Radar Lei <r...@pivotal.io> wrote:

> HAWQ committers,
>
> Per the discussion in "Apache HAWQ graduation from incubator?" [1], we
> want to setup the PMC as part of HAWQ graduation resolution.
>
> So we'd like to confirm whether you want to remain as a committer/PMC
> member of Apache HAWQ project?
>
> If you'd like to remain with HAWQ project, it's welcome and please
> *respond** 'Yes'* in this thread, or *respond 'No'* if you are not
> interested in any more. Thanks.
>
> This thread will be available for at least 72 hours, after that, we will
> send individual confirm emails.
>
> [1] https://lists.apache.org/thread.html/b4a0b5671ce377b3d51c9b7ab00496
> a1eebfcbf1696ce8b67e078c64@%3Cdev.hawq.apache.org%3E
>
> Regards,
> Radar
>

Remain with HAWQ project or not?

[Radar Lei]

HAWQ committers,

Per the discussion in "Apache HAWQ graduation from incubator?" [1], we want
to setup the PMC as part of HAWQ graduation resolution.

So we'd like to confirm whether you want to remain as a committer/PMC
member of Apache HAWQ project?

If you'd like to remain with HAWQ project, it's welcome and please *respond**
'Yes'* in this thread, or *respond 'No'* if you are not interested in any
more. Thanks.

This thread will be available for at least 72 hours, after that, we will
send individual confirm emails.

[1]
https://lists.apache.org/thread.html/b4a0b5671ce377b3d51c9b7ab00496a1eebfcbf1696ce8b67e078c64@%3Cdev.hawq.apache.org%3E

Regards,
Radar

Re: [DISCUSS] Apache HAWQ Graduation from Incubator

[Radar Lei]

Thanks Alan, this is very helpful. It would be great we still can have
mentors to stay as HAWQ PMC members.

I'd volunteer to proceed and collect all the current committers's responds.
I plan to send the email to dev list after this weekend, and then send
individual emails to other committers and our mentors one week after.

Regards,
Radar

On Fri, May 4, 2018 at 2:18 AM, Alan Gates <alanfga...@gmail.com> wrote:

> On Thu, May 3, 2018 at 1:14 AM, Radar Lei <r...@pivotal.io> wrote:
>
> > Hi Roman,
> >
> > Thanks for your recognition and suggestion.
> >
> > Would you please give some guidance on:
> >
> > 1. Currently we have more than 50 committers on the list, should we send
> > email to each committer individually to get their confirmation? Or we can
> > just send one public email to ask in private/dev list?
> >
> ​Start with an email to the dev list.  80% of the people who are going to
> respond will respond there.  After a few days send individual emails to
> those who don't respond on the dev list.
> ​
>
>
> > 2. What if someone want to keep to be a committer but not be a PMC
> member?
> >
> ​That's fine, if it's what they want.  But you'll need to note that clearly
> when we write up the graduation resolution as there may be questions on why
> different people are being granted different status.​
>
>
> >
> > 3. What if we can not get respond from some committers in time? E.g. one
> > week.
> >
> ​I would include in the email a time limit, probably closer to two weeks
> than one so that anyone who is on vacation has a better chance of seeing
> it.  That two weeks can be from the time you send the original email on the
> dev list.  Anyone who does not respond in the given time frame will be
> assumed to not want to continue with HAWQ.
>
> As another note, it is traditional to ask the mentors if they wish to stay
> on as PMC members.  For new projects I think it is very valuable to have at
> least one experienced Apache person on the PMC, even if he or she is not
> contributing code.
>
> Alan.​
>
>
> >
> >
> >
> > Regards,
> > Radar
> >
> > On Wed, May 2, 2018 at 5:40 AM, Roman Shaposhnik <ro...@shaposhnik.org>
> > wrote:
> >
> > > I think in general it makes sense to me. Looking at:
> > >http://incubator.apache.org/projects/hawq.html
> > > I'd like to suggest that perhaps you may want to
> > > explicitly ask folks on the committers list whether
> > > they wish to remain with the project or not (I suspect
> > > some of them will decline the invitation -- but perhaps
> > > I'm wrong).
> > >
> > > If the list of committers will match the list of PPMC
> > > closes -- then I'd also suggest considering making
> > > PMC == committers as part of your graduation
> > > resolution.
> > >
> > > Other than that -- I think you will make a great community.
> > >
> > > Thanks,
> > > Roman.
> > >
> > > On Sat, Apr 28, 2018 at 4:02 AM, 刘奎恩(局外) <kuien@alibaba-inc.com>
> > > wrote:
> > > > Great news, I have no reason to reject this wonderful idea. One
> > > consideration is may we have time window to do a bit more efforts on
> > > execuation engine? because these derivative startup projects (hawq+,
> > > deepgreen) all claims their superiority on execuation engine. Maybe,
> Hawq
> > > may co-plan it with Greenplum, it is one common issue in both data
> > > warehouse.
> > > > -——
> > > > Kuien Liu/奎恩
> > > > 
> > --发件人：Radar
> > > Lei <r...@pivotal.io>发送时间：2018年4月26日(星期四) 10:16收件人：dev <
> > > dev@hawq.incubator.apache.org>主 题：[DISCUSS] Apache HAWQ Graduation
> from
> > > Incubator
> > > > Hi All,
> > > >
> > > > With the Apache HAWQ community keep healthy and growing, I think now
> > > would
> > > > be a good time to discuss where we stand regarding to graduation from
> > the
> > > > Incubator, and what requirements remains.
> > > >
> > > > Apache HAWQ entered incubation in September of 2015, since then, the
> > HAWQ
> > > > community learned a lot about how to do things in Apache ways. Now we
> > > are a
> > > > very helpful and engaged community, ready to help on all questions
> > > > from the HAWQ
> > > > community. We delivered four releases including two binary releases,
> > now
> > > we
> > > > can do self-driving releases in good cadence. The community is
> growing,
> > > >  new committers and PPMC members keep joining. We addressed the
> > maturity
> > > > issues one by one followed by Apache Project Maturity Model [1],
> > > currently all
> > > > the License and IP issues we can see got resolved.
> > > >
> > > > Base on those, I believes it's time for us to graduate to TLP. [2]
> > > >
> > > > Any thoughts? And welcome advice from HAWQ Mentors?
> > > >
> > > > Thanks,
> > > >
> > > > [1] https://cwiki.apache.org/confluence/display/HAWQ/ASF+
> > > Maturity+Evaluation
> > > > [2] https://incubator.apache.org/guides/graduation.html#grad
> > > > uating_to_a_top_level_project
> > > >
> > > > Regards,
> > > > Radar
> > >
> >
>

Re: [DISCUSS] Apache HAWQ Graduation from Incubator

[Radar Lei]

Hi Roman,

Thanks for your recognition and suggestion.

Would you please give some guidance on:

1. Currently we have more than 50 committers on the list, should we send
email to each committer individually to get their confirmation? Or we can
just send one public email to ask in private/dev list?
2. What if someone want to keep to be a committer but not be a PMC member?

3. What if we can not get respond from some committers in time? E.g. one
week.



Regards,
Radar

On Wed, May 2, 2018 at 5:40 AM, Roman Shaposhnik <ro...@shaposhnik.org>
wrote:

> I think in general it makes sense to me. Looking at:
>http://incubator.apache.org/projects/hawq.html
> I'd like to suggest that perhaps you may want to
> explicitly ask folks on the committers list whether
> they wish to remain with the project or not (I suspect
> some of them will decline the invitation -- but perhaps
> I'm wrong).
>
> If the list of committers will match the list of PPMC
> closes -- then I'd also suggest considering making
> PMC == committers as part of your graduation
> resolution.
>
> Other than that -- I think you will make a great community.
>
> Thanks,
> Roman.
>
> On Sat, Apr 28, 2018 at 4:02 AM, 刘奎恩(局外) <kuien@alibaba-inc.com>
> wrote:
> > Great news, I have no reason to reject this wonderful idea. One
> consideration is may we have time window to do a bit more efforts on
> execuation engine? because these derivative startup projects (hawq+,
> deepgreen) all claims their superiority on execuation engine. Maybe, Hawq
> may co-plan it with Greenplum, it is one common issue in both data
> warehouse.
> > -——
> > Kuien Liu/奎恩
> > --发件人：Radar
> Lei <r...@pivotal.io>发送时间：2018年4月26日(星期四) 10:16收件人：dev <
> dev@hawq.incubator.apache.org>主 题：[DISCUSS] Apache HAWQ Graduation from
> Incubator
> > Hi All,
> >
> > With the Apache HAWQ community keep healthy and growing, I think now
> would
> > be a good time to discuss where we stand regarding to graduation from the
> > Incubator, and what requirements remains.
> >
> > Apache HAWQ entered incubation in September of 2015, since then, the HAWQ
> > community learned a lot about how to do things in Apache ways. Now we
> are a
> > very helpful and engaged community, ready to help on all questions
> > from the HAWQ
> > community. We delivered four releases including two binary releases, now
> we
> > can do self-driving releases in good cadence. The community is growing,
> >  new committers and PPMC members keep joining. We addressed the maturity
> > issues one by one followed by Apache Project Maturity Model [1],
> currently all
> > the License and IP issues we can see got resolved.
> >
> > Base on those, I believes it's time for us to graduate to TLP. [2]
> >
> > Any thoughts? And welcome advice from HAWQ Mentors?
> >
> > Thanks,
> >
> > [1] https://cwiki.apache.org/confluence/display/HAWQ/ASF+
> Maturity+Evaluation
> > [2] https://incubator.apache.org/guides/graduation.html#grad
> > uating_to_a_top_level_project
> >
> > Regards,
> > Radar
>

[DISCUSS] Apache HAWQ Graduation from Incubator

[Radar Lei]

Hi All,

With the Apache HAWQ community keep healthy and growing, I think now would
be a good time to discuss where we stand regarding to graduation from the
Incubator, and what requirements remains.

Apache HAWQ entered incubation in September of 2015, since then, the HAWQ
community learned a lot about how to do things in Apache ways. Now we are a
very helpful and engaged community, ready to help on all questions
from the HAWQ
community. We delivered four releases including two binary releases, now we
can do self-driving releases in good cadence. The community is growing,
 new committers and PPMC members keep joining. We addressed the maturity
issues one by one followed by Apache Project Maturity Model [1], currently all
the License and IP issues we can see got resolved.

Base on those, I believes it's time for us to graduate to TLP. [2]

Any thoughts? And welcome advice from HAWQ Mentors?

Thanks,

[1] https://cwiki.apache.org/confluence/display/HAWQ/ASF+Maturity+Evaluation
[2] https://incubator.apache.org/guides/graduation.html#grad
uating_to_a_top_level_project

Regards,
Radar

Re: New Committer: Lav Jain

[Radar Lei]

Congrats to Lav.

Regards,
Radar

On Wed, Apr 25, 2018 at 3:26 AM, Shivram Mani 
wrote:

> The Project Management Committee (PMC) for Apache HAWQ (incubating)
> has invited *Lav Jain* to become a committer and we are pleased to
> announce that he has accepted.
> Being a committer enables easier contribution to the project since
> there is no need to go via the patch submission process. This should
> enable better productivity.
> Please join us in congratulating him and we are looking forward to
> collaborating with him in the open source community.
> His contribution includes (but not limited to):
>
>
> His contribution includes (but not limited to):
>   - *12 commits* with some major components in hawq involved,
>including contributions to *PXF* and* Ranger Plugin Service*
>   - Numerous other contributions including documentation, test, build,
> command line
>tools, code refactor, etc
>   - *300+ commits* to HAWQ and PXF *CI and testing*
>
> HAWQ-1584. Propogate bridge endIteration exception during write
> HAWQ-1580. Set PXF port number to 5888 for GPDB
> HAWQ-1545. Fix pxf-private.classpath file for Cloudera tarball distro
> HAWQ-1533. Fix compiler warnings when building PXF code
> HAWQ-1523. Provide ability to bundle PXF artifacts into a single tarbal
> HAWQ-1492. Enable packaging for PXF JDBC plugin.
> HAWQ-1461. Improve partition parameters validation for PXF-JDBC plugin.
> HAWQ-1454. Exclude certain jars from Ranger Plugin Service packaging
> HAWQ-1415. Set the default_value of JAVA_HOME for running RPS
> HAWQ-1420. Enable build of Ranger plugin open source RPM
> HAWQ-1415. Set the default_value of JAVA_HOME for running RPS
>
> **JIRA*:*
> *11 JIRAs* were created or assigned to Lav
> 10 JIRAs are closed and 1 JIRA is still open
> https://issues.apache.org/jira/browse/HAWQ-1492?jql=
> project%20%3D%20HAWQ%20AND%20(assignee%20in%20(lavjain)%
> 20OR%20reporter%20in%20(lavjain))
>
> *Related Contributions*
> Lav has also been a very active contributor to the *HAWQ Ambari project*
> *40 JIRAs* were created or assigned to Lav
> https://issues.apache.org/jira/browse/AMBARI-21377?jql=
> project%20%3D%20AMBARI%20AND%20(assignee%20in%20(lavjain)%
> 20OR%20reporter%20in%20(lavjain))
>
>
>
>
> --
> Shivram Mani
>

Re: New Committer: Shubham Sharma

[Radar Lei]

Congrats to Shubham!

Regards,
Radar

On Wed, Apr 25, 2018 at 3:28 AM, Shivram Mani 
wrote:

> The Project Management Committee (PMC) for Apache HAWQ (incubating)
> has invited *Shubham Sharma* to become a committer and we are pleased
> to announce that he has accepted.
> Being a committer enables easier contribution to the project since
> there is no need to go via the patch submission process. This should
> enable better productivity.
> Please join us in congratulating him and we are looking forward to
> collaborating with him in the open source community.
>
>
> His contribution includes (but not limited to):
>
>
>
> 15 commits to components in hawq, including contributions to PXF and master
> branch CI
> - Reference - https://github.com/apache/incubator-hawq/pulls?utf8=%E2%
> 9C%93=is%3Apr+author%3Aoutofmem0ry
>
> HAWQ-1572: Fix travis ci build failure. Thrift/Boost incompatibility
> HAWQ-1559: Fix travis ci build failure due to missing apr package
> HAWQ-1553: Add option to hawq extract to specify log directory
> HAWQ-1368: Add option to hawq register to specify log directory
> HAWQ-1549: Fix standby resync when using -M smart/fast/immediate
> HAWQ-1548: Correct ambiguous message while logging hawq utilization
> HAWQ-1527: Enabled partition filtering for integral data types
> HAWQ-1524:  Fix travis ci build failure caused after protobuf upgrade to
> 3.4
> HAWQ-1198: Fix to filter out irrelevant fragments while accessing Hive
> HAWQ-1273:  Removed incorrect references from gplogfilter
> HAWQ-1504:  Fixed namenode hang during docker container restart
> HAWQ-1503: Fixed ftp link causing build failure on centos-6 docker
> HAWQ-1495: Corrected answer file to match insert statement
> HAWQ-1373: Added feature to reload GUC values using hawq reload-config
> HAWQ-1480: Added feature for packing a core file
>
> 4 commits to hawq documentation (https://github.com/apache/incubator-hawq
> -docs/pulls?utf8=%E2%9C%93=is%3Apr+author%3Aoutofmem0ry)
>  - Two closed and two open pull requests
>
> HAWQ-1562: Fixed incorrect path to default log directory
> HAWQ-1368: Add option to hawq register to specify log directory
> HAWQ-1480: Added documentation for packcore utility
> HAWQ-1553: Add option to hawq extract to specify log directory
>
> *JIRA:*
> 12 JIRAs were created by Shubham. Pull requests for all 13 issues has been
> merged to master.
> https://issues.apache.org/jira/browse/HAWQ-1572?jql=
> project%20%3D%20HAWQ%20AND%20reporter%20in%20(outofmemory)
> 3 JIRAs were assigned to Shubham. Pull requests for these issues has been
> merged to master. [HAWQ-1198, HAWQ-1368, HAWQ-1373 ]
>
> Other contributions include code review(https://github.com/
> apache/incubator-
> hawq/pull/1318) and email discussions.
>
> --
> Shivram Mani
>

Re: Podling Report Reminder - April 2018

[Radar Lei]

Thanks Lei, nice report.

Please check below comments:

Three most important issues to address in the move towards
> graduation:

Should be one?

> Three committer candidates passed the voting process:

Should be two.

When were the last committers or PPMC members elected?
>1) Amy BAI:  Nov 1, 2017
>2) ChunLing WANG: Nov 1, 2017
>3) Hongxu MA: Nov 4, 2017


Maybe we should update with the latest committer votes.

Regards,
Radar

On Wed, Apr 4, 2018 at 10:11 PM, Lei Chang  wrote:

> Hi Guys,
>
> Here is the podling report. Please feel free to add your comments.
>
> Cheers
> Lei
>
>
> 
> 
> HAWQ
>
> HAWQ is an advanced enterprise SQL on Hadoop analytic engine built around a
> robust and high-performance massively-parallel processing (MPP) SQL engine
> evolved from Greenplum Database.
>
> HAWQ has been incubating since 2015-09-04.
>
> Three most important issues to address in the move towards
> graduation:
>
>   1. Continue to improve the project's release cadence. To this
>  end we plan on expanding automation services to support
>  increased developer participation. (HAWQ-127)
>
>
> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need
> to be aware of?
>
>  Nothing urgent at this time.
>
> How has the community developed since the last report?
>
> 1. Conference Talks :
>
>  * HAWQ on Microsoft Azure Cloud. Microsoft Incubator Talk (Speaker: Lei
> Chang, Mar 21, 2018)
>
>
> 2. Three committer candidates passed the voting process:
>
>1) Shubham SHARMA
>2) Lav JAIN
>
>
> How has the project developed since the last report?
>
>
> 1. HAWQ 2.3 released. It includes the following features.
>
>   1) New Feature: HAWQ Core supports plugable external storage framework.
>   2) New Feature: HAWQ Ranger supports RPS HA.
>   3) New Feature: HAWQ Ranger supports Kerberos authentication.
>   4) New Feature: HAWQ Core supports HDFS TDE (Transparent Data Encryption)
> through libHdfs3.
>   5) Licenses: Fix PXF license files located in PXF jar files.
>   6) Licenses: Check Apache HAWQ mandatory libraries to match LC20, LC30
> license criteria.
>   7) Build: Release build project
>   8) Bug fixes.
>
> Project page link:
> https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.3.0.0-
> incubating+Release
>
> 2. HAWQ 2.4 release plan was proposed.
>
>   1) New Feature: Pluggable Vectorized Execution Engine on HAWQ.
>   2) New Feature: Support Runtime Filter for HAWQ local hash join.
>   3) New Feature: Support accessing Hive table data by the new Pluggable
> Storage Framework.
>   4) Bug fixes.
>
> Project page link:
> https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.4.0.0-
> incubating+Release
>
>
> How would you assess the podling's maturity?
> Please feel free to add your own commentary.
>
>   [ ] Initial setup
>   [ ] Working towards first release
>   [ ] Community building
>   [X] Nearing graduation
>   [ ] Other:
>
> Date of last release:
>
> 2018-03-12, Apache HAWQ 2.3.0.0
>
> When were the last committers or PPMC members elected?
>
>1) Amy BAI:  Nov 1, 2017
>2) ChunLing WANG: Nov 1, 2017
>3) Hongxu MA: Nov 4, 2017
>
>
> Signed-off-by:
>
>   [](hawq) Alan Gates
>  Comments:
>   [ ](hawq) Konstantin Boudnik
>  Comments:
>   [ ](hawq) Justin Erenkrantz
>  Comments:
>   [ ](hawq) Thejas Nair
>  Comments:
>   [](hawq) Roman Shaposhnik
>  Comments:
>
> On Wed, Apr 4, 2018 at 8:55 AM,  wrote:
>
> > Dear podling,
> >
> > This email was sent by an automated system on behalf of the Apache
> > Incubator PMC. It is an initial reminder to give you plenty of time to
> > prepare your quarterly board report.
> >
> > The board meeting is scheduled for Wed, 18 April 2018, 10:30 am PDT.
> > The report for your podling will form a part of the Incubator PMC
> > report. The Incubator PMC requires your report to be submitted 2 weeks
> > before the board meeting, to allow sufficient time for review and
> > submission (Wed, April 04).
> >
> > Please submit your report with sufficient time to allow the Incubator
> > PMC, and subsequently board members to review and digest. Again, the
> > very latest you should submit your report is 2 weeks prior to the board
> > meeting.
> >
> > Thanks,
> >
> > The Apache Incubator PMC
> >
> > Submitting your Report
> >
> > --
> >
> > Your report should contain the following:
> >
> > *   Your project name
> > *   A brief description of your project, which assumes no knowledge of
> > the project or necessarily of its field
> > *   A list of the three most important issues to address in the move
> > towards graduation.
> > *   Any issues that the Incubator PMC or ASF Board might wish/need to be
> > aware of
> > *   How has the community developed since the last report
> > *   How has the project developed since the last report.
> > *   How does the podling rate their 

Apache HAWQ 2.4.0.0-incubating Release Plan/Timeframe

[Radar Lei]

Hi Apache HAWQ community,

We delivered latest Apache HAWQ release two weeks ago, I think it's time to
get the next release moving along.

I drafted a plan for the release Apache HAWQ 2.4.0.0-incubating, now it's
open for discussion, please help to provide feedback.

Release Version:

Apache HAWQ 2.4.0.0-incubating


Plan Scope:

New Feature: Pluggable Vectorized Execution Engine on HAWQ.
New Feature: Support Runtime Filter for HAWQ local hash join.
New Feature: Support accessing Hive table data by Pluggable Storage
Framework.
Bug fixes.


Release Type:

Source release and binary release.


Timeline:

Target to end of June 2018 : branch cut
Target to end of July 2018 : release



For details, you can check in the wiki page:

https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.4.0.0-incubating+Release


Please feel free to discuss and provide feedback, thanks.

Regards,
Radar

Re: Design of Runtime Filter for HAWQ

[Radar Lei]

Nice design, good to see HAWQ performance improve.

Thanks Wen.

Regards,
Radar

On Tue, Mar 27, 2018 at 10:50 PM, Lirong Jian  wrote:

> Great job, Wen.
>
>
> Best,
> Lirong
>
> Lirong Jian
> HashData Inc.
>
> 2018-03-26 15:09 GMT+08:00 Wen Lin :
>
> > Hi, All,
> >
> > A draft design of runtime filter for HAWQ has been uploaded to jira
> > HAWQ-1597.
> > https://issues.apache.org/jira/browse/HAWQ-1597
> > Please feel free to review it and give any suggestion.
> >
> > Thanks!
> >
>

Re: [ANNOUNCE] Apache HAWQ 2.3.0.0-incubating Release

[Radar Lei]

Thanks all!

It's my honor to be release manager of next Apache HAWQ release, I will
draft and send out a release plan soon.

Regards,
Radar

On Mon, Mar 26, 2018 at 1:56 PM, Hongxu Ma <inte...@outlook.com> wrote:

> +1 for radar,  
>
> 在 23/03/2018 10:04, Radar Lei 写道:
> > Thanks Yi for your great work on Apache HAWQ 2.3.0.0-incubating release.
> >
> > I would like to volunteer to be next release manager of Apache HAWQ.
> >
> > As a committer, I used to work close with previous excellent release
> > managers to prepare the releases, now I want to contribute more on Apache
> > HAWQ releases. I hope we can keep HAWQ release regularly and always keep
> > Apache HAWQ project in a healthy status. Thanks.
> >
> >
> > Regards,
> > Radar
> >
> > On Fri, Mar 23, 2018 at 8:48 AM, Yi JIN <y...@apache.org> wrote:
> >
> >> Thanks everyone for making this real, especially the help from Ruilong
> and
> >> Radar!
> >>
> >> To make it a more successful apache project and community, we need to
> keep
> >> the release cadence. Who would like to be volunteer for the next
> >> release manager
> >> and drive the effort? Thanks.
> >>
> >> Best,
> >> Yi (yjin)
> >>
> >> On Fri, Mar 23, 2018 at 11:40 AM, Lei Chang <chang.lei...@gmail.com>
> >> wrote:
> >>
> >>> congrats!
> >>>
> >>> Cheers
> >>> Lei
> >>>
> >>>
> >>>
> >>> On Wed, Mar 21, 2018 at 1:58 PM, Yi JIN <y...@apache.org> wrote:
> >>>
> >>>> Thanks John, I fixed it and resent the announcement.
> >>>>
> >>>> Best,
> >>>> Yi (yjin)
> >>>>
> >>>> On Wed, Mar 21, 2018 at 4:14 PM, Amy Bai <a...@pivotal.io> wrote:
> >>>>
> >>>>> Congratulations! Thanks Yi for this release!
> >>>>>
> >>>>> On Wed, Mar 21, 2018 at 11:15 AM, John D. Ament <
> >> johndam...@apache.org
> >>>>> wrote:
> >>>>>
> >>>>>> Dropping announce@.
> >>>>>>
> >>>>>> On Tue, Mar 20, 2018 at 10:28 PM Yi JIN <y...@apache.org> wrote:
> >>>>>>
> >>>>>>> Apache HAWQ (incubating) Project Team is proud to announce Apache
> >>>>>>> HAWQ 2.3.0.0-incubating has been released.
> >>>>>>>
> >>>>>>> Apache HAWQ (incubating) combines exceptional MPP-based analytics
> >>>>>>> performance, robust ANSI SQL compliance, Hadoop ecosystem
> >>>>>>> integration and manageability, and flexible data-store format
> >>>>>>> support, all natively in Hadoop, no connectors required. Built
> >>>>>>> from a decade’s worth of massively parallel processing (MPP)
> >>>>>>> expertise developed through the creation of the Pivotal
> >>>>>>> Greenplum® enterprise database and open source PostgreSQL, HAWQ
> >>>>>>> enables to you to swiftly and interactively query Hadoop data,
> >>>>>>> natively via HDFS.
> >>>>>>>
> >>>>>>> *Download Link*:
> >>>>>>>
> >>>>>>> https://dist.apache.org/repos/dist/release/incubator/hawq/2.
> >>>>>> 3.0.0-incubating/
> >>>>>>>
> >>>>>> This has been a hot topic as of late.  This link is incorrect.
> >>> Please
> >>>>>> review http://www.apache.org/dev/release-publishing#
> >>> distribution_dist
> >>>>> and
> >>>>>> related pages and fix the link.  Please resend this email once
> >> you've
> >>>>>> corrected it.
> >>>>>>
> >>>>>>
> >>>>>>> *About this release*
> >>>>>>> This is a release having both source code and binary
> >>>>>>>
> >>>>>>> All changes:
> >>>>>>>
> >>>>>>> https://cwiki.apache.org/confluence/display/HAWQ/
> >>>> Apache+HAWQ+2.3.0.0-
> >>>>>> incubating+Release
> >>>>>>>
> >>>>>>> *HAWQ Resources:*
> >>>>>>>
> >>>>>>> - JIRA: https://issues.apache.org/jira/browse/HAWQ
> >>>>>>> - Wiki:
> >>>>>>> https://cwiki.apache.org/confluence/display/HAWQ/
> >> Apache+HAWQ+Home
> >>>>>>> - Mailing list(s): dev@hawq.incubator.apache.org
> >>>>>>>u...@hawq.incubator.apache.org
> >>>>>>>
> >>>>>>> *Know more about HAWQ:*
> >>>>>>> http://hawq.apache.org
> >>>>>>>
> >>>>>>> - Apache HAWQ (incubating) Team
> >>>>>>>
> >>>>>>> =
> >>>>>>> *Disclaimer*
> >>>>>>>
> >>>>>>> Apache HAWQ (incubating) is an effort undergoing incubation at
> >> The
> >>>>>>> Apache Software Foundation (ASF), sponsored by the name of Apache
> >>>>>>> Incubator PMC. Incubation is required of all newly accepted
> >>>>>>> projects until a further review indicates that the
> >>>>>>> infrastructure, communications, and decision making process have
> >>>>>>> stabilized in a manner consistent with other successful ASF
> >>>>>>> projects. While incubation status is not necessarily a reflection
> >>>>>>> of the completeness or stability of the code, it does indicate
> >>>>>>> that the project has yet to be fully endorsed by the ASF.
> >>>>>>>
>
> --
> Regards,
> Hongxu.
>
>

Re: [ANNOUNCE] Apache HAWQ 2.3.0.0-incubating Release

[Radar Lei]

Thanks Yi for your great work on Apache HAWQ 2.3.0.0-incubating release.

I would like to volunteer to be next release manager of Apache HAWQ.

As a committer, I used to work close with previous excellent release
managers to prepare the releases, now I want to contribute more on Apache
HAWQ releases. I hope we can keep HAWQ release regularly and always keep
Apache HAWQ project in a healthy status. Thanks.


Regards,
Radar

On Fri, Mar 23, 2018 at 8:48 AM, Yi JIN  wrote:

> Thanks everyone for making this real, especially the help from Ruilong and
> Radar!
>
> To make it a more successful apache project and community, we need to keep
> the release cadence. Who would like to be volunteer for the next
> release manager
> and drive the effort? Thanks.
>
> Best,
> Yi (yjin)
>
> On Fri, Mar 23, 2018 at 11:40 AM, Lei Chang 
> wrote:
>
> > congrats!
> >
> > Cheers
> > Lei
> >
> >
> >
> > On Wed, Mar 21, 2018 at 1:58 PM, Yi JIN  wrote:
> >
> > > Thanks John, I fixed it and resent the announcement.
> > >
> > > Best,
> > > Yi (yjin)
> > >
> > > On Wed, Mar 21, 2018 at 4:14 PM, Amy Bai  wrote:
> > >
> > > > Congratulations! Thanks Yi for this release!
> > > >
> > > > On Wed, Mar 21, 2018 at 11:15 AM, John D. Ament <
> johndam...@apache.org
> > >
> > > > wrote:
> > > >
> > > > > Dropping announce@.
> > > > >
> > > > > On Tue, Mar 20, 2018 at 10:28 PM Yi JIN  wrote:
> > > > >
> > > > > > Apache HAWQ (incubating) Project Team is proud to announce Apache
> > > > > > HAWQ 2.3.0.0-incubating has been released.
> > > > > >
> > > > > > Apache HAWQ (incubating) combines exceptional MPP-based analytics
> > > > > > performance, robust ANSI SQL compliance, Hadoop ecosystem
> > > > > > integration and manageability, and flexible data-store format
> > > > > > support, all natively in Hadoop, no connectors required. Built
> > > > > > from a decade’s worth of massively parallel processing (MPP)
> > > > > > expertise developed through the creation of the Pivotal
> > > > > > Greenplum® enterprise database and open source PostgreSQL, HAWQ
> > > > > > enables to you to swiftly and interactively query Hadoop data,
> > > > > > natively via HDFS.
> > > > > >
> > > > > > *Download Link*:
> > > > > >
> > > > > > https://dist.apache.org/repos/dist/release/incubator/hawq/2.
> > > > > 3.0.0-incubating/
> > > > > >
> > > > > >
> > > > > This has been a hot topic as of late.  This link is incorrect.
> > Please
> > > > > review http://www.apache.org/dev/release-publishing#
> > distribution_dist
> > > > and
> > > > > related pages and fix the link.  Please resend this email once
> you've
> > > > > corrected it.
> > > > >
> > > > >
> > > > > > *About this release*
> > > > > > This is a release having both source code and binary
> > > > > >
> > > > > > All changes:
> > > > > >
> > > > > > https://cwiki.apache.org/confluence/display/HAWQ/
> > > Apache+HAWQ+2.3.0.0-
> > > > > incubating+Release
> > > > > >
> > > > > >
> > > > > > *HAWQ Resources:*
> > > > > >
> > > > > >- JIRA: https://issues.apache.org/jira/browse/HAWQ
> > > > > >- Wiki:
> > > > > > https://cwiki.apache.org/confluence/display/HAWQ/
> Apache+HAWQ+Home
> > > > > >- Mailing list(s): dev@hawq.incubator.apache.org
> > > > > >   u...@hawq.incubator.apache.org
> > > > > >
> > > > > > *Know more about HAWQ:*
> > > > > > http://hawq.apache.org
> > > > > >
> > > > > > - Apache HAWQ (incubating) Team
> > > > > >
> > > > > > =
> > > > > > *Disclaimer*
> > > > > >
> > > > > > Apache HAWQ (incubating) is an effort undergoing incubation at
> The
> > > > > > Apache Software Foundation (ASF), sponsored by the name of Apache
> > > > > > Incubator PMC. Incubation is required of all newly accepted
> > > > > > projects until a further review indicates that the
> > > > > > infrastructure, communications, and decision making process have
> > > > > > stabilized in a manner consistent with other successful ASF
> > > > > > projects. While incubation status is not necessarily a reflection
> > > > > > of the completeness or stability of the code, it does indicate
> > > > > > that the project has yet to be fully endorsed by the ASF.
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] Apache HAWQ 2.3.0.0-incubating Release

[Radar Lei]

Congrats!
Thanks Yi for driving this release.

Regards,
Radar

On Wed, Mar 21, 2018 at 10:27 AM, Yi JIN  wrote:

> Apache HAWQ (incubating) Project Team is proud to announce Apache
> HAWQ 2.3.0.0-incubating has been released.
>
> Apache HAWQ (incubating) combines exceptional MPP-based analytics
> performance, robust ANSI SQL compliance, Hadoop ecosystem
> integration and manageability, and flexible data-store format
> support, all natively in Hadoop, no connectors required. Built
> from a decade’s worth of massively parallel processing (MPP)
> expertise developed through the creation of the Pivotal
> Greenplum® enterprise database and open source PostgreSQL, HAWQ
> enables to you to swiftly and interactively query Hadoop data,
> natively via HDFS.
>
> *Download Link*:
> https://dist.apache.org/repos/dist/release/incubator/hawq/2.
> 3.0.0-incubating/
>
> *About this release*
> This is a release having both source code and binary
>
> All changes:
> https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.3.0.0-
> incubating+Release
>
>
> *HAWQ Resources:*
>
>- JIRA: https://issues.apache.org/jira/browse/HAWQ
>- Wiki: https://cwiki.apache.org/confluence/display/HAWQ/
> Apache+HAWQ+Home
>- Mailing list(s): dev@hawq.incubator.apache.org
>   u...@hawq.incubator.apache.org
>
> *Know more about HAWQ:*
> http://hawq.apache.org
>
> - Apache HAWQ (incubating) Team
>
> =
> *Disclaimer*
>
> Apache HAWQ (incubating) is an effort undergoing incubation at The
> Apache Software Foundation (ASF), sponsored by the name of Apache
> Incubator PMC. Incubation is required of all newly accepted
> projects until a further review indicates that the
> infrastructure, communications, and decision making process have
> stabilized in a manner consistent with other successful ASF
> projects. While incubation status is not necessarily a reflection
> of the completeness or stability of the code, it does indicate
> that the project has yet to be fully endorsed by the ASF.
>

Re: [VOTE]: Apache HAWQ 2.3.0.0-incubating Release (RC2)

[Radar Lei]

+1

I downloaded the source tarball, checked:

The hashes are correct.

Rat checks passed.
Checked LICENSE, NOTICE, DISCLAIMER files.

Compiled HAWQ successfully, installed successfully.

Run test queries and verified the version numbers.

Checked previous rc1 issues, all fixed.


I also downloaded the binary tarball, checked:

The hashes are correct.

On a clean CentOS7 instance, I installed all the rpm packages following by
the binary install instruction page.
The installation is successfully and HAWQ is running well.

The installed packages have correct version numbers.



Regards,
Radar

On Tue, Feb 27, 2018 at 12:19 PM, Yi JIN  wrote:

> Hi All,
>
> This is the vote for Apache HAWQ (incubating) 2.3.0.0-incubating Release
> Candidate 2 (RC2). It is a source release for HAWQ core, PXF, and Ranger;
> and binary release for HAWQ core,  PXF and Ranger. We have rpm package
> involved for the binary release.
>
> The vote will run for at least 72 hours and will close on Saturday, March
> 3rd, 2017. Thanks.
>
> 1. Wiki page of the release:
> *https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.3.0.0-
> incubating+Release
>  incubating+Release>*
>
>
> 2. Release Notes (Apache Jira generated):
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> version=12340262=Html=12318826
>
>
> 3. Release verification steps can be found at:
> For source tarball: https://cwiki.apache.org/confluence/display/HAWQ/
> Release+Process%3A+Step+by+step+guide#ReleaseProcess:
> Stepbystepguide-ValidatetheReleaseCandidate
> For rpm package: https://cwiki.apache.org/confluence/display/HAWQ/
> Build+Package+and+Install+with+RPM
>
>
> 4. Git release branch:
> https://git-wip-us.apache.org/repos/asf?p=incubator-hawq.
> git;a=shortlog;h=refs/heads/2.3.0.0-incubating
>
> 5. Source and Binary release balls with signare:
> https://dist.apache.org/repos/dist/dev/incubator/hawq/2.3.0.
> 0-incubating.RC2/
>
>
> 6. Keys to verify the signature of the release artifact are available at:
> https://dist.apache.org/repos/dist/dev/incubator/hawq/KEYS
>
>
> 7. The artifact(s) has been signed with Key ID: CE60F90D1333092A
>
> 8. Fixed issues in RC2.
> https://issues.apache.org/jira/browse/HAWQ-1589
> https://issues.apache.org/jira/browse/HAWQ-1590
>
> REMINDER: Please provide details of what you have tried and verified before
> your vote conclusion. Thanks!
>
>
> Please vote accordingly:
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>
>
> Best regards,
> Yi (yjin)
>

Welcome to attend HAWQ meetup in Shanghai

[Radar Lei]

Hi All,

We will host an Apache HAWQ meetup in Pivotal Shanghai office on Jan.12,
you are welcome to join if you are interested in.

Please check below presentation topics:

   - HAWQ best practice (Hongxu Ma)
   - Introduction to HAWQ resource management (Wen Lin)
   - How Alibaba use HAWQ in 11.11 shopping festival (Kuien Liu)
   - Architecture evolution of Apache HAWQ (Ivan Weng && Amy Bai)


*When:  13:00pm on Friday, Jan.12, 2018.*
*Where: 118 Yunnan Road, Huangpu District，Shanghai 200021*

For registration, please register with below link(in Chinese language) or
send email to me:

http://www.hdb.com/party/sfkt2.html

Thanks.


Regards,
Radar

Re: Travis CI build failing for master repo - HAWQ-1572

[Radar Lei]

Thrift version higher than 0.9.x might not break the current feature tests,
but may have potential issues. Any one have context of this history, please
give your inputs, thanks.

Shubham, I think we can fix the CI first since Travis is only used for HAWQ
build verification and some simple tests, then we can investigate the
thrift version issue later. Thanks.

Regards,
Radar

On Fri, Dec 15, 2017 at 1:57 PM, Shubham Sharma <ssha...@pivotal.io> wrote:

> Hi Radar, am not aware of compatibility problems with higher version of
> thrift. From what I tested with thrift 0.11and boost 1.60 the build
> passes(as
> seen in the new pr <https://github.com/apache/incubator-hawq/pull/1323>)
> am
> not sure if it might cause any issues later during usage. Before this
> failure, recent travis CI was using thrift 0.10 which had a successful
> build as well. What were the compatibility problems that were observed
> before ?
> Even with thrift 0.9.x we would have to switch to a lower version of boost
> because the current default version of travis ci's mac image installs boost
> 1.65.1 which has removed the tr1 module and will cause failure.
>
> On Thu, Dec 14, 2017 at 9:38 PM, Radar Lei <r...@pivotal.io> wrote:
>
> > Thanks Shubham to address this issue.
> >
> > I noticed thrift is upgraded to 0.11, I remember HAWQ have compatible
> > problem with newer version of thrift.
> >
> > So another question is should we use thrift newer than 0.9.x?
> >
> > Regards,
> > Radar
> >
> > On Fri, Dec 15, 2017 at 8:47 AM, Shubham Sharma <ssha...@pivotal.io>
> > wrote:
> >
> > > Hi,
> > >
> > > The travis CI build is failing for master and new commits. The CI is
> > > erroring out with
> > >
> > > configure: error: thrift is required
> > > The command “./configure” failed and exited with 1 during .
> > >
> > > I was able to reproduce this issue and looking at the config.log it
> looks
> > > like it is failing at the line below while running a conftest.cpp -
> > >
> > > /usr/local/include/thrift/stdcxx.h:32:10: fatal error:
> > > 'boost/tr1/functional.hpp' file not found
> > >
> > > The root cause of the problem is compatibility of thrift 0.11 with
> boost
> > > 1.65.1 . Travis recently upgraded there xcode to 9.2 and list of
> default
> > > packages now contains boost 1.65.1 and thrift 0.11.
> > >
> > > Thrift uses stdcxx.h
> > > <https://github.com/apache/thrift/blob/041c3c777db9639b0a9195bc6aa1f9
> > > 35501fd506/lib/cpp/src/thrift/stdcxx.h#L32>
> > > which
> > > includes boost/tr1/functional.hpp library. The support for tr1 has been
> > > removed in boost 1.65, see here
> > > <http://www.boost.org/users/history/version_1_65_0.html> under topic
> > > “Removed Libraries”.
> > > Since tr1 library is no longer present in boost 1.65, this causes
> thrift
> > to
> > > fail and eventually ./configure fails
> > >
> > >
> > > *Solution*
> > > As a solution I recommend that we uninstall boost 1.65 and install
> boost
> > > 1.60(the last compatible build with thrift). Tested on my fork of hawq
> > repo
> > > with this solution and the build passes.
> > >
> > > I am not sure if this is a problem with thrift that they are not yet
> > > compatible with boost 1.65 yet or a problem with travis ci that they
> have
> > > included two incompatible versions. Will love to hear community's
> > thoughts
> > > on it.
> > >
> > > --
> > > Regards,
> > > Shubham
> > >
> >
>
>
>
> --
> Regards,
> Shubham Sharma
> Staff Customer Engineer
> Pivotal Global Support Services
> ssha...@pivotal.io
> Direct Tel: +1(510)-304-8201
> Office Hours: Mon-Fri 9:00 am to 5:00 pm PDT
> Out of Office Hours Contact +1 877-477-2269
>

Re: Travis CI build failing for master repo - HAWQ-1572

[Radar Lei]

Thanks Shubham to address this issue.

I noticed thrift is upgraded to 0.11, I remember HAWQ have compatible
problem with newer version of thrift.

So another question is should we use thrift newer than 0.9.x?

Regards,
Radar

On Fri, Dec 15, 2017 at 8:47 AM, Shubham Sharma  wrote:

> Hi,
>
> The travis CI build is failing for master and new commits. The CI is
> erroring out with
>
> configure: error: thrift is required
> The command “./configure” failed and exited with 1 during .
>
> I was able to reproduce this issue and looking at the config.log it looks
> like it is failing at the line below while running a conftest.cpp -
>
> /usr/local/include/thrift/stdcxx.h:32:10: fatal error:
> 'boost/tr1/functional.hpp' file not found
>
> The root cause of the problem is compatibility of thrift 0.11 with boost
> 1.65.1 . Travis recently upgraded there xcode to 9.2 and list of default
> packages now contains boost 1.65.1 and thrift 0.11.
>
> Thrift uses stdcxx.h
>  35501fd506/lib/cpp/src/thrift/stdcxx.h#L32>
> which
> includes boost/tr1/functional.hpp library. The support for tr1 has been
> removed in boost 1.65, see here
>  under topic
> “Removed Libraries”.
> Since tr1 library is no longer present in boost 1.65, this causes thrift to
> fail and eventually ./configure fails
>
>
> *Solution*
> As a solution I recommend that we uninstall boost 1.65 and install boost
> 1.60(the last compatible build with thrift). Tested on my fork of hawq repo
> with this solution and the build passes.
>
> I am not sure if this is a problem with thrift that they are not yet
> compatible with boost 1.65 yet or a problem with travis ci that they have
> included two incompatible versions. Will love to hear community's thoughts
> on it.
>
> --
> Regards,
> Shubham
>

Re: Travis CI failing for Hawq

[Radar Lei]

Thanks you Shubham.

The fix is merged to master.

Regards,
Radar

On Fri, Dec 1, 2017 at 10:27 AM, Shubham Sharma <ssha...@pivotal.io> wrote:

> Travis CI build is green again.
> ​
>
> On Thu, Nov 30, 2017 at 6:12 PM, Shubham Sharma <ssha...@pivotal.io>
> wrote:
>
> > Thanks Radar, already submitted a PR https://github.com/apache/
> > incubator-hawq/pull/1317 . The build is in progress and passed the
> > failure stage. Hopefully build should complete in another 10 minutes.
> >
> > On Thu, Nov 30, 2017 at 6:08 PM, Radar Lei <r...@pivotal.io> wrote:
> >
> >> Hi Shubham,
> >>
> >> Thanks for help addressing this issue. I think you can just go ahead and
> >> fix it.
> >>
> >> For the question why master is still green, I think there is no PR was
> >> merged to master in the passed, so only PRs were tested by Travis.
> >>
> >> Regards,
> >> Radar
> >>
> >> On Fri, Dec 1, 2017 at 3:04 AM, Shubham Sharma <ssha...@pivotal.io>
> >> wrote:
> >>
> >> > Hello folks,
> >> >
> >> > It looks like our Travis build is broken. I first noticed this for my
> >> own
> >> > fork's build and saw the same behavior in apache github repo as well.
> >> It is
> >> > failing with the error below
> >> >
> >> > configure: error: Please install apr from http://apr.apache.org/ and
> >> > add dir of 'apr-1-config' to env variable
> >> > '/Users/travis/.rvm/gems/ruby-2.4.2/bin:/Users/travis/.rvm/
> >> > gems/ruby-2.4.2@global/bin:/Users/travis/.rvm/rubies/ruby-
> >> > 2.4.2/bin:/Users/travis/.rvm/bin:/Users/travis/bin:/Users/
> >> > travis/.local/bin:/Users/travis/.nvm/versions/node/v6.
> >> > 11.4/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin'.
> >> >
> >> > The command "./configure" failed and exited with 1 during .
> >> >
> >> > Your build has been stopped.
> >> >
> >> > /Users/travis/.travis/job_stages: line 166: shell_session_update:
> >> > command not found
> >> >
> >> > Looked into it, the builds started failing November 28th. This is
> around
> >> > the same time when Travis CI upgraded their default xcode version to
> >> 8.3.
> >> > Here
> >> > is the notification
> >> > <https://blog.travis-ci.com/2017-11-21-xcode8-3-default-
> image-announce>
> >> .
> >> >
> >> > Have identified a potential fix and tested it for my fork, the build
> >> > completes successfully. Currently we don't install apr using brew
> >> install,
> >> > which is one of the pre-requisites as mentioned in the hawq incubator
> >> wiki.
> >> > The fix is to "brew install apr" and then force link it to the path
> >> using
> >> > "brew link apr --force. This resolves the problem.
> >> >
> >> > But I have couple of additional questions -
> >> >
> >> > 1. How did the apr get installed before, was it installed with some
> >> other
> >> > package. Asking this as few packages have been removed from the
> default
> >> > image in xcode 8.3
> >> >
> >> > 2. Though the build for branches is failing continuously, why the
> build
> >> > status for master is still green ?
> >> >
> >> > Anyhow, since apr is a dependency for our project my proposal is to
> add
> >> a
> >> > brew install to travis.yml to avoid failure due to such upgrade in
> >> future.
> >> >
> >> > Let me know your thoughts, I can create a JIRA and already have a PR
> >> ready.
> >> >
> >> > --
> >> > Regards,
> >> > Shubham
> >> >
> >>
> >
> >
> >
> > --
> > Regards,
> > Shubham Sharma
> > Staff Customer Engineer
> > Pivotal Global Support Services
> > ssha...@pivotal.io
> > Direct Tel: +1(510)-304-8201 <(510)%20304-8201>
> > Office Hours: Mon-Fri 9:00 am to 5:00 pm PDT
> > Out of Office Hours Contact +1 877-477-2269 <(877)%20477-2269>
> >
>
>
>
> --
> Regards,
> Shubham Sharma
> Staff Customer Engineer
> Pivotal Global Support Services
> ssha...@pivotal.io
> Direct Tel: +1(510)-304-8201
> Office Hours: Mon-Fri 9:00 am to 5:00 pm PDT
> Out of Office Hours Contact +1 877-477-2269
>

Re: Travis CI failing for Hawq

[Radar Lei]

Hi Shubham,

Thanks for help addressing this issue. I think you can just go ahead and
fix it.

For the question why master is still green, I think there is no PR was
merged to master in the passed, so only PRs were tested by Travis.

Regards,
Radar

On Fri, Dec 1, 2017 at 3:04 AM, Shubham Sharma  wrote:

> Hello folks,
>
> It looks like our Travis build is broken. I first noticed this for my own
> fork's build and saw the same behavior in apache github repo as well. It is
> failing with the error below
>
> configure: error: Please install apr from http://apr.apache.org/ and
> add dir of 'apr-1-config' to env variable
> '/Users/travis/.rvm/gems/ruby-2.4.2/bin:/Users/travis/.rvm/
> gems/ruby-2.4.2@global/bin:/Users/travis/.rvm/rubies/ruby-
> 2.4.2/bin:/Users/travis/.rvm/bin:/Users/travis/bin:/Users/
> travis/.local/bin:/Users/travis/.nvm/versions/node/v6.
> 11.4/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin'.
>
> The command "./configure" failed and exited with 1 during .
>
> Your build has been stopped.
>
> /Users/travis/.travis/job_stages: line 166: shell_session_update:
> command not found
>
> Looked into it, the builds started failing November 28th. This is around
> the same time when Travis CI upgraded their default xcode version to 8.3.
> Here
> is the notification
>  .
>
> Have identified a potential fix and tested it for my fork, the build
> completes successfully. Currently we don't install apr using brew install,
> which is one of the pre-requisites as mentioned in the hawq incubator wiki.
> The fix is to "brew install apr" and then force link it to the path using
> "brew link apr --force. This resolves the problem.
>
> But I have couple of additional questions -
>
> 1. How did the apr get installed before, was it installed with some other
> package. Asking this as few packages have been removed from the default
> image in xcode 8.3
>
> 2. Though the build for branches is failing continuously, why the build
> status for master is still green ?
>
> Anyhow, since apr is a dependency for our project my proposal is to add a
> brew install to travis.yml to avoid failure due to such upgrade in future.
>
> Let me know your thoughts, I can create a JIRA and already have a PR ready.
>
> --
> Regards,
> Shubham
>

Re: Apache HAWQ (incubating) 2.3.0.0 suggestion

[Radar Lei]

Hi All,

As Ed mentioned, we already have sufficient content(TDE, Ranger
HA/Kerberos) to do a release, and seems Pluggable storage feature need more
time to finish review and tests.

So  how about we do this release without pluggable storage? We can make
another release soon if pluggable storage is ready for production, any
comments? Thanks.


Regards,
Radar

On Wed, Nov 15, 2017 at 8:44 AM, Lei Chang  wrote:

> Cool. Looks your guys' work is quite related to the pluggable storage
> feature (HAWQ-786).
>
> The feature implements a native C interface for external formats and it is
> several times faster than the current JAVA interface. And the feature has
> been validated in Oushu version for some time.
>
> If you guys are interested in the alignment of the feature, we can discuss
> it on the JIRA.
>
> Cheers
> Lei
>
>
>
>
> On Tue, Nov 14, 2017 at 2:02 PM, 刘奎恩(局外) 
> wrote:
>
> >
> > Missed out DEV.
> >
> > --
> > 发件人：刘奎恩(局外) 
> > 发送时间：2017年11月14日(星期二) 11:15
> > 收件人：Lei Chang 
> > 主 题：回复：Apache HAWQ (incubating) 2.3.0.0 suggestion
> >
> > Hi Dr. Chang,
> >
> > Good, thanks for useful suggestion. Now there is:
> > https://issues.apache.org/jira/browse/HAWQ-1550
> >
> > We are trying to integrate Hawq engine onto MaxCompute (former name
> ODPS, within
> > Aliyun Cloud), Seahawks, it is led by Mr. Chen Xia. We develop a
> component
> > AXF (borrow idea from PXF) to connect data sources from Druid and
> > MaxCompute, it works well for this 11.11 battle, but it is not
> stand-alone
> > program.
> >
> > -——
> > Kuien Liu/奎恩
> >
> > --
> > 发件人：Lei Chang 
> > 发送时间：2017年11月14日(星期二) 10:28
> > 收件人：dev ; 刘奎恩(局外) <
> > kuien@alibaba-inc.com>
> > 主 题：Re: Apache HAWQ (incubating) 2.3.0.0 suggestion
> >
> >
> > Kuien, we are welcoming any good contributions from the community.
> >
> > Looks hawq_log_master_concise is a good enhancement, I'd like suggest you
> > create a JIRA and we can discuss it on the JIRA.
> >
> > Can you explain more about Druid Wrapper & MaxCompute Wrapper? What are
> > the use cases here?
> >
> > Cheers
> > Lei
> >
> >
> >
> >
> >
> > On Tue, Nov 14, 2017 at 10:05 AM, 刘奎恩(局外) 
> > wrote:
> > Hi Ed and Hawq,
> > I have add two GUCs (log_max_size, log_max_age) to control the logfile
> > size on master, otherwise the query to hawq_toolkit.hawq_log_master_
> concise
> > will be slower and slower, and disk usage on master is hard to constrain.
> > If some gys are interested to this, I can submit it to Hawq.
> > Besides, our team has introduced many interesting features on hawq, but
> > most of them are deep coupling with Alibaba Cloud. Some of them may be
> > (maybe not) seperated alone, such as Druid Wrapper, MaxCompute Wrapper. I
> > will discuss with teammates Mr. Chen Xia, Mr. Zhiyong Dai et al.
> >
> > 祝好！刘奎恩/局外
> > --发件人：Ed
> > Espino 发送时间：2017年11月13日(星期一) 23:25收件人：dev <
> > dev@hawq.incubator.apache.org>主 题：Apache HAWQ (incubating) 2.3.0.0
> > suggestion
> > HAWQ,
> >
> > I feel we have sufficient content to warrant a 2.3.0.0
> > release. It has been
> > 4+ months since our last release (2.2.0.0). I suggest we include the
> > pluggable storage feature (HAWQ-786 - assigned to Chiyang Wan -
> > chiyang10...@gmail.com) plus others (as appropriate) in
> > the 2.3.0.0 queue in
> > the next release.
> >
> > Are there any outstanding issues that are mandatory for
> > the 2.3.0.0 release?
> > What do others think?
> >
> > -=e
> >
> > --
> > *Ed Espino*
> >
> >
> >
> >
>

Re: I want to contribute code to Apache HAWQ

[Radar Lei]

Chiyang,

Welcome and looking forward to see your contributions.

Regards,
Radar

On Wed, Nov 15, 2017 at 9:17 AM, 陶征霖  wrote:

> Welcome to Apache HAWQ Family
>
> 2017-11-13 11:12 GMT+08:00 Shubham Sharma :
>
> > Welcome to the Apache HAWQ community Chiyang. Looking forward to your
> > contributions.
> >
> > Following links will help you get started -
> >
> > - Contribution guidelines -
> > https://cwiki.apache.org/confluence/display/HAWQ/Contributing+to+HAWQ
> > - Development Environment setup -
> > https://cwiki.apache.org/confluence/display/HAWQ/Build+and+Install
> > - Frequently faced problems and their solutions -
> > https://cwiki.apache.org/confluence/pages/viewpage.
> action?pageId=65144284
> >
> > Hope this helps.
> >
> >
> > On Sun, Nov 12, 2017 at 7:00 PM, Lei Chang 
> wrote:
> >
> > > Cool.  Welcome to Apache HAWQ.
> > >
> > > And looking forwards to seeing your updates on the feature.
> > >
> > > Cheers
> > > Lei
> > >
> > >
> > >
> > >
> > > On Mon, Nov 13, 2017 at 10:54 AM, Chiyang Wan 
> > > wrote:
> > >
> > > > Hello, everyone. I want to contribute code to Apache HAWQ, especially
> > for
> > > > HAWQ-786(Framework to support pluggable formats and file systems).
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> > Shubham Sharma
> >
>

Re: Hawq standby sync fails if there are existing connections to master

[Radar Lei]

Hi Shubham,

The behavior is intentional but use '-M' option to force it should be good.
Thanks.

Regards,
Radar

On Tue, Nov 14, 2017 at 10:05 AM, Shubham Sharma  wrote:

> To close this thread, the source of this issue appears to be
> tools/bin/hawq_ctl method _resync_standby. When this is called the command
> formation does not include stop_mode options as passed to the arguments.
>
>  def _resync_standby(self):
> logger.info("Re-sync standby")
> cmd = "%s; hawq stop master -a;" % source_hawq_env
> check_return_code(local_ssh(cmd, logger), logger, "Stop hawq
> cluster failed, exit")
> ..
> ..
>
> JIRA ID - HAWQ-1549
>
> On Mon, Nov 13, 2017 at 5:56 PM, Shubham Sharma 
> wrote:
>
> > Thanks, I think I found out where the issue is. Will create a JIRA and
> > submit PR.
> >
> > On Mon, Nov 13, 2017 at 5:54 PM, Lei Chang 
> wrote:
> >
> >> oh, yes, make sense, I overlooked the -M option here.
> >>
> >> Here we should respect the input "-M" option.
> >>
> >> Cheers
> >> Lei
> >>
> >>
> >>
> >> On Tue, Nov 14, 2017 at 8:46 AM, Shubham Sharma 
> >> wrote:
> >>
> >> > Thanks for the response Lei.
> >> >
> >> > I understand this completely and agree with the behavior that we
> should
> >> not
> >> > brutally terminate connections.
> >> >
> >> > However, the case here is, am deliberately trying to use stop mode
> using
> >> > command `hawq init standby -n -v -M fast`. At this point as a user I
> >> > understand that connections will be terminated without warning. When
> >> > passing option `-M fast` hawq should not interrupt me.
> >> >
> >> > Basically, hawq command line is not respecting `-M fast`. Whereas hawq
> >> init
> >> > standby --help documents this option. Let me know if this makes sense.
> >> >
> >> >
> >> > hawq init standby --help
> >> >
> >> > Usage: HAWQ management scripts options
> >> >
> >> >
> >> > Options:
> >> >
> >> >   -h, --helpshow this help message and exit
> >> >
> >> >   -a, --prompt  Execute automatically
> >> >
> >> >   -M STOP_MODE, --mode=STOP_MODE
> >> >
> >> > HAWQ stop mode: smart/fast/immediate
> >> >
> >> >
> >> > On Mon, Nov 13, 2017 at 4:32 PM, Lei Chang 
> >> wrote:
> >> >
> >> > > Hi Shubham,
> >> > >
> >> > > The behavior is intentional. If there are connections when HAWQ init
> >> > > standby, it is better to warn the client instead of cutting the
> >> > connections
> >> > > brutally.
> >> > >
> >> > > Cheers
> >> > > Lei
> >> > >
> >> > >
> >> > >
> >> > > On Mon, Nov 13, 2017 at 11:58 AM, Shubham Sharma <
> ssha...@pivotal.io>
> >> > > wrote:
> >> > >
> >> > > > Hello folks,
> >> > > >
> >> > > > Recently observed a behaviour while re-syncing standby from hawq
> >> > command
> >> > > > line.
> >> > > >
> >> > > > Here are the reproduction steps -
> >> > > >
> >> > > > 1 - Open a client connection to hawq using psql
> >> > > > 2 - From a different terminal run command - hawq init standby -n
> -v
> >> -M
> >> > > fast
> >> > > > 3 - Standby resync fails with error
> >> > > >
> >> > > >
> >> > > > 20171113:03:49:21:158354 hawq_stop:hdp3:gpadmin-[WARNING]:-There
> >> are
> >> > > > other connections to this instance, shutdown mode smart aborted
> >> > > >
> >> > > > 20171113:03:49:21:158354 hawq_stop:hdp3:gpadmin-[WARNING]:-Either
> >> > > > remove connections, or use 'hawq stop master -M fast' or 'hawq
> stop
> >> > > > master -M immediate'
> >> > > >
> >> > > > 20171113:03:49:21:158354 hawq_stop:hdp3:gpadmin-[WARNING]:-See
> hawq
> >> > > > stop --help for all options
> >> > > >
> >> > > > 20171113:03:49:21:158354 hawq_stop:hdp3:gpadmin-[ERROR]:-Active
> >> > > > connections. Aborting shutdown...
> >> > > >
> >> > > > 20171113:03:49:21:158143 hawq_init:hdp3:gpadmin-[ERROR]:-Stop
> hawq
> >> > > > cluster failed, exit
> >> > > >
> >> > > > 4 - My understanding is when -M (stop mode) is passed it should
> >> > terminate
> >> > > > existing client connections. Also, it seems like a good practice
> to
> >> > > > terminate client connections before standby master resync.
> >> > > >
> >> > > > Is this an expected behavior in hawq ? If not, I can open a JIRA
> and
> >> > work
> >> > > > on a pull request to fix this.
> >> > > >
> >> > > > Looking forward to your thoughts on this.
> >> > > > ​
> >> > > > --
> >> > > > Regards,
> >> > > > Shubham Sharma
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Regards,
> >> > Shubham Sharma
> >> > Staff Customer Engineer
> >> > Pivotal Global Support Services
> >> > ssha...@pivotal.io
> >> > Direct Tel: +1(510)-304-8201
> >> > Office Hours: Mon-Fri 9:00 am to 5:00 pm PDT
> >> > Out of Office Hours Contact +1 877-477-2269
> >> >
> >>
> >
> >
> >
> > --
> > Regards,
> > Shubham Sharma
> > Staff Customer Engineer
> > Pivotal Global Support Services
> > ssha...@pivotal.io
> > Direct Tel: +1(510)-304-8201 <(510)%20304-8201>
> > Office 

New committer: Hongxu Ma

[Radar Lei]

The Project Management Committee (PMC) for Apache HAWQ (incubating) has
invited Hongxu Ma to become a committer and we are pleased to announce that
he has accepted.
Being a committer enables easier contribution to the project since there is
no need to go via the patch submission process. This should enable better
productivity.
Please join us in congratulating him and we are looking forward to
collaborating with him in the open source community.

His contribution includes (but not limited to):

*Direct contribution to code base:*

   - 26 commits in total with some major components in hawq involved,
   including contributions to Apache Ranger integration, TDE support
   and command line tools.
*https://github.com/apache/incubator-hawq/commits?author=interma
   *
   - 27 closed PRs:
*https://github.com/apache/incubator-hawq/pulls?utf8=%E2%9C%93=is%3Apr%20is%3Aclosed%20author%3Ainterma
   
*
   - 11 Jiras on Apache Ranger integration
  - https://issues.apache.org/jira/browse/HAWQ-1493 Integrate Ranger
  lookup JAAS configuration in ranger-admin plugin jar
  - https://issues.apache.org/jira/browse/HAWQ-1485 Use user/password
  instead of credentials cache in Ranger lookup for HAWQ with Kerberos
  enabled.
  - https://issues.apache.org/jira/browse/HAWQ-1477 Ranger-plugin
  connect to Ranger admin under kerberos security.
  - https://issues.apache.org/jira/browse/HAWQ-1456 Copy RPS
  configuration files to standby in specific scenarios
  - https://issues.apache.org/jira/browse/HAWQ-1436 Implement RPS High
  availability on HAWQ
  - https://issues.apache.org/jira/browse/HAWQ-1410 Add basic test case
  for hcatalog with ranger
  - https://issues.apache.org/jira/browse/HAWQ-1405 'hawq stop
  --reload' should not stop RPS
  - https://issues.apache.org/jira/browse/HAWQ-1393 'hawq stop cluster'
  failed when rps.sh have some path errors (e.g. CATALINA_HOME)
  - https://issues.apache.org/jira/browse/HAWQ-1365 Print out detailed
  schema information for tables which the user doesn't have privileges
  - https://issues.apache.org/jira/browse/HAWQ-1279 Force to recompute
  namespace_path when enable_ranger
  - https://issues.apache.org/jira/browse/HAWQ-1257 If user doesn't
  have privileges on certain objects, need return user which specific table
  he doesn't have right.
   - 4 Jiras on hawq TDE support
  - https://issues.apache.org/jira/browse/HAWQ-1520 gpcheckhdfs should
  skip hdfs trash directory
  - https://issues.apache.org/jira/browse/HAWQ-1510 Add TDE-related
  functionality into hawq command line tools
  - https://issues.apache.org/jira/browse/HAWQ-1506 Support
  multi-append a file within encryption zone
  - https://issues.apache.org/jira/browse/HAWQ-1193 TDE support in HAWQ
   - 5 Jiras on improvements including documentation, test, build, command
   line tools, code refactor.
  - https://issues.apache.org/jira/browse/HAWQ-1385 hawq_ctl stop
  failed when master is down
  - https://issues.apache.org/jira/browse/HAWQ-870 Allocate target's
  tuple table slot in PortalHeapMemory during split partition
  - https://issues.apache.org/jira/browse/HAWQ-513 initdb.c failed on
  OSX 10.11.3 due to fgets error
  - https://issues.apache.org/jira/browse/HAWQ-1380 Keep hawq_toolkit
  schema check in HAWQ native side
  - https://issues.apache.org/jira/browse/HAWQ-1286 Reduce unnecessary
  calls of namespace check when run \d


   - 3 Jiras on bug fixes including test failure, core-dump
   - https://issues.apache.org/jira/browse/HAWQ-1508 Fix travis broken
  caused by libssl path
  - https://issues.apache.org/jira/browse/HAWQ-1381 Core dump when
  execute 'select * from hawq_toolkit.__hawq_log_master_ext;' on macOS
  - https://issues.apache.org/jira/browse/HAWQ-1226 HAWQ core dump due
  to enable ranger while RPS is down

*Indirect contributions to code base: *
Provides a lot of valuable comments for PRs and help improve the quality of
the codes.
Reviewed for 59 closed RPs:
*https://github.com/apache/incubator-hawq/pulls?utf8=%E2%9C%93=is%3Apr%20commenter%3Ainterma
*

*Design Doc:*
Complete 2 design docs with other commiters.
1, https://issues.apache.org/jira/browse/HAWQ-1193 TDE support in HAWQ
https://issues.apache.org/jira/secure/attachment/12844669/
HAWQ_TDE_Design_ver0.2%20.pdf
2, https://issues.apache.org/jira/browse/HAWQ-1436 Implement RPS High
availability on HAWQ
https://issues.apache.org/jira/secure/attachment/12864941/
RPSHADesign_v0.1.pdf

*HAWQ Meetup:*
One topic on Apache HAWQ Meetup July 2017, Beijing
HAWQ TDE security integration

Re: HAWQ-421 - Proposal to remove references of GPHOME and greenplum_path.sh

[Radar Lei]

Hi Shubham,

I would recommend we hold on this change until we have fully understanding
of the affects and the benefit we can get.

I can think about below items:

   - Require a lot of changes to development CI and Test cases.
   - Require changes of user defined scripts and cases.
   - Might affect the user who are using GPDB clients to access HAWQ.
   - Might affect other projects which are related to HAWQ, e.g. Madlib.
   - It's better to have such changes at a major release, like 3.0.

So this is my opinion, welcome other comments. Thanks.


Regards,
Radar

On Mon, Sep 4, 2017 at 8:45 AM, Shubham Sharma  wrote:

> Hi,
>
> HAWQ-421 describes the confusion created by GPHOME and greenplum_path.sh,
> which also seems out of place with the project name HAWQ. This issue is
> marked as backlog for a while and I wanted to take the opportunity to
> replace these references with something related to our project.
>
> Since these changes almost touch the entire project and the naming changes
> will stick going on forward, wanted to bounce off some ideas with the
> community, let me know which naming convention sounds better.
>
> 1 - Replace GPHOME with
>
> a - HAWQHOME
> b - HAWQ_HOME
> c - HAWQ_CONF_DIR
>
> 2 - Replace greenplum_path.sh with
>
> a - hawq_env.sh
> b - hawq_path.sh
>
> I prefer 1-b and 2-a
>
> Please feel free to add other options. Appreciate your inputs.
>
> --
> Regards,
> Shubham
>

Re: Apache HAWQ 2.3.0.0 release plan

[Radar Lei]

Thanks Yi.

Cut release branch at early/middle September looks fine.

I would suggest we can add TDE support in the release plan too.

Regards,
Radar

On Tue, Jul 25, 2017 at 2:37 PM, Lili Ma  wrote:

> Hi Yi,
>
> What's the status of pluggable external storage?
>
> Also, I wonder whether we can add TDE in next release.
>
> Thanks
> Lili
>
> 2017-07-25 13:25 GMT+08:00 Vineet Goel :
>
> > Sounds good to me, Yi.
> > If there are any other suggestions from other members, those are welcome
> > too.
> >
> > Thank you.
> >
> >
> >
> > On Thu, Jul 20, 2017 at 6:14 PM Yi JIN  wrote:
> >
> > > Hi Vineet,
> > >
> > > Thank you for your comment, I will add timeline on release page as part
> > of
> > > the plan.
> > >
> > > Basically, I expect by 30th Oct., we can have tar balls available for
> > > downloading, and before that I reserve about 1.5 month for voting
> > progress
> > > according to previous experiences. So basically, I think we can cut
> > branch
> > > for new release at mid Sept. So that means we have no more than 2
> months
> > > for fixing and new feature delivery. As delivering pluggable external
> > > storage requires some time. If necessary, I think it is ok to move
> target
> > > date 2 weeks earlier. Does this rough time schedule make sense? What's
> > your
> > > opinion?
> > >
> > > I will add Umbrella JIRAS on that page as well.
> > >
> > > Best,
> > > Yi Jin (yjin)
> > >
> > > On Thu, Jul 20, 2017 at 3:52 PM, Vineet Goel 
> wrote:
> > >
> > > > Hi Yi,
> > > >
> > > > Thank you for the initiative. I have some feedback that we can
> discuss
> > > here
> > > > and collect further input on.
> > > >
> > > > a) Putting a timeline on releases (such as October 31st here) may
> have
> > > some
> > > > downsides to the process. My thoughts are that more frequent releases
> > > give
> > > > users access to features and fixes faster, from a stable branch.
> > Release
> > > > momentum is good to have in any project. For instance, master branch
> is
> > > > already 75+ commits ahead of 2.2.0.0-incubating, since that branch
> was
> > > cut
> > > > a a couple of months back.
> > > >
> > > > b) Also, it may help to add Umbrella JIRAs on that page, associated
> to
> > > the
> > > > new features. If certain features are too large to finish in a given
> > > > release, they can always ride the next release vehicle.
> > > >
> > > > It would be good to hear thoughts from others.
> > > >
> > > > Thanks!
> > > > Vineet
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Jul 19, 2017 at 6:15 PM Yi JIN  wrote:
> > > >
> > > > > Hi Apache HAWQ community,
> > > > >
> > > > > Thank you guys for your trust, I drafted one release plan for next
> > > > version
> > > > > 2.3.0.0 that has been update in wiki as below. Please feel free to
> > > > comment
> > > > > and any input is welcome. Thank you again.
> > > > >
> > > > >
> > > > > https://cwiki.apache.org/confluence/display/HAWQ/
> > Apache+HAWQ+2.3.0.0-
> > > > incubating+Release
> > > > >
> > > > >
> > > > > Best,
> > > > > Yi Jin (yjin)
> > > > >
> > > >
> > >
> >
>

Re: Re: [ANNOUNCE] Apache HAWQ 2.2.0.0-incubating Released

[Radar Lei]

+1 to Yi to be next release manager.

BTW, thanks Ruilong's great effort to deliver the binary release as release
manager, I know there is a lot works to handle binary artifacts licenses.

Regards,
Radar

On Thu, Jul 13, 2017 at 3:16 PM, Wen Lin  wrote:

> Congratulations!
> Thanks Ruilong for all the efforts on release!
> Thanks Yi for volunteering for next release!
>
> Regards!
>
> On Thu, Jul 13, 2017 at 3:04 PM, Ed Espino  wrote:
>
> > Yi,
> >
> > +1 to your offer to be the Release Manager for the next Apache HAWQ
> > release. We all know your past contributions well. Thank you for
> > volunteering.
> >
> > Regards,
> > -=e
> >
> > On Wed, Jul 12, 2017 at 11:48 PM, Yi JIN  wrote:
> >
> > > Hi Ruilong,
> > >
> > > I would like to take this responsibility as a volunteer for the next
> > > release. As a committer I used to contribute a lot of code to Apache
> > HAWQ,
> > > consequently besides code work, if possible I would like to contribute
> > more
> > > in another way and learn more about growing an Apache project.
> > >
> > > Best,
> > > Yi (yjin)
> > >
> > > On Thu, Jul 13, 2017 at 4:43 PM, HuoRuilong 
> wrote:
> > >
> > > > Great step towards a mature hawq and active community! Thanks
> everyone
> > > for
> > > > making this real, especially the help from Ed!
> > > >
> > > > To make it a more successful apache project and community, we need to
> > > keep
> > > > the release cadence. Who would like to be volunteer for the next
> > release
> > > > manager and drive the effort? Thanks.
> > > >
> > > > Best regards,
> > > > Ruilong Huo
> > > >
> > > >
> > > > At 2017-07-13 14:39:21, "Lili Ma"  wrote:
> > > > >Congratulations everyone :)
> > > > >
> > > > >We're stepping further towards graduation!
> > > > >
> > > > >Best Regards,
> > > > >Lili
> > > > >
> > > > >2017-07-13 13:16 GMT+08:00 Ed Espino :
> > > > >
> > > > >> Congratulations to everyone on the first Apache HAWQ release with
> > > > >> convenience binaries. Special thanks to Ruilong for his excellent
> > > > release
> > > > >> management guidance.
> > > > >>
> > > > >> I'm very proud to be part of a great dev team.
> > > > >>
> > > > >> Cheers,
> > > > >> -=e
> > > > >>
> > > > >> On Wed, Jul 12, 2017 at 10:00 PM, 陶征霖 
> wrote:
> > > > >>
> > > > >> > Congrats!
> > > > >> >
> > > > >> > 2017-07-13 9:55 GMT+08:00 Yandong Yao :
> > > > >> >
> > > > >> > > Great achievement, Congrats!
> > > > >> > >
> > > > >> > > On Thu, Jul 13, 2017 at 8:46 AM, Lei Chang <
> > > chang.lei...@gmail.com>
> > > > >> > wrote:
> > > > >> > >
> > > > >> > > > Congrats!
> > > > >> > > >
> > > > >> > > > Cheers
> > > > >> > > > Lei
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Wed, Jul 12, 2017 at 3:27 PM, Ruilong Huo <
> h...@apache.org
> > >
> > > > >> wrote:
> > > > >> > > >
> > > > >> > > > > Hi All,
> > > > >> > > > >
> > > > >> > > > > The Apache HAWQ (incubating) Project Team is proud to
> > announce
> > > > >> > > > > the release of Apache HAWQ 2.2.0.0-incubating.
> > > > >> > > > >
> > > > >> > > > > This is a source code and binary release.
> > > > >> > > > >
> > > > >> > > > > ABOUT HAWQ
> > > > >> > > > > Apache HAWQ (incubating) combines exceptional MPP-based
> > > > analytics
> > > > >> > > > > performance, robust ANSI SQL compliance, Hadoop ecosystem
> > > > >> integration
> > > > >> > > > > and manageability, and flexible data-store format support,
> > all
> > > > >> > > > > natively in Hadoop, no connectors required.
> > > > >> > > > >
> > > > >> > > > > Built from a decade’s worth of massively parallel
> processing
> > > > (MPP)
> > > > >> > > > > expertise developed through the creation of open source
> > > > Greenplum®
> > > > >> > > > > Database and PostgreSQL, HAWQ enables you to
> > > > >> > > > > swiftly and interactively query Hadoop data, natively via
> > > HDFS.
> > > > >> > > > >
> > > > >> > > > > FEATURES AND ENHANCEMENTS INCLUDED IN THIS RELEASE
> > > > >> > > > > - CentOS 7.x support
> > > > >> > > > > Apache HAWQ is improved to be compatible with CentOS 7.x
> > along
> > > > with
> > > > >> > > 6.x.
> > > > >> > > > >
> > > > >> > > > > - Apache Ranger integration
> > > > >> > > > > Integrate Apache HAWQ with Apache Ranger through HAWQ
> Ranger
> > > > Plugin
> > > > >> > > > Service
> > > > >> > > > > which is a RESTful service. It enables users to use Apache
> > > > Ranger
> > > > >> to
> > > > >> > > > > authorize
> > > > >> > > > > user access to Apache HAWQ resources. It also manages all
> > > Hadoop
> > > > >> > > > > components’
> > > > >> > > > > authorization policies with the same user interface,
> policy
> > > > store,
> > > > >> > and
> > > > >> > > > > auditing
> > > > >> > > > > stores.
> > > > >> > > > >
> > > > >> > > > > - PXF ORC profile
> > > > >> > > > > Fully supports PXF with Optimized Row Columnar (ORC) file
> > > > format.
> > > > >> > > > >
> > > > >> > > 

Re: [ANNOUNCE] Apache HAWQ 2.2.0.0-incubating Released

[Radar Lei]

Great achievement! Congratulations!

Regards,
Radar

On Wed, Jul 12, 2017 at 5:38 PM, Hongxu Ma  wrote:

> Congratulations!
>
> Wish HAWQ getting better in future!
>
> 在 12/07/2017 15:27, Ruilong Huo 写道:
> > Hi All,
> >
> > The Apache HAWQ (incubating) Project Team is proud to announce
> > the release of Apache HAWQ 2.2.0.0-incubating.
> >
> > This is a source code and binary release.
> >
> > ABOUT HAWQ
> > Apache HAWQ (incubating) combines exceptional MPP-based analytics
> > performance, robust ANSI SQL compliance, Hadoop ecosystem integration
> > and manageability, and flexible data-store format support, all
> > natively in Hadoop, no connectors required.
> >
> > Built from a decade’s worth of massively parallel processing (MPP)
> > expertise developed through the creation of open source Greenplum®
> > Database and PostgreSQL, HAWQ enables you to
> > swiftly and interactively query Hadoop data, natively via HDFS.
> >
> > FEATURES AND ENHANCEMENTS INCLUDED IN THIS RELEASE
> > - CentOS 7.x support
> > Apache HAWQ is improved to be compatible with CentOS 7.x along with 6.x.
> >
> > - Apache Ranger integration
> > Integrate Apache HAWQ with Apache Ranger through HAWQ Ranger Plugin
> Service
> > which is a RESTful service. It enables users to use Apache Ranger to
> authorize
> > user access to Apache HAWQ resources. It also manages all Hadoop
> components’
> > authorization policies with the same user interface, policy store, and
> auditing
> > stores.
> >
> > - PXF ORC profile
> > Fully supports PXF with Optimized Row Columnar (ORC) file format.
> >
> > - Fixes and enhancements on Apache HAWQ resource manager, query
> execution, dispatcher,
> > catalog, management utilities and more.
> >
> > JIRA GENERATED RELEASE NOTES
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12318826=12339641
> >
> > RELEASE ARTIFACTS ARE AVAILABLE AT
> > http://apache.org/dyn/closer.cgi/incubator/hawq/2.2.0.0-incubating
> >
> > SHA256 & MD5 SIGNATURES (verify your downloads <
> https://www.apache.org/dyn/closer.cgi#verify>):
> > https://dist.apache.org/repos/dist/release/incubator/hawq/2.
> 2.0.0-incubating
> >
> > PGP KEYS
> > https://dist.apache.org/repos/dist/release/incubator/hawq/KEYS
> >
> > DOCUMENTATION
> > http://hawq.incubator.apache.org/docs/userguide/2.2.0.0-incubating
> >
> > HAWQ RESOURCES
> > - JIRA: https://issues.apache.org/jira/browse/HAWQ
> > - Wiki: https://cwiki.apache.org/confluence/display/HAWQ/
> Apache+HAWQ+Home
> > - Mailing list:
> > dev@hawq.incubator.apache.org
> > u...@hawq.incubator.apache.org
> >
> > LEARN MORE ABOUT HAWQ
> > http://hawq.apache.org
> >
> > Best regards,
> > - Apache HAWQ (incubating) Team
> >
> > ==
> > DISCLAIMER
> >
> > Apache HAWQ (incubating) is an effort undergoing incubation at the Apache
> > Software Foundation (ASF), sponsored by the Apache Incubator PMC.
> >
> > Incubation is required of all newly accepted projects until a further
> > review indicates that the infrastructure, communications, and decision
> > making process have stabilized in a manner consistent with other
> > successful ASF projects.
> >
> > While incubation status is not necessarily a reflection of the
> > completeness or stability of the code, it does indicate that the
> > project has yet to be fully endorsed by the ASF.
> >
> > Best regards,
> > Ruilong Huo
>
> --
> Regards,
> Hongxu.
>
>

Re: GNU SASL Library - Libgsasl dependency for libhdfs3?

[Radar Lei]

Hi Ed,

Per my understanding, libgsasl is a build and runtime dependence of HAWQ,
but it's not included in HAWQ source code and HAWQ binary release tarball.

If you check the HAWQ rpm dependences, you can see 'libgsasl' is listed in
'hawq.spec', user need to install it before install HAWQ.

So I think we don't need care if it's Apache License compatible and don't
need to include it in our license file. Thanks.



Regards,
Radar

On Fri, Jul 7, 2017 at 1:34 PM, Ed Espino  wrote:

> HAWQ Dev,
>
> I have been filling out the project's Apache Project Maturity Model at the
> following location:
> https://cwiki.apache.org/confluence/display/HAWQ/
> Apache+Project+Maturity+Model
>
> In trying to gather information for the Licenses and Copyright section
> (specifically: LC20 and LC30), I started to review dependent third party
> components. I reviewed the project's LICENSE file (
> https://github.com/apache/incubator-hawq/blob/master/LICENSE). In
> addition,
> I reviewed the build and install steps from the wiki. I noticed one
> component (libgsasl 1.8.0) which is listed but isn't listed in the LICENSE
> file. This component is a build dependency of the libhdfs3 component (
> https://github.com/apache/incubator-hawq/blob/master/
> depends/libhdfs3/README.md).
> The component libhdfs3 will not build without the development
> libgsasl (header and libs) available. The component libgsasl is the GNU
> SASL Library (https://www.gnu.org/software/gsasl/).
>
> Here is it's licensing information:
>
> The core GNU SASL library, and most mechanisms, are licensed under the GNU
> Lesser General Public version 2.1 (or later). It is distributed separately,
> as the "libgsasl" package. The GNU SASL command line application, self test
> suite and more are licensed under the GNU General Public License version 3
> (or later). The "gsasl" package distribution includes the library part as
> well, so you do not need to install two packages.
>
> To the best of my knowledge, this dependent component libgsasl is not
> compatible with Apache Software projects. Have there been any discussions
> in the past on its use in HAWQ? I couldn't find any in the mail archives. I
> could be entirely wrong and this could be a false alarm. If anyone can
> provide some information, it would be greatly appreciated.
>
> Regards,
> -=e
>
> --
> *Ed Espino*
>

[jira] [Created] (HAWQ-1425) Incorrect init cluster error message while ssh connection failed

[Radar Lei (JIRA)]

Radar Lei created HAWQ-1425:
---

 Summary: Incorrect init cluster error message while ssh connection 
failed
 Key: HAWQ-1425
 URL: https://issues.apache.org/jira/browse/HAWQ-1425
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Command Line Tools
Reporter: Radar Lei
Assignee: Ed Espino


If password-less ssh connection is not set, run hawq init cluster will give 
wrong error message:
'This can be run on master host only'.

Actually this might caused by ssh connection to master host failed. We should 
handle this situation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HAWQ-1421) Improve PXF rpm package name format and dependencies

[Radar Lei (JIRA)]

Radar Lei created HAWQ-1421:
---

 Summary: Improve PXF rpm package name format and dependencies
 Key: HAWQ-1421
 URL: https://issues.apache.org/jira/browse/HAWQ-1421
 Project: Apache HAWQ
  Issue Type: Improvement
  Components: Build
Reporter: Radar Lei
Assignee: Ed Espino
 Fix For: 2.2.0.0-incubating


If we build pxf rpm package by 'make rpm', we will get below pxf packages:
{quote}
  apache-tomcat-7.0.62-el6.noarch.rpm
  pxf-3.2.1.0-root.el6.noarch.rpm
  pxf-hbase_3_2_1_0-3.2.1.0-root.el6.noarch.rpm
  pxf-hdfs_3_2_1_0-3.2.1.0-root.el6.noarch.rpm
  pxf-hive_3_2_1_0-3.2.1.0-root.el6.noarch.rpm
  pxf-jdbc_3_2_1_0-3.2.1.0-root.el6.noarch.rpm
  pxf-json_3_2_1_0-3.2.1.0-root.el6.noarch.rpm
  pxf-service_3_2_1_0-3.2.1.0-root.el6.noarch.rpm
{quote}

These rpm packages have dependencies on Apache Hadoop components only, some 
other Hadoop distributes can't satisfy it. E.g. :
{quote}
rpm -ivh pxf-hdfs_3_2_1_0-3.2.1.0-root.el6.noarch.rpm
error: Failed dependencies:
pxf-service_3_2_1_0 >= 3.2.1.0 is needed by 
pxf-hdfs_3_2_1_0-0:3.2.1.0-root.el6.noarch
hadoop >= 2.7.1 is needed by pxf-hdfs_3_2_1_0-0:3.2.1.0-root.el6.noarch
hadoop-mapreduce >= 2.7.1 is needed by 
pxf-hdfs_3_2_1_0-0:3.2.1.0-root.el6.noarch
{quote}

We'd better make the rpm package name format and dependencies better. 
  1. Remove the version string like '3_2_1_0'.
  2. Remove the user name from the build environment.
  3. Consider do we need to include the apache-tomcat rpm package into HAWQ rpm 
release tarball.
  4. Improve the hard code 'el6' string. (This might be optinal)
  5. Improve the dependencies, including the dependencies between these pxf rpm 
packages.





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HAWQ-1401) DESTDIR option not functioning well in 'make install'

[Radar Lei (JIRA)]

Radar Lei created HAWQ-1401:
---

 Summary: DESTDIR option not functioning well in 'make install'
 Key: HAWQ-1401
 URL: https://issues.apache.org/jira/browse/HAWQ-1401
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Build
Reporter: Radar Lei
Assignee: Ed Espino
 Fix For: 2.2.0.0-incubating


Currently 'make install DESTDIR=/destdir/usr/local/apache-hawq' is not working 
for all the files. 

For example: our configuration files under 'etc' folder. Build rpm from source 
tarball relies on this 'DESTDIR' option.

We should make sure DESTDIR is working for all the files in the install list.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HAWQ-1392) Support RPM build from HAWQ source tarball

[Radar Lei (JIRA)]

Radar Lei created HAWQ-1392:
---

 Summary: Support RPM build from HAWQ source tarball
 Key: HAWQ-1392
 URL: https://issues.apache.org/jira/browse/HAWQ-1392
 Project: Apache HAWQ
  Issue Type: Task
  Components: Build
Reporter: Radar Lei
Assignee: Ed Espino
 Fix For: backlog


Once we achieve the target of make rpm package from pre-build binary tarball, 
we should be able to make HAWQ rpm package from source code tarball too.

1. We should have a default compile option for the build, and user can define  
the compile options by themselves.
2. Add build requires.
3. Add make options to build and collect all HAWQ/PXF/Ranger rpm packages.
4. ...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HAWQ-1316) Feature test compile failed on on Centos7.3

[Radar Lei (JIRA)]

Radar Lei created HAWQ-1316:
---

 Summary: Feature test compile failed on on Centos7.3
 Key: HAWQ-1316
 URL: https://issues.apache.org/jira/browse/HAWQ-1316
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Build
Reporter: Radar Lei
Assignee: Ed Espino
 Fix For: backlog


While compiling HAWQ feature test on rhel7, I see below error.

make -C lib all
make[1]: Entering directory 
`/tmp/build/78017950/hdb_apache/src/test/feature/lib'
ar -r libtest.a 
/tmp/build/78017950/hdb_apache/src/test/feature/lib/xml_parser.o 
/tmp/build/78017950/hdb_apache/src/test/feature/lib/command.o 
/tmp/build/78017950/hdb_apache/src/test/feature/lib/hawq_config.o 
/tmp/build/78017950/hdb_apache/src/test/feature/lib/hawq_scp.o 
/tmp/build/78017950/hdb_apache/src/test/feature/lib/string_util.o 
/tmp/build/78017950/hdb_apache/src/test/feature/lib/hdfs_config.o 
/tmp/build/78017950/hdb_apache/src/test/feature/lib/data_gen.o 
/tmp/build/78017950/hdb_apache/src/test/feature/lib/psql.o 
/tmp/build/78017950/hdb_apache/src/test/feature/lib/yarn_config.o 
/tmp/build/78017950/hdb_apache/src/test/feature/lib/sql_util.o 
/tmp/build/78017950/hdb_apache/src/test/feature/lib/file_replace.o
ar: creating libtest.a
make[1]: Leaving directory `/tmp/build/78017950/hdb_apache/src/test/feature/lib'
g++ -I/usr/include -I/usr/local/include -I/usr/include/libxml2 
-I/tmp/build/78017950/hdb_apache/src/test/feature/ 
-I/tmp/build/78017950/hdb_apache/src/test/feature/ManagementTool/ 
-I/tmp/build/78017950/hdb_apache/src/test/feature/lib/ 
-I/tmp/build/78017950/hdb_apache/src/interfaces/libpq 
-I/tmp/build/78017950/hdb_apache/src/interfaces 
-I/tmp/build/78017950/hdb_apache/src/include  
-I/tmp/build/78017950/hdb_apache/depends/thirdparty/googletest/googletest/include
 
-I/tmp/build/78017950/hdb_apache/depends/thirdparty/googletest/googlemock/include
 -Wall -O0 -g -std=c++11 test_main.o ao/TestAoSnappy.o utility/test_copy.o 
utility/test_cmd.o testlib/test_lib.o ExternalSource/test_errortbl.o 
ExternalSource/test_external_oid.o ExternalSource/test_exttab.o 
query/test_create_type_composite.o query/test_prepare.o query/test_portal.o 
query/test_temp.o query/test_polymorphism.o query/test_nested_case_null.o 
query/test_sequence.o query/test_parser.o query/test_information_schema.o 
query/test_gp_dist_random.o query/test_rowtypes.o query/test_insert.o 
query/test_aggregate.o parquet/test_parquet.o transactions/test_transactions.o 
ddl/test_database.o partition/test_partition.o catalog/test_alter_table.o 
catalog/test_type.o catalog/test_alter_owner.o catalog/test_guc.o 
catalog/test_create_table.o toast/TestToast.o planner/test_subplan.o 
UDF/TestUDF.o PreparedStatement/TestPreparedStatement.o lib/xml_parser.o 
lib/command.o lib/hawq_config.o lib/hawq_scp.o lib/string_util.o 
lib/hdfs_config.o lib/data_gen.o lib/psql.o lib/yarn_config.o lib/sql_util.o 
lib/file_replace.o ManagementTool/test_hawq_register_usage2_case1.o 
ManagementTool/test_hawq_register_usage2_case2.o 
ManagementTool/test_hawq_register_usage1.o 
ManagementTool/test_hawq_register_rollback.o 
ManagementTool/test_hawq_register_partition.o -L../../../src/port 
-L../../../src/port -Wl,--as-needed 
-L/tmp/build/78017950/hdb_apache/depends/libhdfs3/build/install/usr/local/hawq/lib
 
-L/tmp/build/78017950/hdb_apache/depends/libyarn/build/install/usr/local/hawq/lib
 -Wl,-rpath,'/usr/local/hawq/lib',--enable-new-dtags -L/usr/local/lib 
-L/usr/lib -L/tmp/build/78017950/hdb_apache/src/test/feature/ 
-L/tmp/build/78017950/hdb_apache/src/test/feature/lib/ 
-L/tmp/build/78017950/hdb_apache/src/interfaces/libpq 
-L/tmp/build/78017950/hdb_apache/depends/thirdparty/googletest/build/googlemock 
-L/tmp/build/78017950/hdb_apache/depends/thirdparty/googletest/build/googlemock/gtest
 -lpgport -ljson-c -levent -lyaml -lsnappy -lbz2 -lrt -lz -lreadline -lcrypt 
-ldl -lm  -lcurl   -lyarn -lkrb5 -lgtest -lpq -lxml2 -ltest -o feature-test
/usr/bin/ld: /usr/lib/libgtest.a(gtest-all.cc.o): undefined reference to symbol 
'pthread_key_delete@@GLIBC_2.2.5'
/usr/lib64/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
make: *** [all] Error 1


After append '-l pthread', the compile passed. now this happens on centos7.3 
version, centos7.2 working fine.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HAWQ-1229) Remove unused option in hawq config

[Radar Lei (JIRA)]

Radar Lei created HAWQ-1229:
---

 Summary: Remove unused option in hawq config
 Key: HAWQ-1229
 URL: https://issues.apache.org/jira/browse/HAWQ-1229
 Project: Apache HAWQ
  Issue Type: Improvement
  Components: Command Line Tools
Reporter: Radar Lei
Assignee: Ed Espino


hawq config --help offers --verbose option but it is not actually possible. We 
should remove this '--verbose' option.

See:
[gpadmin@test1 ~]$ hawq config --help

usage: hawq config [--options]

The "options" are:
   -c --change Changes a configuration parameter setting.
   -s --show   Shows the value for a specified configuration parameter.
   -l --list   Lists all configuration parameters.
   -q --quiet  Run in quiet mode.
   -v --verboseDisplays detailed status.
   -r --remove HAWQ GUC name to be removed.
   --skipvalidationSkip the system validation checks.
   --ignore-bad-hosts  Skips copying configuration files on host on which SSH 
fails

See 'hawq --help' for more information on other commands.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-1227) HAWQ init fails if User name contains capital character

[Radar Lei (JIRA)]

Radar Lei created HAWQ-1227:
---

 Summary: HAWQ init fails if User name contains capital character
 Key: HAWQ-1227
 URL: https://issues.apache.org/jira/browse/HAWQ-1227
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Command Line Tools
Reporter: Radar Lei
Assignee: Ed Espino


If user name contains capital character, then HAWQ initialize will failed to 
load toolkit file.

See log:
[WARNING]:-psql:/tmp/_gp_toolkit_tmp_20161219_135052:1: ERROR:  role "Tester" 
does not exist
psql:/tmp/_gp_toolkit_tmp_20161219_135052:1: ERROR:  role "tester" does not 
exist

We should add single quote to  ROLENAME in toolket sql file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-1156) HAWQArray restricted HAWQ version

[Radar Lei (JIRA)]

Radar Lei created HAWQ-1156:
---

 Summary: HAWQArray restricted HAWQ version
 Key: HAWQ-1156
 URL: https://issues.apache.org/jira/browse/HAWQ-1156
 Project: Apache HAWQ
  Issue Type: Improvement
  Components: Command Line Tools
Reporter: Radar Lei
Assignee: Lei Chang


Currently HAWQArray restricted HAWQ version to be '2.0', which makes some tools 
can't works with higher HAWQ version or user defined HAWQ version.

See:
tools/bin/hawqpylib/hawqarray.py



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-1129) plr-hawq default install directory should be the same as HAWQ install directory

[Radar Lei (JIRA)]

Radar Lei created HAWQ-1129:
---

 Summary: plr-hawq default install directory should be the same as 
HAWQ install directory
 Key: HAWQ-1129
 URL: https://issues.apache.org/jira/browse/HAWQ-1129
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Command Line Tools
Reporter: Radar Lei
Assignee: Lei Chang
 Fix For: 2.0.1.0-incubating


Currently plr-hawq default install path is '/usr/local/hawq_2_0_1_0-2.0.1.0', 
this is different from the hawq default install path '/usr/local/hawq_2_0_1_0'.

We should keep them the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-1121) Madlib gppkg install hang while installation actually finished

[Radar Lei (JIRA)]

Radar Lei created HAWQ-1121:
---

 Summary: Madlib gppkg install hang while installation actually 
finished
 Key: HAWQ-1121
 URL: https://issues.apache.org/jira/browse/HAWQ-1121
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Command Line Tools
Reporter: Radar Lei
Assignee: Lei Chang


When doing Madlib installation with gppkg package, the installation process 
will hang there while all the installation steps already finished.

Currently we can only install Madlib by rpm install, we should fix this to 
support Madlib gppkg installation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-1067) Append hawq version number to plr-hawq rpm pakcage name

[Radar Lei (JIRA)]

Radar Lei created HAWQ-1067:
---

 Summary: Append hawq version number to plr-hawq rpm pakcage name
 Key: HAWQ-1067
 URL: https://issues.apache.org/jira/browse/HAWQ-1067
 Project: Apache HAWQ
  Issue Type: Task
  Components: Build
Reporter: Radar Lei
Assignee: Lei Chang
 Fix For: 2.0.1.0-incubating


Now plr-hawq using plr vesion. It's not easy to align with HAWQ version.

So we'd better make below changes:
1. Add hawq version string to plr-hawq rpm package name.
2. Create a virtual rpm package to make sure user have same install interface.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-1028) Add '-d' option for hawq state to be compatible with Ambari

[Radar Lei (JIRA)]

Radar Lei created HAWQ-1028:
---

 Summary: Add '-d' option for hawq state to be compatible with 
Ambari
 Key: HAWQ-1028
 URL: https://issues.apache.org/jira/browse/HAWQ-1028
 Project: Apache HAWQ
  Issue Type: Task
  Components: Command Line Tools
Reporter: Radar Lei
Assignee: Lei Chang


Previously we removed the legacy option '-d' '--datadir' from 'hawq state' 
command. This option is used to specify the master data directory, but we never 
used it in our command line tools.

Now we found this unused option is used by current version Ambari, and will 
cause Ambari check HAWQ status failed if we removed it. So to be compatible 
with Ambari, we need to add it back until Ambari do not use this option.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-1009) Remove requirement of environment value 'MASTER_DATA_DIRECTORY'

[Radar Lei (JIRA)]

Radar Lei created HAWQ-1009:
---

 Summary: Remove requirement of environment value 
'MASTER_DATA_DIRECTORY'
 Key: HAWQ-1009
 URL: https://issues.apache.org/jira/browse/HAWQ-1009
 Project: Apache HAWQ
  Issue Type: Task
Reporter: Radar Lei
Assignee: Lei Chang


In old HAWQ command line tools, some tools will require 'MASTER_DATA_DIRECTORY' 
environment to be set. 

Since now we define 'MASTER_DATA_DIRECTORY' in hawq-site.xml, we do not need to 
set it anymore. So we'd better remove these requirements to avoid user 
confusion.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-926) Remove pycrypto from source code

[Radar Lei (JIRA)]

Radar Lei created HAWQ-926:
--

 Summary: Remove pycrypto from source code
 Key: HAWQ-926
 URL: https://issues.apache.org/jira/browse/HAWQ-926
 Project: Apache HAWQ
  Issue Type: Task
  Components: Command Line Tools
Reporter: Radar Lei
Assignee: Lei Chang


Seems there is some concerns of it's license:
https://github.com/dlitz/pycrypto/blob/master/COPYRIGHT

Let's remove it out of Apache HAWQ source code. User would need to use 'pip 
install pycrypto' to install it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-881) Output of 'hawq stop --reload' is not correct

[Radar Lei (JIRA)]

Radar Lei created HAWQ-881:
--

 Summary: Output of 'hawq stop --reload' is not correct
 Key: HAWQ-881
 URL: https://issues.apache.org/jira/browse/HAWQ-881
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Command Line Tools
Reporter: Radar Lei
Assignee: Lei Chang


hawq stop has an option to reload configuration changes. However when you 
reload the log information displayed suggests that the cluster is stopped.

We should log 'reload' instead of 'stop' while running 'hawq stop -u/--reload'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-880) Output of 'hawq stop --reload' is not correct

[Radar Lei (JIRA)]

Radar Lei created HAWQ-880:
--

 Summary: Output of 'hawq stop --reload' is not correct
 Key: HAWQ-880
 URL: https://issues.apache.org/jira/browse/HAWQ-880
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Command Line Tools
Reporter: Radar Lei
Assignee: Lei Chang


hawq stop has an option to reload configuration changes. However when you 
reload the log information displayed suggests that the cluster is stopped.

We should log 'reload' instead of 'stop' while running 'hawq stop -u/--reload'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-875) Upgrade HAWQ version to 2.0.1.0

[Radar Lei (JIRA)]

Radar Lei created HAWQ-875:
--

 Summary: Upgrade HAWQ version to 2.0.1.0
 Key: HAWQ-875
 URL: https://issues.apache.org/jira/browse/HAWQ-875
 Project: Apache HAWQ
  Issue Type: Task
  Components: Build
Reporter: Radar Lei
Assignee: Lei Chang


Since we already have a branch for HAWQ 2.0.0.0, maybe we should start to  use 
a new version number '2.0.1.0'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-872) HAWQ check fails if '--kerberos' not given when kerberos is actually enabled

[Radar Lei (JIRA)]

Radar Lei created HAWQ-872:
--

 Summary: HAWQ check fails if '--kerberos' not given when kerberos 
is actually enabled 
 Key: HAWQ-872
 URL: https://issues.apache.org/jira/browse/HAWQ-872
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Command Line Tools
Reporter: Radar Lei
Assignee: Lei Chang


When kerberos is enabled, we run 'hawq check' without '--kerberos' option, it 
will give 'global name 'hadoop' is not defined' error.


See:
checkFailed(host.hostname, "HAWQ configuration: expected '%s' for '%s', 
actual value is '%s'" % ('simple', 'hadoop.security.authentication', 
actual_config[hadoop.security.authentication]))
NameError: global name 'hadoop' is not defined



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-871) HAWQ CHECK looking for wrong YARN HA parameters

[Radar Lei (JIRA)]

Radar Lei created HAWQ-871:
--

 Summary: HAWQ CHECK looking for wrong YARN HA parameters
 Key: HAWQ-871
 URL: https://issues.apache.org/jira/browse/HAWQ-871
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Command Line Tools
Reporter: Radar Lei
Assignee: Lei Chang


When YARN HA is enabled in HDP, the following four params don't actually exist. 
However, hawq check has been hard-coded to look for them, causing a failure. 
yarn.resourcemanager.address.rm1
yarn.resourcemanager.address.rm2
yarn.resourcemanager.scheduler.address.rm1
yarn.resourcemanager.scheduler.address.rm2 


**Correct params:**
yarn.resourcemanager.hostname.rm1
yarn.resourcemanager.hostname.rm2
yarn.resourcemanager.scheduler.address (apparently, there is no rm1/rm2 
specified for scheduler)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-863) Add python module 'pycrypto'

[Radar Lei (JIRA)]

Radar Lei created HAWQ-863:
--

 Summary: Add python module 'pycrypto'
 Key: HAWQ-863
 URL: https://issues.apache.org/jira/browse/HAWQ-863
 Project: Apache HAWQ
  Issue Type: Task
  Components: Build, Command Line Tools
Reporter: Radar Lei
Assignee: Lei Chang


Previously we added python modules which HAWQ needed, but 'pycrypto' not added 
as we think it's not needed by HAWQ now.

Actually 'hawq ssh-exkeys' still need it, and it's license is 'Public domain' 
which is compatible with HAWQ. So we should add it in.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-838) Replace python module paramiko

[Radar Lei (JIRA)]

Radar Lei created HAWQ-838:
--

 Summary: Replace python module paramiko
 Key: HAWQ-838
 URL: https://issues.apache.org/jira/browse/HAWQ-838
 Project: Apache HAWQ
  Issue Type: Sub-task
  Components: Command Line Tools
Reporter: Radar Lei
Assignee: Lei Chang


Since paramiko's license is conflict with Apach HAWQ's license, so we can't 
deliver it with HAWQ package.

To save user's time to install paramiko manually, we should remove paramiko 
from HAWQ code, and find a good replacement which don't have license conflict.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-837) Add python modules into HAWQ code except paramiko

[Radar Lei (JIRA)]

Radar Lei created HAWQ-837:
--

 Summary: Add python modules into HAWQ code except paramiko
 Key: HAWQ-837
 URL: https://issues.apache.org/jira/browse/HAWQ-837
 Project: Apache HAWQ
  Issue Type: Sub-task
  Components: Build
Reporter: Radar Lei
Assignee: Lei Chang


Add below python modules which compatible with Apache HAWQ license. Paramiko 
won't be added.

figleaf BSD
pg8000  BSD
pygresqlPython
simplejson  MIT
pyyaml  MIT
lockfileOSI Approved :: MIT License
psi MIT
pychecker   BSD-like
unittest2   OSI Approved :: BSD License
pycryptoPublic domain



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)