Re: Re: [DISCUSS] Apache HAWQ Graduation from Incubator

[Yi JIN]

+1 to Lei, I fully support Lei Change as Apache HAWQ project PMC Chairman,
he deserves this role, not only outstanding contributions to this project
in a very long period till today also his solid leadership and vision.

Best,
Yi  Jin

On Fri, Jun 1, 2018 at 3:17 AM, Ed Espino  wrote:

> Ruilong,
>
> I also give my full support for Lei Chang as the Apache HAWQ project's
> initial PMC Chairman. His leadership and vision have contributed immensely
> to the project.
>
> Regards,
> -=e
>
> On Thu, May 31, 2018 at 6:32 AM, Ruilong Huo  wrote:
>
> > Great progress towards Apache HAWQ graduation! Thanks Radar for pushing
> > this forward!
> >
> > I would like to nominate Lei Chang as the PMC Chair. He initiated HAWQ
> > project several years ago, led the development, brought it to Apache
> > incubation, and has always been active in HAWQ community to make it a
> > world-leading big data product as well as a successfully Apache project.
> > There is no doubt that he is perfect for the role. And I believe he will
> > continue to share his insights and go even further with HAWQ after
> > graduation.
> >
> >
> > Best regards,
> > Ruilong Huo
> >
> >
> > At 2018-05-31 15:40:42, "Radar Lei"  wrote:
> > >Just find some good material for nominating chair from Roman's email,
> > >thanks Roman.
> > >
> > >I think we can follow this to nominate a Chair in this thread too. Guys
> > >please help to nominate or self-nominate. Thanks a lot.
> > >
> > >See:
> > >At the very minimum your resolution will contain: 1. A name of the
> project
> > >2. A list of proposed PMC 3. A proposed PMC chair
> > >
> > >On #3 I typically recommend podlings I mento to setup a rotating chair
> > >policy. This is, in no way, an ASF requirement so feel free to ignore
> it,
> > >but it worked well before. The chair will be expected up for rotation
> > every
> > >year. It will be more that ok for the same person to self-nominate once
> > the
> > >year is up -- but at the same time it'll be up to the same person to
> > >actually kick off a thread asking if anybody else is interested in
> serving
> > >as a chair for the next year. Of course, if there multiple candidates
> > there
> > >will have to be a vote.
> > >
> > >
> > >Regards,
> > >Radar
> > >
> > >On Tue, May 29, 2018 at 10:05 PM, Radar Lei  wrote:
> > >
> > >> Thanks Roman.
> > >>
> > >> This make sense, I will start to draft the resolution. BTW, we would
> > need
> > >> to nominate a chair, I guess it's the last piece we need to draft the
> > >> resolution.
> > >>
> > >> Regards,
> > >> Radar
> > >>
> > >> On Tue, May 29, 2018 at 11:24 AM, Roman Shaposhnik <
> > ro...@shaposhnik.org>
> > >> wrote:
> > >>
> > >>> On Sun, May 27, 2018 at 11:37 PM, Radar Lei  wrote:
> > >>> > Hi Roman,
> > >>> >
> > >>> > We have confirmed with each HAWQ committer whether they want to
> > remain
> > >>> with
> > >>> > HAWQ project.  As a summary, 37 PPMC members(including two mentors)
> > and
> > >>> 7
> > >>> > committers confirmed they want to remain with HAWQ. [1] The total
> > >>> committers
> > >>> > number 44 seems pretty close with PPMC member number 37, is it good
> > >>> enough
> > >>> > to make PMC == committers as our graduation resolution?
> > >>>
> > >>> PMC == committers in this case makes perfect sense to me!
> > >>>
> > >>> > Should we update the whimsy and project webpage now or do update
> > after
> > >>> graduation?
> > >>>
> > >>> It really doesn't matter much. Your next step is to draft a
> resolution
> > >>> similar to:
> > >>> https://www.mail-archive.com/general@incubator.apache.org/ms
> > >>> g56982.html
> > >>>
> > >>> and start a [DISCUSS] thread similar to the above.
> > >>>
> > >>> Makes sense?
> > >>>
> > >>> Thanks,
> > >>> Roman.
> > >>>
> > >>
> > >>
> >
>

Re: Remain with HAWQ project or not?

[Yi JIN]

Yes， I would like to remain a committer/PMC member of Apache HAWQ project.

Best,
Yi (yjin)

On Wed, May 9, 2018 at 12:41 AM, Ed Espino  wrote:

> Yes I would like to remain a committer/PMC member of Apache HAWQ project.
>
> Regards,
> -=e
>
> On Mon, May 7, 2018 at 1:11 AM, Radar Lei  wrote:
>
> > HAWQ committers,
> >
> > Per the discussion in "Apache HAWQ graduation from incubator?" [1], we
> want
> > to setup the PMC as part of HAWQ graduation resolution.
> >
> > So we'd like to confirm whether you want to remain as a committer/PMC
> > member of Apache HAWQ project?
> >
> > If you'd like to remain with HAWQ project, it's welcome and please
> > *respond**
> > 'Yes'* in this thread, or *respond 'No'* if you are not interested in any
> > more. Thanks.
> >
> > This thread will be available for at least 72 hours, after that, we will
> > send individual confirm emails.
> >
> > [1]
> > https://lists.apache.org/thread.html/b4a0b5671ce377b3d51c9b7ab00496
> > a1eebfcbf1696ce8b67e078c64@%3Cdev.hawq.apache.org%3E
> >
> > Regards,
> > Radar
> >
>

Re: New Committer: Shubham Sharma

[Yi JIN]

Congratulations!

Yi (yjin)


On Thu, Apr 26, 2018 at 1:26 PM, Shubham Sharma  wrote:

> Thank you everyone. I'm honored to be given this opportunity and really
> excited to become a committer.
>
> On Tue, Apr 24, 2018 at 11:02 PM, Ivan Weng  wrote:
>
> > Congrats to Shubham.
> >
> > Regards,
> > Ivan
> >
> > On Wed, Apr 25, 2018 at 10:59 AM, Yandong Yao  wrote:
> >
> > > Congrats!
> > >
> > > On Wed, Apr 25, 2018 at 10:06 AM, Radar Lei  wrote:
> > >
> > > > Congrats to Shubham!
> > > >
> > > > Regards,
> > > > Radar
> > > >
> > > > On Wed, Apr 25, 2018 at 3:28 AM, Shivram Mani <
> shivram.m...@gmail.com>
> > > > wrote:
> > > >
> > > > > The Project Management Committee (PMC) for Apache HAWQ (incubating)
> > > > > has invited *Shubham Sharma* to become a committer and we are
> pleased
> > > > > to announce that he has accepted.
> > > > > Being a committer enables easier contribution to the project since
> > > > > there is no need to go via the patch submission process. This
> should
> > > > > enable better productivity.
> > > > > Please join us in congratulating him and we are looking forward to
> > > > > collaborating with him in the open source community.
> > > > >
> > > > >
> > > > > His contribution includes (but not limited to):
> > > > >
> > > > >
> > > > >
> > > > > 15 commits to components in hawq, including contributions to PXF
> and
> > > > master
> > > > > branch CI
> > > > > - Reference - https://github.com/apache/
> > incubator-hawq/pulls?utf8=%E2%
> > > > > 9C%93=is%3Apr+author%3Aoutofmem0ry
> > > > >
> > > > > HAWQ-1572: Fix travis ci build failure. Thrift/Boost
> incompatibility
> > > > > HAWQ-1559: Fix travis ci build failure due to missing apr package
> > > > > HAWQ-1553: Add option to hawq extract to specify log directory
> > > > > HAWQ-1368: Add option to hawq register to specify log directory
> > > > > HAWQ-1549: Fix standby resync when using -M smart/fast/immediate
> > > > > HAWQ-1548: Correct ambiguous message while logging hawq utilization
> > > > > HAWQ-1527: Enabled partition filtering for integral data types
> > > > > HAWQ-1524:  Fix travis ci build failure caused after protobuf
> upgrade
> > > to
> > > > > 3.4
> > > > > HAWQ-1198: Fix to filter out irrelevant fragments while accessing
> > Hive
> > > > > HAWQ-1273:  Removed incorrect references from gplogfilter
> > > > > HAWQ-1504:  Fixed namenode hang during docker container restart
> > > > > HAWQ-1503: Fixed ftp link causing build failure on centos-6 docker
> > > > > HAWQ-1495: Corrected answer file to match insert statement
> > > > > HAWQ-1373: Added feature to reload GUC values using hawq
> > reload-config
> > > > > HAWQ-1480: Added feature for packing a core file
> > > > >
> > > > > 4 commits to hawq documentation (https://github.com/apache/
> > > > incubator-hawq
> > > > > -docs/pulls?utf8=%E2%9C%93=is%3Apr+author%3Aoutofmem0ry)
> > > > >  - Two closed and two open pull requests
> > > > >
> > > > > HAWQ-1562: Fixed incorrect path to default log directory
> > > > > HAWQ-1368: Add option to hawq register to specify log directory
> > > > > HAWQ-1480: Added documentation for packcore utility
> > > > > HAWQ-1553: Add option to hawq extract to specify log directory
> > > > >
> > > > > *JIRA:*
> > > > > 12 JIRAs were created by Shubham. Pull requests for all 13 issues
> has
> > > > been
> > > > > merged to master.
> > > > > https://issues.apache.org/jira/browse/HAWQ-1572?jql=
> > > > > project%20%3D%20HAWQ%20AND%20reporter%20in%20(outofmemory)
> > > > > 3 JIRAs were assigned to Shubham. Pull requests for these issues
> has
> > > been
> > > > > merged to master. [HAWQ-1198, HAWQ-1368, HAWQ-1373 ]
> > > > >
> > > > > Other contributions include code review(https://github.com/
> > > > > apache/incubator-
> > > > > hawq/pull/1318) and email discussions.
> > > > >
> > > > > --
> > > > > Shivram Mani
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Best Regards,
> > > Yandong
> > >
> >
>
>
>
> --
> Regards,
> Shubham Sharma
> Staff Customer Engineer
> Pivotal Global Support Services
> ssha...@pivotal.io
> Direct Tel: +1(510)-304-8201
> Office Hours: Mon-Fri 9:00 am to 5:00 pm PDT
> Out of Office Hours Contact +1 877-477-2269
>

Re: New Committer: Lav Jain

[Yi JIN]

Congratulations!

Yi (yjin)

On Thu, Apr 26, 2018 at 1:18 PM, Hongxu Ma  wrote:

> Congrats!
>
> 在 25/04/2018 03:26, Shivram Mani 写道:
> > The Project Management Committee (PMC) for Apache HAWQ (incubating)
> > has invited *Lav Jain* to become a committer and we are pleased to
> > announce that he has accepted.
> > Being a committer enables easier contribution to the project since
> > there is no need to go via the patch submission process. This should
> > enable better productivity.
> > Please join us in congratulating him and we are looking forward to
> > collaborating with him in the open source community.
> > His contribution includes (but not limited to):
> >
> >
> > His contribution includes (but not limited to):
> >- *12 commits* with some major components in hawq involved,
> > including contributions to *PXF* and* Ranger Plugin Service*
> >- Numerous other contributions including documentation, test, build,
> > command line
> > tools, code refactor, etc
> >- *300+ commits* to HAWQ and PXF *CI and testing*
> >
> > HAWQ-1584. Propogate bridge endIteration exception during write
> > HAWQ-1580. Set PXF port number to 5888 for GPDB
> > HAWQ-1545. Fix pxf-private.classpath file for Cloudera tarball distro
> > HAWQ-1533. Fix compiler warnings when building PXF code
> > HAWQ-1523. Provide ability to bundle PXF artifacts into a single tarbal
> > HAWQ-1492. Enable packaging for PXF JDBC plugin.
> > HAWQ-1461. Improve partition parameters validation for PXF-JDBC plugin.
> > HAWQ-1454. Exclude certain jars from Ranger Plugin Service packaging
> > HAWQ-1415. Set the default_value of JAVA_HOME for running RPS
> > HAWQ-1420. Enable build of Ranger plugin open source RPM
> > HAWQ-1415. Set the default_value of JAVA_HOME for running RPS
> >
> > **JIRA*:*
> > *11 JIRAs* were created or assigned to Lav
> > 10 JIRAs are closed and 1 JIRA is still open
> > https://issues.apache.org/jira/browse/HAWQ-1492?jql=
> > project%20%3D%20HAWQ%20AND%20(assignee%20in%20(lavjain)%
> > 20OR%20reporter%20in%20(lavjain))
> >
> > *Related Contributions*
> > Lav has also been a very active contributor to the *HAWQ Ambari project*
> > *40 JIRAs* were created or assigned to Lav
> > https://issues.apache.org/jira/browse/AMBARI-21377?jql=
> > project%20%3D%20AMBARI%20AND%20(assignee%20in%20(lavjain)%
> > 20OR%20reporter%20in%20(lavjain))
> >
> >
> >
> >
>
> --
> Regards,
> Hongxu.
>
>

Re: [ANNOUNCE] Apache HAWQ 2.3.0.0-incubating Release

[Yi JIN]

Thank you Radar!

Best,
Yi (yjin)

On Mon, Mar 26, 2018 at 6:25 PM, Radar Lei <r...@pivotal.io> wrote:

> Thanks all!
>
> It's my honor to be release manager of next Apache HAWQ release, I will
> draft and send out a release plan soon.
>
> Regards,
> Radar
>
> On Mon, Mar 26, 2018 at 1:56 PM, Hongxu Ma <inte...@outlook.com> wrote:
>
> > +1 for radar,  
> >
> > 在 23/03/2018 10:04, Radar Lei 写道:
> > > Thanks Yi for your great work on Apache HAWQ 2.3.0.0-incubating
> release.
> > >
> > > I would like to volunteer to be next release manager of Apache HAWQ.
> > >
> > > As a committer, I used to work close with previous excellent release
> > > managers to prepare the releases, now I want to contribute more on
> Apache
> > > HAWQ releases. I hope we can keep HAWQ release regularly and always
> keep
> > > Apache HAWQ project in a healthy status. Thanks.
> > >
> > >
> > > Regards,
> > > Radar
> > >
> > > On Fri, Mar 23, 2018 at 8:48 AM, Yi JIN <y...@apache.org> wrote:
> > >
> > >> Thanks everyone for making this real, especially the help from Ruilong
> > and
> > >> Radar!
> > >>
> > >> To make it a more successful apache project and community, we need to
> > keep
> > >> the release cadence. Who would like to be volunteer for the next
> > >> release manager
> > >> and drive the effort? Thanks.
> > >>
> > >> Best,
> > >> Yi (yjin)
> > >>
> > >> On Fri, Mar 23, 2018 at 11:40 AM, Lei Chang <chang.lei...@gmail.com>
> > >> wrote:
> > >>
> > >>> congrats!
> > >>>
> > >>> Cheers
> > >>> Lei
> > >>>
> > >>>
> > >>>
> > >>> On Wed, Mar 21, 2018 at 1:58 PM, Yi JIN <y...@apache.org> wrote:
> > >>>
> > >>>> Thanks John, I fixed it and resent the announcement.
> > >>>>
> > >>>> Best,
> > >>>> Yi (yjin)
> > >>>>
> > >>>> On Wed, Mar 21, 2018 at 4:14 PM, Amy Bai <a...@pivotal.io> wrote:
> > >>>>
> > >>>>> Congratulations! Thanks Yi for this release!
> > >>>>>
> > >>>>> On Wed, Mar 21, 2018 at 11:15 AM, John D. Ament <
> > >> johndam...@apache.org
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Dropping announce@.
> > >>>>>>
> > >>>>>> On Tue, Mar 20, 2018 at 10:28 PM Yi JIN <y...@apache.org> wrote:
> > >>>>>>
> > >>>>>>> Apache HAWQ (incubating) Project Team is proud to announce Apache
> > >>>>>>> HAWQ 2.3.0.0-incubating has been released.
> > >>>>>>>
> > >>>>>>> Apache HAWQ (incubating) combines exceptional MPP-based analytics
> > >>>>>>> performance, robust ANSI SQL compliance, Hadoop ecosystem
> > >>>>>>> integration and manageability, and flexible data-store format
> > >>>>>>> support, all natively in Hadoop, no connectors required. Built
> > >>>>>>> from a decade’s worth of massively parallel processing (MPP)
> > >>>>>>> expertise developed through the creation of the Pivotal
> > >>>>>>> Greenplum® enterprise database and open source PostgreSQL, HAWQ
> > >>>>>>> enables to you to swiftly and interactively query Hadoop data,
> > >>>>>>> natively via HDFS.
> > >>>>>>>
> > >>>>>>> *Download Link*:
> > >>>>>>>
> > >>>>>>> https://dist.apache.org/repos/dist/release/incubator/hawq/2.
> > >>>>>> 3.0.0-incubating/
> > >>>>>>>
> > >>>>>> This has been a hot topic as of late.  This link is incorrect.
> > >>> Please
> > >>>>>> review http://www.apache.org/dev/release-publishing#
> > >>> distribution_dist
> > >>>>> and
> > >>>>>> related pages and fix the link.  Please resend this email once
> > >> you've
> > >>>>>> corrected it.
> > >>>>>>
> > >>>>>>
> > >>>>>>> *About this release*
> > >>>>>>> This is a release having both source code and binary
> > >>>>>>>
> > >>>>>>> All changes:
> > >>>>>>>
> > >>>>>>> https://cwiki.apache.org/confluence/display/HAWQ/
> > >>>> Apache+HAWQ+2.3.0.0-
> > >>>>>> incubating+Release
> > >>>>>>>
> > >>>>>>> *HAWQ Resources:*
> > >>>>>>>
> > >>>>>>> - JIRA: https://issues.apache.org/jira/browse/HAWQ
> > >>>>>>> - Wiki:
> > >>>>>>> https://cwiki.apache.org/confluence/display/HAWQ/
> > >> Apache+HAWQ+Home
> > >>>>>>> - Mailing list(s): dev@hawq.incubator.apache.org
> > >>>>>>>u...@hawq.incubator.apache.org
> > >>>>>>>
> > >>>>>>> *Know more about HAWQ:*
> > >>>>>>> http://hawq.apache.org
> > >>>>>>>
> > >>>>>>> - Apache HAWQ (incubating) Team
> > >>>>>>>
> > >>>>>>> =
> > >>>>>>> *Disclaimer*
> > >>>>>>>
> > >>>>>>> Apache HAWQ (incubating) is an effort undergoing incubation at
> > >> The
> > >>>>>>> Apache Software Foundation (ASF), sponsored by the name of Apache
> > >>>>>>> Incubator PMC. Incubation is required of all newly accepted
> > >>>>>>> projects until a further review indicates that the
> > >>>>>>> infrastructure, communications, and decision making process have
> > >>>>>>> stabilized in a manner consistent with other successful ASF
> > >>>>>>> projects. While incubation status is not necessarily a reflection
> > >>>>>>> of the completeness or stability of the code, it does indicate
> > >>>>>>> that the project has yet to be fully endorsed by the ASF.
> > >>>>>>>
> >
> > --
> > Regards,
> > Hongxu.
> >
> >
>

Re: [ANNOUNCE] Apache HAWQ 2.3.0.0-incubating Release

[Yi JIN]

+1

Thank you Radar for taking this work! You have solid project management and
release manage experiences, I'm pretty sure you are good at this job :)

Best,
Yi (yjin)

On Fri, Mar 23, 2018 at 1:04 PM, Radar Lei <r...@pivotal.io> wrote:

> Thanks Yi for your great work on Apache HAWQ 2.3.0.0-incubating release.
>
> I would like to volunteer to be next release manager of Apache HAWQ.
>
> As a committer, I used to work close with previous excellent release
> managers to prepare the releases, now I want to contribute more on Apache
> HAWQ releases. I hope we can keep HAWQ release regularly and always keep
> Apache HAWQ project in a healthy status. Thanks.
>
>
> Regards,
> Radar
>
> On Fri, Mar 23, 2018 at 8:48 AM, Yi JIN <y...@apache.org> wrote:
>
> > Thanks everyone for making this real, especially the help from Ruilong
> and
> > Radar!
> >
> > To make it a more successful apache project and community, we need to
> keep
> > the release cadence. Who would like to be volunteer for the next
> > release manager
> > and drive the effort? Thanks.
> >
> > Best,
> > Yi (yjin)
> >
> > On Fri, Mar 23, 2018 at 11:40 AM, Lei Chang <chang.lei...@gmail.com>
> > wrote:
> >
> > > congrats!
> > >
> > > Cheers
> > > Lei
> > >
> > >
> > >
> > > On Wed, Mar 21, 2018 at 1:58 PM, Yi JIN <y...@apache.org> wrote:
> > >
> > > > Thanks John, I fixed it and resent the announcement.
> > > >
> > > > Best,
> > > > Yi (yjin)
> > > >
> > > > On Wed, Mar 21, 2018 at 4:14 PM, Amy Bai <a...@pivotal.io> wrote:
> > > >
> > > > > Congratulations! Thanks Yi for this release!
> > > > >
> > > > > On Wed, Mar 21, 2018 at 11:15 AM, John D. Ament <
> > johndam...@apache.org
> > > >
> > > > > wrote:
> > > > >
> > > > > > Dropping announce@.
> > > > > >
> > > > > > On Tue, Mar 20, 2018 at 10:28 PM Yi JIN <y...@apache.org> wrote:
> > > > > >
> > > > > > > Apache HAWQ (incubating) Project Team is proud to announce
> Apache
> > > > > > > HAWQ 2.3.0.0-incubating has been released.
> > > > > > >
> > > > > > > Apache HAWQ (incubating) combines exceptional MPP-based
> analytics
> > > > > > > performance, robust ANSI SQL compliance, Hadoop ecosystem
> > > > > > > integration and manageability, and flexible data-store format
> > > > > > > support, all natively in Hadoop, no connectors required. Built
> > > > > > > from a decade’s worth of massively parallel processing (MPP)
> > > > > > > expertise developed through the creation of the Pivotal
> > > > > > > Greenplum® enterprise database and open source PostgreSQL, HAWQ
> > > > > > > enables to you to swiftly and interactively query Hadoop data,
> > > > > > > natively via HDFS.
> > > > > > >
> > > > > > > *Download Link*:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/release/incubator/hawq/2.
> > > > > > 3.0.0-incubating/
> > > > > > >
> > > > > > >
> > > > > > This has been a hot topic as of late.  This link is incorrect.
> > > Please
> > > > > > review http://www.apache.org/dev/release-publishing#
> > > distribution_dist
> > > > > and
> > > > > > related pages and fix the link.  Please resend this email once
> > you've
> > > > > > corrected it.
> > > > > >
> > > > > >
> > > > > > > *About this release*
> > > > > > > This is a release having both source code and binary
> > > > > > >
> > > > > > > All changes:
> > > > > > >
> > > > > > > https://cwiki.apache.org/confluence/display/HAWQ/
> > > > Apache+HAWQ+2.3.0.0-
> > > > > > incubating+Release
> > > > > > >
> > > > > > >
> > > > > > > *HAWQ Resources:*
> > > > > > >
> > > > > > >- JIRA: https://issues.apache.org/jira/browse/HAWQ
> > > > > > >- Wiki:
> > > > > > > https://cwiki.apache.org/confluence/display/HAWQ/
> > Apache+HAWQ+Home
> > > > > > >- Mailing list(s): dev@hawq.incubator.apache.org
> > > > > > >   u...@hawq.incubator.apache.org
> > > > > > >
> > > > > > > *Know more about HAWQ:*
> > > > > > > http://hawq.apache.org
> > > > > > >
> > > > > > > - Apache HAWQ (incubating) Team
> > > > > > >
> > > > > > > =
> > > > > > > *Disclaimer*
> > > > > > >
> > > > > > > Apache HAWQ (incubating) is an effort undergoing incubation at
> > The
> > > > > > > Apache Software Foundation (ASF), sponsored by the name of
> Apache
> > > > > > > Incubator PMC. Incubation is required of all newly accepted
> > > > > > > projects until a further review indicates that the
> > > > > > > infrastructure, communications, and decision making process
> have
> > > > > > > stabilized in a manner consistent with other successful ASF
> > > > > > > projects. While incubation status is not necessarily a
> reflection
> > > > > > > of the completeness or stability of the code, it does indicate
> > > > > > > that the project has yet to be fully endorsed by the ASF.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] Apache HAWQ 2.3.0.0-incubating Release

[Yi JIN]

Thanks everyone for making this real, especially the help from Ruilong and
Radar!

To make it a more successful apache project and community, we need to keep
the release cadence. Who would like to be volunteer for the next
release manager
and drive the effort? Thanks.

Best,
Yi (yjin)

On Fri, Mar 23, 2018 at 11:40 AM, Lei Chang <chang.lei...@gmail.com> wrote:

> congrats!
>
> Cheers
> Lei
>
>
>
> On Wed, Mar 21, 2018 at 1:58 PM, Yi JIN <y...@apache.org> wrote:
>
> > Thanks John, I fixed it and resent the announcement.
> >
> > Best,
> > Yi (yjin)
> >
> > On Wed, Mar 21, 2018 at 4:14 PM, Amy Bai <a...@pivotal.io> wrote:
> >
> > > Congratulations! Thanks Yi for this release!
> > >
> > > On Wed, Mar 21, 2018 at 11:15 AM, John D. Ament <johndam...@apache.org
> >
> > > wrote:
> > >
> > > > Dropping announce@.
> > > >
> > > > On Tue, Mar 20, 2018 at 10:28 PM Yi JIN <y...@apache.org> wrote:
> > > >
> > > > > Apache HAWQ (incubating) Project Team is proud to announce Apache
> > > > > HAWQ 2.3.0.0-incubating has been released.
> > > > >
> > > > > Apache HAWQ (incubating) combines exceptional MPP-based analytics
> > > > > performance, robust ANSI SQL compliance, Hadoop ecosystem
> > > > > integration and manageability, and flexible data-store format
> > > > > support, all natively in Hadoop, no connectors required. Built
> > > > > from a decade’s worth of massively parallel processing (MPP)
> > > > > expertise developed through the creation of the Pivotal
> > > > > Greenplum® enterprise database and open source PostgreSQL, HAWQ
> > > > > enables to you to swiftly and interactively query Hadoop data,
> > > > > natively via HDFS.
> > > > >
> > > > > *Download Link*:
> > > > >
> > > > > https://dist.apache.org/repos/dist/release/incubator/hawq/2.
> > > > 3.0.0-incubating/
> > > > >
> > > > >
> > > > This has been a hot topic as of late.  This link is incorrect.
> Please
> > > > review http://www.apache.org/dev/release-publishing#
> distribution_dist
> > > and
> > > > related pages and fix the link.  Please resend this email once you've
> > > > corrected it.
> > > >
> > > >
> > > > > *About this release*
> > > > > This is a release having both source code and binary
> > > > >
> > > > > All changes:
> > > > >
> > > > > https://cwiki.apache.org/confluence/display/HAWQ/
> > Apache+HAWQ+2.3.0.0-
> > > > incubating+Release
> > > > >
> > > > >
> > > > > *HAWQ Resources:*
> > > > >
> > > > >- JIRA: https://issues.apache.org/jira/browse/HAWQ
> > > > >- Wiki:
> > > > > https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+Home
> > > > >- Mailing list(s): dev@hawq.incubator.apache.org
> > > > >   u...@hawq.incubator.apache.org
> > > > >
> > > > > *Know more about HAWQ:*
> > > > > http://hawq.apache.org
> > > > >
> > > > > - Apache HAWQ (incubating) Team
> > > > >
> > > > > =
> > > > > *Disclaimer*
> > > > >
> > > > > Apache HAWQ (incubating) is an effort undergoing incubation at The
> > > > > Apache Software Foundation (ASF), sponsored by the name of Apache
> > > > > Incubator PMC. Incubation is required of all newly accepted
> > > > > projects until a further review indicates that the
> > > > > infrastructure, communications, and decision making process have
> > > > > stabilized in a manner consistent with other successful ASF
> > > > > projects. While incubation status is not necessarily a reflection
> > > > > of the completeness or stability of the code, it does indicate
> > > > > that the project has yet to be fully endorsed by the ASF.
> > > > >
> > > >
> > >
> >
>

[ANNOUNCE] Apache HAWQ 2.3.0.0-incubating Release

[Yi JIN]

Apache HAWQ (incubating) Project Team is proud to announce Apache
HAWQ 2.3.0.0-incubating has been released.

Apache HAWQ (incubating) combines exceptional MPP-based analytics
performance, robust ANSI SQL compliance, Hadoop ecosystem
integration and manageability, and flexible data-store format
support, all natively in Hadoop, no connectors required. Built
from a decade’s worth of massively parallel processing (MPP)
expertise developed through the creation of the Pivotal
Greenplum® enterprise database and open source PostgreSQL, HAWQ
enables to you to swiftly and interactively query Hadoop data,
natively via HDFS.

*Download Link*:

http://hawq.apache.org/#download



*About this release*
This is a release having both source code and binary

All changes:
https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.3.0.0-incubating+Release


*HAWQ Resources:*

   - JIRA: https://issues.apache.org/jira/browse/HAWQ
   - Wiki: https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+Home
   - Mailing list(s): dev@hawq.incubator.apache.org
  u...@hawq.incubator.apache.org

*Know more about HAWQ:*
http://hawq.apache.org

- Apache HAWQ (incubating) Team

=
*Disclaimer*

Apache HAWQ (incubating) is an effort undergoing incubation at The
Apache Software Foundation (ASF), sponsored by the name of Apache
Incubator PMC. Incubation is required of all newly accepted
projects until a further review indicates that the
infrastructure, communications, and decision making process have
stabilized in a manner consistent with other successful ASF
projects. While incubation status is not necessarily a reflection
of the completeness or stability of the code, it does indicate
that the project has yet to be fully endorsed by the ASF.

Re: [ANNOUNCE] Apache HAWQ 2.3.0.0-incubating Release

[Yi JIN]

Thanks John, I fixed it and resent the announcement.

Best,
Yi (yjin)

On Wed, Mar 21, 2018 at 4:14 PM, Amy Bai <a...@pivotal.io> wrote:

> Congratulations! Thanks Yi for this release!
>
> On Wed, Mar 21, 2018 at 11:15 AM, John D. Ament <johndam...@apache.org>
> wrote:
>
> > Dropping announce@.
> >
> > On Tue, Mar 20, 2018 at 10:28 PM Yi JIN <y...@apache.org> wrote:
> >
> > > Apache HAWQ (incubating) Project Team is proud to announce Apache
> > > HAWQ 2.3.0.0-incubating has been released.
> > >
> > > Apache HAWQ (incubating) combines exceptional MPP-based analytics
> > > performance, robust ANSI SQL compliance, Hadoop ecosystem
> > > integration and manageability, and flexible data-store format
> > > support, all natively in Hadoop, no connectors required. Built
> > > from a decade’s worth of massively parallel processing (MPP)
> > > expertise developed through the creation of the Pivotal
> > > Greenplum® enterprise database and open source PostgreSQL, HAWQ
> > > enables to you to swiftly and interactively query Hadoop data,
> > > natively via HDFS.
> > >
> > > *Download Link*:
> > >
> > > https://dist.apache.org/repos/dist/release/incubator/hawq/2.
> > 3.0.0-incubating/
> > >
> > >
> > This has been a hot topic as of late.  This link is incorrect.  Please
> > review http://www.apache.org/dev/release-publishing#distribution_dist
> and
> > related pages and fix the link.  Please resend this email once you've
> > corrected it.
> >
> >
> > > *About this release*
> > > This is a release having both source code and binary
> > >
> > > All changes:
> > >
> > > https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.3.0.0-
> > incubating+Release
> > >
> > >
> > > *HAWQ Resources:*
> > >
> > >- JIRA: https://issues.apache.org/jira/browse/HAWQ
> > >- Wiki:
> > > https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+Home
> > >- Mailing list(s): dev@hawq.incubator.apache.org
> > >   u...@hawq.incubator.apache.org
> > >
> > > *Know more about HAWQ:*
> > > http://hawq.apache.org
> > >
> > > - Apache HAWQ (incubating) Team
> > >
> > > =
> > > *Disclaimer*
> > >
> > > Apache HAWQ (incubating) is an effort undergoing incubation at The
> > > Apache Software Foundation (ASF), sponsored by the name of Apache
> > > Incubator PMC. Incubation is required of all newly accepted
> > > projects until a further review indicates that the
> > > infrastructure, communications, and decision making process have
> > > stabilized in a manner consistent with other successful ASF
> > > projects. While incubation status is not necessarily a reflection
> > > of the completeness or stability of the code, it does indicate
> > > that the project has yet to be fully endorsed by the ASF.
> > >
> >
>

[ANNOUNCE] Apache HAWQ 2.3.0.0-incubating Release

[Yi JIN]

Apache HAWQ (incubating) Project Team is proud to announce Apache
HAWQ 2.3.0.0-incubating has been released.

Apache HAWQ (incubating) combines exceptional MPP-based analytics
performance, robust ANSI SQL compliance, Hadoop ecosystem
integration and manageability, and flexible data-store format
support, all natively in Hadoop, no connectors required. Built
from a decade’s worth of massively parallel processing (MPP)
expertise developed through the creation of the Pivotal
Greenplum® enterprise database and open source PostgreSQL, HAWQ
enables to you to swiftly and interactively query Hadoop data,
natively via HDFS.

*Download Link*:

http://www.apache.org/dist/incubator/hawq/2.3.0.0-incubating/



*About this release*
This is a release having both source code and binary

All changes:
https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.3.0.0-incubating+Release


*HAWQ Resources:*

   - JIRA: https://issues.apache.org/jira/browse/HAWQ
   - Wiki: https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+Home
   - Mailing list(s): dev@hawq.incubator.apache.org
  u...@hawq.incubator.apache.org

*Know more about HAWQ:*
http://hawq.apache.org

- Apache HAWQ (incubating) Team

=
*Disclaimer*

Apache HAWQ (incubating) is an effort undergoing incubation at The
Apache Software Foundation (ASF), sponsored by the name of Apache
Incubator PMC. Incubation is required of all newly accepted
projects until a further review indicates that the
infrastructure, communications, and decision making process have
stabilized in a manner consistent with other successful ASF
projects. While incubation status is not necessarily a reflection
of the completeness or stability of the code, it does indicate
that the project has yet to be fully endorsed by the ASF.

[ANNOUNCE] Apache HAWQ 2.3.0.0-incubating Release

[Yi JIN]

Apache HAWQ (incubating) Project Team is proud to announce Apache
HAWQ 2.3.0.0-incubating has been released.

Apache HAWQ (incubating) combines exceptional MPP-based analytics
performance, robust ANSI SQL compliance, Hadoop ecosystem
integration and manageability, and flexible data-store format
support, all natively in Hadoop, no connectors required. Built
from a decade’s worth of massively parallel processing (MPP)
expertise developed through the creation of the Pivotal
Greenplum® enterprise database and open source PostgreSQL, HAWQ
enables to you to swiftly and interactively query Hadoop data,
natively via HDFS.

*Download Link*:
https://dist.apache.org/repos/dist/release/incubator/hawq/2.3.0.0-incubating/

*About this release*
This is a release having both source code and binary

All changes:
https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.3.0.0-incubating+Release


*HAWQ Resources:*

   - JIRA: https://issues.apache.org/jira/browse/HAWQ
   - Wiki: https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+Home
   - Mailing list(s): dev@hawq.incubator.apache.org
  u...@hawq.incubator.apache.org

*Know more about HAWQ:*
http://hawq.apache.org

- Apache HAWQ (incubating) Team

=
*Disclaimer*

Apache HAWQ (incubating) is an effort undergoing incubation at The
Apache Software Foundation (ASF), sponsored by the name of Apache
Incubator PMC. Incubation is required of all newly accepted
projects until a further review indicates that the
infrastructure, communications, and decision making process have
stabilized in a manner consistent with other successful ASF
projects. While incubation status is not necessarily a reflection
of the completeness or stability of the code, it does indicate
that the project has yet to be fully endorsed by the ASF.

Re: Re: [VOTE]: Apache HAWQ 2.3.0.0-incubating Release (RC2)

[Yi JIN]

Guys, this is a reminder, please vote asap. Thanks

Best,
Yi (yjin)

On Fri, Mar 2, 2018 at 1:07 PM, Hubert Zhang <hzh...@pivotal.io> wrote:

> +1
> Build and Installed. Tests passed.
>
> On Thu, Mar 1, 2018 at 2:01 PM, Ruilong Huo <h...@apache.org> wrote:
>
> > +1 for the HAWQ 2.3.0.0-incubating RC2
> >
> >
> > Here are the checks have been done:
> >
> >
> > 1. Reviewed LICENSE, NOTICE, DISCLAIMER, and pom.xml.
> >
> >
> > 2. Passed RAT configuration check successfully.
> >
> >
> > 3. Passed source and rpm package signature, md5 and sha256 checksum.
> >
> >
> > 4. Compiled from source tarball and installed RC2 with feature check
> > successful.
> >
> >
> > 5. Downloaded rpm tarball, installed hawq on CentOS 7.2 VM following
> > https://cwiki.apache.org/confluence/display/HAWQ/Build+
> > Package+and+Install+with+RPM. The initialization and basic query passed.
> >
> >
> > Best regards,
> > Ruilong Huo
> > At 2018-03-01 13:50:58, "Bai Jie" <amybai6...@gmail.com> wrote:
> > >Build from branch 2.3.0.0 source code, installed , init , run feature
> test
> > >and simple queries. Looks good to me. +1
> > >
> > >On Wed, Feb 28, 2018 at 11:24 AM, Hongxu Ma <inte...@outlook.com>
> wrote:
> > >
> > >> +1
> > >>
> > >> Both source and rpm package are verified in my environment: Red Hat
> > >> Enterprise Linux Server release 7.2
> > >> Include installation and execute some simple queries.
> > >>
> > >> 在 27/02/2018 12:19, Yi JIN 写道:
> > >> > Hi All,
> > >> >
> > >> > This is the vote for Apache HAWQ (incubating) 2.3.0.0-incubating
> > Release
> > >> > Candidate 2 (RC2). It is a source release for HAWQ core, PXF, and
> > Ranger;
> > >> > and binary release for HAWQ core,  PXF and Ranger. We have rpm
> package
> > >> > involved for the binary release.
> > >> >
> > >> > The vote will run for at least 72 hours and will close on Saturday,
> > March
> > >> > 3rd, 2017. Thanks.
> > >> >
> > >> > 1. Wiki page of the release:
> > >> > *https://cwiki.apache.org/confluence/display/HAWQ/
> > Apache+HAWQ+2.3.0.0-
> > >> incubating+Release
> > >> > <https://cwiki.apache.org/confluence/display/HAWQ/
> > Apache+HAWQ+2.3.0.0-
> > >> incubating+Release>*
> > >> >
> > >> >
> > >> > 2. Release Notes (Apache Jira generated):
> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > >> > version=12340262=Html=12318826
> > >> >
> > >> >
> > >> > 3. Release verification steps can be found at:
> > >> > For source tarball: https://cwiki.apache.org/
> confluence/display/HAWQ/
> > >> > Release+Process%3A+Step+by+step+guide#ReleaseProcess:
> > >> > Stepbystepguide-ValidatetheReleaseCandidate
> > >> > For rpm package: https://cwiki.apache.org/confluence/display/HAWQ/
> > >> > Build+Package+and+Install+with+RPM
> > >> >
> > >> >
> > >> > 4. Git release branch:
> > >> > https://git-wip-us.apache.org/repos/asf?p=incubator-hawq.
> > >> > git;a=shortlog;h=refs/heads/2.3.0.0-incubating
> > >> >
> > >> > 5. Source and Binary release balls with signare:
> > >> > https://dist.apache.org/repos/dist/dev/incubator/hawq/2.3.0.
> > >> > 0-incubating.RC2/
> > >> >
> > >> >
> > >> > 6. Keys to verify the signature of the release artifact are
> available
> > at:
> > >> > https://dist.apache.org/repos/dist/dev/incubator/hawq/KEYS
> > >> >
> > >> >
> > >> > 7. The artifact(s) has been signed with Key ID: CE60F90D1333092A
> > >> >
> > >> > 8. Fixed issues in RC2.
> > >> > https://issues.apache.org/jira/browse/HAWQ-1589
> > >> > https://issues.apache.org/jira/browse/HAWQ-1590
> > >> >
> > >> > REMINDER: Please provide details of what you have tried and verified
> > >> before
> > >> > your vote conclusion. Thanks!
> > >> >
> > >> >
> > >> > Please vote accordingly:
> > >> > [ ] +1 approve
> > >> > [ ] +0 no opinion
> > >> > [ ] -1 disapprove (and reason why)
> > >> >
> > >> >
> > >> > Best regards,
> > >> > Yi (yjin)
> > >> >
> > >>
> > >> --
> > >> Regards,
> > >> Hongxu.
> > >>
> > >>
> >
>
>
>
> --
> Thanks
>
> Hubert Zhang
>

[VOTE]: Apache HAWQ 2.3.0.0-incubating Release (RC2)

[Yi JIN]

Hi All,

This is the vote for Apache HAWQ (incubating) 2.3.0.0-incubating Release
Candidate 2 (RC2). It is a source release for HAWQ core, PXF, and Ranger;
and binary release for HAWQ core,  PXF and Ranger. We have rpm package
involved for the binary release.

The vote will run for at least 72 hours and will close on Saturday, March
3rd, 2017. Thanks.

1. Wiki page of the release:
*https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.3.0.0-incubating+Release
*


2. Release Notes (Apache Jira generated):
https://issues.apache.org/jira/secure/ReleaseNote.jspa?
version=12340262=Html=12318826


3. Release verification steps can be found at:
For source tarball: https://cwiki.apache.org/confluence/display/HAWQ/
Release+Process%3A+Step+by+step+guide#ReleaseProcess:
Stepbystepguide-ValidatetheReleaseCandidate
For rpm package: https://cwiki.apache.org/confluence/display/HAWQ/
Build+Package+and+Install+with+RPM


4. Git release branch:
https://git-wip-us.apache.org/repos/asf?p=incubator-hawq.
git;a=shortlog;h=refs/heads/2.3.0.0-incubating

5. Source and Binary release balls with signare:
https://dist.apache.org/repos/dist/dev/incubator/hawq/2.3.0.
0-incubating.RC2/


6. Keys to verify the signature of the release artifact are available at:
https://dist.apache.org/repos/dist/dev/incubator/hawq/KEYS


7. The artifact(s) has been signed with Key ID: CE60F90D1333092A

8. Fixed issues in RC2.
https://issues.apache.org/jira/browse/HAWQ-1589
https://issues.apache.org/jira/browse/HAWQ-1590

REMINDER: Please provide details of what you have tried and verified before
your vote conclusion. Thanks!


Please vote accordingly:
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)


Best regards,
Yi (yjin)

Re: [VOTE]: Apache HAWQ 2.3.0.0-incubating Release (RC1)

[Yi JIN]

Hi all,

Thank you very much for providing these helpful comments. I'm preparing RC2
and will have it available for vote again soon. All issues mentioned will
be fixed.

BTW, please provide what you did to verify the candidate by listing bullets
before showing your vote conclusion.

Best,
Yi(yjin)

On Sat, Feb 24, 2018 at 2:16 PM, Ming Li <m...@pivotal.io> wrote:

> +1.
>
> On Sat, Feb 24, 2018 at 10:23 AM, Bai Jie <amybai6...@gmail.com> wrote:
>
> > Build from branch 2.3.0.0 source code, installed , init , run basic test
> > and simple queries. Looks good to me. +1
> >
> > On Sat, Feb 24, 2018 at 9:37 AM, Ivan Weng <iw...@pivotal.io> wrote:
> >
> > > Compile, install, init and run simple testing successfully. +1
> > >
> > > Regards,
> > > Ivan
> > >
> > > On Fri, Feb 23, 2018 at 10:35 AM, Hubert Zhang <hzh...@pivotal.io>
> > wrote:
> > >
> > > > +1
> > > >
> > > > On Wed, Feb 21, 2018 at 6:29 PM, Hongxu Ma <inte...@outlook.com>
> > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Both source and rpm package are verified in my environment: Red Hat
> > > > > Enterprise Linux Server release 7.2
> > > > > Include installation and execute some simple queries.
> > > > >
> > > > > 在 20/02/2018 10:42, Yi JIN 写道:
> > > > > > Hi All,
> > > > > >
> > > > > > This is the vote for Apache HAWQ (incubating) 2.3.0.0-incubating
> > > > Release
> > > > > > Candidate 1 (RC1). It is a source release for HAWQ core, PXF, and
> > > > Ranger;
> > > > > > and binary release for HAWQ core,  PXF and Ranger. We have rpm
> > > package
> > > > > > involved for the binary release.
> > > > > >
> > > > > > The vote will run for at least 72 hours and will close on
> Saturday,
> > > Feb
> > > > > 24,
> > > > > > 2017. Thanks.
> > > > > >
> > > > > > 1. Wiki page of the release:
> > > > > > *https://cwiki.apache.org/confluence/display/HAWQ/
> > > Apache+HAWQ+2.3.0.0-
> > > > > incubating+Release
> > > > > > <https://cwiki.apache.org/confluence/display/HAWQ/
> > > Apache+HAWQ+2.3.0.0-
> > > > > incubating+Release>*
> > > > > >
> > > > > >
> > > > > > 2. Release Notes (Apache Jira generated):
> > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > > > > version=12340262=Html=12318826
> > > > > >
> > > > > >
> > > > > > 3. Release verification steps can be found at:
> > > > > > For source tarball: https://cwiki.apache.org/
> > confluence/display/HAWQ
> > > > > > /Release+Process%3A+Step+by+step+guide#ReleaseProcess:
> > > Stepbystepguide-
> > > > > > ValidatetheReleaseCandidate
> > > > > > For rpm package: https://cwiki.apache.org/
> confluence/display/HAWQ
> > > > > > /Build+Package+and+Install+with+RPM
> > > > > >
> > > > > >
> > > > > > 4. Git release branch:
> > > > > > https://git-wip-us.apache.org/repos/asf?p=incubator-hawq.
> > > > > git;a=shortlog;h=refs/heads/2.3.0.0-incubating
> > > > > >
> > > > > > 5. Source and Binary release balls with signare:
> > > > > > https://dist.apache.org/repos/dist/dev/incubator/hawq/2.3.0.
> > > > > 0-incubating.RC1/
> > > > > >
> > > > > >
> > > > > > 6. Keys to verify the signature of the release artifact are
> > available
> > > > at:
> > > > > > https://dist.apache.org/repos/dist/dev/incubator/hawq/KEYS
> > > > > >
> > > > > >
> > > > > > 7. The artifact(s) has been signed with Key ID: CE60F90D1333092A
> > > > > >
> > > > > >
> > > > > > Please vote accordingly:
> > > > > > [ ] +1 approve
> > > > > > [ ] +0 no opinion
> > > > > > [ ] -1 disapprove (and reason why)
> > > > > >
> > > > > >
> > > > > > Best regards,
> > > > > > Yi (yjin)
> > > > > >
> > > > >
> > > > > --
> > > > > Regards,
> > > > > Hongxu.
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Thanks
> > > >
> > > > Hubert Zhang
> > > >
> > >
> >
>

[VOTE]: Apache HAWQ 2.3.0.0-incubating Release (RC1)

[Yi JIN]

Hi All,

This is the vote for Apache HAWQ (incubating) 2.3.0.0-incubating Release
Candidate 1 (RC1). It is a source release for HAWQ core, PXF, and Ranger;
and binary release for HAWQ core,  PXF and Ranger. We have rpm package
involved for the binary release.

The vote will run for at least 72 hours and will close on Saturday, Feb 24,
2017. Thanks.

1. Wiki page of the release:
*https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.3.0.0-incubating+Release
*


2. Release Notes (Apache Jira generated):
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12340262=Html=12318826


3. Release verification steps can be found at:
For source tarball: https://cwiki.apache.org/confluence/display/HAWQ
/Release+Process%3A+Step+by+step+guide#ReleaseProcess:Stepbystepguide-
ValidatetheReleaseCandidate
For rpm package: https://cwiki.apache.org/confluence/display/HAWQ
/Build+Package+and+Install+with+RPM


4. Git release branch:
https://git-wip-us.apache.org/repos/asf?p=incubator-hawq.git;a=shortlog;h=refs/heads/2.3.0.0-incubating

5. Source and Binary release balls with signare:
https://dist.apache.org/repos/dist/dev/incubator/hawq/2.3.0.0-incubating.RC1/


6. Keys to verify the signature of the release artifact are available at:
https://dist.apache.org/repos/dist/dev/incubator/hawq/KEYS


7. The artifact(s) has been signed with Key ID: CE60F90D1333092A


Please vote accordingly:
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)


Best regards,
Yi (yjin)

[jira] [Created] (HAWQ-1586) Update version from 2.2.0.0 to 2.3.0.0

[Yi Jin (JIRA)]

Yi Jin created HAWQ-1586:


 Summary: Update version from 2.2.0.0 to 2.3.0.0
 Key: HAWQ-1586
 URL: https://issues.apache.org/jira/browse/HAWQ-1586
 Project: Apache HAWQ
  Issue Type: Task
  Components: Build
Reporter: Yi Jin
Assignee: Radar Lei
 Fix For: 2.3.0.0-incubating


Update version number from 2.2.0.0 to 2.3.0.0 for release of 2.3.0.0 incubating



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Apache HAWQ 2.3.0.0-incubating release work starts

[Yi JIN]

Thank you Ed! Definitely need your help to make whole release process
smooth. Thanks again!

Yi

On Wed, Jan 31, 2018 at 8:32 PM, 刘奎恩(局外) <kuien@alibaba-inc.com> wrote:

> Yi, thanks for your efforts. I guess the demo will NOT cover the whole
> feature of ORC format. If ORC will not be covered in next release, do you
> have any plan on it in near future? looking forward to performance of this
> framework which supports zero-copy.
> -——
> Kuien Liu/奎恩
> --------------发件人：Yi
> JIN <jinyi.apa...@gmail.com>发送时间：2018年1月30日(星期二) 20:20收件人：dev <
> dev@hawq.incubator.apache.org>主 题：Re: Apache HAWQ 2.3.0.0-incubating
> release work starts
> Yes, I think in this release the pluggable storage should
> be ready for use,
> but will be lack of demo to be added in next release. Also we will have
> nice features involved like TDE, ranger, etc.
>
> On Tue, Jan 30, 2018 at 10:46 PM, 刘奎恩(局外) <kuien@alibaba-inc.com
> > wrote:
>
> > great! Does it mean the pluggable storage framework is ready?
> >
> >
> >
> > 来自钉钉专属商务邮箱
> --发件人：Yi
> > JIN<y...@apache.org>日 期：2018年01月30日 18:53:55收件人：<dev@hawq.
> > incubator.apache.org>主 题：Apache HAWQ 2.3.0.0-incubating release work
> > startsHi all,
> >
> > As all planed features and bug fixes are done and
> delivered, I will start
> > release process tomorrow. Thanks for your support.
> >
> > Best,
> > Yi
> >
> >
>

Re: Apache HAWQ 2.3.0.0-incubating release work starts

[Yi JIN]

Yes, I think in this release the pluggable storage should be ready for use,
but will be lack of demo to be added in next release. Also we will have
nice features involved like TDE, ranger, etc.

On Tue, Jan 30, 2018 at 10:46 PM, 刘奎恩(局外) <kuien@alibaba-inc.com> wrote:

> great! Does it mean the pluggable storage framework is ready?
>
>
>
> 来自钉钉专属商务邮箱----------发件人：Yi
> JIN<y...@apache.org>日 期：2018年01月30日 18:53:55收件人：<dev@hawq.
> incubator.apache.org>主 题：Apache HAWQ 2.3.0.0-incubating release work
> startsHi all,
>
> As all planed features and bug fixes are done and delivered, I will start
> release process tomorrow. Thanks for your support.
>
> Best,
> Yi
>
>

Apache HAWQ 2.3.0.0-incubating release work starts

[Yi JIN]

Hi all,

As all planed features and bug fixes are done and delivered, I will start
release process tomorrow. Thanks for your support.

Best,
Yi

Re: Apache HAWQ (incubating) 2.3.0.0 suggestion

[Yi JIN]

Hi Radar,

According to my input, the pluggable storage framework is one desired
feature from marketing, as it is key to reenforce HAWQ performance
advantages and its code is nowunder refactor for delivery as I have seen
some activities relating to this feature, I think we just need the last
mile to have it contributed in Apache HAWQ. Therefore, I prefer to give
more patience to have one stronger release besides those amazing features
we have done, like TDE, ranger etc. Thanks.

Best,
Yi

On Thu, Nov 30, 2017 at 8:45 PM, Radar Lei  wrote:

> Hi All,
>
> As Ed mentioned, we already have sufficient content(TDE, Ranger
> HA/Kerberos) to do a release, and seems Pluggable storage feature need more
> time to finish review and tests.
>
> So  how about we do this release without pluggable storage? We can make
> another release soon if pluggable storage is ready for production, any
> comments? Thanks.
>
>
> Regards,
> Radar
>
> On Wed, Nov 15, 2017 at 8:44 AM, Lei Chang  wrote:
>
> > Cool. Looks your guys' work is quite related to the pluggable storage
> > feature (HAWQ-786).
> >
> > The feature implements a native C interface for external formats and it
> is
> > several times faster than the current JAVA interface. And the feature has
> > been validated in Oushu version for some time.
> >
> > If you guys are interested in the alignment of the feature, we can
> discuss
> > it on the JIRA.
> >
> > Cheers
> > Lei
> >
> >
> >
> >
> > On Tue, Nov 14, 2017 at 2:02 PM, 刘奎恩(局外) 
> > wrote:
> >
> > >
> > > Missed out DEV.
> > >
> > > --
> > > 发件人：刘奎恩(局外) 
> > > 发送时间：2017年11月14日(星期二) 11:15
> > > 收件人：Lei Chang 
> > > 主 题：回复：Apache HAWQ (incubating) 2.3.0.0 suggestion
> > >
> > > Hi Dr. Chang,
> > >
> > > Good, thanks for useful suggestion. Now there is:
> > > https://issues.apache.org/jira/browse/HAWQ-1550
> > >
> > > We are trying to integrate Hawq engine onto MaxCompute (former name
> > ODPS, within
> > > Aliyun Cloud), Seahawks, it is led by Mr. Chen Xia. We develop a
> > component
> > > AXF (borrow idea from PXF) to connect data sources from Druid and
> > > MaxCompute, it works well for this 11.11 battle, but it is not
> > stand-alone
> > > program.
> > >
> > > -——
> > > Kuien Liu/奎恩
> > >
> > > --
> > > 发件人：Lei Chang 
> > > 发送时间：2017年11月14日(星期二) 10:28
> > > 收件人：dev ; 刘奎恩(局外) <
> > > kuien@alibaba-inc.com>
> > > 主 题：Re: Apache HAWQ (incubating) 2.3.0.0 suggestion
> > >
> > >
> > > Kuien, we are welcoming any good contributions from the community.
> > >
> > > Looks hawq_log_master_concise is a good enhancement, I'd like suggest
> you
> > > create a JIRA and we can discuss it on the JIRA.
> > >
> > > Can you explain more about Druid Wrapper & MaxCompute Wrapper? What are
> > > the use cases here?
> > >
> > > Cheers
> > > Lei
> > >
> > >
> > >
> > >
> > >
> > > On Tue, Nov 14, 2017 at 10:05 AM, 刘奎恩(局外) 
> > > wrote:
> > > Hi Ed and Hawq,
> > > I have add two GUCs (log_max_size, log_max_age) to control the logfile
> > > size on master, otherwise the query to hawq_toolkit.hawq_log_master_
> > concise
> > > will be slower and slower, and disk usage on master is hard to
> constrain.
> > > If some gys are interested to this, I can submit it to Hawq.
> > > Besides, our team has introduced many interesting features on hawq, but
> > > most of them are deep coupling with Alibaba Cloud. Some of them may be
> > > (maybe not) seperated alone, such as Druid Wrapper, MaxCompute
> Wrapper. I
> > > will discuss with teammates Mr. Chen Xia, Mr. Zhiyong Dai et al.
> > >
> > > 祝好！刘奎恩/局外
> > > 
> --发件人：Ed
> > > Espino 发送时间：2017年11月13日(星期一) 23:25收件人：dev <
> > > dev@hawq.incubator.apache.org>主 题：Apache HAWQ (incubating) 2.3.0.0
> > > suggestion
> > > HAWQ,
> > >
> > > I feel we have sufficient content to warrant a 2.3.0.0
> > > release. It has been
> > > 4+ months since our last release (2.2.0.0). I suggest we include the
> > > pluggable storage feature (HAWQ-786 - assigned to Chiyang Wan -
> > > chiyang10...@gmail.com) plus others (as appropriate) in
> > > the 2.3.0.0 queue in
> > > the next release.
> > >
> > > Are there any outstanding issues that are mandatory for
> > > the 2.3.0.0 release?
> > > What do others think?
> > >
> > > -=e
> > >
> > > --
> > > *Ed Espino*
> > >
> > >
> > >
> > >
> >
>

Re: New committer: Hongxu Ma

[Yi JIN]

Congratulations!

On Thu, Nov 2, 2017 at 6:10 PM, Hongxu Ma  wrote:

> Thanks all nice buddy!
> It's a great honor for me.
>
> 在 02/11/2017 00:05, Lei Chang 写道:
> > Congrats!
> >
> > Cheers
> > Lei
> >
> >
> > On Wed, Nov 1, 2017 at 11:33 PM, Brian Lu  wrote:
> >
> >> Congrats Hongxu!
> >>
> >> Best Regards,
> >> Brian
> >>
> >> On Wed, Nov 1, 2017 at 10:44 PM, Shubham Sharma 
> >> wrote:
> >>
> >>> Congrats Hongxu !
> >>>
> >>> On Wed, Nov 1, 2017 at 3:15 AM, stanly sheng 
> >>> wrote:
> >>>
>  Congrats, Hongxu!
> 
>  2017-11-01 14:16 GMT+08:00 Hubert Zhang :
> 
> > Congrats to Hongxu!
> >
> > On Wed, Nov 1, 2017 at 2:06 PM, Radar Lei  wrote:
> >
> >> The Project Management Committee (PMC) for Apache HAWQ (incubating)
> >>> has
> >> invited Hongxu Ma to become a committer and we are pleased to
> >>> announce
> > that
> >> he has accepted.
> >> Being a committer enables easier contribution to the project since
>  there
> > is
> >> no need to go via the patch submission process. This should enable
>  better
> >> productivity.
> >> Please join us in congratulating him and we are looking forward to
> >> collaborating with him in the open source community.
> >>
> >> His contribution includes (but not limited to):
> >>
> >> *Direct contribution to code base:*
> >>
> >> - 26 commits in total with some major components in hawq
> >> involved,
> >> including contributions to Apache Ranger integration, TDE
> >> support
> >> and command line tools.
> >> *https://github.com/apache/incubator-hawq/commits?author=interma
> >>  >> interma
>  *
> >> - 27 closed PRs:
> >> *https://github.com/apache/incubator-hawq/pulls?utf8=%E2%
> >> 9C%93=is%3Apr%20is%3Aclosed%20author%3Ainterma
> >>  >> 9C%93=is%3Apr%20is%3Aclosed%20author%3Ainterma>*
> >> - 11 Jiras on Apache Ranger integration
> >>- https://issues.apache.org/jira/browse/HAWQ-1493 Integrate
>  Ranger
> >>lookup JAAS configuration in ranger-admin plugin jar
> >>- https://issues.apache.org/jira/browse/HAWQ-1485 Use
> > user/password
> >>instead of credentials cache in Ranger lookup for HAWQ with
> > Kerberos
> >>enabled.
> >>- https://issues.apache.org/jira/browse/HAWQ-1477
> >>> Ranger-plugin
> >>connect to Ranger admin under kerberos security.
> >>- https://issues.apache.org/jira/browse/HAWQ-1456 Copy RPS
> >>configuration files to standby in specific scenarios
> >>- https://issues.apache.org/jira/browse/HAWQ-1436 Implement
> >>> RPS
> > High
> >>availability on HAWQ
> >>- https://issues.apache.org/jira/browse/HAWQ-1410 Add basic
> >>> test
> >> case
> >>for hcatalog with ranger
> >>- https://issues.apache.org/jira/browse/HAWQ-1405 'hawq stop
> >>--reload' should not stop RPS
> >>- https://issues.apache.org/jira/browse/HAWQ-1393 'hawq stop
> >> cluster'
> >>failed when rps.sh have some path errors (e.g. CATALINA_HOME)
> >>- https://issues.apache.org/jira/browse/HAWQ-1365 Print out
> > detailed
> >>schema information for tables which the user doesn't have
> > privileges
> >>- https://issues.apache.org/jira/browse/HAWQ-1279 Force to
> > recompute
> >>namespace_path when enable_ranger
> >>- https://issues.apache.org/jira/browse/HAWQ-1257 If user
>  doesn't
> >>have privileges on certain objects, need return user which
>  specific
> >> table
> >>he doesn't have right.
> >> - 4 Jiras on hawq TDE support
> >>- https://issues.apache.org/jira/browse/HAWQ-1520
> >> gpcheckhdfs
> > should
> >>skip hdfs trash directory
> >>- https://issues.apache.org/jira/browse/HAWQ-1510 Add
>  TDE-related
> >>functionality into hawq command line tools
> >>- https://issues.apache.org/jira/browse/HAWQ-1506 Support
> >>multi-append a file within encryption zone
> >>- https://issues.apache.org/jira/browse/HAWQ-1193 TDE
> >> support
> >>> in
> >> HAWQ
> >> - 5 Jiras on improvements including documentation, test, build,
> > command
> >> line tools, code refactor.
> >>- https://issues.apache.org/jira/browse/HAWQ-1385 hawq_ctl
> >>> stop
> >>failed when master is down
> >>- https://issues.apache.org/jira/browse/HAWQ-870 Allocate
>  target's
> >>tuple table slot in PortalHeapMemory during split 

Re: New committer: Chunling Wang

[Yi JIN]

Congratulations!

On Thu, Nov 2, 2017 at 3:05 AM, Lei Chang  wrote:

> Congrats!
>
> Cheers
> Lei
>
>
> On Wed, Nov 1, 2017 at 11:42 PM, Ed Espino  wrote:
>
> > Congratulations Chunling.
> >
> > Regards,
> > -=e
> >
> > On Wed, Nov 1, 2017 at 8:33 AM, Brian Lu  wrote:
> >
> > > Congrats Chunling!
> > >
> > > Best Regards,
> > > Brian
> > >
> > > On Wed, Nov 1, 2017 at 10:45 PM, Shubham Sharma 
> > > wrote:
> > >
> > > > Congratulations Chunling !
> > > >
> > > > On Wed, Nov 1, 2017 at 3:15 AM, stanly sheng <
> stanly.sxi...@gmail.com>
> > > > wrote:
> > > >
> > > > > Congrats, Chunling !
> > > > >
> > > > > 2017-11-01 14:15 GMT+08:00 Hubert Zhang :
> > > > >
> > > > > > Congrats to Chunling
> > > > > >
> > > > > > On Wed, Nov 1, 2017 at 2:10 PM, yanqing weng 
> > > wrote:
> > > > > >
> > > > > > > The Project Management Committee (PMC) for Apache HAWQ
> > (incubating)
> > > > has
> > > > > > > invited Chunling Wang to become a committer and we are pleased
> to
> > > > > > announce
> > > > > > > that
> > > > > > > she has accepted.
> > > > > > > Being a committer enables easier contribution to the project
> > since
> > > > > there
> > > > > > is
> > > > > > > no need to go via the patch submission process. This should
> > enable
> > > > > better
> > > > > > > productivity.
> > > > > > > Please join us in congratulating her and we are looking forward
> > to
> > > > > > > collaborating with her in the open source community.
> > > > > > >
> > > > > > > Her contribution includes (but not limited to):
> > > > > > >
> > > > > > > *Direct contribution to code base:*
> > > > > > >
> > > > > > >- *37 commits* in total with some major components in hawq
> > > > involved,
> > > > > > >including contributions to Apache Ranger integration, hawq
> > > > register
> > > > > > >and command line tools. https://github.com/apac
> > > > > > >he/incubator-hawq/commits?author=wcl14
> > > > > > > > wcl14>
> > > > > > >- *29 closed PRs*: https://github.com/apache
> > > > > > >/incubator-hawq/pulls?utf8=%E2%9C%93=is%3Apr%20is%
> 3Aclosed
> > > > > > >%20author%3Awcl14
> > > > > > > > > > > > > 9C%93=is%3Apr%20is%3Aclosed%20author%3Awcl14>
> > > > > > >- *2 improvements* including documentation, test, build,
> > command
> > > > > line
> > > > > > >tools, code refactor.
> > > > > > >   - HAWQ-1358  > jira/browse/HAWQ-1358
> > > >.
> > > > > > >   Refactor gpfdist library in featuretest.
> > > > > > >   - HAWQ-1377  > jira/browse/HAWQ-1377
> > > >.
> > > > > Add
> > > > > > >   more information for Ranger related GUCs in default
> > > > > hawq-site.xml.
> > > > > > >- *13 bug fixes* including test failure, ranger, register,
> > > > > core-dump,
> > > > > > >command line tools, build components.
> > > > > > >   - HAWQ-1020  > jira/browse/HAWQ-1020
> > > >.
> > > > > Fix
> > > > > > >   bugs to let feature tests TestCommonLib.TestHdfsConfig
> and
> > > > > > >   TestCommonLib.TestYanConfig run in concourse
> > > > > > >   - HAWQ-1037  > jira/browse/HAWQ-1037
> > > >.
> > > > > > > Modify
> > > > > > >   way to get HDFS port in TestHawqRegister
> > > > > > >   - HAWQ-1037  > jira/browse/HAWQ-1037
> > > >.
> > > > > Get
> > > > > > >   HDFS namenode host from HAWQ catalog tables instead of
> from
> > > > HDFS
> > > > > > >   configuration file
> > > > > > >   - HAWQ-1037  > jira/browse/HAWQ-1037
> > > >.
> > > > > Add
> > > > > > >   get OS user in hdfs_config when is not set
> > > > > > >   - HAWQ-1113  > jira/browse/HAWQ-1113
> > > >
> > > > > > >   . Fix bug when files in yaml
> is
> > > > > > > disordered, hawq
> > > > > > >   register error in force mode.
> > > > > > >   - HAWQ-1145  > jira/browse/HAWQ-1145
> > > >.
> > > > > Add
> > > > > > >   UDF gp_relfile_insert_for_register and add insert
> metadata
> > > into
> > > > > > >   gp_relfile_node and gp_persistent_relfile_node for HAWQ
> > > > register
> > > > > > >   - HAWQ-1237 <>https://issues.apache.org/
> > > jira/browse/HAWQ-1237.
> > > > > > > Modify
> > > > > > >   hard code 'select' privilege in
> create_ranger_request_json_
> > > > > batch()
> > > > > > > in
> > > > > > >   rangerrest.c
> > > > > > >   - HAWQ-1239  > jira/browse/HAWQ-1239
> > > >.
> > > > > > Fail
> > > > > > >   to call pg_rangercheck_batch() when when 'rte->rtekind !=
> > > > > > > RTE_RELATION' or
> > > > > > >   'requiredPerms == 0'
> > 

Re: New Committer: Amy Bai

[Yi JIN]

Congratulations!

On Thu, Nov 2, 2017 at 3:06 AM, Lei Chang  wrote:

> Congrats!
>
> Cheers
> Lei
>
> On Wed, Nov 1, 2017 at 2:02 PM, Wen Lin  wrote:
>
> > Hi,
> >
> > The Project Management Committee (PMC) for Apache HAWQ (incubating) has
> > invited Amy Bai to become a committer and we are pleased to announce that
> > she has accepted.
> > Being a committer enables easier contribution to the project since there
> is
> > no need to go via the patch submission process. This should enable better
> > productivity. Please join us in congratulating her and we are looking
> > forward to collaborating with her in the open source community. Her
> > contribution includes (but not limited to):
> > List contributions to code base, documentation, code review, discussion
> in
> > mailing list, JIRA, etc.
> >
> > Regards!
> >
>

[NEED HELP] Apache HAWQ Incubator PXF license issue to fix

[Yi JIN]

Hi Guys,

Apache HAWQ Incubator is recently actively moving, there are several big
things on going as i believe we will see the progress soon. While as one
planed item. There is a license issue in PXF component. I paste the link
and content as below for reference. Can anyone volunteer to help fix this
so that we can have it done in version 2.3.0.0? Many thanks!

Best,
Yi (yjin)

https://issues.apache.org/jira/browse/HAWQ-1496?filter=12341362

Deliver LICENSE, NOTICE and DISCLAIMER in main PXF RPM in which all others
are dependent. Currently, they are delivered in all supplied PXF jar files.
This was identified during IPMC voting process for the 2.2.0.0 convenience
binary release artifacts (
https://lists.apache.org/thread.html/35f0f37fb96ad88b1b7cee286640a94894a5b6956b9ae41a7d554ed6@%3Cgeneral.incubator.apache.org%3E
)

Status Update : Apache HAWQ 2.3.0.0-incubating Release

[Yi JIN]

Hi guys,

Apache HAWQ 2.3.0.0 incubating release management page is basically
available for checking now. I am going to add more content to make it a
nice view to learn the whole release easily.

As to this release, We will have amazing new features and key bug fixes
involved! Of course, we are always working on its way to graduation
considering its building facility, license issues, etc. Please check the
link below for details.

https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.3.0.0-incubating+Release

Always please feel free to touch me through y...@apache.org if I missed
anything or made anything wrongly reported. This is an amazing team work
and thank you guys! Have a great weekend.

Best,
Yi Jin (yjin)

[jira] [Created] (HAWQ-1514) TDE feature makes libhdfs3 require openssl1.1

[Yi Jin (JIRA)]

Yi Jin created HAWQ-1514:


 Summary: TDE feature makes libhdfs3 require openssl1.1
 Key: HAWQ-1514
 URL: https://issues.apache.org/jira/browse/HAWQ-1514
 Project: Apache HAWQ
  Issue Type: Task
  Components: libhdfs
Reporter: Yi Jin
Assignee: Radar Lei


New TDE feature delivered in libhdfs3 requires specific version of openssl, at 
least per my test, 1.0.21 does not work, and 1.1 source code built library 
passed.

So maybe we need some build and installation instruction improvement. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HAWQ-1512) Check Apache HAWQ mandatory libraries to match LC20, LC30 license criteria

[Yi Jin (JIRA)]

Yi Jin created HAWQ-1512:


 Summary: Check Apache HAWQ mandatory libraries to match LC20, LC30 
license criteria
 Key: HAWQ-1512
 URL: https://issues.apache.org/jira/browse/HAWQ-1512
 Project: Apache HAWQ
  Issue Type: Task
  Components: Build
Reporter: Yi Jin
Assignee: Radar Lei
 Fix For: 2.3.0.0-incubating


Check Apache HAWQ mandatory libraries to match LC20, LC30 license criteria

Check the following page for the criteria

https://cwiki.apache.org/confluence/display/HAWQ/ASF+Maturity+Evaluation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Apache HAWQ 2.3.0.0 release plan

[Yi JIN]

Hi Shivram,

This framework is not ORC oriented, it does not care any specific file
system or format in fact per my understanding.

And yes, I have required that its design proposal should be available in
community for comment before delivery.

Best,
Yi

On Thu, Aug 3, 2017 at 3:53 AM, Shivram Mani <shivram.m...@gmail.com> wrote:

> Yi, for the pluggable storage framework to be merged to master we need to
> have a more detailed proposal talking about design/implementation etc. It
> has been more than a year since the first ORC related draft was first
> published for the pluggable framework. August timeline seems very unlikely.
>
> On Tue, Jul 25, 2017 at 5:45 AM, Yi JIN <y...@apache.org> wrote:
>
> > Thank Lili and Radar for your comment.
> >
> > Following Vineet's comment, I have changed timeline to have branch cut
> > scheduled at 1st Sept.
> >
> > And TDE will be involved in this release per your requests. Please Radar
> > help plan TDE delivery in Aug.
> >
> > Currently, pluggable external storage supporting framework per my
> > understanding is under construction at a wrap-up phase, it will be
> smoothly
> > delivered into repo in Aug as well when it has enough test cases
> verified.
> >
> > Best,
> > Yi
> >
> > On Tue, Jul 25, 2017 at 6:39 PM, Radar Lei <r...@pivotal.io> wrote:
> >
> > > Thanks Yi.
> > >
> > > Cut release branch at early/middle September looks fine.
> > >
> > > I would suggest we can add TDE support in the release plan too.
> > >
> > > Regards,
> > > Radar
> > >
> > > On Tue, Jul 25, 2017 at 2:37 PM, Lili Ma <lil...@apache.org> wrote:
> > >
> > > > Hi Yi,
> > > >
> > > > What's the status of pluggable external storage?
> > > >
> > > > Also, I wonder whether we can add TDE in next release.
> > > >
> > > > Thanks
> > > > Lili
> > > >
> > > > 2017-07-25 13:25 GMT+08:00 Vineet Goel <vvin...@apache.org>:
> > > >
> > > > > Sounds good to me, Yi.
> > > > > If there are any other suggestions from other members, those are
> > > welcome
> > > > > too.
> > > > >
> > > > > Thank you.
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Jul 20, 2017 at 6:14 PM Yi JIN <jinyi.apa...@gmail.com>
> > wrote:
> > > > >
> > > > > > Hi Vineet,
> > > > > >
> > > > > > Thank you for your comment, I will add timeline on release page
> as
> > > part
> > > > > of
> > > > > > the plan.
> > > > > >
> > > > > > Basically, I expect by 30th Oct., we can have tar balls available
> > for
> > > > > > downloading, and before that I reserve about 1.5 month for voting
> > > > > progress
> > > > > > according to previous experiences. So basically, I think we can
> cut
> > > > > branch
> > > > > > for new release at mid Sept. So that means we have no more than 2
> > > > months
> > > > > > for fixing and new feature delivery. As delivering pluggable
> > external
> > > > > > storage requires some time. If necessary, I think it is ok to
> move
> > > > target
> > > > > > date 2 weeks earlier. Does this rough time schedule make sense?
> > > What's
> > > > > your
> > > > > > opinion?
> > > > > >
> > > > > > I will add Umbrella JIRAS on that page as well.
> > > > > >
> > > > > > Best,
> > > > > > Yi Jin (yjin)
> > > > > >
> > > > > > On Thu, Jul 20, 2017 at 3:52 PM, Vineet Goel <vvin...@apache.org
> >
> > > > wrote:
> > > > > >
> > > > > > > Hi Yi,
> > > > > > >
> > > > > > > Thank you for the initiative. I have some feedback that we can
> > > > discuss
> > > > > > here
> > > > > > > and collect further input on.
> > > > > > >
> > > > > > > a) Putting a timeline on releases (such as October 31st here)
> may
> > > > have
> > > > > > some
> > > > > > > downsides to the process. My thoughts are that more frequent
> > > releases
> > > > > > 

Re: Apache HAWQ 2.3.0.0 release plan

[Yi JIN]

Thank Lili and Radar for your comment.

Following Vineet's comment, I have changed timeline to have branch cut
scheduled at 1st Sept.

And TDE will be involved in this release per your requests. Please Radar
help plan TDE delivery in Aug.

Currently, pluggable external storage supporting framework per my
understanding is under construction at a wrap-up phase, it will be smoothly
delivered into repo in Aug as well when it has enough test cases verified.

Best,
Yi

On Tue, Jul 25, 2017 at 6:39 PM, Radar Lei <r...@pivotal.io> wrote:

> Thanks Yi.
>
> Cut release branch at early/middle September looks fine.
>
> I would suggest we can add TDE support in the release plan too.
>
> Regards,
> Radar
>
> On Tue, Jul 25, 2017 at 2:37 PM, Lili Ma <lil...@apache.org> wrote:
>
> > Hi Yi,
> >
> > What's the status of pluggable external storage?
> >
> > Also, I wonder whether we can add TDE in next release.
> >
> > Thanks
> > Lili
> >
> > 2017-07-25 13:25 GMT+08:00 Vineet Goel <vvin...@apache.org>:
> >
> > > Sounds good to me, Yi.
> > > If there are any other suggestions from other members, those are
> welcome
> > > too.
> > >
> > > Thank you.
> > >
> > >
> > >
> > > On Thu, Jul 20, 2017 at 6:14 PM Yi JIN <jinyi.apa...@gmail.com> wrote:
> > >
> > > > Hi Vineet,
> > > >
> > > > Thank you for your comment, I will add timeline on release page as
> part
> > > of
> > > > the plan.
> > > >
> > > > Basically, I expect by 30th Oct., we can have tar balls available for
> > > > downloading, and before that I reserve about 1.5 month for voting
> > > progress
> > > > according to previous experiences. So basically, I think we can cut
> > > branch
> > > > for new release at mid Sept. So that means we have no more than 2
> > months
> > > > for fixing and new feature delivery. As delivering pluggable external
> > > > storage requires some time. If necessary, I think it is ok to move
> > target
> > > > date 2 weeks earlier. Does this rough time schedule make sense?
> What's
> > > your
> > > > opinion?
> > > >
> > > > I will add Umbrella JIRAS on that page as well.
> > > >
> > > > Best,
> > > > Yi Jin (yjin)
> > > >
> > > > On Thu, Jul 20, 2017 at 3:52 PM, Vineet Goel <vvin...@apache.org>
> > wrote:
> > > >
> > > > > Hi Yi,
> > > > >
> > > > > Thank you for the initiative. I have some feedback that we can
> > discuss
> > > > here
> > > > > and collect further input on.
> > > > >
> > > > > a) Putting a timeline on releases (such as October 31st here) may
> > have
> > > > some
> > > > > downsides to the process. My thoughts are that more frequent
> releases
> > > > give
> > > > > users access to features and fixes faster, from a stable branch.
> > > Release
> > > > > momentum is good to have in any project. For instance, master
> branch
> > is
> > > > > already 75+ commits ahead of 2.2.0.0-incubating, since that branch
> > was
> > > > cut
> > > > > a a couple of months back.
> > > > >
> > > > > b) Also, it may help to add Umbrella JIRAs on that page, associated
> > to
> > > > the
> > > > > new features. If certain features are too large to finish in a
> given
> > > > > release, they can always ride the next release vehicle.
> > > > >
> > > > > It would be good to hear thoughts from others.
> > > > >
> > > > > Thanks!
> > > > > Vineet
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Jul 19, 2017 at 6:15 PM Yi JIN <y...@apache.org> wrote:
> > > > >
> > > > > > Hi Apache HAWQ community,
> > > > > >
> > > > > > Thank you guys for your trust, I drafted one release plan for
> next
> > > > > version
> > > > > > 2.3.0.0 that has been update in wiki as below. Please feel free
> to
> > > > > comment
> > > > > > and any input is welcome. Thank you again.
> > > > > >
> > > > > >
> > > > > > https://cwiki.apache.org/confluence/display/HAWQ/
> > > Apache+HAWQ+2.3.0.0-
> > > > > incubating+Release
> > > > > >
> > > > > >
> > > > > > Best,
> > > > > > Yi Jin (yjin)
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Apache HAWQ 2.3.0.0 release plan

[Yi JIN]

Hi Vineet,

Thank you for your comment, I will add timeline on release page as part of
the plan.

Basically, I expect by 30th Oct., we can have tar balls available for
downloading, and before that I reserve about 1.5 month for voting progress
according to previous experiences. So basically, I think we can cut branch
for new release at mid Sept. So that means we have no more than 2 months
for fixing and new feature delivery. As delivering pluggable external
storage requires some time. If necessary, I think it is ok to move target
date 2 weeks earlier. Does this rough time schedule make sense? What's your
opinion?

I will add Umbrella JIRAS on that page as well.

Best,
Yi Jin (yjin)

On Thu, Jul 20, 2017 at 3:52 PM, Vineet Goel <vvin...@apache.org> wrote:

> Hi Yi,
>
> Thank you for the initiative. I have some feedback that we can discuss here
> and collect further input on.
>
> a) Putting a timeline on releases (such as October 31st here) may have some
> downsides to the process. My thoughts are that more frequent releases give
> users access to features and fixes faster, from a stable branch. Release
> momentum is good to have in any project. For instance, master branch is
> already 75+ commits ahead of 2.2.0.0-incubating, since that branch was cut
> a a couple of months back.
>
> b) Also, it may help to add Umbrella JIRAs on that page, associated to the
> new features. If certain features are too large to finish in a given
> release, they can always ride the next release vehicle.
>
> It would be good to hear thoughts from others.
>
> Thanks!
> Vineet
>
>
>
>
> On Wed, Jul 19, 2017 at 6:15 PM Yi JIN <y...@apache.org> wrote:
>
> > Hi Apache HAWQ community,
> >
> > Thank you guys for your trust, I drafted one release plan for next
> version
> > 2.3.0.0 that has been update in wiki as below. Please feel free to
> comment
> > and any input is welcome. Thank you again.
> >
> >
> > https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.3.0.0-
> incubating+Release
> >
> >
> > Best,
> > Yi Jin (yjin)
> >
>

Apache HAWQ 2.3.0.0 release plan

[Yi JIN]

Hi Apache HAWQ community,

Thank you guys for your trust, I drafted one release plan for next version
2.3.0.0 that has been update in wiki as below. Please feel free to comment
and any input is welcome. Thank you again.

https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.3.0.0-incubating+Release


Best,
Yi Jin (yjin)

Re: Re: [ANNOUNCE] Apache HAWQ 2.2.0.0-incubating Released

[Yi JIN]

Hi Ruilong,

I would like to take this responsibility as a volunteer for the next
release. As a committer I used to contribute a lot of code to Apache HAWQ,
consequently besides code work, if possible I would like to contribute more
in another way and learn more about growing an Apache project.

Best,
Yi (yjin)

On Thu, Jul 13, 2017 at 4:43 PM, HuoRuilong  wrote:

> Great step towards a mature hawq and active community! Thanks everyone for
> making this real, especially the help from Ed!
>
> To make it a more successful apache project and community, we need to keep
> the release cadence. Who would like to be volunteer for the next release
> manager and drive the effort? Thanks.
>
> Best regards,
> Ruilong Huo
>
>
> At 2017-07-13 14:39:21, "Lili Ma"  wrote:
> >Congratulations everyone :)
> >
> >We're stepping further towards graduation!
> >
> >Best Regards,
> >Lili
> >
> >2017-07-13 13:16 GMT+08:00 Ed Espino :
> >
> >> Congratulations to everyone on the first Apache HAWQ release with
> >> convenience binaries. Special thanks to Ruilong for his excellent
> release
> >> management guidance.
> >>
> >> I'm very proud to be part of a great dev team.
> >>
> >> Cheers,
> >> -=e
> >>
> >> On Wed, Jul 12, 2017 at 10:00 PM, 陶征霖  wrote:
> >>
> >> > Congrats!
> >> >
> >> > 2017-07-13 9:55 GMT+08:00 Yandong Yao :
> >> >
> >> > > Great achievement, Congrats!
> >> > >
> >> > > On Thu, Jul 13, 2017 at 8:46 AM, Lei Chang 
> >> > wrote:
> >> > >
> >> > > > Congrats!
> >> > > >
> >> > > > Cheers
> >> > > > Lei
> >> > > >
> >> > > >
> >> > > > On Wed, Jul 12, 2017 at 3:27 PM, Ruilong Huo 
> >> wrote:
> >> > > >
> >> > > > > Hi All,
> >> > > > >
> >> > > > > The Apache HAWQ (incubating) Project Team is proud to announce
> >> > > > > the release of Apache HAWQ 2.2.0.0-incubating.
> >> > > > >
> >> > > > > This is a source code and binary release.
> >> > > > >
> >> > > > > ABOUT HAWQ
> >> > > > > Apache HAWQ (incubating) combines exceptional MPP-based
> analytics
> >> > > > > performance, robust ANSI SQL compliance, Hadoop ecosystem
> >> integration
> >> > > > > and manageability, and flexible data-store format support, all
> >> > > > > natively in Hadoop, no connectors required.
> >> > > > >
> >> > > > > Built from a decade’s worth of massively parallel processing
> (MPP)
> >> > > > > expertise developed through the creation of open source
> Greenplum®
> >> > > > > Database and PostgreSQL, HAWQ enables you to
> >> > > > > swiftly and interactively query Hadoop data, natively via HDFS.
> >> > > > >
> >> > > > > FEATURES AND ENHANCEMENTS INCLUDED IN THIS RELEASE
> >> > > > > - CentOS 7.x support
> >> > > > > Apache HAWQ is improved to be compatible with CentOS 7.x along
> with
> >> > > 6.x.
> >> > > > >
> >> > > > > - Apache Ranger integration
> >> > > > > Integrate Apache HAWQ with Apache Ranger through HAWQ Ranger
> Plugin
> >> > > > Service
> >> > > > > which is a RESTful service. It enables users to use Apache
> Ranger
> >> to
> >> > > > > authorize
> >> > > > > user access to Apache HAWQ resources. It also manages all Hadoop
> >> > > > > components’
> >> > > > > authorization policies with the same user interface, policy
> store,
> >> > and
> >> > > > > auditing
> >> > > > > stores.
> >> > > > >
> >> > > > > - PXF ORC profile
> >> > > > > Fully supports PXF with Optimized Row Columnar (ORC) file
> format.
> >> > > > >
> >> > > > > - Fixes and enhancements on Apache HAWQ resource manager, query
> >> > > > execution,
> >> > > > > dispatcher,
> >> > > > > catalog, management utilities and more.
> >> > > > >
> >> > > > > JIRA GENERATED RELEASE NOTES
> >> > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> >> > > > > projectId=12318826=12339641
> >> > > > >
> >> > > > > RELEASE ARTIFACTS ARE AVAILABLE AT
> >> > > > > http://apache.org/dyn/closer.cgi/incubator/hawq/2.2.0.0-
> incubating
> >> > > > >
> >> > > > > SHA256 & MD5 SIGNATURES (verify your downloads <
> >> > > > > https://www.apache.org/dyn/closer.cgi#verify>):
> >> > > > > https://dist.apache.org/repos/dist/release/incubator/hawq/2.
> >> > > > > 2.0.0-incubating
> >> > > > >
> >> > > > > PGP KEYS
> >> > > > > https://dist.apache.org/repos/dist/release/incubator/hawq/KEYS
> >> > > > >
> >> > > > > DOCUMENTATION
> >> > > > > http://hawq.incubator.apache.org/docs/userguide/2.2.0.0-
> incubating
> >> > > > >
> >> > > > > HAWQ RESOURCES
> >> > > > > - JIRA: https://issues.apache.org/jira/browse/HAWQ
> >> > > > > - Wiki: https://cwiki.apache.org/confluence/display/HAWQ/
> >> > > > Apache+HAWQ+Home
> >> > > > > - Mailing list:
> >> > > > > dev@hawq.incubator.apache.org
> >> > > > > u...@hawq.incubator.apache.org
> >> > > > >
> >> > > > > LEARN MORE ABOUT HAWQ
> >> > > > > http://hawq.apache.org
> >> > > > >
> >> > > > > Best regards,
> >> > > > > - Apache HAWQ (incubating) Team
> >> > > > >
> >> > > > > ==
> >> > > > > 

Re: [VOTE]: Apache HAWQ 2.2.0.0-incubating Release (RC3)

[Yi JIN]

+1

downloaded src, verified checksum verification
init hawq successfully and run basic query on mac.

Best,
Yi (yjin)

On Thu, Jun 29, 2017 at 9:04 PM, Xiang Sheng  wrote:

> +1
>
> downloaded src, verified PGP sinatures and SHA256/MD5 checksum verification
> of src release,
> rat check passed, compile passed, install passed.
> init hawq successfully and run basic query successfully.
> on centos6-vm.
>
>
>
> On Wed, Jun 28, 2017 at 9:50 PM, Radar Da lei  wrote:
>
> > +1
> >
> > I downloaded the source code tarball, RAT check passed. Compiled,
> installed
> > and initialized HAWQ successfully. InstallCheck-good tests passed.
> >
> > I also downloaded the binary release tarball, installed all the packages
> by
> > following the wiki page on a clean redhat 7 vm:
> > https://cwiki.apache.org/confluence/display/HAWQ/Build+
> > Package+and+Install+with+RPM
> > After installed the binary packages, I initialized hdfs and hawq
> > successfully, basic queries passed. Also verified the
> > License/Notice/DISCLAIMER files are installed with the binary packages.
> >
> >
> >
> >
> >
> >
> > Regards,
> > Radar
> >
> > On Wed, Jun 28, 2017 at 11:44 AM, Ruilong Huo  wrote:
> >
> > > Hi All,
> > >
> > >
> > > This is the vote for Apache HAWQ (incubating) 2.2.0.0-incubating
> Release
> > > Candidate 3 (RC3). It is a source release for HAWQ core, PXF, and
> Ranger;
> > > and binary release for HAWQ core, and PXF. We have rpm package for the
> > > binary release. Though it is not perfect to have binary tarball absent,
> > we
> > > can add that in the future binary release if there is demand in the
> > > community.
> > >
> > >
> > > The vote will run for at least 72 hours and will close on Saturday,
> July
> > > 1, 2017.
> > >
> > >
> > > 1. Issue fixed in RC3:
> > > https://issues.apache.org/jira/browse/HAWQ-1475: Add LICENSE, NOTICE,
> > and
> > > DISCLAIMER files for c/c++ components for Apache HAWQ binary release
> > > https://issues.apache.org/jira/browse/HAWQ-1489: Fix PXF jar files,
> add
> > > LICENSE, NOTICE, and DISCLAIMER files for Apache HAWQ binary release
> > >
> > >
> > > 2. Wiki page of the release:
> > > https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+2.2.0.0-
> > > incubating+Release
> > >
> > >
> > > 3. Release Notes (Apache Jira generated):
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > > projectId=12318826=12339641
> > >
> > >
> > > 4. Release verification steps can be found at:
> > > For source tarball: https://cwiki.apache.org/confluence/display/HAWQ/
> > > Release+Process%3A+Step+by+step+guide#ReleaseProcess:Stepbystepguide-
> > > ValidatetheReleaseCandidate
> > > For rpm package: https://cwiki.apache.org/
> confluence/display/HAWQ/Build+
> > > Package+and+Install+with+RPM
> > >
> > >
> > > 5. The tag to be voted on: 2.2.0.0-incubating-rc3 (
> > > c08aa560a65af25f1357fc49d736407aff124d05), located here:
> > > https://git-wip-us.apache.org/repos/asf?p=incubator-hawq.
> git;a=commit;h=
> > > c08aa560a65af25f1357fc49d736407aff124d05
> > >
> > >
> > > 6. Git release branch:
> > > https://git-wip-us.apache.org/repos/asf?p=incubator-hawq.
> > > git;a=shortlog;h=refs/heads/2.2.0.0-incubating
> > >
> > >
> > > 7. Source release:
> > > Package: https://dist.apache.org/repos/dist/dev/incubator/hawq/2.2.0.
> > > 0-incubating.RC3/apache-hawq-src-2.2.0.0-incubating.tar.gz
> > > PGP Signature: https://dist.apache.org/repos/
> > > dist/dev/incubator/hawq/2.2.0.0-incubating.RC3/apache-hawq-
> > > src-2.2.0.0-incubating.tar.gz.asc
> > > SHA256 Hash: https://dist.apache.org/repos/
> dist/dev/incubator/hawq/2.2.0
> > .
> > > 0-incubating.RC3/apache-hawq-src-2.2.0.0-incubating.tar.gz.sha256
> > > MD5 Hash: https://dist.apache.org/repos/dist/dev/incubator/hawq/2.2.0.
> > > 0-incubating.RC3/apache-hawq-src-2.2.0.0-incubating.tar.gz.md5
> > >
> > >
> > > 8. Binary release:
> > > Package: https://dist.apache.org/repos/dist/dev/incubator/hawq/2.2.0.
> > > 0-incubating.RC3/apache-hawq-rpm-2.2.0.0-incubating.tar.gz
> > > PGP Signature: https://dist.apache.org/repos/
> > > dist/dev/incubator/hawq/2.2.0.0-incubating.RC3/apache-hawq-
> > > rpm-2.2.0.0-incubating.tar.gz.asc
> > > SHA256 Hash: https://dist.apache.org/repos/
> dist/dev/incubator/hawq/2.2.0
> > .
> > > 0-incubating.RC3/apache-hawq-rpm-2.2.0.0-incubating.tar.gz.sha256
> > > MD5 Hash: https://dist.apache.org/repos/dist/dev/incubator/hawq/2.2.0.
> > > 0-incubating.RC3/apache-hawq-rpm-2.2.0.0-incubating.tar.gz.md5
> > >
> > >
> > > 9. Keys to verify the signature of the release artifact are available
> at:
> > > https://dist.apache.org/repos/dist/dev/incubator/hawq/KEYS
> > >
> > >
> > > 10. The artifact(s) has been signed with Key ID: 1B8B6872
> > >
> > >
> > >
> > >
> > > Please vote accordingly:
> > > [ ] +1 approve
> > > [ ] +0 no opinion
> > > [ ] -1 disapprove (and reason why)
> > >
> > >
> > > Best regards,
> > > Ruilong Huo
> >
>
>
>
> --
> Best Regards,
> Xiang Sheng
>

Re: New committer: Xiang Sheng

[Yi Jin]

Congratulations Xiang!

Best
Yi

On Wed, May 17, 2017 at 11:53 AM, Wen Lin  wrote:

> Hi,
>
> The Project Management Committee (PMC) for Apache HAWQ (incubating) has
> invited Xiang Sheng to become a committer and we are pleased to announce
> that he has accepted.
> Being a committer enables easier contribution to the project since there is
> no need to go via the patch submission process. This should enable better
> productivity. Please join us in congratulating him and we are looking
> forward to collaborating with him in the open source community. His
> contribution includes (but not limited to):
> List contributions to code base, documentation, code review, discussion in
> mailing list, JIRA, etc.
>
> Regards!
> Wen
>

Re: [VOTE] New committer: Xiang Sheng

[Yi Jin]

+1

Xiang showed great code contribution in Apache HAWQ project including
fixing bugs and contributing new features. His work shows his deep insight
of this project and effective development work, he is also active in both
user and dev mail list. Therefore, I think he deserves an Apache HAWQ
committer.

Best,
Yi

On Tue, May 9, 2017 at 5:40 PM, Lili Ma  wrote:

> +1 for Xiang!
>
> Xiang has contributed a lot to Apache HAWQ project, including Ranger
> integration, Hawq Register implementation and Resource Manager bug fix.
> Also Xiang answered a lot of questions for HAWQ in usr/dev mail list and
> StackOverFlow channel.  He also shared the tech talk in Apache HAWQ Meetup.
>
> I think Xiang is well deserved to become a Apache HAWQ committer.
>
> Thanks
> Lili
>
> 2017-05-09 14:18 GMT+08:00 Wen Lin :
>
> > Hi All,
> >
> > This is a VOTE email for promoting candidate *Xiang Sheng* (with github
> id
> > *stanlyxiang*) from contributor to committer, who has been contributing
> to
> > Apache HAWQ (incubating) in last one and half years (from Nov. 2015 to
> May
> > 2017). Please give +1, 0 or -1 with reasons in this email thread.
> >
> > His contribution includes (but not limited to):
> > *Direct contribution to code base:*
> >
> >- 43 commits in total with some major components in hawq involved,
> >including contributions to Apache Ranger integration, hawq register
> >and command line tools, resource manager.
> >https://github.com/apache/incubator-hawq/commits?author=stanlyxiang
> >- 41 closed PRs: https://github.com/apache/incubator-hawq/pulls?q=is%
> >3Apr+is%3Aclosed+author%3Astanlyxiang
> > > 3Apr+is%3Aclosed+author%3Astanlyxiang>
> >
> >- 13 improvements including documentation, test, build, command line
> >tools, code refactor.
> >
> >
> >- HAWQ-140 Add more
> >   information in HAWQ build instructions file
> >   - HAWQ-143 Add
> >   informations in Apache-HAWQ README.md
> >   - HAWQ-154Update
> >   BUILD_INSTRUCTIONS file for dependencies install method and
> > ambiguous word.
> >   - HAWQ-203Add a
> guc
> >   for debug metadata, datalocality time stat.
> >   - HAWQ-265  >    >Change metadata
> >   share memory flush strategy to prevent out of share memory problem
> > when
> >   create too many hdfs_file.
> >   - HAWQ-279    Add
> 2
> >   guc in template-hawq-site
> >   - HAWQ-284  Add a
> > udf
> >   for new metadata flush strategy testing.
> >   - HAWQ-313 Fix
> >   dereference pointer before null check
> >   - HAWQ-475 Add
> >   build_type gcov for code coverage.
> >   - HAWQ-486 gpcheck
> >   can’t find namenode with Ambari install PHD
> >   - HAWQ-498  Update
> >   property value in gpcheck.cnf
> >   - HAWQ-1430 
> Update
> >   ranger related log level to avoid log flood
> >   - HAWQ-77  Fix
> source
> >   code comment for new ALTER/CREATE RESOURCE QUEUE
> >- 10 bug fixes including test failure, resource manager, core-dump,
> >command line tools, build components.
> >
> >
> >- HAWQ-295 New
> metadata
> >   flush strategy remove 1 entry every time flush due to flush
> condition
> >   wrong.
> >   - HAWQ-998  Fix
> test
> >   for aggregate-with-null test.
> >   - HAWQ-1051 
> > failing
> >   in reverse DNS lookup causes resource manager core dump
> >   - HAWQ-1076.
> Fixed
> >   USAGE privilege bug on nextval(sequence) when optimizer on
> >   - HAWQ-1117  RM
> >   crash when init db after configure with param '--enable-cassert'
> >   - HAWQ-1160  Hawq
> >   checkperf does not handle hostfile correctly
> >   - HAWQ-1170 
> Crash
> >   at cleanup_allocation_algorithm() when enable ‘—enable-cassert’
> > option
> >   - HAWQ-1208 
> > Porting
> >   gpdb 

[jira] [Created] (HAWQ-1439) tolerate system time changed to earlier point when checking resource context timeout

[Yi Jin (JIRA)]

Yi Jin created HAWQ-1439:


 Summary: tolerate system time changed to earlier point when 
checking resource context timeout
 Key: HAWQ-1439
 URL: https://issues.apache.org/jira/browse/HAWQ-1439
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Resource Manager
Reporter: Yi Jin
Assignee: Ed Espino
 Fix For: 2.3.0.0-incubating


When the system time is changed to an earlier point, the resource context maybe 
timed out because the context's latest action time is larger than checking time.

This fix is to adjust latest action time to new system time when it is adjusted 
to earlier point.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [VOTE] Apache HAWQ Incubator "powered by" logo vote

[Yi Jin]

the log, with "HAWQ" text, but without drop shadow.

Best
Yi

On Mon, Apr 24, 2017 at 12:31 PM, Hubert Zhang  wrote:

> +1 for HAWQ logo, with "HAWQ" text, with drop shadow
>
> On Sun, Apr 23, 2017 at 5:08 AM, Alex (Oleksandr) Diachenko <
> odiache...@pivotal.io> wrote:
>
> > +1 for HAWQ logo, with "HAWQ" text, with drop shadow
> >
> > On Sat, Apr 22, 2017 at 9:11 AM, Lili Ma  wrote:
> >
> > > +1 HAWQ logo, with "HAWQ" text, with drop shadow
> > >
> > > On Sat, Apr 22, 2017 at 2:43 AM, Ivan Weng  wrote:
> > >
> > > > +1 HAWQ logo, with "HAWQ" text, with drop shadow
> > > >
> > > >
> > > > Regards,
> > > > Ivan
> > > >
> > > > On Sat, Apr 22, 2017 at 7:30 AM, Shivram Mani <
> shivram.m...@gmail.com>
> > > > wrote:
> > > >
> > > > > +1 HAWQ logo, with "HAWQ" text, with drop shadow
> > > > >
> > > > > On Fri, Apr 21, 2017 at 10:48 AM, Ed Espino 
> > wrote:
> > > > >
> > > > > > I have uploaded a few variations of the Apache HAWQ "powered by"
> > > logo.
> > > > > > Please select your preference via vote. This vote will remain
> open
> > > for
> > > > at
> > > > > > least 72 hours.
> > > > > >
> > > > > > [ ] HAWQ logo, no text, no drop shadow
> > > > > > [ ] HAWQ logo, no text, with drop shadow
> > > > > > [ ] HAWQ logo, with "HAWQ" text, no drop shadow
> > > > > > [ ] HAWQ logo, with "HAWQ" text, with drop shadow
> > > > > >
> > > > > > The full variations and additional sizes can be viewed here:
> > > > > > https://cwiki.apache.org/confluence/pages/viewpage.
> > > > > action?pageId=69407067
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > *Ed Espino*
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > shivram mani
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Thanks
>
> Hubert Zhang
>

[jira] [Created] (HAWQ-1433) ALTER RESOURCE QUEUE DDL does not check the format of attribute MEMORY_CLUSTER_LIMIT

[Yi Jin (JIRA)]

Yi Jin created HAWQ-1433:


 Summary: ALTER RESOURCE QUEUE DDL does not check the format of 
attribute MEMORY_CLUSTER_LIMIT
 Key: HAWQ-1433
 URL: https://issues.apache.org/jira/browse/HAWQ-1433
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Resource Manager
Reporter: Yi Jin
Assignee: Ed Espino
 Fix For: 2.3.0.0-incubating


Shubham Sharma <topologicalqu...@gmail.com>
2:11 PM (2 hours ago)

to user, sebastiao.gone. 
Hello Sebastio, I think you have encountered the following issue - 

1 - Problem -  alter resource queue pg_default with 
(CORE_LIMIT_CLUSTER/MEMORY_LIMIT_CLUSTER=90);

gpadmin=# select * from pg_resqueue;
  rsqname   | parentoid | activestats | memorylimit | corelimit | resovercommit 
| allocpolicy | vsegresourcequota | nvsegupperlimit | nvseglowerlimit | nvseg
upperlimitperseg | nvseglowerlimitperseg | creationtime |  updatetime   
| status 
+---+-+-+---+---+-+---+-+-+--
-+---+--+---+
 pg_root| 0 |  -1 | 100%| 100%  | 2 
| even|   |   0 |   0 |  
   0 | 0 |  |   
| branch
 pg_default |  9800 |  20 | 50% | 50%   | 2 
| even| mem:256mb |   0 |   0 |  
   0 | 0 |  | 2017-04-12 
22:45:55.056102+01 | 
(2 rows)

gpadmin=# alter resource queue pg_default with (CORE_LIMIT_CLUSTER=90);
ALTER QUEUE

gpadmin=# select * from test;
 a 
---
(0 rows)
gpadmin=# \q

2 - restart hawq cluster

3 - ERROR

[gpadmin@hdp3 ~]$ psql
psql (8.2.15)
Type "help" for help.
gpadmin=# select * from test;
WARNING:  FD 31 having errors raised. errno 104
ERROR:  failed to register in resource manager, failed to receive content 
(pquery.c:787)

3 - alter resource queue pg_default with 
(CORE_LIMIT_CLUSTER/MEMORY_LIMIT_CLUSTER=50%); --Let's switch back
! Not allowed !
alter resource queue pg_default with (CORE_LIMIT_CLUSTER=50%);
WARNING:  FD 33 having errors raised. errno 104
ERROR:  failed to register in resource manager, failed to receive content 
(resqueuecommand.c:364)

4 -  How to fix - Please be extra careful while using this.
gpadmin=# begin;
BEGIN
gpadmin=# set allow_system_table_mods='dml';
SET
gpadmin=# select * from pg_resqueue where corelimit=90;
  rsqname   | parentoid | activestats | memorylimit | corelimit | resovercommit 
| allocpolicy | vsegresourcequota | nvsegupperlimit | nvseglowerlimit | nvseg
upperlimitperseg | nvseglowerlimitperseg | creationtime |  updatetime   
| status 
+---+-+-+---+---+-+---+-+-+--
-+---+--+---+
 pg_default |  9800 |  20 | 50% | 90| 2 
| even| mem:256mb |   0 |   0 |  
   0 | 0 |  | 2017-04-12 
22:59:30.092823+01 | 
(1 row)
gpadmin=# update pg_resqueue set corelimit='50%' where corelimit=90;
UPDATE 1
gpadmin=# commit;
COMMIT

5 - System should be back to normal

gpadmin=# select * from test;
 a 
---
(0 rows)


Regards,
Shubh




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [VOTE]: Apache HAWQ 2.2.0.0-incubating Release (RC2)

[Yi Jin]

+1.

Checked wiki page, release notes.
Using Centos 7 downloaded rpm, installed, initialized hawq cluster,
started/stopped cluster, ran basic queries.

On Wed, Apr 12, 2017 at 8:17 PM, Amy Bai  wrote:

> +1.
>
> Checked wiki page, release notes.
> Using RHEL 7 downloaded rpm, installed, initialized hawq cluster,
> started/stopped cluster, ran basic queries.
>
> Regards,
> Amy
>
> On Wed, Apr 12, 2017 at 5:47 PM, Paul Guo  wrote:
>
> > +1
> >
> > Tried compiling with simple options and then installing the binaries,
> > also tried installing with the rpm packets following the wiki page.
> >
> > I tested some simple queries with the default setting.
> >
> >
> > 2017-04-12 16:39 GMT+08:00 Radar Da lei :
> >
> > > +1
> > >
> > > Following the install wiki page step by step, I spend only 15 minutes
> to
> > > get Bigtop Hadoop and HAWQ binary tarball installed and initialized
> > > successfully on a brand new RHEL 7 server.
> > >
> > > Compile and install passed with the release source tarball.
> > >
> > > RAT checks passed.
> > >
> > > Verified the previous bugs are fixed.
> > >
> > >
> > >
> > >
> > >
> > > Regards,
> > > Radar
> > >
> > > On Wed, Apr 12, 2017 at 2:17 PM, Hubert Zhang 
> wrote:
> > >
> > > > +1
> > > > Download the rpm and install successfully on centos7.
> > > > Run SQL successfully with ranger enabled.
> > > >
> > > > On Wed, Apr 12, 2017 at 6:18 AM, Kavinder Dhaliwal <
> > kdhali...@pivotal.io
> > > >
> > > > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Successfully Installed bigtop hadoop rpms on centos7
> > > > > Successfully installed hawq & pxf rpms from the binary release
> > > > > Successfully verified PGP signatures and checksums
> > > > > Successfully created a native table, inserted tuples and selected
> > > tuples
> > > > > Successfully created an external table with PXF and query a hive
> > table
> > > > via
> > > > > the HiveORC profile
> > > > >
> > > > > On Mon, Apr 10, 2017 at 3:13 AM, Ruilong Huo 
> > wrote:
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > This is the vote for Apache HAWQ (incubating) 2.2.0.0-incubating
> > > > > > Release Candidate 2 (RC2). It is a source release and binary
> > > > > > release. We have rpm package for the binary release. Though it is
> > > > > > not perfect to have binary tarball absent, we can add that in the
> > > > future
> > > > > > binary release if there is demand in the community.
> > > > > >
> > > > > > The vote will run for at least 72 hours and will close on
> Thursday,
> > > > April
> > > > > > 13, 2017.
> > > > > >
> > > > > > 1. Issue fixed in RC2:
> > > > > > https://issues.apache.org/jira/browse/HAWQ-1425: Incorrect init
> > > > cluster
> > > > > > error message while ssh connection failed
> > > > > >
> > > > > > 2. Wiki page of the release:
> > > > > > https://cwiki.apache.org/confluence/display/HAWQ/
> > > Apache+HAWQ+2.2.0.0-
> > > > > > incubating+Release
> > > > > >
> > > > > > 3. Release Notes (Apache Jira generated):
> > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
> > > > > > ctId=12318826=12339641
> > > > > >
> > > > > > 4. Release verification steps can be found at:
> > > > > > For source tarball: https://cwiki.apache.org/
> > > > confluence/display/HAWQ/Re
> > > > > > lease+Process%3A+Step+by+step+guide#ReleaseProcess:Stepbyste
> > > > > > pguide-ValidatetheReleaseCandidate
> > > > > > For rpm package: https://cwiki.apache.org/
> > confluence/display/HAWQ/Bu
> > > > > > ild+Package+and+Install+with+RPM
> > > > > >
> > > > > > 5. The tag to be voted on: 2.2.0.0-incubating-rc2 (
> > > > > > 32e01c76b54782d0367204737d412cdf5b6146ff), located here:
> > > > > > https://git-wip-us.apache.org/repos/asf?p=incubator-hawq.
> > > > git;a=commit;h=
> > > > > > 32e01c76b54782d0367204737d412cdf5b6146ff
> > > > > >
> > > > > > 6. Git release branch:
> > > > > > https://git-wip-us.apache.org/repos/asf?p=incubator-hawq.git
> > > > > > ;a=shortlog;h=refs/heads/2.2.0.0-incubating
> > > > > >
> > > > > > 7. Source release:
> > > > > > Package: https://dist.apache.org/repos/dist/dev/incubator/
> > > > > > hawq/2.2.0.0-incubating.RC2/apache-hawq-src-2.2.0.0-
> > > incubating.tar.gz
> > > > > > PGP Signature: https://dist.apache.org/repos/dist/dev/
> > > > > > incubator/hawq/2.2.0.0-incubating.RC2/apache-hawq-
> > > > > > src-2.2.0.0-incubating.tar.gz.asc
> > > > > > SHA256 Hash: https://dist.apache.org/repos/
> > dist/dev/incubator/hawq/
> > > > > > 2.2.0.0-incubating.RC2/apache-hawq-src-2.2.0.0-incubating.
> > > > tar.gz.sha256
> > > > > > MD5 Hash: https://dist.apache.org/repos/dist/dev/incubator/hawq/
> > > > > > 2.2.0.0-incubating.RC2/apache-hawq-src-2.2.0.0-incubating.
> > tar.gz.md5
> > > > > >
> > > > > > 8. Binary release:
> > > > > > Package: https://dist.apache.org/repos/dist/dev/incubator/
> > > > > > hawq/2.2.0.0-incubating.RC2/apache-hawq-rpm-2.2.0.0-
> > > incubating.tar.gz
> > > > > > PGP Signature: 

Re: [VOTE]: Apache HAWQ 2.2.0.0-incubating Release

[Yi Jin]

+1.

Checked wiki page, release notes.
Using MacOS downloaded and verified package of both rpm and source code
including (asc, md5, sha256)
Using MacOS downloaded source code tar ball, configured, compiled,
initialized hawq cluster and started/stoped cluster, ran basic feature
tests.
Using Centos7 downloaded rpm, installed, initialized hawq cluster,
started/stopped cluster, ran basic queries.

Best,
Yi


On Sun, Apr 2, 2017 at 1:29 AM, Ruilong Huo  wrote:

> Hi All,
>
> This is the vote for Apache HAWQ (incubating) 2.2.0.0-incubating
> Release Candidate 1 (RC1). It is a source release and binary
> release. We have rpm package for the binary release. Though it is
> not perfect to have binary tarball absent, we can add that in the future
> binary release if there is demand in the community.
>
> The vote will run for at least 120 hours and will close on Thursday,
> April 6, 2017 as there is weekend and holiday in next few days.
>
> 1. Wiki page of the release:
> https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ
> +2.2.0.0-incubating+Release
>
> 2. Release Notes (Apache Jira generated):
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
> ctId=12318826=12339641
>
> 3. Release verification steps can be found at:
> For source tarball:
> https://cwiki.apache.org/confluence/display/HAWQ/Release+Pro
> cess%3A+Step+by+step+guide#ReleaseProcess:Stepbystepguide-
> ValidatetheReleaseCandidate
> For rpm package:
> https://cwiki.apache.org/confluence/display/HAWQ/Build+Packa
> ge+and+Install+with+RPM
>
> 4. The tag to be voted on: 2.2.0.0-incubating-rc1
> (18e928c91e36ec0bf6af7f9760fc8a7445537816), located here:
> https://git-wip-us.apache.org/repos/asf?p=incubator-hawq.git
> ;a=commit;h=18e928c91e36ec0bf6af7f9760fc8a7445537816
>
> 5. Git release branch:
> https://git-wip-us.apache.org/repos/asf?p=incubator-hawq.git
> ;a=shortlog;h=refs/heads/2.2.0.0-incubating
>
> 6. Source release:
> Package:
> https://dist.apache.org/repos/dist/dev/incubator/hawq/2.2.0.
> 0-incubating.RC1/apache-hawq-src-2.2.0.0-incubating.tar.gz
> PGP Signature:
> https://dist.apache.org/repos/dist/dev/incubator/hawq/2.2.0.
> 0-incubating.RC1/apache-hawq-src-2.2.0.0-incubating.tar.gz.asc
> SHA256
> 
> Hash:
> https://dist.apache.org/repos/dist/dev/incubator/hawq/2.2.0.
> 0-incubating.RC1/apache-hawq-src-2.2.0.0-incubating.tar.gz.sha256
> MD5
> 
> Hash:
> https://dist.apache.org/repos/dist/dev/incubator/hawq/2.2.0.
> 0-incubating.RC1/apache-hawq-src-2.2.0.0-incubating.tar.gz.md5
>
> 7. Binary release:
> Package:
> https://dist.apache.org/repos/dist/dev/incubator/hawq/2.2.0.
> 0-incubating.RC1/apache-hawq-rpm-2.2.0.0-incubating.tar.gz
> PGP Signature:
> https://dist.apache.org/repos/dist/dev/incubator/hawq/2.2.0.
> 0-incubating.RC1/apache-hawq-rpm-2.2.0.0-incubating.tar.gz.asc
> SHA256
> 
> Hash:
> https://dist.apache.org/repos/dist/dev/incubator/hawq/2.2.0.
> 0-incubating.RC1/apache-hawq-rpm-2.2.0.0-incubating.tar.gz.sha256
> MD5
> 
> Hash:
> https://dist.apache.org/repos/dist/dev/incubator/hawq/2.2.0.
> 0-incubating.RC1/apache-hawq-rpm-2.2.0.0-incubating.tar.gz.md5
>
> 8. Keys to verify the signature of the release artifact are available at:
> https://dist.apache.org/repos/dist/dev/incubator/hawq/KEYS
>
> 9. The artifact(s) has been signed with Key ID: 1B8B6872
>
> To help in tallying the vote, Apache HAWQ Incubator PMC members
> (http://home.apache.org/phonebook.html?pmc=incubator) please be sure
> to indicate "(binding)" with your vote.
>
> Please vote accordingly:
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>

[jira] [Created] (HAWQ-1400) Add a small sleeping period in feature test utility before dropping test database

[Yi Jin (JIRA)]

Yi Jin created HAWQ-1400:


 Summary: Add a small sleeping period in feature test utility 
before dropping test database
 Key: HAWQ-1400
 URL: https://issues.apache.org/jira/browse/HAWQ-1400
 Project: Apache HAWQ
  Issue Type: Improvement
  Components: Tests
Reporter: Yi Jin
Assignee: Jiali Yao


This improvement is to raise the stability of feature test. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [VOTE] Release Apache HAWQ 2.1.0.0-incubating (RC4)

[Yi Jin]

+1

Downloaded source code, compile and installed, basically tested.

Yi Jin

On Wed, Feb 22, 2017 at 12:21 AM, Radar Da lei <r...@pivotal.io> wrote:

> +1
>
> Downloaded source tarball, compiled, installed, initialized successfully.
>
>
>
> Regards,
> Radar
>
> On Tue, Feb 21, 2017 at 6:33 PM, Ivan Weng <iw...@pivotal.io> wrote:
>
> > +1
> >
> > Compile, install, license check and basic test are OK in my laptop.
> >
> >
> > Regards,
> > Ivan
> >
> > On Tue, Feb 21, 2017 at 6:26 PM, Paul Guo <paul...@gmail.com> wrote:
> >
> > > +1 ("paulguo")
> > >
> > > compile and install, basic license check, regression/feature testing.
> > >
> > > 2017-02-21 18:04 GMT+08:00 Xiang Sheng <xsh...@pivotal.io>:
> > >
> > > > +1 for successfully mvn verify  &&  configure && make && make install
> > &&
> > > > hawq start
> > > >
> > > > On Tue, Feb 21, 2017 at 4:09 PM, Hubert Zhang <hzh...@pivotal.io>
> > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > On Mon, Feb 20, 2017 at 11:12 AM, Ed Espino <esp...@apache.org>
> > wrote:
> > > > >
> > > > > > This is the vote for Apache HAWQ (incubating) 2.1.0.0-incubating
> > > > > > Release Candidate 4 (RC4).  This is a Source only release.
> > > > > >
> > > > > > The vote will run for at least 72 hours and will close on Monday,
> > > > > > February 22, 2017.
> > > > > >
> > > > > > Issues fixed in RC4:
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1343 - Travis.ci
> > build
> > > is
> > > > > > passing.
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1344 - Rollback
> > Apache
> > > > HAWQ
> > > > > > version to 2.1, update PXF to 3.2
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1134 - Add Bigtop
> > > layout
> > > > > > specific pxf-private classpath
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1340 - Remove
> unused
> > > > file -
> > > > > > tools/bin/disqual.ksh
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1330 - Adding
> > > additional
> > > > > ASF
> > > > > > headers to test files.
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1314 - Added
> upgrade
> > > for
> > > > > > function pxf_get_item_fields.
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1323 - Add license
> > > header
> > > > > > added to each source file in test directory
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1338 - Fixed writer
> > > > process
> > > > > > doesn't exit nicely in some case
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1334 - QD thread
> > should
> > > > set
> > > > > > error code if failing so that the main process for the query
> could
> > > exit
> > > > > > soon
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1315 - Function
> > > > > > validateResourcePoolStatus() in resourcepool.c is logging the
> wrong
> > > > > > information
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1314 - Added
> upgrade
> > > for
> > > > > > function pxf_get_item_fields.
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1326 - Cancel the
> > query
> > > > > > earlier if one of the segments for the query crashes
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1329 - pg_catalog
> > view
> > > > > > fallback failed.
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1327 - Move ID from
> > > > struct
> > > > > > SegStatData to struct SegInfoData so that ID could be used in QD.
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1324 - Fixed crash
> at
> > > > query
> > > > > > cancel, signal hanler cannot call unsafe elog()
> > > > > > * https://issues.apache.org/jira/browse/HAWQ-1321 - failNames
> > > wrongly
> > > > > uses
> > > > > > memory context to build message when 

[jira] [Created] (HAWQ-1321) failNames wrongly uses memory context to build message when ANALYZE failed

[Yi Jin (JIRA)]

Yi Jin created HAWQ-1321:


 Summary: failNames wrongly uses memory context to build message 
when ANALYZE failed
 Key: HAWQ-1321
 URL: https://issues.apache.org/jira/browse/HAWQ-1321
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Core
Reporter: Yi Jin
Assignee: Ed Espino
 Fix For: 2.2.0.0-incubating


I find one bug exist in generating error message for ANALYZE when the message 
size is large. 

In analyzeStmt(), there is a variable called failNames. It is initialized in 
caller's memory context, but it repallocs memory in relation context, and it is 
freed in statement context. This is bug of wrongly using memory context. when 
the relation and statement context are dropped at then end of function 
analyzeStmt(), pat of its content will be flushed with 0. This explain why 
another block's header was randomly wiped out in the bug.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [VOTE]: Apache HAWQ 2.1.0.0-incubating Release Candidate 3 (RC3)

[Yi Jin]

Checked

1） download code, unzip and build
2)  ran basic test
3) read release notes

+1 approve

Yi


On Mon, Feb 6, 2017 at 10:28 PM, Paul Guo  wrote:

> Ed, I think we could extend the deadline a bit since I know some people was
> just back from vacation.
>
> 2017-02-03 18:07 GMT+08:00 Ed Espino :
>
> > This is the vote for Apache HAWQ (incubating) 2.1.0.0-incubating Release
> > Candidate 3 (RC3).  This is a Source only release.
> >
> > The vote will run for at least 72 hours and will close on Monday,
> February
> > 6, 2017.
> >
> > Issues fixed in RC3:
> > * https://issues.apache.org/jira/browse/HAWQ-1308 - Fix Javadocs compile
> > warnings on JDK-8 for PXF
> > * https://issues.apache.org/jira/browse/HAWQ-1309 - PXF Service must
> > default to port 51200 and user pxf
> > * https://issues.apache.org/jira/browse/HAWQ-1306 - PXF no longer builds
> > with JDK 1.8 (after commit 6fa1ced2)
> > * https://issues.apache.org/jira/browse/HAWQ-1297 - Make PXF install
> ready
> > from source code
> > * https://issues.apache.org/jira/browse/HAWQ-1228 - Use profile based on
> > file format in HCatalog integration(HiveRC, HiveText profiles)
> > * https://issues.apache.org/jira/browse/HAWQ-1297 - Make PXF install
> ready
> > from source
> >
> > Issues fixed in RC2:
> > * https://issues.apache.org/jira/browse/HAWQ-1267 - NOTICE file need to
> be
> > updated to reflect the right year for copyright
> > * https://issues.apache.org/jira/browse/HAWQ-1268 - pom.xml need to be
> > updated to reflect the correct version for apache hawq 2.1.0.0-incubating
> > release
> > * https://issues.apache.org/jira/browse/HAWQ-1277 - The "make" command
> > generates an error on CentOS 7 when --with-perl is run in configure.
> >
> > Issues deferred from previous vote(s):
> > * [RC1] https://issues.apache.org/jira/browse/HAWQ-1278 - Investigate
> > installcheck-good issue on Mac OSX
> >
> > Release Notes (Apache Jira generated):
> > *
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > projectId=12318826=12338900
> >
> > Release verification steps can be found at:
> > *
> > https://cwiki.apache.org/confluence/display/HAWQ/
> > Release+Process%3A+Step+by+step+guide#ReleaseProcess:Stepbystepguide-
> > ValidatetheReleaseCandidate
> >
> > The tag to be voted on: 2.1.0.0-incubating-rc3
> > (12c7df017551f1c3b0deb38c7243db3e018ef62c), located here:
> > *
> > https://git-wip-us.apache.org/repos/asf?p=incubator-hawq.git;a=commit;h=
> > 12c7df017551f1c3b0deb38c7243db3e018ef62c
> >
> > Git release branch:
> > *
> > https://git-wip-us.apache.org/repos/asf?p=incubator-hawq.
> > git;a=shortlog;h=refs/heads/2.1.0.0-incubating
> >
> > Source release package:
> > *
> > https://dist.apache.org/repos/dist/dev/incubator/hawq/2.1.0.
> > 0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz
> >
> > Source release verification:
> >
> > * PGP Signature:
> >
> > https://dist.apache.org/repos/dist/dev/incubator/hawq/2.1.0.
> > 0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc
> > * SHA256/MD5 Hash:
> >
> > https://dist.apache.org/repos/dist/dev/incubator/hawq/2.1.0.
> > 0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256
> >
> > https://dist.apache.org/repos/dist/dev/incubator/hawq/2.1.0.
> > 0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5
> >
> > Keys to verify the signature of the release artifact are available at:
> > * https://dist.apache.org/repos/dist/dev/incubator/hawq/KEYS
> >
> > The artifact(s) has been signed with Key ID: 57325522
> >
> > Previous Vote Thread(s):
> > * RC1 [VOTE] Thread
> >
> > https://lists.apache.org/thread.html/882dc11880f8794fc5603aee470c0e
> > 5f912f579a7c247c270dbeb9a4@%3Cdev.hawq.apache.org%3E
> > * RC1 [RESULT][VOTE] Thread
> >
> > https://lists.apache.org/thread.html/ada53fa0dc9547bee47d8e73cc09ec
> > 3e99e6335c291a78155b5af139@%3Cdev.hawq.apache.org%3E
> > * RC2 [VOTE] Thread
> >
> > https://lists.apache.org/thread.html/02cb5e2218c516b3c17c3819afdcc4
> > fdbff670a0c01918168650e981@%3Cdev.hawq.apache.org%3E
> > * RC2 [RESULT][VOTE] Thread
> >
> > https://lists.apache.org/thread.html/de507d211356e00e1c07b857811a89
> > 258755feaacf1533e2d49cbfb9@%3Cdev.hawq.apache.org%3E
> >
> > To help in tallying the vote, Apache HAWQ Incubating PMC members (
> > http://home.apache.org/phonebook.html?pmc=incubator) please be sure to
> > indicate "(binding)" with your vote.
> >
> > Please vote accordingly:
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove (and reason why)
> >
> > Regards,
> > -=e
> >
> > --
> > *Ed Espino*
> > *esp...@apache.org *
> >
>

[jira] [Created] (HAWQ-1310) Reformat resource_negotiator()

[Yi Jin (JIRA)]

Yi Jin created HAWQ-1310:


 Summary: Reformat resource_negotiator()
 Key: HAWQ-1310
 URL: https://issues.apache.org/jira/browse/HAWQ-1310
 Project: Apache HAWQ
  Issue Type: Improvement
  Components: Core
Reporter: Yi Jin
Assignee: Ed Espino
 Fix For: 2.2.0.0-incubating


The indents in function resource_negotiator() is not aligned. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: New Committer: Lisa Owen

[Yi Jin]

Hi Lisa,

Congratulations!

Best,
Yi

On Wed, Feb 1, 2017 at 11:25 AM, Ed Espino  wrote:

> The Project Management Committee (PMC) for Apache HAWQ (incubating) has
> invited Lisa Owen to become a committer and we are pleased to announce that
> she has accepted.
>
> Her documentation contributions include (but not limited to):
>
>- 223 commits (incubator-hawq-docs
>) in
>total which span key components of hawq.
>- 61 closed PRs:
>https://github.com/apache/incubator-hawq-docs/pulls?q=
> is%3Apr+is%3Aclosed+author%3Alisakowen
>- 33 HAWQ Jiras reported:
>https://issues.apache.org/jira/browse/HAWQ-1273?jql=
> reporter%20in%20(lisakowen)
>
> Being a committer enables easier contribution to the project. This should
> enable improved productivity.
>
> Please join us in congratulating her.  We are looking forward to a closer
> collaboration with the open source community.
>
> *CONGRATULATIONS LISA!!!*
>
> Warm Regards,
> -=Ed Espino
>
> --
> *Ed Espino*
> *esp...@apache.org *
>

Re: hawq_rm_stmt_vseg_memory and hawq_rm_nvseg_perquery_perseg_limit

[Yi Jin]

I created https://issues.apache.org/jira/browse/HAWQ-1299 to track this,
When have an agreement about the limit, I will fix it at once.

Please go there and comment for this improvement. Thank you.

Yi

On Mon, Jan 30, 2017 at 3:56 PM, Yi Jin <y...@pivotal.io> wrote:

> Jon, Sorry for late response, I find I missed this one.
>
> With regard to the memory quota of one vseg, I think I know your
> consideration now. It makes sense. What's your expectation for the memory
> limit? 32GB or 64GB? I think I can open a jira to track this and make
> decision which limit we can extend to.
>
> And for  hawq_rm_nvseg_perquery_limit, I think if you dont want this
> limits your query resource in a large cluster ,just set it to a large value
> 65535 for example, then I think you can achieve your goal, saying using
> hawq_rm_nvseg_perquery_perseg_limit to dynamically limit the number of
> vsegs for one query.
>
> Yi
>
> On Tue, Jan 24, 2017 at 3:21 AM, Jon Roberts <jrobe...@pivotal.io> wrote:
>
>> It is common in Greenplum to use "explain analyze" to look at the query
>> plan and if the query spills to disk, then increasing the statement memory
>> will typically increase the speed of the query.  With hash distributed
>> tables in Hawq, this will be true too and the only way, short of rewriting
>> the query, to make it perform better.
>>
>> For the really large Greenplum clusters out there, we don't see benefits
>> from running 8 segments per host.  Instead, we dial this back to 4
>> segments
>> because the network traffic in the interconnect becomes the bottleneck.
>> For Hawq where the clusters will likely be much bigger, I think we will
>> have the same network limitations.  So with random distribution, we will
>> probably reduce the number of vsegs to get better performance on the
>> network side and then use more memory per vseg to prevent spilling to disk
>> and use more of the available resources.  That is why we need the ability
>> to allocate more than 16GB of RAM per vseg.
>>
>> On a different but related note, I would like to see
>> hawq_rm_nvseg_perquery_limit be changed to not determine the number of
>> vsegs per query across the entire cluster but at the datanode level.  That
>> would make it much easier to expand and shrink a cluster without having to
>> adjust GUCs each time.
>>
>>
>>
>> Jon Roberts
>> Principal Engineer | jrobe...@pivotal.io | 615-426-8661
>>
>> On Sun, Jan 22, 2017 at 10:05 PM, Yi Jin <y...@pivotal.io> wrote:
>>
>> > Hi Jon,
>> >
>> > For scenario 1, it is true. However, do you think it necessary to make
>> one
>> > vseg having so high memory quota setting? Of course, I think if we force
>> > hawq_rm_stmt_nvseg a value different with bucket number, optimizer will
>> > give different plan. This is way to force high concurrency and high
>> > resource occupation.
>> >
>> > For scenario 2, I think if we want to release more concurrency in
>> executing
>> > one query, we should advice customer to
>> > increase hawq_rm_nvseg_perquery_limit (default 512)
>> > and hawq_rm_nvseg_perquery_perseg_limit(default 6), then we choose an
>> > appropriate vseg mem quota, as a result HAWQ will make decision to
>> leverage
>> > more concurrency and more resource for big queries automatically.  I
>> mean
>> > HAWQ will choose the number of virtual segments automatically, not
>> always
>> > high number of virtual segments.
>> >
>> > For scenario 3, It is recommended not to assign all system memory to
>> HAWQ
>> > in the segment side. It is a good idea to leave some for system and for
>> the
>> > other applications.
>> >
>> > I think it is fine to support higher memory limit for one virtual
>> segment,
>> > but still I have no idea why it is necessary.
>> >
>> > As HAWQ uses different memory allocation strategy comparing with
>> Greenplum,
>> > it is true, the external behavior of HAWQ is different. Virtual segment
>> > (VSEG) is a critical concept in HAWQ to manage resource consumption and
>> > query QE concurrency, in fact for Greenplum, the QE concurrency is
>> fixed,
>> > i.e. the number of QEs for one query execution is fixed, this is why
>> > Greenplum user just needs to think of memory consumption in one
>> segment, as
>> > all segments consume the same amount of memory for the same query. While
>> > back to HAWQ, this number is dynamic, you can easily change resource
>> queue
>> > de

Re: hawq_rm_stmt_vseg_memory and hawq_rm_nvseg_perquery_perseg_limit

[Yi Jin]

Jon, Sorry for late response, I find I missed this one.

With regard to the memory quota of one vseg, I think I know your
consideration now. It makes sense. What's your expectation for the memory
limit? 32GB or 64GB? I think I can open a jira to track this and make
decision which limit we can extend to.

And for  hawq_rm_nvseg_perquery_limit, I think if you dont want this limits
your query resource in a large cluster ,just set it to a large value 65535
for example, then I think you can achieve your goal, saying using
hawq_rm_nvseg_perquery_perseg_limit to dynamically limit the number of
vsegs for one query.

Yi

On Tue, Jan 24, 2017 at 3:21 AM, Jon Roberts <jrobe...@pivotal.io> wrote:

> It is common in Greenplum to use "explain analyze" to look at the query
> plan and if the query spills to disk, then increasing the statement memory
> will typically increase the speed of the query.  With hash distributed
> tables in Hawq, this will be true too and the only way, short of rewriting
> the query, to make it perform better.
>
> For the really large Greenplum clusters out there, we don't see benefits
> from running 8 segments per host.  Instead, we dial this back to 4 segments
> because the network traffic in the interconnect becomes the bottleneck.
> For Hawq where the clusters will likely be much bigger, I think we will
> have the same network limitations.  So with random distribution, we will
> probably reduce the number of vsegs to get better performance on the
> network side and then use more memory per vseg to prevent spilling to disk
> and use more of the available resources.  That is why we need the ability
> to allocate more than 16GB of RAM per vseg.
>
> On a different but related note, I would like to see
> hawq_rm_nvseg_perquery_limit be changed to not determine the number of
> vsegs per query across the entire cluster but at the datanode level.  That
> would make it much easier to expand and shrink a cluster without having to
> adjust GUCs each time.
>
>
>
> Jon Roberts
> Principal Engineer | jrobe...@pivotal.io | 615-426-8661
>
> On Sun, Jan 22, 2017 at 10:05 PM, Yi Jin <y...@pivotal.io> wrote:
>
> > Hi Jon,
> >
> > For scenario 1, it is true. However, do you think it necessary to make
> one
> > vseg having so high memory quota setting? Of course, I think if we force
> > hawq_rm_stmt_nvseg a value different with bucket number, optimizer will
> > give different plan. This is way to force high concurrency and high
> > resource occupation.
> >
> > For scenario 2, I think if we want to release more concurrency in
> executing
> > one query, we should advice customer to
> > increase hawq_rm_nvseg_perquery_limit (default 512)
> > and hawq_rm_nvseg_perquery_perseg_limit(default 6), then we choose an
> > appropriate vseg mem quota, as a result HAWQ will make decision to
> leverage
> > more concurrency and more resource for big queries automatically.  I mean
> > HAWQ will choose the number of virtual segments automatically, not always
> > high number of virtual segments.
> >
> > For scenario 3, It is recommended not to assign all system memory to HAWQ
> > in the segment side. It is a good idea to leave some for system and for
> the
> > other applications.
> >
> > I think it is fine to support higher memory limit for one virtual
> segment,
> > but still I have no idea why it is necessary.
> >
> > As HAWQ uses different memory allocation strategy comparing with
> Greenplum,
> > it is true, the external behavior of HAWQ is different. Virtual segment
> > (VSEG) is a critical concept in HAWQ to manage resource consumption and
> > query QE concurrency, in fact for Greenplum, the QE concurrency is fixed,
> > i.e. the number of QEs for one query execution is fixed, this is why
> > Greenplum user just needs to think of memory consumption in one segment,
> as
> > all segments consume the same amount of memory for the same query. While
> > back to HAWQ, this number is dynamic, you can easily change resource
> queue
> > definition or statement resource definition to change it, you don't have
> to
> > install and config more segments in those nodes to force the QE
> > concurrency.
> >
> > Consider a scenario that there are a lot of small scale queries
> > concurrently arrived, HAWQ just allocate a small number of vsegs for each
> > query without resource waste and this makes HAWQ able to support high
> > concurrency much better than traditional MPP-architecture database to
> > achieve high throughput. Opposite, for those large scale queries, HAWQ
> can
> > automatically increase number of vseg to a very high number if you set

Re: build fail on arch linux

[Yi Jin]

Yes, I met similar issue, Bison 3.x does not work for HAWQ now.

On Mon, Jan 30, 2017 at 12:37 PM, Dmitry Bouzolin <
dbouzo...@yahoo.com.invalid> wrote:

> Hi Lei,
> I use Bison 3.0.2. And looks actually like a bug in gram.c source for this
> Bison version.The function refers yyscanner which is not defined. I will
> reach out Bison bug list.Thanks for reply!
>
> On Sunday, January 29, 2017 8:09 PM, Lei Chang 
> wrote:
>
>
>  Hi Dmitry,
>
> Which bison version do you use? Looks this is a known issue when compiling
> hawq on latest bison (3.x) version.  Bison 2.x version should work.
>
> Thanks
> Lei
>
>
>
>
> On Mon, Jan 30, 2017 at 3:41 AM, Dmitry Bouzolin <
> dbouzo...@yahoo.com.invalid> wrote:
>
> > Hi All,
> > Yes, I know arch linux is not supported, however I appreciate any clues
> on
> > why the build would fail like so:
> >
> > make -C caql allmake[4]: Entering directory
> '/data/src/incubator-hawq/src/
> > backend/catalog/caql'
> > gcc -O3 -std=gnu99  -Wall -Wmissing-prototypes -Wpointer-arith
> > -Wendif-labels -Wformat-security -fno-strict-aliasing -fwrapv
> > -fno-aggressive-loop-optimizations  -I/usr/include/libxml2
> > -I../../../../src/include -D_GNU_SOURCE  -I/data/src/incubator-hawq/
> > depends/libhdfs3/build/install/opt/hawq/include
> > -I/data/src/incubator-hawq/depends/libyarn/build/install/
> opt/hawq/include
> > -c -o gram.o gram.c
> > gram.c: In function ‘caql_yyparse’:
> > gram.c:1368:41: error: ‘yyscanner’ undeclared (first use in this
> function)
> >yychar = yylex (, , yyscanner);
> >  ^
> > gram.c:1368:41: note: each undeclared identifier is reported only once
> for
> > each function it appears in
> > : recipe for target 'gram.o' failed
> >
> > If I build on CentOS, I get different make like for this target and build
> > succeeds:
> > make -C caql all
> > make[4]: Entering directory `/data/src/incubator-hawq/src/
> > backend/catalog/caql'
> > gcc -O3 -std=gnu99  -Wall -Wmissing-prototypes -Wpointer-arith
> > -Wendif-labels -Wformat-security -fno-strict-aliasing -fwrapv
> > -fno-aggressive-loop-optimizations  -I/usr/include/libxml2
> > -I../../../../src/include -D_GNU_SOURCE  -I/data/src/incubator-hawq/
> > depends/libhdfs3/build/install/opt/hawq/include
> > -I/data/src/incubator-hawq/depends/libyarn/build/install/
> opt/hawq/include
> > -c -o caqlanalyze.o caqlanalyze.c
> >
> > The difference is in input and output file. The same line in Arch
> > completes successfully. All dependencies are in place.
> >
> > Thanks, Dmitry.
> >
>
>
>

Re: hawq_rm_stmt_vseg_memory and hawq_rm_nvseg_perquery_perseg_limit

[Yi Jin]

Hi Jon,

For scenario 1, it is true. However, do you think it necessary to make one
vseg having so high memory quota setting? Of course, I think if we force
hawq_rm_stmt_nvseg a value different with bucket number, optimizer will
give different plan. This is way to force high concurrency and high
resource occupation.

For scenario 2, I think if we want to release more concurrency in executing
one query, we should advice customer to
increase hawq_rm_nvseg_perquery_limit (default 512)
and hawq_rm_nvseg_perquery_perseg_limit(default 6), then we choose an
appropriate vseg mem quota, as a result HAWQ will make decision to leverage
more concurrency and more resource for big queries automatically.  I mean
HAWQ will choose the number of virtual segments automatically, not always
high number of virtual segments.

For scenario 3, It is recommended not to assign all system memory to HAWQ
in the segment side. It is a good idea to leave some for system and for the
other applications.

I think it is fine to support higher memory limit for one virtual segment,
but still I have no idea why it is necessary.

As HAWQ uses different memory allocation strategy comparing with Greenplum,
it is true, the external behavior of HAWQ is different. Virtual segment
(VSEG) is a critical concept in HAWQ to manage resource consumption and
query QE concurrency, in fact for Greenplum, the QE concurrency is fixed,
i.e. the number of QEs for one query execution is fixed, this is why
Greenplum user just needs to think of memory consumption in one segment, as
all segments consume the same amount of memory for the same query. While
back to HAWQ, this number is dynamic, you can easily change resource queue
definition or statement resource definition to change it, you don't have to
install and config more segments in those nodes to force the QE
concurrency.

Consider a scenario that there are a lot of small scale queries
concurrently arrived, HAWQ just allocate a small number of vsegs for each
query without resource waste and this makes HAWQ able to support high
concurrency much better than traditional MPP-architecture database to
achieve high throughput. Opposite, for those large scale queries, HAWQ can
automatically increase number of vseg to a very high number if you set
those limit gucs high as well.

Best,
Yi

On Mon, Jan 23, 2017 at 12:30 PM, Jon Roberts <jrobe...@pivotal.io> wrote:

> I've been thinking about these scenarios:
>
> 1.  Hash distributed tables with fixed number of buckets.
> If the tables were built using the defaults, buckets = 6 * number of
> nodes.  So you basically have 6 vsegs per host.  Multiply that by 16GB and
> you only can use 96GB of the 256GB of RAM per node.
>
> 2.  A user has random tables but doesn't understand they can increase the
> number of vsegs.
> This will be common for users that come from Greenplum.  They again can
> only set statement member to 16GB so they are stuck with a max of 96GB of
> RAM usage.
>
> 3.  User increases vsegs and statement memory.
> Possibly run out of memory if too aggressive with settings.
>
> - I think we should be able to specify statement memory higher than 16GB.
> Maybe the limit should be something much higher such as 1TB.
>
> - The optimizer should limit the number of vsegs based on statement memory
> setting to prevent OOM.  You could do the opposite too.  (limit memory and
> use the vseg setting provided)  Greenplum can limit the amount of memory
> but we have two dials to adjust with vsegs and memory.
>
>
> Jon Roberts
>
> On Sun, Jan 22, 2017 at 5:20 PM, Yi Jin <y...@pivotal.io> wrote:
>
> > Hi Jon,
> >
> > That guc setting limit means for one virtual segment, the maximum
> > consumable memory is 16GB, for one segment/node, there maybe multiple
> vsegs
> > allocated to run queries, so if a node has 256gb expected to be consumed
> by
> > HAWQ, it will have at most 16 vsegs running concurrently.
> >
> > hawq_rm_stmt_vseg_memory is for setting statement level vseg memory
> > consumption, it is required to specify hawq_rm_stmt_nvseg as well, only
> > when hawq_rm_stmt_nvseg is greater than 0, hawq_rm_stmt_vseg_memory is
> > activated regardless the original target resource queue vseg resource
> quota
> > definition. For example, you can set hawq_rm_stmt_vseg_memory as 16gb,
> > hawq_rm_stmt_nvseg
> > as 256, if you have a cluster having 256gb * 16 nodes and your target
> > resource queue can use 100% cluster resource, you will have 16 vsegs
> > running per node to consume all memory resource for this query.
> >
> > Best,
> > Yi
> >
> > On Sat, Jan 21, 2017 at 3:40 PM, Lei Chang <lei_ch...@apache.org> wrote:
> >
> > > hawq_rm_stmt_vseg_memory and hawq_rm_stmt_nvseg need to be used
> together
> > to
> >

Re: hawq_rm_stmt_vseg_memory and hawq_rm_nvseg_perquery_perseg_limit

[Yi Jin]

Hi Jon,

That guc setting limit means for one virtual segment, the maximum
consumable memory is 16GB, for one segment/node, there maybe multiple vsegs
allocated to run queries, so if a node has 256gb expected to be consumed by
HAWQ, it will have at most 16 vsegs running concurrently.

hawq_rm_stmt_vseg_memory is for setting statement level vseg memory
consumption, it is required to specify hawq_rm_stmt_nvseg as well, only
when hawq_rm_stmt_nvseg is greater than 0, hawq_rm_stmt_vseg_memory is
activated regardless the original target resource queue vseg resource quota
definition. For example, you can set hawq_rm_stmt_vseg_memory as 16gb,
hawq_rm_stmt_nvseg
as 256, if you have a cluster having 256gb * 16 nodes and your target
resource queue can use 100% cluster resource, you will have 16 vsegs
running per node to consume all memory resource for this query.

Best,
Yi

On Sat, Jan 21, 2017 at 3:40 PM, Lei Chang  wrote:

> hawq_rm_stmt_vseg_memory and hawq_rm_stmt_nvseg need to be used together to
> set the specific number of segments and the vseg memory. And
> hawq_rm_stmt_nvseg should be less than hawq_rm_nvseg_perquery_perseg_
> limit.
>
> set hawq_rm_stmt_vseg_memory = '2GB';set hawq_rm_stmt_nvseg = 6;
>
> looks 16GB is somewhat small for big dedicated machines: if 16GB is per
> virtual segment memory, if 8 segment is used, it only use 128GB.
>
> Cheers
> Lei
>
>
> On Fri, Jan 20, 2017 at 9:11 PM, Jon Roberts  wrote:
>
> > Why is there a limit of 16GB for hawq_rm_stmt_vseg_memory?  A cluster
> with
> > 256GB per node and dedicated for HAWQ may certainly want to utilize more
> > memory per segment.  Is there something I'm missing regarding statement
> > memory?
> >
> > Secondly, does the number of vsegs for a query get influenced by the
> > statement memory or does it just look at the plan and
> > hawq_rm_nvseg_perquery_perseg_limit?
> >
> >
> > Jon Roberts
> >
>

[jira] [Created] (HAWQ-1285) resource manager outputs uninitialized string as host name

[Yi Jin (JIRA)]

Yi Jin created HAWQ-1285:


 Summary: resource manager outputs uninitialized string as host name
 Key: HAWQ-1285
 URL: https://issues.apache.org/jira/browse/HAWQ-1285
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Resource Manager
Reporter: Yi Jin
Assignee: Ed Espino
 Fix For: 2.2.0.0-incubating


In yarn mode, when the host is not registered yet, RM will generate the 
following log message which makes log unable to read to identify which host is 
being processed.

2017-01-11 10:57:37.265071 
GMT,,,p48567,th5266578240,con4,,seg-1,"LOG","0","Resource 
manager adjusts segment ~@ original global resource manager resource capacity 
from (188416 MB, 50 CORE) to (188416 MB, 46 
CORE)",,,0,,"resourcepool.c",4700,



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [VOTE]: Apache HAWQ 2.1.0.0-incubating Release

[Yi Jin]

+1

Built, installed, init cluster and stop/start cluster.

Yi

On Thu, Jan 12, 2017 at 9:27 PM, Ming Li  wrote:

> Built, installed, init cluster successfully.
> But 2 failures when running 'make installcheck-good':
>
> -
> test errortbl ... FAILED (5.19 sec)
> test hcatalog_lookup  ... FAILED (16.60 sec)
>
>
>
> On Thu, Jan 12, 2017 at 5:52 PM, Hubert Zhang  wrote:
>
> > Built and installed successfully!
> > +1
> >
> > On Thu, Jan 12, 2017 at 3:13 AM, Hong  wrote:
> >
> > > Build succeeded on new MBP, RAT check passed.
> > > +1 on this great release! Good job!
> > >
> > > Regards,
> > > Hong
> > >
> > > 2017-01-10 22:16 GMT-05:00 Ed Espino :
> > >
> > > > This is the vote for 2.1.0.0-incubating of Apache HAWQ (incubating).
> > > This
> > > > is a Source only release.
> > > >
> > > > The vote will run for at least 72 hours and will close on Jan 14,
> 2017.
> > > >
> > > > Release Notes (Jira generated):
> > > >
> > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > > > projectId=12318826=12338900
> > > >
> > > > Release verification steps can be found at:
> > > >
> > > > https://cwiki.apache.org/confluence/display/HAWQ/
> > > > Release+Process%3A+Step+by+step+guide#ReleaseProcess:Stepbys
> tepguide-
> > > > ValidatetheReleaseCandidate
> > > >
> > > > Git branch for the release:
> > > > https://github.com/apache/incubator-hawq/tree/2.1.0.0-incubating
> > > > Sources for the release:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/incubator/hawq/2.1.0.
> > > > 0-incubating.RC1/apache-hawq-src-2.1.0.0-incubating.tar.gz
> > > > Source release verification:
> > > > PGP Signature:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/incubator/hawq/2.1.0.
> > > > 0-incubating.RC1/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc
> > > > MD5/SHA256 Hash:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/incubator/hawq/2.1.0.
> > > > 0-incubating.RC1/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5
> > > >
> > > > https://dist.apache.org/repos/dist/dev/incubator/hawq/2.1.0.
> > > > 0-incubating.RC1/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256
> > > > Keys to verify the signature of the release artifact are available
> at:
> > > > https://dist.apache.org/repos/dist/dev/incubator/hawq/KEYS
> > > > The artifact(s) have been signed with Key : 57325522
> > > >
> > > > Please vote accordingly:
> > > > [ ] +1  approve
> > > > [ ] +0  no opinion
> > > > [ ] -1  disapprove (and reason why)
> > > >
> > > >
> > > > --
> > > > *Ed Espino*
> > > > *esp...@apache.org *
> > > >
> > >
> >
> >
> >
> > --
> > Thanks
> >
> > Hubert Zhang
> >
>

[jira] [Created] (HAWQ-1258) segment resource manager does not switch back when it cannot resolve standby host name

[Yi Jin (JIRA)]

Yi Jin created HAWQ-1258:


 Summary: segment resource manager does not switch back when it 
cannot resolve standby host name
 Key: HAWQ-1258
 URL: https://issues.apache.org/jira/browse/HAWQ-1258
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Resource Manager
Reporter: Yi Jin
Assignee: Ed Espino
 Fix For: 2.2.0.0-incubating


When segment resource manager finds master resource manager is not available, 
it should switch to standby server, however, when the standby server cannot be 
resolved, it does not switch back to original master server side. And it also 
should avoid switching to standby if there is no standby which is set as value 
'none'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Enforce "Fix version" when closing a fixed JIRA

[Yi Jin]

+1, which version tag should we use?

On Thu, Dec 29, 2016 at 12:41 AM, Ruilong Huo  wrote:

> Strong +1 on this so that user can have clue on the hawq version which has
> the fix available.
>
> Best regards,
> Ruilong Huo
>
> On Wed, Dec 28, 2016 at 1:23 PM, Paul Guo  wrote:
>
> > I noticed that when I close a JIRA after I fixed it, the version info is
> > not mandatory.
> > I think we should enforce this info. Anyone has any concern on this idea?
> > Thanks.
> >
>

Re: unrecognized configuration parameter "ONETARY"

[Yi Jin]

Jon,

Thank you, yes, I am looking into this bug now.

Best,
Yi

On Thu, Dec 29, 2016 at 2:33 AM, Jon Roberts <jrobe...@pivotal.io> wrote:

> I created a Jira for this bug:
> https://issues.apache.org/jira/browse/HAWQ-1244
>
> Jon Roberts
> Principal Engineer | jrobe...@pivotal.io | 615-426-8661
>
> On Wed, Dec 21, 2016 at 12:58 AM, Ming Li <m...@pivotal.io> wrote:
>
> > Thanks Radar,
> >
> > It is strange that 'unrecognized configuration' will issue FATAL error, I
> > config in hawq-site.xml, and grep pg_log on master and segment date
> > directory, no FATAL error of 'unrecognized configuration'. Any ideas?
> >
> > On Wed, Dec 21, 2016 at 2:45 PM, Radar Da lei <r...@pivotal.io> wrote:
> >
> > > Hi Ming,
> > >
> > > 'hawq_lc_monetary' is only used to store user defined '--lc-monetary'
> > value
> > > then pass below option to initdb:
> > >   '--lc-monetary=${hawq_lc_monetary} '
> > >
> > > So finally we give '--lc-monetary=en_US.utf8 ' to initdb. Base on this
> I
> > > don't think the GUC 'ONETARY' have any dependencies with hawq admin
> > > tools. Thanks.
> > >
> > > Regards,
> > > Radar
> > >
> > > On Tue, Dec 20, 2016 at 10:18 AM, Ming Li <m...@pivotal.io> wrote:
> > >
> > >> Hi Radar,
> > >>
> > >> It seems that the config param 'hawq_lc_monetary' can only be
> recognized
> > >> by hawq admin tools in python, and set hawq guc 'lc-monetary' finally.
> > >>
> > >> My question is: Why not directly using 'lc-monetary' instead of '
> > >> hawq_lc_monetary' in the hawq admin tools? Is there any possible to
> > >> conflict?
> > >>
> > >> Thanks.
> > >>
> > >> On Tue, Dec 20, 2016 at 6:45 AM, Yi Jin <y...@pivotal.io> wrote:
> > >>
> > >>> Thank you jon, I will specially recheck the guc population logic. Yi
> > >>>
> > >>> On Tue, Dec 20, 2016 at 4:07 AM, Jon Roberts <jrobe...@pivotal.io>
> > >>> wrote:
> > >>>
> > >>> > The segment actually has this error:
> > >>> > 2016-12-14 13:47:34.760839
> > >>> > UTC,"gpadmin","gpadmin",p737499,th542214432,"172.21.
> > >>> > 13.196","40327",2016-12-14
> > >>> > 13:47:34 UTC,0,con23798,,seg-1,
> "FATAL","42704","unrecognized
> > >>> > configuration parameter ""ONETARY""",,,0,,"guc.c",10006,
> > >>> >
> > >>> > This made me check out the configs because the error was raised
> from
> > >>> > guc.c.  I found that /usr/local/hawq/etc/_mgmt_config has
> "onetary"
> > >>> text:
> > >>> > hawq_lc_monetary=en_US.utf8
> > >>> >
> > >>> > I'm using nodes with 24 drives so I have 24 temp directories set
> for
> > >>> both
> > >>> > the master and the segments.  Thinking that possibly the problem is
> > >>> related
> > >>> > to the number of vSegs * the size of the config file exceeding a
> > >>> variable
> > >>> > size, I decided to reduce the temp directories down to just two for
> > >>> both
> > >>> > the master and segments.
> > >>> >
> > >>> > After restarting HAWQ, I could get the query to use 16 and even 24
> > >>> vSegs
> > >>> > without a problem.
> > >>> >
> > >>> > So maybe the guc logic needs revisiting to make sure there isn't an
> > >>> issue
> > >>> > with parsing the GUCs when there are many vSegs and many temp
> > >>> directories.
> > >>> > I would think this would be a problem even with the default number
> of
> > >>> vSegs
> > >>> > per host on really large clusters.
> > >>> >
> > >>> >
> > >>> > Jon Roberts
> > >>> > Principal Engineer | jrobe...@pivotal.io | 615-426-8661
> > >>> <(615)%20426-8661>
> > >>> >
> > >>> > On Mon, Dec 19, 2016 at 4:44 AM, Yi Jin <y...@pivotal.io> wrote:
> > >>> >
> > >>> > > Hi Jon,
> > >>> > >
> > >>> > > I think there is no ONTARY configuration item

[jira] [Created] (HAWQ-1242) hawq-site.xml default content has wrong guc variable names

[Yi Jin (JIRA)]

Yi Jin created HAWQ-1242:


 Summary: hawq-site.xml default content has wrong guc variable names
 Key: HAWQ-1242
 URL: https://issues.apache.org/jira/browse/HAWQ-1242
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Core
Reporter: Yi Jin
Assignee: Ed Espino


Reported by Paul Guo

I happened to check the log and found some confusing logs.
Have not looked into this yet. Anyone sawq this before?

2016-12-02 09:28:05.942583
GMT,,,p366111,th4549860160,,,seg-1,"LOG","0","NOTE:
Recognized configuration:
hawq_dfs_url=localhost:8020/hawq_default""processXMLNode","kvproperties.c",134,

..

2016-12-02 09:28:05.948051
GMT,,,p366111,th4549860160,,,seg-1,"LOG","42704","unrecognized
configuration parameter
""hawq_standby_address_host""""set_config_option","guc.c",10041,
2016-12-02 09:28:05.948203
GMT,,,p366111,th4549860160,,,seg-1,"LOG","42704","unrecognized
configuration parameter
""hawq_dfs_url""""set_config_option","guc.c",10041,
2016-12-02 09:28:05.948349
GMT,,,p366111,th4549860160,,,seg-1,"LOG","42704","unrecognized
configuration parameter
""hawq_master_directory""""set_config_option","guc.c",10041,
2016-12-02 09:28:05.948493
GMT,,,p366111,th4549860160,,,seg-1,"LOG","42704","unrecognized
configuration parameter
""hawq_segment_directory""""set_config_option","guc.c",10041,



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: "unrecognized configuration parameter" in masterdd/pg_log/startup.log

[Yi Jin]

I will open a JIRA to fix it by providing correct guc names in
hawq-site.xml file.

Yi

On Wed, Dec 28, 2016 at 3:27 PM, Yi Jin <y...@pivotal.io> wrote:

> Hi Paul,
>
> HAWQ loads hawq-site.xml and set configuration one by one regardless
> whether the guc is valid, so that's why those non-existing guc variables
> are logged out. I think it is from default hawq-site.xml file. Share we
> update the xml file as the fix?
>
> Best,
> Yi
>
> On Sat, Dec 24, 2016 at 12:37 AM, Paul Guo <p...@pivotal.io> wrote:
>
>> Hi yi,
>>
>> It seems that more than 1 people are confused by this behaviour. I just
>> talked with
>>
>> Radar without checking the code details. He said RM will read this and
>> should filter
>>
>> this. Could you verify please? Could we easily fix this?
>>
>>
>> (Previous email seems to be blocked so I fwd and then replied using my
>> company email).
>>
>> Thanks.
>>
>>
>> -- Forwarded message --
>> From: Paul Guo <paul...@gmail.com>
>> Date: 2016-12-03 11:08 GMT+08:00
>> Subject: "unrecognized configuration parameter" in
>> masterdd/pg_log/startup.log
>> To: dev@hawq.incubator.apache.org
>>
>>
>> I happened to check the log and found some confusing logs.
>> Have not looked into this yet. Anyone sawq this before?
>>
>> 2016-12-02 09:28:05.942583 GMT,,,p366111,th4549860160
>> ,,,seg-1,"LOG","0","NOTE: Recognized configuration:
>> hawq_dfs_url=localhost:8020/hawq_default""processXML
>> Node","kvproperties.c",134,
>>
>> ..
>>
>> 2016-12-02 09:28:05.948051 GMT,,,p366111,th4549860160
>> ,,,seg-1,"LOG","42704","unrecognized configuration parameter
>> ""hawq_standby_address_host""""set_config_option","guc.c",10041,
>> 2016-12-02 09:28:05.948203 GMT,,,p366111,th4549860160
>> ,,,seg-1,"LOG","42704","unrecognized configuration parameter
>> ""hawq_dfs_url""""set_config_option","guc.c",10041,
>> 2016-12-02 09:28:05.948349 GMT,,,p366111,th4549860160
>> ,,,seg-1,"LOG","42704","unrecognized configuration parameter
>> ""hawq_master_directory""""set_config_option","guc.c",10041,
>> 2016-12-02 09:28:05.948493 GMT,,,p366111,th4549860160
>> ,,,seg-1,"LOG","42704","unrecognized configuration parameter
>> ""hawq_segment_directory""""set_config_option","guc.c",10041,
>>
>>
>>
>>
>>
>>
>

Re: unrecognized configuration parameter "ONETARY"

[Yi Jin]

Thank you jon, I will specially recheck the guc population logic. Yi

On Tue, Dec 20, 2016 at 4:07 AM, Jon Roberts <jrobe...@pivotal.io> wrote:

> The segment actually has this error:
> 2016-12-14 13:47:34.760839
> UTC,"gpadmin","gpadmin",p737499,th542214432,"172.21.
> 13.196","40327",2016-12-14
> 13:47:34 UTC,0,con23798,,seg-1,"FATAL","42704","unrecognized
> configuration parameter ""ONETARY""",,,0,,"guc.c",10006,
>
> This made me check out the configs because the error was raised from
> guc.c.  I found that /usr/local/hawq/etc/_mgmt_config has "onetary" text:
> hawq_lc_monetary=en_US.utf8
>
> I'm using nodes with 24 drives so I have 24 temp directories set for both
> the master and the segments.  Thinking that possibly the problem is related
> to the number of vSegs * the size of the config file exceeding a variable
> size, I decided to reduce the temp directories down to just two for both
> the master and segments.
>
> After restarting HAWQ, I could get the query to use 16 and even 24 vSegs
> without a problem.
>
> So maybe the guc logic needs revisiting to make sure there isn't an issue
> with parsing the GUCs when there are many vSegs and many temp directories.
> I would think this would be a problem even with the default number of vSegs
> per host on really large clusters.
>
>
> Jon Roberts
> Principal Engineer | jrobe...@pivotal.io | 615-426-8661
>
> On Mon, Dec 19, 2016 at 4:44 AM, Yi Jin <y...@pivotal.io> wrote:
>
> > Hi Jon,
> >
> > I think there is no ONTARY configuration item, so maybe I need the full
> log
> > containing that error to check the error routine.
> >
> > Best,
> > Yi
> >
> > On Mon, Dec 19, 2016 at 5:39 PM, Paul Guo <paul...@gmail.com> wrote:
> >
> > > You could grep the log to see whether there is "ONETARY" setting before
> > > this error occurs, and also grep configuration files and related test
> > files
> > > to find who tried to set this.
> > >
> > > 2016-12-14 22:36 GMT+08:00 Jon Roberts <jrobe...@pivotal.io>:
> > >
> > > > I'm getting the error message: unrecognized configuration parameter
> > > > "ONETARY" with a few of the TPC-DS queries where I'm increasing the
> > > number
> > > > of vsegs to get better performance.  The error message alone is
> > confusing
> > > > so even if I am doing something wrong, the message should be
> improved.
> > > >
> > > > My environment:
> > > >
> > > > AWS d2.8xlarge nodes
> > > > - 24 2 TB disks
> > > > - 252 GB RAM
> > > > - 36 cores
> > > > - Centos 6
> > > > - 10 nodes
> > > > - 1 admin node
> > > > - 10GB network
> > > > - 7 TB of data
> > > > - Standard TPC-DS Queries
> > > > - hawq_rm_memory_limit_perseg = 200gb
> > > > - hawq_rm_stmt_vseg_memory = 16gb
> > > > - Random distribution on all tables
> > > >
> > > > I'm tried reducing the statement memory but that doesn't change
> > anything.
> > > >
> > > > Query 88 is a good example of this because it fails quickly.
> > > >
> > > > set hawq_rm_nvseg_perquery_perseg_limit=12;
> > > >
> > > > time psql -f 188.tpcds.88.sql
> > > > SET
> > > > Timing is on.
> > > > SET
> > > > Time: 0.157 ms
> > > >  h8_30_to_9 | h9_to_9_30 | h9_30_to_10 | h10_to_10_30 | h10_30_to_11
> |
> > > > h11_to_11_30 | h11_30_to_12 | h12_to_12_30
> > > > ++-+--+-
> > > > -+--+--+--
> > > >16279055 |   32496701 |32493080 | 48732586 | 48782652
> |
> > > > 28460584 | 28453299 | 32518016
> > > > (1 row)
> > > >
> > > > Time: 259695.969 ms
> > > >
> > > > real 4m19.706s
> > > > user 0m0.001s
> > > > sys 0m0.003s
> > > >
> > > > Next:
> > > > set hawq_rm_nvseg_perquery_perseg_limit=14;
> > > >
> > > > time psql -f 188.tpcds.88.sql
> > > > SET
> > > > Timing is on.
> > > > SET
> > > > Time: 0.171 ms
> > > > psql:188.tpcds.88.sql:95: ERROR:  Error dispatching to seg25
> > > > ip-172-21-13-189.ec2.internal:4: connection pointer is NULL
> > > > DETAIL:  Master unable to connect to seg25
> > > > ip-172-21-13-189.ec2.internal:4: FATAL:  unrecognized
> > configuration
> > > > parameter "ONETARY"
> > > >
> > > > real 0m8.787s
> > > > user 0m0.003s
> > > > sys 0m0.002s
> > > >
> > > >
> > > > Jon Roberts
> > > >
> > >
> >
>

Re: unrecognized configuration parameter "ONETARY"

[Yi Jin]

Hi Jon,

I think there is no ONTARY configuration item, so maybe I need the full log
containing that error to check the error routine.

Best,
Yi

On Mon, Dec 19, 2016 at 5:39 PM, Paul Guo  wrote:

> You could grep the log to see whether there is "ONETARY" setting before
> this error occurs, and also grep configuration files and related test files
> to find who tried to set this.
>
> 2016-12-14 22:36 GMT+08:00 Jon Roberts :
>
> > I'm getting the error message: unrecognized configuration parameter
> > "ONETARY" with a few of the TPC-DS queries where I'm increasing the
> number
> > of vsegs to get better performance.  The error message alone is confusing
> > so even if I am doing something wrong, the message should be improved.
> >
> > My environment:
> >
> > AWS d2.8xlarge nodes
> > - 24 2 TB disks
> > - 252 GB RAM
> > - 36 cores
> > - Centos 6
> > - 10 nodes
> > - 1 admin node
> > - 10GB network
> > - 7 TB of data
> > - Standard TPC-DS Queries
> > - hawq_rm_memory_limit_perseg = 200gb
> > - hawq_rm_stmt_vseg_memory = 16gb
> > - Random distribution on all tables
> >
> > I'm tried reducing the statement memory but that doesn't change anything.
> >
> > Query 88 is a good example of this because it fails quickly.
> >
> > set hawq_rm_nvseg_perquery_perseg_limit=12;
> >
> > time psql -f 188.tpcds.88.sql
> > SET
> > Timing is on.
> > SET
> > Time: 0.157 ms
> >  h8_30_to_9 | h9_to_9_30 | h9_30_to_10 | h10_to_10_30 | h10_30_to_11 |
> > h11_to_11_30 | h11_30_to_12 | h12_to_12_30
> > ++-+--+-
> > -+--+--+--
> >16279055 |   32496701 |32493080 | 48732586 | 48782652 |
> > 28460584 | 28453299 | 32518016
> > (1 row)
> >
> > Time: 259695.969 ms
> >
> > real 4m19.706s
> > user 0m0.001s
> > sys 0m0.003s
> >
> > Next:
> > set hawq_rm_nvseg_perquery_perseg_limit=14;
> >
> > time psql -f 188.tpcds.88.sql
> > SET
> > Timing is on.
> > SET
> > Time: 0.171 ms
> > psql:188.tpcds.88.sql:95: ERROR:  Error dispatching to seg25
> > ip-172-21-13-189.ec2.internal:4: connection pointer is NULL
> > DETAIL:  Master unable to connect to seg25
> > ip-172-21-13-189.ec2.internal:4: FATAL:  unrecognized configuration
> > parameter "ONETARY"
> >
> > real 0m8.787s
> > user 0m0.003s
> > sys 0m0.002s
> >
> >
> > Jon Roberts
> >
>

Re: Apache HAWQ release manager volunteer

[Yi Jin]

Thank you Ed! :)

Yi

On Wed, Dec 7, 2016 at 4:18 PM, Ivan Weng  wrote:

> +1. Thanks Ed.
>
>
> Regards,
> Ivan
>
> On Wed, Dec 7, 2016 at 1:06 PM, Wen Lin  wrote:
>
> > +1. Thank you, Ed!
> >
> > On Wed, Dec 7, 2016 at 12:10 PM, Paul Guo  wrote:
> >
> > > +1. Thanks.
> > >
> > > 2016-12-07 11:12 GMT+08:00 Ed Espino :
> > >
> > > > HAWQ dev community,
> > > >
> > > > I am volunteering to be the release manager for the second Apache
> HAWQ
> > > > release.  I will be following the guidelines set forth in the
> project's
> > > > Release Management wiki section as well as the Apache documentation:
> > > >
> > > > o Apache HAWQ wiki Release Management:
> > > > https://cwiki.apache.org/confluence/display/HAWQ/
> > Release+Management
> > > >
> > > > o A Guide To Release Management During Incubation (DRAFT):
> > > > http://incubator.apache.org/guides/releasemanagement.html
> > > >
> > > > NOTE: As needed, I will be updating the project's Release Management
> > wiki
> > > > to reflect updates and filling possible gaps to the release process.
> > > >
> > > > Assuming there are no objections, I will do my best to send a release
> > > plan
> > > > within 48 hours.
> > > >
> > > > Regards,
> > > > -=e
> > > >
> > > > --
> > > > *Ed Espino*
> > > > *esp...@apache.org *
> > > >
> > >
> >
>

[jira] [Created] (HAWQ-1174) double type core counter of container set has precision issue

[Yi Jin (JIRA)]

Yi Jin created HAWQ-1174:


 Summary: double type core counter of container set has precision 
issue
 Key: HAWQ-1174
 URL: https://issues.apache.org/jira/browse/HAWQ-1174
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Resource Manager
Reporter: Yi Jin
Assignee: Lei Chang


This can be reproduced when we have 2 segments, each has 75gb, 15core. cluster 
memory-core ratio is 5gb per core, and then we trigger resource consumption by 
allocating and returning 256mb * 12 vsegs.

(gdb) p *containerset
$6 = {Allocated = {MemoryMB = 5120, Core = 1, Ratio = 0}, Available = {MemoryMB 
= 5120,
Core = 0.99956, Ratio = 0}, Containers = 0x7ff07a93c870}
(gdb) quit
A debugging session is active.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: new committer: Paul Guo

[Yi Jin]

Congratulations! Paul! :)

On Fri, Nov 4, 2016 at 3:48 PM, Lili Ma  wrote:

> Congratulations, Paul!
>
> On Fri, Nov 4, 2016 at 12:08 PM, Hong Wu  wrote:
>
> > Wow! Congrats Paul.
> >
> >
> > > 在 2016年11月4日，上午11:51，Wen Lin  写道：
> > >
> > > Paul,
> > > Congratulations!
> > >
> > >> On Fri, Nov 4, 2016 at 11:34 AM, Ruilong Huo  wrote:
> > >>
> > >> The Project Management Committee (PMC) for Apache HAWQ (incubating)
> has
> > >> invited Paul Guo to become a committer and we are pleased to announce
> > that
> > >> he has accepted.
> > >>
> > >> Being a committer enables easier contribution to the project since
> > there is
> > >> no need to go via the patch submission process. This should enable
> > better
> > >> productivity.
> > >>
> > >> Please join us in congratulating him and we are looking forward to
> > >> collaborating with him in the open source community.
> > >>
> > >> His contribution includes (but not limited to):
> > >>
> > >>   - *Direct contribution to code base*
> > >>  - *56 commits in total which span most of the key components of
> > >>  hawq. This demonstrate concrete knowledge and in depth
> > >> understanding of the
> > >>  product*
> > >>  - 56 closed PRs: https://github.com/apache
> > >>  /incubator-hawq/pulls?q=is%3Apr+is%3Aclosed+author%3Apaul-guo-
> > >>   > >> 3Apr+is%3Aclosed+author%3Apaul-guo->
> > >>  - *9 features, enhancement and code refactor including storage
> and
> > >>  compression, command line tool and management utility,
> procedural
> > >>  language, configure and build system, test infrastructure and
> test,
> > >> etc*
> > >> - HAWQ-774 
> Add
> > >> snappy compression support to row oriented storage
> > >> - HAWQ-984 
> > hawq
> > >> config is too slow
> > >> - HAWQ-775 
> > >> Provide
> > >> a seperate PLR package
> > >> - HAWQ-751 
> Add
> > >> plr, pgcrypto, gporca into Apache HAWQ
> > >> - HAWQ-744 
> Add
> > >> plperl code
> > >> - HAWQ-1007 
> > Add
> > >> the pgcrypto code into hawq
> > >> - HAWQ-394 
> > >> Remove
> > >> pgcrypto from code base
> > >> - HAWQ-914 
> > >> Improve
> > >> user experience of HAWQ's build infrastructure
> > >> - HAWQ-1081 
> > >> Check
> > >> missing perl modules (at least JSON) in configure
> > >> - HAWQ-867 
> > >> Replace
> > >> the git-submobule mechanism with git-clone
> > >> - HAWQ-711 
> > >> Integrate
> > >> libhdfs3 and libyarn makefile into hawq
> > >> - HAWQ-878 
> Add
> > >> googletest cases for the ao snappy compression support
> > >> - HAWQ-876 
> Add
> > >> the support for initFile option of gpdiff.pl in hawq
> googletest
> > >> framework
> > >> - HAWQ-917 
> > >> Refactor
> > >> feature tests for data type check with new googletest
> framework
> > >>  - *34 bug fixes in key components including storage, compression,
> > >>  dispatcher, command line tool, management utilty, configure and
> > build
> > >>  system, and test infrastructure, etc*
> > >> - HAWQ-1097 
> > >> Makefile
> > >> is not friendly to ORCA updating
> > >> - HAWQ-1092 
> > >> lc_collate
> > >> and lc_ctype do not work after setting through hawq init
> > >> - HAWQ-1077 
> > >> Table
> > >> insert hangs due to stack overwrite which is caused by a bug
> > >> in ao snappy
> > >> code
> > >> - HAWQ-1041 
> > >> Some
> > >> "top_srcdir" should be "top_builddir" in makefiles of
> > >> libhdfs3 and libyarn
> > >> - HAWQ-1021 
> > >> Need
> > >> to log for some local_ssh function calls.
> > >> - HAWQ-980 
> > hawq
> > >> does not handle guc value with space properly
> > >> - HAWQ-977 

Re: [VOTE] Apache HAWQ 2.0.0.0-incubating RC4

[Yi Jin]

+1

* Review LICENSE
* Review NOTICE
* Review DISCLAIMER
* Ran basic build and test.

Best,
Yi

On Thu, Sep 29, 2016 at 6:06 PM, Hong Wu  wrote:

> +1 on build, install, license check.
>
> Hong
>
> 2016-09-28 18:01 GMT+08:00 Radar Da lei :
>
> > Ran RAT checks, build, install and ran simple queries. All passed.
> >
> > +1
> >
> > Regards,
> > Radar
> >
> > On Wed, Sep 28, 2016 at 5:58 PM, Paul Guo  wrote:
> >
> > > +1. Built and simply tested on a centos 6.7 system. Also run license
> > check.
> > >
> > > 2016-09-28 2:59 GMT+08:00 Goden Yao :
> > >
> > > > Hi All , just a reminder that we're looking to close the vote by
> Friday
> > > > 12pm.
> > > > Please reply this thread with you opinions after downloading the
> > tarball
> > > > binary.
> > > >
> > > > On Mon, Sep 26, 2016 at 5:17 PM Ed Espino  wrote:
> > > >
> > > > > Goden, That makes sense.  As they can, the community should attempt
> > to
> > > > > build and test and report issues. Primary focus will be on the IP
> > > aspects
> > > > > of the release.
> > > > >
> > > > > Cheers,
> > > > > -=e
> > > > >
> > > > > On Mon, Sep 26, 2016 at 5:12 PM, Goden Yao 
> > > wrote:
> > > > >
> > > > > > Thanks Ed for sharing the details.
> > > > > > Although this is a IP related source tarball only release, it's
> > still
> > > > > > recommended to build the source tarball on your local machine and
> > let
> > > > the
> > > > > > community know if you see any issues (may not be blocker of the
> > > > release).
> > > > > >
> > > > > > On Sun, Sep 25, 2016 at 8:55 PM Ed Espino 
> > wrote:
> > > > > >
> > > > > > > FYI: I will be holding a remote session with the HAWQ team
> based
> > in
> > > > > > Beijing
> > > > > > > to review the basic steps I used for the Apache HAWQ
> > > > 2.0.0.0-incubator
> > > > > > RC4
> > > > > > > review. Needless to say I learned a significant amount over the
> > > past
> > > > > few
> > > > > > > weeks and will be sharing that experience.
> > > > > > >
> > > > > > > -=e
> > > > > > >
> > > > > > > On Sat, Sep 24, 2016 at 11:15 AM, Ed Espino  >
> > > > wrote:
> > > > > > >
> > > > > > > > +1
> > > > > > > >
> > > > > > > > Here are the review items I performed to support my vote:
> > > > > > > >
> > > > > > > > * Review LICENSE (including supporting files in licenses
> > > directory)
> > > > > > > > * Review NOTICE
> > > > > > > > * Review DISCLAIMER
> > > > > > > > * Review pom.xml (Apache RAT configuration)
> > > > > > > > * Checked the source package signature
> > > > > > > > * Successfully ran an Apache RAT check (maven verify) with
> > > results
> > > > > > below
> > > > > > > > (full rat.txt attached for reference):
> > > > > > > > * Review JIRAS related to this release as identified by
> release
> > > > > > manager.
> > > > > > > > All issues for the release have been resolved.
> > > > > > > >  https://cwiki.apache.org/confluence/display/HAWQ/HAWQ+
> > > > > > > > Release+2.0.0.0-incubating)
> > > > > > > >
> > > > > > > > NOTE(s):
> > > > > > > > * As this is an IP focused release, I did not perform a
> build.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > --
> > > > > > > > *Ed Espino*
> > > > > > > > *esp...@apache.org *
> > > > > > > >
> > > > > > > >
> > > > > > > > #
> > > > > > > > APACHE RAT EXECUTION
> > > > > > > > #
> > > > > > > >
> > > > > > > > ✔ ~/workspace/HAWQ-projects/apache-hawq-src-2.0.0.0-
> incubating
> > > > > > > > 10:58 $ mvn verify
> > > > > > > > [INFO] Scanning for projects...
> > > > > > > > [INFO]
> > > > > > > >
> > > > > > > > [INFO] --
> > > > --
> > > > > > > > 
> > > > > > > > [INFO] Building hawq 2.0
> > > > > > > > [INFO] --
> > > > --
> > > > > > > > 
> > > > > > > > [INFO]
> > > > > > > > [INFO] --- apache-rat-plugin:0.11:check (default) @ hawq ---
> > > > > > > > [INFO] 51 implicit excludes (use -debug for more details).
> > > > > > > > [INFO] Exclude: tools/bin/src/stream/*
> > > > > > > > [INFO] Exclude: tools/bin/pythonSrc/PyGreSQL-4.0/**
> > > > > > > > [INFO] Exclude: tools/bin/pythonSrc/lockfile-0.9.1/**
> > > > > > > > [INFO] Exclude: tools/bin/pythonSrc/PSI-0.3b2_gp/**
> > > > > > > > [INFO] Exclude: tools/bin/ext/simplejson/**
> > > > > > > > [INFO] Exclude: tools/bin/ext/yaml/**
> > > > > > > > [INFO] Exclude: depends/libhdfs3/CMake/**
> > > > > > > > [INFO] Exclude: depends/libyarn/CMake/**
> > > > > > > > [INFO] Exclude: tools/bin/pythonSrc/pychecker-0.8.18/**
> > > > > > > > [INFO] Exclude: tools/bin/pythonSrc/unittest2-0.5.1/**
> > > > > > > > [INFO] Exclude: tools/bin/ext/figleaf/**
> > > > > > > > [INFO] Exclude: tools/bin/ext/pg8000/**
> > > > > > > > [INFO] Exclude: 

Re: Rename "greenplum" to "hawq"

[Yi Jin]

I think it is a must-do, but some concerns of customer using convention and
legacy applications, scripts etc.

On Wed, Jul 13, 2016 at 1:44 PM, 陶征霖  wrote:

> Good idea, but need quite a lot of effort and may also affect custormer
> behavior. Should handle it carefully.
>
> 2016-07-13 9:54 GMT+08:00 Ivan Weng :
>
> > Agree with this good idea. But as Paul said, there are maybe already many
> > users use greeenplum_path.sh or something else in their environment. So
> we
> > need to think about it.
> >
> >
> > Regards,
> > Ivan
> >
> > On Wed, Jul 13, 2016 at 9:31 AM, Paul Guo  wrote:
> >
> > > I've asked this before. Seems that affects some old users. I'm not sure
> > > about the details.
> > > I agree that we should change it to a better name in a release.
> > >
> > > 2016-07-13 9:25 GMT+08:00 Roman Shaposhnik :
> > >
> > > > On Tue, Jul 12, 2016 at 6:21 PM, Xiang Sheng 
> > wrote:
> > > > > Agree . @xunzhang.
> > > > > However , some greenplum strings can be easily replaced , but there
> > are
> > > > too
> > > > > many in the code or comments.  Changing all of them costs too much
> > > > efforts.
> > > > >
> > > > > So changing the strings that users can see is enough.
> > > >
> > > > Huge +1 to this! Btw, is this something we may be able to tackle in
> our
> > > > next Apache release?
> > > >
> > > > Thanks,
> > > > Roman.
> > > >
> > >
> >
>
>
>
> --
> Thanks,
> Zhenglin
>

Re: Question on hawq_rm_nvseg_perquery_limit

[Yi Jin]

Hi Vineet,

Some my comment.

For question 1.
Yes,
perquery_limit is introduced mainly for restrict resource usage in large
scale cluster; perquery_perseg_limit is to avoid allocating too many
processes in one segment, which may cause serious performance issue. So,
two gucs are for different performance aspects. Along with the variation of
cluster scale, one of the two limits actually takes effect. We dont have to
let both active for resource allocation.

For question 2.

In fact, perquery_perseg_limit is a general resource restriction for all
queries not only hash table queries and external table queries, this is why
this guc is not merged with another one. For example, when we run some
queries upon random distributed tables, it does not make sense to let
resource manager refer a guc for hash table.

For the last topic item.

In my opinion, it is not necessary to adjust hawq_rm_nvseg_perquery_limit,
say, we just need to leave it unchanged and actually not active until we
really want to run a large-scale HAWQ cluster, for example, 100+ nodes.

Best,
Yi

On Wed, Jul 13, 2016 at 1:18 PM, Vineet Goel  wrote:

> Hi all,
>
> I’m trying to document some GUC usage in detail and have questions on
> hawq_rm_nvseg_perquery_limit and hawq_rm_nvseg_perquery_perseg_limit
> tuning.
>
> *hawq_rm_nvseg_perquery_limit* = (default value = 512) . Let’s call it
> *perquery_limit* in short.
> *hawq_rm_nvseg_perquery_perseg_limit* (default value = 6) . Let’s call it
> *perquery_perseg_limit* in short.
>
>
> 1) Is there ever any benefit in having perquery_limit *greater than*
> (perquery_perseg_limit * segment host count) ?
> For example in a 10-node cluster, HAWQ will never allocate more than (GUC
> default 6 * 10 =) 60 v-segs, so the perquery_limit default of 512 doesn’t
> have any effect. It seems perquery_limit overrides (takes effect)
> perquery_perseg_limit only when it’s value is less than
> (perquery_perseg_limit * segment host count).
>
> Is that the correct assumption? That would make sense, as users may want to
> keep a check on how much processing a single query can take up (that
> implies that the limit must be lower than the total possible v-segs). Or,
> it may make sense in large clusters (100-nodes or more) where we need to
> limit the pressure on HDFS.
>
>
> 2) Now, if the purpose of hawq_rm_nvseg_perquery_limit is to keep a check
> on single query resource usage (by limiting the # of v-segs), doesn’t if
> affect default_hash_table_bucket_number because queries will fail when
> *default_hash_table_bucket_number* is greater than
> hawq_rm_nvseg_perquery_limit ? In that case, the purpose of
> hawq_rm_nvseg_perquery_limit conflicts with the ability to run queries on
> HASH dist tables. This then means that tuning hawq_rm_nvseg_perquery_limit
> down is not a good idea, which seems conflicting to the purpose of the GUC
> (in relation to other GUC).
>
>
> Perhaps someone can provide some examples on *how and when would you
> tune hawq_rm_nvseg_perquery_limit* in this 10-node example:
>
> *Defaults on a 10-node cluster are:*
> a) *hawq_rm_nvseg_perquery_perseg_limit* = 6 (hence ability to spin up 6 *
> 10 = 60 total v-segs for random tables)
> b) *hawq_rm_nvseg_perquery_limit* = 512 (but HAWQ will never dispatch more
> than 60 v-segs on random table, so value of 512 does not seem practical)
> c) *default_hash_table_bucket_number* = 60 (6 * 10)
>
>
>
> Thanks
> Vineet
>

[jira] [Created] (HAWQ-817) remove deprecated test cases from checkinstall-good

[Yi Jin (JIRA)]

Yi Jin created HAWQ-817:
---

 Summary: remove deprecated test cases from checkinstall-good
 Key: HAWQ-817
 URL: https://issues.apache.org/jira/browse/HAWQ-817
 Project: Apache HAWQ
  Issue Type: New Feature
  Components: Core
Reporter: Yi Jin
Assignee: Lei Chang






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-802) Refactor alter owner installcheck case using new test framework

[Yi Jin (JIRA)]

Yi Jin created HAWQ-802:
---

 Summary: Refactor alter owner installcheck case using new test 
framework
 Key: HAWQ-802
 URL: https://issues.apache.org/jira/browse/HAWQ-802
 Project: Apache HAWQ
  Issue Type: New Feature
  Components: Core
Reporter: Yi Jin
Assignee: Lei Chang






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: HAWQ YARN RPC Errors

[Yi Jin]

Hi Gagan,

I find you code is not up to date, can you update to latest code base and
try again, I am not sure if this is a fixed issue.

And let me and Wen know if it occurs after using the latest version code.
Please also provide us full log file.

Best,
Yi

On Tue, May 17, 2016 at 1:08 PM, Gagan Brahmi  wrote:

> it is a 152kb file.
>
> I have renamed the file as hawq_master_rm_error.txt. Please find it
> attached.
>
>
> Regards,
> Gagan Brahmi
>
> On Mon, May 16, 2016 at 7:59 PM, Wen Lin  wrote:
> > Hi, Gagan,
> >
> > Where is the log? There is no attachment in your email.
> >
> > Thanks!
> >
> > On Sun, May 15, 2016 at 1:24 AM, Gagan Brahmi 
> wrote:
> >
> >> Hi Wen,
> >>
> >> Please find attached logs which has a few instances of the occurrence
> >> of the error.
> >>
> >>
> >> Regards,
> >> Gagan Brahmi
> >>
> >> On Thu, May 12, 2016 at 7:33 PM, Wen Lin  wrote:
> >> > Hi, Gagan,
> >> >
> >> > It seems a sync failure between QD and Resource Manager. Not related
> to
> >> > libyarn 's RPC.
> >> > Would you like to attach the master's log file? Thanks!
> >> >
> >> > On Fri, May 13, 2016 at 12:58 AM, Gagan Brahmi  >
> >> > wrote:
> >> >
> >> >> Hi Team,
> >> >>
> >> >> Do we have some recommended tuning for the RPC warning/errors
> >> >> encountered intermittently?
> >> >>
> >> >> The error which is seen is the following:
> >> >>
> >> >> WARNING:  Sync RPC framework (inet) finds exception raised.
> >> >> ERROR:  failed to return resource to resource manager, failed to
> >> >> receive content (pquery.c:991)
> >> >>
> >> >> This error however, disappears when we retry the query. There are
> >> >> cases when the query is to be retried more than once.
> >> >>
> >> >> The error looks to be invoked when COMM2RM_CLIENT_FAIL_RECV is
> >> encountered.
> >> >>
> >> >> The setup is using YARN resource manager. And the following is the
> >> >> yarn-client configuration used:
> >> >>
> >> >> 
> >> >>
> >> >> 
> >> >>   hadoop.security.authentication
> >> >>   kerberos
> >> >> 
> >> >>
> >> >> 
> >> >>   rpc.client.connect.retry
> >> >>   10
> >> >> 
> >> >>
> >> >> 
> >> >>   rpc.client.connect.tcpnodelay
> >> >>   true
> >> >> 
> >> >>
> >> >> 
> >> >>   rpc.client.connect.timeout
> >> >>   60
> >> >> 
> >> >>
> >> >> 
> >> >>   rpc.client.max.idle
> >> >>   1
> >> >> 
> >> >>
> >> >> 
> >> >>   rpc.client.ping.interval
> >> >>   1
> >> >> 
> >> >>
> >> >> 
> >> >>   rpc.client.read.timeout
> >> >>   360
> >> >> 
> >> >>
> >> >> 
> >> >>   rpc.client.socket.linger.timeout
> >> >>   -1
> >> >> 
> >> >>
> >> >> 
> >> >>   rpc.client.timeout
> >> >>   360
> >> >> 
> >> >>
> >> >> 
> >> >>   rpc.client.write.timeout
> >> >>   360
> >> >> 
> >> >>
> >> >> 
> >> >>   yarn.client.failover.max.attempts
> >> >>   15
> >> >> 
> >> >>
> >> >>   
> >> >>
> >> >> I would appreciate some recommendations.
> >> >>
> >> >>
> >> >> Regards,
> >> >> Gagan Brahmi
> >> >>
> >>
>

Re: [VOTE] HAWQ 2.0.0-beta-incubating RC4

[Yi Jin]

+1

What I have done:
1) Downloaded, deployed and tested the project.
2) Reviewed LICENSE, COPYRIGHT, DISCLAIMER and NOTICE files.

Best,
Yi Jin

On Tue, Jan 26, 2016 at 4:14 PM, Ruilong Huo <r...@pivotal.io> wrote:

> Downloaded, deployed, tested and reviewed it. +1
>
> Best regards,
> Ruilong Huo
>
> On Tue, Jan 26, 2016 at 12:41 PM, Lei Chang <lei_ch...@apache.org> wrote:
>
> > +1 (binding)
> >
> > Cheers
> > Lei
> >
> >
> > On Tue, Jan 26, 2016 at 6:33 AM, Roman Shaposhnik <ro...@shaposhnik.org>
> > wrote:
> >
> > > Lei, can you please explicitly cast +1/-1 vote?
> > >
> > > Thanks,
> > > Roman.
> > >
> > > On Sun, Jan 24, 2016 at 7:50 PM, Lei Chang <lei_ch...@apache.org>
> wrote:
> > > > Look good!
> > > >
> > > > Run mvn verify, get:
> > > >
> > > > *
> > > >
> > > > Summary
> > > >
> > > > ---
> > > >
> > > > Generated at: 2016-01-24T19:50:43-08:00
> > > >
> > > > Notes: 5
> > > >
> > > > Binaries: 0
> > > >
> > > > Archives: 0
> > > >
> > > > Standards: 2627
> > > >
> > > > Apache Licensed: 1796
> > > >
> > > > Generated Documents: 0
> > > >
> > > > JavaDocs are generated and so license header is optional
> > > >
> > > > Generated files do not required license headers
> > > >
> > > > 0 Unknown Licenses
> > > >
> > > > ***
> > > >
> > > > Unapproved licenses:
> > > >
> > > > ***
> > > >
> > > > Cheers
> > > >
> > > > Lei Chang
> > > >
> > > > On Sat, Jan 23, 2016 at 9:14 AM, Roman Shaposhnik <
> > ro...@shaposhnik.org>
> > > > wrote:
> > > >
> > > >> On Fri, Jan 22, 2016 at 11:20 AM, Caleb Welton <cwel...@pivotal.io>
> > > wrote:
> > > >> > First question for me is:
> > > >> > - We have a couple existing jiras on IP clearance, including
> > > >> >   - https://issues.apache.org/jira/browse/HAWQ-184
> > > >> >   - https://issues.apache.org/jira/browse/HAWQ-207
> > > >> >
> > > >> > In particular if HAWQ-184 has *not* been resolved then how are we
> > > clear
> > > >> of
> > > >> > our IP issues?  I see that there was a commit associated with the
> > > issue,
> > > >> so
> > > >> > perhaps this is just a lack of jira hygiene?
> > > >>
> > > >> I wasn't too eager to resolve 184 until the vote passes,
> > > >> as for 207 -- that ended up being fixed in a different way
> > > >> and I resolve it as not applicable anymore.
> > > >>
> > > >> Thanks,
> > > >> Roman.
> > > >>
> > >
> >
>