[jira] [Created] (HAWQ-721) New feature test skeleton
hongwu created HAWQ-721: --- Summary: New feature test skeleton Key: HAWQ-721 URL: https://issues.apache.org/jira/browse/HAWQ-721 Project: Apache HAWQ Issue Type: New Feature Components: Build Reporter: hongwu Assignee: Lei Chang -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-460) thrift_devel library should be provided
[ https://issues.apache.org/jira/browse/HAWQ-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275952#comment-15275952 ] Konstantin Boudnik commented on HAWQ-460: - The main concern is that binaries of the library are coming from who-knows-where. That's the main and only concern > thrift_devel library should be provided > > > Key: HAWQ-460 > URL: https://issues.apache.org/jira/browse/HAWQ-460 > Project: Apache HAWQ > Issue Type: Bug > Components: Core >Affects Versions: 2.0.0-beta-incubating >Reporter: Konstantin Boudnik >Assignee: Lei Chang >Priority: Blocker > Fix For: 2.0.0-beta-incubating > > > As discussed elsewhere, the dependency on native thrift_devel is external > (non-ASF), mandatory and yet is a source one. Let's make sure the library is > either incorporated into HAWQ code, or added as a contrib, or something else. > Because an external mandatory source code dependency doesn't sit well with > Apache release expectations, hence is the blocker for the release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-716) Support code coverage build and report in hawq Makefile
[ https://issues.apache.org/jira/browse/HAWQ-716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275895#comment-15275895 ] ASF GitHub Bot commented on HAWQ-716: - Github user huor commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/636#discussion_r62446449 --- Diff: src/bin/gpfdist/Makefile --- @@ -22,7 +22,8 @@ include $(top_builddir)/src/Makefile.global LIBS := -levent -lyaml -lz -lbz2 -lssl -lcrypto ifeq ($(BUILD_TYPE), gcov) -LIBS := -lgcov $(LIBS) +LIBS := -lgcov $(LIBS) --- End diff -- The indent of LIBS is correct since it is consistent with CFLAGS in next line. > Support code coverage build and report in hawq Makefile > --- > > Key: HAWQ-716 > URL: https://issues.apache.org/jira/browse/HAWQ-716 > Project: Apache HAWQ > Issue Type: Improvement > Components: Build >Reporter: Ruilong Huo >Assignee: Ruilong Huo > > We need to support code coverage build and report in hawq Makefile. The scope > include: > 1. Build hawq/libhdfs3/libyarn binaries with code coverage enabled > 2. Generate unified code coverage report for hawq/libhdfs3/libyarn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-716) Support code coverage build and report in hawq Makefile
[ https://issues.apache.org/jira/browse/HAWQ-716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275893#comment-15275893 ] ASF GitHub Bot commented on HAWQ-716: - Github user xunzhang commented on the pull request: https://github.com/apache/incubator-hawq/pull/636#issuecomment-217767172 LGTM > Support code coverage build and report in hawq Makefile > --- > > Key: HAWQ-716 > URL: https://issues.apache.org/jira/browse/HAWQ-716 > Project: Apache HAWQ > Issue Type: Improvement > Components: Build >Reporter: Ruilong Huo >Assignee: Ruilong Huo > > We need to support code coverage build and report in hawq Makefile. The scope > include: > 1. Build hawq/libhdfs3/libyarn binaries with code coverage enabled > 2. Generate unified code coverage report for hawq/libhdfs3/libyarn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-716) Support code coverage build and report in hawq Makefile
[ https://issues.apache.org/jira/browse/HAWQ-716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275889#comment-15275889 ] ASF GitHub Bot commented on HAWQ-716: - Github user xunzhang commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/636#discussion_r62446101 --- Diff: src/bin/gpfdist/Makefile --- @@ -22,7 +22,8 @@ include $(top_builddir)/src/Makefile.global LIBS := -levent -lyaml -lz -lbz2 -lssl -lcrypto ifeq ($(BUILD_TYPE), gcov) -LIBS := -lgcov $(LIBS) +LIBS := -lgcov $(LIBS) --- End diff -- indent > Support code coverage build and report in hawq Makefile > --- > > Key: HAWQ-716 > URL: https://issues.apache.org/jira/browse/HAWQ-716 > Project: Apache HAWQ > Issue Type: Improvement > Components: Build >Reporter: Ruilong Huo >Assignee: Ruilong Huo > > We need to support code coverage build and report in hawq Makefile. The scope > include: > 1. Build hawq/libhdfs3/libyarn binaries with code coverage enabled > 2. Generate unified code coverage report for hawq/libhdfs3/libyarn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-708) Consume latest GPORCA features up to version 1.630
[ https://issues.apache.org/jira/browse/HAWQ-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275884#comment-15275884 ] ASF GitHub Bot commented on HAWQ-708: - Github user liming01 commented on the pull request: https://github.com/apache/incubator-hawq/pull/631#issuecomment-217765803 Hi @xinzweb, I have merged it, please close this PR. Thanks. > Consume latest GPORCA features up to version 1.630 > -- > > Key: HAWQ-708 > URL: https://issues.apache.org/jira/browse/HAWQ-708 > Project: Apache HAWQ > Issue Type: Improvement > Components: Optimizer >Reporter: xin zhang >Assignee: Amr El-Helw > > There are additional GPORCA features benefiting the HAWQ community, and this > ticket is to bump the GPORCA from version 1.627 to 1.630. Here is list of new > features in GPORCA: > -- 1.628 and rename EatMalloc to EatTracker >Better memory tracking. > -- 1.629 to include new GUC for Join Exploration > New GUC optimizer_join_arity_for_associativity_commutativity is added to > control Join exploration, hence speed up optimization time [#116333899] > For example, `set optimizer_join_arity_for_associativity_commutativity=x` > will > hint at the optimizer to stop exploring join associativity and join > commutativity transformations when an n-ary join operator has more than > x children during optimization, pruning quite a bit of the search space. > -- 1.630 to fix CTE Predicate pushdown > This fix will prevent predicate pushdown with volatile function like > random(), which caused wrong result if predicates were duplicated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-711) Integrate libhdfs3 and libyarn makefile into hawq
[ https://issues.apache.org/jira/browse/HAWQ-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275878#comment-15275878 ] ASF GitHub Bot commented on HAWQ-711: - Github user xunzhang commented on the pull request: https://github.com/apache/incubator-hawq/pull/638#issuecomment-217765195 +1 > Integrate libhdfs3 and libyarn makefile into hawq > - > > Key: HAWQ-711 > URL: https://issues.apache.org/jira/browse/HAWQ-711 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Ming LI >Assignee: Ming LI > > We put source code of libyarn and libhdfs3 into hawq code base(in /depends), > but we still need separate command to build these 2 depend lib before > building hawq. > We need to integrate them into hawq build scripts, so that they will be > automatically built before hawq build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-711) Integrate libhdfs3 and libyarn makefile into hawq
[ https://issues.apache.org/jira/browse/HAWQ-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275877#comment-15275877 ] ASF GitHub Bot commented on HAWQ-711: - Github user huor commented on the pull request: https://github.com/apache/incubator-hawq/pull/638#issuecomment-217765182 +1 > Integrate libhdfs3 and libyarn makefile into hawq > - > > Key: HAWQ-711 > URL: https://issues.apache.org/jira/browse/HAWQ-711 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Ming LI >Assignee: Ming LI > > We put source code of libyarn and libhdfs3 into hawq code base(in /depends), > but we still need separate command to build these 2 depend lib before > building hawq. > We need to integrate them into hawq build scripts, so that they will be > automatically built before hawq build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-711) Integrate libhdfs3 and libyarn makefile into hawq
[ https://issues.apache.org/jira/browse/HAWQ-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275868#comment-15275868 ] ASF GitHub Bot commented on HAWQ-711: - Github user paul-guo- commented on the pull request: https://github.com/apache/incubator-hawq/pull/638#issuecomment-217763593 To Huor, I checked the source code. It seems that it is easy to add some Makefile targets for the issues you mentioned. To better track them, I'd leave them to another issue. To xunzhang, yes, we should do them in configure or make to make the build of libhdfs3 and libyarn transparent to users. Currently doing it in "make pre_config" (dependency of "make all") seems to be better to handle. Maybe I misunderstood your words. Thanks all. > Integrate libhdfs3 and libyarn makefile into hawq > - > > Key: HAWQ-711 > URL: https://issues.apache.org/jira/browse/HAWQ-711 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Ming LI >Assignee: Ming LI > > We put source code of libyarn and libhdfs3 into hawq code base(in /depends), > but we still need separate command to build these 2 depend lib before > building hawq. > We need to integrate them into hawq build scripts, so that they will be > automatically built before hawq build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-711) Integrate libhdfs3 and libyarn makefile into hawq
[ https://issues.apache.org/jira/browse/HAWQ-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275859#comment-15275859 ] ASF GitHub Bot commented on HAWQ-711: - Github user paul-guo- commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/638#discussion_r62444174 --- Diff: configure --- @@ -11461,157 +11427,6 @@ rm -f core conftest.err conftest.$ac_objext \ pgac_args="$pgac_args with_xml_prefix with_xml_exec_prefix enable_xmltest" CFLAGS="$CFLAGS $XML_CPPFLAGS" -# libhdfs3 --- End diff -- We do not build libhdfs3 separately now so we do not need to check the dependency. > Integrate libhdfs3 and libyarn makefile into hawq > - > > Key: HAWQ-711 > URL: https://issues.apache.org/jira/browse/HAWQ-711 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Ming LI >Assignee: Ming LI > > We put source code of libyarn and libhdfs3 into hawq code base(in /depends), > but we still need separate command to build these 2 depend lib before > building hawq. > We need to integrate them into hawq build scripts, so that they will be > automatically built before hawq build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-711) Integrate libhdfs3 and libyarn makefile into hawq
[ https://issues.apache.org/jira/browse/HAWQ-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275834#comment-15275834 ] ASF GitHub Bot commented on HAWQ-711: - Github user xunzhang commented on the pull request: https://github.com/apache/incubator-hawq/pull/638#issuecomment-217758533 I think it is just a workaround solution. HAWQ use autotools but in this pull request, `libyarn` and `libhdfs3` use makefile which is dangerous for portability. I think a better solution is to write a shell/python script which is transparent to user that could build `libyarn` and `libhdfs3` before building `hawq`. > Integrate libhdfs3 and libyarn makefile into hawq > - > > Key: HAWQ-711 > URL: https://issues.apache.org/jira/browse/HAWQ-711 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Ming LI >Assignee: Ming LI > > We put source code of libyarn and libhdfs3 into hawq code base(in /depends), > but we still need separate command to build these 2 depend lib before > building hawq. > We need to integrate them into hawq build scripts, so that they will be > automatically built before hawq build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-711) Integrate libhdfs3 and libyarn makefile into hawq
[ https://issues.apache.org/jira/browse/HAWQ-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275832#comment-15275832 ] ASF GitHub Bot commented on HAWQ-711: - Github user xunzhang commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/638#discussion_r62442543 --- Diff: configure --- @@ -11461,157 +11427,6 @@ rm -f core conftest.err conftest.$ac_objext \ pgac_args="$pgac_args with_xml_prefix with_xml_exec_prefix enable_xmltest" CFLAGS="$CFLAGS $XML_CPPFLAGS" -# libhdfs3 --- End diff -- Why delete configure test for libhdfs3 here? > Integrate libhdfs3 and libyarn makefile into hawq > - > > Key: HAWQ-711 > URL: https://issues.apache.org/jira/browse/HAWQ-711 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Ming LI >Assignee: Ming LI > > We put source code of libyarn and libhdfs3 into hawq code base(in /depends), > but we still need separate command to build these 2 depend lib before > building hawq. > We need to integrate them into hawq build scripts, so that they will be > automatically built before hawq build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (HAWQ-415) Include pygresql source code in code base
[
https://issues.apache.org/jira/browse/HAWQ-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Chang closed HAWQ-415.
--
Resolution: Fixed
> Include pygresql source code in code base
> -
>
> Key: HAWQ-415
> URL: https://issues.apache.org/jira/browse/HAWQ-415
> Project: Apache HAWQ
> Issue Type: Improvement
> Components: Build
>Affects Versions: 2.0.0-beta-incubating
>Reporter: Goden Yao
>Assignee: Lei Chang
> Fix For: 2.0.0-beta-incubating
>
>
> This issue is being discussed on [this
> thread|http://mail-archives.apache.org/mod_mbox/incubator-hawq-dev/201602.mbox/%3C20160209140623.GA21642@tpx%3E].
> We have steps like:
> {code}yum install -y postgresql-devel{code}
> followed by
> {code}yum erase -y postgresql postgresql-libs postgresql-devel{code}
> This is a result of depending on the Python
> pygresql module in the hawq management scripts (written in python).
> Pygresql itself requires postgres (actually HAWQ itself would be fine) in
> order to get the libpq bindings compiled correctly.
> Pygresql library license (http://www.pygresql.org/copyright.html) looks
> compatible with Apache license.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HAWQ-460) thrift_devel library should be provided
[ https://issues.apache.org/jira/browse/HAWQ-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275812#comment-15275812 ] Lei Chang commented on HAWQ-460: [~cos], Looks thrift is from apache (https://thrift.apache.org/). Do you have any other concerns? If no, I will close this JIRA. Thanks! > thrift_devel library should be provided > > > Key: HAWQ-460 > URL: https://issues.apache.org/jira/browse/HAWQ-460 > Project: Apache HAWQ > Issue Type: Bug > Components: Core >Affects Versions: 2.0.0-beta-incubating >Reporter: Konstantin Boudnik >Assignee: Lei Chang >Priority: Blocker > Fix For: 2.0.0-beta-incubating > > > As discussed elsewhere, the dependency on native thrift_devel is external > (non-ASF), mandatory and yet is a source one. Let's make sure the library is > either incorporated into HAWQ code, or added as a contrib, or something else. > Because an external mandatory source code dependency doesn't sit well with > Apache release expectations, hence is the blocker for the release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-394) Remove pgcrypto from code base
[ https://issues.apache.org/jira/browse/HAWQ-394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei Chang updated HAWQ-394: --- Assignee: Paul Guo (was: Lei Chang) > Remove pgcrypto from code base > -- > > Key: HAWQ-394 > URL: https://issues.apache.org/jira/browse/HAWQ-394 > Project: Apache HAWQ > Issue Type: Task > Components: Build >Reporter: Goden Yao >Assignee: Paul Guo >Priority: Blocker > Fix For: 2.0.0-beta-incubating > > > Based on IPMC release review, it's better as reviewed to remove pgcrypto > component from open source code base. > https://github.com/apache/incubator-hawq/tree/master/contrib/pgcrypto -- This message was sent by Atlassian JIRA (v6.3.4#6332)
