[GitHub] incubator-hawq pull request #1195: HAWQ-326. Support RPM package for Apache ...
Github user paul-guo- commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/1195#discussion_r108334685 --- Diff: contrib/hawq-package/build_hawq_rpm.sh --- @@ -0,0 +1,77 @@ +#!/bin/bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +CUR_DIR=$(pwd) + +if [ -f /etc/redhat-release ] ; then +DISTRO_VERSION=$(cat /etc/redhat-release | sed s/.*release\ // | sed s/\ .*//) +DISTRO_MAJOR_VERSION=$(echo ${DISTRO_VERSION} | awk -F '.' '{print $1}') +OS_TYPE="el${DISTRO_MAJOR_VERSION}" +elif [ -f /etc/SuSE-release ] ; then +DISTRO_VERSION=$(cat /etc/SuSE-release | tr "\n" ' ' | sed s/.*=\ //) +DISTRO_MAJOR_VERSION=$(echo ${DISTRO_VERSION} |awk -F '.' '{print $1}') +OS_TYPE="suse${DISTRO_MAJOR_VERSION}" +fi + +HAWQ_RELEASE_VERSION=$(cat ../../getversion| grep ^GP_VERSION | cut -d '=' -f2 | sed 's|"||g' | cut -d '-' -f1) + +echo "HAWQ_RELEASE_VERSION is $HAWQ_RELEASE_VERSION" + +if [ -d rpmbuild ]; then +echo "Clean rpmbuild directory." +rm -rf rpmbuild +fi + +echo "Make directories: rpmbuild/BUILD rpmbuild/RPMS rpmbuild/SOURCES rpmbuild/SPECS rpmbuild/SRPMS" +mkdir -p rpmbuild/BUILD rpmbuild/RPMS rpmbuild/SOURCES rpmbuild/SPECS rpmbuild/SRPMS + +# Copy HAWQ RPM configuration file for the build +cp hawq.spec rpmbuild/SPECS/ + +HAWQ_SOURCE_TARBALL_FILE=apache-hawq-src-${HAWQ_RELEASE_VERSION}-incubating.tar.gz + +# Get PATH where to find HAWQ source code tarball +if [ -z ${HAWQ_SOURCE_TARBALL_PATH} ]; then +HAWQ_SOURCE_TARBALL_PATH=${CUR_DIR} +fi + +# Copy HAWQ source code tarball for rpm build +if [ -f "${HAWQ_SOURCE_TARBALL_PATH}/${HAWQ_SOURCE_TARBALL_FILE}" ]; then +cp ${HAWQ_SOURCE_TARBALL_PATH}/${HAWQ_SOURCE_TARBALL_FILE} rpmbuild/SOURCES/ +else +echo "Can not find ${HAWQ_SOURCE_TARBALL_PATH}/${HAWQ_SOURCE_TARBALL_FILE} " +exit 1 +fi + +pushd rpmbuild > /dev/null +RPM_TOP_DIR=$(pwd) + +echo 'rpmbuild --define "_topdir ${RPM_TOP_DIR}" \ + --define "_hawq_version ${HAWQ_RELEASE_VERSION}" \ + --define "_rpm_os_version ${OS_TYPE}" \ + -bb SPECS/hawq.spec' --- End diff -- You do not need to do like this. You could use the switch: "set -x" and "set +x" --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1195: HAWQ-326. Support RPM package for Apache ...
Github user paul-guo- commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/1195#discussion_r108334385 --- Diff: contrib/hawq-package/build_hawq_rpm.sh --- @@ -0,0 +1,77 @@ +#!/bin/bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +CUR_DIR=$(pwd) + +if [ -f /etc/redhat-release ] ; then +DISTRO_VERSION=$(cat /etc/redhat-release | sed s/.*release\ // | sed s/\ .*//) +DISTRO_MAJOR_VERSION=$(echo ${DISTRO_VERSION} | awk -F '.' '{print $1}') +OS_TYPE="el${DISTRO_MAJOR_VERSION}" +elif [ -f /etc/SuSE-release ] ; then +DISTRO_VERSION=$(cat /etc/SuSE-release | tr "\n" ' ' | sed s/.*=\ //) +DISTRO_MAJOR_VERSION=$(echo ${DISTRO_VERSION} |awk -F '.' '{print $1}') +OS_TYPE="suse${DISTRO_MAJOR_VERSION}" +fi + +HAWQ_RELEASE_VERSION=$(cat ../../getversion| grep ^GP_VERSION | cut -d '=' -f2 | sed 's|"||g' | cut -d '-' -f1) --- End diff -- i.e. I expect the rpm build scripts and spec could be quite independent. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1195: HAWQ-326. Support RPM package for Apache ...
Github user paul-guo- commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/1195#discussion_r108334298 --- Diff: contrib/hawq-package/build_hawq_rpm.sh --- @@ -0,0 +1,77 @@ +#!/bin/bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +CUR_DIR=$(pwd) + +if [ -f /etc/redhat-release ] ; then +DISTRO_VERSION=$(cat /etc/redhat-release | sed s/.*release\ // | sed s/\ .*//) +DISTRO_MAJOR_VERSION=$(echo ${DISTRO_VERSION} | awk -F '.' '{print $1}') +OS_TYPE="el${DISTRO_MAJOR_VERSION}" +elif [ -f /etc/SuSE-release ] ; then +DISTRO_VERSION=$(cat /etc/SuSE-release | tr "\n" ' ' | sed s/.*=\ //) +DISTRO_MAJOR_VERSION=$(echo ${DISTRO_VERSION} |awk -F '.' '{print $1}') +OS_TYPE="suse${DISTRO_MAJOR_VERSION}" +fi + +HAWQ_RELEASE_VERSION=$(cat ../../getversion| grep ^GP_VERSION | cut -d '=' -f2 | sed 's|"||g' | cut -d '-' -f1) --- End diff -- I think we should allow HAWQ_RELEASE_VERSION (and document it) to be predefined (if not predefined then fallback to call getversion) so that users do not need to build a rpm whose version is different than the codebase. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1194: Hawq 1396. Fix the bug when query hcatalog via P...
Github user interma commented on the issue: https://github.com/apache/incubator-hawq/pull/1194 merged --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1194: Hawq 1396. Fix the bug when query hcatalog via P...
Github user linwen commented on the issue: https://github.com/apache/incubator-hawq/pull/1194 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1194: Hawq 1396. Fix the bug when query hcatalog via P...
Github user interma commented on the issue: https://github.com/apache/incubator-hawq/pull/1194 All fixed. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1197: [HAWQ-1415] Set the default_value of JAVA...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-hawq/pull/1197 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1188: HAWQ-1406. Update HAWQ product version st...
Github user huor closed the pull request at: https://github.com/apache/incubator-hawq/pull/1188 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1195: HAWQ-326. Support RPM package for Apache HAWQ.
Github user huor commented on the issue: https://github.com/apache/incubator-hawq/pull/1195 +1 for making the rpm build available for hawq! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (HAWQ-1408) PANICs during COPY ... FROM STDIN
[ https://issues.apache.org/jira/browse/HAWQ-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming LI resolved HAWQ-1408. --- Resolution: Fixed > PANICs during COPY ... FROM STDIN > - > > Key: HAWQ-1408 > URL: https://issues.apache.org/jira/browse/HAWQ-1408 > Project: Apache HAWQ > Issue Type: Bug > Components: Core >Affects Versions: backlog >Reporter: Ming LI >Assignee: Ming LI > Fix For: 2.1.0.0-incubating > > > We found PANIC (and respective core dumps). From the initial analysis from > the logs and core dump, the query causing this PANIC is a "COPY ... FROM > STDIN". This query does not always panic. > This kind of queries are executed from Java/Scala code (by one of IG Spark > Jobs). Connection to the DB is managed by connection pool (commons-dbcp2) and > validated on borrow by “select 1” validation query. IG is using > postgresql-9.4-1206-jdbc41 as a java driver to create those connections. I > believe they should be using the driver from DataDirect, available in PivNet; > however, I haven't found hard evidence pointing the driver as a root cause. > My initial analysis on the packcore for the master PANIC. Not sure if this > helps or makes sense. > This is the backtrace of the packcore for process 466858: > {code} > (gdb) bt > #0 0x7fd875f906ab in raise () from > /data/logs/52280/packcore-core.postgres.466858/lib64/libpthread.so.0 > #1 0x008c0b19 in SafeHandlerForSegvBusIll (postgres_signal_arg=11, > processName=) at elog.c:4519 > #2 > #3 0x0053b9c3 in SetSegnoForWrite (existing_segnos=0x4c46ff0, > existing_segnos@entry=0x0, relid=relid@entry=1195061, > segment_num=segment_num@entry=6, forNewRel=forNewRel@entry=0 '\000', > keepHash=keepHash@entry=1 '\001') at appendonlywriter.c:1166 > #4 0x0053c08f in assignPerRelSegno > (all_relids=all_relids@entry=0x2b96d68, segment_num=6) at > appendonlywriter.c:1212 > #5 0x005f79e8 in DoCopy (stmt=stmt@entry=0x2b2a3d8, > queryString=) at copy.c:1591 > #6 0x007ef737 in ProcessUtility > (parsetree=parsetree@entry=0x2b2a3d8, queryString=0x2c2f550 "COPY > mis_data_ig_client_derived_attributes.client_derived_attributes_src (id, > tracking_id, name, value_string, value_timestamp, value_number, > value_boolean, environment, account, channel, device, feat"..., > params=0x0, isTopLevel=isTopLevel@entry=1 '\001', > dest=dest@entry=0x2b2a7c8, completionTag=completionTag@entry=0x7ffcb5e318e0 > "") at utility.c:1076 > #7 0x007ea95e in PortalRunUtility (portal=portal@entry=0x2b8eab0, > utilityStmt=utilityStmt@entry=0x2b2a3d8, isTopLevel=isTopLevel@entry=1 > '\001', dest=dest@entry=0x2b2a7c8, > completionTag=completionTag@entry=0x7ffcb5e318e0 "") at pquery.c:1969 > #8 0x007ec13e in PortalRunMulti (portal=portal@entry=0x2b8eab0, > isTopLevel=isTopLevel@entry=1 '\001', dest=dest@entry=0x2b2a7c8, > altdest=altdest@entry=0x2b2a7c8, > completionTag=completionTag@entry=0x7ffcb5e318e0 "") at pquery.c:2079 > #9 0x007ede95 in PortalRun (portal=portal@entry=0x2b8eab0, > count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=1 '\001', > dest=dest@entry=0x2b2a7c8, altdest=altdest@entry=0x2b2a7c8, > completionTag=completionTag@entry=0x7ffcb5e318e0 "") at pquery.c:1596 > #10 0x007e5ad9 in exec_simple_query > (query_string=query_string@entry=0x2b29100 "COPY > mis_data_ig_client_derived_attributes.client_derived_attributes_src (id, > tracking_id, name, value_string, value_timestamp, value_number, > value_boolean, environment, account, channel, device, feat"..., > seqServerHost=seqServerHost@entry=0x0, > seqServerPort=seqServerPort@entry=-1) at postgres.c:1816 > #11 0x007e6cb2 in PostgresMain (argc=, argv= out>, argv@entry=0x29d7820, username=0x29d75d0 "mis_ig") at postgres.c:4840 > #12 0x00799540 in BackendRun (port=0x29afc50) at postmaster.c:5915 > #13 BackendStartup (port=0x29afc50) at postmaster.c:5484 > #14 ServerLoop () at postmaster.c:2163 > #15 0x0079c309 in PostmasterMain (argc=, > argv=) at postmaster.c:1454 > #16 0x004a4209 in main (argc=9, argv=0x29af010) at main.c:226 > {code} > Jumping into the frame 3 and running info locals, we found something odd for > "status" variable: > {code} > (gdb) f 3 > #3 0x0053b9c3 in SetSegnoForWrite (existing_segnos=0x4c46ff0, > existing_segnos@entry=0x0, relid=relid@entry=1195061, > segment_num=segment_num@entry=6, forNewRel=forNewRel@entry=0 '\000', > keepHash=keepHash@entry=1 '\001') at appendonlywriter.c:1166 > 1166 appendonlywriter.c: No such file or directory. > (gdb) info locals > status = 0x0 > [...] > {code} > This panic comes from this piece of code in "appendonlywritter.c": > {code} > for (int i = 0; i < segment_num; i++) > { > AOSegfileStatus *status =
[jira] [Comment Edited] (HAWQ-1408) PANICs during COPY ... FROM STDIN
[ https://issues.apache.org/jira/browse/HAWQ-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940431#comment-15940431 ] Ming LI edited comment on HAWQ-1408 at 3/28/17 2:20 AM: In case of there is no reproduce steps, I only went through the code to find out possible root cause. Now I get one possible cause. The possible reason is related to https://issues.apache.org/jira/browse/HAWQ-642. (1) When keepHash, we can not guarantee that only generate remaining_num is enough to alloc seg file for all segment_num. Because the next new generated seg file number may have same hash key id with the old one. (2) When !keepHash, now the remaining_num returned from addCandidateSegno() is not precise. So we need to fix it to meet the need of HAWQ-642. (3) At the final call to addCandidateSegno(), we should keep monitoring the remaining_num instead of at the beginning because of the reason (1). So that even the new seg file is not actually used by this query ( maybe for hash key conflict), we can continue to alloc enough seg files for this query. @lilima1 @hubertzhang , please correct me if I am wrong. Thanks. was (Author: mli): In case of there is no reproduce steps, I only went through the code to find out possible root cause. Now I get one possible cause. The possible reason is related https://issues.apache.org/jira/browse/HAWQ-642. (1) When keepHash, we can not guarantee that only generate remaining_num is enough to alloc seg file for all segment_num. Because the next new generated seg file number may have same hash key id with the old one. (2) When !keepHash, now the remaining_num returned from addCandidateSegno() is not precise. So we need to fix it to meet the need of HAWQ-642. (3) At the final call to addCandidateSegno(), we should keep monitoring the remaining_num instead of at the beginning because of the reason (1). So that even the new seg file is not actually used by this query ( maybe for hash key conflict), we can continue to alloc enough seg files for this query. @lilima1 @hubertzhang , please correct me if I am wrong. Thanks. > PANICs during COPY ... FROM STDIN > - > > Key: HAWQ-1408 > URL: https://issues.apache.org/jira/browse/HAWQ-1408 > Project: Apache HAWQ > Issue Type: Bug > Components: Core >Affects Versions: backlog >Reporter: Ming LI >Assignee: Ming LI > Fix For: 2.1.0.0-incubating > > > We found PANIC (and respective core dumps). From the initial analysis from > the logs and core dump, the query causing this PANIC is a "COPY ... FROM > STDIN". This query does not always panic. > This kind of queries are executed from Java/Scala code (by one of IG Spark > Jobs). Connection to the DB is managed by connection pool (commons-dbcp2) and > validated on borrow by “select 1” validation query. IG is using > postgresql-9.4-1206-jdbc41 as a java driver to create those connections. I > believe they should be using the driver from DataDirect, available in PivNet; > however, I haven't found hard evidence pointing the driver as a root cause. > My initial analysis on the packcore for the master PANIC. Not sure if this > helps or makes sense. > This is the backtrace of the packcore for process 466858: > {code} > (gdb) bt > #0 0x7fd875f906ab in raise () from > /data/logs/52280/packcore-core.postgres.466858/lib64/libpthread.so.0 > #1 0x008c0b19 in SafeHandlerForSegvBusIll (postgres_signal_arg=11, > processName=) at elog.c:4519 > #2 > #3 0x0053b9c3 in SetSegnoForWrite (existing_segnos=0x4c46ff0, > existing_segnos@entry=0x0, relid=relid@entry=1195061, > segment_num=segment_num@entry=6, forNewRel=forNewRel@entry=0 '\000', > keepHash=keepHash@entry=1 '\001') at appendonlywriter.c:1166 > #4 0x0053c08f in assignPerRelSegno > (all_relids=all_relids@entry=0x2b96d68, segment_num=6) at > appendonlywriter.c:1212 > #5 0x005f79e8 in DoCopy (stmt=stmt@entry=0x2b2a3d8, > queryString=) at copy.c:1591 > #6 0x007ef737 in ProcessUtility > (parsetree=parsetree@entry=0x2b2a3d8, queryString=0x2c2f550 "COPY > mis_data_ig_client_derived_attributes.client_derived_attributes_src (id, > tracking_id, name, value_string, value_timestamp, value_number, > value_boolean, environment, account, channel, device, feat"..., > params=0x0, isTopLevel=isTopLevel@entry=1 '\001', > dest=dest@entry=0x2b2a7c8, completionTag=completionTag@entry=0x7ffcb5e318e0 > "") at utility.c:1076 > #7 0x007ea95e in PortalRunUtility (portal=portal@entry=0x2b8eab0, > utilityStmt=utilityStmt@entry=0x2b2a3d8, isTopLevel=isTopLevel@entry=1 > '\001', dest=dest@entry=0x2b2a7c8, > completionTag=completionTag@entry=0x7ffcb5e318e0 "") at pquery.c:1969 > #8 0x007ec13e in PortalRunMulti (portal=portal@entry=0x2b8eab0, >
[jira] [Resolved] (HAWQ-1045) Update PXF rpm to include virtual RPM
[ https://issues.apache.org/jira/browse/HAWQ-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr Diachenko resolved HAWQ-1045. --- Resolution: Fixed > Update PXF rpm to include virtual RPM > - > > Key: HAWQ-1045 > URL: https://issues.apache.org/jira/browse/HAWQ-1045 > Project: Apache HAWQ > Issue Type: Task > Components: Build, PXF >Reporter: Goden Yao >Assignee: Shivram Mani > Fix For: 2.2.0.0-incubating > > > This is a requirement from Ambari integration. Side by side installation > scenario to do HAWQ upgrade and verification. > The following rpm names and packaging strategy was agreed upon during the > meeting with HAWQ, HAWQ-Ambari team and [~rvs] > The pxf component rpms will be renamed to include the package version as part > of the package name. > Example: > o pxf-hdf_3_0_1-3.0.1-1088.el6.noarch.rpm > o pxf-json_3_0_1-3.0.1-1088.el6.noarch.rpm > o pxf-hbase_3_0_1-3.0.1-1088.el6.noarch.rpm > o pxf-hive_3_0_1-3.0.1-1088.el6.noarch.rpm > o pxf-service_3_0_1-3.0.1-1088.el6.noarch.rpm (dependency: > apache-tomcat-7.0.62) > PXF virtual RPM by the name pxf will have a direct dependency on the > individual PXF RPMS (pxf-hdfs,pxf-hive,pxf-hbase). Will also be responsible > for creating the symlinks (/usr/lib/pxf and /etc/pxf/conf). > Example: > o pxf-3.0.1.0-1088.el6.noarch.rpm > Dependencies: > o pxf-hdf_3_0_1-3.0.1-1088.el6.noarch.rpm > o pxf-json_3_0_1-3.0.1-1088.el6.noarch.rpm > o pxf-hbase_3_0_1-3.0.1-1088.el6.noarch.rpm > o pxf-hive_3_0_1-3.0.1-1088.el6.noarch.rpm > o pxf-service_3_0_1-3.0.1-1088.el6.noarch.rpm > We will be removing the RPM dependancies on hive and hbase from pxf-hive and > pxf-hbase respectively. > Create and update the necessary rpm spec files to produce the above mentioned > rpms. > The rationale behind including the version in the pxf component rpm names is > to allow both version of the rpms to be installed side by side to felicitate > HAWQ upgrade. In the absence of the package name update, the package will > always remove the artifacts created by the old package. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HAWQ-1314) Post-upgrade pxf_get_item_fields function break
[ https://issues.apache.org/jira/browse/HAWQ-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr Diachenko resolved HAWQ-1314. --- Resolution: Fixed > Post-upgrade pxf_get_item_fields function break > --- > > Key: HAWQ-1314 > URL: https://issues.apache.org/jira/browse/HAWQ-1314 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog, Hcatalog, PXF >Reporter: Gagan Brahmi >Assignee: Oleksandr Diachenko > Fix For: 2.2.0.0-incubating > > > After upgrading from HDB 2.0.0.0 to 2.0.1.0 the pxf_get_item_fields function > stopped working. The following is the message reported: > {code} > postgres=# SELECT * FROM pxf_get_item_fields('Hive','default.hive_table'); > ERROR: function return row and query-specified return row do not match > DETAIL: Returned row contains 5 attributes, but query expects 4. > {code} > The describe table function which uses this function reports the same thing. > {code}\d hcatalog.default.hive_table{code} > The reason being the change in the number of out columns from this function. > Previously the following were the columns reported: > {code} > postgres=# \df pxf_get_item_fields > > List of functions >Schema |Name | Result data type | > Argument data types > | Type > +-+--+--+ > pg_catalog | pxf_get_item_fields | SETOF record | profile text, pattern > text, OUT path text, OUT itemname text, OUT fieldname text, OUT fieldtype > text | normal > (1 row) > {code} > With the newer version we have an extra column > {code} > postgres=# \df pxf_get_item_fields > > List of functions >Schema |Name | Result data type | > Argument data types > | Type > +-+--++ > pg_catalog | pxf_get_item_fields | SETOF record | profile text, pattern > text, OUT path text, OUT itemname text, OUT fieldname text, OUT fieldtype > text, OUT sourcefieldtype text | normal > (1 row) > {code} > The solution to make the describe table and pxf_get_item_fields function > working is to update the pg_proc table under pg_catalog schema. > This issue is a request to either an update in the documentation which > mentions this requirement or improving the upgrade process which will take > care of catalog update to reflect new behavior of the function. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HAWQ-953) Pushed down filter fails when quering partitioned Hive tables
[ https://issues.apache.org/jira/browse/HAWQ-953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr Diachenko resolved HAWQ-953. -- Resolution: Duplicate > Pushed down filter fails when quering partitioned Hive tables > - > > Key: HAWQ-953 > URL: https://issues.apache.org/jira/browse/HAWQ-953 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Oleksandr Diachenko >Assignee: Oleksandr Diachenko > Fix For: 2.2.0.0-incubating > > > After code changes in HAWQ-779 Hawq started sending filters to Hive, which > fails on partitioned tables. > {code} > # \d hive_partitions_all_types > External table "public.hive_partitions_all_types" > Column |Type | Modifiers > +-+--- > t1 | text| > t2 | text| > num1 | integer | > dub1 | double precision| > dec1 | numeric | > tm | timestamp without time zone | > r | real| > bg | bigint | > b | boolean | > tn | smallint| > sml| smallint| > dt | date| > vc1| character varying(5)| > c1 | character(3)| > bin| bytea | > Type: readable > Encoding: UTF8 > Format type: custom > Format options: formatter 'pxfwritable_import' > External location: > pxf://127.0.0.1:51200/hive_many_partitioned_table?PROFILE=Hive > pxfautomation=# SELECT t1, t2, bg FROM hive_partitions_all_types where bg = > 23456789; > ERROR: remote component error (500) from '127.0.0.1:51200': type Exception > report message MetaException(message:Filtering is supported only on > partition keys of type string)description The server encountered an > internal error that prevented it from fulfilling this request.exception > javax.servlet.ServletException: MetaException(message:Filtering is supported > only on partition keys of type string) (libchurl.c:878) > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HAWQ-124) Create Project Maturity Model summary file
[ https://issues.apache.org/jira/browse/HAWQ-124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Espino resolved HAWQ-124. Resolution: Fixed This has been done on the HAWQ wiki: https://cwiki.apache.org/confluence/display/HAWQ/Apache+Project+Maturity+Model > Create Project Maturity Model summary file > -- > > Key: HAWQ-124 > URL: https://issues.apache.org/jira/browse/HAWQ-124 > Project: Apache HAWQ > Issue Type: Improvement > Components: Core >Reporter: Caleb Welton >Assignee: Ed Espino > Fix For: 2.2.0.0-incubating > > > Graduating from an Apache Incubator project requires showing the Apache > Incubator IPMC that we have reached a level of maturity as an incubator > project. One tool that can be used to assess our maturity is the [Apache > Project Maturity Model > Document|https://community.apache.org/apache-way/apache-project-maturity-model.html]. > > I propose we do something similar to what Groovy did and include a Project > Maturity Self assessment in our source code and evaluate ourselves with > respect to project maturity with each of our reports. > To do: > 1. Create a MATURITY.adoc file in our root project directory containing our > self assessment. > See > https://github.com/apache/groovy/blob/67b87a3592f13a6281f5b20081c37a66c80079b9/MATURITY.adoc > as an example document in the Groovy project. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HAWQ-331) Fix HAWQ Jenkins pullrequest build reporting
[ https://issues.apache.org/jira/browse/HAWQ-331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Espino resolved HAWQ-331. Resolution: Fixed This has been fixed in the Apache Jenkins build projects in use by HAWQ. > Fix HAWQ Jenkins pullrequest build reporting > > > Key: HAWQ-331 > URL: https://issues.apache.org/jira/browse/HAWQ-331 > Project: Apache HAWQ > Issue Type: Task > Components: Build >Reporter: Goden Yao >Assignee: Ed Espino > Fix For: 2.2.0.0-incubating > > > https://builds.apache.org/job/HAWQ-build-pullrequest/83/console > It has been recently discovered that Jenkins reports SUCCESS even when a > build was actually failed. > *Acceptance Criteria* > 1. No false SUCCESS when it was a failure > 2. Include Installcheck good, unit tests in the build process -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HAWQ-309) Support Centos/RHEL 7
[ https://issues.apache.org/jira/browse/HAWQ-309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Espino resolved HAWQ-309. Resolution: Fixed This is done in HAWQ 2.2.0.0-incubating release. > Support Centos/RHEL 7 > -- > > Key: HAWQ-309 > URL: https://issues.apache.org/jira/browse/HAWQ-309 > Project: Apache HAWQ > Issue Type: Wish > Components: Build >Reporter: Lei Chang >Assignee: Ed Espino > Fix For: 2.2.0.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (HAWQ-1332) Can not grant database and schema privileges without table privileges in ranger or ranger plugin service
[ https://issues.apache.org/jira/browse/HAWQ-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunling Wang closed HAWQ-1332. --- > Can not grant database and schema privileges without table privileges in > ranger or ranger plugin service > > > Key: HAWQ-1332 > URL: https://issues.apache.org/jira/browse/HAWQ-1332 > Project: Apache HAWQ > Issue Type: Bug > Components: Security >Reporter: Chunling Wang >Assignee: Alexander Denissov > Fix For: 2.2.0.0-incubating > > Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png > > > We try to grant database connect and schema usage privileges to a non-super > user to connect database. We find that if we set policy with database and > schema included, but with table excluded, we can not connect database. But if > we include table, we can connect to database. We think there may be bug in > Ranger Plugin Service or Ranger. Here are steps to reproduce it. > 1. create a new user "usertest1" in database: > {code} > $ psql postgres > psql (8.2.15) > Type "help" for help. > postgres=# CREATE USER usertest1; > NOTICE: resource queue required -- using default resource queue "pg_default" > CREATE ROLE > postgres=# > {code} > 2. add user "usertest1" in pg_hba.conf > {code} > local all usertest1 trust > {code} > 3. set policy with database and schema included, with table excluded > !screenshot-1.png|width=800,height=400! > 4. connect database with user "usertest1" but failed with permission denied > {code} > $ psql postgres -U usertest1 > psql: FATAL: permission denied for database "postgres" > DETAIL: User does not have CONNECT privilege. > {code} > 5. set policy with database, schema and table included > !screenshot-2.png|width=800,height=400! > 6. connect database with user "usertest1" and succeed > {code} > $ psql postgres -U usertest1 > psql (8.2.15) > Type "help" for help. > postgres=# > {code} > But if we do not set table as "*", and specify table like "a", we can not > access database either. > !screenshot-3.png|width=800,height=400! -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HAWQ-1332) Can not grant database and schema privileges without table privileges in ranger or ranger plugin service
[ https://issues.apache.org/jira/browse/HAWQ-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunling Wang resolved HAWQ-1332. - Resolution: Not A Problem > Can not grant database and schema privileges without table privileges in > ranger or ranger plugin service > > > Key: HAWQ-1332 > URL: https://issues.apache.org/jira/browse/HAWQ-1332 > Project: Apache HAWQ > Issue Type: Bug > Components: Security >Reporter: Chunling Wang >Assignee: Alexander Denissov > Fix For: 2.2.0.0-incubating > > Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png > > > We try to grant database connect and schema usage privileges to a non-super > user to connect database. We find that if we set policy with database and > schema included, but with table excluded, we can not connect database. But if > we include table, we can connect to database. We think there may be bug in > Ranger Plugin Service or Ranger. Here are steps to reproduce it. > 1. create a new user "usertest1" in database: > {code} > $ psql postgres > psql (8.2.15) > Type "help" for help. > postgres=# CREATE USER usertest1; > NOTICE: resource queue required -- using default resource queue "pg_default" > CREATE ROLE > postgres=# > {code} > 2. add user "usertest1" in pg_hba.conf > {code} > local all usertest1 trust > {code} > 3. set policy with database and schema included, with table excluded > !screenshot-1.png|width=800,height=400! > 4. connect database with user "usertest1" but failed with permission denied > {code} > $ psql postgres -U usertest1 > psql: FATAL: permission denied for database "postgres" > DETAIL: User does not have CONNECT privilege. > {code} > 5. set policy with database, schema and table included > !screenshot-2.png|width=800,height=400! > 6. connect database with user "usertest1" and succeed > {code} > $ psql postgres -U usertest1 > psql (8.2.15) > Type "help" for help. > postgres=# > {code} > But if we do not set table as "*", and specify table like "a", we can not > access database either. > !screenshot-3.png|width=800,height=400! -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HAWQ-1333) Change access mode of source files for HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-1333: -- Fix Version/s: (was: 2.2.0.0-incubating) 2.3.0.0-incubating > Change access mode of source files for HAWQ > - > > Key: HAWQ-1333 > URL: https://issues.apache.org/jira/browse/HAWQ-1333 > Project: Apache HAWQ > Issue Type: Bug > Components: Tests >Reporter: Amy >Assignee: Amy > Fix For: 2.3.0.0-incubating > > > There are several source files's access mode is 755 in HAWQ, e.g. *.c *.cpp > *.h files. In order to guarantee the security, will change the source files' > access mode to 644. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HAWQ-991) "HAWQ register" could register tables according to .yml configuration file
[ https://issues.apache.org/jira/browse/HAWQ-991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-991: - Fix Version/s: (was: 2.2.0.0-incubating) 2.3.0.0-incubating > "HAWQ register" could register tables according to .yml configuration file > -- > > Key: HAWQ-991 > URL: https://issues.apache.org/jira/browse/HAWQ-991 > Project: Apache HAWQ > Issue Type: New Feature > Components: Command Line Tools >Affects Versions: 2.1.0.0-incubating >Reporter: hongwu >Assignee: hongwu > Fix For: 2.3.0.0-incubating > > > Scenario: > 1. For cluster Disaster Recovery. Two clusters co-exist, periodically import > data from Cluster A to Cluster B. Need Register data to Cluster B. > 2. For the rollback of table. Do checkpoints somewhere, and need to rollback > to previous checkpoint. > Description: > Register according to .yml configuration file. > hawq register [-h hostname] [-p port] [-U username] [-d databasename] [-c > config] [--force][--repair] > Behaviors: > 1. If table doesn't exist, will automatically create the table and register > the files in .yml configuration file. Will use the filesize specified in .yml > to update the catalog table. > 2. If table already exist, and neither --force nor --repair configured. Do > not create any table, and directly register the files specified in .yml file > to the table. Note that if the file is under table directory in HDFS, will > throw error, say, to-be-registered files should not under the table path. > 3. If table already exist, and --force is specified. Will clear all the > catalog contents in pg_aoseg.pg_paqseg_$relid while keep the files on HDFS, > and then re-register all the files to the table. This is for scenario 2. > 4. If table already exist, and --repair is specified. Will change both file > folder and catalog table pg_aoseg.pg_paqseg_$relid to the state which .yml > file configures. Note may some new generated files since the checkpoint may > be deleted here. Also note the all the files in .yml file should all under > the table folder on HDFS. Limitation: Do not support cases for hash table > redistribution, table truncate and table drop. This is for scenario 3. > Requirements: > 1. To be registered file path has to colocate with HAWQ in the same HDFS > cluster. > 2. If to be registered is a hash table, the registered file number should be > one or multiple times or hash table bucket number. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HAWQ-127) Create CI projects for HAWQ releases
[ https://issues.apache.org/jira/browse/HAWQ-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-127: - Fix Version/s: (was: 2.2.0.0-incubating) 2.3.0.0-incubating > Create CI projects for HAWQ releases > > > Key: HAWQ-127 > URL: https://issues.apache.org/jira/browse/HAWQ-127 > Project: Apache HAWQ > Issue Type: Bug > Components: Build >Affects Versions: 2.1.0.0-incubating >Reporter: Lei Chang >Assignee: Radar Lei > Fix For: 2.3.0.0-incubating > > > Create Jenkins projects that build HAWQ binary, source tarballs and docker > images, and run sanity tests including at least installcheck-good tests for > each commit. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HAWQ-127) Create CI projects for HAWQ releases
[ https://issues.apache.org/jira/browse/HAWQ-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-127: - Affects Version/s: 2.1.0.0-incubating > Create CI projects for HAWQ releases > > > Key: HAWQ-127 > URL: https://issues.apache.org/jira/browse/HAWQ-127 > Project: Apache HAWQ > Issue Type: Bug > Components: Build >Affects Versions: 2.1.0.0-incubating >Reporter: Lei Chang >Assignee: Radar Lei > Fix For: 2.3.0.0-incubating > > > Create Jenkins projects that build HAWQ binary, source tarballs and docker > images, and run sanity tests including at least installcheck-good tests for > each commit. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (HAWQ-1406) Update HAWQ product version strings to 2.2.0.0 (HAWQ/HAWQ Ambari Plugin) & 3.2.0.0 (PXF)
[ https://issues.apache.org/jira/browse/HAWQ-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo closed HAWQ-1406. - > Update HAWQ product version strings to 2.2.0.0 (HAWQ/HAWQ Ambari Plugin) & > 3.2.0.0 (PXF) > > > Key: HAWQ-1406 > URL: https://issues.apache.org/jira/browse/HAWQ-1406 > Project: Apache HAWQ > Issue Type: Task > Components: Build >Affects Versions: 2.1.0.0-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.2.0.0-incubating > > > Need to update the HAWQ (2.2.0.0), HAWQ Ambari Plugin (2.2.0.0) and PXF > (3.2.0.0) versions so that we can clearly identify Apache HAWQ > 2.2.0.0-incubating artifacts. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HAWQ-1406) Update HAWQ product version strings to 2.2.0.0 (HAWQ/HAWQ Ambari Plugin) & 3.2.0.0 (PXF)
[ https://issues.apache.org/jira/browse/HAWQ-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo resolved HAWQ-1406. --- Resolution: Fixed > Update HAWQ product version strings to 2.2.0.0 (HAWQ/HAWQ Ambari Plugin) & > 3.2.0.0 (PXF) > > > Key: HAWQ-1406 > URL: https://issues.apache.org/jira/browse/HAWQ-1406 > Project: Apache HAWQ > Issue Type: Task > Components: Build >Affects Versions: 2.1.0.0-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.2.0.0-incubating > > > Need to update the HAWQ (2.2.0.0), HAWQ Ambari Plugin (2.2.0.0) and PXF > (3.2.0.0) versions so that we can clearly identify Apache HAWQ > 2.2.0.0-incubating artifacts. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HAWQ-1310) Reformat resource_negotiator()
[ https://issues.apache.org/jira/browse/HAWQ-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-1310: -- Fix Version/s: (was: 2.2.0.0-incubating) 2.3.0.0-incubating > Reformat resource_negotiator() > -- > > Key: HAWQ-1310 > URL: https://issues.apache.org/jira/browse/HAWQ-1310 > Project: Apache HAWQ > Issue Type: Improvement > Components: Core >Affects Versions: 2.1.0.0-incubating >Reporter: Yi Jin >Assignee: Yi Jin > Fix For: 2.3.0.0-incubating > > > The indents in function resource_negotiator() is not aligned. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HAWQ-1310) Reformat resource_negotiator()
[ https://issues.apache.org/jira/browse/HAWQ-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-1310: -- Affects Version/s: 2.1.0.0-incubating > Reformat resource_negotiator() > -- > > Key: HAWQ-1310 > URL: https://issues.apache.org/jira/browse/HAWQ-1310 > Project: Apache HAWQ > Issue Type: Improvement > Components: Core >Affects Versions: 2.1.0.0-incubating >Reporter: Yi Jin >Assignee: Yi Jin > Fix For: 2.2.0.0-incubating > > > The indents in function resource_negotiator() is not aligned. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HAWQ-1416) hawq_toolkit administrative schema missing in HAWQ installation
Vineet Goel created HAWQ-1416: - Summary: hawq_toolkit administrative schema missing in HAWQ installation Key: HAWQ-1416 URL: https://issues.apache.org/jira/browse/HAWQ-1416 Project: Apache HAWQ Issue Type: Bug Components: Command Line Tools, DDL Reporter: Vineet Goel Assignee: Ed Espino Fix For: 2.3.0.0-incubating hawq_toolkit administrative schema is not pre-installed with HAWQ, but should actually be available once HAWQ is installed and initialized. Current workaround seems to be a manual command to install it: psql -f /usr/local/hawq/share/postgresql/gp_toolkit.sql -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HAWQ-1414) Bump up PXF version to 3.2.1.0
[ https://issues.apache.org/jira/browse/HAWQ-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr Diachenko resolved HAWQ-1414. --- Resolution: Fixed > Bump up PXF version to 3.2.1.0 > -- > > Key: HAWQ-1414 > URL: https://issues.apache.org/jira/browse/HAWQ-1414 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Oleksandr Diachenko >Assignee: Oleksandr Diachenko > Fix For: 2.2.0.0-incubating > > > Update PXF version to 3.2.1.0 for upcoming release. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] incubator-hawq pull request #1196: HAWQ-1414. Bump up PXF version to 3.2.1.0...
Github user sansanichfb closed the pull request at: https://github.com/apache/incubator-hawq/pull/1196 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1197: [HAWQ-1415] Set the default_value of JAVA...
GitHub user ljainpivotalio opened a pull request: https://github.com/apache/incubator-hawq/pull/1197 [HAWQ-1415] Set the default_value of JAVA_HOME for running RPS You can merge this pull request into a Git repository by running: $ git pull https://github.com/ljainpivotalio/incubator-hawq HAWQ-1415 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1197.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1197 commit fe4af61c29d09eb4ace738fc9fea9df64b81622d Author: ljainpivotalioDate: 2017-03-27T23:28:42Z [#142344489] Set the default_value of JAVA_HOME for running RPS --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1179: HAWQ-1395. PXF build script to support pa...
Github user sansanichfb closed the pull request at: https://github.com/apache/incubator-hawq/pull/1179 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (HAWQ-1415) Set JAVA_HOME / JRE_HOME for RPS in the scripts
Alexander Denissov created HAWQ-1415: Summary: Set JAVA_HOME / JRE_HOME for RPS in the scripts Key: HAWQ-1415 URL: https://issues.apache.org/jira/browse/HAWQ-1415 Project: Apache HAWQ Issue Type: Sub-task Components: Security Reporter: Alexander Denissov Assignee: Ed Espino JAVA_HOME or JRE_HOME need to be set for RPS to run. Ambari installs JDK under /usr/jdk64 by default. Until Ambari starts managing RPS, we should do the following: - discover the JDK under /usr/jdk64 - if found, set this value as default value for JAVA_HOME property in rps.properties - export JAVA_HOME in rps.sh If the JDK is not available in /usr/jdk64, warn the user that JAVA_HOME must be setup. The logic can go into enable-ranger-plugin.sh script. Once Ambari starts managing the RPS configuration, the default can be changed to /usr/java/default targeted for non-Ambari installations. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] incubator-hawq pull request #1196: HAWQ-1414. Bump up PXF version to 3.2.1.0...
GitHub user sansanichfb opened a pull request: https://github.com/apache/incubator-hawq/pull/1196 HAWQ-1414. Bump up PXF version to 3.2.1.0. You can merge this pull request into a Git repository by running: $ git pull https://github.com/sansanichfb/incubator-hawq HAWQ-1414 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1196.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1196 commit 9a4ff843f91171aff8c0e8311ff920674aa39cc0 Author: Oleksandr DiachenkoDate: 2017-03-27T21:00:31Z HAWQ-1414. Bump up PXF version to 3.2.1.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (HAWQ-1414) Bump up PXF version to 3.2.1.0
[ https://issues.apache.org/jira/browse/HAWQ-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr Diachenko updated HAWQ-1414: -- Fix Version/s: 2.2.0.0-incubating > Bump up PXF version to 3.2.1.0 > -- > > Key: HAWQ-1414 > URL: https://issues.apache.org/jira/browse/HAWQ-1414 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Oleksandr Diachenko >Assignee: Ed Espino > Fix For: 2.2.0.0-incubating > > > Update PXF version to 3.2.1.0 for upcoming release. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (HAWQ-1414) Bump up PXF version to 3.2.1.0
[ https://issues.apache.org/jira/browse/HAWQ-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr Diachenko reassigned HAWQ-1414: - Assignee: Oleksandr Diachenko (was: Ed Espino) > Bump up PXF version to 3.2.1.0 > -- > > Key: HAWQ-1414 > URL: https://issues.apache.org/jira/browse/HAWQ-1414 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Oleksandr Diachenko >Assignee: Oleksandr Diachenko > Fix For: 2.2.0.0-incubating > > > Update PXF version to 3.2.1.0 for upcoming release. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HAWQ-1414) Bump up PXF version to 3.2.1.0
Oleksandr Diachenko created HAWQ-1414: - Summary: Bump up PXF version to 3.2.1.0 Key: HAWQ-1414 URL: https://issues.apache.org/jira/browse/HAWQ-1414 Project: Apache HAWQ Issue Type: Improvement Components: PXF Reporter: Oleksandr Diachenko Assignee: Ed Espino Update PXF version to 3.2.1.0 for upcoming release. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] incubator-hawq issue #1177: HAWQ-326. Support RPM package for Apache HAWQ.
Github user radarwave commented on the issue: https://github.com/apache/incubator-hawq/pull/1177 This PR is for build HAWQ rpm from binary tarball, since now we chose using source tarball to build rpm packages, so open another PR for the review. New PR link is: https://github.com/apache/incubator-hawq/pull/1195 Will close this PR if no further comments. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1195: HAWQ-326. Support RPM package for Apache HAWQ.
Github user radarwave commented on the issue: https://github.com/apache/incubator-hawq/pull/1195 @shivzone @denalex @huor @paul-guo- @edespino Please help to review, including PXF/Ranger-plugin changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1195: HAWQ-326. Support RPM package for Apache ...
GitHub user radarwave opened a pull request: https://github.com/apache/incubator-hawq/pull/1195 HAWQ-326. Support RPM package for Apache HAWQ. 1. Add a RPM spec file to support building HAWQ rpm package from source code tarball. 2. Add 'make rpm' command to build HAWQ/PXF/Ranger-plugin rpm packages and make into a binary tarball from source code tarball. You can merge this pull request into a Git repository by running: $ git pull https://github.com/radarwave/incubator-hawq rpm-tarball Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1195.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1195 commit 7826a5ff525cb0cf2cfcd201c608d9d215734a37 Author: rleiDate: 2017-03-17T02:36:01Z HAWQ-326. Support RPM package for Apache HAWQ. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1194: Hawq 1396. Fix the bug when query hcatalo...
Github user linwen commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/1194#discussion_r108129930 --- Diff: src/test/feature/Ranger/test_ranger.cpp --- @@ -314,6 +314,35 @@ TEST_F(TestHawqRanger, ResourceIncludeATest) { } } +TEST_F(TestHawqRanger, HcatalogTest) { + SQLUtility util; + if (util.getGUCValue("hawq_acl_type") == "ranger") + { + /* +* create a table in hive and populate some rows +*/ + clearEnv(, "pxf", 1); + string rootPath(util.getTestRootPath()); + string sqlPath = rootPath + "/Ranger/data/testhive.sql"; + auto cmd = hawq::test::stringFormat("hive -f %s", sqlPath.c_str()); + Command::getCommandStatus(cmd); + + /* +* create a user and query this table, fail. +*/ + addUser(, "pxf", 1, false); + runSQLFile(, "pxf", "fail", 1); + + /* +* add an allow policy for this user and query again, succeed. --- End diff -- This line of comment is not accurate. Should be "allow policies" --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1194: Hawq 1396. Fix the bug when query hcatalo...
Github user linwen commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/1194#discussion_r108130047 --- Diff: src/test/feature/Ranger/test_ranger.cpp --- @@ -314,6 +314,35 @@ TEST_F(TestHawqRanger, ResourceIncludeATest) { } } +TEST_F(TestHawqRanger, HcatalogTest) { + SQLUtility util; + if (util.getGUCValue("hawq_acl_type") == "ranger") + { + /* +* create a table in hive and populate some rows +*/ + clearEnv(, "pxf", 1); --- End diff -- should clean env for 2 and 3, added from line:340 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (HAWQ-1166) More friendly error message after hawqregister with original table
[ https://issues.apache.org/jira/browse/HAWQ-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiang Sheng updated HAWQ-1166: -- Fix Version/s: (was: 2.2.0.0-incubating) 2.3.0.0-incubating > More friendly error message after hawqregister with original table > -- > > Key: HAWQ-1166 > URL: https://issues.apache.org/jira/browse/HAWQ-1166 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Command Line Tools >Reporter: hongwu >Assignee: Xiang Sheng > Fix For: 2.3.0.0-incubating > > > After hawqregister, if users access the original table, he will get message > below. Although this is a undefined behavior, should we write some record > into catalog table to return instructive error message in this case? > {code} > ERROR: file open error in file > 'hdfs://localhost:8020/hawq_default/16385/16387/16733/1' for relation 't': No > such file or directory (seg0 localhost:4 pid=43107) > DETAIL: > File does not exist: /hawq_default/16385/16387/16733/1 > at > org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:71) > at > org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:61) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1828) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1799) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1712) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:587) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:365) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2045) > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HAWQ-1147) Analyze will ERROR after register multiple parquet data files to a parquet table while enable debug and cassert in hawq configure
[ https://issues.apache.org/jira/browse/HAWQ-1147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiang Sheng updated HAWQ-1147: -- Fix Version/s: (was: 2.2.0.0-incubating) 2.3.0.0-incubating > Analyze will ERROR after register multiple parquet data files to a parquet > table while enable debug and cassert in hawq configure > - > > Key: HAWQ-1147 > URL: https://issues.apache.org/jira/browse/HAWQ-1147 > Project: Apache HAWQ > Issue Type: Bug > Components: Core >Affects Versions: 2.1.0.0-incubating >Reporter: Xiang Sheng >Assignee: Xiang Sheng > Fix For: 2.3.0.0-incubating > > > Enable debug and cassert in hawq configure, analyze will report Unexpected > internal error after register multiple parquet data files to the table. > reproduce steps: > reproduce steps: > 1. ./configure --enable-debug --enable-cassert > 2. make -j8 > 3. make install > 4. hawq init cluster -a > 5. hadoop fs -mkdir hdfs://localhost:8020/hawq_register_test > 6. hadoop fs -put > $hawq_home/src/test/feature/ManagementTool/test_hawq_register_hawq.paq > hdfs://localhost:8020/hawq_register_test/hawq1.paq > 7. create table t (i int) with (appendonly=true, orientation=parquet); > 8. hawq register -d postgres -f > hdfs://localhost:8020/hawq_register_test/hawq1.paq t > 9. > postgres=# select oid from pg_class where relname = 't'; > oid >+--+ > 24586 > (1 row) > postgres=# select * from pg_aoseg.pg_paqseg_24586; > segno | eof | tupcount | eofuncompressed > ---+-+--+- > 1 | 657 | -1 | -1 > (1 rows) > postgres=#analyze t; > FATAL: Unexpected internal error (analyze.c:1718) > DETAIL: FailedAssertion("!(relTuples > -1.0)", File: "analyze.c", Line: 1718) > HINT: Process 43356 will wait for gp_debug_linger=120 seconds before > termination. > Note that its locks and other resources will not be released until then. > server closed the connection unexpectedly > This probably means the server terminated abnormally > before or while processing the request. > The connection to the server was lost. Attempting reset: Succeeded -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (HAWQ-1367) hawq can access to user tables that have no permission with fallback check table.
[ https://issues.apache.org/jira/browse/HAWQ-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunling Wang closed HAWQ-1367. --- > hawq can access to user tables that have no permission with fallback check > table. > -- > > Key: HAWQ-1367 > URL: https://issues.apache.org/jira/browse/HAWQ-1367 > Project: Apache HAWQ > Issue Type: Bug > Components: Security >Reporter: Xiang Sheng >Assignee: Chunling Wang > Fix For: 2.2.0.0-incubating > > > if a user have access to catalog table and he have no access to user table b. > he can access to table b using "select * from catalog_table, b;" -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HAWQ-1367) hawq can access to user tables that have no permission with fallback check table.
[ https://issues.apache.org/jira/browse/HAWQ-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunling Wang resolved HAWQ-1367. - Resolution: Fixed > hawq can access to user tables that have no permission with fallback check > table. > -- > > Key: HAWQ-1367 > URL: https://issues.apache.org/jira/browse/HAWQ-1367 > Project: Apache HAWQ > Issue Type: Bug > Components: Security >Reporter: Xiang Sheng >Assignee: Chunling Wang > Fix For: 2.2.0.0-incubating > > > if a user have access to catalog table and he have no access to user table b. > he can access to table b using "select * from catalog_table, b;" -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HAWQ-1377) Add more information for Ranger related GUCs in default hawq-site.xml
[ https://issues.apache.org/jira/browse/HAWQ-1377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunling Wang resolved HAWQ-1377. - Resolution: Fixed > Add more information for Ranger related GUCs in default hawq-site.xml > - > > Key: HAWQ-1377 > URL: https://issues.apache.org/jira/browse/HAWQ-1377 > Project: Apache HAWQ > Issue Type: Improvement > Components: Security >Reporter: Chunling Wang >Assignee: Chunling Wang > Fix For: 2.2.0.0-incubating > > > We should add default GUCs for Ranger in sample hawq-site.xml, just as what > resource manager does, so that users don't need to refer to the documents for > detailed GUC names. > The output content should be like follows: > {code} > > > hawq_acl_type > standalone > > > hawq_rps_address_host > localhost > > > hawq_rps_address_suffix > rps > > > hawq_rps_address_port > 8432 > > > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (HAWQ-1377) Add more information for Ranger related GUCs in default hawq-site.xml
[ https://issues.apache.org/jira/browse/HAWQ-1377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunling Wang closed HAWQ-1377. --- > Add more information for Ranger related GUCs in default hawq-site.xml > - > > Key: HAWQ-1377 > URL: https://issues.apache.org/jira/browse/HAWQ-1377 > Project: Apache HAWQ > Issue Type: Improvement > Components: Security >Reporter: Chunling Wang >Assignee: Chunling Wang > Fix For: 2.2.0.0-incubating > > > We should add default GUCs for Ranger in sample hawq-site.xml, just as what > resource manager does, so that users don't need to refer to the documents for > detailed GUC names. > The output content should be like follows: > {code} > > > hawq_acl_type > standalone > > > hawq_rps_address_host > localhost > > > hawq_rps_address_suffix > rps > > > hawq_rps_address_port > 8432 > > > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] incubator-hawq pull request #1193: HAWQ-1406. Update hawq version in pom.xml...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-hawq/pull/1193 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1193: HAWQ-1406. Update hawq version in pom.xml for ha...
Github user radarwave commented on the issue: https://github.com/apache/incubator-hawq/pull/1193 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1194: Hawq 1396. Fix the bug when query hcatalog via P...
Github user interma commented on the issue: https://github.com/apache/incubator-hawq/pull/1194 @linwen @ictmalili @wcl14 @stanlyxiang help to review, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1194: Hawq 1396. Fix the bug when query hcatalo...
GitHub user interma opened a pull request: https://github.com/apache/incubator-hawq/pull/1194 Hawq 1396. Fix the bug when query hcatalog via PXF, add test add hcatalog test You can merge this pull request into a Git repository by running: $ git pull https://github.com/interma/interma-hawq hawq-1396 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1194.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1194 commit 055e5219abdd934db5da9904bc0024858abff14b Author: intermaDate: 2017-03-27T05:44:18Z add hcatalog test case commit 1731879d4781a44e08b0aadb2c5466c980763458 Author: interma Date: 2017-03-27T09:24:46Z fix bug and add ans-files --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1193: HAWQ-1406. Update hawq version in pom.xml...
GitHub user huor opened a pull request: https://github.com/apache/incubator-hawq/pull/1193 HAWQ-1406. Update hawq version in pom.xml for hawq 2.2.0.0 incubating release You can merge this pull request into a Git repository by running: $ git pull https://github.com/huor/incubator-hawq huor_hawq2.2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1193.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1193 commit 4921e92c46733d024629ecf02d7d01709fcd6c2f Author: Ruilong HuoDate: 2017-03-27T09:09:31Z HAWQ-1406. Update hawq version in pom.xml for hawq 2.2.0.0 incubating release --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1192: HAWQ-1412. Fix inconsistent json file for catalo...
Github user paul-guo- commented on the issue: https://github.com/apache/incubator-hawq/pull/1192 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1188: HAWQ-1406. Update HAWQ product version strings f...
Github user paul-guo- commented on the issue: https://github.com/apache/incubator-hawq/pull/1188 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1190: HAWQ-1411. Fix inconsistent json file for catalo...
Github user paul-guo- commented on the issue: https://github.com/apache/incubator-hawq/pull/1190 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1188: HAWQ-1406. Update HAWQ product version strings f...
Github user radarwave commented on the issue: https://github.com/apache/incubator-hawq/pull/1188 LGTM +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1188: HAWQ-1406. Update HAWQ product version strings f...
Github user huor commented on the issue: https://github.com/apache/incubator-hawq/pull/1188 The json file has been updated based on latest code base. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (HAWQ-1413) Generate tools/bin/gppylib/data/*.json automatically
Paul Guo created HAWQ-1413: -- Summary: Generate tools/bin/gppylib/data/*.json automatically Key: HAWQ-1413 URL: https://issues.apache.org/jira/browse/HAWQ-1413 Project: Apache HAWQ Issue Type: Bug Components: Build Reporter: Paul Guo Assignee: Ed Espino The json files below are used for various releases, however maybe we should generate them automatically in Makefile and should not keep them in codebase. 1.1.json 1.2.json 1.3.json 2.0.json 2.1.json -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] incubator-hawq pull request #1192: HAWQ-1412. Fix inconsistent json file for...
GitHub user huor opened a pull request: https://github.com/apache/incubator-hawq/pull/1192 HAWQ-1412. Fix inconsistent json file for catalog of hawq 2.0 The 2.0 json file is created based on hawq 2.0.0.0-incubating code base and instruction at https://cwiki.apache.org/confluence/display/HAWQ/Build+and+Install. ``` cd src/include/catalog/ perl tidycat.pl -dd 2.0.json -df json *.h mv 2.0.json ../../../tools/bin/gppylib/data/2.0.json ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/huor/incubator-hawq huor_hawq2.0 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1192.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1192 commit eeb2be67029bf483af0c0d0d43c2459157eccfa1 Author: Ruilong HuoDate: 2017-03-27T06:46:44Z HAWQ-1412. Fix inconsistent json file for catalog of hawq 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1191: HAWQ-1412. Fix inconsistent json file for...
Github user huor closed the pull request at: https://github.com/apache/incubator-hawq/pull/1191 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1191: HAWQ-1412. Fix inconsistent json file for...
GitHub user huor opened a pull request: https://github.com/apache/incubator-hawq/pull/1191 HAWQ-1412. Fix inconsistent json file for catalog of hawq 2.0 The 2.0 json file is created based on hawq 2.0.0.0-incubating code base and instruction at https://cwiki.apache.org/confluence/display/HAWQ/Build+and+Install. ``` cd src/include/catalog/ perl tidycat.pl -dd 2.0.json -df json *.h mv 2.0.json ../../../tools/bin/gppylib/data/2.0.json ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/huor/incubator-hawq huor_hawq2.0 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1191.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1191 commit 49ceffaffcd742dc9caf1716a91a9cbf6bc1d51e Author: rleiDate: 2016-07-11T02:22:29Z HAWQ-901 Add retries to standby master start check commit fe405ab62957900f35cc478a13b6ad4c45892131 Author: rlei Date: 2016-07-11T08:47:04Z HAWQ-892. Set HAWQ version to 2.0.0.0-incubating commit 727117ff4bc9b892c69b7c29514aa73049ab6949 Author: Goden Yao Date: 2016-07-12T21:25:14Z HAWQ-919. Add missing ASF license header. commit 8083da2867dc9a351809f45746a4c52c271c50fc Author: C.J. Jameson and Devadass Santhosh Sampath Date: 2016-03-22T17:32:03Z HAWQ-537. Accept filenames with spaces commit 7a36744eb1aee9c284b204c6d879a2bb9831a782 Author: Yancheng Luo Date: 2016-06-07T10:09:44Z HAWQ-784. Add tests to hawq register for data types and NULL check. commit 290e94d59f7e24831c1374c689e8c83a1dad5a87 Author: xunzhang Date: 2016-06-16T13:56:56Z HAWQ-806. Add feature test for subplan with new framework commit ca9a4a990e0afe30e492edb1b7d2eee4af63b785 Author: xunzhang Date: 2016-07-09T04:56:22Z HAWQ-807. Refactor feature test for gpcopy with new framework. commit cf7793f0d17299a049a2a61ffa2ed8cedd6194a0 Author: xunzhang Date: 2016-07-09T15:15:00Z HAWQ-911. Optimize and refactor makefiles for feature test framework. commit b1810879b9818271f67d1b23a30a002619e6727f Author: Wen Lin Date: 2016-07-12T02:13:30Z HAWQ-912. Skip temporary Directories checking for master/standby commit 7703959366b4ea4cbb971010ba13acd9bba45675 Author: Paul Guo Date: 2016-07-12T10:27:26Z HAWQ-915. RAT is failing on the latest master commit b0d1615534c6ccc31b45127c7cbbb14325ec339b Author: rlei Date: 2016-07-14T02:40:42Z HAWQ-926. Remove pycrypto from HAWQ source code User need to install it by pip before install HAWQ commit f9608a5329b7142e0135ee50bd05755ce645c37a Author: xunzhang Date: 2016-07-15T10:09:29Z HAWQ-622. Update libhdfs3 readme, clean old strings. commit e308af60bb56b26de57b0c3971388edc0f108faa Author: Goden Yao Date: 2016-07-19T00:04:44Z HAWQ-937. Clean up 'Pivotal Extension Framework' and polish wording commit f05c19ed166850b5b7275cba2ce322ff380d3311 Author: Paul Guo Date: 2016-07-12T06:12:19Z HAWQ-917. Refactor feature tests for data type check with new googletest framework commit 0e6fe7a4f1431ed5d01f0fdc8b48e648869c177e Author: xunzhang Date: 2016-07-12T06:14:08Z HAWQ-808. Refactor feature test for external_oid with new framework. commit a2a79c6da01972bf66f0ef51e0acd69a3196b58d Author: Wen Lin Date: 2016-07-13T01:39:49Z HAWQ-918. Fix memtuple forming bug when null-saved size is larger than 32763 bytes commit e65d463c42081d63b9219d768d8a6eacbf11183d Author: ztao1987 Date: 2016-07-14T02:57:08Z HAWQ-924. Refactor feature test for querycontext with new test framework commit 0081706dcf35c81e5365929e8089f861fd37cd6c Author: YI JIN Date: 2016-07-14T04:55:30Z HAWQ-894. Add feature test for polymorphism with new test framework commit 2e1b093f35faba2511eafcd0a2826ce8a7d5afb9 Author: YI JIN Date: 2016-07-14T08:58:33Z HAWQ-894. Add missed answer file for polymorphism case commit ed443c10a44b956dd288395a0144a8c16004583e Author: YI JIN Date: 2016-07-18T05:30:07Z HAWQ-896. Add feature test for create table with new test framework commit 9bdad43ebbbcefce23db193c3a7dd62ea6a3d805 Author: xunzhang Date: 2016-07-19T01:49:38Z HAWQ-933. Warning fix for gpnetbench. commit 9bc040d92d8c5153b6034e75ebf3a4ea97e31ac7 Author: Roman Shaposhnik Date: 2016-09-06T18:15:38Z Including Crypto NOTICE in the README.md commit 928eade37f42295402986a7fa7759e45ae6278e6
[jira] [Updated] (HAWQ-1412) Inconsistent json file for catalog of hawq 2.0
[ https://issues.apache.org/jira/browse/HAWQ-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-1412: -- Description: To generate catalog information for hawq, we need to make sure the right version of metadata information is used. For hawq 2.2, the tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need to correct that for hawq 2.0. (was: To generate catalog information for hawq, we need to make sure the right version of metadata information is used. For hawq 2.2, the tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need to correct that for hawq 2.1.) > Inconsistent json file for catalog of hawq 2.0 > -- > > Key: HAWQ-1412 > URL: https://issues.apache.org/jira/browse/HAWQ-1412 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Affects Versions: 2.0.0.0-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.0.0.0-incubating > > > To generate catalog information for hawq, we need to make sure the right > version of metadata information is used. For hawq 2.2, the > tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in > [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need > to correct that for hawq 2.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HAWQ-1412) Inconsistent json file for catalog of hawq 2.0
[ https://issues.apache.org/jira/browse/HAWQ-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-1412: -- Affects Version/s: (was: 2.1.0.0-incubating) 2.0.0.0-incubating > Inconsistent json file for catalog of hawq 2.0 > -- > > Key: HAWQ-1412 > URL: https://issues.apache.org/jira/browse/HAWQ-1412 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Affects Versions: 2.0.0.0-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.0.0.0-incubating > > > To generate catalog information for hawq, we need to make sure the right > version of metadata information is used. For hawq 2.2, the > tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in > [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need > to correct that for hawq 2.1. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HAWQ-1412) Inconsistent json file for catalog of hawq 2.0
Ruilong Huo created HAWQ-1412: - Summary: Inconsistent json file for catalog of hawq 2.0 Key: HAWQ-1412 URL: https://issues.apache.org/jira/browse/HAWQ-1412 Project: Apache HAWQ Issue Type: Bug Components: Catalog Affects Versions: 2.1.0.0-incubating Reporter: Ruilong Huo Assignee: Ruilong Huo Fix For: 2.1.0.0-incubating To generate catalog information for hawq, we need to make sure the right version of metadata information is used. For hawq 2.2, the tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need to correct that for hawq 2.1. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HAWQ-1412) Inconsistent json file for catalog of hawq 2.0
[ https://issues.apache.org/jira/browse/HAWQ-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-1412: -- Fix Version/s: (was: 2.1.0.0-incubating) 2.0.0.0-incubating > Inconsistent json file for catalog of hawq 2.0 > -- > > Key: HAWQ-1412 > URL: https://issues.apache.org/jira/browse/HAWQ-1412 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Affects Versions: 2.0.0.0-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.0.0.0-incubating > > > To generate catalog information for hawq, we need to make sure the right > version of metadata information is used. For hawq 2.2, the > tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in > [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need > to correct that for hawq 2.1. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HAWQ-1411) Inconsistent json file for catalog of hawq 2.1
[ https://issues.apache.org/jira/browse/HAWQ-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15942738#comment-15942738 ] Ruilong Huo commented on HAWQ-1411: --- Seems good for catalog improvement. Please create another jira for it so that we can enable it for hawq 2.2 afterwards. For existing hawq 2.0 and 2.1, let's use current solution for now. > Inconsistent json file for catalog of hawq 2.1 > -- > > Key: HAWQ-1411 > URL: https://issues.apache.org/jira/browse/HAWQ-1411 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Affects Versions: 2.1.0.0-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.1.0.0-incubating > > > To generate catalog information for hawq, we need to make sure the right > version of metadata information is used. For hawq 2.2, the > tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in > [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need > to correct that for hawq 2.1. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] incubator-hawq pull request #1190: HAWQ-1411. Fix inconsistent json file for...
GitHub user huor opened a pull request: https://github.com/apache/incubator-hawq/pull/1190 HAWQ-1411. Fix inconsistent json file for catalog of hawq 2.1 You can merge this pull request into a Git repository by running: $ git pull https://github.com/huor/incubator-hawq huor_hawq2.1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1190.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1190 commit a75d01984b568eb43072553bfe5c3f95b7c5dcb4 Author: Ruilong HuoDate: 2017-03-27T06:31:27Z HAWQ-1411. Fix inconsistent json file for catalog of hawq 2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (HAWQ-1411) Inconsistent json file for catalog of hawq 2.1
[ https://issues.apache.org/jira/browse/HAWQ-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-1411: -- Description: To generate catalog information for hawq, we need to make sure the right version of metadata information is used. For hawq 2.2, the tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need to correct that for hawq 2.1. (was: To generate catalog information for hawq, we need to make sure the right version of metadata information is used. For hawq 2.2, the tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need to correct that for hawq 2.0 and 2.1.) > Inconsistent json file for catalog of hawq 2.1 > -- > > Key: HAWQ-1411 > URL: https://issues.apache.org/jira/browse/HAWQ-1411 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Affects Versions: 2.1.0.0-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.1.0.0-incubating > > > To generate catalog information for hawq, we need to make sure the right > version of metadata information is used. For hawq 2.2, the > tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in > [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need > to correct that for hawq 2.1. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HAWQ-1411) Inconsistent json file for catalog of hawq 2.1
[ https://issues.apache.org/jira/browse/HAWQ-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-1411: -- Affects Version/s: 2.1.0.0-incubating > Inconsistent json file for catalog of hawq 2.1 > -- > > Key: HAWQ-1411 > URL: https://issues.apache.org/jira/browse/HAWQ-1411 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Affects Versions: 2.1.0.0-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.1.0.0-incubating > > > To generate catalog information for hawq, we need to make sure the right > version of metadata information is used. For hawq 2.2, the > tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in > [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need > to correct that for hawq 2.0 and 2.1. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HAWQ-1411) Inconsistent json file for catalog of hawq 2.1
[ https://issues.apache.org/jira/browse/HAWQ-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-1411: -- Summary: Inconsistent json file for catalog of hawq 2.1 (was: Inconsistent json file for catalog of hawq 2.0 and 2.1) > Inconsistent json file for catalog of hawq 2.1 > -- > > Key: HAWQ-1411 > URL: https://issues.apache.org/jira/browse/HAWQ-1411 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Affects Versions: 2.1.0.0-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.1.0.0-incubating > > > To generate catalog information for hawq, we need to make sure the right > version of metadata information is used. For hawq 2.2, the > tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in > [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need > to correct that for hawq 2.0 and 2.1. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HAWQ-1411) Inconsistent json file for catalog of hawq 2.1
[ https://issues.apache.org/jira/browse/HAWQ-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15942715#comment-15942715 ] Paul Guo commented on HAWQ-1411: Can we automate this (e.g. generate it automatically based on version?) > Inconsistent json file for catalog of hawq 2.1 > -- > > Key: HAWQ-1411 > URL: https://issues.apache.org/jira/browse/HAWQ-1411 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Affects Versions: 2.1.0.0-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.1.0.0-incubating > > > To generate catalog information for hawq, we need to make sure the right > version of metadata information is used. For hawq 2.2, the > tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in > [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need > to correct that for hawq 2.0 and 2.1. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HAWQ-1411) Inconsistent json file for catalog of hawq 2.1
[ https://issues.apache.org/jira/browse/HAWQ-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-1411: -- Fix Version/s: 2.1.0.0-incubating > Inconsistent json file for catalog of hawq 2.1 > -- > > Key: HAWQ-1411 > URL: https://issues.apache.org/jira/browse/HAWQ-1411 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Affects Versions: 2.1.0.0-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.1.0.0-incubating > > > To generate catalog information for hawq, we need to make sure the right > version of metadata information is used. For hawq 2.2, the > tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in > [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need > to correct that for hawq 2.1. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (HAWQ-1411) Inconsistent json file for catalog of hawq 2.0 and 2.1
[ https://issues.apache.org/jira/browse/HAWQ-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo reassigned HAWQ-1411: - Assignee: Ruilong Huo (was: Ed Espino) > Inconsistent json file for catalog of hawq 2.0 and 2.1 > -- > > Key: HAWQ-1411 > URL: https://issues.apache.org/jira/browse/HAWQ-1411 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Ruilong Huo >Assignee: Ruilong Huo > > To generate catalog information for hawq, we need to make sure the right > version of metadata information is used. For hawq 2.2, the > tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in > [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need > to correct that for hawq 2.0 and 2.1. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HAWQ-1411) Inconsistent json file for catalog of hawq 2.0 and 2.1
Ruilong Huo created HAWQ-1411: - Summary: Inconsistent json file for catalog of hawq 2.0 and 2.1 Key: HAWQ-1411 URL: https://issues.apache.org/jira/browse/HAWQ-1411 Project: Apache HAWQ Issue Type: Bug Components: Catalog Reporter: Ruilong Huo Assignee: Ed Espino To generate catalog information for hawq, we need to make sure the right version of metadata information is used. For hawq 2.2, the tools/bin/gppylib/data/2.2.json is created based on 2.2 code base in [HAWQ-1406|https://issues.apache.org/jira/browse/HAWQ-1406]. However, we need to correct that for hawq 2.0 and 2.1. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] incubator-hawq issue #1188: HAWQ-1406. Update HAWQ product version strings f...
Github user huor commented on the issue: https://github.com/apache/incubator-hawq/pull/1188 @stanlyxiang, you are right. The json need to be consistently for corresponding hawq version. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---