[GitHub] incubator-hawq issue #839: HAWQ-983. Fix bug that minirepro generates wrong ...

[hsyuan]

Github user hsyuan commented on the issue:

https://github.com/apache/incubator-hawq/pull/839
  
@d @oarap please take a look. This is ported from gpdb.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #838: HAWQ-934. Populate canSetTag of PlannedStmt from ...

[hsyuan]

Github user hsyuan commented on the issue:

https://github.com/apache/incubator-hawq/pull/838
  
@vraghavan78 @d @oarap need you guys kind review.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #835: HAWQ-980. hawq does not handle guc value w...

[paul-guo-]

Github user paul-guo- closed the pull request at:

https://github.com/apache/incubator-hawq/pull/835


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #843: HAWQ-914. Improve user experience of HAWQ'...

[paul-guo-]

Github user paul-guo- closed the pull request at:

https://github.com/apache/incubator-hawq/pull/843


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #839: HAWQ-983. Fix bug that minirepro generates wrong ...

[hsyuan]

Github user hsyuan commented on the issue:

https://github.com/apache/incubator-hawq/pull/839
  
ping, will anyone work on merging this pull request?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #838: HAWQ-934. Populate canSetTag of PlannedStmt from ...

[hsyuan]

Github user hsyuan commented on the issue:

https://github.com/apache/incubator-hawq/pull/838
  
ping, any updates?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #859: HAWQ-1011. Check whether the table to be register...

[xunzhang]

Github user xunzhang commented on the issue:

https://github.com/apache/incubator-hawq/pull/859
  
cc @ictmalili @huor 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #859: HAWQ-1011. Check whether the table to be r...

[xunzhang]

GitHub user xunzhang opened a pull request:

https://github.com/apache/incubator-hawq/pull/859

HAWQ-1011. Check whether the table to be registered is existed.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xunzhang/incubator-hawq HAWQ-1011

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/859.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #859


commit 400a88e55974b27cd33c1a36c7088d49d07e5185
Author: xunzhang 
Date:   2016-08-25T03:35:28Z

HAWQ-1011. Check whether the table to be registered is existed




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...

[denalex]

Github user denalex commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/857#discussion_r76180262
  
--- Diff: contrib/hawq-ambari-plugin/pom.xml ---
@@ -0,0 +1,125 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/maven-v4_0_0.xsd;>
+
+  4.0.0
+  org.apache.hawq
+  hawq-ambari-plugin
+  2.0.1
+  hawq-ambari-plugin
+  http://maven.apache.org
+
+  
+1.0.0
+UTF-8
+${basedir}/target/resources-temp
+/var/lib/hawq
+${hawq.lib.dir}/staging
+  
+
+  
+
+
+  
+org.apache.maven.plugins
+maven-resources-plugin
+3.0.1
+
+
+  
+none
+
+  copy-resources
+
+  
+
+
+
+  
+${basedir}/build.properties
+  
+  ${resources.temp}
+  
+
+  ${basedir}/src/main/resources/
+  true
+
+  
+
+  
+
+  
+org.codehaus.mojo
+rpm-maven-plugin
+2.1.4
+
+
+  
+none
+
+  rpm
+
+  
+
+
+
+  org.apache.hawq
+  
+HAWQ plugin contains Ambari's stack definition for HAWQ.
+When installed, Ambari will be able to support HAWQ as a 
service.
+  
+  x86_64
+  ${build_number}%{?dist}
+  
+ambari-server = 2.2
+python = 2.6
+  
+  
+
+  ${hawq.lib.staging.dir}
+  
+
+  ${resources.temp}/services
+
+  
+
+
+  ${hawq.lib.dir}
+  755
+  
+
+  ${resources.temp}/utils
+
+  
+
+  
+  
+

[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...

[denalex]

Github user denalex commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/857#discussion_r76180202
  
--- Diff: contrib/hawq-ambari-plugin/build.properties ---
@@ -0,0 +1,8 @@
+hawq.release.version=2.0.1
+hawq.common.services.version=2.0.0
+pxf.release.version=3.0.1
+pxf.common.services.version=3.0.0
+hawq.repo.prefix=hawq
+hawq.addons.repo.prefix=hawq-add-ons
+repository.version=2.0.1.0
+default.stack=HDP-2.4
--- End diff --

why a separate file for these ? why not have them inside pom.xml ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...

[ljainpivotalio]

Github user ljainpivotalio commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/857#discussion_r76179421
  
--- Diff: contrib/hawq-ambari-plugin/README.md ---
@@ -0,0 +1,97 @@
+# HAWQ Ambari Plugin
+
+hawq-ambari-plugin helps users install HAWQ and PXF using Ambari.
+
+To ensure that Ambari recognizes HAWQ and PXF as services that can be 
installed for a specific stack, the following manual steps are required:
+ * Add HAWQ and PXF metainfo.xml files (containing metadata about the 
service) under the stack to be installed.
+ * Add repositories where HAWQ and PXF rpms reside, so that Ambari can use 
it during installation. This requires updating repoinfo.xml under the stack 
HAWQ and PXF is to be added.
+ * If a stack is already installed using Ambari, add repositories to the 
existing stack using the Ambari REST API.
+
+
+## Source Code
+Source code directory structure of the hawq-ambari-plugin is as follows:
+```
+hawq-ambari-plugin
++-- README.md
++-- build.properties
++-- pom.xml
++-- src
++-- main
++-- resources
++-- services
+¦   +-- HAWQ
+¦   ¦   +-- metainfo.xml
+¦   +-- PXF
+¦   +-- metainfo.xml
++-- utils
++-- add-hawq.py
+```
+
+### build.properties
+[build.properties](build.properties) contains properties required for 
building the plugin.
+
+### metainfo.xml
+[metainfo.xml](src/main/resources/services/HAWQ/metainfo.xml) contains the 
metadata about the service. The metainfo.xml specifies that the service 
definition is to be inherited from Ambari common-services. HAWQ and PXF 
common-services code can be found under [Apache Ambari 
repository](https://github.com/apache/ambari/tree/trunk/ambari-server/src/main/resources/common-services/).
+
+### add-hawq.py
+[add-hawq.py](src/main/resources/utils/add-hawq.py) deploys HAWQ and PXF 
metainfo.xml files under the stack and adds the repositories to Ambari.
+
+
+## Building the plugin
+***Build Environment***: centos6 is the typical operating system used for 
building.
+
+Properties specified in the [build.properties](build.properties) file:
+
+| Property | Description | Value |
+| --- | --- | --- |
+| hawq.release.version | Release version of HAWQ | 2.0.1 |
+| hawq.common.services.version | HAWQ common services code in Ambari to be 
inherited | 2.0.0 |
+| pxf.release.version | Release version of PXF | 3.0.1 |
+| pxf.common.services.version | PXF common services code in Ambari to be 
inherited | 3.0.0 |
+| hawq.repo.prefix | Repository name for HAWQ core repository  | hawq |
+| hawq.addons.repo.prefix | Repository name for HAWQ Add Ons repository  | 
hawq-add-ons |
+| repository.version | Repository Version to be used in repository 
information | 2.0.1.0 |
+| default.stack | Default stack under which, metainfo.xml and repositories 
have to be added | HDP-2.4 |
+
+To build the rpm for hawq-ambari-plugin, change the 
[build.properties](build.properties) file with the required parameters and run 
```mvn install``` command under hawq-ambari-plugin directory:
+```
+$ pwd
+incubator-hawq/contrib/hawq-ambari-plugin
+$ mvn install
--- End diff --

mvn clean resources:copy-resources rpm:rpm -Dbuild_number="${BUILD_NUMBER}"



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...

[ljainpivotalio]

Github user ljainpivotalio commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/857#discussion_r76179141
  
--- Diff: contrib/hawq-ambari-plugin/src/main/resources/utils/add-hawq.py 
---
@@ -0,0 +1,498 @@
+#!/usr/bin/env python
+
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+import base64
+import getpass
+import json
+import os
+import shutil
+import socket
+import urllib2
+import xml.etree.ElementTree as ET
+from optparse import OptionParser
+
+DEFAULT_STACK = '${default.stack}'
+SUPPORTED_OS_LIST = ['redhat6']
+HAWQ_LIB_STAGING_DIR = '${hawq.lib.staging.dir}'
+REPO_VERSION = '${repository.version}'
+HAWQ_REPO = '${hawq.repo.prefix}'
+HAWQ_ADD_ONS_REPO = '${hawq.addons.repo.prefix}'
+
+REPO_INFO = {
+  HAWQ_REPO: {
+'repoid': '-'.join([HAWQ_REPO, REPO_VERSION]),
+'input_param': '--hawqrepo',
+'optional': False
+  },
+  HAWQ_ADD_ONS_REPO: {
+'repoid': '-'.join([HAWQ_ADD_ONS_REPO, REPO_VERSION]),
+'input_param': '--addonsrepo',
+'optional': True
+  }
+}
+
+
+class APIClient:
+  """
+  Class which interacts with Ambari Server API
+  """
+
+  # Base API URL points to localhost. This script is to be executed on the 
Ambari Server
+  BASE_API_URL = 'http://localhost:8080/api/v1'
+
+  def __init__(self, user, password):
+self.user = user
+self.password = password
+self.encoded_credentials = base64.encodestring(self.user + ':' + 
self.password).replace('\n', '')
+
+  def __request(self, method, url_path, headers=None, data=None):
+"""
+Creates API requests and packages response into the following format: 
(response code, response body in json object)
+"""
+headers = headers if headers is not None else {}
+headers['Authorization'] = 'Basic {0}'.format(self.encoded_credentials)
+
+req = urllib2.Request(self.BASE_API_URL + url_path, data, headers)
+req.get_method = lambda: method
+response = urllib2.urlopen(req)
+response_str = response.read()
+
+return response.getcode(), json.loads(response_str) if response_str 
else None
+
+  def verify_api_reachable(self):
+"""
+Returns true if Ambari Server is reachable through API
+"""
+try:
+  status_code, _ = self.__request('GET', '/stacks')
+except Exception as e:
+  if type(e) == urllib2.HTTPError and e.code == 403:
+raise Exception('Invalid username and/or password.')
+  elif type(e) == urllib2.URLError:
+raise Exception('Ambari-server is not running. Please start 
ambari-server.')
+  else:
+raise Exception('Unable to connect to Ambari Server.\n' + str(e))
+
+  def get_cluster_name(self):
+"""
+Returns the name of the installed cluster
+"""
+_, response_json = self.__request('GET', '/clusters')
+return None if len(response_json['items']) == 0 else 
response_json['items'][0]['Clusters']['cluster_name']
+
+  def get_stack_info(self, cluster_name):
+"""
+Returns stack information (stack name, stack version, repository 
version) of stack installed on cluster
+"""
+_, response_json = self.__request('GET',
+  
'/clusters/{0}/stack_versions?ClusterStackVersions/state.matches(CURRENT)'.format(
+  cluster_name))
+if 'items' not in response_json or len(response_json['items']) == 0:
+  raise Exception('No Stack found to be installed on the cluster 
{0}'.format(cluster_name))
+stack_versions = response_json['items'][0]['ClusterStackVersions']
+return stack_versions['stack'], stack_versions['version'], 
stack_versions['repository_version']
+
+  def get_existing_repository_info(self, stack_name, stack_version, 
repository_version):
+"""
+Returns existing repo information for a given stack
+"""

[GitHub] incubator-hawq pull request #858: HAWQ-1012. Append test yaml files for hawq...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/858


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #858: HAWQ-1012. Append test yaml files for hawq regist...

[yaoj2]

Github user yaoj2 commented on the issue:

https://github.com/apache/incubator-hawq/pull/858
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #858: HAWQ-1012. Append test yaml files for hawq regist...

[huor]

Github user huor commented on the issue:

https://github.com/apache/incubator-hawq/pull/858
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #858: HAWQ-1012. Append test yaml files for hawq regist...

[xunzhang]

Github user xunzhang commented on the issue:

https://github.com/apache/incubator-hawq/pull/858
  
cc @huor @yaoj2 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #858: HAWQ-1012. Append test yaml files for hawq...

[xunzhang]

GitHub user xunzhang opened a pull request:

https://github.com/apache/incubator-hawq/pull/858

HAWQ-1012. Append test yaml files for hawq register feature tests.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xunzhang/incubator-hawq HAWQ-1012-HOTFIX

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/858.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #858


commit b29dc418a08f352618b732e6889a6ca4a1d6a627
Author: xunzhang 
Date:   2016-08-25T02:33:29Z

HAWQ-1012. Append test yaml files for hawq register feature tests.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (HAWQ-1012) Check whether the input yaml file for hawq register is valid.

[hongwu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hongwu resolved HAWQ-1012.
--
Resolution: Implemented

> Check whether the input yaml file for hawq register is valid.
> -
>
> Key: HAWQ-1012
> URL: https://issues.apache.org/jira/browse/HAWQ-1012
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Command Line Tools
>Affects Versions: 2.0.1.0-incubating
>Reporter: hongwu
>Assignee: hongwu
> Fix For: 2.0.1.0-incubating
>
>
> To avoid error-typed input yaml configuration file for hawq register, we need 
> to add a function to check whether the input yaml file is valid or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #855: HAWQ-1012. Check whether the input yaml fi...

[xunzhang]

Github user xunzhang closed the pull request at:

https://github.com/apache/incubator-hawq/pull/855


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (HAWQ-1012) Check whether the input yaml file for hawq register is valid.

[hongwu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hongwu updated HAWQ-1012:
-
Affects Version/s: 2.0.1.0-incubating

> Check whether the input yaml file for hawq register is valid.
> -
>
> Key: HAWQ-1012
> URL: https://issues.apache.org/jira/browse/HAWQ-1012
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Command Line Tools
>Affects Versions: 2.0.1.0-incubating
>Reporter: hongwu
>Assignee: hongwu
> Fix For: 2.0.1.0-incubating
>
>
> To avoid error-typed input yaml configuration file for hawq register, we need 
> to add a function to check whether the input yaml file is valid or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (HAWQ-256) Integrate Security with Apache Ranger

[Hubert Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hubert Zhang updated HAWQ-256:
--
Comment: was deleted

(was: Agree with [~vVineet] If enable_ranger is ON in Hawq, we should disable 
grant/revoke/reassign statement on CLI. Here reassign is a Hawq statement which 
changes the ownership of database objects owned by a database role to other 
role.)

> Integrate Security with Apache Ranger
> -
>
> Key: HAWQ-256
> URL: https://issues.apache.org/jira/browse/HAWQ-256
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: PXF, Security
>Reporter: Michael Andre Pearce (IG)
>Assignee: Lili Ma
> Fix For: backlog
>
> Attachments: HAWQRangerSupportDesign.pdf
>
>
> Integrate security with Apache Ranger for a unified Hadoop security solution. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-256) Integrate Security with Apache Ranger

[Hubert Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436134#comment-15436134
 ] 

Hubert Zhang commented on HAWQ-256:
---

Agree with [~vVineet] If enable_ranger is ON in Hawq, we should disable 
grant/revoke/reassign statement on CLI. Here reassign is a Hawq statement which 
changes the ownership of database objects owned by a database role to other 
role.

> Integrate Security with Apache Ranger
> -
>
> Key: HAWQ-256
> URL: https://issues.apache.org/jira/browse/HAWQ-256
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: PXF, Security
>Reporter: Michael Andre Pearce (IG)
>Assignee: Lili Ma
> Fix For: backlog
>
> Attachments: HAWQRangerSupportDesign.pdf
>
>
> Integrate security with Apache Ranger for a unified Hadoop security solution. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-256) Integrate Security with Apache Ranger

[Hubert Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436135#comment-15436135
 ] 

Hubert Zhang commented on HAWQ-256:
---

Agree with [~vVineet] If enable_ranger is ON in Hawq, we should disable 
grant/revoke/reassign statement on CLI. Here reassign is a Hawq statement which 
changes the ownership of database objects owned by a database role to other 
role.

> Integrate Security with Apache Ranger
> -
>
> Key: HAWQ-256
> URL: https://issues.apache.org/jira/browse/HAWQ-256
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: PXF, Security
>Reporter: Michael Andre Pearce (IG)
>Assignee: Lili Ma
> Fix For: backlog
>
> Attachments: HAWQRangerSupportDesign.pdf
>
>
> Integrate security with Apache Ranger for a unified Hadoop security solution. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #856: HAWQ-984. hawq config is too slow.

[radarwave]

Github user radarwave commented on the issue:

https://github.com/apache/incubator-hawq/pull/856
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...

[mithmatt]

GitHub user mithmatt opened a pull request:

https://github.com/apache/incubator-hawq/pull/857

HAWQ-1013. Move HAWQ Ambari plugin to Apache HAWQ



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mithmatt/incubator-hawq hawq-ambari-plugin

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/857.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #857


commit d77f210326acb4bcbcae4e96a405d7141b41a639
Author: Matt 
Date:   2016-08-24T23:37:48Z

HAWQ-1013. Move HAWQ Ambari plugin to Apache HAWQ




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (HAWQ-1019) clarify database application interfaces discussion

[Lisa Owen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lisa Owen updated HAWQ-1019:

Priority: Minor  (was: Major)

> clarify database application interfaces discussion
> --
>
> Key: HAWQ-1019
> URL: https://issues.apache.org/jira/browse/HAWQ-1019
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lisa Owen
>Assignee: Lei Chang
>Priority: Minor
> Fix For: 2.0.1.0-incubating
>
>
> discussion of drivers for database application interfaces needs to be 
> clarified.
> relevant incubator-hawq-docs file:  
> clientaccess/g-database-application-interfaces.html.md.erb 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-1019) clarify database application interfaces discussion

[Lisa Owen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lisa Owen updated HAWQ-1019:

Fix Version/s: 2.0.1.0-incubating

> clarify database application interfaces discussion
> --
>
> Key: HAWQ-1019
> URL: https://issues.apache.org/jira/browse/HAWQ-1019
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lisa Owen
>Assignee: Lei Chang
>Priority: Minor
> Fix For: 2.0.1.0-incubating
>
>
> discussion of drivers for database application interfaces needs to be 
> clarified.
> relevant incubator-hawq-docs file:  
> clientaccess/g-database-application-interfaces.html.md.erb 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-1019) clarify database application interfaces discussion

[Lisa Owen (JIRA)]

Lisa Owen created HAWQ-1019:
---

 Summary: clarify database application interfaces discussion
 Key: HAWQ-1019
 URL: https://issues.apache.org/jira/browse/HAWQ-1019
 Project: Apache HAWQ
  Issue Type: Improvement
  Components: Documentation
Reporter: Lisa Owen
Assignee: Lei Chang


discussion of drivers for database application interfaces needs to be clarified.

relevant incubator-hawq-docs file:  
clientaccess/g-database-application-interfaces.html.md.erb 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-762) Hive aggregation queries through PXF sometimes hang

[Goden Yao (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Goden Yao updated HAWQ-762:
---
Description: 
Reproduce Steps:
{code}
select count(*) from hcatalog.default.hivetable;
{code}

sometimes, this query will hang and we see from pxf logs that Hive thrift 
server cannot be connected from PXF agent. 
While users can still visit hive metastore (through HUE) and execute the same 
query.

After a restart of PXF agent, this query goes through without issues.

*Troubleshooting Guide*
- check catalina.out (tomcat) and pxf-service.log to see if the query request 
gets to tomcat/pxf webapp, any exceptions happened during the time window
- enable {code}log_min_messages=DEBUG2{code} to see at which step the query is 
stuck
- try:
{code}
curl http:///pxf/ProtocolVersion
{code}
where URI is the hostname or IP of the machine you installed PXF, port is 
usually 51200 if you didn’t change it.
The response you’ll get if PXF service is running OK:
{code}
{version: v14}
{code}

  was:
Reproduce Steps:
{code}
select count(*) from hcatalog.default.hivetable;
{code}

sometimes, this query will hang and we see from pxf logs that Hive thrift 
server cannot be connected from PXF agent. 
While users can still visit hive metastore (through HUE) and execute the same 
query.

After a restart of PXF agent, this query goes through without issues.

*Troubleshooting Guide*
- check catalina.out (tomcat) logs to see if the query request gets to tomcat
- enable {code}log_min_messages=DEBUG2{code} to see at which step the query is 
stuck
- try:
{code}
curl http:///pxf/ProtocolVersion
{code}
where URI is the hostname or IP of the machine you installed PXF, port is 
usually 51200 if you didn’t change it.
The response you’ll get if PXF service is running OK:
{code}
{version: v14}
{code}


> Hive aggregation queries through PXF sometimes hang
> ---
>
> Key: HAWQ-762
> URL: https://issues.apache.org/jira/browse/HAWQ-762
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Hcatalog, PXF
>Affects Versions: 2.0.0.0-incubating
>Reporter: Oleksandr Diachenko
>Assignee: Goden Yao
>  Labels: performance
> Fix For: backlog
>
>
> Reproduce Steps:
> {code}
> select count(*) from hcatalog.default.hivetable;
> {code}
> sometimes, this query will hang and we see from pxf logs that Hive thrift 
> server cannot be connected from PXF agent. 
> While users can still visit hive metastore (through HUE) and execute the same 
> query.
> After a restart of PXF agent, this query goes through without issues.
> *Troubleshooting Guide*
> - check catalina.out (tomcat) and pxf-service.log to see if the query request 
> gets to tomcat/pxf webapp, any exceptions happened during the time window
> - enable {code}log_min_messages=DEBUG2{code} to see at which step the query 
> is stuck
> - try:
> {code}
> curl http:///pxf/ProtocolVersion
> {code}
> where URI is the hostname or IP of the machine you installed PXF, port is 
> usually 51200 if you didn’t change it.
> The response you’ll get if PXF service is running OK:
> {code}
> {version: v14}
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-762) Hive aggregation queries through PXF sometimes hang

[Goden Yao (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Goden Yao updated HAWQ-762:
---
Description: 
Reproduce Steps:
{code}
select count(*) from hcatalog.default.hivetable;
{code}

sometimes, this query will hang and we see from pxf logs that Hive thrift 
server cannot be connected from PXF agent. 
While users can still visit hive metastore (through HUE) and execute the same 
query.

After a restart of PXF agent, this query goes through without issues.

*Troubleshooting Guide*
- check catalina.out (tomcat) logs to see if the query request gets to tomcat
- enable {code}log_min_messages=DEBUG2{code} to see at which step the query is 
stuck
- try:
{code}
curl http:///pxf/ProtocolVersion
{code}
where URI is the hostname or IP of the machine you installed PXF, port is 
usually 51200 if you didn’t change it.
The response you’ll get if PXF service is running OK:
{code}
{version: v14}
{code}

  was:
Reproduce Steps:
{code}
select count(*) from hcatalog.default.hivetable;
{code}

sometimes, this query will hang and we see from pxf logs that Hive thrift 
server cannot be connected from PXF agent. 
While users can still visit hive metastore (through HUE) and execute the same 
query.

After a restart of PXF agent, this query goes through without issues.

**troubleshooting guide**
- check catalina.out (tomcat) logs to see if the query request gets to tomcat
- enable {code}log_min_messages=DEBUG2{code} to see at which step the query is 
stuck
- try:
{code}
curl http:///pxf/ProtocolVersion
{code}
where URI is the hostname or IP of the machine you installed PXF, port is 
usually 51200 if you didn’t change it.
The response you’ll get if PXF service is running OK:
{code}
{version: v14}
{code}


> Hive aggregation queries through PXF sometimes hang
> ---
>
> Key: HAWQ-762
> URL: https://issues.apache.org/jira/browse/HAWQ-762
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Hcatalog, PXF
>Affects Versions: 2.0.0.0-incubating
>Reporter: Oleksandr Diachenko
>Assignee: Goden Yao
>  Labels: performance
> Fix For: backlog
>
>
> Reproduce Steps:
> {code}
> select count(*) from hcatalog.default.hivetable;
> {code}
> sometimes, this query will hang and we see from pxf logs that Hive thrift 
> server cannot be connected from PXF agent. 
> While users can still visit hive metastore (through HUE) and execute the same 
> query.
> After a restart of PXF agent, this query goes through without issues.
> *Troubleshooting Guide*
> - check catalina.out (tomcat) logs to see if the query request gets to tomcat
> - enable {code}log_min_messages=DEBUG2{code} to see at which step the query 
> is stuck
> - try:
> {code}
> curl http:///pxf/ProtocolVersion
> {code}
> where URI is the hostname or IP of the machine you installed PXF, port is 
> usually 51200 if you didn’t change it.
> The response you’ll get if PXF service is running OK:
> {code}
> {version: v14}
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-762) Hive aggregation queries through PXF sometimes hang

[Goden Yao (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Goden Yao updated HAWQ-762:
---
Description: 
Reproduce Steps:
{code}
select count(*) from hcatalog.default.hivetable;
{code}

sometimes, this query will hang and we see from pxf logs that Hive thrift 
server cannot be connected from PXF agent. 
While users can still visit hive metastore (through HUE) and execute the same 
query.

After a restart of PXF agent, this query goes through without issues.

**troubleshooting guide**
- check catalina.out (tomcat) logs to see if the query request gets to tomcat
- enable {code}log_min_messages=DEBUG2{code} to see at which step the query is 
stuck
- try:
{code}
curl http:///pxf/ProtocolVersion
{code}
where URI is the hostname or IP of the machine you installed PXF, port is 
usually 51200 if you didn’t change it.
The response you’ll get if PXF service is running OK:
{code}
{version: v14}
{code}

  was:
Reproduce Steps:
{code}
select count(*) from hcatalog.default.hivetable;
{code}

sometimes, this query will hang and we see from pxf logs that Hive thrift 
server cannot be connected from PXF agent. 
While users can still visit hive metastore (through HUE) and execute the same 
query.

After a restart of PXF agent, this query goes through without issues.



> Hive aggregation queries through PXF sometimes hang
> ---
>
> Key: HAWQ-762
> URL: https://issues.apache.org/jira/browse/HAWQ-762
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Hcatalog, PXF
>Affects Versions: 2.0.0.0-incubating
>Reporter: Oleksandr Diachenko
>Assignee: Goden Yao
>  Labels: performance
> Fix For: backlog
>
>
> Reproduce Steps:
> {code}
> select count(*) from hcatalog.default.hivetable;
> {code}
> sometimes, this query will hang and we see from pxf logs that Hive thrift 
> server cannot be connected from PXF agent. 
> While users can still visit hive metastore (through HUE) and execute the same 
> query.
> After a restart of PXF agent, this query goes through without issues.
> **troubleshooting guide**
> - check catalina.out (tomcat) logs to see if the query request gets to tomcat
> - enable {code}log_min_messages=DEBUG2{code} to see at which step the query 
> is stuck
> - try:
> {code}
> curl http:///pxf/ProtocolVersion
> {code}
> where URI is the hostname or IP of the machine you installed PXF, port is 
> usually 51200 if you didn’t change it.
> The response you’ll get if PXF service is running OK:
> {code}
> {version: v14}
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #854: HAWQ-997. HAWQ doesn't send PXF data type ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/854


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #853: HAWQ-967. Extend Projection info to includ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/853


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (HAWQ-997) HAWQ doesn't send PXF data type with precision

[Oleksandr Diachenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleksandr Diachenko resolved HAWQ-997.
--
Resolution: Fixed

> HAWQ doesn't send PXF data type with precision 
> ---
>
> Key: HAWQ-997
> URL: https://issues.apache.org/jira/browse/HAWQ-997
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: PXF
>Reporter: Shivram Mani
>Assignee: Oleksandr Diachenko
> Fix For: backlog
>
>
> HAWQ/PXF sends via the Rest api information about ATTR and types using 
> x-gp-attr-typename. Attributes such as varchar(3) char(3) are sent as varchar 
> and char. This causes HAWQ-992 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1013) Move HAWQ Ambari plugin to Apache HAWQ

[Vineet Goel (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435483#comment-15435483
 ] 

Vineet Goel commented on HAWQ-1013:
---

RPM packaging is a build-time decision, separate from source code management. 
There are pros and cons to both. Separate RPM allows the flexibility of a quick 
install required on Ambari-server node only, where HAWQ binaries may not be 
necessary (depending on cluster size and topology).

> Move HAWQ Ambari plugin to Apache HAWQ
> --
>
> Key: HAWQ-1013
> URL: https://issues.apache.org/jira/browse/HAWQ-1013
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: Ambari
>Reporter: Matt
>Assignee: Alexander Denissov
>  Labels: UX
> Fix For: backlog
>
>
> To add HAWQ and PXF to Ambari, users have to follow certain manual steps. 
> These manual steps include:
> - Adding HAWQ and PXF metainfo.xml files (containing metadata about the 
> service) under the stack to be installed.
> - Adding repositories, where HAWQ and PXF rpms reside so that Ambari can use 
> it during installation. This requires updating repoinfo.xml under the stack 
> HAWQ and PXF is being added to.
> - Adding repositories to an existing stack managed by Ambari requires adding 
> the repositories using the Ambari REST endpoint.
> The HAWQ Ambari plugin automates the above steps using a script.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-967) Extend Projection info to include filter attributes

[Oleksandr Diachenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleksandr Diachenko resolved HAWQ-967.
--
Resolution: Fixed

> Extend Projection info to include filter attributes
> ---
>
> Key: HAWQ-967
> URL: https://issues.apache.org/jira/browse/HAWQ-967
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: PXF
>Reporter: Shivram Mani
>Assignee: Oleksandr Diachenko
>Priority: Blocker
> Fix For: backlog
>
>
> HAWQ-927 includes query projection columns as part of the projection info 
> sent from HAWQ to PXF.
> For queries where filter attributes are different from projection attributes, 
> PXF would return data with NULL values in the filter attributes.
> e.g. a table "test" has 2 columns say: c1 int, c2 int
> {code}
> select c1 from test where c2 > 0;
> {code}
> In the case above, as c2 is not in column projection, pxf will return records 
> like (1, NULL), (2, NULL) ... as part of the implementation in HAWQ-927 
> Due to this HAWQ wouldn't have the necessary data to carry out filters once 
> it receives data back from the underlying external dataset. via PXF and wrong 
> result will be returned to users
> The projection information must be a union of the internal HAWQ projection 
> info and the attributes in the filters.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1013) Move HAWQ Ambari plugin to Apache HAWQ

[Goden Yao (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435475#comment-15435475
 ] 

Goden Yao commented on HAWQ-1013:
-

Thanks for the clarification - in that case, I don't think it'd be necessary to 
package this python script into a separate RPM - -which is the case today I 
suppose?
The python script should be part of HAWQ utility and packaged within HAWQ RPM.

> Move HAWQ Ambari plugin to Apache HAWQ
> --
>
> Key: HAWQ-1013
> URL: https://issues.apache.org/jira/browse/HAWQ-1013
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: Ambari
>Reporter: Matt
>Assignee: Alexander Denissov
>  Labels: UX
> Fix For: backlog
>
>
> To add HAWQ and PXF to Ambari, users have to follow certain manual steps. 
> These manual steps include:
> - Adding HAWQ and PXF metainfo.xml files (containing metadata about the 
> service) under the stack to be installed.
> - Adding repositories, where HAWQ and PXF rpms reside so that Ambari can use 
> it during installation. This requires updating repoinfo.xml under the stack 
> HAWQ and PXF is being added to.
> - Adding repositories to an existing stack managed by Ambari requires adding 
> the repositories using the Ambari REST endpoint.
> The HAWQ Ambari plugin automates the above steps using a script.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #853: HAWQ-967. Extend Projection info to includ...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/853#discussion_r76117988
  
--- Diff: src/backend/access/external/pxffilters.c ---
@@ -451,6 +452,51 @@ opexpr_to_pxffilter(OpExpr *expr, PxfFilterDesc 
*filter)
return false;
 }
 
+static List*
+append_attr_from_var(Var* var, List* attrs)
+{
+   AttrNumber varattno = var->varattno;
+   /* system attr not supported */
+   if (varattno > InvalidAttrNumber)
--- End diff --

@kavinderd, yes, physical attributes should have positive value, some 
system attributes have -1, and we are not projecting them.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #854: HAWQ-997. HAWQ doesn't send PXF data type with pr...

[GodenYao]

Github user GodenYao commented on the issue:

https://github.com/apache/incubator-hawq/pull/854
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-1013) Move HAWQ Ambari plugin to Apache HAWQ

[Vineet Goel (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435462#comment-15435462
 ] 

Vineet Goel commented on HAWQ-1013:
---

This JIRA is actually not related to moving Ambari plugin to HAWQ repo, perhaps 
the description should be changed.
Ambari plugin requires Ambari to be aware of a service, and that plugin code 
already resides in Apache Ambari repo, here:
https://github.com/apache/ambari/tree/trunk/ambari-server/src/main/resources/common-services/HAWQ/2.0.0

In order to enable the Ambari plugin, a script is required during HAWQ 
install-time to make an Ambari “Stack” aware of the service being managed. That 
script and implementation is tightly coupled with the service (HAWQ) itself and 
should be managed in the service (HAWQ) repo. Hope this helps.

> Move HAWQ Ambari plugin to Apache HAWQ
> --
>
> Key: HAWQ-1013
> URL: https://issues.apache.org/jira/browse/HAWQ-1013
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: Ambari
>Reporter: Matt
>Assignee: Alexander Denissov
>  Labels: UX
> Fix For: backlog
>
>
> To add HAWQ and PXF to Ambari, users have to follow certain manual steps. 
> These manual steps include:
> - Adding HAWQ and PXF metainfo.xml files (containing metadata about the 
> service) under the stack to be installed.
> - Adding repositories, where HAWQ and PXF rpms reside so that Ambari can use 
> it during installation. This requires updating repoinfo.xml under the stack 
> HAWQ and PXF is being added to.
> - Adding repositories to an existing stack managed by Ambari requires adding 
> the repositories using the Ambari REST endpoint.
> The HAWQ Ambari plugin automates the above steps using a script.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HAWQ-992) PXF Hive data type check in Fragmenter too restrictive

[Oleksandr Diachenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleksandr Diachenko reassigned HAWQ-992:


Assignee: Oleksandr Diachenko  (was: Shivram Mani)

> PXF Hive data type check in Fragmenter too restrictive
> --
>
> Key: HAWQ-992
> URL: https://issues.apache.org/jira/browse/HAWQ-992
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: PXF
>Reporter: Shivram Mani
>Assignee: Oleksandr Diachenko
> Fix For: backlog
>
>
> HiveDataFragmenter used by both HiveText and HiveRC profiles has a very 
> strict type check. 
> Hawq type numeric(10,10) is compatible with hive's decimal(10,10)
> But Hawq type numeric is not compatible with hive's decimal(10,10)
> Similar issue exits with other data types which have variable optional 
> arguments. The type check should be modified to allow hawq type that is a 
> compabitle type but without optional precision/length arguments to work with 
> the corresponding hive type.
> Support following additional hive data types: date, varchar, char



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HAWQ-997) HAWQ doesn't send PXF data type with precision

[Oleksandr Diachenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433993#comment-15433993
 ] 

Oleksandr Diachenko edited comment on HAWQ-997 at 8/24/16 6:27 PM:
---

HAWQ datatypes with precision:

* {noformat}bit [ (n) ]

* bit varying [ (n) ]

* character [ (n) ]

* character varying [ (n) ]

* decimal [ (p, s) ]

* interval [ (p) ]

* time [ (p) ] [ without time zone ]

* time [ (p) ] with time zone

* timestamp [ (p) ] [without time zone ]

* timestamp [ (p) ] with time zone
{noformat}


was (Author: odiachenko):
HAWQ datatypes with precision:

* {noformat}bit [ (n) ]{noformat}

* bit varying [ (n) ]

* character [ (n) ]

* character varying [ (n) ]

* decimal [ (p, s) ]

* interval [ (p) ]

* time [ (p) ] [ without time zone ]

* time [ (p) ] with time zone

* timestamp [ (p) ] [without time zone ]

* timestamp [ (p) ] with time zone

> HAWQ doesn't send PXF data type with precision 
> ---
>
> Key: HAWQ-997
> URL: https://issues.apache.org/jira/browse/HAWQ-997
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: PXF
>Reporter: Shivram Mani
>Assignee: Goden Yao
> Fix For: backlog
>
>
> HAWQ/PXF sends via the Rest api information about ATTR and types using 
> x-gp-attr-typename. Attributes such as varchar(3) char(3) are sent as varchar 
> and char. This causes HAWQ-992 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HAWQ-997) HAWQ doesn't send PXF data type with precision

[Oleksandr Diachenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433993#comment-15433993
 ] 

Oleksandr Diachenko edited comment on HAWQ-997 at 8/24/16 6:27 PM:
---

HAWQ datatypes with precision:

{noformat}
* bit [ (n) ]

* bit varying [ (n) ]

* character [ (n) ]

* character varying [ (n) ]

* decimal [ (p, s) ]

* interval [ (p) ]

* time [ (p) ] [ without time zone ]

* time [ (p) ] with time zone

* timestamp [ (p) ] [without time zone ]

* timestamp [ (p) ] with time zone
{noformat}


was (Author: odiachenko):
HAWQ datatypes with precision:

* {noformat}bit [ (n) ]

* bit varying [ (n) ]

* character [ (n) ]

* character varying [ (n) ]

* decimal [ (p, s) ]

* interval [ (p) ]

* time [ (p) ] [ without time zone ]

* time [ (p) ] with time zone

* timestamp [ (p) ] [without time zone ]

* timestamp [ (p) ] with time zone
{noformat}

> HAWQ doesn't send PXF data type with precision 
> ---
>
> Key: HAWQ-997
> URL: https://issues.apache.org/jira/browse/HAWQ-997
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: PXF
>Reporter: Shivram Mani
>Assignee: Goden Yao
> Fix For: backlog
>
>
> HAWQ/PXF sends via the Rest api information about ATTR and types using 
> x-gp-attr-typename. Attributes such as varchar(3) char(3) are sent as varchar 
> and char. This causes HAWQ-992 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HAWQ-997) HAWQ doesn't send PXF data type with precision

[Oleksandr Diachenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433993#comment-15433993
 ] 

Oleksandr Diachenko edited comment on HAWQ-997 at 8/24/16 6:27 PM:
---

HAWQ datatypes with precision:

* {noformat}bit [ (n) ]{noformat}

* bit varying [ (n) ]

* character [ (n) ]

* character varying [ (n) ]

* decimal [ (p, s) ]

* interval [ (p) ]

* time [ (p) ] [ without time zone ]

* time [ (p) ] with time zone

* timestamp [ (p) ] [without time zone ]

* timestamp [ (p) ] with time zone


was (Author: odiachenko):
HAWQ datatypes with precision:

* {noformat}bit [ (n) ]

* bit varying [ (n) ]

* character [ (n) ]

* character varying [ (n) ]

* decimal [ (p, s) ]

* interval [ (p) ]

* time [ (p) ] [ without time zone ]

* time [ (p) ] with time zone

* timestamp [ (p) ] [without time zone ]

* timestamp [ (p) ] with time zone

> HAWQ doesn't send PXF data type with precision 
> ---
>
> Key: HAWQ-997
> URL: https://issues.apache.org/jira/browse/HAWQ-997
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: PXF
>Reporter: Shivram Mani
>Assignee: Goden Yao
> Fix For: backlog
>
>
> HAWQ/PXF sends via the Rest api information about ATTR and types using 
> x-gp-attr-typename. Attributes such as varchar(3) char(3) are sent as varchar 
> and char. This causes HAWQ-992 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HAWQ-997) HAWQ doesn't send PXF data type with precision

[Oleksandr Diachenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433993#comment-15433993
 ] 

Oleksandr Diachenko edited comment on HAWQ-997 at 8/24/16 6:27 PM:
---

HAWQ datatypes with precision:

* {noformat}bit [ (n) ]

* bit varying [ (n) ]

* character [ (n) ]

* character varying [ (n) ]

* decimal [ (p, s) ]

* interval [ (p) ]

* time [ (p) ] [ without time zone ]

* time [ (p) ] with time zone

* timestamp [ (p) ] [without time zone ]

* timestamp [ (p) ] with time zone


was (Author: odiachenko):
HAWQ datatypes with precision:

* bit [ (n) ]

* bit varying [ (n) ]

* character [ (n) ]

* character varying [ (n) ]

* decimal [ (p, s) ]

* interval [ (p) ]

* time [ (p) ] [ without time zone ]

* time [ (p) ] with time zone

* timestamp [ (p) ] [without time zone ]

* timestamp [ (p) ] with time zone

> HAWQ doesn't send PXF data type with precision 
> ---
>
> Key: HAWQ-997
> URL: https://issues.apache.org/jira/browse/HAWQ-997
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: PXF
>Reporter: Shivram Mani
>Assignee: Goden Yao
> Fix For: backlog
>
>
> HAWQ/PXF sends via the Rest api information about ATTR and types using 
> x-gp-attr-typename. Attributes such as varchar(3) char(3) are sent as varchar 
> and char. This causes HAWQ-992 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HAWQ-997) HAWQ doesn't send PXF data type with precision

[Oleksandr Diachenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433993#comment-15433993
 ] 

Oleksandr Diachenko edited comment on HAWQ-997 at 8/24/16 6:26 PM:
---

HAWQ datatypes with precision:

* bit [ (n) ]

* bit varying [ (n) ]

* character [ (n) ]

* character varying [ (n) ]

* decimal [ (p, s) ]

* interval [ (p) ]

* time [ (p) ] [ without time zone ]

* time [ (p) ] with time zone

* timestamp [ (p) ] [without time zone ]

* timestamp [ (p) ] with time zone


was (Author: odiachenko):
HAWQ datatypes with precision:

* numeric(p,s)

> HAWQ doesn't send PXF data type with precision 
> ---
>
> Key: HAWQ-997
> URL: https://issues.apache.org/jira/browse/HAWQ-997
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: PXF
>Reporter: Shivram Mani
>Assignee: Goden Yao
> Fix For: backlog
>
>
> HAWQ/PXF sends via the Rest api information about ATTR and types using 
> x-gp-attr-typename. Attributes such as varchar(3) char(3) are sent as varchar 
> and char. This causes HAWQ-992 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #854: HAWQ-997. HAWQ doesn't send PXF data type with pr...

[kavinderd]

Github user kavinderd commented on the issue:

https://github.com/apache/incubator-hawq/pull/854
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #853: HAWQ-967. Extend Projection info to include filte...

[kavinderd]

Github user kavinderd commented on the issue:

https://github.com/apache/incubator-hawq/pull/853
  
Nice work! LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #853: HAWQ-967. Extend Projection info to includ...

[kavinderd]

Github user kavinderd commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/853#discussion_r76100388
  
--- Diff: src/backend/access/external/pxffilters.c ---
@@ -451,6 +452,51 @@ opexpr_to_pxffilter(OpExpr *expr, PxfFilterDesc 
*filter)
return false;
 }
 
+static List*
+append_attr_from_var(Var* var, List* attrs)
+{
+   AttrNumber varattno = var->varattno;
+   /* system attr not supported */
+   if (varattno > InvalidAttrNumber)
--- End diff --

So the `varattno` is only valid when greater than `InvalidAttrNumber`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #837: HAWQ-779 support pxf filter pushdwon at th...

[kavinderd]

Github user kavinderd commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/837#discussion_r76098154
  
--- Diff: src/backend/optimizer/plan/createplan.c ---
@@ -1146,7 +1146,7 @@ static char** create_pxf_plan(char **segdb_file_map, 
RelOptInfo *rel, int total_
Relation relation = RelationIdGetRelation(planner_rt_fetch(scan_relid, 
ctx->root)->relid);
segdb_work_map = map_hddata_2gp_segments(uri_str, 

 total_segs, segs_participating,
-   
 relation, NULL);
+   
 relation, ctx->root->parse->jointree->quals);
--- End diff --

Can you ensure that the filters are only passed when the guc 
`pxf_enable_filter_pushdown` is enabled.  Such as is done here: 
https://github.com/apache/incubator-hawq/blob/316c5048f303513c40e8194b15024df12a3d76ad/src/backend/access/external/pxffilters.c#L554
 

Currently if this guc is disabled filters should not be passed to PXF


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #855: HAWQ-1012. Check whether the input yaml file for ...

[ictmalili]

Github user ictmalili commented on the issue:

https://github.com/apache/incubator-hawq/pull/855
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #856: HAWQ-984. hawq config is too slow.

[xunzhang]

Github user xunzhang commented on the issue:

https://github.com/apache/incubator-hawq/pull/856
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #856: HAWQ-984. hawq config is too slow.

[paul-guo-]

GitHub user paul-guo- opened a pull request:

https://github.com/apache/incubator-hawq/pull/856

HAWQ-984. hawq config is too slow.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/paul-guo-/incubator-hawq python

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/856.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #856


commit 7e74c67bc5be81bbe4aab6a654872d3b0ce39c90
Author: Paul Guo 
Date:   2016-08-24T09:48:14Z

HAWQ-984. hawq config is too slow.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-256) Integrate Security with Apache Ranger

[Vineet Goel (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15434467#comment-15434467
 ] 

Vineet Goel commented on HAWQ-256:
--

I found this in the Hive documentation:

"The ADMIN permission in Ranger is the equivalent to the WITH GRANT OPTION in 
SQL standard-based authorization. However, the ADMIN permission gives the 
grantee the ability to grant all permissions rather than just the permissions 
possessed by the grantor. With SQL standard-based authorization, the WITH GRANT 
OPTION applies only to permissions possessed by the grantor."

This seems to suggest that "WITH GRANT OPTION" doesn't translate into same 
behavior at the Ranger level. This is understandable and acceptable I think. 
Ranger users and Component (Hive or HAWQ) users are likely two separate groups 
and they don't need to cross in their functions. This likely means, WITH GRANT 
OPTION on the CLI probably doesn't propagate into any Ranger policy updates and 
is ignored?

Secondly, I'm late to this discussion, but it seems like [~bosco] was 
suggesting to design in such a way that "native component CLI commands" should 
not be encouraged, but rather, only Ranger UI/APIs should be used to set those 
policies (if Ranger authentication is switched ON in the component). If that's 
the case, I like that idea, to reduce design complexity. Hence, Authentication 
changes made with GRANT and REVOKE statements on component CLI must be disabled 
if Ranger authentication is switched ON. If Ranger is not in use, native 
component behavior remains unchanged. Users are expected not to flip back and 
forth between using Ranger and not using Ranger.

> Integrate Security with Apache Ranger
> -
>
> Key: HAWQ-256
> URL: https://issues.apache.org/jira/browse/HAWQ-256
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: PXF, Security
>Reporter: Michael Andre Pearce (IG)
>Assignee: Lili Ma
> Fix For: backlog
>
> Attachments: HAWQRangerSupportDesign.pdf
>
>
> Integrate security with Apache Ranger for a unified Hadoop security solution. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)