[GitHub] incubator-hawq pull request #1096: HAWQ-1282. Shared Input Scan may result i...

[zhangh43]

Github user zhangh43 commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1096#discussion_r96807336
  
--- Diff: src/include/executor/nodeShareInputScan.h ---
@@ -30,6 +30,8 @@
 #define NODESHAREINPUTSCAN_H
 
 #include "nodes/execnodes.h"
+
+#define MAX_RETRY_TIMES 5
--- End diff --

I remove timeout retry max times.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #1096: HAWQ-1282. Shared Input Scan may result in endle...

[linwen]

Github user linwen commented on the issue:

https://github.com/apache/incubator-hawq/pull/1096
  
+1 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1096: HAWQ-1282. Shared Input Scan may result i...

[zhangh43]

Github user zhangh43 commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1096#discussion_r96802952
  
--- Diff: src/include/executor/nodeShareInputScan.h ---
@@ -30,6 +30,8 @@
 #define NODESHAREINPUTSCAN_H
 
 #include "nodes/execnodes.h"
+
+#define MAX_RETRY_TIMES 5
--- End diff --

I think writer_wait_for_acks is not a must one. and for orca is on, 
writer_wait_for_acks is not called at all.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (HAWQ-1283) error: ‘privilege’ undeclared - when make project

[Xiang Sheng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiang Sheng updated HAWQ-1283:
--
Component/s: Security

> error: ‘privilege’ undeclared - when make project
> -
>
> Key: HAWQ-1283
> URL: https://issues.apache.org/jira/browse/HAWQ-1283
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Security
>Reporter: Devin Jia
>Assignee: Xiang Sheng
> Fix For: 2.2.0.0-incubating
>
>
> when i "git pull" the newest source code,and make project, encounter error:
> {quote}
> ./configure --prefix=/opt/hawq-build --enable-depend --enable-cassert 
> --enable-debug
> make install
> {quote}
> ..
> ..
> gcc -O0 -std=gnu99  -Wall -Wmissing-prototypes -Wpointer-arith  
> -Wendif-labels -Wformat-security -fno-strict-aliasing -fwrapv -g -ggdb  
> -I/usr/include/libxml2 -I../../../src/include -D_GNU_SOURCE  
> -I/opt/incubator-hawq/depends/libhdfs3/build/install/opt/hawq-build/include 
> -I/opt/incubator-hawq/depends/libyarn/build/install/opt/hawq-build/include  
> -c -o rangerrest.o rangerrest.c -MMD -MP -MF .deps/rangerrest.Po
> {color:red}
> rangerrest.c: In function ‘create_ranger_request_json’:
> rangerrest.c:165: error: ‘privilege’ undeclared (first use in this function)
> rangerrest.c:165: error: (Each undeclared identifier is reported only once
> rangerrest.c:165: error: for each function it appears in.)
> {color}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-1283) error: ‘privilege’ undeclared - when make project

[Xiang Sheng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiang Sheng updated HAWQ-1283:
--
Fix Version/s: 2.2.0.0-incubating

> error: ‘privilege’ undeclared - when make project
> -
>
> Key: HAWQ-1283
> URL: https://issues.apache.org/jira/browse/HAWQ-1283
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Security
>Reporter: Devin Jia
>Assignee: Xiang Sheng
> Fix For: 2.2.0.0-incubating
>
>
> when i "git pull" the newest source code,and make project, encounter error:
> {quote}
> ./configure --prefix=/opt/hawq-build --enable-depend --enable-cassert 
> --enable-debug
> make install
> {quote}
> ..
> ..
> gcc -O0 -std=gnu99  -Wall -Wmissing-prototypes -Wpointer-arith  
> -Wendif-labels -Wformat-security -fno-strict-aliasing -fwrapv -g -ggdb  
> -I/usr/include/libxml2 -I../../../src/include -D_GNU_SOURCE  
> -I/opt/incubator-hawq/depends/libhdfs3/build/install/opt/hawq-build/include 
> -I/opt/incubator-hawq/depends/libyarn/build/install/opt/hawq-build/include  
> -c -o rangerrest.o rangerrest.c -MMD -MP -MF .deps/rangerrest.Po
> {color:red}
> rangerrest.c: In function ‘create_ranger_request_json’:
> rangerrest.c:165: error: ‘privilege’ undeclared (first use in this function)
> rangerrest.c:165: error: (Each undeclared identifier is reported only once
> rangerrest.c:165: error: for each function it appears in.)
> {color}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HAWQ-1283) error: ‘privilege’ undeclared - when make project

[Xiang Sheng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiang Sheng reassigned HAWQ-1283:
-

Assignee: Xiang Sheng  (was: Ed Espino)

> error: ‘privilege’ undeclared - when make project
> -
>
> Key: HAWQ-1283
> URL: https://issues.apache.org/jira/browse/HAWQ-1283
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Devin Jia
>Assignee: Xiang Sheng
>
> when i "git pull" the newest source code,and make project, encounter error:
> {quote}
> ./configure --prefix=/opt/hawq-build --enable-depend --enable-cassert 
> --enable-debug
> make install
> {quote}
> ..
> ..
> gcc -O0 -std=gnu99  -Wall -Wmissing-prototypes -Wpointer-arith  
> -Wendif-labels -Wformat-security -fno-strict-aliasing -fwrapv -g -ggdb  
> -I/usr/include/libxml2 -I../../../src/include -D_GNU_SOURCE  
> -I/opt/incubator-hawq/depends/libhdfs3/build/install/opt/hawq-build/include 
> -I/opt/incubator-hawq/depends/libyarn/build/install/opt/hawq-build/include  
> -c -o rangerrest.o rangerrest.c -MMD -MP -MF .deps/rangerrest.Po
> {color:red}
> rangerrest.c: In function ‘create_ranger_request_json’:
> rangerrest.c:165: error: ‘privilege’ undeclared (first use in this function)
> rangerrest.c:165: error: (Each undeclared identifier is reported only once
> rangerrest.c:165: error: for each function it appears in.)
> {color}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1283) error: ‘privilege’ undeclared - when make project

[Xiang Sheng (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829427#comment-15829427
 ] 

Xiang Sheng commented on HAWQ-1283:
---

Hi [~jiadx]Do you pull the latest code from github  ?  
I just pulled the latest code and configure as you wrote "./configure 
--prefix=/opt/hawq-build --enable-depend --enable-cassert --enable-debug" . 
Then I do "make -j8" to compile and "make install" to install. All succeed. 

Please refer to  
https://github.com/apache/incubator-hawq/blob/master/src/backend/libpq/rangerrest.c
 and I don't find the undeclared variable.  Could you confirm that ? Or maybe 
there are some old objects file, you could try "make distclean" first and then 
configure & make & make install. 

Thanks

> error: ‘privilege’ undeclared - when make project
> -
>
> Key: HAWQ-1283
> URL: https://issues.apache.org/jira/browse/HAWQ-1283
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Devin Jia
>Assignee: Ed Espino
>
> when i "git pull" the newest source code,and make project, encounter error:
> {quote}
> ./configure --prefix=/opt/hawq-build --enable-depend --enable-cassert 
> --enable-debug
> make install
> {quote}
> ..
> ..
> gcc -O0 -std=gnu99  -Wall -Wmissing-prototypes -Wpointer-arith  
> -Wendif-labels -Wformat-security -fno-strict-aliasing -fwrapv -g -ggdb  
> -I/usr/include/libxml2 -I../../../src/include -D_GNU_SOURCE  
> -I/opt/incubator-hawq/depends/libhdfs3/build/install/opt/hawq-build/include 
> -I/opt/incubator-hawq/depends/libyarn/build/install/opt/hawq-build/include  
> -c -o rangerrest.o rangerrest.c -MMD -MP -MF .deps/rangerrest.Po
> {color:red}
> rangerrest.c: In function ‘create_ranger_request_json’:
> rangerrest.c:165: error: ‘privilege’ undeclared (first use in this function)
> rangerrest.c:165: error: (Each undeclared identifier is reported only once
> rangerrest.c:165: error: for each function it appears in.)
> {color}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #731: HAWQ-830. Fix wrong result in CTE query due to CT...

[huor]

Github user huor commented on the issue:

https://github.com/apache/incubator-hawq/pull/731
  
Close the pr for now. Need to merge the fix after wrong result and hang in 
CTE query is fixed. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #731: HAWQ-830. Fix wrong result in CTE query du...

[huor]

Github user huor closed the pull request at:

https://github.com/apache/incubator-hawq/pull/731


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-1087) Need clarification on Wiki page for Build Instructions

[Ruilong Huo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829421#comment-15829421
 ] 

Ruilong Huo commented on HAWQ-1087:
---

[~espino], did you get a chance to help David to get access of hawq wiki?

> Need clarification on Wiki page for Build Instructions
> --
>
> Key: HAWQ-1087
> URL: https://issues.apache.org/jira/browse/HAWQ-1087
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Documentation
>Reporter: Goden Yao
>Assignee: David Yozie
>Priority: Minor
> Fix For: backlog
>
>
> From [~jmclean], this is during a VOTE review. 
> {quote}
> Also currently looks like openssl may be a little broken and info on the wiki 
> may need updating:
> {code}
> brew link --force openssl
> Warning: Refusing to link: openssl
> {code}
> Linking keg-only openssl means you may end up linking against the insecure,
> deprecated system OpenSSL while using the headers from Homebrew's openssl.
> Instead, pass the full include/library paths to your compiler e.g.:
>  {code} -I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib{code}
> I needed up doing this to make it compile:
> make CFLAGS="-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib"
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HAWQ-1278) Investigate installcheck-good issue on Mac OSX

[Ruilong Huo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829417#comment-15829417
 ] 

Ruilong Huo edited comment on HAWQ-1278 at 1/19/17 6:58 AM:


[~espino], did you get a chance to help David to get access of hawq wiki?


was (Author: huor):
[~espino], did you get a chance to grant access of hawq wiki to David?

> Investigate installcheck-good issue on Mac OSX
> --
>
> Key: HAWQ-1278
> URL: https://issues.apache.org/jira/browse/HAWQ-1278
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Tests
>Reporter: Ed Espino
>Assignee: Ruilong Huo
> Fix For: 2.2.0.0-incubating
>
>
> I am filing this as a place holder for the Mac OSX installcheck-good 
> investigation work.  Ming Li originally reported installcheck-good testing 
> issues with errtable and hcatalog_lookup test suites.
> This issue is not seen on CentOS 6 & 7 environments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1278) Investigate installcheck-good issue on Mac OSX

[Ruilong Huo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829417#comment-15829417
 ] 

Ruilong Huo commented on HAWQ-1278:
---

[~espino], did you get a chance to grant access of hawq wiki to David?

> Investigate installcheck-good issue on Mac OSX
> --
>
> Key: HAWQ-1278
> URL: https://issues.apache.org/jira/browse/HAWQ-1278
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Tests
>Reporter: Ed Espino
>Assignee: Ruilong Huo
> Fix For: 2.2.0.0-incubating
>
>
> I am filing this as a place holder for the Mac OSX installcheck-good 
> investigation work.  Ming Li originally reported installcheck-good testing 
> issues with errtable and hcatalog_lookup test suites.
> This issue is not seen on CentOS 6 & 7 environments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-1283) error: ‘privilege’ undeclared - when make project

[Devin Jia (JIRA)]

Devin Jia created HAWQ-1283:
---

 Summary: error: ‘privilege’ undeclared - when make project
 Key: HAWQ-1283
 URL: https://issues.apache.org/jira/browse/HAWQ-1283
 Project: Apache HAWQ
  Issue Type: Bug
Reporter: Devin Jia
Assignee: Ed Espino


when i "git pull" the newest source code,and make project, encounter error:

{quote}
./configure --prefix=/opt/hawq-build --enable-depend --enable-cassert 
--enable-debug
make install
{quote}
..
..
gcc -O0 -std=gnu99  -Wall -Wmissing-prototypes -Wpointer-arith  -Wendif-labels 
-Wformat-security -fno-strict-aliasing -fwrapv -g -ggdb  -I/usr/include/libxml2 
-I../../../src/include -D_GNU_SOURCE  
-I/opt/incubator-hawq/depends/libhdfs3/build/install/opt/hawq-build/include 
-I/opt/incubator-hawq/depends/libyarn/build/install/opt/hawq-build/include  -c 
-o rangerrest.o rangerrest.c -MMD -MP -MF .deps/rangerrest.Po
{color:red}
rangerrest.c: In function ‘create_ranger_request_json’:
rangerrest.c:165: error: ‘privilege’ undeclared (first use in this function)
rangerrest.c:165: error: (Each undeclared identifier is reported only once
rangerrest.c:165: error: for each function it appears in.)
{color}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-1193) TDE support in HAWQ

[Hongxu Ma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongxu Ma updated HAWQ-1193:

Assignee: Ivan Weng  (was: Hongxu Ma)

> TDE support in HAWQ
> ---
>
> Key: HAWQ-1193
> URL: https://issues.apache.org/jira/browse/HAWQ-1193
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: libhdfs
>Reporter: Hongxu Ma
>Assignee: Ivan Weng
> Fix For: backlog
>
> Attachments: HAWQ_TDE_Design_ver0.1.pdf, HAWQ_TDE_Design_ver0.2 .pdf
>
>
>  TDE(transparently data encrypted) has been supported after hadoop 2.6:
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html
> https://issues.apache.org/jira/browse/HDFS-6134
> Use TDE can promise：
> 1, hdfs file is encrypted.
> 2, network transfer between hdfs and libhdfs client is encrypted.
> So hawq will update libhdfs3 to support it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #1096: HAWQ-1282. Shared Input Scan may result i...

[ictmalili]

Github user ictmalili commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1096#discussion_r96797703
  
--- Diff: src/include/executor/nodeShareInputScan.h ---
@@ -30,6 +30,8 @@
 #define NODESHAREINPUTSCAN_H
 
 #include "nodes/execnodes.h"
+
+#define MAX_RETRY_TIMES 5
--- End diff --

Is it appropriate to set the value as 5? What if the network environment is 
bad?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (HAWQ-1194) Add EncryptionZones related RPC

[Hongxu Ma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongxu Ma updated HAWQ-1194:

Assignee: Amy  (was: Hongxu Ma)

> Add EncryptionZones related RPC
> ---
>
> Key: HAWQ-1194
> URL: https://issues.apache.org/jira/browse/HAWQ-1194
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: libhdfs
>Reporter: Hongxu Ma
>Assignee: Amy
> Fix For: backlog
>
>
> Add createEncryption, getEZForPath, listEncryptionZones RPC



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-1194) Add EncryptionZones related RPC

[Hongxu Ma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongxu Ma updated HAWQ-1194:

Description: Add createEncryption, getEZForPath, listEncryptionZones RPC

> Add EncryptionZones related RPC
> ---
>
> Key: HAWQ-1194
> URL: https://issues.apache.org/jira/browse/HAWQ-1194
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: libhdfs
>Reporter: Hongxu Ma
>Assignee: Hongxu Ma
> Fix For: backlog
>
>
> Add createEncryption, getEZForPath, listEncryptionZones RPC



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-1194) Add EncryptionZones related RPC

[Hongxu Ma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongxu Ma updated HAWQ-1194:

Summary: Add EncryptionZones related RPC  (was: add EncryptionZones related 
RPC)

> Add EncryptionZones related RPC
> ---
>
> Key: HAWQ-1194
> URL: https://issues.apache.org/jira/browse/HAWQ-1194
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: libhdfs
>Reporter: Hongxu Ma
>Assignee: Hongxu Ma
> Fix For: backlog
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-1194) add EncryptionZones related RPC

[Hongxu Ma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongxu Ma updated HAWQ-1194:

Summary: add EncryptionZones related RPC  (was:  Todo)

> add EncryptionZones related RPC
> ---
>
> Key: HAWQ-1194
> URL: https://issues.apache.org/jira/browse/HAWQ-1194
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: libhdfs
>Reporter: Hongxu Ma
>Assignee: Hongxu Ma
> Fix For: backlog
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-1279) Force to recompute namespace_path when enable_ranger

[Hongxu Ma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongxu Ma closed HAWQ-1279.
---
Resolution: Fixed

fixed

> Force to recompute namespace_path when enable_ranger
> 
>
> Key: HAWQ-1279
> URL: https://issues.apache.org/jira/browse/HAWQ-1279
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: PXF, Security
>Reporter: Hongxu Ma
>Assignee: Hongxu Ma
> Fix For: backlog
>
>
> namespace_path is cached in each psql session and the cache invalidation is 
> triggered by Grant/Revoke SQL.
> When enable_ranger, Grant/Revoke SQL is no longer use, so the cache prevent a 
> ack-check request sending.
> Example:
> {code}
> // create table t(i int); => failed
> [{""resource"":{""database"":""postgres"",""schema"":""pg_catalog""},""privileges"":[""usage""],""allowed"":true}]
> [{""resource"":{""database"":""postgres"",""schema"":""pg_catalog""},""privileges"":[""usage""],""allowed"":true}]
> [{""resource"":{""database"":""postgres"",""schema"":""public""},""privileges"":[""usage""],""allowed"":false}]
> // grant usage and create permissions to public schema in ranger and try 
> again => failed again
> [{""resource"":{""database"":""postgres"",""schema"":""pg_catalog""},""privileges"":[""usage""],""allowed"":true}]
> [{""resource"":{""database"":""postgres"",""schema"":""pg_catalog""},""privileges"":[""usage""],""allowed"":true}]
> // why not send a request for USAGE??
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #1096: HAWQ-1282. Shared Input Scan may result i...

[zhangh43]

GitHub user zhangh43 opened a pull request:

https://github.com/apache/incubator-hawq/pull/1096

HAWQ-1282. Shared Input Scan may result in endless loop.

There are residual process after running some queries. Through the call 
stack, we find that there is an endless loop in function writer_wait_for_acks() 
in shared input scan.
This PR includes:
1 add max retry times to avoid this problem.
2 fix file handler leak in retry_read() and retry_write() of shared input 
scan.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zhangh43/incubator-hawq hawq1282

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/1096.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1096


commit 0fef36c5f9dbd39d9cee99170886ce034c67f992
Author: hubertzhang 
Date:   2017-01-19T05:58:48Z

HAWQ-1282. Shared Input Scan may result in endless loop.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1094: HAWQ-1279. Force to recompute namespace_p...

[interma]

Github user interma closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1094


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #1094: HAWQ-1279. Force to recompute namespace_path whe...

[interma]

Github user interma commented on the issue:

https://github.com/apache/incubator-hawq/pull/1094
  
Already merged.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (HAWQ-1282) Shared Input Scan may result in endless loop.

[Hubert Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hubert Zhang updated HAWQ-1282:
---
Summary: Shared Input Scan may result in endless loop.  (was: Shared Input 
Scan may result in endless loop)

> Shared Input Scan may result in endless loop.
> -
>
> Key: HAWQ-1282
> URL: https://issues.apache.org/jira/browse/HAWQ-1282
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core
>Reporter: Hubert Zhang
>Assignee: Hubert Zhang
>
> There are residual process after running some queries.  Through the call 
> stack, we find that there is an endless loop in function 
> writer_wait_for_acks() in shared input scan.
> We plan to add max retry times to avoid this problem.
> Also, there are some file handler leaks in retry_read() and retry_write() of 
> shared input scan.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-1282) Shared Input Scan may result in endless loop

[Hubert Zhang (JIRA)]

Hubert Zhang created HAWQ-1282:
--

 Summary: Shared Input Scan may result in endless loop
 Key: HAWQ-1282
 URL: https://issues.apache.org/jira/browse/HAWQ-1282
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Core
Reporter: Hubert Zhang
Assignee: Ed Espino


There are residual process after running some queries.  Through the call stack, 
we find that there is an endless loop in function writer_wait_for_acks() in 
shared input scan.
We plan to add max retry times to avoid this problem.
Also, we fix file handler leak in retry_read and retry_write.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-1282) Shared Input Scan may result in endless loop

[Hubert Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hubert Zhang updated HAWQ-1282:
---
Description: 
There are residual process after running some queries.  Through the call stack, 
we find that there is an endless loop in function writer_wait_for_acks() in 
shared input scan.
We plan to add max retry times to avoid this problem.
Also, there are some file handler leaks in retry_read() and retry_write() of 
shared input scan.

  was:
There are residual process after running some queries.  Through the call stack, 
we find that there is an endless loop in function writer_wait_for_acks() in 
shared input scan.
We plan to add max retry times to avoid this problem.
Also, we fix file handler leak in retry_read and retry_write.


> Shared Input Scan may result in endless loop
> 
>
> Key: HAWQ-1282
> URL: https://issues.apache.org/jira/browse/HAWQ-1282
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core
>Reporter: Hubert Zhang
>Assignee: Hubert Zhang
>
> There are residual process after running some queries.  Through the call 
> stack, we find that there is an endless loop in function 
> writer_wait_for_acks() in shared input scan.
> We plan to add max retry times to avoid this problem.
> Also, there are some file handler leaks in retry_read() and retry_write() of 
> shared input scan.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HAWQ-1282) Shared Input Scan may result in endless loop

[Hubert Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hubert Zhang reassigned HAWQ-1282:
--

Assignee: Hubert Zhang  (was: Ed Espino)

> Shared Input Scan may result in endless loop
> 
>
> Key: HAWQ-1282
> URL: https://issues.apache.org/jira/browse/HAWQ-1282
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core
>Reporter: Hubert Zhang
>Assignee: Hubert Zhang
>
> There are residual process after running some queries.  Through the call 
> stack, we find that there is an endless loop in function 
> writer_wait_for_acks() in shared input scan.
> We plan to add max retry times to avoid this problem.
> Also, we fix file handler leak in retry_read and retry_write.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #1095: HAWQ-1276. hawq should error out directly...

[stanlyxiang]

Github user stanlyxiang closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1095


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #1095: HAWQ-1276. hawq should error out directly when r...

[stanlyxiang]

Github user stanlyxiang commented on the issue:

https://github.com/apache/incubator-hawq/pull/1095
  
merged into master. closed. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-24) Support superuser to GRANT/REVOKE CREATION privilege to/from non-superuser on TABLESPACE

[MongJu Jung (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-24?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829370#comment-15829370
 ] 

MongJu Jung commented on HAWQ-24:
-

the below method works.

PGOPTIONS='-c gp_session_role=utility' psql

hope this helps!!

> Support superuser to GRANT/REVOKE CREATION privilege to/from non-superuser on 
> TABLESPACE
> 
>
> Key: HAWQ-24
> URL: https://issues.apache.org/jira/browse/HAWQ-24
> Project: Apache HAWQ
>  Issue Type: Wish
>  Components: DDL, Storage
>Reporter: Ruilong Huo
>Assignee: Lei Chang
> Fix For: backlog
>
>
> It raises error "Cannot support GRANT/REVOKE on TABLESPACE statement" while 
> following the HAWQ guide 
> (http://hawq.docs.pivotal.io/docs-gpdb/admin_guide/ddl/ddl-tablespace.html) 
> to GRANT/REVOKE CREATION privilege to/from non-superuser on TABLESPACE.
> {code}
> gpadmin=# GRANT CREATE ON TABLESPACE fstbs TO tstuser;
> ERROR:  Cannot support GRANT/REVOKE on TABLESPACE statement
> {code}
> As a consequence, with the user as SUPERUSER is possible to create tables on 
> top of the tablespace, but with a user as NOSUPERUSER its not possible:
> {code}
> tstuser=> CREATE TABLE testfs3 ( col01 INTEGER ) TABLESPACE fstbs;
> NOTICE:  Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 
> 'col01' as the Greenplum Database data distribution key for this table.
> HINT:  The 'DISTRIBUTED BY' clause determines the distribution of data. Make 
> sure column(s) chosen are the optimal data distribution key to minimize skew.
> ERROR:  permission denied for tablespace fstbs
> {code}
>  
> {code}
> gpadmin=# alter user tstuser with superuser;
> ALTER ROLE
> [gpadmin@ai2hdm1 ~]$ psql -d tstuser -U tstuser
> Password for user tstuser: 
> psql (8.2.15)
> Type "help" for help.
> tstuser=# CREATE TABLE testfs3 ( col01 INTEGER ) TABLESPACE fstbs;
> NOTICE:  Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 
> 'col01' as the Greenplum Database data distribution key for this table.
> HINT:  The 'DISTRIBUTED BY' clause determines the distribution of data. Make 
> sure column(s) chosen are the optimal data distribution key to minimize skew.
> CREATE TABLE
> {code}
> Due to security consideration, it is not acceptable for some HAWQ users to 
> always use SUPERUSER to create TABLESPACE.  Thus, we need to support:
> 1. Superuser can GRANT/REVOKE CREATION privilege to/from non-superuser on 
> TABLESPACE.
> 2. Non-supuser can create TABLESPACE once it is granted with creation 
> privilege.
> 3. Non-superuser to GRANT/REVOKE CREATION privilege on TABLESPACE to other 
> users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-1276) The error message is not friendly when ranger plugin service is unavailable.

[Xiang Sheng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiang Sheng closed HAWQ-1276.
-

> The error message is not friendly when ranger plugin service is unavailable.
> 
>
> Key: HAWQ-1276
> URL: https://issues.apache.org/jira/browse/HAWQ-1276
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Security
>Reporter: Xiang Sheng
>Assignee: Xiang Sheng
> Fix For: 2.2.0.0-incubating
>
>
> The error message is not friendly when RPS cannot be connected.
> We expect a more clear message printed out, for example, "cannot connect to 
> RPS service: host/port/service  ".
> What's more, there are many other error and log messages and log level should 
> be refined to more friendly. Some unused log should be removed. 
> {code}
> MacBook-Pro:incubator-hawq malili$ psql -d postgres
> psql (8.2.15)
> Type "help" for help.
> postgres=# \l
> WARNING:  curl_easy_perform() failed: Couldn't connect to server
> LINE 5: FROM pg_catalog.pg_database d
>  ^
> ERROR:  permission denied for schema pg_catalog
> LINE 5: FROM pg_catalog.pg_database d
>  ^
> postgres=#
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-1276) The error message is not friendly when ranger plugin service is unavailable.

[Xiang Sheng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiang Sheng resolved HAWQ-1276.
---
Resolution: Fixed

> The error message is not friendly when ranger plugin service is unavailable.
> 
>
> Key: HAWQ-1276
> URL: https://issues.apache.org/jira/browse/HAWQ-1276
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Security
>Reporter: Xiang Sheng
>Assignee: Xiang Sheng
> Fix For: 2.2.0.0-incubating
>
>
> The error message is not friendly when RPS cannot be connected.
> We expect a more clear message printed out, for example, "cannot connect to 
> RPS service: host/port/service  ".
> What's more, there are many other error and log messages and log level should 
> be refined to more friendly. Some unused log should be removed. 
> {code}
> MacBook-Pro:incubator-hawq malili$ psql -d postgres
> psql (8.2.15)
> Type "help" for help.
> postgres=# \l
> WARNING:  curl_easy_perform() failed: Couldn't connect to server
> LINE 5: FROM pg_catalog.pg_database d
>  ^
> ERROR:  permission denied for schema pg_catalog
> LINE 5: FROM pg_catalog.pg_database d
>  ^
> postgres=#
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-256) Integrate Security with Apache Ranger

[Lili Ma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lili Ma updated HAWQ-256:
-
Component/s: (was: PXF)

> Integrate Security with Apache Ranger
> -
>
> Key: HAWQ-256
> URL: https://issues.apache.org/jira/browse/HAWQ-256
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: Security
>Reporter: Michael Andre Pearce (IG)
>Assignee: Lili Ma
> Fix For: backlog
>
> Attachments: HAWQRangerSupportDesign.pdf, 
> HAWQRangerSupportDesign_v0.2.pdf, HAWQRangerSupportDesign_v0.3.pdf
>
>
> Integrate security with Apache Ranger for a unified Hadoop security solution. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #1092: HAWQ-1203. Ranger Plugin Service Implemen...

[ictmalili]

Github user ictmalili commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1092#discussion_r96780739
  
--- Diff: ranger-plugin/scripts/register_hawq.sh ---
@@ -0,0 +1,217 @@
+#!/usr/bin/env bash
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+function usage() {
+  echo "USAGE: register_hawq.sh -r ranger_host:ranger_port -u ranger_user 
-p ranger_password -h hawq_host:hawq_port -w hawq_user -q hawq_password"
--- End diff --

@denalex May we change it adding indication the meaning to register to 
Ranger, such as  register_hawq_to_ranger.sh, or anything else?   Because HAWQ 
now has a management tool named as hawq register


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (HAWQ-1281) Add integration tests to Ranger Plugin Service project

[Alexander Denissov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Denissov reassigned HAWQ-1281:


Assignee: Alexander Denissov  (was: Ed Espino)

> Add integration tests to Ranger Plugin Service project
> --
>
> Key: HAWQ-1281
> URL: https://issues.apache.org/jira/browse/HAWQ-1281
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Security
>Reporter: Alexander Denissov
>Assignee: Alexander Denissov
> Fix For: backlog
>
>
> /ranger-plugin/service module need additional integration test cases:
> - test cases to check policies with * values for parent resources and 
> "public" for group.
> - test cases for isAccessAllowed call for schema
> - test cases for isAccessAllowed call for table
> - test cases for isAccessAllowed call for sequence



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-1281) Add integration tests to Ranger Plugin Service project

[Alexander Denissov (JIRA)]

Alexander Denissov created HAWQ-1281:


 Summary: Add integration tests to Ranger Plugin Service project
 Key: HAWQ-1281
 URL: https://issues.apache.org/jira/browse/HAWQ-1281
 Project: Apache HAWQ
  Issue Type: Sub-task
  Components: Security
Reporter: Alexander Denissov
Assignee: Ed Espino


/ranger-plugin/service module need additional integration test cases:
- test cases to check policies with * values for parent resources and "public" 
for group.
- test cases for isAccessAllowed call for schema
- test cases for isAccessAllowed call for table
- test cases for isAccessAllowed call for sequence



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-1203) Implement Ranger Plugin Service which holds HAWQ Ranger Plugin and provide REST Service

[Alexander Denissov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Denissov resolved HAWQ-1203.
--
   Resolution: Fixed
Fix Version/s: (was: backlog)
   2.2.0.0-incubating

> Implement Ranger Plugin Service which holds HAWQ Ranger Plugin and provide 
> REST Service
> ---
>
> Key: HAWQ-1203
> URL: https://issues.apache.org/jira/browse/HAWQ-1203
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Security
>Reporter: Lili Ma
>Assignee: Alexander Denissov
> Fix For: 2.2.0.0-incubating
>
> Attachments: RangerPluginServiceRPS.pdf
>
>
> Per design, we want to create a separate RPS service which hosts HAWQ Ranger 
> plugin service and in charge of handling HAWQ ACL request and periodically 
> fetch policies from Ranger server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-1119) create new documentation topic for PXF writable profiles

[Lisa Owen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lisa Owen closed HAWQ-1119.
---

> create new documentation topic for PXF writable profiles
> 
>
> Key: HAWQ-1119
> URL: https://issues.apache.org/jira/browse/HAWQ-1119
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lisa Owen
>Assignee: David Yozie
>Priority: Minor
> Fix For: 2.2.0.0-incubating
>
>
> certain profiles supported by the existing PXF plug-ins support writable 
> tables.  create some documentation content for these profiles.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1119) create new documentation topic for PXF writable profiles

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828830#comment-15828830
 ] 

ASF GitHub Bot commented on HAWQ-1119:
--

Github user lisakowen closed the pull request at:

https://github.com/apache/incubator-hawq-docs/pull/46


> create new documentation topic for PXF writable profiles
> 
>
> Key: HAWQ-1119
> URL: https://issues.apache.org/jira/browse/HAWQ-1119
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lisa Owen
>Assignee: David Yozie
>Priority: Minor
> Fix For: 2.2.0.0-incubating
>
>
> certain profiles supported by the existing PXF plug-ins support writable 
> tables.  create some documentation content for these profiles.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #731: HAWQ-830. Fix wrong result in CTE query due to CT...

[edespino]

Github user edespino commented on the issue:

https://github.com/apache/incubator-hawq/pull/731
  
@huor - Yes let's close this PR for now.  Thanks, -=e


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1092: HAWQ-1203. Ranger Plugin Service Implemen...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1092


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-1119) create new documentation topic for PXF writable profiles

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828539#comment-15828539
 ] 

ASF GitHub Bot commented on HAWQ-1119:
--

Github user lisakowen commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r96705982
  
--- Diff: markdown/pxf/HDFSWritablePXF.html.md.erb ---
@@ -396,7 +394,7 @@ The field type of `recordkey` must correspond to the 
key type, much as the other
 
 ### Example: Using Record Keys
 
-Create an external readable table to access the record keys from the 
writable table `pxf_tbl_seqwrit` that you created in [Example: Writing Data 
Using the SequenceWritable Profile](#profile_hdfsseqwritable_writing).  The 
`recordkey` is of type `int8`. (We did not identify a record key when creating 
the writable table.
+Create an external readable table to access the record keys from the 
writable table `pxf_tbl_seqwrit` that you created in [Example: Writing Data 
Using the SequenceWritable Profile](#profile_hdfsseqwritable_writing).  The 
`recordkey` is of type `int8`. (You did not identify a record key when you 
created the writable table.
--- End diff --

those two sentences are not directly related.  the type of recordkey will 
always be int8.  i am going to just remove the "You did not identify a record 
key..." sentence as a variant is used after the example.



> create new documentation topic for PXF writable profiles
> 
>
> Key: HAWQ-1119
> URL: https://issues.apache.org/jira/browse/HAWQ-1119
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lisa Owen
>Assignee: David Yozie
>Priority: Minor
> Fix For: 2.2.0.0-incubating
>
>
> certain profiles supported by the existing PXF plug-ins support writable 
> tables.  create some documentation content for these profiles.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1119) create new documentation topic for PXF writable profiles

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828529#comment-15828529
 ] 

ASF GitHub Bot commented on HAWQ-1119:
--

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r96702964
  
--- Diff: markdown/pxf/HDFSWritablePXF.html.md.erb ---
@@ -413,4 +411,4 @@ gpadmin=# SELECT * FROM read_pxf_tbl_seqwrit_RECKEY;
 (2 rows)
 ```
 
-The `recordkey` is displayed as `0`; a record key was not identified when 
you inserted entries in the writable table.
+The `recordkey` is displayed as `0`; you did not identify a record key 
when you inserted entries in the writable table.
--- End diff --

Edit:  The `recordkey` value is `0` because you did not identify a record 
key when you inserted entries into the writable table.


> create new documentation topic for PXF writable profiles
> 
>
> Key: HAWQ-1119
> URL: https://issues.apache.org/jira/browse/HAWQ-1119
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lisa Owen
>Assignee: David Yozie
>Priority: Minor
> Fix For: 2.2.0.0-incubating
>
>
> certain profiles supported by the existing PXF plug-ins support writable 
> tables.  create some documentation content for these profiles.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1119) create new documentation topic for PXF writable profiles

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828530#comment-15828530
 ] 

ASF GitHub Bot commented on HAWQ-1119:
--

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r96702640
  
--- Diff: markdown/pxf/HDFSWritablePXF.html.md.erb ---
@@ -136,7 +136,7 @@ Perform the following operations to use the PXF 
`HdfsTextSimple` profile to crea
FORMAT 'CSV';
 ```
 
-The table is created with the `'CSV'` `FORMAT` because the delimiter 
character used when creating the writable table was a comma `,`.
+The table is created with the `'CSV'` `FORMAT` because the delimiter 
character with which you created the writable table was a comma `,`.
--- End diff --

This sounds a little awkward.  Edit to:

"Specify the the `'CSV'` `FORMAT` for the readable table, because you 
created the writable table with a comma ',' as the delimiter character.


> create new documentation topic for PXF writable profiles
> 
>
> Key: HAWQ-1119
> URL: https://issues.apache.org/jira/browse/HAWQ-1119
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lisa Owen
>Assignee: David Yozie
>Priority: Minor
> Fix For: 2.2.0.0-incubating
>
>
> certain profiles supported by the existing PXF plug-ins support writable 
> tables.  create some documentation content for these profiles.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1119) create new documentation topic for PXF writable profiles

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828528#comment-15828528
 ] 

ASF GitHub Bot commented on HAWQ-1119:
--

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r96702781
  
--- Diff: markdown/pxf/HDFSWritablePXF.html.md.erb ---
@@ -396,7 +394,7 @@ The field type of `recordkey` must correspond to the 
key type, much as the other
 
 ### Example: Using Record Keys
 
-Create an external readable table to access the record keys from the 
writable table `pxf_tbl_seqwrit` that you created in [Example: Writing Data 
Using the SequenceWritable Profile](#profile_hdfsseqwritable_writing).  The 
`recordkey` is of type `int8`. (We did not identify a record key when creating 
the writable table.
+Create an external readable table to access the record keys from the 
writable table `pxf_tbl_seqwrit` that you created in [Example: Writing Data 
Using the SequenceWritable Profile](#profile_hdfsseqwritable_writing).  The 
`recordkey` is of type `int8`. (You did not identify a record key when you 
created the writable table.
--- End diff --

Edit last sentence:  The `recordkey` is of type `int8` because you did not 
identify a record key when you created the writable table.


> create new documentation topic for PXF writable profiles
> 
>
> Key: HAWQ-1119
> URL: https://issues.apache.org/jira/browse/HAWQ-1119
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lisa Owen
>Assignee: David Yozie
>Priority: Minor
> Fix For: 2.2.0.0-incubating
>
>
> certain profiles supported by the existing PXF plug-ins support writable 
> tables.  create some documentation content for these profiles.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1278) Investigate installcheck-good issue on Mac OSX

[David Yozie (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828478#comment-15828478
 ] 

David Yozie commented on HAWQ-1278:
---

[~espino][~huor] - It looks like we're still using the wiki 
(https://cwiki.apache.org/confluence/display/HAWQ/HAWQ+Release+2.1.0.0-incubating+Release?show-miniview)
 for publishing release notes.  I'm still trying to edit permissions on the 
wiki.  Nobody on the docs team can edit the wiki content :(

> Investigate installcheck-good issue on Mac OSX
> --
>
> Key: HAWQ-1278
> URL: https://issues.apache.org/jira/browse/HAWQ-1278
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Tests
>Reporter: Ed Espino
>Assignee: Ruilong Huo
> Fix For: 2.2.0.0-incubating
>
>
> I am filing this as a place holder for the Mac OSX installcheck-good 
> investigation work.  Ming Li originally reported installcheck-good testing 
> issues with errtable and hcatalog_lookup test suites.
> This issue is not seen on CentOS 6 & 7 environments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1087) Need clarification on Wiki page for Build Instructions

[David Yozie (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828465#comment-15828465
 ] 

David Yozie commented on HAWQ-1087:
---

[~huor] - I still don't seem to have access to the wiki.  Clicking the edit 
button will only allow me to add labels, not edit the content.

> Need clarification on Wiki page for Build Instructions
> --
>
> Key: HAWQ-1087
> URL: https://issues.apache.org/jira/browse/HAWQ-1087
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Documentation
>Reporter: Goden Yao
>Assignee: David Yozie
>Priority: Minor
> Fix For: backlog
>
>
> From [~jmclean], this is during a VOTE review. 
> {quote}
> Also currently looks like openssl may be a little broken and info on the wiki 
> may need updating:
> {code}
> brew link --force openssl
> Warning: Refusing to link: openssl
> {code}
> Linking keg-only openssl means you may end up linking against the insecure,
> deprecated system OpenSSL while using the headers from Homebrew's openssl.
> Instead, pass the full include/library paths to your compiler e.g.:
>  {code} -I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib{code}
> I needed up doing this to make it compile:
> make CFLAGS="-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib"
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HAWQ-1280) Add documentation for gp_interconnect* parameters

[David Yozie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Yozie reassigned HAWQ-1280:
-

Assignee: David Yozie  (was: Ed Espino)

> Add documentation for  gp_interconnect* parameters
> --
>
> Key: HAWQ-1280
> URL: https://issues.apache.org/jira/browse/HAWQ-1280
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Jane Beckman
>Assignee: David Yozie
>Priority: Trivial
> Fix For: 2.2.0.0-incubating
>
>
> The following configuration parameters are present in HAWQ and need to be 
> added to the documentation:
> gp_interconnect_cache_future_packets
> gp_interconnect_default_rtt
> gp_interconnect_min_retries_before_timeout
> gp_interconnect_min_rto
> gp_interconnect_timer_checking_period
> gp_interconnect_timer_period
> gp_interconnect_transmit_timeout



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-1280) Add documentation for gp_interconnect* parameters

[Jane Beckman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jane Beckman updated HAWQ-1280:
---
Priority: Trivial  (was: Major)

> Add documentation for  gp_interconnect* parameters
> --
>
> Key: HAWQ-1280
> URL: https://issues.apache.org/jira/browse/HAWQ-1280
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Jane Beckman
>Assignee: Ed Espino
>Priority: Trivial
> Fix For: 2.2.0.0-incubating
>
>
> The following configuration parameters are present in HAWQ and need to be 
> added to the documentation:
> gp_interconnect_cache_future_packets
> gp_interconnect_default_rtt
> gp_interconnect_min_retries_before_timeout
> gp_interconnect_min_rto
> gp_interconnect_timer_checking_period
> gp_interconnect_timer_period
> gp_interconnect_transmit_timeout



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-1280) Add documentation for gp_interconnect* parameters

[Jane Beckman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jane Beckman updated HAWQ-1280:
---
Component/s: Documentation

> Add documentation for  gp_interconnect* parameters
> --
>
> Key: HAWQ-1280
> URL: https://issues.apache.org/jira/browse/HAWQ-1280
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Jane Beckman
>Assignee: Ed Espino
> Fix For: 2.2.0.0-incubating
>
>
> The following configuration parameters are present in HAWQ and need to be 
> added to the documentation:
> gp_interconnect_cache_future_packets
> gp_interconnect_default_rtt
> gp_interconnect_min_retries_before_timeout
> gp_interconnect_min_rto
> gp_interconnect_timer_checking_period
> gp_interconnect_timer_period
> gp_interconnect_transmit_timeout



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #731: HAWQ-830. Fix wrong result in CTE query due to CT...

[huor]

Github user huor commented on the issue:

https://github.com/apache/incubator-hawq/pull/731
  
@edespino, it is better to merge this fix along with extra fix for wrong 
result and hang in CTE query. Given there is a lot of effort to fix the wrong 
result and hang in CTE, I think we can close this pr without merge it for now. 
We can do that when they are fixed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #1093: HAWQ-1193. Add createEncryption, getEZForPath, l...

[jiny2]

Github user jiny2 commented on the issue:

https://github.com/apache/incubator-hawq/pull/1093
  
+1 LGTM, please add some comments to explain the logic if necessary. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Comment Edited] (HAWQ-1278) Investigate installcheck-good issue on Mac OSX

[Ruilong Huo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827764#comment-15827764
 ] 

Ruilong Huo edited comment on HAWQ-1278 at 1/18/17 10:02 AM:
-

[~espino] and [~dyozie], per investigation with [~abai] and [~liming01], the 
root cause is that: 1) error table suite failure is due to gpfidst issue; 2) 
gpfdist failure is due to the client receive no response from gpfdist service 
after the http request is sent. To be specific, there is a bug in curl-7.51.0 
on osx 10.12 while curl-7.43.0 is good on that.

As for doc, we need to add below information:
{noformat}
Know issue: HAWQ-1278: curl 7.51.0 bug on osx 10.12 introduce gpfdist client 
fail to get response from gpfdist service after http request is sent.
Solution: use curl 7.43
{noformat}


was (Author: huor):
[~espino] and [~dyozie], per investigation with [~abai] and [~liming01], the 
root cause is that: 1) error table suite failure is due to gpfidst issue; 2) 
gpfdist failure is due to the client receive no response from gpfdist service 
after the http request is sent. To be specific, there is a bug in curl-7.51.0 
on osx 10.12 while curl-7.43.0 is good on that.

As for doc, we need to add below information:
{noformat}
Know issue: HAWQ-1278: curl 7.51.0 bug on osx 10.12 introduce gpfdist client 
fail to get response from gpfdist service after http request is sent.
The solution: use curl 7.43
{noformat}

> Investigate installcheck-good issue on Mac OSX
> --
>
> Key: HAWQ-1278
> URL: https://issues.apache.org/jira/browse/HAWQ-1278
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Tests
>Reporter: Ed Espino
>Assignee: Ruilong Huo
> Fix For: 2.2.0.0-incubating
>
>
> I am filing this as a place holder for the Mac OSX installcheck-good 
> investigation work.  Ming Li originally reported installcheck-good testing 
> issues with errtable and hcatalog_lookup test suites.
> This issue is not seen on CentOS 6 & 7 environments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1278) Investigate installcheck-good issue on Mac OSX

[Ruilong Huo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827764#comment-15827764
 ] 

Ruilong Huo commented on HAWQ-1278:
---

[~espino] and [~dyozie], per investigation with [~abai] and [~liming01], the 
root cause is that: 1) error table suite failure is due to gpfidst issue; 2) 
gpfdist failure is due to the client receive no response from gpfdist service 
after the http request is sent. To be specific, there is a bug in curl-7.51.0 
on osx 10.12 while curl-7.43.0 is good on that.

As for doc, we need to add below information:
{noformat}
Know issue: HAWQ-1278: curl 7.51.0 bug on osx 10.12 introduce gpfdist client 
fail to get response from gpfdist service after http request is sent.
The solution: use curl 7.43
{noformat}

> Investigate installcheck-good issue on Mac OSX
> --
>
> Key: HAWQ-1278
> URL: https://issues.apache.org/jira/browse/HAWQ-1278
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Tests
>Reporter: Ed Espino
>Assignee: Ruilong Huo
> Fix For: 2.2.0.0-incubating
>
>
> I am filing this as a place holder for the Mac OSX installcheck-good 
> investigation work.  Ming Li originally reported installcheck-good testing 
> issues with errtable and hcatalog_lookup test suites.
> This issue is not seen on CentOS 6 & 7 environments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1278) Investigate installcheck-good issue on Mac OSX

[Ming LI (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827658#comment-15827658
 ] 

Ming LI commented on HAWQ-1278:
---

On macOS 10.12.2, after changed curl version from 7.51.0 to 7.43.0, the test 
errortbl ran clean.
So it should be a curl bug on this version of platform. Thanks.

> Investigate installcheck-good issue on Mac OSX
> --
>
> Key: HAWQ-1278
> URL: https://issues.apache.org/jira/browse/HAWQ-1278
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Tests
>Reporter: Ed Espino
>Assignee: Ruilong Huo
> Fix For: 2.2.0.0-incubating
>
>
> I am filing this as a place holder for the Mac OSX installcheck-good 
> investigation work.  Ming Li originally reported installcheck-good testing 
> issues with errtable and hcatalog_lookup test suites.
> This issue is not seen on CentOS 6 & 7 environments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #1093: HAWQ-1193. Add createEncryption, getEZFor...

[interma]

Github user interma commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1093#discussion_r96584985
  
--- Diff: depends/libhdfs3/src/client/hdfs.h ---
@@ -482,6 +482,30 @@ int hdfsCreateDirectory(hdfsFS fs, const char * path);
 int hdfsSetReplication(hdfsFS fs, const char * path, int16_t replication);
 
 /**
+ * hdfsEncryptionZoneInfo- Information about an encryption zone.
+ */
+typedef struct {
+int mSuite; /* the suite of encryption zone */
+int mCryptoProtocolVersion; /* the version of crypto protocol */
+int64_t mId; /* the id of encryption zone */
+char * mPath; /* the path of encryption zone */
+char * mKeyName; /* the key name of encryption zone */
+} hdfsEncryptionZoneInfo;
+
+
+/**
+ * hdfsEncryptionFileInfo - Information about an encryption file/directory.
+ */
+typedef struct {
+int mSuite; /* the suite of encryption file/directory */
+int mCryptoProtocolVersion; /* the version of crypto protocol */
+char * mKey; /* the key of encryption file/directory */
+char * mKeyName; /* the key name of encryption file/directory */
+char * mIv; /* the iv of encryption file/directory */
+char * mEzKeyVersionName; /* the version encryption file/directory */
--- End diff --

The 4 `char *` field, change to array(e.g. `char mKey[16];` ) or new a 
buffer(e.g. call `Strdup()` ) is better?

Don't change is also ok, and should use them carefully in future code 
(avoid the space of their point is invalid).



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-1087) Need clarification on Wiki page for Build Instructions

[Ruilong Huo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827599#comment-15827599
 ] 

Ruilong Huo commented on HAWQ-1087:
---

[~dyozie], I have the access to hawq wiki: 
https://cwiki.apache.org/confluence/display/HAWQ/Build+and+Install

> Need clarification on Wiki page for Build Instructions
> --
>
> Key: HAWQ-1087
> URL: https://issues.apache.org/jira/browse/HAWQ-1087
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Documentation
>Reporter: Goden Yao
>Assignee: David Yozie
>Priority: Minor
> Fix For: backlog
>
>
> From [~jmclean], this is during a VOTE review. 
> {quote}
> Also currently looks like openssl may be a little broken and info on the wiki 
> may need updating:
> {code}
> brew link --force openssl
> Warning: Refusing to link: openssl
> {code}
> Linking keg-only openssl means you may end up linking against the insecure,
> deprecated system OpenSSL while using the headers from Homebrew's openssl.
> Instead, pass the full include/library paths to your compiler e.g.:
>  {code} -I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib{code}
> I needed up doing this to make it compile:
> make CFLAGS="-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib"
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #1095: HAWQ-1276. hawq should error out directly when r...

[interma]

Github user interma commented on the issue:

https://github.com/apache/incubator-hawq/pull/1095
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #1095: HAWQ-1276. hawq should error out directly when r...

[stanlyxiang]

Github user stanlyxiang commented on the issue:

https://github.com/apache/incubator-hawq/pull/1095
  
cc @ictmalili @interma 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---