[jira] [Commented] (HAWQ-897) Add feature test for create table distribution with new test framework

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387150#comment-15387150
 ] 

ASF GitHub Bot commented on HAWQ-897:
-

Github user yaoj2 commented on the issue:

https://github.com/apache/incubator-hawq/pull/807
  
LGTM


> Add feature test for create table distribution with new test framework
> --
>
> Key: HAWQ-897
> URL: https://issues.apache.org/jira/browse/HAWQ-897
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Tests
>Reporter: Lin Wen
>Assignee: Lin Wen
> Fix For: 2.0.1.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #807: HAWQ-897. Add feature test for create table distr...

[yaoj2]

Github user yaoj2 commented on the issue:

https://github.com/apache/incubator-hawq/pull/807
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-940) Kerberos Ticket Expired for LibYARN Operations

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387148#comment-15387148
 ] 

ASF GitHub Bot commented on HAWQ-940:
-

Github user jiny2 commented on the issue:

https://github.com/apache/incubator-hawq/pull/804
  
LGTM +1


> Kerberos Ticket Expired for LibYARN Operations
> --
>
> Key: HAWQ-940
> URL: https://issues.apache.org/jira/browse/HAWQ-940
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: libyarn
>Reporter: Lin Wen
>Assignee: Lin Wen
> Fix For: 2.0.1.0-incubating
>
>
> HAWQ's libhdfs3 and libyarn use a same kerberos keyfile. 
> Whenever a hdfs operation is triggered, a function named login() is called, 
> in login() function, this ticket is initialized by "kinit". 
> But for libyarn, login() function is only called during the resource broker 
> process starts. So if HAWQ starts up and there is no query for a long 
> period(24 hours in kerberos's configure file, krb.conf), this ticket will 
> expire, and HAWQ fails to register itself in Hadoop YARN.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #804: HAWQ-940. Fix Kerberos ticket expired for libyarn...

[jiny2]

Github user jiny2 commented on the issue:

https://github.com/apache/incubator-hawq/pull/804
  
LGTM +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-897) Add feature test for create table distribution with new test framework

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387147#comment-15387147
 ] 

ASF GitHub Bot commented on HAWQ-897:
-

Github user linwen commented on the issue:

https://github.com/apache/incubator-hawq/pull/807
  
+1


> Add feature test for create table distribution with new test framework
> --
>
> Key: HAWQ-897
> URL: https://issues.apache.org/jira/browse/HAWQ-897
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Tests
>Reporter: Lin Wen
>Assignee: Lin Wen
> Fix For: 2.0.1.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-934) Populate canSetTag of PlannedStmt from Query object

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387135#comment-15387135
 ] 

ASF GitHub Bot commented on HAWQ-934:
-

Github user hsyuan commented on the issue:

https://github.com/apache/incubator-hawq/pull/799
  
@changleicn @wengyanqing @paul-guo- 
Please take a look.


> Populate canSetTag of PlannedStmt from Query object
> ---
>
> Key: HAWQ-934
> URL: https://issues.apache.org/jira/browse/HAWQ-934
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Optimizer
>Reporter: Haisheng Yuan
>Assignee: Venkatesh
> Fix For: 2.0.1.0-incubating
>
>
> HAWQ generated an error if a single query resulted in multiple query plans 
> because of rule transformation and the plans were produced by PQO. This is 
> because of an incorrect directive in the plan to lock the same resource more 
> than once.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #799: HAWQ-934. Populate canSetTag of PlannedStmt from ...

[hsyuan]

Github user hsyuan commented on the issue:

https://github.com/apache/incubator-hawq/pull/799
  
@changleicn @wengyanqing @paul-guo- 
Please take a look.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-897) Add feature test for create table distribution with new test framework

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387136#comment-15387136
 ] 

ASF GitHub Bot commented on HAWQ-897:
-

GitHub user jiny2 opened a pull request:

https://github.com/apache/incubator-hawq/pull/807

HAWQ-897. Add feature test for create table distribution with new test 
framework



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jiny2/incubator-hawq HAWQ-897

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/807.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #807


commit 5fb757d8597b6dfa3e64cda81260e4d819e1793c
Author: YI JIN 
Date:   2016-07-21T04:28:24Z

HAWQ-897. Add feature test for create table distribution with new test 
framework




> Add feature test for create table distribution with new test framework
> --
>
> Key: HAWQ-897
> URL: https://issues.apache.org/jira/browse/HAWQ-897
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Tests
>Reporter: Lin Wen
>Assignee: Lin Wen
> Fix For: 2.0.1.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #807: HAWQ-897. Add feature test for create tabl...

[jiny2]

GitHub user jiny2 opened a pull request:

https://github.com/apache/incubator-hawq/pull/807

HAWQ-897. Add feature test for create table distribution with new test 
framework



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jiny2/incubator-hawq HAWQ-897

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/807.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #807


commit 5fb757d8597b6dfa3e64cda81260e4d819e1793c
Author: YI JIN 
Date:   2016-07-21T04:28:24Z

HAWQ-897. Add feature test for create table distribution with new test 
framework




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-938) Remove ivy.xml in gpopt and read orca version from header file

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387128#comment-15387128
 ] 

ASF GitHub Bot commented on HAWQ-938:
-

Github user hsyuan commented on the issue:

https://github.com/apache/incubator-hawq/pull/806
  
Thanks, will take care of it.


> Remove ivy.xml in gpopt and read orca version from header file
> --
>
> Key: HAWQ-938
> URL: https://issues.apache.org/jira/browse/HAWQ-938
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Optimizer
>Reporter: Haisheng Yuan
>Assignee: Haisheng Yuan
> Fix For: 2.0.1.0-incubating
>
>
> Currently, if we want to upgrade orca or gpos, we need change the orca SHA as 
> well as the version number in ivy.xml. The function gp_opt_version() returns 
> version number that is read from ivy.xml, which is not a right way. It should 
> only be dependent on the source file of orca and gpos.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #806: HAWQ-938. Remove ivy.xml in gpopt and read orca v...

[hsyuan]

Github user hsyuan commented on the issue:

https://github.com/apache/incubator-hawq/pull/806
  
Thanks, will take care of it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Closed] (HAWQ-943) Various issues in hawq register feature_test cases

[Paul Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Guo closed HAWQ-943.
-
Resolution: Fixed

> Various issues in hawq register feature_test cases
> --
>
> Key: HAWQ-943
> URL: https://issues.apache.org/jira/browse/HAWQ-943
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Paul Guo
> Fix For: 2.0.1.0-incubating
>
>
> 1) Do not assume the test database is postgres.
> Use HAWQ_DB which is defined in sql_util.h
> 2) Use error immune options when creating a new hdfs file or directory.
> e.g. mkdir -p, put -f.
> Since nonexistence of those files/directories are not guaranteed.
> e.g. Previous test run was terminated by ctrl+c.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HAWQ-943) Various issues in hawq register feature_test cases

[Paul Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Guo reassigned HAWQ-943:
-

Assignee: Paul Guo  (was: Lei Chang)

> Various issues in hawq register feature_test cases
> --
>
> Key: HAWQ-943
> URL: https://issues.apache.org/jira/browse/HAWQ-943
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Paul Guo
> Fix For: 2.0.1.0-incubating
>
>
> 1) Do not assume the test database is postgres.
> Use HAWQ_DB which is defined in sql_util.h
> 2) Use error immune options when creating a new hdfs file or directory.
> e.g. mkdir -p, put -f.
> Since nonexistence of those files/directories are not guaranteed.
> e.g. Previous test run was terminated by ctrl+c.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-940) Kerberos Ticket Expired for LibYARN Operations

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387080#comment-15387080
 ] 

ASF GitHub Bot commented on HAWQ-940:
-

GitHub user linwen reopened a pull request:

https://github.com/apache/incubator-hawq/pull/804

HAWQ-940. Fix Kerberos ticket expired for libyarn operations

Please review, thanks! 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/linwen/incubator-hawq hawq_940

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/804.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #804


commit 0942630825f6c84da155bd2d9aec4831f7e4d049
Author: Wen Lin 
Date:   2016-07-20T09:07:01Z

HAWQ-940. Fix Kerberos ticket expired for libyarn operations




> Kerberos Ticket Expired for LibYARN Operations
> --
>
> Key: HAWQ-940
> URL: https://issues.apache.org/jira/browse/HAWQ-940
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: libyarn
>Reporter: Lin Wen
>Assignee: Lin Wen
> Fix For: 2.0.1.0-incubating
>
>
> HAWQ's libhdfs3 and libyarn use a same kerberos keyfile. 
> Whenever a hdfs operation is triggered, a function named login() is called, 
> in login() function, this ticket is initialized by "kinit". 
> But for libyarn, login() function is only called during the resource broker 
> process starts. So if HAWQ starts up and there is no query for a long 
> period(24 hours in kerberos's configure file, krb.conf), this ticket will 
> expire, and HAWQ fails to register itself in Hadoop YARN.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-940) Kerberos Ticket Expired for LibYARN Operations

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387079#comment-15387079
 ] 

ASF GitHub Bot commented on HAWQ-940:
-

Github user linwen commented on the issue:

https://github.com/apache/incubator-hawq/pull/804
  
Another way of fix is to add the ticket check in resource broker process 
loop, for every time interval, login() is called. But this fix has to keep 
another variable to record the last updated time, which is duplicate with 
login(). 


> Kerberos Ticket Expired for LibYARN Operations
> --
>
> Key: HAWQ-940
> URL: https://issues.apache.org/jira/browse/HAWQ-940
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: libyarn
>Reporter: Lin Wen
>Assignee: Lin Wen
> Fix For: 2.0.1.0-incubating
>
>
> HAWQ's libhdfs3 and libyarn use a same kerberos keyfile. 
> Whenever a hdfs operation is triggered, a function named login() is called, 
> in login() function, this ticket is initialized by "kinit". 
> But for libyarn, login() function is only called during the resource broker 
> process starts. So if HAWQ starts up and there is no query for a long 
> period(24 hours in kerberos's configure file, krb.conf), this ticket will 
> expire, and HAWQ fails to register itself in Hadoop YARN.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #804: HAWQ-940. Fix Kerberos ticket expired for ...

[linwen]

GitHub user linwen reopened a pull request:

https://github.com/apache/incubator-hawq/pull/804

HAWQ-940. Fix Kerberos ticket expired for libyarn operations

Please review, thanks! 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/linwen/incubator-hawq hawq_940

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/804.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #804


commit 0942630825f6c84da155bd2d9aec4831f7e4d049
Author: Wen Lin 
Date:   2016-07-20T09:07:01Z

HAWQ-940. Fix Kerberos ticket expired for libyarn operations




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #804: HAWQ-940. Fix Kerberos ticket expired for libyarn...

[linwen]

Github user linwen commented on the issue:

https://github.com/apache/incubator-hawq/pull/804
  
Another way of fix is to add the ticket check in resource broker process 
loop, for every time interval, login() is called. But this fix has to keep 
another variable to record the last updated time, which is duplicate with 
login(). 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (HAWQ-940) Kerberos Ticket Expired for LibYARN Operations

[Lin Wen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lin Wen reassigned HAWQ-940:


Assignee: Lin Wen  (was: Lei Chang)

> Kerberos Ticket Expired for LibYARN Operations
> --
>
> Key: HAWQ-940
> URL: https://issues.apache.org/jira/browse/HAWQ-940
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: libyarn
>Reporter: Lin Wen
>Assignee: Lin Wen
> Fix For: 2.0.1.0-incubating
>
>
> HAWQ's libhdfs3 and libyarn use a same kerberos keyfile. 
> Whenever a hdfs operation is triggered, a function named login() is called, 
> in login() function, this ticket is initialized by "kinit". 
> But for libyarn, login() function is only called during the resource broker 
> process starts. So if HAWQ starts up and there is no query for a long 
> period(24 hours in kerberos's configure file, krb.conf), this ticket will 
> expire, and HAWQ fails to register itself in Hadoop YARN.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-940) Kerberos Ticket Expired for LibYARN Operations

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387063#comment-15387063
 ] 

ASF GitHub Bot commented on HAWQ-940:
-

Github user linwen commented on the issue:

https://github.com/apache/incubator-hawq/pull/804
  
After re-think this fix, I think it's better to check ticket expiration in 
resource broker process loop.
so close it. 


> Kerberos Ticket Expired for LibYARN Operations
> --
>
> Key: HAWQ-940
> URL: https://issues.apache.org/jira/browse/HAWQ-940
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: libyarn
>Reporter: Lin Wen
>Assignee: Lei Chang
> Fix For: 2.0.1.0-incubating
>
>
> HAWQ's libhdfs3 and libyarn use a same kerberos keyfile. 
> Whenever a hdfs operation is triggered, a function named login() is called, 
> in login() function, this ticket is initialized by "kinit". 
> But for libyarn, login() function is only called during the resource broker 
> process starts. So if HAWQ starts up and there is no query for a long 
> period(24 hours in kerberos's configure file, krb.conf), this ticket will 
> expire, and HAWQ fails to register itself in Hadoop YARN.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #804: HAWQ-940. Fix Kerberos ticket expired for libyarn...

[linwen]

Github user linwen commented on the issue:

https://github.com/apache/incubator-hawq/pull/804
  
After re-think this fix, I think it's better to check ticket expiration in 
resource broker process loop.
so close it. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-940) Kerberos Ticket Expired for LibYARN Operations

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387064#comment-15387064
 ] 

ASF GitHub Bot commented on HAWQ-940:
-

Github user linwen closed the pull request at:

https://github.com/apache/incubator-hawq/pull/804


> Kerberos Ticket Expired for LibYARN Operations
> --
>
> Key: HAWQ-940
> URL: https://issues.apache.org/jira/browse/HAWQ-940
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: libyarn
>Reporter: Lin Wen
>Assignee: Lei Chang
> Fix For: 2.0.1.0-incubating
>
>
> HAWQ's libhdfs3 and libyarn use a same kerberos keyfile. 
> Whenever a hdfs operation is triggered, a function named login() is called, 
> in login() function, this ticket is initialized by "kinit". 
> But for libyarn, login() function is only called during the resource broker 
> process starts. So if HAWQ starts up and there is no query for a long 
> period(24 hours in kerberos's configure file, krb.conf), this ticket will 
> expire, and HAWQ fails to register itself in Hadoop YARN.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #804: HAWQ-940. Fix Kerberos ticket expired for ...

[linwen]

Github user linwen closed the pull request at:

https://github.com/apache/incubator-hawq/pull/804


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-943) Various issues in hawq register feature_test cases

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387059#comment-15387059
 ] 

ASF GitHub Bot commented on HAWQ-943:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/805


> Various issues in hawq register feature_test cases
> --
>
> Key: HAWQ-943
> URL: https://issues.apache.org/jira/browse/HAWQ-943
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
> Fix For: 2.0.1.0-incubating
>
>
> 1) Do not assume the test database is postgres.
> Use HAWQ_DB which is defined in sql_util.h
> 2) Use error immune options when creating a new hdfs file or directory.
> e.g. mkdir -p, put -f.
> Since nonexistence of those files/directories are not guaranteed.
> e.g. Previous test run was terminated by ctrl+c.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #805: HAWQ-943. Various issues in hawq register ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/805


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-943) Various issues in hawq register feature_test cases

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386997#comment-15386997
 ] 

ASF GitHub Bot commented on HAWQ-943:
-

Github user ictmalili commented on the issue:

https://github.com/apache/incubator-hawq/pull/805
  
LGTM. +1


> Various issues in hawq register feature_test cases
> --
>
> Key: HAWQ-943
> URL: https://issues.apache.org/jira/browse/HAWQ-943
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
> Fix For: 2.0.1.0-incubating
>
>
> 1) Do not assume the test database is postgres.
> Use HAWQ_DB which is defined in sql_util.h
> 2) Use error immune options when creating a new hdfs file or directory.
> e.g. mkdir -p, put -f.
> Since nonexistence of those files/directories are not guaranteed.
> e.g. Previous test run was terminated by ctrl+c.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #805: HAWQ-943. Various issues in hawq register feature...

[ictmalili]

Github user ictmalili commented on the issue:

https://github.com/apache/incubator-hawq/pull/805
  
LGTM. +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #806: HAWQ-938. Remove ivy.xml in gpopt and read orca v...

[paul-guo-]

Github user paul-guo- commented on the issue:

https://github.com/apache/incubator-hawq/pull/806
  
+1

By the way, I just noticed the following code in gpopt.mk

UNAME = $(shell uname)
UNAME_P = $(shell uname -p)
UNAME_M = $(shell uname -m)
ARCH_OS = GPOS_$(UNAME)
ARCH_CPU = GPOS_$(UNAME_P)

ifeq (Darwin, $(UNAME))
LDSFX = dylib
else
LDSFX = so
endif

Please use  DLSUFFIX and PORTNAME which are defined in src/Makefile.global
(and its including makefiles),
and are generated with the configure command. This is another issue, you
could make the change in the future.

2016-07-21 9:21 GMT+08:00 Lei Chang :

> @paul-guo-  to review.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> 
,
> or mute the thread
> 

> .
>



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #800: HAWQ-936. Add GUC for array expansion in ORCA opt...

[changleicn]

Github user changleicn commented on the issue:

https://github.com/apache/incubator-hawq/pull/800
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-936) Add GUC for array expansion in ORCA optimizer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386950#comment-15386950
 ] 

ASF GitHub Bot commented on HAWQ-936:
-

Github user changleicn commented on the issue:

https://github.com/apache/incubator-hawq/pull/800
  
LGTM


> Add GUC for array expansion in ORCA optimizer
> -
>
> Key: HAWQ-936
> URL: https://issues.apache.org/jira/browse/HAWQ-936
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: Optimizer
>Reporter: Haisheng Yuan
>Assignee: Venkatesh
> Fix For: 2.0.1.0-incubating
>
>
> Consider the query with the following pattern select * from foo where foo.a 
> IN {1,2,3,...}. Currently, when the number of constants in the IN subquery is 
> large, the query optimization time is unacceptable. This is stopping 
> customers from turning Orca on by default since many of the queries are 
> generated queries with such a pattern.
> The root cause is due to the expansion of the IN subquery into an expression 
> in disjunctive normal form. The objective of this story is to disable this 
> expansion when the number of constants in the IN list is large.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #806: HAWQ-938. Remove ivy.xml in gpopt and read orca v...

[changleicn]

Github user changleicn commented on the issue:

https://github.com/apache/incubator-hawq/pull/806
  
@paul-guo- to review.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-938) Remove ivy.xml in gpopt and read orca version from header file

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386947#comment-15386947
 ] 

ASF GitHub Bot commented on HAWQ-938:
-

Github user changleicn commented on the issue:

https://github.com/apache/incubator-hawq/pull/806
  
@paul-guo- to review.


> Remove ivy.xml in gpopt and read orca version from header file
> --
>
> Key: HAWQ-938
> URL: https://issues.apache.org/jira/browse/HAWQ-938
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Optimizer
>Reporter: Haisheng Yuan
>Assignee: Haisheng Yuan
> Fix For: 2.0.1.0-incubating
>
>
> Currently, if we want to upgrade orca or gpos, we need change the orca SHA as 
> well as the version number in ivy.xml. The function gp_opt_version() returns 
> version number that is read from ivy.xml, which is not a right way. It should 
> only be dependent on the source file of orca and gpos.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #796: HAWQ-927. Pass ProjectionInfo data to PXF

[shivzone]

Github user shivzone commented on the issue:

https://github.com/apache/incubator-hawq/pull/796
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-927) Send Projection Info Data from HAWQ to PXF

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386817#comment-15386817
 ] 

ASF GitHub Bot commented on HAWQ-927:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/796


> Send Projection Info Data from HAWQ to PXF
> --
>
> Key: HAWQ-927
> URL: https://issues.apache.org/jira/browse/HAWQ-927
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: External Tables, PXF
>Reporter: Kavinder Dhaliwal
>Assignee: Kavinder Dhaliwal
> Fix For: backlog
>
>
> To achieve column projection at the level of PXF or the underlying readers we 
> need to first send this data as a Header/Param to PXF. Currently, PXF has no 
> knowledge whether a query requires all columns or a subset of columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-936) Add GUC for array expansion in ORCA optimizer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386816#comment-15386816
 ] 

ASF GitHub Bot commented on HAWQ-936:
-

Github user hsyuan commented on the issue:

https://github.com/apache/incubator-hawq/pull/800
  
This PR will give https://github.com/apache/incubator-hawq/pull/795 a free 
ride.


> Add GUC for array expansion in ORCA optimizer
> -
>
> Key: HAWQ-936
> URL: https://issues.apache.org/jira/browse/HAWQ-936
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: Optimizer
>Reporter: Haisheng Yuan
>Assignee: Venkatesh
> Fix For: 2.0.1.0-incubating
>
>
> Consider the query with the following pattern select * from foo where foo.a 
> IN {1,2,3,...}. Currently, when the number of constants in the IN subquery is 
> large, the query optimization time is unacceptable. This is stopping 
> customers from turning Orca on by default since many of the queries are 
> generated queries with such a pattern.
> The root cause is due to the expansion of the IN subquery into an expression 
> in disjunctive normal form. The objective of this story is to disable this 
> expansion when the number of constants in the IN list is large.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #796: HAWQ-927. Pass ProjectionInfo data to PXF

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/796


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #800: HAWQ-936. Add GUC for array expansion in ORCA opt...

[hsyuan]

Github user hsyuan commented on the issue:

https://github.com/apache/incubator-hawq/pull/800
  
This PR will give https://github.com/apache/incubator-hawq/pull/795 a free 
ride.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-927) Send Projection Info Data from HAWQ to PXF

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386809#comment-15386809
 ] 

ASF GitHub Bot commented on HAWQ-927:
-

Github user shivzone commented on the issue:

https://github.com/apache/incubator-hawq/pull/796
  
+1


> Send Projection Info Data from HAWQ to PXF
> --
>
> Key: HAWQ-927
> URL: https://issues.apache.org/jira/browse/HAWQ-927
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: External Tables, PXF
>Reporter: Kavinder Dhaliwal
>Assignee: Kavinder Dhaliwal
> Fix For: backlog
>
>
> To achieve column projection at the level of PXF or the underlying readers we 
> need to first send this data as a Header/Param to PXF. Currently, PXF has no 
> knowledge whether a query requires all columns or a subset of columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-938) Remove ivy.xml in gpopt and read orca version from header file

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386634#comment-15386634
 ] 

ASF GitHub Bot commented on HAWQ-938:
-

Github user hsyuan commented on the issue:

https://github.com/apache/incubator-hawq/pull/806
  
@changleicn @yaoj2 @wengyanqing 
Please take a look.


> Remove ivy.xml in gpopt and read orca version from header file
> --
>
> Key: HAWQ-938
> URL: https://issues.apache.org/jira/browse/HAWQ-938
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Optimizer
>Reporter: Haisheng Yuan
>Assignee: Haisheng Yuan
> Fix For: 2.0.1.0-incubating
>
>
> Currently, if we want to upgrade orca or gpos, we need change the orca SHA as 
> well as the version number in ivy.xml. The function gp_opt_version() returns 
> version number that is read from ivy.xml, which is not a right way. It should 
> only be dependent on the source file of orca and gpos.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #806: HAWQ-938. Remove ivy.xml in gpopt and read orca v...

[hsyuan]

Github user hsyuan commented on the issue:

https://github.com/apache/incubator-hawq/pull/806
  
@changleicn @yaoj2 @wengyanqing 
Please take a look.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-938) Remove ivy.xml in gpopt and read orca version from header file

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386632#comment-15386632
 ] 

ASF GitHub Bot commented on HAWQ-938:
-

GitHub user hsyuan opened a pull request:

https://github.com/apache/incubator-hawq/pull/806

HAWQ-938. Remove ivy.xml in gpopt and read orca version from header file

The old mechanism extracted the version numbers from the Ivy config file,
which doesn't do the right thing if you build without Ivy. Using the
version headers is simpler, anyway. Also removed `ivy.xml` and 
`ivy-build.xml`
under `gpopt` folder.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hsyuan/incubator-hawq HAWQ-938

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/806.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #806


commit 2a2a89cc6b950d4067a8e4d8a5e79b2f7b1cf839
Author: Haisheng Yuan 
Date:   2016-07-20T20:14:25Z

HAWQ-938. Remove ivy.xml in gpopt and read orca version from header file

The old mechanism extracted the version numbers from the Ivy config file,
which doesn't do the right thing if you build without Ivy. Using the
version headers is simpler, anyway.




> Remove ivy.xml in gpopt and read orca version from header file
> --
>
> Key: HAWQ-938
> URL: https://issues.apache.org/jira/browse/HAWQ-938
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Optimizer
>Reporter: Haisheng Yuan
>Assignee: Haisheng Yuan
> Fix For: 2.0.1.0-incubating
>
>
> Currently, if we want to upgrade orca or gpos, we need change the orca SHA as 
> well as the version number in ivy.xml. The function gp_opt_version() returns 
> version number that is read from ivy.xml, which is not a right way. It should 
> only be dependent on the source file of orca and gpos.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #806: HAWQ-938. Remove ivy.xml in gpopt and read...

[hsyuan]

GitHub user hsyuan opened a pull request:

https://github.com/apache/incubator-hawq/pull/806

HAWQ-938. Remove ivy.xml in gpopt and read orca version from header file

The old mechanism extracted the version numbers from the Ivy config file,
which doesn't do the right thing if you build without Ivy. Using the
version headers is simpler, anyway. Also removed `ivy.xml` and 
`ivy-build.xml`
under `gpopt` folder.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hsyuan/incubator-hawq HAWQ-938

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/806.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #806


commit 2a2a89cc6b950d4067a8e4d8a5e79b2f7b1cf839
Author: Haisheng Yuan 
Date:   2016-07-20T20:14:25Z

HAWQ-938. Remove ivy.xml in gpopt and read orca version from header file

The old mechanism extracted the version numbers from the Ivy config file,
which doesn't do the right thing if you build without Ivy. Using the
version headers is simpler, anyway.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (HAWQ-938) Remove ivy.xml in gpopt and read orca version from header file

[Haisheng Yuan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haisheng Yuan reassigned HAWQ-938:
--

Assignee: Haisheng Yuan  (was: Venkatesh)

> Remove ivy.xml in gpopt and read orca version from header file
> --
>
> Key: HAWQ-938
> URL: https://issues.apache.org/jira/browse/HAWQ-938
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Optimizer
>Reporter: Haisheng Yuan
>Assignee: Haisheng Yuan
> Fix For: 2.0.1.0-incubating
>
>
> Currently, if we want to upgrade orca or gpos, we need change the orca SHA as 
> well as the version number in ivy.xml. The function gp_opt_version() returns 
> version number that is read from ivy.xml, which is not a right way. It should 
> only be dependent on the source file of orca and gpos.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-942) Need to .gitignore and make-clean some test files which are generated after running feature/test

[Goden Yao (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Goden Yao updated HAWQ-942:
---
Fix Version/s: 2.0.1.0-incubating

> Need to .gitignore and make-clean some test files which are generated after 
> running feature/test
> 
>
> Key: HAWQ-942
> URL: https://issues.apache.org/jira/browse/HAWQ-942
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
> Fix For: 2.0.1.0-incubating
>
>
> After running feature tests, there will be some generated output files, like 
> .out files (result files after running .sql file), .out.diff files (If .out 
> files does not match .ans files, there will be files which include the diff 
> part), generated sql and and ans files, etc.
> We need to clean up them in the Makefile clean/distclean target. Besides we 
> need to mask them in "git status" outputs, probably .out.diff is an exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-943) Various issues in hawq register feature_test cases

[Goden Yao (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Goden Yao updated HAWQ-943:
---
Fix Version/s: 2.0.1.0-incubating

> Various issues in hawq register feature_test cases
> --
>
> Key: HAWQ-943
> URL: https://issues.apache.org/jira/browse/HAWQ-943
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
> Fix For: 2.0.1.0-incubating
>
>
> 1) Do not assume the test database is postgres.
> Use HAWQ_DB which is defined in sql_util.h
> 2) Use error immune options when creating a new hdfs file or directory.
> e.g. mkdir -p, put -f.
> Since nonexistence of those files/directories are not guaranteed.
> e.g. Previous test run was terminated by ctrl+c.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-927) Send Projection Info Data from HAWQ to PXF

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386188#comment-15386188
 ] 

ASF GitHub Bot commented on HAWQ-927:
-

Github user xunzhang commented on the issue:

https://github.com/apache/incubator-hawq/pull/796
  
LGTM. Remember rebasing the commit before checking in.  


> Send Projection Info Data from HAWQ to PXF
> --
>
> Key: HAWQ-927
> URL: https://issues.apache.org/jira/browse/HAWQ-927
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: External Tables, PXF
>Reporter: Kavinder Dhaliwal
>Assignee: Kavinder Dhaliwal
> Fix For: backlog
>
>
> To achieve column projection at the level of PXF or the underlying readers we 
> need to first send this data as a Header/Param to PXF. Currently, PXF has no 
> knowledge whether a query requires all columns or a subset of columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #796: HAWQ-927. Pass ProjectionInfo data to PXF

[xunzhang]

Github user xunzhang commented on the issue:

https://github.com/apache/incubator-hawq/pull/796
  
LGTM. Remember rebasing the commit before checking in. 🍺 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-927) Send Projection Info Data from HAWQ to PXF

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386155#comment-15386155
 ] 

ASF GitHub Bot commented on HAWQ-927:
-

Github user kavinderd commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/796#discussion_r71559534
  
--- Diff: src/backend/access/external/pxfheaders.c ---
@@ -158,6 +166,29 @@ static void add_tuple_desc_httpheader(CHURL_HEADERS 
headers, Relation rel)
pfree(formatter.data);
 }
 
+static void add_projection_desc_httpheader(CHURL_HEADERS headers, 
ProjectionInfo *projInfo) {
+   int i;
+   char long_number[sizeof(int32) * 8];
+   int *varNumbers = projInfo->pi_varNumbers;
+   StringInfoData formatter;
+   initStringInfo();
+
+/* Convert the number of projection columns to a string */
+pg_ltoa(list_length(projInfo->pi_targetlist), long_number);
+churl_headers_append(headers, "X-GP-ATTRS-PROJ", long_number);
+
+   for(i = 0; i < list_length(projInfo->pi_targetlist); i++) {
--- End diff --

Yes it will be in another PR related to 
[this](https://issues.apache.org/jira/browse/HAWQ-583?jql=project%20%3D%20HAWQ%20AND%20resolution%20%3D%20Unresolved%20AND%20assignee%20%3D%20kavinderd%20ORDER%20BY%20priority%20DESC)
 Jira.


> Send Projection Info Data from HAWQ to PXF
> --
>
> Key: HAWQ-927
> URL: https://issues.apache.org/jira/browse/HAWQ-927
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: External Tables, PXF
>Reporter: Kavinder Dhaliwal
>Assignee: Kavinder Dhaliwal
> Fix For: backlog
>
>
> To achieve column projection at the level of PXF or the underlying readers we 
> need to first send this data as a Header/Param to PXF. Currently, PXF has no 
> knowledge whether a query requires all columns or a subset of columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #796: HAWQ-927. Pass ProjectionInfo data to PXF

[kavinderd]

Github user kavinderd commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/796#discussion_r71559534
  
--- Diff: src/backend/access/external/pxfheaders.c ---
@@ -158,6 +166,29 @@ static void add_tuple_desc_httpheader(CHURL_HEADERS 
headers, Relation rel)
pfree(formatter.data);
 }
 
+static void add_projection_desc_httpheader(CHURL_HEADERS headers, 
ProjectionInfo *projInfo) {
+   int i;
+   char long_number[sizeof(int32) * 8];
+   int *varNumbers = projInfo->pi_varNumbers;
+   StringInfoData formatter;
+   initStringInfo();
+
+/* Convert the number of projection columns to a string */
+pg_ltoa(list_length(projInfo->pi_targetlist), long_number);
+churl_headers_append(headers, "X-GP-ATTRS-PROJ", long_number);
+
+   for(i = 0; i < list_length(projInfo->pi_targetlist); i++) {
--- End diff --

Yes it will be in another PR related to 
[this](https://issues.apache.org/jira/browse/HAWQ-583?jql=project%20%3D%20HAWQ%20AND%20resolution%20%3D%20Unresolved%20AND%20assignee%20%3D%20kavinderd%20ORDER%20BY%20priority%20DESC)
 Jira.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-927) Send Projection Info Data from HAWQ to PXF

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386147#comment-15386147
 ] 

ASF GitHub Bot commented on HAWQ-927:
-

Github user kavinderd commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/796#discussion_r71558813
  
--- Diff: src/backend/access/external/fileam.c ---
@@ -454,6 +454,21 @@ external_stopscan(FileScanDesc scan)
}
 }
 
+/* 
+ * external_getnext_init - prepare ExternalSelectDesc struct 
before external_getnext
+/* 
+ */
+
+ExternalSelectDesc
+external_getnext_init(PlanState *state) {
+   ExternalSelectDesc desc = (ExternalSelectDesc) 
palloc0(sizeof(ExternalSelectDescData));
--- End diff --

I missed adding `pfree` for `desc`. I added it to the end of 
`ExternalNext()`


> Send Projection Info Data from HAWQ to PXF
> --
>
> Key: HAWQ-927
> URL: https://issues.apache.org/jira/browse/HAWQ-927
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: External Tables, PXF
>Reporter: Kavinder Dhaliwal
>Assignee: Kavinder Dhaliwal
> Fix For: backlog
>
>
> To achieve column projection at the level of PXF or the underlying readers we 
> need to first send this data as a Header/Param to PXF. Currently, PXF has no 
> knowledge whether a query requires all columns or a subset of columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #796: HAWQ-927. Pass ProjectionInfo data to PXF

[kavinderd]

Github user kavinderd commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/796#discussion_r71558813
  
--- Diff: src/backend/access/external/fileam.c ---
@@ -454,6 +454,21 @@ external_stopscan(FileScanDesc scan)
}
 }
 
+/* 
+ * external_getnext_init - prepare ExternalSelectDesc struct 
before external_getnext
+/* 
+ */
+
+ExternalSelectDesc
+external_getnext_init(PlanState *state) {
+   ExternalSelectDesc desc = (ExternalSelectDesc) 
palloc0(sizeof(ExternalSelectDescData));
--- End diff --

I missed adding `pfree` for `desc`. I added it to the end of 
`ExternalNext()`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-932) HAWQ fails to query external table defined with "localhost" in URL

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386111#comment-15386111
 ] 

ASF GitHub Bot commented on HAWQ-932:
-

Github user kavinderd commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/803#discussion_r71555148
  
--- Diff: src/backend/access/external/libchurl.c ---
@@ -312,6 +312,14 @@ CHURL_HANDLE churl_init_upload(const char* url, 
CHURL_HEADERS headers)
context->upload = true;
clear_error_buffer(context);
 
+   /* needed to resolve pxf service address */
+   struct curl_slist *resolve_hosts = NULL;
+   char *pxf_host_entry = (char *) palloc0(strlen(pxf_service_address) + 
strlen(LocalhostIpV4Entry) + 1);
--- End diff --

Is `pxf_host_entry` pfree'd?


> HAWQ fails to query external table defined with "localhost" in URL
> --
>
> Key: HAWQ-932
> URL: https://issues.apache.org/jira/browse/HAWQ-932
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: External Tables, PXF
>Reporter: Goden Yao
>Assignee: Oleksandr Diachenko
> Fix For: 2.0.1.0-incubating
>
>
> Originally reported by [~jpatel] when he's making a docker image based on 
> HAWQ 2.0.0.0-incubating dev build. Investigated by [~odiachenko]
> There is workaround to define it with 127.0.0.1, but there is not a 
> workaround for querying tables using HCatalog integration.
> It used to work before.
> {code}
> template1=# CREATE EXTERNAL TABLE ext_table1 (t1text, t2text,
> num1  integer, dub1  double precision) LOCATION
> (E'pxf://localhost:51200/hive_small_data?PROFILE=Hive') FORMAT 'CUSTOM'
> (formatter='pxfwritable_import');*
> CREATE EXTERNAL TABLE
> template1=# select * from ext_table1;
> ERROR:  remote component error (0): (libchurl.c:898)*
> {code}
> When I turned on debug mode in curl, I found this error in logs - "*
> Closing connection 0".
> I found a workaround, to set CURLOPT_RESOLVE option in curl:
> {code}
> struct curl_slist *host = NULL;
> host = curl_slist_append(NULL, "localhost:51200:127.0.0.1");*
> set_curl_option(context, CURLOPT_RESOLVE, host);
> {code}
> It seems like an issue with DNS cache,



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #803: HAWQ-932. Added pxf service address to cur...

[kavinderd]

Github user kavinderd commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/803#discussion_r71555148
  
--- Diff: src/backend/access/external/libchurl.c ---
@@ -312,6 +312,14 @@ CHURL_HANDLE churl_init_upload(const char* url, 
CHURL_HEADERS headers)
context->upload = true;
clear_error_buffer(context);
 
+   /* needed to resolve pxf service address */
+   struct curl_slist *resolve_hosts = NULL;
+   char *pxf_host_entry = (char *) palloc0(strlen(pxf_service_address) + 
strlen(LocalhostIpV4Entry) + 1);
--- End diff --

Is `pxf_host_entry` pfree'd?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #803: HAWQ-932. Added pxf service address to curl resol...

[kavinderd]

Github user kavinderd commented on the issue:

https://github.com/apache/incubator-hawq/pull/803
  
LGTM. Agree with @GodenYao on refactoring out the common code


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-932) HAWQ fails to query external table defined with "localhost" in URL

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386103#comment-15386103
 ] 

ASF GitHub Bot commented on HAWQ-932:
-

Github user kavinderd commented on the issue:

https://github.com/apache/incubator-hawq/pull/803
  
LGTM. Agree with @GodenYao on refactoring out the common code


> HAWQ fails to query external table defined with "localhost" in URL
> --
>
> Key: HAWQ-932
> URL: https://issues.apache.org/jira/browse/HAWQ-932
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: External Tables, PXF
>Reporter: Goden Yao
>Assignee: Oleksandr Diachenko
> Fix For: 2.0.1.0-incubating
>
>
> Originally reported by [~jpatel] when he's making a docker image based on 
> HAWQ 2.0.0.0-incubating dev build. Investigated by [~odiachenko]
> There is workaround to define it with 127.0.0.1, but there is not a 
> workaround for querying tables using HCatalog integration.
> It used to work before.
> {code}
> template1=# CREATE EXTERNAL TABLE ext_table1 (t1text, t2text,
> num1  integer, dub1  double precision) LOCATION
> (E'pxf://localhost:51200/hive_small_data?PROFILE=Hive') FORMAT 'CUSTOM'
> (formatter='pxfwritable_import');*
> CREATE EXTERNAL TABLE
> template1=# select * from ext_table1;
> ERROR:  remote component error (0): (libchurl.c:898)*
> {code}
> When I turned on debug mode in curl, I found this error in logs - "*
> Closing connection 0".
> I found a workaround, to set CURLOPT_RESOLVE option in curl:
> {code}
> struct curl_slist *host = NULL;
> host = curl_slist_append(NULL, "localhost:51200:127.0.0.1");*
> set_curl_option(context, CURLOPT_RESOLVE, host);
> {code}
> It seems like an issue with DNS cache,



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-932) HAWQ fails to query external table defined with "localhost" in URL

[Kavinder Dhaliwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386100#comment-15386100
 ] 

Kavinder Dhaliwal commented on HAWQ-932:


[~odiachenko] Good work on finding the fix for this. Do you know why we didn't 
notice this issue earlier?

> HAWQ fails to query external table defined with "localhost" in URL
> --
>
> Key: HAWQ-932
> URL: https://issues.apache.org/jira/browse/HAWQ-932
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: External Tables, PXF
>Reporter: Goden Yao
>Assignee: Oleksandr Diachenko
> Fix For: 2.0.1.0-incubating
>
>
> Originally reported by [~jpatel] when he's making a docker image based on 
> HAWQ 2.0.0.0-incubating dev build. Investigated by [~odiachenko]
> There is workaround to define it with 127.0.0.1, but there is not a 
> workaround for querying tables using HCatalog integration.
> It used to work before.
> {code}
> template1=# CREATE EXTERNAL TABLE ext_table1 (t1text, t2text,
> num1  integer, dub1  double precision) LOCATION
> (E'pxf://localhost:51200/hive_small_data?PROFILE=Hive') FORMAT 'CUSTOM'
> (formatter='pxfwritable_import');*
> CREATE EXTERNAL TABLE
> template1=# select * from ext_table1;
> ERROR:  remote component error (0): (libchurl.c:898)*
> {code}
> When I turned on debug mode in curl, I found this error in logs - "*
> Closing connection 0".
> I found a workaround, to set CURLOPT_RESOLVE option in curl:
> {code}
> struct curl_slist *host = NULL;
> host = curl_slist_append(NULL, "localhost:51200:127.0.0.1");*
> set_curl_option(context, CURLOPT_RESOLVE, host);
> {code}
> It seems like an issue with DNS cache,



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-943) Various issues in hawq register feature_test cases

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385608#comment-15385608
 ] 

ASF GitHub Bot commented on HAWQ-943:
-

Github user yaoj2 commented on the issue:

https://github.com/apache/incubator-hawq/pull/805
  
LGTM


> Various issues in hawq register feature_test cases
> --
>
> Key: HAWQ-943
> URL: https://issues.apache.org/jira/browse/HAWQ-943
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
>
> 1) Do not assume the test database is postgres.
> Use HAWQ_DB which is defined in sql_util.h
> 2) Use error immune options when creating a new hdfs file or directory.
> e.g. mkdir -p, put -f.
> Since nonexistence of those files/directories are not guaranteed.
> e.g. Previous test run was terminated by ctrl+c.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #805: HAWQ-943. Various issues in hawq register feature...

[yaoj2]

Github user yaoj2 commented on the issue:

https://github.com/apache/incubator-hawq/pull/805
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #805: HAWQ-943. Various issues in hawq register ...

[paul-guo-]

GitHub user paul-guo- opened a pull request:

https://github.com/apache/incubator-hawq/pull/805

HAWQ-943. Various issues in hawq register feature_test cases



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/paul-guo-/incubator-hawq test3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/805.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #805


commit 6f95bd9bfe2df9b0a45878a841bdbb47d8ecbf53
Author: Paul Guo 
Date:   2016-07-20T09:35:47Z

HAWQ-943. Various issues in hawq register feature_test cases




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (HAWQ-943) Various issues in hawq register feature_test cases

[Paul Guo (JIRA)]

Paul Guo created HAWQ-943:
-

 Summary: Various issues in hawq register feature_test cases
 Key: HAWQ-943
 URL: https://issues.apache.org/jira/browse/HAWQ-943
 Project: Apache HAWQ
  Issue Type: Bug
Reporter: Paul Guo
Assignee: Lei Chang


1) Do not assume the test database is postgres.
Use HAWQ_DB which is defined in sql_util.h

2) Use error immune options when creating a new hdfs file or directory.
e.g. mkdir -p, put -f.
Since nonexistence of those files/directories are not guaranteed.
e.g. Previous test run was terminated by ctrl+c.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-940) Kerberos Ticket Expired for LibYARN Operations

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385590#comment-15385590
 ] 

ASF GitHub Bot commented on HAWQ-940:
-

GitHub user linwen opened a pull request:

https://github.com/apache/incubator-hawq/pull/804

HAWQ-940. Fix Kerberos ticket expired for libyarn operations

Please review, thanks! 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/linwen/incubator-hawq hawq_940

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/804.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #804


commit 0942630825f6c84da155bd2d9aec4831f7e4d049
Author: Wen Lin 
Date:   2016-07-20T09:07:01Z

HAWQ-940. Fix Kerberos ticket expired for libyarn operations




> Kerberos Ticket Expired for LibYARN Operations
> --
>
> Key: HAWQ-940
> URL: https://issues.apache.org/jira/browse/HAWQ-940
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: libyarn
>Reporter: Lin Wen
>Assignee: Lei Chang
> Fix For: 2.0.1.0-incubating
>
>
> HAWQ's libhdfs3 and libyarn use a same kerberos keyfile. 
> Whenever a hdfs operation is triggered, a function named login() is called, 
> in login() function, this ticket is initialized by "kinit". 
> But for libyarn, login() function is only called during the resource broker 
> process starts. So if HAWQ starts up and there is no query for a long 
> period(24 hours in kerberos's configure file, krb.conf), this ticket will 
> expire, and HAWQ fails to register itself in Hadoop YARN.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #804: HAWQ-940. Fix Kerberos ticket expired for ...

[linwen]

GitHub user linwen opened a pull request:

https://github.com/apache/incubator-hawq/pull/804

HAWQ-940. Fix Kerberos ticket expired for libyarn operations

Please review, thanks! 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/linwen/incubator-hawq hawq_940

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/804.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #804


commit 0942630825f6c84da155bd2d9aec4831f7e4d049
Author: Wen Lin 
Date:   2016-07-20T09:07:01Z

HAWQ-940. Fix Kerberos ticket expired for libyarn operations




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---