date:20170215

[GitHub] incubator-hawq pull request #1124: HAWQ-1330. More ASF header updates (inclu...

2017-02-15 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1124


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1127: HAWQ-1336. Travis CI "brew" updates

2017-02-15 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1127


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (HAWQ-1325) Allow queries related to pg_temp if ranger is enable

2017-02-15 Thread Lin Wen (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lin Wen resolved HAWQ-1325.
---
   Resolution: Fixed
Fix Version/s: (was: 2.3.0.0-incubating)
   2.2.0.0-incubating

> Allow queries related to pg_temp if ranger is enable
> 
>
> Key: HAWQ-1325
> URL: https://issues.apache.org/jira/browse/HAWQ-1325
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Security
>Reporter: Lin Wen
>Assignee: Lin Wen
> Fix For: 2.2.0.0-incubating
>
>
> Queries related to temp will send request to RPS, asking the privilege of 
> schema "pg_temp_XXX", like this:
> ./hawq-2017-02-13_142852.csv:2017-02-13 14:29:29.718445 
> CST,"linw","postgres",p71787,th-1324481600,"[local]",,2017-02-13 14:29:01 
> CST,8477,con13,cmd3,seg-1,,,x8477,sx1,"DEBUG3","0","send json request 
> to ranger : { ""requestId"": ""3"", ""user"": ""linw"", ""clientIp"": 
> ""127.0.0.1"", ""context"": ""select * from temp1;"", ""access"": [ { 
> ""resource"": { ""database"": ""postgres"", ""schema"": ""pg_temp_13"", 
> ""table"": ""temp1"" }, ""privileges"": [ ""select"" ] } ] }",,"select * 
> from temp1;",0,,"rangerrest.c",454,
> In order to better control, for pg_temp_XX schema and objects in that schema, 
> we should fall back these checks to catalog without sending requests to RPS. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (HAWQ-1338) In some case writer process crashed when running 'hawq stop cluster'

2017-02-15 Thread Ming LI (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ming LI resolved HAWQ-1338.
---
   Resolution: Fixed
Fix Version/s: backlog

> In some case writer process crashed when running 'hawq stop cluster'
> 
>
> Key: HAWQ-1338
> URL: https://issues.apache.org/jira/browse/HAWQ-1338
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core
>Reporter: Ming LI
>Assignee: Ming LI
> Fix For: backlog
>
>
> On master node of test machine, some process doesn't exit nicely, and core 
> dump after a while.
> {code}
> --- The running log  -
> 2/12/17 11:33:59 PM PST: 
> --
> 2/12/17 11:33:59 PM PST: Check if postgres/java processes are closed properly:
> 2/12/17 11:33:59 PM PST: 
> --
> 2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test1: 
> 2/12/17 11:33:59 PM PST: gpadmin5279  1  0 22:53 ?00:00:03 
> postgres: port 31000, master logger process   
>   
>   
> 2/12/17 11:33:59 PM PST: gpadmin5283  1  0 22:53 ?00:00:01 
> postgres: port 31000, writer process  
>   
>   
> 2/12/17 11:33:59 PM PST: root  23864 24  1 23:37 ?00:00:01 
> /usr/libexec/abrt-hook-ccpp 6 18446744073709551615 5283 501 501 1486971433 
> postgres
> 2/12/17 11:33:59 PM PST: -
> 2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test2: 
> 2/12/17 11:33:59 PM PST: -
> 2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test3: 
> 2/12/17 11:33:59 PM PST: -
> 2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test4: 
> 2/12/17 11:33:59 PM PST: -
> 2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test5: 
> 2/12/17 11:33:59 PM PST: -
> 2/12/17 11:33:59 PM PST: ERROR: Postgres process not closed on test1, please 
> check.
> 2/12/17 11:33:59 PM PST: 
> --
> --- The call stack -
> (gdb) bt
> #0  0x0032214325e5 in raise () from /lib64/libc.so.6
> #1  0x003221433dc5 in abort () from /lib64/libc.so.6
> #2  0x0096433a in errfinish (dummy=0) at elog.c:686
> #3  0x009665bd in elog_finish (elevel=22, fmt=0xc53af0 "process is 
> dying from critical section") at elog.c:1463
> #4  0x0086c11d in proc_exit_prepare (code=1) at ipc.c:153
> #5  0x0086c0a9 in proc_exit (code=1) at ipc.c:93
> #6  0x00964300 in errfinish (dummy=0) at elog.c:670
> #7  0x00825121 in ServiceClientRead (serviceClient=0xfc73f0, 
> response=0x7fffb96842de, responseLen=1,
> timeout=0x7fffb96842c0) at service.c:523
> #8  0x00824f7b in ServiceClientReceiveResponse 
> (serviceClient=0xfc73f0, response=0x7fffb96842de, responseLen=1,
> timeout=0x7fffb96842c0) at service.c:480
> #9  0x0082bce1 in WalSendServerClientReceiveResponse 
> (walSendResponse=0x7fffb96842de, timeout=0x7fffb96842c0)
> at walsendserver.c:372
> #10 0x0051596d in XLogQDMirrorWaitForResponse (waitForever=0 '\000') 
> at xlog.c:1919
> #11 0x00515c0c in XLogQDMirrorWrite (startidx=0, npages=1, 
> timeLineID=1, logId=0, logSeg=1, logOff=13729792)
> at xlog.c:2005
> #12 0x00516615 in XLogWrite (WriteRqst=..., flexible=0 '\000', 
> xlog_switch=0 '\000') at xlog.c:2354
> #13 0x00516d68 in XLogFlush (record=...) at xlog.c:2572
> #14 0x00522f88 in CreateCheckPoint (shutdown=1 '\001', force=1 
> '\001') at xlog.c:8136
> #15 0x0052277b in ShutdownXLOG (code=0, arg=0) at xlog.c:7865
> #16 0x00821f42 in BackgroundWriterMain () at bgwriter.c:318
> #17 0x0059c9f1 in AuxiliaryProcessMain (argc=2, argv=0x7fffb9684b60) 
> at bootstrap.c:467
> #18 0x0083c7b0 in StartChildProcess (type=BgWriterProcess) at 
> postmaster.c:6836
> #19 0x00838f39 in CommenceNormalOperations () at postmaster.c:3618
> #20 0x0083984a in do_reaper () at postmaster.c:4021
> #21 0x00835e97 in ServerLoop () at postmaster.c:2136
> #22 0x0083500f in PostmasterMain (argc=9, argv=0x288bd10) at 
> postmaster.c:1454
> #23 0x007612af in main (argc=9, argv=0x288bd10) at main.c:226
> {code}



--
This

[GitHub] incubator-hawq issue #1127: HAWQ-1336. Travis CI "brew" updates

2017-02-15 Thread paul-guo-

Github user paul-guo- commented on the issue:

https://github.com/apache/incubator-hawq/pull/1127
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1129: HAWQ-1325. Allow queries related to pg_te...

2017-02-15 Thread linwen

Github user linwen closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1129


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #1124: HAWQ-1330. More ASF header updates (including po...

2017-02-15 Thread huor

Github user huor commented on the issue:

https://github.com/apache/incubator-hawq/pull/1124
  
@edespino, thanks for the update. +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (HAWQ-1325) Allow queries related to pg_temp if ranger is enable

2017-02-15 Thread Lin Wen (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lin Wen reassigned HAWQ-1325:
-

Assignee: Lin Wen  (was: Ed Espino)

> Allow queries related to pg_temp if ranger is enable
> 
>
> Key: HAWQ-1325
> URL: https://issues.apache.org/jira/browse/HAWQ-1325
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Security
>Reporter: Lin Wen
>Assignee: Lin Wen
> Fix For: 2.3.0.0-incubating
>
>
> Queries related to temp will send request to RPS, asking the privilege of 
> schema "pg_temp_XXX", like this:
> ./hawq-2017-02-13_142852.csv:2017-02-13 14:29:29.718445 
> CST,"linw","postgres",p71787,th-1324481600,"[local]",,2017-02-13 14:29:01 
> CST,8477,con13,cmd3,seg-1,,,x8477,sx1,"DEBUG3","0","send json request 
> to ranger : { ""requestId"": ""3"", ""user"": ""linw"", ""clientIp"": 
> ""127.0.0.1"", ""context"": ""select * from temp1;"", ""access"": [ { 
> ""resource"": { ""database"": ""postgres"", ""schema"": ""pg_temp_13"", 
> ""table"": ""temp1"" }, ""privileges"": [ ""select"" ] } ] }",,"select * 
> from temp1;",0,,"rangerrest.c",454,
> In order to better control, for pg_temp_XX schema and objects in that schema, 
> we should fall back these checks to catalog without sending requests to RPS. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HAWQ-1339) Cache lookup failed after explain OLAP grouping query

2017-02-15 Thread Amy (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amy updated HAWQ-1339:
--
Description: 
Some OLAP grouping query may error out with "division by zero", and when do 
query explain, notice of "cache lookup failed for attribute 7 of relation 75036 
(lsyscache.c:437)" occurred.
{code}
postgres=# SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
TO_CHAR(COALESCE(MAX(DISTINCT 
floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
postgres-# FROM sale,customer,vendor
postgres-# WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
postgres-# GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
ERROR:  division by zero  (seg0 localhost:4 pid=25205)
postgres=#
postgres=# explain SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
TO_CHAR(COALESCE(MAX(DISTINCT 
floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
FROM sale,customer,vendor
WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
NOTICE:  cache lookup failed for attribute 7 of relation 75036 (lsyscache.c:437)
{code}

The reproduction steps are:
{code}
Step 1: Prepare schema and data using attached olap_setup.sql

Step 2: Run below OLAP grouping query
-- OLAP query involving MAX() function
SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), TO_CHAR(COALESCE(MAX(DISTINCT 
floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
FROM sale,customer,vendor
WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;

explain SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
TO_CHAR(COALESCE(MAX(DISTINCT 
floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
FROM sale,customer,vendor
WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
{code}

  was:
Some explain on OLAP grouping query may encounter error "cache lookup failed 
for attribute 7 of relation 75036 (lsyscache.c:437)".
'''
postgres=# SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
TO_CHAR(COALESCE(MAX(DISTINCT 
floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
postgres-# FROM sale,customer,vendor
postgres-# WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
postgres-# GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
ERROR:  division by zero  (seg0 localhost:4 pid=25205)
postgres=#
postgres=# explain SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
TO_CHAR(COALESCE(MAX(DISTINCT floor 
'9 99.   FROM sale,customer,vendor  
  WHERE sale.cn=customer.cn AND 
sale.vn=vendor.vn  GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
NOTICE:  cache lookup failed for attribute 7 of relation 75036 (lsyscache.c:437)
'''

The reproduction steps are:
'''
Step 1: Prepare schema and data using attached olap_setup.sql

Step 2: Run below OLAP grouping query
-- OLAP query involving MAX() function
SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), TO_CHAR(COALESCE(MAX(DISTINCT

[jira] [Updated] (HAWQ-1339) Cache lookup failed after explain OLAP grouping query

2017-02-15 Thread Amy (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amy updated HAWQ-1339:
--
Attachment: olap_setup.sql

> Cache lookup failed after explain OLAP grouping query
> -
>
> Key: HAWQ-1339
> URL: https://issues.apache.org/jira/browse/HAWQ-1339
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Catalog
>Reporter: Amy
>Assignee: Amy
> Fix For: 2.3.0.0-incubating
>
> Attachments: olap_setup.sql
>
>
> Some explain on OLAP grouping query may encounter error "cache lookup failed 
> for attribute 7 of relation 75036 (lsyscache.c:437)".
> '''
> postgres=# SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
> TO_CHAR(COALESCE(MAX(DISTINCT 
> floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
> postgres-# FROM sale,customer,vendor
> postgres-# WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
> postgres-# GROUP BY 
> ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
>  HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
> ERROR:  division by zero  (seg0 localhost:4 pid=25205)
> postgres=#
> postgres=# explain SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
> TO_CHAR(COALESCE(MAX(DISTINCT floor   
>   '9 99.   FROM sale,customer,vendor  
>   WHERE sale.cn=customer.cn 
> AND sale.vn=vendor.vn  
> GROUP BY 
> ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
>  HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
> NOTICE:  cache lookup failed for attribute 7 of relation 75036 
> (lsyscache.c:437)
> '''
> The reproduction steps are:
> '''
> Step 1: Prepare schema and data using attached olap_setup.sql
> Step 2: Run below OLAP grouping query
> -- OLAP query involving MAX() function
> SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
> TO_CHAR(COALESCE(MAX(DISTINCT 
> floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
> FROM sale,customer,vendor
> WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
> GROUP BY 
> ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
>  HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
> explain SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
> TO_CHAR(COALESCE(MAX(DISTINCT 
> floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
> FROM sale,customer,vendor
> WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
> GROUP BY 
> ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
>  HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
> '''



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HAWQ-1339) Cache lookup failed after explain OLAP grouping query

2017-02-15 Thread Ruilong Huo (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-1339:
--
Description: 
Some explain on OLAP grouping query may encounter error "cache lookup failed 
for attribute 7 of relation 75036 (lsyscache.c:437)".
'''
postgres=# SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
TO_CHAR(COALESCE(MAX(DISTINCT 
floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
postgres-# FROM sale,customer,vendor
postgres-# WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
postgres-# GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
ERROR:  division by zero  (seg0 localhost:4 pid=25205)
postgres=#
postgres=# explain SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
TO_CHAR(COALESCE(MAX(DISTINCT floor 
'9 99.   FROM sale,customer,vendor  
  WHERE sale.cn=customer.cn AND 
sale.vn=vendor.vn  GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
NOTICE:  cache lookup failed for attribute 7 of relation 75036 (lsyscache.c:437)
'''

The reproduction steps are:
'''
Step 1: Prepare schema and data using attached olap_setup.sql

Step 2: Run below OLAP grouping query
-- OLAP query involving MAX() function
SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), TO_CHAR(COALESCE(MAX(DISTINCT 
floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
FROM sale,customer,vendor
WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;

explain SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
TO_CHAR(COALESCE(MAX(DISTINCT 
floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
FROM sale,customer,vendor
WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
'''

  was:
Some OLAP grouping query may error out with "division by zero", and when do 
query explain, notice of "cache lookup failed for attribute 7 of relation 75036 
(lsyscache.c:437)" occurred.
'''
postgres=# SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
TO_CHAR(COALESCE(MAX(DISTINCT 
floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
postgres-# FROM sale,customer,vendor
postgres-# WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
postgres-# GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
ERROR:  division by zero  (seg0 localhost:4 pid=25205)
postgres=#
postgres=# explain SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
TO_CHAR(COALESCE(MAX(DISTINCT floor 
'9 99.   FROM sale,customer,vendor  
  WHERE sale.cn=customer.cn AND 
sale.vn=vendor.vn  GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
NOTICE:  cache lookup failed for attribute 7 of relation 75036 (lsyscache.c:437)
'''

The reproduction steps are:
'''
Step 1: Prepare schema and data using attached olap_setup.sql

Step 2: Run below OLAP grouping query
-- OLAP query involving MAX() function
SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), TO_CHAR(COALESCE(MAX(DISTINCT

[jira] [Assigned] (HAWQ-1339) Cache lookup failed after explain OLAP grouping query

2017-02-15 Thread Amy (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amy reassigned HAWQ-1339:
-

Assignee: Amy  (was: Ed Espino)

> Cache lookup failed after explain OLAP grouping query
> -
>
> Key: HAWQ-1339
> URL: https://issues.apache.org/jira/browse/HAWQ-1339
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Catalog
>Reporter: Amy
>Assignee: Amy
> Fix For: 2.3.0.0-incubating
>
>
> Some OLAP grouping query may error out with "division by zero", and when do 
> query explain, notice of "cache lookup failed for attribute 7 of relation 
> 75036 (lsyscache.c:437)" occurred.
> '''
> postgres=# SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
> TO_CHAR(COALESCE(MAX(DISTINCT 
> floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
> postgres-# FROM sale,customer,vendor
> postgres-# WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
> postgres-# GROUP BY 
> ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
>  HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
> ERROR:  division by zero  (seg0 localhost:4 pid=25205)
> postgres=#
> postgres=# explain SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
> TO_CHAR(COALESCE(MAX(DISTINCT floor   
>   '9 99.   FROM sale,customer,vendor  
>   WHERE sale.cn=customer.cn 
> AND sale.vn=vendor.vn  
> GROUP BY 
> ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
>  HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
> NOTICE:  cache lookup failed for attribute 7 of relation 75036 
> (lsyscache.c:437)
> '''
> The reproduction steps are:
> '''
> Step 1: Prepare schema and data using attached olap_setup.sql
> Step 2: Run below OLAP grouping query
> -- OLAP query involving MAX() function
> SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
> TO_CHAR(COALESCE(MAX(DISTINCT 
> floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
> FROM sale,customer,vendor
> WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
> GROUP BY 
> ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
>  HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
> explain SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
> TO_CHAR(COALESCE(MAX(DISTINCT 
> floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
> FROM sale,customer,vendor
> WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
> GROUP BY 
> ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
>  HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
> '''



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HAWQ-1339) Cache lookup failed after explain OLAP grouping query

2017-02-15 Thread Amy (JIRA)

Amy created HAWQ-1339:
-

 Summary: Cache lookup failed after explain OLAP grouping query
 Key: HAWQ-1339
 URL: https://issues.apache.org/jira/browse/HAWQ-1339
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Catalog
Reporter: Amy
Assignee: Ed Espino
 Fix For: 2.3.0.0-incubating


Some OLAP grouping query may error out with "division by zero", and when do 
query explain, notice of "cache lookup failed for attribute 7 of relation 75036 
(lsyscache.c:437)" occurred.
'''
postgres=# SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
TO_CHAR(COALESCE(MAX(DISTINCT 
floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
postgres-# FROM sale,customer,vendor
postgres-# WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
postgres-# GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
ERROR:  division by zero  (seg0 localhost:4 pid=25205)
postgres=#
postgres=# explain SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
TO_CHAR(COALESCE(MAX(DISTINCT floor 
'9 99.   FROM sale,customer,vendor  
  WHERE sale.cn=customer.cn AND 
sale.vn=vendor.vn  GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
NOTICE:  cache lookup failed for attribute 7 of relation 75036 (lsyscache.c:437)
'''

The reproduction steps are:
'''
Step 1: Prepare schema and data using attached olap_setup.sql

Step 2: Run below OLAP grouping query
-- OLAP query involving MAX() function
SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), TO_CHAR(COALESCE(MAX(DISTINCT 
floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
FROM sale,customer,vendor
WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;

explain SELECT sale.vn,sale.cn,sale.dt,GROUPING(sale.vn), 
TO_CHAR(COALESCE(MAX(DISTINCT 
floor(sale.vn+sale.qty)),0),'.999'),TO_CHAR(COALESCE(VAR_SAMP(floor(sale.pn/sale.prc)),0),'.999'),TO_CHAR(COALESCE(COUNT(floor(sale.qty+sale.prc)),0),'.999')
FROM sale,customer,vendor
WHERE sale.cn=customer.cn AND sale.vn=vendor.vn
GROUP BY 
ROLLUP((sale.prc),(sale.vn,sale.vn),(sale.pn,sale.pn),(sale.dt),(sale.qty,sale.vn,sale.qty)),ROLLUP((sale.pn),(sale.vn,sale.pn),(sale.qty)),(),sale.cn
 HAVING COALESCE(VAR_POP(sale.cn),0) >= 45.5839785564113;
'''



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-hawq pull request #1115: HAWQ-1324. Fixed crash at query cancel, s...

2017-02-15 Thread liming01

Github user liming01 closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1115


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1128: HAWQ-1338. Fixed writer process doesn't e...

2017-02-15 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1128


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #1128: HAWQ-1338. Fixed writer process doesn't exit nic...

2017-02-15 Thread amyrazz44

Github user amyrazz44 commented on the issue:

https://github.com/apache/incubator-hawq/pull/1128
  
LGTM +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #1128: HAWQ-1338. Fixed writer process doesn't exit nic...

2017-02-15 Thread wengyanqing

Github user wengyanqing commented on the issue:

https://github.com/apache/incubator-hawq/pull/1128
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-1338) In some case writer process crashed when running 'hawq stop cluster'

2017-02-15 Thread Ming LI (JIRA)


[ 
https://issues.apache.org/jira/browse/HAWQ-1338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869175#comment-15869175
 ] 

Ming LI commented on HAWQ-1338:
---

(1) From the bt in the core file, it seems that SuppressPanic is true, we still 
report panic even at process exist function. We should suppress the panic if 
there is no side effect. 
(2) Now hawq stop utility stop standby after master stopped successfully or 
failed to stop with 10 minutes timeout, so it seems no need to change.

> In some case writer process crashed when running 'hawq stop cluster'
> 
>
> Key: HAWQ-1338
> URL: https://issues.apache.org/jira/browse/HAWQ-1338
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core
>Reporter: Ming LI
>Assignee: Ming LI
>
> On master node of test machine, some process doesn't exit nicely, and core 
> dump after a while.
> {code}
> --- The running log  -
> 2/12/17 11:33:59 PM PST: 
> --
> 2/12/17 11:33:59 PM PST: Check if postgres/java processes are closed properly:
> 2/12/17 11:33:59 PM PST: 
> --
> 2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test1: 
> 2/12/17 11:33:59 PM PST: gpadmin5279  1  0 22:53 ?00:00:03 
> postgres: port 31000, master logger process   
>   
>   
> 2/12/17 11:33:59 PM PST: gpadmin5283  1  0 22:53 ?00:00:01 
> postgres: port 31000, writer process  
>   
>   
> 2/12/17 11:33:59 PM PST: root  23864 24  1 23:37 ?00:00:01 
> /usr/libexec/abrt-hook-ccpp 6 18446744073709551615 5283 501 501 1486971433 
> postgres
> 2/12/17 11:33:59 PM PST: -
> 2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test2: 
> 2/12/17 11:33:59 PM PST: -
> 2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test3: 
> 2/12/17 11:33:59 PM PST: -
> 2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test4: 
> 2/12/17 11:33:59 PM PST: -
> 2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test5: 
> 2/12/17 11:33:59 PM PST: -
> 2/12/17 11:33:59 PM PST: ERROR: Postgres process not closed on test1, please 
> check.
> 2/12/17 11:33:59 PM PST: 
> --
> --- The call stack -
> (gdb) bt
> #0  0x0032214325e5 in raise () from /lib64/libc.so.6
> #1  0x003221433dc5 in abort () from /lib64/libc.so.6
> #2  0x0096433a in errfinish (dummy=0) at elog.c:686
> #3  0x009665bd in elog_finish (elevel=22, fmt=0xc53af0 "process is 
> dying from critical section") at elog.c:1463
> #4  0x0086c11d in proc_exit_prepare (code=1) at ipc.c:153
> #5  0x0086c0a9 in proc_exit (code=1) at ipc.c:93
> #6  0x00964300 in errfinish (dummy=0) at elog.c:670
> #7  0x00825121 in ServiceClientRead (serviceClient=0xfc73f0, 
> response=0x7fffb96842de, responseLen=1,
> timeout=0x7fffb96842c0) at service.c:523
> #8  0x00824f7b in ServiceClientReceiveResponse 
> (serviceClient=0xfc73f0, response=0x7fffb96842de, responseLen=1,
> timeout=0x7fffb96842c0) at service.c:480
> #9  0x0082bce1 in WalSendServerClientReceiveResponse 
> (walSendResponse=0x7fffb96842de, timeout=0x7fffb96842c0)
> at walsendserver.c:372
> #10 0x0051596d in XLogQDMirrorWaitForResponse (waitForever=0 '\000') 
> at xlog.c:1919
> #11 0x00515c0c in XLogQDMirrorWrite (startidx=0, npages=1, 
> timeLineID=1, logId=0, logSeg=1, logOff=13729792)
> at xlog.c:2005
> #12 0x00516615 in XLogWrite (WriteRqst=..., flexible=0 '\000', 
> xlog_switch=0 '\000') at xlog.c:2354
> #13 0x00516d68 in XLogFlush (record=...) at xlog.c:2572
> #14 0x00522f88 in CreateCheckPoint (shutdown=1 '\001', force=1 
> '\001') at xlog.c:8136
> #15 0x0052277b in ShutdownXLOG (code=0, arg=0) at xlog.c:7865
> #16 0x00821f42 in BackgroundWriterMain () at bgwriter.c:318
> #17 0x0059c9f1 in AuxiliaryProcessMain (argc=2, argv=0x7fffb9684b60) 
> at bootstrap.c:467
> #18 0x0083c7b0 in StartChildProcess (type=BgWriterProcess) at 
> postmaster.c:6836
> #19 0x00838f39 in CommenceNormalOperations () at postmaster.c:3618
> #20 0x0083984a in

[GitHub] incubator-hawq pull request #1128: HAWQ-1338. Fixed writer process doesn't e...

2017-02-15 Thread liming01

GitHub user liming01 opened a pull request:

https://github.com/apache/incubator-hawq/pull/1128

HAWQ-1338. Fixed writer process doesn't exit nicely in some case

Signed-off-by: Ivan Weng 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/liming01/incubator-hawq mli/HAWQ-1338

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/1128.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1128


commit 946fe58ee83eb445849a76524be894e34f822114
Author: Ming LI 
Date:   2017-02-16T04:34:21Z

HAWQ-1338. Fixed writer process doesn't exit nicely in some case

Signed-off-by: Ivan Weng 




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (HAWQ-1338) In some case writer process crashed when running 'hawq stop cluster'

2017-02-15 Thread Ming LI (JIRA)

Ming LI created HAWQ-1338:
-

 Summary: In some case writer process crashed when running 'hawq 
stop cluster'
 Key: HAWQ-1338
 URL: https://issues.apache.org/jira/browse/HAWQ-1338
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Core
Reporter: Ming LI
Assignee: Ed Espino


On master node of test machine, some process doesn't exit nicely, and core dump 
after a while.

{code}
--- The running log  -
2/12/17 11:33:59 PM PST: 
--
2/12/17 11:33:59 PM PST: Check if postgres/java processes are closed properly:
2/12/17 11:33:59 PM PST: 
--
2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test1: 
2/12/17 11:33:59 PM PST: gpadmin5279  1  0 22:53 ?00:00:03 
postgres: port 31000, master logger process 
  
2/12/17 11:33:59 PM PST: gpadmin5283  1  0 22:53 ?00:00:01 
postgres: port 31000, writer process
  
2/12/17 11:33:59 PM PST: root  23864 24  1 23:37 ?00:00:01 
/usr/libexec/abrt-hook-ccpp 6 18446744073709551615 5283 501 501 1486971433 
postgres
2/12/17 11:33:59 PM PST: -
2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test2: 
2/12/17 11:33:59 PM PST: -
2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test3: 
2/12/17 11:33:59 PM PST: -
2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test4: 
2/12/17 11:33:59 PM PST: -
2/12/17 11:33:59 PM PST: Check if postgres|java process is running on test5: 
2/12/17 11:33:59 PM PST: -
2/12/17 11:33:59 PM PST: ERROR: Postgres process not closed on test1, please 
check.
2/12/17 11:33:59 PM PST: 
--
--- The call stack -
(gdb) bt
#0  0x0032214325e5 in raise () from /lib64/libc.so.6
#1  0x003221433dc5 in abort () from /lib64/libc.so.6
#2  0x0096433a in errfinish (dummy=0) at elog.c:686
#3  0x009665bd in elog_finish (elevel=22, fmt=0xc53af0 "process is 
dying from critical section") at elog.c:1463
#4  0x0086c11d in proc_exit_prepare (code=1) at ipc.c:153
#5  0x0086c0a9 in proc_exit (code=1) at ipc.c:93
#6  0x00964300 in errfinish (dummy=0) at elog.c:670
#7  0x00825121 in ServiceClientRead (serviceClient=0xfc73f0, 
response=0x7fffb96842de, responseLen=1,
timeout=0x7fffb96842c0) at service.c:523
#8  0x00824f7b in ServiceClientReceiveResponse (serviceClient=0xfc73f0, 
response=0x7fffb96842de, responseLen=1,
timeout=0x7fffb96842c0) at service.c:480
#9  0x0082bce1 in WalSendServerClientReceiveResponse 
(walSendResponse=0x7fffb96842de, timeout=0x7fffb96842c0)
at walsendserver.c:372
#10 0x0051596d in XLogQDMirrorWaitForResponse (waitForever=0 '\000') at 
xlog.c:1919
#11 0x00515c0c in XLogQDMirrorWrite (startidx=0, npages=1, 
timeLineID=1, logId=0, logSeg=1, logOff=13729792)
at xlog.c:2005
#12 0x00516615 in XLogWrite (WriteRqst=..., flexible=0 '\000', 
xlog_switch=0 '\000') at xlog.c:2354
#13 0x00516d68 in XLogFlush (record=...) at xlog.c:2572
#14 0x00522f88 in CreateCheckPoint (shutdown=1 '\001', force=1 '\001') 
at xlog.c:8136
#15 0x0052277b in ShutdownXLOG (code=0, arg=0) at xlog.c:7865
#16 0x00821f42 in BackgroundWriterMain () at bgwriter.c:318
#17 0x0059c9f1 in AuxiliaryProcessMain (argc=2, argv=0x7fffb9684b60) at 
bootstrap.c:467
#18 0x0083c7b0 in StartChildProcess (type=BgWriterProcess) at 
postmaster.c:6836
#19 0x00838f39 in CommenceNormalOperations () at postmaster.c:3618
#20 0x0083984a in do_reaper () at postmaster.c:4021
#21 0x00835e97 in ServerLoop () at postmaster.c:2136
#22 0x0083500f in PostmasterMain (argc=9, argv=0x288bd10) at 
postmaster.c:1454
#23 0x007612af in main (argc=9, argv=0x288bd10) at main.c:226
{code}






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (HAWQ-1334) QD thread should set error code if failing so that the main process for the query could exit soon

2017-02-15 Thread Paul Guo (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Guo closed HAWQ-1334.
--
   Resolution: Fixed
Fix Version/s: 2.2.0.0-incubating

> QD thread should set error code if failing so that the main process for the 
> query could exit soon
> -
>
> Key: HAWQ-1334
> URL: https://issues.apache.org/jira/browse/HAWQ-1334
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Dispatcher
>Reporter: Paul Guo
>Assignee: Paul Guo
> Fix For: 2.2.0.0-incubating
>
>
> In QD thread function dispmgt_thread_func_run(), if there are failures either 
> due to QE or QD itself, it will cancel the query and then clean up. The main 
> process for the query needs the error code of meleeResults be set so that it 
> soon proceeds to cancel the query, else we have to wait for timeout. 
> Typically dispmgt_thread_func_run() should set the error code, however I 
> found there are some cases who do not handle this, e.g. if poll() fails with 
> ENOMEM. One symptom of this issue is that we could sometimes see hang if a 
> query is canceled for some reasons.
> The potential solution is that:
> 1) We expect each branch jump ("goto error_cleanup") set proper error code 
> itself. It is not an easy job.
> 2) We add a "guard" function in the error_cleanup code to set an error code 
> if it is not set, i.e. 1) is not well done.
> I'd this JIRA cares about 2).
> In general, the cleanup code in QD seems to be really obscure and not 
> elegant. Maybe we should file another JIRA to refactor the error handling 
> logic in it. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-hawq issue #1126: HAWQ-1334. QD thread should set error code if fa...

2017-02-15 Thread paul-guo-

Github user paul-guo- commented on the issue:

https://github.com/apache/incubator-hawq/pull/1126
  
Merged. (Discussed with ruilong. The filename 
executormgr_seterrcode_if_needed does not need a modification while move the 
NULL pointer check into the function.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1126: HAWQ-1334. QD thread should set error cod...

2017-02-15 Thread paul-guo-

Github user paul-guo- closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1126


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Closed] (HAWQ-1315) Function validateResourcePoolStatus() in resourcepool.c is logging the wrong information

2017-02-15 Thread Amy (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amy closed HAWQ-1315.
-
Resolution: Fixed

> Function validateResourcePoolStatus() in resourcepool.c is logging the wrong 
> information
> 
>
> Key: HAWQ-1315
> URL: https://issues.apache.org/jira/browse/HAWQ-1315
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Resource Manager
>Reporter: Xiang Sheng
>Assignee: Amy
> Fix For: 2.3.0.0-incubating
>
>
> The function "validateResourcePoolStatus()" in "resourcepool.c" is not 
> logging the correct information in the message printed by "elog()" function 
> in line 4123. In the snippet below:
> {code}
> if ( totalallocmem > mem || totalalloccore > core )
> {
>  elog(WARNING, "HAWQ RM Validation. Allocated too much resource in 
> resource "
>"pool (%d MB, %lf CORE), maximum capacity (%d MB, %d 
> CORE)",
>totalallocmem,
>totalalloccore,
>core,
>mem);
> }
> {code}
> The third and fourth parameters ('core' and 'mem') are swapped; the third 
> string placeholder should be the maximum memory capacity, but it is printing 
> the cores. The same happens with the fourth string placeholder.
> This leads to log messages of this kind in hawq master log:
> {code}
> 2017-02-02 01:30:03.014708 
> CET,,,p351048,th-3843744960,con4,,seg-1,"WARNING","01000","HAWQ 
> RM Validation. Allocated too much resource in resource pool (49152 MB, 
> 6.00 CORE), maximum capacity (5 MB, 40960 
> CORE)",,,0,,"resourcepool.c",4123,
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-hawq pull request #1125: HAWQ-1315. Function validateResourcePoolS...

2017-02-15 Thread amyrazz44

Github user amyrazz44 closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1125


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (HAWQ-1336) HAWQ Travis CI build service is failing

2017-02-15 Thread Ed Espino (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ed Espino updated HAWQ-1336:

Fix Version/s: 2.2.0.0-incubating

> HAWQ Travis CI build service is failing
> ---
>
> Key: HAWQ-1336
> URL: https://issues.apache.org/jira/browse/HAWQ-1336
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Build
>Reporter: Ed Espino
>Assignee: Ed Espino
> Fix For: 2.2.0.0-incubating
>
>
> The HAWQ Travis CI builds are currently failing with the following error:
> {code}
> Error: libevent-2.0.22 already installed
> ...
> The command "brew install protobuf protobuf-c Gsasl openssl thrift ccache 
> snappy libevent bison cpanm maven" failed and exited with 1 during .
> Your build has been stopped.
> /Users/travis/build.sh: line 155: shell_session_update: command not found
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Reopened] (HAWQ-1157) Make consistent docs link to PostgreSQL 8.2

2017-02-15 Thread Ed Espino (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ed Espino reopened HAWQ-1157:
-

> Make consistent docs link to PostgreSQL 8.2
> ---
>
> Key: HAWQ-1157
> URL: https://issues.apache.org/jira/browse/HAWQ-1157
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Documentation
>Reporter: Jane Beckman
>Assignee: David Yozie
> Fix For: 2.1.0.0-incubating
>
>
> Some links to Postgresql link to other versions of PostgreSQL than 8.2. HAWQ 
> is based on PostgreSQL 8.2.15.  One section mentions PostgreSQL 9.0 
> specifically. These references should standardize on 8.2.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HAWQ-1157) Make consistent docs link to PostgreSQL 8.2

2017-02-15 Thread Ed Espino (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ed Espino updated HAWQ-1157:

Fix Version/s: (was: 2.2.0.0-incubating)

> Make consistent docs link to PostgreSQL 8.2
> ---
>
> Key: HAWQ-1157
> URL: https://issues.apache.org/jira/browse/HAWQ-1157
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Documentation
>Reporter: Jane Beckman
>Assignee: David Yozie
> Fix For: 2.1.0.0-incubating
>
>
> Some links to Postgresql link to other versions of PostgreSQL than 8.2. HAWQ 
> is based on PostgreSQL 8.2.15.  One section mentions PostgreSQL 9.0 
> specifically. These references should standardize on 8.2.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (HAWQ-1267) NOTICE file need to be updated to reflect the right year for copyright

2017-02-15 Thread Ed Espino (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ed Espino closed HAWQ-1267.
---
Resolution: Fixed

> NOTICE file need to be updated to reflect the right year for copyright
> --
>
> Key: HAWQ-1267
> URL: https://issues.apache.org/jira/browse/HAWQ-1267
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Documentation
>Affects Versions: 2.1.0.0-incubating
>Reporter: Ruilong Huo
>Assignee: Ed Espino
> Fix For: 2.1.0.0-incubating
>
>
> The NOTICE file need to updated to reflect the right year for copyright for 
> apache hawq 2.1.0.0-incubating release.
> To be specific, update
> {noformat}
> Apache HAWQ (incubating)
> Copyright 2016 The Apache Software Foundation.
> {noformat}
> to
> {noformat}
> Apache HAWQ (incubating)
> Copyright 2017 The Apache Software Foundation.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (HAWQ-1157) Make consistent docs link to PostgreSQL 8.2

2017-02-15 Thread Ed Espino (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ed Espino closed HAWQ-1157.
---
Resolution: Fixed

> Make consistent docs link to PostgreSQL 8.2
> ---
>
> Key: HAWQ-1157
> URL: https://issues.apache.org/jira/browse/HAWQ-1157
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Documentation
>Reporter: Jane Beckman
>Assignee: David Yozie
> Fix For: 2.1.0.0-incubating
>
>
> Some links to Postgresql link to other versions of PostgreSQL than 8.2. HAWQ 
> is based on PostgreSQL 8.2.15.  One section mentions PostgreSQL 9.0 
> specifically. These references should standardize on 8.2.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HAWQ-1268) pom.xml need to be updated to reflect the correct version for apache hawq 2.1.0.0-incubating release

2017-02-15 Thread Ed Espino (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ed Espino updated HAWQ-1268:

Fix Version/s: (was: 2.2.0.0-incubating)

> pom.xml need to be updated to reflect the correct version for apache hawq 
> 2.1.0.0-incubating release
> 
>
> Key: HAWQ-1268
> URL: https://issues.apache.org/jira/browse/HAWQ-1268
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Documentation
>Affects Versions: 2.1.0.0-incubating
>Reporter: Ruilong Huo
>Assignee: Ed Espino
> Fix For: 2.1.0.0-incubating
>
>
> pom.xml need to be updated to reflect the correct version for apache hawq 
> 2.1.0.0-incubating release. Otherwise, RAT check with shows wrong hawq 
> version.
> {noformat}
> rhuo-mbp:apache-hawq-src-2.1.0.0-incubating rhuo$ mvn verify
> [INFO] Scanning for projects...
> [INFO]
> [INFO] 
> 
> [INFO] Building hawq 2.0
> [INFO] 
> 
> Downloading: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-plugin/0.12/apache-rat-plugin-0.12.pom
> Downloaded: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-plugin/0.12/apache-rat-plugin-0.12.pom
>  (10 KB at 1.1 KB/sec)
> Downloading: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-project/0.12/apache-rat-project-0.12.pom
> Downloaded: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-project/0.12/apache-rat-project-0.12.pom
>  (24 KB at 0.8 KB/sec)
> Downloading: 
> https://repo.maven.apache.org/maven2/org/apache/apache/17/apache-17.pom
> ...
> {noformat}
> To be specific, update
> {noformat}
>   org.apache.hawq
>   hawq
>   2.0
>   pom
> {noformat}
> to
> {noformat}
>   org.apache.hawq
>   hawq
>   2.1
>   pom
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (HAWQ-1268) pom.xml need to be updated to reflect the correct version for apache hawq 2.1.0.0-incubating release

2017-02-15 Thread Ed Espino (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ed Espino resolved HAWQ-1268.
-
Resolution: Fixed

> pom.xml need to be updated to reflect the correct version for apache hawq 
> 2.1.0.0-incubating release
> 
>
> Key: HAWQ-1268
> URL: https://issues.apache.org/jira/browse/HAWQ-1268
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Documentation
>Affects Versions: 2.1.0.0-incubating
>Reporter: Ruilong Huo
>Assignee: Ed Espino
> Fix For: 2.1.0.0-incubating
>
>
> pom.xml need to be updated to reflect the correct version for apache hawq 
> 2.1.0.0-incubating release. Otherwise, RAT check with shows wrong hawq 
> version.
> {noformat}
> rhuo-mbp:apache-hawq-src-2.1.0.0-incubating rhuo$ mvn verify
> [INFO] Scanning for projects...
> [INFO]
> [INFO] 
> 
> [INFO] Building hawq 2.0
> [INFO] 
> 
> Downloading: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-plugin/0.12/apache-rat-plugin-0.12.pom
> Downloaded: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-plugin/0.12/apache-rat-plugin-0.12.pom
>  (10 KB at 1.1 KB/sec)
> Downloading: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-project/0.12/apache-rat-project-0.12.pom
> Downloaded: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-project/0.12/apache-rat-project-0.12.pom
>  (24 KB at 0.8 KB/sec)
> Downloading: 
> https://repo.maven.apache.org/maven2/org/apache/apache/17/apache-17.pom
> ...
> {noformat}
> To be specific, update
> {noformat}
>   org.apache.hawq
>   hawq
>   2.0
>   pom
> {noformat}
> to
> {noformat}
>   org.apache.hawq
>   hawq
>   2.1
>   pom
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Reopened] (HAWQ-1268) pom.xml need to be updated to reflect the correct version for apache hawq 2.1.0.0-incubating release

2017-02-15 Thread Ed Espino (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ed Espino reopened HAWQ-1268:
-

updating fix version

> pom.xml need to be updated to reflect the correct version for apache hawq 
> 2.1.0.0-incubating release
> 
>
> Key: HAWQ-1268
> URL: https://issues.apache.org/jira/browse/HAWQ-1268
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Documentation
>Affects Versions: 2.1.0.0-incubating
>Reporter: Ruilong Huo
>Assignee: Ed Espino
> Fix For: 2.1.0.0-incubating
>
>
> pom.xml need to be updated to reflect the correct version for apache hawq 
> 2.1.0.0-incubating release. Otherwise, RAT check with shows wrong hawq 
> version.
> {noformat}
> rhuo-mbp:apache-hawq-src-2.1.0.0-incubating rhuo$ mvn verify
> [INFO] Scanning for projects...
> [INFO]
> [INFO] 
> 
> [INFO] Building hawq 2.0
> [INFO] 
> 
> Downloading: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-plugin/0.12/apache-rat-plugin-0.12.pom
> Downloaded: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-plugin/0.12/apache-rat-plugin-0.12.pom
>  (10 KB at 1.1 KB/sec)
> Downloading: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-project/0.12/apache-rat-project-0.12.pom
> Downloaded: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-project/0.12/apache-rat-project-0.12.pom
>  (24 KB at 0.8 KB/sec)
> Downloading: 
> https://repo.maven.apache.org/maven2/org/apache/apache/17/apache-17.pom
> ...
> {noformat}
> To be specific, update
> {noformat}
>   org.apache.hawq
>   hawq
>   2.0
>   pom
> {noformat}
> to
> {noformat}
>   org.apache.hawq
>   hawq
>   2.1
>   pom
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Reopened] (HAWQ-1267) NOTICE file need to be updated to reflect the right year for copyright

2017-02-15 Thread Ed Espino (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ed Espino reopened HAWQ-1267:
-

> NOTICE file need to be updated to reflect the right year for copyright
> --
>
> Key: HAWQ-1267
> URL: https://issues.apache.org/jira/browse/HAWQ-1267
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Documentation
>Affects Versions: 2.1.0.0-incubating
>Reporter: Ruilong Huo
>Assignee: Ed Espino
> Fix For: 2.1.0.0-incubating
>
>
> The NOTICE file need to updated to reflect the right year for copyright for 
> apache hawq 2.1.0.0-incubating release.
> To be specific, update
> {noformat}
> Apache HAWQ (incubating)
> Copyright 2016 The Apache Software Foundation.
> {noformat}
> to
> {noformat}
> Apache HAWQ (incubating)
> Copyright 2017 The Apache Software Foundation.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HAWQ-1267) NOTICE file need to be updated to reflect the right year for copyright

2017-02-15 Thread Ed Espino (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ed Espino updated HAWQ-1267:

Fix Version/s: (was: 2.2.0.0-incubating)

> NOTICE file need to be updated to reflect the right year for copyright
> --
>
> Key: HAWQ-1267
> URL: https://issues.apache.org/jira/browse/HAWQ-1267
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Documentation
>Affects Versions: 2.1.0.0-incubating
>Reporter: Ruilong Huo
>Assignee: Ed Espino
> Fix For: 2.1.0.0-incubating
>
>
> The NOTICE file need to updated to reflect the right year for copyright for 
> apache hawq 2.1.0.0-incubating release.
> To be specific, update
> {noformat}
> Apache HAWQ (incubating)
> Copyright 2016 The Apache Software Foundation.
> {noformat}
> to
> {noformat}
> Apache HAWQ (incubating)
> Copyright 2017 The Apache Software Foundation.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (HAWQ-1268) pom.xml need to be updated to reflect the correct version for apache hawq 2.1.0.0-incubating release

2017-02-15 Thread Ed Espino (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ed Espino closed HAWQ-1268.
---

> pom.xml need to be updated to reflect the correct version for apache hawq 
> 2.1.0.0-incubating release
> 
>
> Key: HAWQ-1268
> URL: https://issues.apache.org/jira/browse/HAWQ-1268
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Documentation
>Affects Versions: 2.1.0.0-incubating
>Reporter: Ruilong Huo
>Assignee: Ed Espino
> Fix For: 2.1.0.0-incubating
>
>
> pom.xml need to be updated to reflect the correct version for apache hawq 
> 2.1.0.0-incubating release. Otherwise, RAT check with shows wrong hawq 
> version.
> {noformat}
> rhuo-mbp:apache-hawq-src-2.1.0.0-incubating rhuo$ mvn verify
> [INFO] Scanning for projects...
> [INFO]
> [INFO] 
> 
> [INFO] Building hawq 2.0
> [INFO] 
> 
> Downloading: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-plugin/0.12/apache-rat-plugin-0.12.pom
> Downloaded: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-plugin/0.12/apache-rat-plugin-0.12.pom
>  (10 KB at 1.1 KB/sec)
> Downloading: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-project/0.12/apache-rat-project-0.12.pom
> Downloaded: 
> https://repo.maven.apache.org/maven2/org/apache/rat/apache-rat-project/0.12/apache-rat-project-0.12.pom
>  (24 KB at 0.8 KB/sec)
> Downloading: 
> https://repo.maven.apache.org/maven2/org/apache/apache/17/apache-17.pom
> ...
> {noformat}
> To be specific, update
> {noformat}
>   org.apache.hawq
>   hawq
>   2.0
>   pom
> {noformat}
> to
> {noformat}
>   org.apache.hawq
>   hawq
>   2.1
>   pom
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HAWQ-1277) The "make" command generates an error on CentOS 7 when --with-perl is run in configure.

2017-02-15 Thread Ed Espino (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ed Espino updated HAWQ-1277:

Fix Version/s: (was: 2.2.0.0-incubating)

> The "make" command generates an error on CentOS 7 when --with-perl is run in 
> configure.
> ---
>
> Key: HAWQ-1277
> URL: https://issues.apache.org/jira/browse/HAWQ-1277
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Build
>Reporter: Ed Espino
>Assignee: Ed Espino
> Fix For: 2.1.0.0-incubating
>
> Attachments: plperl-GNUmakefile-xsubpp-rhel7.patch
>
>
> The following "make" error is encountered on CentOS 7 when --with-perl is run 
> in configure:
> {code}
> make[3]: Entering directory 
> `/home/centos/apache-hawq-src-2.1.0.0-incubating/src/pl/plperl'
> "/usr/bin/perl" ./text2macro.pl --strip='^(\#.*|\s*)$' 
> plc_perlboot.pl plc_trusted.pl 
> > perlchunks.h
> "/usr/bin/perl" plperl_opmask.pl plperl_opmask.h
> "/usr/bin/perl" /usr/share/perl5/ExtUtils/xsubpp -typemap 
> /usr/share/perl5/ExtUtils/typemap SPI.xs >SPI.c
> "/usr/bin/perl" /usr/share/perl5/ExtUtils/xsubpp -typemap 
> /usr/share/perl5/ExtUtils/typemap Util.xs >Util.c
> Can't open perl script "/usr/share/perl5/ExtUtils/xsubpp": No such file or 
> directory
> Can't open perl script "/usr/share/perl5/ExtUtils/xsubpp": No such file or 
> directory
> make[3]: *** [SPI.c] Error 2
> make[3]: *** Deleting file `SPI.c'
> make[3]: *** Waiting for unfinished jobs
> make[3]: *** [Util.c] Error 2
> make[3]: *** Deleting file `Util.c'
> make[3]: Leaving directory 
> `/home/centos/apache-hawq-src-2.1.0.0-incubating/src/pl/plperl'
> make[2]: *** [all] Error 1
> make[2]: Leaving directory 
> `/home/centos/apache-hawq-src-2.1.0.0-incubating/src/pl'
> make[1]: *** [all] Error 2
> make[1]: Leaving directory 
> `/home/centos/apache-hawq-src-2.1.0.0-incubating/src'
> make: *** [all] Error 2
> {code}
> I also have found a fix for this which was backported from PostgreSQL to the 
> Greenplum DB master branch a year ago by Heikki 
> Linnakangas (Original PostgreSQL committer 
> Andrew Dunstan >):
> https://github.com/greenplum-db/gpdb/commit/f493756a5ac8a90ff1d5e3b84f38a153c9f80b9c#diff-97d34e5e3684082069b6c85876df9107



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-hawq issue #1123: HAWQ-1331. Update hawq and pxf versions

2017-02-15 Thread shivzone

Github user shivzone commented on the issue:

https://github.com/apache/incubator-hawq/pull/1123
  
@ljainpivotalio i'll update it. ideally we shouldn't be maintaining version 
specific files for releases in git 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #1123: HAWQ-1331. Update hawq and pxf versions

2017-02-15 Thread ljainpivotalio

Github user ljainpivotalio commented on the issue:

https://github.com/apache/incubator-hawq/pull/1123
  
Please create 2.2.json in tools/bin/gppylib/data as a copy of 2.1.json


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-1336) HAWQ Travis CI build service is failing

2017-02-15 Thread Ed Espino (JIRA)


[ 
https://issues.apache.org/jira/browse/HAWQ-1336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868550#comment-15868550
 ] 

Ed Espino commented on HAWQ-1336:
-

PR created: https://github.com/apache/incubator-hawq/pull/1127

This is to help diagnose future issues and a potential fix (brew install --> 
brew reinstall).

> HAWQ Travis CI build service is failing
> ---
>
> Key: HAWQ-1336
> URL: https://issues.apache.org/jira/browse/HAWQ-1336
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Build
>Reporter: Ed Espino
>Assignee: Ed Espino
>
> The HAWQ Travis CI builds are currently failing with the following error:
> {code}
> Error: libevent-2.0.22 already installed
> ...
> The command "brew install protobuf protobuf-c Gsasl openssl thrift ccache 
> snappy libevent bison cpanm maven" failed and exited with 1 during .
> Your build has been stopped.
> /Users/travis/build.sh: line 155: shell_session_update: command not found
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-hawq pull request #1122: HAWQ-1314. Added upgrade for function pxf...

2017-02-15 Thread sansanichfb

Github user sansanichfb closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1122


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1127: HAWQ-1336. Travis CI "brew" updates

2017-02-15 Thread edespino

GitHub user edespino opened a pull request:

https://github.com/apache/incubator-hawq/pull/1127

HAWQ-1336. Travis CI "brew" updates

* Display installed brew components and version after brew update.
* Remove libevent installation.

I am not sure this will fix the Travis build issue.  This will give us 
insight to what components are installed in the base brew installation.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/edespino/incubator-hawq HAWQ-1336

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/1127.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1127






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (HAWQ-1336) HAWQ Travis CI build service is failing

2017-02-15 Thread Ed Espino (JIRA)

Ed Espino created HAWQ-1336:
---

 Summary: HAWQ Travis CI build service is failing
 Key: HAWQ-1336
 URL: https://issues.apache.org/jira/browse/HAWQ-1336
 Project: Apache HAWQ
  Issue Type: Task
  Components: Build
Reporter: Ed Espino
Assignee: Ed Espino


The HAWQ Travis CI builds are currently failing with the following error:

{code}
Error: libevent-2.0.22 already installed
...
The command "brew install protobuf protobuf-c Gsasl openssl thrift ccache 
snappy libevent bison cpanm maven" failed and exited with 1 during .
Your build has been stopped.
/Users/travis/build.sh: line 155: shell_session_update: command not found
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-hawq issue #1124: HAWQ-1330. More ASF header updates (including po...

2017-02-15 Thread edespino

Github user edespino commented on the issue:

https://github.com/apache/incubator-hawq/pull/1124
  
@huor - there are no such rules. Our use is dependent on our needs. My main 
goal is to avoid accidentally excluding files from being scanned.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #1126: HAWQ-1334. QD thread should set error code if fa...

2017-02-15 Thread huor

Github user huor commented on the issue:

https://github.com/apache/incubator-hawq/pull/1126
  
+1 while the func name need to be revised.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #1125: HAWQ-1315. Function validateResourcePoolStatus()...

2017-02-15 Thread jiny2

Github user jiny2 commented on the issue:

https://github.com/apache/incubator-hawq/pull/1125
  
pushed into master branch


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (HAWQ-1335) Need to refactor some QD error handling code.

2017-02-15 Thread Paul Guo (JIRA)

Paul Guo created HAWQ-1335:
--

 Summary: Need to refactor some QD error handling code.
 Key: HAWQ-1335
 URL: https://issues.apache.org/jira/browse/HAWQ-1335
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Dispatcher
Reporter: Paul Guo
Assignee: Ed Espino


While recently I worked on QD related issues I found the QD error handling code 
is really confusing. At least:

1) dispmgt_thread_func_run()
/*
 * Cleanup rules:
 * 1. query cancel, result error, and poll error: mark the executor stop.
 * 2. connection error: mark the gang error. Set by 
workermgr_mark_executor_error().
 */

I do not think the code is handling like this. Maybe I did not understand 
related details. Besides workermgr_mark_executor_error() does not exist also.

2) In executormgr_cancel()

#if 0
if (success)
{
executor->state = QES_STOP;
executor->health = QEH_CANCEL;
}
else
{
/* TODO: log error? how to deal with connection error. */
executormgr_catch_error(executor);
}
#endif 

{
write_log("function executormgr_cancel calling 
executormgr_catch_error");
executormgr_catch_error(executor);
}

Why executormgr_catch_error() is called for all cases? and whether "success" is 
not enough to judge whether executormgr_catch_error() should be called or not.

3) cdbdisp_seterrcode()

if (!dispatchResult->errcode)
{
dispatchResult->errcode =
(errcode == 0) ? ERRCODE_INTERNAL_ERROR : errcode;
if (resultIndex >= 0)
dispatchResult->errindex = resultIndex;
}

Why need to set ERRCODE_INTERNAL_ERROR while leave meleeResults->err code 
alone. This piece of code seems to be totally redundant. 

4) It is splitting general errors with normal cancel kinda of interrupts.
However I still see error code below related to cancellation.
if (errcode == ERRCODE_GP_OPERATION_CANCELED ||
errcode == ERRCODE_QUERY_CANCELED)
Is it possible to combine them to just error code only.

5) dispmgt_thread_func_run() should really set error code for each 
error_cleanup cases.

6) dispmgt_thread_func_run() could quit earlier not just because QE, e.g. lack 
of memory on QD so QD fails in some system calls with ENOMEM, while 
cdbdisp_seterrcode() seems to need a related executor.

7) Probably need to rewrite some of the logs and the comments.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (HAWQ-1321) failNames wrongly uses memory context to build message when ANALYZE failed

2017-02-15 Thread Yi Jin (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Jin resolved HAWQ-1321.
--
Resolution: Fixed

> failNames wrongly uses memory context to build message when ANALYZE failed
> --
>
> Key: HAWQ-1321
> URL: https://issues.apache.org/jira/browse/HAWQ-1321
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.1.0.0-incubating
>Reporter: Yi Jin
>Assignee: Yi Jin
> Fix For: 2.2.0.0-incubating
>
>
> I find one bug exist in generating error message for ANALYZE when the message 
> size is large. 
> In analyzeStmt(), there is a variable called failNames. It is initialized in 
> caller's memory context, but it repallocs memory in relation context, and it 
> is freed in statement context. This is bug of wrongly using memory context. 
> when the relation and statement context are dropped at then end of function 
> analyzeStmt(), pat of its content will be flushed with 0. This explain why 
> another block's header was randomly wiped out in the bug.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HAWQ-1332) Can not grant database and schema privileges without table privileges in ranger or ranger plugin service

2017-02-15 Thread Lili Ma (JIRA)


[ 
https://issues.apache.org/jira/browse/HAWQ-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867573#comment-15867573
 ] 

Lili Ma commented on HAWQ-1332:
---

[~adenisso] Seems that this is a bug from RPS. Could you help see it? Thanks

> Can not grant database and schema privileges without table privileges in 
> ranger or ranger plugin service
> 
>
> Key: HAWQ-1332
> URL: https://issues.apache.org/jira/browse/HAWQ-1332
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Security
>Reporter: Chunling Wang
>Assignee: Alexander Denissov
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png
>
>
> We try to grant database connect and schema usage privileges to a non-super 
> user to connect database. We find that if we set policy with database and 
> schema included, but with table excluded, we can not connect database. But if 
> we include table, we can connect to database. We think there may be bug in 
> Ranger Plugin Service or Ranger. Here are steps to reproduce it.
> 1. create a new user "usertest1" in database:
> {code}
> $ psql postgres
> psql (8.2.15)
> Type "help" for help.
> postgres=# CREATE USER usertest1;
> NOTICE:  resource queue required -- using default resource queue "pg_default"
> CREATE ROLE
> postgres=#
> {code}
> 2. add user "usertest1" in pg_hba.conf
> {code}
> local all usertest1 trust
> {code}
> 3. set policy with database and schema included, with table excluded
> !screenshot-1.png|width=800,height=400!
> 4. connect database with user "usertest1" but failed with permission denied
> {code}
> $ psql postgres -U usertest1
> psql: FATAL:  permission denied for database "postgres"
> DETAIL:  User does not have CONNECT privilege.
> {code}
> 5. set policy with database, schema and table included
> !screenshot-2.png|width=800,height=400!
> 6. connect database with user "usertest1" and succeed
> {code}
> $ psql postgres -U usertest1
> psql (8.2.15)
> Type "help" for help.
> postgres=#
> {code}
> But if we do not set table as "*", and specify table like "a", we can not 
> access database either.
> !screenshot-3.png|width=800,height=400!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (HAWQ-1332) Can not grant database and schema privileges without table privileges in ranger or ranger plugin service

2017-02-15 Thread Lili Ma (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lili Ma reassigned HAWQ-1332:
-

Assignee: Alexander Denissov  (was: Ed Espino)

> Can not grant database and schema privileges without table privileges in 
> ranger or ranger plugin service
> 
>
> Key: HAWQ-1332
> URL: https://issues.apache.org/jira/browse/HAWQ-1332
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Security
>Reporter: Chunling Wang
>Assignee: Alexander Denissov
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png
>
>
> We try to grant database connect and schema usage privileges to a non-super 
> user to connect database. We find that if we set policy with database and 
> schema included, but with table excluded, we can not connect database. But if 
> we include table, we can connect to database. We think there may be bug in 
> Ranger Plugin Service or Ranger. Here are steps to reproduce it.
> 1. create a new user "usertest1" in database:
> {code}
> $ psql postgres
> psql (8.2.15)
> Type "help" for help.
> postgres=# CREATE USER usertest1;
> NOTICE:  resource queue required -- using default resource queue "pg_default"
> CREATE ROLE
> postgres=#
> {code}
> 2. add user "usertest1" in pg_hba.conf
> {code}
> local all usertest1 trust
> {code}
> 3. set policy with database and schema included, with table excluded
> !screenshot-1.png|width=800,height=400!
> 4. connect database with user "usertest1" but failed with permission denied
> {code}
> $ psql postgres -U usertest1
> psql: FATAL:  permission denied for database "postgres"
> DETAIL:  User does not have CONNECT privilege.
> {code}
> 5. set policy with database, schema and table included
> !screenshot-2.png|width=800,height=400!
> 6. connect database with user "usertest1" and succeed
> {code}
> $ psql postgres -U usertest1
> psql (8.2.15)
> Type "help" for help.
> postgres=#
> {code}
> But if we do not set table as "*", and specify table like "a", we can not 
> access database either.
> !screenshot-3.png|width=800,height=400!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-hawq pull request #1113: HAWQ-1321. failNames wrongly uses memory ...

2017-02-15 Thread jiny2

Github user jiny2 closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1113


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #1125: HAWQ-1315. Function validateResourcePoolStatus()...

2017-02-15 Thread jiny2

Github user jiny2 commented on the issue:

https://github.com/apache/incubator-hawq/pull/1125
  
+1 LGMT 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1126: HAWQ-1334. QD thread should set error cod...

2017-02-15 Thread paul-guo-

GitHub user paul-guo- opened a pull request:

https://github.com/apache/incubator-hawq/pull/1126

HAWQ-1334. QD thread should set error code if failing so that the maiâ¦

â¦n process for the query could exit soon

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/paul-guo-/incubator-hawq QD

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/1126.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1126


commit d9f89e093a2689f5ddaf18d5948da2305bba5980
Author: Paul Guo 
Date:   2017-02-15T08:50:07Z

HAWQ-1334. QD thread should set error code if failing so that the main 
process for the query could exit soon




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (HAWQ-1334) QD thread should set error code if failing so that the main process for the query could exit soon

2017-02-15 Thread Paul Guo (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Guo reassigned HAWQ-1334:
--

Assignee: Paul Guo  (was: Ed Espino)

> QD thread should set error code if failing so that the main process for the 
> query could exit soon
> -
>
> Key: HAWQ-1334
> URL: https://issues.apache.org/jira/browse/HAWQ-1334
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Dispatcher
>Reporter: Paul Guo
>Assignee: Paul Guo
>
> In QD thread function dispmgt_thread_func_run(), if there are failures either 
> due to QE or QD itself, it will cancel the query and then clean up. The main 
> process for the query needs the error code of meleeResults be set so that it 
> soon proceeds to cancel the query, else we have to wait for timeout. 
> Typically dispmgt_thread_func_run() should set the error code, however I 
> found there are some cases who do not handle this, e.g. if poll() fails with 
> ENOMEM. One symptom of this issue is that we could sometimes see hang if a 
> query is canceled for some reasons.
> The potential solution is that:
> 1) We expect each branch jump ("goto error_cleanup") set proper error code 
> itself. It is not an easy job.
> 2) We add a "guard" function in the error_cleanup code to set an error code 
> if it is not set, i.e. 1) is not well done.
> I'd this JIRA cares about 2).
> In general, the cleanup code in QD seems to be really obscure and not 
> elegant. Maybe we should file another JIRA to refactor the error handling 
> logic in it. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HAWQ-1334) QD thread should set error code if failing so that the main process for the query could exit soon

2017-02-15 Thread Paul Guo (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Guo updated HAWQ-1334:
---
Description: 
In QD thread function dispmgt_thread_func_run(), if there are failures either 
due to QE or QD itself, it will cancel the query and then clean up. The main 
process for the query needs the error code of meleeResults be set so that it 
soon proceeds to cancel the query, else we have to wait for timeout. Typically 
dispmgt_thread_func_run() should set the error code, however I found there are 
some cases who do not handle this, e.g. if poll() fails with ENOMEM. One 
symptom of this issue is that we could sometimes see hang if a query is 
canceled for some reasons.

The potential solution is that:

1) We expect each branch jump ("goto error_cleanup") set proper error code 
itself. It is not an easy job.
2) We add a "guard" function in the error_cleanup code to set an error code if 
it is not set, i.e. 1) is not well done.

I'd this JIRA cares about 2).

In general, the cleanup code in QD seems to be really obscure and not elegant. 
Maybe we should file another JIRA to refactor the error handling logic in it. 

  was:
In QD thread dispmgt_thread_func_run(), if there are failures either due to QE 
or QD itself, it will cancel the query and then clean up. The main process for 
the query need to have the error code of meleeResults be set so that it soon 
proceed to cancel the query, else we have to wait for timeout. Typically 
dispmgt_thread_func_run() should set the error code, however I found there are 
some cases who do not handle this, e.g. if poll() fails with ENOMEM. One 
symptom of this issue is that we could sometimes see hang if a query is 
canceled for some reasons.

The potential solution is that:

1) We expect each branch jump ("goto error_cleanup") should set proper error 
code it self.
2) We add a "guard" function in the error_cleanup code to set an error code if 
it is not set.

In general, the cleanup code in QD seems to be really obscure and not elegant. 
Maybe we should file another JIRA to refactor the error handling logic in it. 


> QD thread should set error code if failing so that the main process for the 
> query could exit soon
> -
>
> Key: HAWQ-1334
> URL: https://issues.apache.org/jira/browse/HAWQ-1334
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Dispatcher
>Reporter: Paul Guo
>Assignee: Ed Espino
>
> In QD thread function dispmgt_thread_func_run(), if there are failures either 
> due to QE or QD itself, it will cancel the query and then clean up. The main 
> process for the query needs the error code of meleeResults be set so that it 
> soon proceeds to cancel the query, else we have to wait for timeout. 
> Typically dispmgt_thread_func_run() should set the error code, however I 
> found there are some cases who do not handle this, e.g. if poll() fails with 
> ENOMEM. One symptom of this issue is that we could sometimes see hang if a 
> query is canceled for some reasons.
> The potential solution is that:
> 1) We expect each branch jump ("goto error_cleanup") set proper error code 
> itself. It is not an easy job.
> 2) We add a "guard" function in the error_cleanup code to set an error code 
> if it is not set, i.e. 1) is not well done.
> I'd this JIRA cares about 2).
> In general, the cleanup code in QD seems to be really obscure and not 
> elegant. Maybe we should file another JIRA to refactor the error handling 
> logic in it. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HAWQ-1334) QD thread should set error code if failing so that the main process for the query could exit soon

2017-02-15 Thread Paul Guo (JIRA)

Paul Guo created HAWQ-1334:
--

 Summary: QD thread should set error code if failing so that the 
main process for the query could exit soon
 Key: HAWQ-1334
 URL: https://issues.apache.org/jira/browse/HAWQ-1334
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Dispatcher
Reporter: Paul Guo
Assignee: Ed Espino


In QD thread dispmgt_thread_func_run(), if there are failures either due to QE 
or QD itself, it will cancel the query and then clean up. The main process for 
the query need to have the error code of meleeResults be set so that it soon 
proceed to cancel the query, else we have to wait for timeout. Typically 
dispmgt_thread_func_run() should set the error code, however I found there are 
some cases who do not handle this, e.g. if poll() fails with ENOMEM. One 
symptom of this issue is that we could sometimes see hang if a query is 
canceled for some reasons.

The potential solution is that:

1) We expect each branch jump ("goto error_cleanup") should set proper error 
code it self.
2) We add a "guard" function in the error_cleanup code to set an error code if 
it is not set.

In general, the cleanup code in QD seems to be really obscure and not elegant. 
Maybe we should file another JIRA to refactor the error handling logic in it. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HAWQ-1332) Can not grant database and schema privileges without table privileges in ranger or ranger plugin service

2017-02-15 Thread Chunling Wang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunling Wang updated HAWQ-1332:

Description: 
We try to grant database connect and schema usage privileges to a non-super 
user to connect database. We find that if we set policy with database and 
schema included, but with table excluded, we can not connect database. But if 
we include table, we can connect to database. We think there may be bug in 
Ranger Plugin Service or Ranger. Here are steps to reproduce it.

1. create a new user "usertest1" in database:
{code}
$ psql postgres
psql (8.2.15)
Type "help" for help.

postgres=# CREATE USER usertest1;
NOTICE:  resource queue required -- using default resource queue "pg_default"
CREATE ROLE
postgres=#
{code}

2. add user "usertest1" in pg_hba.conf
{code}
local all usertest1 trust
{code}

3. set policy with database and schema included, with table excluded
!screenshot-1.png|width=800,height=400!

4. connect database with user "usertest1" but failed with permission denied
{code}
$ psql postgres -U usertest1
psql: FATAL:  permission denied for database "postgres"
DETAIL:  User does not have CONNECT privilege.
{code}

5. set policy with database, schema and table included
!screenshot-2.png|width=800,height=400!

6. connect database with user "usertest1" and succeed
{code}
$ psql postgres -U usertest1
psql (8.2.15)
Type "help" for help.

postgres=#
{code}

But if we do not set table as "*", and specify table like "a", we can not 
access database either.
!screenshot-3.png|width=800,height=400!

  was:
We try to grant database connect and schema usage privileges to a non-super 
user to connect database. We find that if we set policy with database and 
schema included, but with table excluded, we can not connect database. But if 
we include table, we can connect to database. We think there may be bug in 
Ranger Plugin Service or Ranger. Here are steps to reproduce it.

1. create a new user "usertest1" in database:
{code}
$ psql postgres
psql (8.2.15)
Type "help" for help.

postgres=# CREATE USER usertest1;
NOTICE:  resource queue required -- using default resource queue "pg_default"
CREATE ROLE
postgres=#
{code}

2. add user "usertest1" in pg_hba.conf
{code}
local all usertest1 trust
{code}

3. set policy with database and schema included, with table excluded
!screenshot-1.png|width=800,height=400!

4. connect database with user "usertest1" but failed with permission denied
{code}
$ psql postgres -U usertest1
psql: FATAL:  permission denied for database "postgres"
DETAIL:  User does not have CONNECT privilege.
{code}

5. set policy with database, schema and table included
!screenshot-2.png|width=800,height=400!

6. connect database with user "usertest1" and succeed
{code}
$ psql postgres -U usertest1
psql (8.2.15)
Type "help" for help.

postgres=#
{code}


> Can not grant database and schema privileges without table privileges in 
> ranger or ranger plugin service
> 
>
> Key: HAWQ-1332
> URL: https://issues.apache.org/jira/browse/HAWQ-1332
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Security
>Reporter: Chunling Wang
>Assignee: Ed Espino
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png
>
>
> We try to grant database connect and schema usage privileges to a non-super 
> user to connect database. We find that if we set policy with database and 
> schema included, but with table excluded, we can not connect database. But if 
> we include table, we can connect to database. We think there may be bug in 
> Ranger Plugin Service or Ranger. Here are steps to reproduce it.
> 1. create a new user "usertest1" in database:
> {code}
> $ psql postgres
> psql (8.2.15)
> Type "help" for help.
> postgres=# CREATE USER usertest1;
> NOTICE:  resource queue required -- using default resource queue "pg_default"
> CREATE ROLE
> postgres=#
> {code}
> 2. add user "usertest1" in pg_hba.conf
> {code}
> local all usertest1 trust
> {code}
> 3. set policy with database and schema included, with table excluded
> !screenshot-1.png|width=800,height=400!
> 4. connect database with user "usertest1" but failed with permission denied
> {code}
> $ psql postgres -U usertest1
> psql: FATAL:  permission denied for database "postgres"
> DETAIL:  User does not have CONNECT privilege.
> {code}
> 5. set policy with database, schema and table included
> !screenshot-2.png|width=800,height=400!
> 6. connect database with user "usertest1" and succeed
> {code}
> $ psql postgres -U usertest1
> psql (8.2.15)
> Type "help" for help.
> postgres=#
> {code}
> But if we do not set table as "*", and specify table like "a", we can not 
> access database either.
>

[jira] [Updated] (HAWQ-1332) Can not grant database and schema privileges without table privileges in ranger or ranger plugin service

2017-02-15 Thread Chunling Wang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunling Wang updated HAWQ-1332:

Attachment: screenshot-3.png

> Can not grant database and schema privileges without table privileges in 
> ranger or ranger plugin service
> 
>
> Key: HAWQ-1332
> URL: https://issues.apache.org/jira/browse/HAWQ-1332
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Security
>Reporter: Chunling Wang
>Assignee: Ed Espino
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png
>
>
> We try to grant database connect and schema usage privileges to a non-super 
> user to connect database. We find that if we set policy with database and 
> schema included, but with table excluded, we can not connect database. But if 
> we include table, we can connect to database. We think there may be bug in 
> Ranger Plugin Service or Ranger. Here are steps to reproduce it.
> 1. create a new user "usertest1" in database:
> {code}
> $ psql postgres
> psql (8.2.15)
> Type "help" for help.
> postgres=# CREATE USER usertest1;
> NOTICE:  resource queue required -- using default resource queue "pg_default"
> CREATE ROLE
> postgres=#
> {code}
> 2. add user "usertest1" in pg_hba.conf
> {code}
> local all usertest1 trust
> {code}
> 3. set policy with database and schema included, with table excluded
> !screenshot-1.png|width=800,height=400!
> 4. connect database with user "usertest1" but failed with permission denied
> {code}
> $ psql postgres -U usertest1
> psql: FATAL:  permission denied for database "postgres"
> DETAIL:  User does not have CONNECT privilege.
> {code}
> 5. set policy with database, schema and table included
> !screenshot-2.png|width=800,height=400!
> 6. connect database with user "usertest1" and succeed
> {code}
> $ psql postgres -U usertest1
> psql (8.2.15)
> Type "help" for help.
> postgres=#
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-hawq issue #1125: HAWQ-1315. Function validateResourcePoolStatus()...

2017-02-15 Thread paul-guo-

Github user paul-guo- commented on the issue:

https://github.com/apache/incubator-hawq/pull/1125
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

59 matches

Mail list logo