[jira] [Commented] (HAWQ-1530) Illegally killing a JDBC select query causes locking problems

[Lei Chang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16241629#comment-16241629
 ] 

Lei Chang commented on HAWQ-1530:
-

Thanks [~kuien] .

I think [~yjin] fixed a bug before, it is quite similar to this bug. 

> Illegally killing a JDBC select query causes locking problems
> -
>
> Key: HAWQ-1530
> URL: https://issues.apache.org/jira/browse/HAWQ-1530
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Transaction
>Reporter: Grant Krieger
>Assignee: Radar Lei
>
> Hi,
> When you perform a long running select statement on 2 hawq tables (join) from 
> JDBC and illegally kill the JDBC client (CTRL ALT DEL) before completion of 
> the query the 2 tables remained locked even when the query completes on the 
> server. 
> The lock is visible via PG_locks. One cannot kill the query via SELECT 
> pg_terminate_backend(393937). The only way to get rid of it is to kill -9 
> from linux or restart hawq but this can kill other things as well.
> The JDBC client I am using is Aqua Data Studio.
> I can provide exact steps to reproduce if required
> Thank you
> Grant 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HAWQ-1530) Illegally killing a JDBC select query causes locking problems

[Kuien Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16241605#comment-16241605
 ] 

Kuien Liu commented on HAWQ-1530:
-

My partner, Zhiyong Dai, has prepared a case to reproduce this issue:

1. Submit "SELECT * FROM tbl" through JDBC client;
2. Send SIGTSTP (by ctrl+z) to suspend JDBC client, when the query in #1 is 
running;
3. psql -c 'DROP TABLE tbl', then the request will be hang;
4. psql -c 'select * from tb', the request will be hang as well;
5.When backend finishes query in #1, it replies message to client and HANG at 
send() method, we may see following stack info:

#0  0x7fc8f4972b41 in send () from /lib64/libpthread.so.0
#1  0x0078ff9a in secure_write ()
#2  0x0079c225 in internal_flush ()
#3  0x0079c0e2 in internal_putbytes ()
#4  0x0079c457 in pq_putmessage ()
#5  0x0079cff3 in pq_endmessage ()
#6  0x004c831c in printtup ()
#7  0x007228b1 in ExecSelect ()
#8  0x007226ec in ExecutePlan ()
#9  0x0071e461 in ExecutorRun ()
#10 0x008fa31a in PortalRunSelect ()
#11 0x008f9efa in PortalRun ()
#12 0x008f1458 in exec_execute_message ()
#13 0x008f4f91 in PostgresMain ()
#14 0x0089d8a6 in BackendRun ()
#15 0x0089cd18 in BackendStartup ()
#16 0x00896e89 in ServerLoop ()
#17 0x00895ea8 in PostmasterMain ()
#18 0x007b1baa in main ()

Wish this helps.

> Illegally killing a JDBC select query causes locking problems
> -
>
> Key: HAWQ-1530
> URL: https://issues.apache.org/jira/browse/HAWQ-1530
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Transaction
>Reporter: Grant Krieger
>Assignee: Radar Lei
>
> Hi,
> When you perform a long running select statement on 2 hawq tables (join) from 
> JDBC and illegally kill the JDBC client (CTRL ALT DEL) before completion of 
> the query the 2 tables remained locked even when the query completes on the 
> server. 
> The lock is visible via PG_locks. One cannot kill the query via SELECT 
> pg_terminate_backend(393937). The only way to get rid of it is to kill -9 
> from linux or restart hawq but this can kill other things as well.
> The JDBC client I am using is Aqua Data Studio.
> I can provide exact steps to reproduce if required
> Thank you
> Grant 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] incubator-hawq issue #1296: HAWQ-1496. Add DISCLAIMER, LICENSE and NOTICE fi...

[jiny2]

Github user jiny2 commented on the issue:

https://github.com/apache/incubator-hawq/pull/1296
  
Thank you ed! @edespino 

+1 Look good to me . 


---

[GitHub] incubator-hawq pull request #1307: HAWQ-1544. prompt file count doesn't matc...

[interma]

Github user interma commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1307#discussion_r149278987
  
--- Diff: src/backend/cdb/cdbcat.c ---
@@ -534,3 +536,15 @@ checkPolicyForUniqueIndex(Relation rel, AttrNumber 
*indattr, int nidxatts,
}
}
 }
+
+void 
+GpPolicyReplace(Oid tbloid, const GpPolicy *policy)
+{
+   GpPolicyReplace_inner(tbloid, policy, false);
+}
+void 
+GpPolicyReplaceWithBucketNum(Oid tbloid, const GpPolicy *policy)
--- End diff --

The older `GpPolicyReplace` function doesn't update `bucketnum` field 
though I have already set this field in `policy` structure, because `repl[1] = 
false;`
So I add a new GpPolicyReplaceWithBucketNum function to do it.


---

[GitHub] incubator-hawq issue #1307: HAWQ-1544. prompt file count doesn't match hash ...

[interma]

Github user interma commented on the issue:

https://github.com/apache/incubator-hawq/pull/1307
  
have already added feature-test.

@linwen @amyrazz44 @wengyanqing @radarwave help to review, thanks.


---

[GitHub] incubator-hawq issue #1296: HAWQ-1496. Add DISCLAIMER, LICENSE and NOTICE fi...

[radarwave]

Github user radarwave commented on the issue:

https://github.com/apache/incubator-hawq/pull/1296
  
Thanks @edespino 

So I'm good with it. +1


---

[jira] [Commented] (HAWQ-1543) PXF port,heap size etc should be configurable through pxf-env

[Shivram Mani (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240723#comment-16240723
 ] 

Shivram Mani commented on HAWQ-1543:


Introduced PXF_JVM_OPTS which allows users to configure the heap size. The 
default value of max memory allocation is 2G and initial value of mem 
allocation is 1G.

> PXF port,heap size etc should be configurable through pxf-env
> -
>
> Key: HAWQ-1543
> URL: https://issues.apache.org/jira/browse/HAWQ-1543
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: PXF
>Reporter: Shivram Mani
>Assignee: Shivram Mani
>Priority: Trivial
> Fix For: 2.3.0.0-incubating
>
>
> Currently the configs in pxf-env take effect in the pxf service only upon pxf 
> init. This should be consumed upon pxf start as well.
> Also introduce configuration for pxf heapsize



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] incubator-hawq issue #1307: HAWQ-1544. prompt file count doesn't match hash ...

[interma]

Github user interma commented on the issue:

https://github.com/apache/incubator-hawq/pull/1307
  
How to reproduct this issue, see jira: 
https://issues.apache.org/jira/browse/HAWQ-1544

Root cause:
When create a table, it inserts a row into `gp_distribution_policy ( 
localoid | bucketnum | attrnums )`, and it will update this row on the follow 
reorganize table. 
But it only updates attrnums field, the correct logic also needs updating 
bucketnum field to the current default_hash_table_bucket_number.



---

[GitHub] incubator-hawq pull request #1307: HAWQ-1544. prompt file count doesn't matc...

[interma]

GitHub user interma opened a pull request:

https://github.com/apache/incubator-hawq/pull/1307

HAWQ-1544. prompt file count doesn't match hash bucket number when re…

…organize table

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/interma/interma-hawq hawq-1544

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/1307.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1307


commit c314748bb6bbcfc307bc439fd6a6880cbf70b04a
Author: interma 
Date:   2017-11-06T09:01:59Z

HAWQ-1544. prompt file count doesn't match hash bucket number when 
reorganize table




---

[jira] [Assigned] (HAWQ-1544) prompt file count doesn't match hash bucket number when reorganize table

[Hongxu Ma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongxu Ma reassigned HAWQ-1544:
---

Assignee: Hongxu Ma  (was: Radar Lei)

> prompt file count doesn't match hash bucket number when reorganize table
> 
>
> Key: HAWQ-1544
> URL: https://issues.apache.org/jira/browse/HAWQ-1544
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Catalog
>Reporter: Hongxu Ma
>Assignee: Hongxu Ma
>
> Reproduce:
> {code}
> postgres=# create table t_vm ( i int ) distributed randomly;
> CREATE TABLE
> postgres=# insert into t_vm select generate_series(1,1);
> INSERT 0 1
> postgres=# show default_hash_table_bucket_number ;
>  default_hash_table_bucket_number
> --
>  6
> (1 row)
> postgres=# set default_hash_table_bucket_number=8;
> SET
> postgres=# show default_hash_table_bucket_number ;
>  default_hash_table_bucket_number
> --
>  8
> (1 row)
> postgres=# alter table t_vm set with(reorganize=true) distributed by (i) ;
> ALTER TABLE
> postgres=# select count(*) from t_vm;
> ERROR:  file count 8 in catalog is not in proportion to the bucket number 6 
> of hash table with oid=16619, some data may be lost, if you still want to 
> continue the query by considering the table as random, set GUC 
> allow_file_count_bucket_num_mismatch to on and try again. 
> (cdbdatalocality.c:3776)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HAWQ-1544) prompt file count doesn't match hash bucket number when reorganize table

[Hongxu Ma (JIRA)]

Hongxu Ma created HAWQ-1544:
---

 Summary: prompt file count doesn't match hash bucket number when 
reorganize table
 Key: HAWQ-1544
 URL: https://issues.apache.org/jira/browse/HAWQ-1544
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Catalog
Reporter: Hongxu Ma
Assignee: Radar Lei


Reproduce:
{code}
postgres=# create table t_vm ( i int ) distributed randomly;
CREATE TABLE
postgres=# insert into t_vm select generate_series(1,1);
INSERT 0 1
postgres=# show default_hash_table_bucket_number ;
 default_hash_table_bucket_number
--
 6
(1 row)

postgres=# set default_hash_table_bucket_number=8;
SET
postgres=# show default_hash_table_bucket_number ;
 default_hash_table_bucket_number
--
 8
(1 row)

postgres=# alter table t_vm set with(reorganize=true) distributed by (i) ;
ALTER TABLE
postgres=# select count(*) from t_vm;
ERROR:  file count 8 in catalog is not in proportion to the bucket number 6 of 
hash table with oid=16619, some data may be lost, if you still want to continue 
the query by considering the table as random, set GUC 
allow_file_count_bucket_num_mismatch to on and try again. 
(cdbdatalocality.c:3776)
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)