[jira] [Commented] (HAWQ-94) Storage error when insert a large tuple
[
https://issues.apache.org/jira/browse/HAWQ-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1482#comment-1482
]
ASF GitHub Bot commented on HAWQ-94:
Github user yaoj2 commented on the pull request:
https://github.com/apache/incubator-hawq/pull/95#issuecomment-155678209
+1 For handle blockLimitLen=0
> Storage error when insert a large tuple
> ---
>
> Key: HAWQ-94
> URL: https://issues.apache.org/jira/browse/HAWQ-94
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Storage
>Reporter: Dong Li
>Assignee: Lirong Jian
>
> 1. Set guc value "appendonly_split_write_size_mb"
> hawq config -c appendonly_split_write_size_mb -v 2
> 2.Run sql
> set default_segment_num=1;
> create table eightbytleft_for_readsplit(str varchar) with
> (appendonly=true,blocksize=2097152,checksum=true);
> insert into eightbytleft_for_readsplit select repeat('a',2097136*63-8);
> WARNING: skipping "eightbytleft_for_readsplit" --- error returned: Bad
> append-only storage header of type small content. Header check error 7,
> detail 'Append-only storage header is invalid -- overall block length 2097152
> is > block limit length 0 (smallcontent_bytes_0_3 0x120003ff,
> smallcontent_bytes_4_7 0xfe00)' (cdbappendonlystorageread.c:972) (seg0
> sdw1.hawq.greenplum.com:31100 pid=641344)
> DETAIL:
> Append-Only storage Small Content header: smallcontent_bytes_0_3 0x120003FF,
> smallcontent_bytes_4_7 0xFE00, headerKind = 1, executorBlockKind = 2,
> rowCount = 0, usingChecksums = true, header checksum 0xAD931AF8, block
> checksum 0x3B923D8, dataLength 2097136, compressedLength 0, overallBlockLen
> 2097152
> Scan of Append-Only Row-Oriented relation 'eightbytleft_for_readsplit'.
> Append-Only segment file
> 'hdfs://smdw:9000/hawq/hawq-1446007451/16385/32094/32095/1', block header
> offset in file = 136314880, bufferCount 66
> INFO: ANALYZE completed. Success: 0, Failure: 1 (eightbytleft_for_readsplit)
> INSERT 0 1
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HAWQ-94) Storage error when insert a large tuple
[
https://issues.apache.org/jira/browse/HAWQ-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1475#comment-1475
]
ASF GitHub Bot commented on HAWQ-94:
GitHub user jianlirong opened a pull request:
https://github.com/apache/incubator-hawq/pull/95
HAWQ-94. Handle one case when reaching EOF for AO large tuple
When *blockLimitLen = 0, we should return false to indicate this is the end
of file.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jianlirong/incubator-hawq HAWQ-94
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-hawq/pull/95.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #95
commit 1f93480e3515dbe93eadc0a87011c223baf02dfb
Author: Lirong Jian
Date: 2015-11-11T05:53:25Z
HAWQ-94. Handle one case when reaching EOF for AO large tuple
> Storage error when insert a large tuple
> ---
>
> Key: HAWQ-94
> URL: https://issues.apache.org/jira/browse/HAWQ-94
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Storage
>Reporter: Dong Li
>Assignee: Lirong Jian
>
> 1. Set guc value "appendonly_split_write_size_mb"
> hawq config -c appendonly_split_write_size_mb -v 2
> 2.Run sql
> set default_segment_num=1;
> create table eightbytleft_for_readsplit(str varchar) with
> (appendonly=true,blocksize=2097152,checksum=true);
> insert into eightbytleft_for_readsplit select repeat('a',2097136*63-8);
> WARNING: skipping "eightbytleft_for_readsplit" --- error returned: Bad
> append-only storage header of type small content. Header check error 7,
> detail 'Append-only storage header is invalid -- overall block length 2097152
> is > block limit length 0 (smallcontent_bytes_0_3 0x120003ff,
> smallcontent_bytes_4_7 0xfe00)' (cdbappendonlystorageread.c:972) (seg0
> sdw1.hawq.greenplum.com:31100 pid=641344)
> DETAIL:
> Append-Only storage Small Content header: smallcontent_bytes_0_3 0x120003FF,
> smallcontent_bytes_4_7 0xFE00, headerKind = 1, executorBlockKind = 2,
> rowCount = 0, usingChecksums = true, header checksum 0xAD931AF8, block
> checksum 0x3B923D8, dataLength 2097136, compressedLength 0, overallBlockLen
> 2097152
> Scan of Append-Only Row-Oriented relation 'eightbytleft_for_readsplit'.
> Append-Only segment file
> 'hdfs://smdw:9000/hawq/hawq-1446007451/16385/32094/32095/1', block header
> offset in file = 136314880, bufferCount 66
> INFO: ANALYZE completed. Success: 0, Failure: 1 (eightbytleft_for_readsplit)
> INSERT 0 1
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HAWQ-117) Alter a newly created tablespace table in kerberos mode would cause "ERROR: Append-Only Storage Read could not open segment file"
[ https://issues.apache.org/jira/browse/HAWQ-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1473#comment-1473 ] ASF GitHub Bot commented on HAWQ-117: - Github user jianlirong closed the pull request at: https://github.com/apache/incubator-hawq/pull/75 > Alter a newly created tablespace table in kerberos mode would cause "ERROR: > Append-Only Storage Read could not open segment file" > -- > > Key: HAWQ-117 > URL: https://issues.apache.org/jira/browse/HAWQ-117 > Project: Apache HAWQ > Issue Type: Bug > Components: Storage >Reporter: Lirong Jian >Assignee: Lirong Jian > > create tablespace ts_a4 filespace cdbfast_fs_a; > create tablespace ts_b4 filespace cdbfast_fs_b; > create table fsts_split_list1 (i int) tablespace ts_a4 partition by list(i) ( > partition a values(1, 2, 3, 4) WITH > (appendonly=true,checksum=true,blocksize=819200,compresslevel=8), > partition b values(5, 6, 7, 8), > default partition default_part); > CREATE TABLE > -- Insert few records into the table > insert into fsts_split_list1 values (generate_series(1,10)); > select count(*) from fsts_split_list1; > -- split partition > alter table fsts_split_list1 split partition for(1) at (1,2) into (partition > f1a, partition f1b); > psql:/data/gpadmin/pulse2-agent/agents/agent1/work/HAWQ-main-FeatureTest-opt-security-FIPS/FIPS-GPSQL-schedule_kerberos/cdbunit/cdbfast/storage/tablespace_filespace/gp_filespace_tablespace/transaction_rollback.sql:17: > NOTICE: exchanged partition "a" of relation "fsts_split_list1" with > relation "pg_temp_29565" > psql:/data/gpadmin/pulse2-agent/agents/agent1/work/HAWQ-main-FeatureTest-opt-security-FIPS/FIPS-GPSQL-schedule_kerberos/cdbunit/cdbfast/storage/tablespace_filespace/gp_filespace_tablespace/transaction_rollback.sql:17: > NOTICE: dropped partition "a" for relation "fsts_split_list1" > psql:/data/gpadmin/pulse2-agent/agents/agent1/work/HAWQ-main-FeatureTest-opt-security-FIPS/FIPS-GPSQL-schedule_kerberos/cdbunit/cdbfast/storage/tablespace_filespace/gp_filespace_tablespace/transaction_rollback.sql:17: > NOTICE: CREATE TABLE will create partition "fsts_split_list1_1_prt_f1a" for > table "fsts_split_list1" > psql:/data/gpadmin/pulse2-agent/agents/agent1/work/HAWQ-main-FeatureTest-opt-security-FIPS/FIPS-GPSQL-schedule_kerberos/cdbunit/cdbfast/storage/tablespace_filespace/gp_filespace_tablespace/transaction_rollback.sql:17: > NOTICE: CREATE TABLE will create partition "fsts_split_list1_1_prt_f1b" for > table "fsts_split_list1" > psql:/data/gpadmin/pulse2-agent/agents/agent1/work/HAWQ-main-FeatureTest-opt-security-FIPS/FIPS-GPSQL-schedule_kerberos/cdbunit/cdbfast/storage/tablespace_filespace/gp_filespace_tablespace/transaction_rollback.sql:17: > WARNING: failed to get filesystem credential. (seg0 rhel64-2:21100 > pid=482506) > DETAIL: Success -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-57) Apache License v2.0 File, Notice File Added to Top Level Directory and License Header added to each source file
[ https://issues.apache.org/jira/browse/HAWQ-57?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Wen reassigned HAWQ-57: --- Assignee: Lin Wen > Apache License v2.0 File, Notice File Added to Top Level Directory and > License Header added to each source file > --- > > Key: HAWQ-57 > URL: https://issues.apache.org/jira/browse/HAWQ-57 > Project: Apache HAWQ > Issue Type: Task > Components: Core >Reporter: Goden Yao >Assignee: Lin Wen >Priority: Blocker > Labels: OSS > Fix For: 2.0.0-beta-incubating > > Attachments: NOTICE > > > *Reference* > http://www.apache.org/dev/apply-license > APPLYING THE LICENSE TO NEW SOFTWARE > To apply the ALv2 to a new software distribution, include one copy of the > license text by copying the file: > http://www.apache.org/licenses/LICENSE-2.0.txt > into a file called LICENSE in the top directory of your distribution. If the > distribution is a jar or tar file, try to add the LICENSE file first in order > to place it at the top of the archive. This covers the collective licensing > for the distribution. > In addition, a correct [NOTICE > file|http://apache.org/legal/src-headers.html#notice] MUST be included in the > same directory as the LICENSE file. > Each original source document (code and documentation, but excluding the > LICENSE and NOTICE files) SHOULD include a [short license > header|http://www.apache.org/legal/src-headers.html] at the top. If the > distribution contains documents not covered by CLA, CCLA or Software Grant > (such as third-party libraries) then see the [policy > guide|http://www.apache.org/legal/resolved.html]. > 3rd party source file should not add the license header. > We can use [Apache > Rat|http://creadur.apache.org/rat/index.html#How_Do_I_Use_Rat] for > verification. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-141) memory accessing panic in system test.
[ https://issues.apache.org/jira/browse/HAWQ-141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1415#comment-1415 ] ASF GitHub Bot commented on HAWQ-141: - Github user jiny2 closed the pull request at: https://github.com/apache/incubator-hawq/pull/90 > memory accessing panic in system test. > -- > > Key: HAWQ-141 > URL: https://issues.apache.org/jira/browse/HAWQ-141 > Project: Apache HAWQ > Issue Type: Bug > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > > #0 0x0036a820f5db in raise () from /lib64/libpthread.so.0 > #1 0x00874962 in SafeHandlerForSegvBusIll (processName= optimized out>, postgres_signal_arg=11) at elog.c:4497 > #2 > #3 0x0036a7e84cd0 in bcopy () from /lib64/libc.so.6 > #4 0x008d6f30 in connectToServerRemote (address= out>, port=, clientfd=0x7d89966c) at > network_utils.c:544 > #5 0x008d2218 in callSyncRPCRemote (hostname=0x0, port= optimized out>, sendbuff=, sendbuffsize=-1477534697, > sendmsgid=1, exprecvmsgid=0, recvsmb=0x1f34838) at rmcomm_SyncComm.c:144 > #6 0x008ce329 in callSyncRPCToRM (index=0, errorbuf=0x7d899700 > "\361gv\002", errorbufsize=1024) at rmcomm_QD2RM.c:2459 > #7 unregisterConnectionInRM (index=0, errorbuf=0x7d899700 "\361gv\002", > errorbufsize=1024) at rmcomm_QD2RM.c:517 > #8 0x007baffd in FreeResource (resource=0x1f0d0d0) at pquery.c:986 > #9 0x0065e14c in ExecutorEnd (queryDesc=) at > execMain.c:1280 > #10 0x0060e4b2 in PortalCleanupHelper (portal=) > at portalcmds.c:347 > #11 PortalCleanup (portal=) at portalcmds.c:299 > #12 0x008a2407 in PortalDrop (portal=0x205c9c0, isTopCommit=0 '\000') > at portalmem.c:392 > #13 0x007b613f in exec_simple_query (query_string= out>, seqServerHost=, seqServerPort= out>) at postgres.c:1738 > #14 0x007b7612 in PostgresMain (argc=, > argv=0x1db85e0, username=) at postgres.c:4691 > #15 0x00768c53 in BackendRun (port=0x1d6dc80) at postmaster.c:5845 > #16 BackendStartup (port=0x1d6dc80) at postmaster.c:5438 > #17 0x007693bd in ServerLoop () at postmaster.c:2140 > #18 0x0076b1ce in PostmasterMain (argc=9, argv=0x1d728e0) at > postmaster.c:1432 > #19 0x006c5eea in main (argc=9, argv=0x1d728a0) at main.c:226 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-145) Memory leak when free the argument instance of resource lease heart-beat thread in dispatcher process
[ https://issues.apache.org/jira/browse/HAWQ-145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1414#comment-1414 ] ASF GitHub Bot commented on HAWQ-145: - Github user jiny2 closed the pull request at: https://github.com/apache/incubator-hawq/pull/94 > Memory leak when free the argument instance of resource lease heart-beat > thread in dispatcher process > - > > Key: HAWQ-145 > URL: https://issues.apache.org/jira/browse/HAWQ-145 > Project: Apache HAWQ > Issue Type: Bug > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-145) Memory leak when free the argument instance of resource lease heart-beat thread in dispatcher process
[
https://issues.apache.org/jira/browse/HAWQ-145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1408#comment-1408
]
ASF GitHub Bot commented on HAWQ-145:
-
Github user jiny2 commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/94#discussion_r44499509
--- Diff: src/backend/resourcemanager/communication/rmcomm_QD2RM.c ---
@@ -1438,6 +1438,7 @@ void freeHeartBeatThreadArg(HeartBeatThreadArg *arg)
{
free((*arg)->HostAddrs[i]);
}
+ free((*arg)->HostAddrs);
--- End diff --
In the code, I have logic to prevent this case :)
> Memory leak when free the argument instance of resource lease heart-beat
> thread in dispatcher process
> -
>
> Key: HAWQ-145
> URL: https://issues.apache.org/jira/browse/HAWQ-145
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Resource Manager
>Reporter: Yi Jin
>Assignee: Yi Jin
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HAWQ-94) Storage error when insert a large tuple
[
https://issues.apache.org/jira/browse/HAWQ-94?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lirong Jian reassigned HAWQ-94:
---
Assignee: Lirong Jian (was: Lei Chang)
> Storage error when insert a large tuple
> ---
>
> Key: HAWQ-94
> URL: https://issues.apache.org/jira/browse/HAWQ-94
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Storage
>Reporter: Dong Li
>Assignee: Lirong Jian
>
> 1. Set guc value "appendonly_split_write_size_mb"
> hawq config -c appendonly_split_write_size_mb -v 2
> 2.Run sql
> set default_segment_num=1;
> create table eightbytleft_for_readsplit(str varchar) with
> (appendonly=true,blocksize=2097152,checksum=true);
> insert into eightbytleft_for_readsplit select repeat('a',2097136*63-8);
> WARNING: skipping "eightbytleft_for_readsplit" --- error returned: Bad
> append-only storage header of type small content. Header check error 7,
> detail 'Append-only storage header is invalid -- overall block length 2097152
> is > block limit length 0 (smallcontent_bytes_0_3 0x120003ff,
> smallcontent_bytes_4_7 0xfe00)' (cdbappendonlystorageread.c:972) (seg0
> sdw1.hawq.greenplum.com:31100 pid=641344)
> DETAIL:
> Append-Only storage Small Content header: smallcontent_bytes_0_3 0x120003FF,
> smallcontent_bytes_4_7 0xFE00, headerKind = 1, executorBlockKind = 2,
> rowCount = 0, usingChecksums = true, header checksum 0xAD931AF8, block
> checksum 0x3B923D8, dataLength 2097136, compressedLength 0, overallBlockLen
> 2097152
> Scan of Append-Only Row-Oriented relation 'eightbytleft_for_readsplit'.
> Append-Only segment file
> 'hdfs://smdw:9000/hawq/hawq-1446007451/16385/32094/32095/1', block header
> offset in file = 136314880, bufferCount 66
> INFO: ANALYZE completed. Success: 0, Failure: 1 (eightbytleft_for_readsplit)
> INSERT 0 1
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HAWQ-143) Add informations in Apache-HAWQ README.md
[ https://issues.apache.org/jira/browse/HAWQ-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiang Sheng resolved HAWQ-143. -- Resolution: Fixed > Add informations in Apache-HAWQ README.md > - > > Key: HAWQ-143 > URL: https://issues.apache.org/jira/browse/HAWQ-143 > Project: Apache HAWQ > Issue Type: Improvement > Components: Documentation >Reporter: Xiang Sheng >Assignee: Lei Chang > > Currently there is only one line "Apache HAWQ" in the README.md file . We > should add more introduction information about Apache HAWQ. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-117) Alter a newly created tablespace table in kerberos mode would cause "ERROR: Append-Only Storage Read could not open segment file"
[ https://issues.apache.org/jira/browse/HAWQ-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999847#comment-14999847 ] ASF GitHub Bot commented on HAWQ-117: - Github user liming01 commented on the pull request: https://github.com/apache/incubator-hawq/pull/75#issuecomment-155642515 Looks good. +1 > Alter a newly created tablespace table in kerberos mode would cause "ERROR: > Append-Only Storage Read could not open segment file" > -- > > Key: HAWQ-117 > URL: https://issues.apache.org/jira/browse/HAWQ-117 > Project: Apache HAWQ > Issue Type: Bug > Components: Storage >Reporter: Lirong Jian >Assignee: Lirong Jian > > create tablespace ts_a4 filespace cdbfast_fs_a; > create tablespace ts_b4 filespace cdbfast_fs_b; > create table fsts_split_list1 (i int) tablespace ts_a4 partition by list(i) ( > partition a values(1, 2, 3, 4) WITH > (appendonly=true,checksum=true,blocksize=819200,compresslevel=8), > partition b values(5, 6, 7, 8), > default partition default_part); > CREATE TABLE > -- Insert few records into the table > insert into fsts_split_list1 values (generate_series(1,10)); > select count(*) from fsts_split_list1; > -- split partition > alter table fsts_split_list1 split partition for(1) at (1,2) into (partition > f1a, partition f1b); > psql:/data/gpadmin/pulse2-agent/agents/agent1/work/HAWQ-main-FeatureTest-opt-security-FIPS/FIPS-GPSQL-schedule_kerberos/cdbunit/cdbfast/storage/tablespace_filespace/gp_filespace_tablespace/transaction_rollback.sql:17: > NOTICE: exchanged partition "a" of relation "fsts_split_list1" with > relation "pg_temp_29565" > psql:/data/gpadmin/pulse2-agent/agents/agent1/work/HAWQ-main-FeatureTest-opt-security-FIPS/FIPS-GPSQL-schedule_kerberos/cdbunit/cdbfast/storage/tablespace_filespace/gp_filespace_tablespace/transaction_rollback.sql:17: > NOTICE: dropped partition "a" for relation "fsts_split_list1" > psql:/data/gpadmin/pulse2-agent/agents/agent1/work/HAWQ-main-FeatureTest-opt-security-FIPS/FIPS-GPSQL-schedule_kerberos/cdbunit/cdbfast/storage/tablespace_filespace/gp_filespace_tablespace/transaction_rollback.sql:17: > NOTICE: CREATE TABLE will create partition "fsts_split_list1_1_prt_f1a" for > table "fsts_split_list1" > psql:/data/gpadmin/pulse2-agent/agents/agent1/work/HAWQ-main-FeatureTest-opt-security-FIPS/FIPS-GPSQL-schedule_kerberos/cdbunit/cdbfast/storage/tablespace_filespace/gp_filespace_tablespace/transaction_rollback.sql:17: > NOTICE: CREATE TABLE will create partition "fsts_split_list1_1_prt_f1b" for > table "fsts_split_list1" > psql:/data/gpadmin/pulse2-agent/agents/agent1/work/HAWQ-main-FeatureTest-opt-security-FIPS/FIPS-GPSQL-schedule_kerberos/cdbunit/cdbfast/storage/tablespace_filespace/gp_filespace_tablespace/transaction_rollback.sql:17: > WARNING: failed to get filesystem credential. (seg0 rhel64-2:21100 > pid=482506) > DETAIL: Success -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-143) Add informations in Apache-HAWQ README.md
[ https://issues.apache.org/jira/browse/HAWQ-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999838#comment-14999838 ] ASF GitHub Bot commented on HAWQ-143: - Github user asfgit closed the pull request at: https://github.com/apache/incubator-hawq/pull/91 > Add informations in Apache-HAWQ README.md > - > > Key: HAWQ-143 > URL: https://issues.apache.org/jira/browse/HAWQ-143 > Project: Apache HAWQ > Issue Type: Improvement > Components: Documentation >Reporter: Xiang Sheng >Assignee: Lei Chang > > Currently there is only one line "Apache HAWQ" in the README.md file . We > should add more introduction information about Apache HAWQ. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-145) Memory leak when free the argument instance of resource lease heart-beat thread in dispatcher process
[ https://issues.apache.org/jira/browse/HAWQ-145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999834#comment-14999834 ] ASF GitHub Bot commented on HAWQ-145: - Github user linwen commented on the pull request: https://github.com/apache/incubator-hawq/pull/94#issuecomment-155637129 +1 > Memory leak when free the argument instance of resource lease heart-beat > thread in dispatcher process > - > > Key: HAWQ-145 > URL: https://issues.apache.org/jira/browse/HAWQ-145 > Project: Apache HAWQ > Issue Type: Bug > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-146) Re-enable non-uniform partitioning DDLs
Entong Shen created HAWQ-146: Summary: Re-enable non-uniform partitioning DDLs Key: HAWQ-146 URL: https://issues.apache.org/jira/browse/HAWQ-146 Project: Apache HAWQ Issue Type: Task Components: DDL Reporter: Entong Shen Assignee: Lei Chang Previously we disabled DDLs that could create non-uniform partitioning hierarchy. We are enabling those now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-145) Memory leak when free the argument instance of resource lease heart-beat thread in dispatcher process
[
https://issues.apache.org/jira/browse/HAWQ-145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999791#comment-14999791
]
ASF GitHub Bot commented on HAWQ-145:
-
Github user zhangh43 commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/94#discussion_r44493523
--- Diff: src/backend/resourcemanager/communication/rmcomm_QD2RM.c ---
@@ -1438,6 +1438,7 @@ void freeHeartBeatThreadArg(HeartBeatThreadArg *arg)
{
free((*arg)->HostAddrs[i]);
}
+ free((*arg)->HostAddrs);
--- End diff --
sorry. It's OK to free(malloc(0)).
+1
> Memory leak when free the argument instance of resource lease heart-beat
> thread in dispatcher process
> -
>
> Key: HAWQ-145
> URL: https://issues.apache.org/jira/browse/HAWQ-145
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Resource Manager
>Reporter: Yi Jin
>Assignee: Yi Jin
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HAWQ-145) Memory leak when free the argument instance of resource lease heart-beat thread in dispatcher process
[
https://issues.apache.org/jira/browse/HAWQ-145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999782#comment-14999782
]
ASF GitHub Bot commented on HAWQ-145:
-
Github user zhangh43 commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/94#discussion_r44493044
--- Diff: src/backend/resourcemanager/communication/rmcomm_QD2RM.c ---
@@ -1438,6 +1438,7 @@ void freeHeartBeatThreadArg(HeartBeatThreadArg *arg)
{
free((*arg)->HostAddrs[i]);
}
+ free((*arg)->HostAddrs);
--- End diff --
have a look at the code: tharg->HostAddrs = malloc(sizeof(char *) *
tharg->HostAddrSize);
where tharg->HostAddrSize maybe zero and malloc(0) happens.
what i mean yesterday is to test whether tharg->HostAddrSize > 0 before
free arg->HostAddrs
> Memory leak when free the argument instance of resource lease heart-beat
> thread in dispatcher process
> -
>
> Key: HAWQ-145
> URL: https://issues.apache.org/jira/browse/HAWQ-145
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Resource Manager
>Reporter: Yi Jin
>Assignee: Yi Jin
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HAWQ-145) Memory leak when free the argument instance of resource lease heart-beat thread in dispatcher process
[ https://issues.apache.org/jira/browse/HAWQ-145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999519#comment-14999519 ] ASF GitHub Bot commented on HAWQ-145: - GitHub user jiny2 opened a pull request: https://github.com/apache/incubator-hawq/pull/94 HAWQ-145. Memory leak when free the argument instance of resource lea… This is to fix a memory leak. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jiny2/incubator-hawq HAWQ-145 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/94.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #94 commit f9ce488b2be8dc4852ded450b627c3bea72572b1 Author: Yi Jin Date: 2015-11-10T22:54:42Z HAWQ-145. Memory leak when free the argument instance of resource lease heart-beat thread in dispatcher process > Memory leak when free the argument instance of resource lease heart-beat > thread in dispatcher process > - > > Key: HAWQ-145 > URL: https://issues.apache.org/jira/browse/HAWQ-145 > Project: Apache HAWQ > Issue Type: Bug > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-145) Memory leak when free the argument instance of resource lease heart-beat thread in dispatcher process
Yi Jin created HAWQ-145: --- Summary: Memory leak when free the argument instance of resource lease heart-beat thread in dispatcher process Key: HAWQ-145 URL: https://issues.apache.org/jira/browse/HAWQ-145 Project: Apache HAWQ Issue Type: Bug Components: Resource Manager Reporter: Yi Jin Assignee: Lei Chang -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-145) Memory leak when free the argument instance of resource lease heart-beat thread in dispatcher process
[ https://issues.apache.org/jira/browse/HAWQ-145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Jin reassigned HAWQ-145: --- Assignee: Yi Jin (was: Lei Chang) > Memory leak when free the argument instance of resource lease heart-beat > thread in dispatcher process > - > > Key: HAWQ-145 > URL: https://issues.apache.org/jira/browse/HAWQ-145 > Project: Apache HAWQ > Issue Type: Bug > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (HAWQ-128) Cleanup pg_resqueue impact in catalog and dump corresponding components
[ https://issues.apache.org/jira/browse/HAWQ-128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Jin closed HAWQ-128. --- > Cleanup pg_resqueue impact in catalog and dump corresponding components > --- > > Key: HAWQ-128 > URL: https://issues.apache.org/jira/browse/HAWQ-128 > Project: Apache HAWQ > Issue Type: Bug > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > > Cleanup pg_resqueue impact in catalog and dump corresponding components > 1) Check all catalog changes related to pg_resqueue; > 2) Check all code accessing pg_resqueue; > 3) Check tools using pg_resqueue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (HAWQ-132) Enrich log information when queuing resource request is timed-out.
[ https://issues.apache.org/jira/browse/HAWQ-132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Jin closed HAWQ-132. --- > Enrich log information when queuing resource request is timed-out. > -- > > Key: HAWQ-132 > URL: https://issues.apache.org/jira/browse/HAWQ-132 > Project: Apache HAWQ > Issue Type: Improvement > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > > This is to enrich log information in case some request is timed out, it is > useful to find the timeout reason. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HAWQ-128) Cleanup pg_resqueue impact in catalog and dump corresponding components
[ https://issues.apache.org/jira/browse/HAWQ-128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Jin resolved HAWQ-128. - Resolution: Fixed > Cleanup pg_resqueue impact in catalog and dump corresponding components > --- > > Key: HAWQ-128 > URL: https://issues.apache.org/jira/browse/HAWQ-128 > Project: Apache HAWQ > Issue Type: Bug > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > > Cleanup pg_resqueue impact in catalog and dump corresponding components > 1) Check all catalog changes related to pg_resqueue; > 2) Check all code accessing pg_resqueue; > 3) Check tools using pg_resqueue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HAWQ-132) Enrich log information when queuing resource request is timed-out.
[ https://issues.apache.org/jira/browse/HAWQ-132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Jin resolved HAWQ-132. - Resolution: Fixed > Enrich log information when queuing resource request is timed-out. > -- > > Key: HAWQ-132 > URL: https://issues.apache.org/jira/browse/HAWQ-132 > Project: Apache HAWQ > Issue Type: Improvement > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > > This is to enrich log information in case some request is timed out, it is > useful to find the timeout reason. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (HAWQ-138) Alter resource quene error when memory set 64mb
[ https://issues.apache.org/jira/browse/HAWQ-138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Jin closed HAWQ-138. --- > Alter resource quene error when memory set 64mb > --- > > Key: HAWQ-138 > URL: https://issues.apache.org/jira/browse/HAWQ-138 > Project: Apache HAWQ > Issue Type: Bug > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > > postgres=# alter resource queue pg_default with > (vseg_resource_quota='mem:64mb'); > WARNING: Fail to manipulate resource queue because vsegresourcequota value > mem:64mb is not valid, only 64mb, 128mb, 256mb, 512mb, 1gb, 2gb, 4gb, 8gb, > 16gb are valid. > ERROR: Can not apply ALTER RESOURCE QUEUE because vsegresourcequota value > mem:64mb is not valid, only 64mb, 128mb, 256mb, 512mb, 1gb, 2gb, 4gb, 8gb, > 16gb are valid. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HAWQ-138) Alter resource quene error when memory set 64mb
[ https://issues.apache.org/jira/browse/HAWQ-138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Jin resolved HAWQ-138. - Resolution: Fixed > Alter resource quene error when memory set 64mb > --- > > Key: HAWQ-138 > URL: https://issues.apache.org/jira/browse/HAWQ-138 > Project: Apache HAWQ > Issue Type: Bug > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > > postgres=# alter resource queue pg_default with > (vseg_resource_quota='mem:64mb'); > WARNING: Fail to manipulate resource queue because vsegresourcequota value > mem:64mb is not valid, only 64mb, 128mb, 256mb, 512mb, 1gb, 2gb, 4gb, 8gb, > 16gb are valid. > ERROR: Can not apply ALTER RESOURCE QUEUE because vsegresourcequota value > mem:64mb is not valid, only 64mb, 128mb, 256mb, 512mb, 1gb, 2gb, 4gb, 8gb, > 16gb are valid. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-123) Add 'Partition by' description in psql
[ https://issues.apache.org/jira/browse/HAWQ-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999347#comment-14999347 ] ASF GitHub Bot commented on HAWQ-123: - GitHub user hsyuan opened a pull request: https://github.com/apache/incubator-hawq/pull/93 HAWQ-123. Add 'Partition by' description in psql You can merge this pull request into a Git repository by running: $ git pull https://github.com/hsyuan/incubator-hawq master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/93.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #93 commit 0432996d1b0728de99c5db1028aade67034109db Author: Haisheng Yuan Date: 2015-11-10T19:51:59Z HAWQ-123. Add 'Partition by' description in psql > Add 'Partition by' description in psql > -- > > Key: HAWQ-123 > URL: https://issues.apache.org/jira/browse/HAWQ-123 > Project: Apache HAWQ > Issue Type: Improvement > Components: Command Line Tools >Reporter: Haisheng Yuan >Assignee: Lei Chang >Priority: Minor > > As a Data Engineer, I should be able to see partition key in psql so that I > can better understand the schema of the partition table. > GIVEN I am a Data Engineer > WHEN I run \d on a partition table > THEN I should see the partition key of the current level in the output > Notes: > the sub-children will not be displayed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-123) Add 'Partition by' description in psql
[ https://issues.apache.org/jira/browse/HAWQ-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan updated HAWQ-123: --- Summary: Add 'Partition by' description in psql (was: Data Engineer can see partition key in psql) > Add 'Partition by' description in psql > -- > > Key: HAWQ-123 > URL: https://issues.apache.org/jira/browse/HAWQ-123 > Project: Apache HAWQ > Issue Type: Improvement > Components: Command Line Tools >Reporter: Haisheng Yuan >Assignee: Lei Chang >Priority: Minor > > As a Data Engineer, I should be able to see partition key in psql so that I > can better understand the schema of the partition table. > GIVEN I am a Data Engineer > WHEN I run \d on a partition table > THEN I should see the partition key of the current level in the output > Notes: > the sub-children will not be displayed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-44) Advanced statistics for PXF tables
[ https://issues.apache.org/jira/browse/HAWQ-44?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999115#comment-14999115 ] ASF GitHub Bot commented on HAWQ-44: GitHub user hornn opened a pull request: https://github.com/apache/incubator-hawq/pull/92 Analyze HAWQ-44 Advanced statistics for PXF table. PXF sample rows are collected into a temporary table, where statistics are derived of them in the same way ANALYZE works for hawq tables. Statistics are gathered at 3 stages: 1. Getting general statistics - number of fragments, size of data source, size of first fragment 2. Count of first fragment tuples HAWQ uses these numbers to determine how many tuples are needed, and these parameters are translated to sampling ratio and number of sampled fragments. 3. Sampling the PXF table based on the sampling ratio and number of fragments to be sampled. The returned tuples are saved in a temporary table. On the PXF side, a function has been made to the Fragmenter API, to allow gathering the stats of the first stage. In addition, a mechanism to sample rows on the fly was added to the Bridge. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hornn/incubator-hawq analyze_HAWQ-44 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/92.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #92 commit 4237b106c6b3787b7770162fec9e23cd53e20d5e Author: Noa Horn Date: 2015-10-12T19:20:01Z HAWQ-44. PXF Advanced Statistics: hawq side changes commit 4eeb40f2a5aaded9e241b19580b7f42875853f4b Author: Noa Horn Date: 2015-10-18T10:34:13Z HAWQ-44. PXF Advanced Statistics: java side commit 7c9c64584c7dd4d8b9e4525ef1fa347805b94699 Author: Noa Horn Date: 2015-10-25T09:34:19Z HAWQ-44. documentation commit ca7ebb118047147c95d4c998eb7650a65bc73045 Author: Noa Horn Date: 2015-11-02T15:14:19Z HAWQ-44. Add function to Fragmenter API to retrieve fragments stats, with default implementation. Add specific implementation to HdfsDataFragmenter. Add code in HAWQ to call new API, and clean up code that called analyzer. commit 38ab2e601908f755859fb65329aaf4e3be26ca8c Author: Noa Horn Date: 2015-11-04T21:02:21Z HAWQ-44. Update package name in new files commit 8c7955c82a9d5c2e4a2db9f240d271c46ecb9bf9 Author: Noa Horn Date: 2015-11-06T19:03:09Z HAWQ-44. Disable getFragmentsStats for HBase and Hive fragmenters commit 4a8183a707c21cf242d454a280570c45bcd2d880 Author: Noa Horn Date: 2015-11-10T02:29:07Z HAWQ-44. Change stats to include unit together with size of resource to avoid overflow. commit 49b3e448436c32561af2fc16749ec484216770d6 Author: Noa Horn Date: 2015-11-10T18:48:42Z HAWQ-44. Remove extra lines > Advanced statistics for PXF tables > -- > > Key: HAWQ-44 > URL: https://issues.apache.org/jira/browse/HAWQ-44 > Project: Apache HAWQ > Issue Type: New Feature > Components: PXF >Reporter: Noa Horn >Assignee: Noa Horn > Labels: Performance > > PXF will get full statistics on a table using sampling. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-143) Add informations in Apache-HAWQ README.md
[ https://issues.apache.org/jira/browse/HAWQ-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998317#comment-14998317 ] ASF GitHub Bot commented on HAWQ-143: - Github user changleicn commented on the pull request: https://github.com/apache/incubator-hawq/pull/91#issuecomment-155372550 Good! > Add informations in Apache-HAWQ README.md > - > > Key: HAWQ-143 > URL: https://issues.apache.org/jira/browse/HAWQ-143 > Project: Apache HAWQ > Issue Type: Improvement > Components: Documentation >Reporter: Xiang Sheng >Assignee: Lei Chang > > Currently there is only one line "Apache HAWQ" in the README.md file . We > should add more introduction information about Apache HAWQ. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HAWQ-140) Add more information in HAWQ build instructions file
[ https://issues.apache.org/jira/browse/HAWQ-140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiang Sheng resolved HAWQ-140. -- Resolution: Fixed > Add more information in HAWQ build instructions file > > > Key: HAWQ-140 > URL: https://issues.apache.org/jira/browse/HAWQ-140 > Project: Apache HAWQ > Issue Type: Improvement > Components: Build >Reporter: Xiang Sheng >Assignee: Xiang Sheng > > As hawq has been opened source. there should be a better HAWQ build > instructions file which is located the top directory of HAWQ. The build > instruction file should clarify the environment and dependencies, how to > build , install and test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-140) Add more information in HAWQ build instructions file
[ https://issues.apache.org/jira/browse/HAWQ-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998259#comment-14998259 ] ASF GitHub Bot commented on HAWQ-140: - Github user stanlyxiang closed the pull request at: https://github.com/apache/incubator-hawq/pull/89 > Add more information in HAWQ build instructions file > > > Key: HAWQ-140 > URL: https://issues.apache.org/jira/browse/HAWQ-140 > Project: Apache HAWQ > Issue Type: Improvement > Components: Build >Reporter: Xiang Sheng >Assignee: Xiang Sheng > > As hawq has been opened source. there should be a better HAWQ build > instructions file which is located the top directory of HAWQ. The build > instruction file should clarify the environment and dependencies, how to > build , install and test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-143) Add informations in Apache-HAWQ README.md
[ https://issues.apache.org/jira/browse/HAWQ-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998252#comment-14998252 ] ASF GitHub Bot commented on HAWQ-143: - GitHub user stanlyxiang opened a pull request: https://github.com/apache/incubator-hawq/pull/91 HAWQ-143. Add informations in Apache-HAWQ README.md Add introduction information about Apache HAWQ in README file. You can merge this pull request into a Git repository by running: $ git pull https://github.com/stanlyxiang/incubator-hawq readme Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/91.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #91 commit 5401b10452f513587e3c0bf02486b5c153424b39 Author: stanlyxiang Date: 2015-11-10T08:30:59Z HAWQ-143. Add informations in Apache-HAWQ README.md > Add informations in Apache-HAWQ README.md > - > > Key: HAWQ-143 > URL: https://issues.apache.org/jira/browse/HAWQ-143 > Project: Apache HAWQ > Issue Type: Improvement > Components: Documentation >Reporter: Xiang Sheng >Assignee: Lei Chang > > Currently there is only one line "Apache HAWQ" in the README.md file . We > should add more introduction information about Apache HAWQ. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-144) Build HAWQ on MacOS
[ https://issues.apache.org/jira/browse/HAWQ-144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei Chang updated HAWQ-144: --- Description: Currently, the only tested build platform for HAWQ is redhat 6.x. It will be very nice if it can work on Mac with clang. This will make new contributions much easier. Instructions on building HAWQ on linux is at: https://github.com/apache/incubator-hawq/blob/master/BUILD_INSTRUCTIONS.md was:Currently, the only tested build platform for HAWQ is redhat 6.x. It will be very nice if it can work on Mac with clang. This will make new contributions much easier. > Build HAWQ on MacOS > --- > > Key: HAWQ-144 > URL: https://issues.apache.org/jira/browse/HAWQ-144 > Project: Apache HAWQ > Issue Type: New Feature > Components: Build >Reporter: Lei Chang >Assignee: Lei Chang > > Currently, the only tested build platform for HAWQ is redhat 6.x. It will be > very nice if it can work on Mac with clang. This will make new contributions > much easier. > Instructions on building HAWQ on linux is at: > https://github.com/apache/incubator-hawq/blob/master/BUILD_INSTRUCTIONS.md -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-144) Build HAWQ on MacOS
Lei Chang created HAWQ-144: -- Summary: Build HAWQ on MacOS Key: HAWQ-144 URL: https://issues.apache.org/jira/browse/HAWQ-144 Project: Apache HAWQ Issue Type: New Feature Components: Build Reporter: Lei Chang Assignee: Lei Chang Currently, the only tested build platform for HAWQ is redhat 6.x. It will be very nice if it can work on Mac with clang. This will make new contributions much easier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-143) Add informations in Apache-HAWQ README.md
Xiang Sheng created HAWQ-143: Summary: Add informations in Apache-HAWQ README.md Key: HAWQ-143 URL: https://issues.apache.org/jira/browse/HAWQ-143 Project: Apache HAWQ Issue Type: Improvement Components: Documentation Reporter: Xiang Sheng Assignee: Lei Chang Currently there is only one line "Apache HAWQ" in the README.md file . We should add more introduction information about Apache HAWQ. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
