On 2.12.2014 06:14, Jeff Davis wrote:
> On Sun, 2014-11-30 at 17:49 -0800, Peter Geoghegan wrote:
>> On Mon, Nov 17, 2014 at 11:39 PM, Jeff Davis <pg...@j-davis.com> wrote:
>>> I can also just move isReset there, and keep mem_allocated as a uint64.
>>> That way, if I find later that I want to track the aggregated value for
>>> the child contexts as well, I can split it into two uint32s. I'll hold
>>> off any any such optimizations until I see some numbers from HashAgg
>>> though.
>>
>> I took a quick look at memory-accounting-v8.patch.
>>
>> Is there some reason why mem_allocated is a uint64? All other things
>> being equal, I'd follow the example of tuplesort.c's
>> MemoryContextAllocHuge() API, which (following bugfix commit
>> 79e0f87a1) uses int64 variables to track available memory and so on.
>
> No reason. New version attached; that's the only change.
I've started reviewing this today. It does not apply cleanly on current
head, because of 4a14f13a committed a few days ago. That commit changed
the constants in src/include/utils/hsearch.h, so the patch needs to use
this:
#define HASH_NOCHILDCXT 0x4000 /* ... */
That's the only conflict, and after fixing it it compiles OK. However, I
got a segfault on the very first query I tried :-(
create table test_hash_agg as select i AS a, i AS b, i AS c, i AS d
from generate_series(1,10000000) s(i);
analyze test_hash_agg;
select a, count(*) from test_hash_agg where a < 100000 group by a;
With a fresh cluster (default config), I get this:
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
The backtrace looks like this (full attached):
Program received signal SIGSEGV, Segmentation fault.
advance_transition_function (aggstate=aggstate@entry=0x2b5c5f0,
peraggstate=peraggstate@entry=0x2b5efd0,
pergroupstate=pergroupstate@entry=0x8) at nodeAgg.c:468
468 if (pergroupstate->noTransValue)
(gdb) bt
#0 advance_transition_function at nodeAgg.c:468
#1 0x00000000005c3494 in advance_aggregates at nodeAgg.c:624
#2 0x00000000005c3dc2 in agg_fill_hash_table at nodeAgg.c:1640
#3 ExecAgg (node=node@entry=0x2b5c5f0) at nodeAgg.c:1338
(gdb) print pergroupstate->noTransValue
Cannot access memory at address 0x11
(gdb) print pergroupstate
$1 = (AggStatePerGroup) 0x8
My guess is something is scribbling over the pergroup memory, or maybe
the memory context gets released, or something. In any case, it's easily
reproducible, and apparently deterministic (always exactly the same
values, no randomness).
regards
Tomas
Program received signal SIGSEGV, Segmentation fault.
advance_transition_function (aggstate=aggstate@entry=0x2b5c5f0,
peraggstate=peraggstate@entry=0x2b5efd0, pergroupstate=pergroupstate@entry=0x8)
at nodeAgg.c:468
468 if (pergroupstate->noTransValue)
(gdb) bt
#0 advance_transition_function (aggstate=aggstate@entry=0x2b5c5f0,
peraggstate=peraggstate@entry=0x2b5efd0, pergroupstate=pergroupstate@entry=0x8)
at nodeAgg.c:468
#1 0x00000000005c3494 in advance_aggregates
(aggstate=aggstate@entry=0x2b5c5f0, pergroup=pergroup@entry=0x8) at
nodeAgg.c:624
#2 0x00000000005c3dc2 in agg_fill_hash_table (aggstate=0x2b5c5f0) at
nodeAgg.c:1640
#3 ExecAgg (node=node@entry=0x2b5c5f0) at nodeAgg.c:1338
#4 0x00000000005b6c68 in ExecProcNode (node=node@entry=0x2b5c5f0) at
execProcnode.c:486
#5 0x00000000005b3e90 in ExecutePlan (dest=0xbffa40 <donothingDR>,
direction=<optimized out>, numberTuples=0, sendTuples=1 '\001',
operation=CMD_SELECT, planstate=0x2b5c5f0, estate=0x2b5c4d8) at execMain.c:1480
#6 standard_ExecutorRun (queryDesc=0x2b58d78, direction=<optimized out>,
count=0) at execMain.c:308
#7 0x000000000056df73 in ExplainOnePlan
(plannedstmt=plannedstmt@entry=0x2b58ce0, into=into@entry=0x0,
es=es@entry=0x7fff6fa066c0, queryString=<optimized out>, params=0x0,
planduration=planduration@entry=0x7fff6fa06650) at explain.c:487
#8 0x000000000056e22d in ExplainOneQuery (query=<optimized out>, into=0x0,
es=0x7fff6fa066c0, queryString=0x2b0bf68 "explain analyze select a, count(*)
from test_hash_agg where a < 100000 group by a;", params=0x0) at explain.c:341
#9 0x000000000056e670 in ExplainQuery (stmt=stmt@entry=0x2b0d248,
queryString=0x2b0bf68 "explain analyze select a, count(*) from test_hash_agg
where a < 100000 group by a;", params=params@entry=0x0, dest=0x2b1b490) at
explain.c:232
#10 0x00000000006b5af6 in standard_ProcessUtility (parsetree=0x2b0d248,
queryString=<optimized out>, context=<optimized out>, params=0x0,
dest=<optimized out>, completionTag=0x7fff6fa067d0 "") at utility.c:648
#11 0x00000000006b2c61 in PortalRunUtility (portal=portal@entry=0x2b4c3b8,
utilityStmt=0x2b0d248, isTopLevel=<optimized out>, dest=dest@entry=0x2b1b490,
completionTag=completionTag@entry=0x7fff6fa067d0 "") at pquery.c:1187
#12 0x00000000006b3add in FillPortalStore (portal=portal@entry=0x2b4c3b8,
isTopLevel=isTopLevel@entry=1 '\001') at pquery.c:1061
#13 0x00000000006b45cf in PortalRun (portal=portal@entry=0x2b4c3b8,
count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=1 '\001',
dest=dest@entry=0x2b3e828, altdest=altdest@entry=0x2b3e828,
completionTag=completionTag@entry=0x7fff6fa06bd0 "") at pquery.c:785
#14 0x00000000006b1fe3 in exec_simple_query (query_string=0x2b0bf68 "explain
analyze select a, count(*) from test_hash_agg where a < 100000 group by a;") at
postgres.c:1045
#15 PostgresMain (argc=<optimized out>, argv=argv@entry=0x2aa5d20,
dbname=0x2aa5b88 "test", username=<optimized out>) at postgres.c:4012
#16 0x000000000046278f in BackendRun (port=0x2ac5b10) at postmaster.c:4130
#17 BackendStartup (port=0x2ac5b10) at postmaster.c:3805
#18 ServerLoop () at postmaster.c:1573
#19 0x0000000000658156 in PostmasterMain (argc=argc@entry=3,
argv=argv@entry=0x2aa4900) at postmaster.c:1220
#20 0x000000000046338b in main (argc=3, argv=0x2aa4900) at main.c:220
(gdb) print pergroupstate->noTransValue
Cannot access memory at address 0x11
(gdb) print pergroupstate
$1 = (AggStatePerGroup) 0x8
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
