"Teodor Sigaev" <[EMAIL PROTECTED]> writes: >> And it's not clear _int_gist.c has been running with toasted array values for >> years because it's limited to arrays of 100 integers (or perhaps 200 >> integers, >> there's a factor of 2 in the test). That's not enough to trigger toasting >> unless there are other large columns in the same table. > > That's was intended limitation to prevent indexing of huge arrays. > gist__int_ops compression method is orientated for small and isn't effective > on > big ones.
Right, so it's possible nobody see any toasted arrays with _int_gist.c since they never get very large. It looks like index_form_tuple will never compress anything under 512b so I guess it's safe currently. >> I do know that with packed varlenas I get a crash in g_int_union among other >> places. I can't tell where the datum came from originally and how it ended up >> stored in packed format. > Can you provide your patch (in current state) and test suite? Or backtrace at > least. It doesn't actually crash, it just fails CHECKARRVALID. I added an assertion in there to cause it to generate a core dump. You can download the core dump and binary from http://community.enterprisedb.com/varlena/core._int http://community.enterprisedb.com/varlena/postgres._int The last patch (without the assertion) is at: http://community.enterprisedb.com/varlena/patch-varvarlena-14.patch.gz What I'm seeing is this: (gdb) f 3 #3 0xb7fd924b in inner_int_union (a=0x84e41f4, b=0xb64220d0) at _int_tool.c:81 81 CHECKARRVALID(b); The array is actually garbage: (gdb) p *b $2 = {vl_len_ = 141, ndim = 0, dataoffset = 5888, elemtype = 0} What's going on is that the va_1byte header is 141 which is 0x80 | 13. So it's actually only 13 bytes with a 1 byte header or a 12 byte array: (gdb) p *(varattrib*)b $3 = {va_1byte = {va_header = 141 '\215', va_data = ""}, va_external = { va_header = 141 '\215', va_padding = "\000\000", va_rawsize = 0, va_extsize = 5888, va_valueid = 0, va_toastrelid = 0}, va_compressed = { va_header = 141, va_rawsize = 0, va_data = ""}, va_4byte = { va_header = 141, va_data = ""}} (gdb) bt #0 0xb7e6a947 in raise () from /lib/tls/libc.so.6 #1 0xb7e6c0c9 in abort () from /lib/tls/libc.so.6 #2 0x082fec97 in ExceptionalCondition ( conditionName=0xb7fdd3b9 "!(!((b)->dataoffset != 0))", errorType=0xb7fdd371 "FailedAssertion", fileName=0xb7fdd347 "_int_tool.c", lineNumber=81) at assert.c:51 #3 0xb7fd924b in inner_int_union (a=0x84e41f4, b=0xb64220d0) at _int_tool.c:81 #4 0xb7fd547d in g_int_picksplit (fcinfo=0xbf9e43f0) at _int_gist.c:403 #5 0x08304d9c in FunctionCall2 (flinfo=0xbf9e5a94, arg1=139342312, arg2=3214821160) at fmgr.c:1154 #6 0x08094fd3 in gistUserPicksplit (r=0xb6078d4c, entryvec=0x84e31e8, attno=0, v=0xbf9e4728, itup=0x84e2ddc, len=142, giststate=0xbf9e4b94) at gistsplit.c:306 #7 0x08095deb in gistSplitByKey (r=0xb6078d4c, page=0xb6420220 "", itup=0x84e2ddc, len=142, giststate=0xbf9e4b94, v=0xbf9e4728, entryvec=0x84e31e8, attno=0) at gistsplit.c:548 #8 0x080874bd in gistSplit (r=0xb6078d4c, page=0xb6420220 "", itup=0x84e2ddc, len=142, giststate=0xbf9e4b94) at gist.c:943 #9 0x080850fa in gistplacetopage (state=0xbf9e49e0, giststate=0xbf9e4b94) at gist.c:329 #10 0x080871eb in gistmakedeal (state=0xbf9e49e0, giststate=0xbf9e4b94) at gist.c:873 #11 0x08084f21 in gistdoinsert (r=0xb6078d4c, itup=0x84e2ce4, freespace=819, giststate=0xbf9e4b94) at gist.c:278 #12 0x08084cf5 in gistbuildCallback (index=0xb6078d4c, htup=0x84c8c30, values=0xbf9e4a98, isnull=0xbf9e4a78 "", tupleIsAlive=1 '\001', state=0xbf9e4b94) at gist.c:201 #13 0x080fc81f in IndexBuildHeapScan (heapRelation=0xb60d6860, indexRelation=0xb6078d4c, indexInfo=0x84cd620, callback=0x8084c27 <gistbuildCallback>, callback_state=0xbf9e4b94) at index.c:1548 #14 0x08084bdd in gistbuild (fcinfo=0xbf9e60e8) at gist.c:150 #15 0x08305630 in OidFunctionCall3 (functionId=782, arg1=3054332000, arg2=3053948236, arg3=139253280) at fmgr.c:1460 #16 0x080fc363 in index_build (heapRelation=0xb60d6860, indexRelation=0xb6078d4c, indexInfo=0x84cd620, isprimary=0 '\0') at index.c:1296 #17 0x080fb86a in index_create (heapRelationId=21361, indexRelationName=0x84a531c "text_idx", indexRelationId=21366, indexInfo=0x84cd620, accessMethodObjectId=783, tableSpaceId=0, classObjectId=0x84cd60c, coloptions=0x84cd6ac, reloptions=0, isprimary=0 '\0', isconstraint=0 '\0', allow_system_table_mods=0 '\0', skip_build=0 '\0', concurrent=0 '\0') at index.c:794 #18 0x0815f3e4 in DefineIndex (heapRelation=0x84a5354, indexRelationName=0x84a531c "text_idx", indexRelationId=0, accessMethodName=0x84a5380 "gist", tableSpaceName=0x0, attributeList=0x84a5448, predicate=0x0, rangetable=0x0, options=0x0, unique=0 '\0', primary=0 '\0', isconstraint=0 '\0', is_alter_table=0 '\0', check_rights=1 '\001', skip_build=0 '\0', quiet=0 '\0', concurrent=0 '\0') at indexcmds.c:452 #19 0x0825dcea in ProcessUtility (parsetree=0x84a5464, params=0x0, dest=0x84a54e0, completionTag=0xbf9e687e "") at utility.c:797 #20 0x0825bef9 in PortalRunUtility (portal=0x84ca2ec, utilityStmt=0x84a5464, dest=0x84a54e0, completionTag=0xbf9e687e "") at pquery.c:1176 #21 0x0825c04b in PortalRunMulti (portal=0x84ca2ec, dest=0x84a54e0, altdest=0x84a54e0, completionTag=0xbf9e687e "") at pquery.c:1263 #22 0x0825b786 in PortalRun (portal=0x84ca2ec, count=2147483647, dest=0x84a54e0, altdest=0x84a54e0, completionTag=0xbf9e687e "") at pquery.c:814 #23 0x0825604d in exec_simple_query ( query_string=0x84a4fec "CREATE INDEX text_idx on test__int using gist ( a gist__int_ops );") at postgres.c:953 #24 0x08259c2f in PostgresMain (argc=4, argv=0x844df58, username=0x844de44 "stark") at postgres.c:3434 #25 0x08223d2c in BackendRun (port=0x8461550) at postmaster.c:2974 #26 0x082232b9 in BackendStartup (port=0x8461550) at postmaster.c:2601 #27 0x08220d82 in ServerLoop () at postmaster.c:1214 #28 0x08220728 in PostmasterMain (argc=3, argv=0x844a340) at postmaster.c:967 #29 0x081c426b in main (argc=3, argv=0x844a340) at main.c:188 -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---------------------------(end of broadcast)--------------------------- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
