Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-16 Thread Amit Kapila
On Sat, Nov 8, 2014 at 1:26 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I just pushed this, after some more minor tweaks. Thanks, and please do continue testing! Few typo's and few questions 1. * range. Need to an extra flag in mmtuples for that. */ Datum

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-11 Thread Greg Stark
On Tue, Nov 11, 2014 at 2:14 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I'm not sure why you say opaque is not associated with the specific scan. Are you thinking we could reuse opaque for a future scan? I think we could consider that opaque *is* the place to cache things such as the

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-11 Thread Alvaro Herrera
Greg Stark wrote: On Tue, Nov 11, 2014 at 2:14 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I'm not sure why you say opaque is not associated with the specific scan. Are you thinking we could reuse opaque for a future scan? I think we could consider that opaque *is* the place to

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-11 Thread Greg Stark
On Tue, Nov 11, 2014 at 12:12 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: As far as I understand, the scan keys don't change within any given scan; if they do, the rescan AM method is called, at which point we should reset whatever is cached about the previous scan. But am I guaranteed

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-11 Thread Alvaro Herrera
Greg Stark wrote: On Tue, Nov 11, 2014 at 12:12 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: As far as I understand, the scan keys don't change within any given scan; if they do, the rescan AM method is called, at which point we should reset whatever is cached about the previous

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-11 Thread Greg Stark
It might be clearer to have an opclassinfo and a scaninfo which can store information in separate opc_opaque and scan_opaque fields with distinct liftetimes. In the bloom filter case the longlived info is the (initial?) size of the bloom filter and the number of hash functions. But I still

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-11 Thread Alvaro Herrera
Greg Stark wrote: It might be clearer to have an opclassinfo and a scaninfo which can store information in separate opc_opaque and scan_opaque fields with distinct liftetimes. In the bloom filter case the longlived info is the (initial?) size of the bloom filter and the number of hash

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-11 Thread Greg Stark
On Tue, Nov 11, 2014 at 1:04 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: It might be clearer ... Patches welcome :-) Or perhaps there could still be a single opaque field but have two optional opclass methods scaninit and rescan which allow the op class to set or reset whichever fields

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-10 Thread Alvaro Herrera
Fujii Masao wrote: I got the following PANIC error in the standby server when I set up the replication servers and ran make installcheck. Note that I was repeating the manual CHECKPOINT every second while installcheck was running. Without the checkpoints, I could not reproduce the problem.

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-10 Thread Alvaro Herrera
Alvaro Herrera wrote: Hm, I think I see what's happening. The xl_brin_update record references two buffers, one which is target for the updated tuple and another which is the revmap buffer. When the update target buffer is being first used we set the INIT bit which removes the buffer

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-10 Thread Alvaro Herrera
Greg Stark wrote: 1) The manual describes the exensibility API including the BrinOpcInfo struct -- but it doesn't define the BrinDesc struct that every API method takes. It's not clear what exactly that argument is for or how to make use of it. Hm, I guess this could use some expansion. 2)

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-10 Thread Alvaro Herrera
Greg Stark wrote: On Sun, Nov 9, 2014 at 5:57 PM, Greg Stark st...@mit.edu wrote: 2) The mention about additional opclass operators and to number them from 11 up is fine -- but there's no explanation of how to decide what operators need to be explicitly added like that. Specifically I

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-10 Thread Greg Stark
On Mon, Nov 10, 2014 at 9:31 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Every time the index is accessed, yeah. I'm not sure about figuring the initial creation details. Do you think we need another support procedure to help with that? We can add it if needed; minmax would just

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-10 Thread Alvaro Herrera
Greg Stark wrote: There's another API question I have. To implement Consistent I need to call the hash function which in the case of functions like hashtext could be fairly expensive and I even need to generate multiple hash values(though currently I'm slicing them all from the integer hash

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-09 Thread Heikki Linnakangas
On 11/09/2014 08:06 AM, Michael Paquier wrote: On Sat, Nov 8, 2014 at 5:40 PM, David Rowley dgrowle...@gmail.com wrote: Please find attached another small fix. This time it's just a small typo in the README, and just some updates to some, now outdated docs. Speaking about the feature... The

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-09 Thread Fujii Masao
On Sat, Nov 8, 2014 at 4:56 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I just pushed this, after some more minor tweaks. Nice! Thanks, and please do continue testing! I got the following PANIC error in the standby server when I set up the replication servers and ran make

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-09 Thread Greg Stark
On Sun, Nov 9, 2014 at 9:18 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Speaking of which, Alvaro, any chance we could get such on opclass still included into 9.5? It would be nice to have one, just to be sure that nothing minmax-specific has crept into the BRIN code. I'm trying to

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-09 Thread Greg Stark
On Sun, Nov 9, 2014 at 5:06 PM, Greg Stark st...@mit.edu wrote: I'm trying to do a bloom filter Brin index. I'm already a bit puzzled by a few things but I've just started so maybe it'll become clear. So some quick comments from pretty early goings -- partly because I'm afraid once I get past

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-09 Thread Greg Stark
On Sun, Nov 9, 2014 at 5:57 PM, Greg Stark st...@mit.edu wrote: 2) The mention about additional opclass operators and to number them from 11 up is fine -- but there's no explanation of how to decide what operators need to be explicitly added like that. Specifically I gather from reading minmax

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-09 Thread Alvaro Herrera
Heikki Linnakangas wrote: Speaking of which, Alvaro, any chance we could get such on opclass still included into 9.5? It would be nice to have one, just to be sure that nothing minmax-specific has crept into the BRIN code. Emre Hasegeli contributed a patch for range types. I am hoping he

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-09 Thread Amit Langote
On Sun, Nov 9, 2014 at 10:30 PM, Fujii Masao masao.fu...@gmail.com wrote: On Sat, Nov 8, 2014 at 4:56 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I just pushed this, after some more minor tweaks. Nice! Thanks, and please do continue testing! I got the following PANIC error in the

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-08 Thread David Rowley
On Sat, Nov 8, 2014 at 8:56 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I just pushed this, after some more minor tweaks. Thanks, and please do continue testing! Please find attached another small fix. This time it's just a small typo in the README, and just some updates to some, now

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-08 Thread Michael Paquier
On Sat, Nov 8, 2014 at 5:40 PM, David Rowley dgrowle...@gmail.com wrote: Please find attached another small fix. This time it's just a small typo in the README, and just some updates to some, now outdated docs. Speaking about the feature... The index operators are still named with minmax,

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-07 Thread Alvaro Herrera
I just pushed this, after some more minor tweaks. Thanks, and please do continue testing! -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-07 Thread David Rowley
On Sat, Nov 8, 2014 at 8:56 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I just pushed this, after some more minor tweaks. Thanks, and please do continue testing! I'm having problems getting this to compile on MSVC. Attached is a patch which fixes the problem. There also seems to be

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-07 Thread Tom Lane
David Rowley dgrowle...@gmail.com writes: I'm having problems getting this to compile on MSVC. Attached is a patch which fixes the problem. The committed code is completely broken on compilers that don't accept varargs macros, and this patch will not make them happier. Probably what needs to

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-07 Thread Alvaro Herrera
Tom Lane wrote: David Rowley dgrowle...@gmail.com writes: I'm having problems getting this to compile on MSVC. Attached is a patch which fixes the problem. The committed code is completely broken on compilers that don't accept varargs macros, and this patch will not make them happier. I

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-07 Thread David Rowley
On Sat, Nov 8, 2014 at 8:56 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I just pushed this, after some more minor tweaks. Thanks, and please do continue testing! Here's another small fix for some unused variable warnings. Unfortunately this Microsoft compiler that I'm using does not

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-06 Thread Alvaro Herrera
Jeff Janes wrote: On Wed, Nov 5, 2014 at 12:54 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thanks for the updated patch. Now when I run the test program (version with better error reporting attached), it runs fine until I open a psql session and issue: reindex table foo;

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-05 Thread Jeff Janes
On Tue, Nov 4, 2014 at 2:28 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I was clearly too careless about testing the xlog code --- it had numerous bugs. This version should be a lot better, but there might be problems lurking still as I don't think I covered it all. Let me know if

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-05 Thread Jeff Janes
On Wed, Nov 5, 2014 at 12:54 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thanks for the updated patch. Now when I run the test program (version with better error reporting attached), it runs fine until I open a psql session and issue: reindex table foo; Then it immediately falls over

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-04 Thread Simon Riggs
On 3 November 2014 22:18, Alvaro Herrera alvhe...@2ndquadrant.com wrote: So here's v21. I also attach a partial diff from v20, just in case anyone wants to give it a look. Looks really good. I'd like to reword this sentence in the readme, since one of the main use cases would be tables

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-04 Thread Jeff Janes
On Mon, Nov 3, 2014 at 2:18 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: So here's v21. I also attach a partial diff from v20, just in case anyone wants to give it a look. This needs a bump to 1.3, or the extension won't install: contrib/pageinspect/pageinspect.control During crash

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-03 Thread Jeff Janes
On Mon, Nov 3, 2014 at 2:18 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: [lots] I have fixed all these items in the attached, thanks -- most user-visible change was the pageinspect 1.3 thingy. pg_upgrade from 1.2 works fine now. I also fixed some things Heikki

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-03 Thread Alvaro Herrera
Jeff Janes wrote: On Mon, Nov 3, 2014 at 2:18 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I get a couple compiler warnings with this: brin.c: In function 'brininsert': brin.c:97: warning: 'tupcxt' may be used uninitialized in this function brin.c:98: warning: 'oldcxt' may be used

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-10-30 Thread Robert Haas
+ acronymBRIN/acronym indexes can satisfy queries via the bitmap + scanning facility, and will return all tuples in all pages within The bitmap scanning facility? Does this mean a bitmap index scan? Or something novel to BRIN? I think this could be clearer. + This enables them to work as

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-10-29 Thread Alvaro Herrera
Heikki Linnakangas wrote: On 10/07/2014 01:33 AM, Alvaro Herrera wrote: I added an USE_ASSERTION-only block in brininsert that runs the union support proc and compares the output with the one from regular addValue. I haven't tested this too much yet. Ok, that's better than nothing. I

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-10-07 Thread Heikki Linnakangas
On 10/07/2014 01:33 AM, Alvaro Herrera wrote: Heikki Linnakangas wrote: On 09/23/2014 10:04 PM, Alvaro Herrera wrote: + Open questions + -- + + * Same-size page ranges? + Current related literature seems to consider that each index entry in a + BRIN index must cover the same

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-26 Thread Erik Rijkers
On Tue, September 23, 2014 21:04, Alvaro Herrera wrote: Alvaro Herrera wrote: [minmax-19.patch] [minmax-19a.patch] Although admittedly it is not directly likely for us to need it, and although I see that there is a BRIN Extensibility chapter added (good!), I am still a bit surprised by the

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-24 Thread Heikki Linnakangas
On 09/23/2014 10:04 PM, Alvaro Herrera wrote: + para + The acronymBRIN/acronym implementation in productnamePostgreSQL/productname + is primarily maintained by Aacute;lvaro Herrera. + /para We don't usually have such verbiage in the docs. The GIN and GiST pages do, but I think those

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-23 Thread Alvaro Herrera
Alvaro Herrera wrote: Heikki Linnakangas wrote: If you add a new datatype, and define b-tree operators for it, what is required to create a minmax opclass for it? Would it be possible to generalize the functions in brin_minmax.c so that they can be reused for any datatype (with b-tree

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 3:04 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Alvaro Herrera wrote: Heikki Linnakangas wrote: If you add a new datatype, and define b-tree operators for it, what is required to create a minmax opclass for it? Would it be possible to generalize the

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-23 Thread Michael Paquier
On Wed, Sep 24, 2014 at 8:23 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Sep 23, 2014 at 3:04 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Alvaro Herrera wrote: I will look into adding some testing mechanism for the union support proc; with that I will just consider the patch

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-23 Thread Alvaro Herrera
Robert Haas wrote: With all respect, I think this is a bad idea. I know you've put a lot of energy into this patch and I'm confident it's made a lot of progress. But as with Stephen's patch, the final form deserves a thorough round of looking over by someone else before it goes in. As you

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 7:35 PM, Michael Paquier michael.paqu...@gmail.com wrote: Would this person be it an extra committer or an simple reviewer? It would give more insurance if such huge patches (couple of thousands of lines) get an extra +1 from another committer, proving that the code has

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 9:23 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: With all respect, I think this is a bad idea. I know you've put a lot of energy into this patch and I'm confident it's made a lot of progress. But as with Stephen's patch, the final form

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-23 Thread Alvaro Herrera
Robert Haas wrote: On Tue, Sep 23, 2014 at 9:23 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: If you or somebody else want to give it a look, I have no problem waiting a bit longer. I don't want to delay indefinitely, though, because I think it's better shipped early in the release

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-08 Thread Erik Rijkers
On Mon, September 8, 2014 18:02, Alvaro Herrera wrote: Here's version 18. I have renamed it: These are now BRIN indexes. I get into a BadArgument after: $ cat crash.sql -- drop table if exists t_100_000_000 cascade; create table t_100_000_000 as select cast(i as integer) from

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-08 Thread Alvaro Herrera
Erik Rijkers wrote: Log file says: TRAP: BadArgument(!(((context) != ((void *)0) (const Node*)((context)))-type) == T_AllocSetContext, File: mcxt.c, Line: 752) 2014-09-08 19:54:46.071 CEST 30151 LOG: server process (PID 30336) was terminated by signal 6: Aborted 2014-09-08