Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-20 Thread Merlin Moncure
On Mon, Sep 19, 2011 at 1:53 PM, Claudio Freire klaussfre...@gmail.com wrote: On Mon, Sep 19, 2011 at 3:43 PM, Merlin Moncure mmonc...@gmail.com wrote: To make the test into i/o bound, I change the setrandom from 10 to 1000; this produced some unexpected results. The hash index is

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-20 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: just selects. update test is also very interesting -- the only test I did for for updates is 'update foo set x=x+1' which was a win for btree (20-30% faster typically). perhaps this isn't algorithm induced though -- lack of wal logging could

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-19 Thread Robert Klemme
On Sun, Sep 18, 2011 at 9:31 PM, Stefan Keller sfkel...@gmail.com wrote: I'm simply referring to literature (like the intro Ramakrishnan Gehrke). I just know that Oracle an Mysql actually do have them too and use it without those current implementation specific restrictions in Postgres.

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-19 Thread Merlin Moncure
On Sun, Sep 18, 2011 at 9:59 AM, Stefan Keller sfkel...@gmail.com wrote: Merlin and Jeff, General remark again:It's hard for me to imagine that btree is superior for all the issues mentioned before. I still believe in hash index for primary keys and certain unique constraints where you need

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-19 Thread Robert Klemme
On Mon, Sep 19, 2011 at 4:04 PM, Merlin Moncure mmonc...@gmail.com wrote: On Sun, Sep 18, 2011 at 9:59 AM, Stefan Keller sfkel...@gmail.com wrote: Merlin and Jeff, General remark again:It's hard for me to imagine that btree is superior for all the issues mentioned before. I still believe in

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-19 Thread Vitalii Tymchyshyn
19.09.11 18:19, Robert Klemme написав(ла): On Mon, Sep 19, 2011 at 4:04 PM, Merlin Moncuremmonc...@gmail.com wrote: Postgres's hash index implementation used to be pretty horrible -- it stored the pre-hashed datum in the index which, while making it easier to do certain things, made it

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-19 Thread Tom Lane
Robert Klemme shortcut...@googlemail.com writes: I still haven't seen a solution to locking when a hash table needs resizing. All hashing algorithms I can think of at the moment would require a lock on the whole beast during the resize which makes this type of index impractical for certain

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-19 Thread Vitalii Tymchyshyn
19.09.11 18:19, Robert Klemme написав(ла): I still haven't seen a solution to locking when a hash table needs resizing. All hashing algorithms I can think of at the moment would require a lock on the whole beast during the resize which makes this type of index impractical for certain loads

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-19 Thread Claudio Freire
On Mon, Sep 19, 2011 at 12:54 PM, Vitalii Tymchyshyn tiv...@gmail.com wrote: 19.09.11 18:19, Robert Klemme написав(ла): I still haven't seen a solution to locking when a hash table needs resizing.  All hashing algorithms I can think of at the moment would require a lock on the whole beast

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-19 Thread Jeff Janes
On Mon, Sep 19, 2011 at 8:19 AM, Robert Klemme shortcut...@googlemail.com wrote: On Mon, Sep 19, 2011 at 4:04 PM, Merlin Moncure mmonc...@gmail.com wrote: The other way to go of course is to try and fix up the existing hash index code -- add wal logging, etc. In theory, a customized hash

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-19 Thread Merlin Moncure
On Mon, Sep 19, 2011 at 10:19 AM, Robert Klemme shortcut...@googlemail.com wrote: On Mon, Sep 19, 2011 at 4:04 PM, Merlin Moncure mmonc...@gmail.com wrote: On Sun, Sep 18, 2011 at 9:59 AM, Stefan Keller sfkel...@gmail.com wrote: Merlin and Jeff, General remark again:It's hard for me to

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-19 Thread Claudio Freire
On Mon, Sep 19, 2011 at 3:43 PM, Merlin Moncure mmonc...@gmail.com wrote: To make the test into i/o bound, I change the setrandom from 10 to 1000; this produced some unexpected results. The hash index is pulling about double the tps (~80 vs ~ 40) over the hybrid version. Well, unless

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-18 Thread Stefan Keller
Merlin and Jeff, General remark again:It's hard for me to imagine that btree is superior for all the issues mentioned before. I still believe in hash index for primary keys and certain unique constraints where you need equality search and don't need ordering or range search. 2011/9/17 Jeff Janes

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-18 Thread Kevin Grittner
Stefan Keller wrote: It's hard for me to imagine that btree is superior for all the issues mentioned before. It would be great if you could show a benchmark technique which shows otherwise. -Kevin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-18 Thread Stefan Keller
I'm simply referring to literature (like the intro Ramakrishnan Gehrke). I just know that Oracle an Mysql actually do have them too and use it without those current implementation specific restrictions in Postgres. IMHO by design Hash Index (e.g. linear hashing) work best when: 1. only equal (=)

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-18 Thread Jeff Janes
On Sun, Sep 18, 2011 at 7:59 AM, Stefan Keller sfkel...@gmail.com wrote: Merlin and Jeff, General remark again:It's hard for me to imagine that btree is superior for all the issues mentioned before. I still believe in hash index for primary keys and certain unique constraints where you need

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-17 Thread Jeff Janes
On Tue, Sep 13, 2011 at 5:04 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 14 September 2011 00:04, Stefan Keller sfkel...@gmail.com wrote: Has this been verified on a recent release? I can't believe that hash performs so bad over all these points. Theory tells me otherwise and

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-17 Thread Jeff Janes
On Wed, Sep 14, 2011 at 4:03 PM, Stefan Keller sfkel...@gmail.com wrote: 2011/9/14 Tom Lane t...@sss.pgh.pa.us writes: (...) I think that the current state of affairs is still what depesz said, namely that there might be cases where they'd be a win to use, except the lack of WAL support is a

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-17 Thread Merlin Moncure
On Sat, Sep 17, 2011 at 4:48 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Tue, Sep 13, 2011 at 5:04 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 14 September 2011 00:04, Stefan Keller sfkel...@gmail.com wrote: Has this been verified on a recent release? I can't believe that hash

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-17 Thread Jeff Janes
On Thu, Sep 15, 2011 at 9:20 PM, Merlin Moncure mmonc...@gmail.com wrote: odd: I was pondering Claudio's point about maintenance of hash indexes vs btree and decided to do some more tests.  Something very strange is happening:  I decided to compare 'update v set x=x+1', historically one of

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-16 Thread Merlin Moncure
On Thu, Sep 15, 2011 at 8:00 PM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, Sep 15, 2011 at 5:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: HM, what if you junked the current hash indexam, and just implemented a wrapper over btree so that the 'hash

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-16 Thread Stefan Keller
2011/9/16 Tom Lane t...@sss.pgh.pa.us: I'm not entirely following this eagerness to junk that AM, anyway. We've put a lot of sweat into it over the years, in the hopes that it would eventually be good for something.  It's on the edge of being good for something now, and there's doubtless room

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-16 Thread Tomas Vondra
Dne 15.9.2011 01:40, Tom Lane napsal(a): Stefan Keller sfkel...@gmail.com writes: 2011/9/14 Tom Lane t...@sss.pgh.pa.us writes: (...) I think that the current state of affairs is still what depesz said, namely that there might be cases where they'd be a win to use, except the lack of WAL

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-15 Thread Merlin Moncure
On Wed, Sep 14, 2011 at 4:03 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 14.09.2011 03:24, Tom Lane wrote: The big picture though is that we're not going to remove hash indexes, even if they're nearly useless in themselves, because hash index opclasses provide the

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-15 Thread Claudio Freire
On Thu, Sep 15, 2011 at 5:00 PM, Merlin Moncure mmonc...@gmail.com wrote: HM, what if you junked the current hash indexam, and just implemented a wrapper over btree so that the 'hash index' was just short hand for hashing the value into a standard index? I'm doing this (only by hand, indexing

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-15 Thread Merlin Moncure
On Thu, Sep 15, 2011 at 3:28 PM, Claudio Freire klaussfre...@gmail.com wrote: On Thu, Sep 15, 2011 at 5:00 PM, Merlin Moncure mmonc...@gmail.com wrote: HM, what if you junked the current hash indexam, and just implemented a wrapper over btree so that the 'hash index' was just short hand for

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-15 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: HM, what if you junked the current hash indexam, and just implemented a wrapper over btree so that the 'hash index' was just short hand for hashing the value into a standard index? Surely creating such a wrapper would be *more* work than adding WAL

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-15 Thread Claudio Freire
On Fri, Sep 16, 2011 at 12:38 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not entirely following this eagerness to junk that AM, anyway. We've put a lot of sweat into it over the years, in the hopes that it would eventually be good for something.  It's on the edge of being good for something

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-15 Thread Merlin Moncure
On Thu, Sep 15, 2011 at 5:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: HM, what if you junked the current hash indexam, and just implemented a wrapper over btree so that the 'hash index' was just short hand for hashing the value into a standard index?

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-15 Thread Claudio Freire
On Fri, Sep 16, 2011 at 3:00 AM, Merlin Moncure mmonc...@gmail.com wrote: c:\Program Files\PostgreSQL\9.0\datadir/s | grep 16525 09/15/2011  07:46 PM       224,641,024 16525 c:\Program Files\PostgreSQL\9.0\datadir/s | grep 16526 09/15/2011  07:49 PM       268,451,840 16526 That's not

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-14 Thread Stefan Keller
2011/9/14 Tom Lane t...@sss.pgh.pa.us: (...) I think that the current state of affairs is still what depesz said, namely that there might be cases where they'd be a win to use, except the lack of WAL support is a killer. I imagine somebody will step up and do that eventually. Should I open

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-14 Thread Heikki Linnakangas
On 14.09.2011 09:39, Stefan Keller wrote: Should I open a ticket? What ticket? With whom? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription:

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-14 Thread Leonardo Francalanci
Hash indexes have been improved since 2005 - their performance was improved quite a bit in 9.0. Here's a more recent analysis: http://www.depesz.com/index.php/2010/06/28/should-you-use-hash-index/ The big picture though is that we're not going to remove hash indexes, even if they're

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-14 Thread Stefan Keller
2011/9/14 Tom Lane t...@sss.pgh.pa.us writes: (...) I think that the current state of affairs is still what depesz said, namely that there might be cases where they'd be a win to use, except the lack of WAL support is a killer. I imagine somebody will step up and do that eventually. How

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-14 Thread Tom Lane
Stefan Keller sfkel...@gmail.com writes: 2011/9/14 Tom Lane t...@sss.pgh.pa.us writes: (...) I think that the current state of affairs is still what depesz said, namely that there might be cases where they'd be a win to use, except the lack of WAL support is a killer. I imagine somebody will

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-13 Thread Peter Geoghegan
On 14 September 2011 00:04, Stefan Keller sfkel...@gmail.com wrote: Has this been verified on a recent release? I can't believe that hash performs so bad over all these points. Theory tells me otherwise and http://en.wikipedia.org/wiki/Hash_table seems to be a success. Hash indexes have been

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-13 Thread Tom Lane
Peter Geoghegan pe...@2ndquadrant.com writes: On 14 September 2011 00:04, Stefan Keller sfkel...@gmail.com wrote: Has this been verified on a recent release? I can't believe that hash performs so bad over all these points. Theory tells me otherwise and http://en.wikipedia.org/wiki/Hash_table