Re: [HACKERS] README of hash index

On Fri, Sep 16, 2016 at 04:50:53PM +0530, Amit Kapila wrote: > Currently README of hash module contain algorithms written in below form. > > The insertion algorithm is rather similar: > > pin meta page and take buffer content lock in shared mode > loop: > compute bucket number for target hash key

Re: [HACKERS] Hash Indexes

On Mon, Sep 19, 2016 at 12:14:26PM +0530, Amit Kapila wrote: > On Mon, Sep 19, 2016 at 11:20 AM, Mark Kirkwood > wrote: > > > > > > On 17/09/16 06:38, Andres Freund wrote: > >> > >> On 2016-09-16 09:12:22 -0700, Jeff Janes wrote: > >>> > >>> On Thu, Sep 15, 2016 at 7:23 AM, Andres Freund > >>> wr

Re: [HACKERS] unique index violation after pg_upgrade to PG10

On Tue, Oct 24, 2017 at 01:14:53PM -0500, Justin Pryzby wrote: > I upgrade another instance to PG10 yesterday and this AM found unique key > violations. > > Our application is SELECTing FROM sites WHERE site_location=$1, and if it > doesn't find one, INSERTs one (I know that's racy and not ideal).

Re: [HACKERS] unique index violation after pg_upgrade to PG10

On Tue, Oct 24, 2017 at 01:30:19PM -0500, Justin Pryzby wrote: > On Tue, Oct 24, 2017 at 01:27:14PM -0500, Kenneth Marshall wrote: > > On Tue, Oct 24, 2017 at 01:14:53PM -0500, Justin Pryzby wrote: > > > > Note: > > > I run a script which does various combinations

Re: [HACKERS] bytea_output output of base64

On Thu, Feb 23, 2017 at 03:52:46PM -0800, David Fetter wrote: > On Thu, Feb 23, 2017 at 05:55:37PM -0500, Bruce Momjian wrote: > > On Thu, Feb 23, 2017 at 04:08:58PM -0500, Tom Lane wrote: > > > Bruce Momjian writes: > > > > Is there a reason we don't support base64 as a bytea_output output > > >

Re: [HACKERS] Hash Functions

On Wed, Aug 16, 2017 at 05:58:41PM -0400, Tom Lane wrote: > Robert Haas writes: > > Attached is a quick sketch of how this could perhaps be done (ignoring > > for the moment the relatively-boring opclass pushups). It introduces > > a new function hash_any_extended which differs from hash_any() in

Re: [HACKERS] CTE inlining

On Wed, May 03, 2017 at 02:33:05PM -0300, Alvaro Herrera wrote: > David Fetter wrote: > > > When we add a "temporary" GUC, we're taking on a gigantic burden. > > Either we support it forever somehow, or we put it on a deprecation > > schedule immediately and expect to be answering questions about

Re: [HACKERS] Hash Functions

On Fri, May 12, 2017 at 02:23:14PM -0400, Robert Haas wrote: > > What about integers? I think we're already assuming two's-complement > arithmetic, which I think means that the only problem with making the > hash values portable for integers is big-endian vs. little-endian. > That's sounds solvea

Re: [HACKERS] WIP: Data at rest encryption

On Wed, Jun 14, 2017 at 12:04:26PM +0300, Aleksander Alekseev wrote: > Hi Ants, > > On Tue, Jun 13, 2017 at 09:07:49AM -0400, Peter Eisentraut wrote: > > On 6/12/17 17:11, Ants Aasma wrote: > > > I'm curious if the community thinks this is a feature worth having? > > > Even considering that securi

Re: [HACKERS] Report: Linux huge pages with Postgres

On Sat, Nov 27, 2010 at 02:27:12PM -0500, Tom Lane wrote: > We've gotten a few inquiries about whether Postgres can use "huge pages" > under Linux. In principle that should be more efficient for large shmem > regions, since fewer TLB entries are needed to support the address > space. I spent a bi

Re: [HACKERS] Suggesting a libpq addition

On Mon, Dec 06, 2010 at 10:14:55AM -0500, Merlin Moncure wrote: > On Mon, Dec 6, 2010 at 9:55 AM, Marc Balmer wrote: > > Am 06.12.10 15:37, schrieb Merlin Moncure: > >> On Sun, Dec 5, 2010 at 5:10 AM, Magnus Hagander > >> wrote: > >>> On Sun, Dec 5, 2010 at 10:22, Marc Balmer wrote: > I am

Re: [HACKERS] Why percent_rank is so slower than rank?

On Thu, Dec 09, 2010 at 05:18:57PM -0500, Tom Lane wrote: > I wrote: > > We're throwing away one tuple at a time as we advance forward through > > the tuplestore, and moving 10+ tuple pointers each time. Ugh. > > This code was all right when written, because (IIRC) the mergejoin > > case was a

Re: [HACKERS] Default mode for shutdown

On Wed, Dec 15, 2010 at 09:39:12AM -0500, Tom Lane wrote: > Magnus Hagander writes: > > I'm sure this has been up before, but hey, let's take it another round. > > Why don't we change the default shutdown mode for pg_ctl from "smart" > > to "fast"? I've never come across a single usecase where "sm

Re: [HACKERS] Extensions, patch v20 (bitrot fixes)

On Mon, Dec 20, 2010 at 02:10:39PM -0500, Tom Lane wrote: > David Fetter writes: > > On Mon, Dec 20, 2010 at 08:01:42PM +0100, Martijn van Oosterhout wrote: > >> I think you mean Unicode is not a superset of all character sets. I've > >> heard this before but never found what's missing. [citation

Re: [HACKERS] Extensions, patch v20 (bitrot fixes)

On Mon, Dec 20, 2010 at 03:08:48PM -0500, Tom Lane wrote: > Kenneth Marshall writes: > > On Mon, Dec 20, 2010 at 02:10:39PM -0500, Tom Lane wrote: > >> [citation needed]? Exactly what characters are missing, and why would > >> the Unicode people have chosen to lea

Re: [HACKERS] [FeatureRequest] Base Convert Function

On Tue, Dec 21, 2010 at 11:28:17PM +0200, Pavel Golub wrote: > Hello, Pavel. > > You wrote: > > PS> Hello > > PS> Dne 21. prosince 2010 21:11 Tom Mudru??ka > napsal(a): > >> > >> Thx for you answers :-) > >> Well... i know that i can write my own plugin and i am familiar with C so > >> thi

Re: [HACKERS] Why is sorting on two columns so slower than sorting on one column?

On Thu, Dec 23, 2010 at 02:33:12AM -0500, Jie Li wrote: > Hi, > > Here is the test table, > > postgres=# \d big_wf > Table "public.big_wf" > Column | Type | Modifiers > +-+--- > age| integer | > id | integer | > > postgres=# \dt+ big_wf >

Re: [HACKERS] Why is sorting on two columns so slower than sortingon one column?

On Thu, Dec 23, 2010 at 10:19:46PM +0800, Li Jie wrote: > Hi Ken, > > Thanks for your tips! Yes it is the case, and I run another query sorting on > the second column whose values are random. > > postgres=# explain analyze select * from big_wf order by id; >

Re: [HACKERS] Why is sorting on two columns so slower thansortingon one column?

On Thu, Dec 23, 2010 at 10:42:26PM +0800, Li Jie wrote: > - Original Message - > From: "Kenneth Marshall" > To: "Li Jie" > Cc: "pgsql-hackers" > Sent: Thursday, December 23, 2010 10:30 PM > Subject: Re: [HACKERS] Why is sorting on t

Re: [HACKERS] Avoiding bad prepared-statement plans.

On Thu, Feb 18, 2010 at 08:31:05PM -0600, David Christensen wrote: > > On Feb 18, 2010, at 2:19 PM, Pierre C wrote: > >> >>> What about catching the error in the application and INSERT'ing into the >>> current preprepare.relation table? The aim would be to do that in dev or >>> in pre-prod environm

Re: [HACKERS] pgbouncer + psql 9.0a4

Hi Garick, Add an ignore_startup_parameters to your pgbouncer.ini file with application_name. Cheers, Ken On Fri, Feb 26, 2010 at 11:26:23AM -0500, Garick Hamlin wrote: > I was just trying out 9.0a4 and I noticed. That I can't connect to > pgbouncer with psql from 9.0a4 as a result of the set

Re: [HACKERS] Can we still trust plperl?

On Thu, Mar 11, 2010 at 09:31:46AM -0500, Andrew Dunstan wrote: > > Last night my attention was drawn to this: > > > > I'm wondering if we can reasonably continue to support plperl as a trusted > lang

[HACKERS] construct_array() use with PQexec with binary data

Dear PostgreSQL development community, I am working on adapting a regular PQexec() call to use binary transmission of the parameters. One of the parameters is an array of BIGINT. Looking in include/utils/array.h, it appears that construct_array() will do exactly what I need to get an array to pass

Re: [HACKERS] Generating Lots of PKs with nextval(): A Feature Proposal

Hi Peter, All you need to do is define your own sequence with an increment of 500. Look at: http://www.postgresql.org/docs/8.4/static/sql-createsequence.html Regards, Ken On Fri, May 14, 2010 at 02:56:18PM -0400, Peter Crabtree wrote: > Recently, in preparation for migrating an application to p

Re: [HACKERS] BYTEA / DBD::Pg change in 9.0 beta

On Tue, May 18, 2010 at 03:26:17PM -0600, Alex Hunsaker wrote: > On Tue, May 18, 2010 at 15:20, Jesper Krogh wrote: > > On 2010-05-18 23:12, Alex Hunsaker wrote: > >> > >> set bytea_output 'escape'; > > > > That was it. Knowing what the problem was I had no problem finding it in the > > release no

Re: [HACKERS] BYTEA / DBD::Pg change in 9.0 beta

On Wed, May 19, 2010 at 10:54:01AM -0400, Robert Haas wrote: > On Wed, May 19, 2010 at 10:17 AM, Stefan Kaltenbrunner > wrote: > > On 05/19/2010 08:13 AM, Tom Lane wrote: > >> Bernd Helmle writes: > >>> --On 18. Mai 2010 23:20:26 +0200 Jesper Krogh wrote: > May I ask whats the reason is for

Re: [HACKERS] Another Modest Proposal: Platforms

On Wed, Sep 22, 2010 at 01:17:54PM -0700, David Fetter wrote: > Folks, > > While it's interesting to note, in an historical sense, that a > platform most recently updated when 1999 was still in the future, I > think it's time we did a little pruning. > > We can start by supporting only platforms

Re: [HACKERS] Why is time with timezone 12 bytes?

On Wed, Sep 22, 2010 at 10:54:53PM +0100, Thom Brown wrote: > On 22 September 2010 22:01, Josh Berkus wrote: > > All, > > > > I was just checking on our year-2027 compliance, and happened to notice > > that time with time zone takes up 12 bytes. ?This seems peculiar, given > > that timestamp with

Re: [HACKERS] Why do we have a database specification in .pgpass?

On Thu, Oct 14, 2010 at 06:09:35AM +0200, Dennis Bj??rklund wrote: > > We have a database specification in .pgpass: > > > > hostname:port:database:username:password > > > > What is the purpose of 'database' since username/password combinations > > are global, not per database? I would like to

Re: [HACKERS] PostgreSQL and HugePage

On Wed, Oct 20, 2010 at 10:10:00AM -0400, Tom Lane wrote: > Robert Haas writes: > > On Tue, Oct 19, 2010 at 11:30 PM, daveg wrote: > >> On Wed, Oct 20, 2010 at 04:08:37PM +1300, Mark Kirkwood wrote: > >>> Heh - provided you specify > >>> SHM_HUGETLB > >>> in the relevant call that is :-) > > >>

Re: [HACKERS] Hash support for arrays

On Tue, Nov 02, 2010 at 04:42:19PM -0400, Tom Lane wrote: > Robert Haas writes: > > On Tue, Nov 2, 2010 at 11:21 AM, Tom Lane wrote: > >> Really? ?I think "I don't understand when this fails" isn't obviously > >> better than being able to predict when it fails ... > > > Isn't that the whole poin

Re: [HACKERS] Hash support for arrays

On Wed, Nov 03, 2010 at 10:24:16AM +0100, Nicolas Barbier wrote: > 2010/11/2 Kenneth Marshall : > > > Given that our hash implimentation mixes the input data well (It does. > > I tested it.) then a simple rotate-and-xor method is all that should > > be needed to ma

Re: [HACKERS] Hash support for arrays

On Thu, Nov 04, 2010 at 10:00:40AM +, Dean Rasheed wrote: > On 3 November 2010 09:24, Nicolas Barbier wrote: > > 2010/11/2 Kenneth Marshall : > > > >> Given that our hash implimentation mixes the input data well (It does. > >> I tested it.) then a simple

Re: [HACKERS] Protecting against unexpected zero-pages: proposal

On Tue, Nov 09, 2010 at 02:05:57PM -0500, Robert Haas wrote: > On Tue, Nov 9, 2010 at 12:31 PM, Greg Stark wrote: > > On Tue, Nov 9, 2010 at 5:06 PM, Aidan Van Dyk wrote: > >> So, for getting checksums, we have to offer up a few things: > >> 1) zero-copy writes, we need to buffer the write to get

Re: [HACKERS] unlogged tables

On Wed, Nov 17, 2010 at 02:16:06PM -0500, Tom Lane wrote: > Robert Haas writes: > > On Wed, Nov 17, 2010 at 1:46 PM, Tom Lane wrote: > >> Robert is probably going to object that he wanted to prevent any > >> fsyncing for unlogged tables, but the discussion over in pgsql-general > >> is crystal cl

Re: EOL for 8.2 (was Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers)

On Thu, Apr 21, 2011 at 06:04:09PM +0100, Dave Page wrote: > On Thu, Apr 21, 2011 at 5:59 PM, Tom Lane wrote: > > [ man, this thread has totally outlived its title, could we change that? > > ?I'll start with this subtopic ] > > > > Robert Haas writes: > >> In fact, I've been wondering if we shoul

Re: [HACKERS] LIKE, CHAR(), and trailing spaces

On Wed, Feb 02, 2011 at 07:48:38PM -0500, Bruce Momjian wrote: > Brendan Jurd wrote: > > On 3 February 2011 10:54, Bruce Momjian wrote: > > > It seems LIKE is considering the trailing CHAR(10) field spaces as > > > significant, even though our documentations says: > > > > > -- snip -- > > > > > >

Re: [HACKERS] Correctly producing array literals for prepared statements

On Wed, Feb 23, 2011 at 09:34:06AM -0600, Merlin Moncure wrote: > On Tue, Feb 22, 2011 at 10:16 PM, Peter Geoghegan > wrote: > > I'm investigating the possibility of developing a utility function for > > our C++ client library, libpqxx, that produces array literals that can > > be used in prepared

Re: [HACKERS] Correctly producing array literals for prepared statements

On Wed, Feb 23, 2011 at 03:34:45PM -0500, Andrew Chernow wrote: > On 2/23/2011 3:06 PM, Peter Geoghegan wrote: >> On 23 February 2011 15:34, Merlin Moncure wrote: >>> You can send nested arrays safely. You just have to be very formal >>> about escaping *everything* both as you get it and as it go

Re: [HACKERS] beta3 & the open items list

On Sun, Jun 20, 2010 at 03:01:04PM -0500, Kevin Grittner wrote: > "Joshua D. Drake" wrote: > > > Can someone tell me what we are going to do about firewalls that > > impose their own rules outside of the control of the DBA? > > Has anyone actually seen a firewall configured for something so >

Re: [HACKERS] pg_basebackup wish list

On Tue, Jul 12, 2016 at 11:06:39AM -0700, Jeff Janes wrote: > On Tue, Jul 12, 2016 at 10:48 AM, Peter Eisentraut > wrote: > > On 7/12/16 12:53 PM, Jeff Janes wrote: > >> The --help message for pg_basebackup says: > >> > >> -Z, --compress=0-9 compress tar output with given compression level > >

[HACKERS] Hash index with larger hashes

Hello Developers, I have been following the recent discussions on increasing the size of the hash function used in Postgres and the work to provide WAL and performance improvements for hash indexes. I know it was mentioned when we moved to the new hashing functions, but the existing functions do

Re: [HACKERS] Reduction in WAL for UPDATEs

On Wed, Mar 28, 2007 at 08:07:14AM +0100, Simon Riggs wrote: > On Tue, 2007-03-27 at 20:48 -0400, Tom Lane wrote: > > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > > It seems possible to reduce overall WAL volume by roughly 25% on common > > > workloads by optimising the way UPDATE statements gener

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

On Wed, Apr 25, 2007 at 10:00:16AM +0200, Zeugswetter Andreas ADI SD wrote: > > > > 1) To deal with partial/inconsisitent write to the data file at > crash > > > recovery, we need full page writes at the first modification to > pages > > > after each checkpoint. It consumes much of WAL space. >

Re: [HACKERS] Bugtraq: Having Fun With PostgreSQL

On Sat, Jun 23, 2007 at 12:02:43PM -0400, Tom Lane wrote: > Magnus Hagander <[EMAIL PROTECTED]> writes: > >> One thing I've thought about doing is to remove the default in initdb > >> completely and *force* the user to choose auth type. Packagers can > >> then just use that to set ident or whatever

Re: [HACKERS] todo: Hash index creation

On Wed, Jun 27, 2007 at 08:36:54PM -0400, Tom Lane wrote: > Heikki Linnakangas <[EMAIL PROTECTED]> writes: > > [EMAIL PROTECTED] wrote: > >> Is anyone currently working on this TODO item? > >> "During index creation, pre-sort the tuples to improve build speed" > > > If you want to work on hash i

Re: [HACKERS] todo: Hash index creation

On Thu, Jul 05, 2007 at 12:26:45PM +0100, Heikki Linnakangas wrote: > Kenneth Marshall wrote: > >I definitely agree with Tom's assessment. If we cannot need to make the > >hash index as performant as it is in theory, none of the other refinements > >are worth it. You woul

Re: [HACKERS] Query plan and execution time of a query

Mickael, Not knowing your query, it sounds like your method is working correctly. It is quite normal to have the initial query take longer than subsequent queries. This is a cache effect and is what databases, in general, strive for performance-wise. I suspect that the second time you run the quer

Re: [HACKERS] tsearch2 in PostgreSQL 8.3?

On Tue, Aug 14, 2007 at 03:15:44PM -0400, Alvaro Herrera wrote: > Bruce Momjian wrote: > > A lot of work has been done to try to get /contrib/tsearch2 into the > > core backend for 8.3, but we have hit a roadblock in how to handle > > multiple text search configurations. (FYI, the documentation is

[HACKERS] Hash index todo list item

Dear PostgreSQL Hackers: After following the hackers mailing list for quite a while, I am going to start investigating what will need to be done to improve hash index performance. Below are the pieces of this project that I am currently considering: 1. Characterize the current hash index implemen

Re: [HACKERS] Hash index todo list item

On Sun, Sep 02, 2007 at 10:41:22PM -0400, Tom Lane wrote: > Kenneth Marshall <[EMAIL PROTECTED]> writes: > > ... This is the rough plan. Does anyone see anything critical that > > is missing at this point? > > Sounds pretty good. Let me brain-dump one item on you: one

Re: [HACKERS] Hash index todo list item

On Mon, Sep 03, 2007 at 10:33:54AM +0100, Simon Riggs wrote: > > > > This is the rough plan. Does anyone see anything critical that > > is missing at this point? Please send me any suggestions for test > > data and various performance test ideas, since I will be working > > on that first. > > Sou

Re: [HACKERS] Hash index todo list item

On Mon, Sep 03, 2007 at 05:20:34PM -0700, Ben Tilly wrote: > > That raises a very random thought. One of the nicer features of > Oracle is the ability to have function-based indexes. So you could > index, say, trim(lower(person.name)). There are a *lot* of practical > situations where that come

Re: [HACKERS] Hash index todo list item

On Sun, Sep 02, 2007 at 01:04:04PM -0500, Kenneth Marshall wrote: > Dear PostgreSQL Hackers: > > After following the hackers mailing list for quite a while, > I am going to start investigating what will need to be done > to improve hash index performance. Below are the pieces of

Re: [HACKERS] Hash index todo list item

On Thu, Sep 06, 2007 at 11:53:45AM -0400, Mark Mielke wrote: > Hannu Krosing wrote: One approahc is not to mix hashes, but to partition the hash, so that each column gets its N bits in the hash. >>> How does that help? You still need all the keys to find out which >>> bucket to lo

Re: [HACKERS] Hash index todo list item

On Fri, Sep 07, 2007 at 09:50:07AM -0400, Mark Mielke wrote: > Kenneth Marshall wrote: >> On Thu, Sep 06, 2007 at 11:56:25PM -0700, Neil Conway wrote: >> >>> You might find this patch useful: >>> >>> http://archives.postgre

Re: [HACKERS] Hash index todo list item

On Fri, Sep 07, 2007 at 12:55:37PM +0100, Heikki Linnakangas wrote: > Neil Conway wrote: > > You might find this patch useful: > > > > http://archives.postgresql.org/pgsql-patches/2005-05/msg00164.php > > Oh, I had forgot about that. > > > It implements the "just store the hash in the index"

Re: [HACKERS] Hash index todo list item

On Thu, Sep 06, 2007 at 11:56:25PM -0700, Neil Conway wrote: > On Sun, 2007-02-09 at 13:04 -0500, Kenneth Marshall wrote: > > 2. Evaluate the performance of different hash index implementations > >and/or changes to the current implementation. My current plan is >

Re: [HACKERS] Hash index todo list item

On Thu, Sep 06, 2007 at 11:56:25PM -0700, Neil Conway wrote: > On Sun, 2007-02-09 at 13:04 -0500, Kenneth Marshall wrote: > > 2. Evaluate the performance of different hash index implementations > >and/or changes to the current implementation. My current plan is >

Re: [HACKERS] Hash index todo list item

On Fri, Sep 07, 2007 at 10:36:41AM -0400, Brian Hurt wrote: > Kenneth Marshall wrote: > >> I understand that a hash value is a many-to-one mapping. That is the >> point of the flag in the index. The flag means that there is only one >> item in the heap corresponding to

Re: [HACKERS] Hash index todo list item

On Fri, Sep 07, 2007 at 11:08:13AM -0400, Brian Hurt wrote: > Kenneth Marshall wrote: > >>>> >>> How likely is it that you will get a hash collision, two strings that are >>> different that will hash to the same value? To avoid this requires a >&g

Re: [HACKERS] Hash index todo list item

On Fri, Sep 07, 2007 at 10:30:30AM -0400, Mark Mielke wrote: > Kenneth Marshall wrote: >> I understand that a hash value is a many-to-one mapping. That is the >> point of the flag in the index. The flag means that there is only one >> item in the heap corresponding to that has

Re: [HACKERS] Hash index todo list item

On Fri, Sep 07, 2007 at 10:36:41AM -0400, Brian Hurt wrote: > Kenneth Marshall wrote: > >> I understand that a hash value is a many-to-one mapping. That is the >> point of the flag in the index. The flag means that there is only one >> item in the heap corresponding to

Re: [HACKERS] Hash index todo list item

On Sat, Sep 08, 2007 at 05:14:09PM -0400, Mark Mielke wrote: > Kenneth Marshall wrote: >> Continuing this train of thought While it would make sense for larger >> keys to store the hash in the index, if the key is smaller, particularly >> if it is of fixed size, it woul

Re: [HACKERS] Hash index todo list item

On Sun, Sep 02, 2007 at 10:41:22PM -0400, Tom Lane wrote: > Kenneth Marshall <[EMAIL PROTECTED]> writes: > > ... This is the rough plan. Does anyone see anything critical that > > is missing at this point? > > Sounds pretty good. Let me brain-dump one item on you: one

Re: [HACKERS] Hash index todo list item

On Thu, Sep 20, 2007 at 05:12:45PM -0700, Tom Raney wrote: > We are pleased to announce an upcoming patch to the hash index code > which improves build time and index size, based on this item in the > TODO list: > During index creation, pre-sort the tuples to improve build speed > http://archives.p

Re: [HACKERS] Hash index todo list item

On Tue, Sep 25, 2007 at 03:35:47PM +0100, Gregory Stark wrote: > "Kenneth Marshall" <[EMAIL PROTECTED]> writes: > > > On Thu, Sep 20, 2007 at 05:12:45PM -0700, Tom Raney wrote: > > > >> Using our implementation, build times and index sizes are >

Re: [HACKERS] Hash index todo list item

On Wed, Sep 05, 2007 at 03:07:03PM -0500, Kenneth Marshall wrote: > On Sun, Sep 02, 2007 at 01:04:04PM -0500, Kenneth Marshall wrote: > > Dear PostgreSQL Hackers: > > > > After following the hackers mailing list for quite a while, > > I am going to start investigatin

Re: [HACKERS] Hash index todo list item

Tom, That is great. I am looking forward to your patch. After the issues that you needed to address, I think that it would be reasonable to add a few more user settings for the hash index. Fill-factor is too course a knob. The others that I have been considering are: maxtuples - Not really the ma

Re: [HACKERS] Hash index todo list item

Tom, Thank you for the update. I am currently working on updating the patch Neil Conway sent in against 8.0-ish that stores only the hash in the index and locates the entries within the page using a binary search. Then I will fold in your recent update. On Sun, Oct 21, 2007 at 01:13:48PM -0700, T

Re: [HACKERS] Hash index todo list item

On Thu, Sep 13, 2007 at 02:02:14PM -0700, Neil Conway wrote: > On Fri, 2007-09-07 at 08:29 -0500, Kenneth Marshall wrote: > > This is a great starting point. I would appreciate it if you have the > > time and could make it apply cleanly to HEAD. > > Just to give you an update

Re: [HACKERS] Hash index todo list item

On Thu, Sep 13, 2007 at 02:02:14PM -0700, Neil Conway wrote: > On Fri, 2007-09-07 at 08:29 -0500, Kenneth Marshall wrote: > > This is a great starting point. I would appreciate it if you have the > > time and could make it apply cleanly to HEAD. > > Just to give you an update

Re: [HACKERS] Index trouble with 8.3b4

On Mon, Jan 14, 2008 at 10:10:54PM -0500, Tom Lane wrote: > Gregory Stark <[EMAIL PROTECTED]> writes: > > "Tom Lane" <[EMAIL PROTECTED]> writes: > >> I went through all of the heap_beginscan calls in the code last night. > >> pgstattuple was broken but AFAICS none of the other callers care about >

Re: [HACKERS] WAL logging of hash indexes

On Tue, Jan 15, 2008 at 07:18:17PM -0800, John Smith wrote: > http://archives.postgresql.org/pgsql-hackers/2007-10/msg01468.php, > > "... I very much want to encourage authors of new Resource Managers and it > looks like we may be getting at least 3 new RMs that produce WAL > records: hash indexes

Re: [HACKERS] [PATCHES] Proposed patch: synchronized_scanning GUCvariable

On Tue, Jan 29, 2008 at 10:40:40AM +0100, Zeugswetter Andreas ADI SD wrote: > > > It's a good point that we don't want pg_dump to screw up the cluster > > order, but that's the only use case I've seen this far for disabling > > sync scans. Even that wouldn't matter much if our estimate for > >

Re: [HACKERS] [PATCHES] Proposed patch: synchronized_scanning GUCvariable

On Wed, Jan 30, 2008 at 10:56:47AM +0100, Zeugswetter Andreas ADI SD wrote: > > > > The plural seems better to me; there's no such thing as a solitary > > > synchronized scan, no? The whole point of the feature is to affect > > > the behavior of multiple scans. > > > > +1. The plural is importan

Re: [HACKERS] Reverse key index

Pretty neat. It may be a possible alternative to the use of the hash index in some applications. Cheers, Ken On Sun, Feb 03, 2008 at 07:13:23PM -0800, Gurjeet Singh wrote: > Hi All, > > I have wanted to create a reverse key index for some time now, and it > seems that an evening of reading a

Re: [HACKERS] [PATCH]-hash index improving

On Thu, Jul 17, 2008 at 12:42:39PM -0400, Alvaro Herrera wrote: > Xiao Meng escribi?: > > The patch store hash code only in the index tuple. > > It based on Neil Conway's patch with an old version of PostgreSQL. > > It passes the regression test but I didn't test the performance yet. > > Anyone in

Re: [HACKERS] [PATCH]-hash index improving

On Thu, Jul 17, 2008 at 02:00:07PM -0400, Jonah H. Harris wrote: > On Thu, Jul 17, 2008 at 1:54 PM, Kenneth Marshall <[EMAIL PROTECTED]> wrote: > >> I think having the HASHVALUE_ONLY define is not a good idea -- it just > >> makes the patch harder to read. I suggest

Re: [HACKERS] [PATCH]-hash index improving

On Thu, Jul 17, 2008 at 04:24:28PM -0400, Jonah H. Harris wrote: > On Thu, Jul 17, 2008 at 5:26 AM, Xiao Meng <[EMAIL PROTECTED]> wrote: > > The patch store hash code only in the index tuple. > > It based on Neil Conway's patch with an old version of PostgreSQL. > > It passes the regression test b

Re: [HACKERS] [PATCH]-hash index improving

I just ran my original 16M word test case against the patched version, and like Tom noted below, the tuples per bucket calculation is wrong which results in identical index sizes for both the original version and the hash-value-only version. > I suppose that the main point of #1 is to reduce index

Re: [HACKERS] [PATCH]-hash index improving

FYI, I just patched the fill-factor calculation and re-ran my test. The index size dropped from 513M to 43M which is the same disk footprint as the corresponding btree index. Have a nice weekend. Ken On Fri, Jul 18, 2008 at 12:23:14PM -0500, Kenneth Marshall wrote: > I just ran my original

Re: [HACKERS] [PATCH]-hash index improving

On Tue, Jul 22, 2008 at 08:36:34PM -0700, Dann Corbit wrote: > > -Original Message- > > From: [EMAIL PROTECTED] [mailto:pgsql-hackers- > > [EMAIL PROTECTED] On Behalf Of Xiao Meng > > Sent: Tuesday, July 22, 2008 7:57 PM > > To: Simon Riggs > > Cc: pgsql-hackers@postgresql.org > > Subject:

Re: [HACKERS] [RFC] Unsigned integer support.

Hi Ryan, I agree, I have had applications use uint types to avoid using a larger data type. I have actually had to patch an application developed for MySQL uint8 to signed int8 on PostgreSQL. In that case, the only operations that were performed where assignment and lookup. If we need to use the n

Re: [HACKERS] migrate data 6.5.3 -> 8.3.1

When upgrading, you use the pg_dump from the new version to dump the old database. Then it can take care of incidental changes during the process. I think that the mailing list archives have articles on upgrading from v6.5. I do not think that you can go straight from v6.5 to v8.3. You will almost

Re: [HACKERS] Patch: plan invalidation vs stored procedures

On Wed, Aug 20, 2008 at 09:16:56AM -0400, Andrew Sullivan wrote: > On Wed, Aug 20, 2008 at 03:12:43PM +0300, Asko Oja wrote: > > > - If there is nothing that can be done in 8.3 at least warning should be > > added into the documentation. It will be just one more don't in our long > > list don'ts

Re: [HACKERS] Need more reviewers!

On Thu, Sep 04, 2008 at 02:01:18PM -0400, Tom Lane wrote: > "Jonah H. Harris" <[EMAIL PROTECTED]> writes: > > I'll push forward on reviewing and testing Xiao's hash index > > improvements for inclusion into core. Though, someone will still need > > to review my stuff. > > I think what the hash in

Re: [HACKERS] 8.3 vs HEAD difference in Interval output?

On Thu, Oct 09, 2008 at 11:50:17AM -0700, Ron Mayer wrote: > Kevin Grittner wrote: > "Kevin Grittner" <[EMAIL PROTECTED]> wrote: >> Even more surprising is the behavior for interval(1) here: >> [ some context with nonsurprising examples removed ...] >> ccdev=# select '1 year 2 mons 3 days

Re: [HACKERS] 8.3 vs HEAD difference in Interval output?

On Thu, Oct 09, 2008 at 02:47:24PM -0500, Kevin Grittner wrote: > >>> Kenneth Marshall <[EMAIL PROTECTED]> wrote: > > >>> Even more surprising is the behavior for interval(1) here: > >>> [ some context with nonsurprising examples removed ...] >

Re: [HACKERS] CLUSTER, REINDEX, VACUUM in "read only" transaction?

On Fri, Oct 10, 2008 at 09:41:39AM -0400, Tom Lane wrote: > So I was looking for other omissions in utility.c, and I noticed that > check_xact_readonly() doesn't reject CLUSTER, REINDEX, or VACUUM. > Now the notion of "read only" that we're trying to enforce is pretty > weak (I think it's effective

Re: [HACKERS] minimal update

On Wed, Oct 22, 2008 at 06:05:26PM -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > >> On Wed, Oct 22, 2008 at 3:24 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > >>> "Minimal" really fails to convey the point here IMHO. How about > >>> something like "suppress_no_op_updates_trigger"?

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

On Tue, Oct 28, 2008 at 09:28:38AM -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Sun, 2008-10-26 at 21:49 -0400, Tom Lane wrote: > >> So I'm concluding that we can easily afford to switch to > >> tuplestore-always operation, especially if we are willing to put any > >> eff

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

generalized CRC. Also, unless you can inline your CRC the Jenkins lookup3 is 5n+20 where CRC is 9n+3. Regards, Ken > On Tue, 4 Nov 2008, Kenneth Marshall wrote: > >> Sorry about the delay for this update to the new hash >> index implementation. I was trying to get the WAL loggi

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

Oleg, Here is a little more information on the use of CRC32 as a hash function, with some warning caveats: http://home.comcast.net/~bretm/hash/8.html Regards, Ken On Tue, Nov 04, 2008 at 03:15:44PM -0600, Kenneth Marshall wrote: > On Tue, Nov 04, 2008 at 11:32:47PM +0300, Oleg Bartunov wr

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

Sorry about the delay for this update to the new hash index implementation. I was trying to get the WAL logging in place and forgot to post the actual patch. The WAL for hash indexes will need to wait for 8.5, but I did want to add back in the piece of the Bob Jenkins 2006 hash function that was st

[HACKERS] Tests citext casts

I installed and ran the citext tests both with and without the patch and had failures both times. The patch applied cleanly and the "make;make install" completed without errors. I have attached the two regression.diffs files, one without the patch applied and the other with the patch. Regards, Ken

Re: [RRR] [HACKERS] Tests citext casts

On Wed, Nov 05, 2008 at 09:04:04AM -0800, David E. Wheeler wrote: > On Nov 5, 2008, at 6:40 AM, Kenneth Marshall wrote: > >> I installed and ran the citext tests both with and without >> the patch and had failures both times. The patch applied >> cleanly and the "

Re: [RRR] [HACKERS] Tests citext casts

On Fri, Nov 07, 2008 at 10:15:17AM -0800, David E. Wheeler wrote: > On Nov 5, 2008, at 12:34 PM, Kenneth Marshall wrote: > >> I am using the anonymous CVS repository, it returns the following >> information in pg_catalog.pg_settings: > > What is lc_collate set to? > &g

Re: [HACKERS] [RRR] Tests citext casts - reviewed

The patch for the citext tests applied to module cleanly and the patched files resulted in a clean "make installcheck" run for the citext module. My previous problem was the result of not testing with a C locale database. This patch is ready to be applied. Regards, Ken Marshall -- Sent via pgsql

Re: [HACKERS] WIP parallel restore patch

Okay, I have had a chance to run some timing benchmarks. Here are my results for the parallel pg_restore patch: Ken -- Server settings: max_connections = 100 # (change requires restart) shared_buffers = 256MB

  1   2   3   >