Re: [HACKERS] Disabling trust/ident authentication configure option

2015-05-06 Thread Magnus Hagander
On May 7, 2015 12:41 AM, "Heikki Linnakangas" wrote: > > On 05/07/2015 01:32 AM, Jim Nasby wrote: >> >> On 5/6/15 12:56 PM, Peter Eisentraut wrote: I think this is a sufficiently general requirement to warrant including > > an option to disable this, as most hardening guides I ha

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE/IGNORE 4.0

2015-05-06 Thread Andreas Karlsson
On 05/06/2015 09:51 PM, Heikki Linnakangas wrote: So, yes, DO NOTHING does very little - and that is its appeal. Supporting this behavior does not short change those who actually care about the existing tuple sticking around for the duration of their transaction - they have a way of doing that. I

Re: [HACKERS] Disabling trust/ident authentication configure option

2015-05-06 Thread Heikki Linnakangas
On 05/07/2015 01:32 AM, Jim Nasby wrote: On 5/6/15 12:56 PM, Peter Eisentraut wrote: I think this is a sufficiently general requirement to warrant including an option to disable this, as most hardening guides I have seen for PostgreSQL unconditionally require to disable trust authentication and

Re: [HACKERS] Disabling trust/ident authentication configure option

2015-05-06 Thread Josh Berkus
On 05/06/2015 02:13 PM, Tom Lane wrote: > Andrew Dunstan writes: >> (Personally I think there's a very good case for completely ripping out >> RFC1413 ident auth. I've not seen it used in a great long while, and >> it's always been a security risk.) > > FWIW, I agree with that --- or at least m

Re: [HACKERS] Disabling trust/ident authentication configure option

2015-05-06 Thread Jim Nasby
On 5/6/15 12:56 PM, Peter Eisentraut wrote: I think this is a sufficiently general requirement to warrant including >an option to disable this, as most hardening guides I have seen for >PostgreSQL unconditionally require to disable trust authentication and >disabling it in the code removes the ne

Re: [HACKERS] BRIN range operator class

2015-05-06 Thread Tom Lane
Alvaro Herrera writes: > Let's think together and try to find a reasonable way to get the union > procedures tested regularly. It is pretty clear that having them run > only when the race condition occurs is not acceptable; bugs go > unnoticed. [ just a drive-by comment... ] Maybe you could set

Re: [HACKERS] BRIN range operator class

2015-05-06 Thread Alvaro Herrera
I again have to refuse the notion that removing the assert-only block without any replacement is acceptable. I just spent a lot of time tracking down what turned out to be a bug in your patch 07: /* Adjust maximum, if B's max is greater than A's max */ - needsadj = FunctionCall2Coll

Re: [HACKERS] Patch for bug #12845 (GB18030 encoding)

2015-05-06 Thread Tom Lane
Robert Haas writes: > On Wed, May 6, 2015 at 11:13 AM, Alvaro Herrera > wrote: >> Maybe not, but at the very least we should consider getting it fixed in >> 9.5 rather than waiting a full development cycle. Same as in >> https://www.postgresql.org/message-id/20150428131549.ga25...@momjian.us >>

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-05-06 Thread Heikki Linnakangas
On 05/07/2015 12:01 AM, Andres Freund wrote: On 2015-05-06 23:48:18 +0300, Heikki Linnakangas wrote: I'll see about fixing that. It's not just a matter of creating another alias for the same rel, I'm afraid: "foo.t" is supposed to refer to the tuple that we attempted to insert, like it does with

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-05-06 Thread Heikki Linnakangas
On 05/07/2015 12:18 AM, Andres Freund wrote: On 2015-05-07 00:10:22 +0300, Heikki Linnakangas wrote: Right, that's the idea. Indexes are just an implementation detail - I think that's a distinction just about no user out there cares about. Unfortunately you can't create a "partial constraint

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-05-06 Thread Andres Freund
On 2015-05-07 00:10:22 +0300, Heikki Linnakangas wrote: > Right, that's the idea. Indexes are just an implementation detail - I think that's a distinction just about no user out there cares about. > Unfortunately you can't create a "partial constraint" - you'll have to > create a partial index. I

Re: [HACKERS] Disabling trust/ident authentication configure option

2015-05-06 Thread Tom Lane
Andrew Dunstan writes: > (Personally I think there's a very good case for completely ripping out > RFC1413 ident auth. I've not seen it used in a great long while, and > it's always been a security risk.) FWIW, I agree with that --- or at least making it a not-built-by-default option. Probably

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-05-06 Thread Heikki Linnakangas
On 05/06/2015 11:05 PM, Peter Geoghegan wrote: On Wed, May 6, 2015 at 7:53 AM, Andres Freund wrote: In this variant, you explicitly specify the constraint by name. I do think it's a bit sad to not be able to specify unique indexes that aren't constraints. So I'd like to have a corresponding O

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-05-06 Thread Peter Geoghegan
On Wed, May 6, 2015 at 2:01 PM, Andres Freund wrote: > How about > 6. The tablename and EXCLUDED? Possibility with the ability to specify >an AS for INSERT INTO foo AS whatever? > > From an implementation pov that'd be simple ;) That's what I wanted to do when I realized what Andres wanted to

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-05-06 Thread Andres Freund
On 2015-05-06 23:48:18 +0300, Heikki Linnakangas wrote: > I'll see about fixing that. It's not just a matter of creating another alias > for the same rel, I'm afraid: "foo.t" is supposed to refer to the tuple that > we attempted to insert, like it does without the ON CONFLICT. I'm not sure what yo

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-05-06 Thread Peter Geoghegan
On Wed, May 6, 2015 at 1:48 PM, Heikki Linnakangas wrote: >> TARGET is also very descriptive, because it situationally describes >> either the existing tuple actually present in the table, or (from a >> RETURNING clause) the final tuple present in the table post-UPDATE. >> We use the term "target"

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-05-06 Thread Heikki Linnakangas
Andres pointed out on IM that the TARGET alias is a bit crummy. In particular, adding an ON CONFLICT DO UPDATE can make a RETURNING clause invalid, because we change the alias of the target rel: create table foo (id int4 primary key, t text); This works: postgres=# insert into foo (id, t) val

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-05-06 Thread Andres Freund
On 2015-05-06 13:37:07 -0700, Peter Geoghegan wrote: > On Wed, May 6, 2015 at 1:22 PM, Andres Freund wrote: > > That it supports exclusion constraints? > > But so does just naming the index. I don't think it's significant that > exclusion operators are in pg_constraint -- you could just as easily

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-05-06 Thread Peter Geoghegan
On Wed, May 6, 2015 at 1:22 PM, Andres Freund wrote: > That it supports exclusion constraints? But so does just naming the index. I don't think it's significant that exclusion operators are in pg_constraint -- you could just as easily name the index, since that's all you ultimately end up with an

Re: [HACKERS] Disabling trust/ident authentication configure option

2015-05-06 Thread Andrew Dunstan
On 05/06/2015 04:19 PM, Robert Haas wrote: On Wed, May 6, 2015 at 3:57 PM, Andrew Dunstan wrote: I don't necessarily object to this idea, but I do think we need to ensure that we don't allow both trust and peer to be disabled (which means on Windows you would not be able to disable trust). Oth

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-05-06 Thread Andres Freund
On 2015-05-06 13:05:16 -0700, Peter Geoghegan wrote: > On Wed, May 6, 2015 at 7:53 AM, Andres Freund wrote: > >> In this variant, you explicitly specify the constraint by name. > > > > I do think it's a bit sad to not be able to specify unique indexes that > > aren't constraints. So I'd like to ha

Re: [HACKERS] Disabling trust/ident authentication configure option

2015-05-06 Thread Robert Haas
On Wed, May 6, 2015 at 3:57 PM, Andrew Dunstan wrote: > I don't necessarily object to this idea, but I do think we need to ensure > that we don't allow both trust and peer to be disabled (which means on > Windows you would not be able to disable trust). Otherwise this becomes a > footgun which wou

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE/IGNORE 4.0

2015-05-06 Thread Andres Freund
On 2015-05-06 22:51:43 +0300, Heikki Linnakangas wrote: > Yeah, I agree that DO NOTHING should not lock the rows. It might make sense > to have a DO LOCK variant, which locks the rows, although I don't > immediately see what the use case would be. If you want to do something more complicated with

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-05-06 Thread Peter Geoghegan
On Wed, May 6, 2015 at 7:53 AM, Andres Freund wrote: >> In this variant, you explicitly specify the constraint by name. > > I do think it's a bit sad to not be able to specify unique indexes that > aren't constraints. So I'd like to have a corresponding ON INDEX - which > would be trivial. Then w

Re: [HACKERS] Disabling trust/ident authentication configure option

2015-05-06 Thread Andrew Dunstan
On 05/06/2015 10:47 AM, Alvaro Herrera wrote: I don't necessarily agree with the patch as proposed. I would rather have a comma-separated list of methods, as in: --disable-auth=ident,peer which lets you choose what to disable without hardcoded choices. Due to the nature of autoconf, t

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE/IGNORE 4.0

2015-05-06 Thread Heikki Linnakangas
On 05/06/2015 10:47 PM, Peter Geoghegan wrote: On Wed, May 6, 2015 at 8:20 AM, Andres Freund wrote: On 2015-05-05 15:00:56 -0700, Peter Geoghegan wrote: Locking the row is not "nothing", though. If you want to lock the row, use an UPSERT with a tautologically false WHERE clause (like "WHERE fa

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE/IGNORE 4.0

2015-05-06 Thread Peter Geoghegan
On Wed, May 6, 2015 at 8:20 AM, Andres Freund wrote: > On 2015-05-05 15:00:56 -0700, Peter Geoghegan wrote: >> Locking the row is not "nothing", though. If you want to lock the row, >> use an UPSERT with a tautologically false WHERE clause (like "WHERE >> false"). > > That's not the same. For one

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-05-06 Thread Peter Geoghegan
On Tue, May 5, 2015 at 5:27 AM, Heikki Linnakangas wrote: > To recap, there are three variants: > > A. INSERT ... ON CONFLICT DO NOTHING > > No arbiter is specified. This means that a conflict on any unique or > exclusion constraint is not allowed (and will do nothing instead). This > variant is o

Re: [HACKERS] Manipulating complex types as non-contiguous structures in-memory

2015-05-06 Thread Pavel Stehule
2015-05-06 18:54 GMT+02:00 Tom Lane : > Pavel Stehule writes: > > 2015-05-06 15:50 GMT+02:00 Tom Lane : > >> Another way to look at it is that in this example, plpgsql's attempts to > >> force the "a" array into expanded form are a mistake: we never get any > >> benefit because array_cat() just w

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE/IGNORE 4.0

2015-05-06 Thread Peter Geoghegan
On Tue, May 5, 2015 at 10:31 AM, Andres Freund wrote: > Another thing I'm wondering about is dealing with deferrable > constraints/deferred indexes. > > a) Why does ExecCheckIndexConstraints() check for indisimmediate for >*all* indexes and not just when it's an arbiter index? That seems >

Re: [HACKERS] Disabling trust/ident authentication configure option

2015-05-06 Thread Peter Eisentraut
On 5/6/15 6:02 AM, Volker Aßmann wrote: > Well "trust" actually does not sound that dangerous in case you only > take a quick glance at the documentation - "trust PostgreSQL to do the > right thing?" Hah, we could rename it to "wideopen". > Please note that the patch does nothing by default, it j

Re: [HACKERS] is possible to upgrade from 9.2 to 9.4 with pg_upgrade

2015-05-06 Thread Jeff Janes
On Wed, May 6, 2015 at 10:26 AM, Jeff Janes wrote: > On Wed, May 6, 2015 at 6:16 AM, Pavel Stehule > wrote: > >> >> >> 2015-05-06 15:15 GMT+02:00 Alvaro Herrera : >> >>> Pavel Stehule wrote: >>> > Hi >>> > >>> > I am working on preparation the migration from 9.2 to 9.4 >>> > >>> > pg_upgrade fai

Re: [HACKERS] is possible to upgrade from 9.2 to 9.4 with pg_upgrade

2015-05-06 Thread Jeff Janes
On Wed, May 6, 2015 at 6:16 AM, Pavel Stehule wrote: > > > 2015-05-06 15:15 GMT+02:00 Alvaro Herrera : > >> Pavel Stehule wrote: >> > Hi >> > >> > I am working on preparation the migration from 9.2 to 9.4 >> > >> > pg_upgrade fails >> > >> > pg_upgrade -b /usr/lib64/pgsql/postgresql-9.2/bin -B /

Re: [HACKERS] Manipulating complex types as non-contiguous structures in-memory

2015-05-06 Thread Tom Lane
Pavel Stehule writes: > 2015-05-06 15:50 GMT+02:00 Tom Lane : >> Another way to look at it is that in this example, plpgsql's attempts to >> force the "a" array into expanded form are a mistake: we never get any >> benefit because array_cat() just wants it in flat form again, and delivers >> it in

Re: [HACKERS] Patch for bug #12845 (GB18030 encoding)

2015-05-06 Thread Robert Haas
On Wed, May 6, 2015 at 11:13 AM, Alvaro Herrera wrote: >> It's a behavior change, so I don't think we would consider a back-patch. > > Maybe not, but at the very least we should consider getting it fixed in > 9.5 rather than waiting a full development cycle. Same as in > https://www.postgresql.or

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE/IGNORE 4.0

2015-05-06 Thread Andres Freund
On 2015-05-05 15:00:56 -0700, Peter Geoghegan wrote: > Locking the row is not "nothing", though. If you want to lock the row, > use an UPSERT with a tautologically false WHERE clause (like "WHERE > false"). That's not the same. For one it "breaks" RETURNING which is a death knell, for another it's

Re: [HACKERS] Patch for bug #12845 (GB18030 encoding)

2015-05-06 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, May 6, 2015 at 10:55 AM, Alvaro Herrera > wrote: > > Robert Haas wrote: > >> On Tue, May 5, 2015 at 9:04 AM, Arjen Nienhuis > >> wrote: > >> > Can someone look at this patch. It should fix bug #12845. > >> > > >> > The current tests for conversions are very minimal.

Re: [HACKERS] Patch for bug #12845 (GB18030 encoding)

2015-05-06 Thread Robert Haas
On Wed, May 6, 2015 at 10:55 AM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Tue, May 5, 2015 at 9:04 AM, Arjen Nienhuis >> wrote: >> > Can someone look at this patch. It should fix bug #12845. >> > >> > The current tests for conversions are very minimal. I expanded them a >> > bit for this

Re: [HACKERS] Patch for bug #12845 (GB18030 encoding)

2015-05-06 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, May 5, 2015 at 9:04 AM, Arjen Nienhuis wrote: > > Can someone look at this patch. It should fix bug #12845. > > > > The current tests for conversions are very minimal. I expanded them a > > bit for this bug. > > > > I think the binary search in the .map files should be

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-05-06 Thread Andres Freund
On 2015-05-05 15:27:09 +0300, Heikki Linnakangas wrote: > I'm a bit late to the party as I haven't paid much attention to the syntax > before, but let me give some comments on this "arbiter index inference" > thingie. > > > To recap, there are three variants: > > A. INSERT ... ON CONFLICT DO NOT

Re: [HACKERS] Disabling trust/ident authentication configure option

2015-05-06 Thread Alvaro Herrera
Robert Haas wrote: > I frankly find that a bit difficult to swallow. You think that > everyone knows that bad passwords are a problem, but some people might > not realize that an authentication method called "trust" might not be > secure? Ultimately, what we offer to users is choice of a few opt

Re: [HACKERS] Manipulating complex types as non-contiguous structures in-memory

2015-05-06 Thread Pavel Stehule
2015-05-06 15:50 GMT+02:00 Tom Lane : > Pavel Stehule writes: > > Multidimensional append is slower 2x .. but it is probably corner case > > > declare a int[] := '{}'; begin for i in 1..9 loop a := a || ARRAY[[i > > ]]; end loop; raise notice '%', 'aa'; end$$ language plpgsql; > > Yeah, that'

Re: [HACKERS] Patch for bug #12845 (GB18030 encoding)

2015-05-06 Thread Robert Haas
On Tue, May 5, 2015 at 9:04 AM, Arjen Nienhuis wrote: > Can someone look at this patch. It should fix bug #12845. > > The current tests for conversions are very minimal. I expanded them a > bit for this bug. > > I think the binary search in the .map files should be removed but I > leave that for a

Re: [HACKERS] Manipulating complex types as non-contiguous structures in-memory

2015-05-06 Thread Tom Lane
Pavel Stehule writes: > Multidimensional append is slower 2x .. but it is probably corner case > declare a int[] := '{}'; begin for i in 1..9 loop a := a || ARRAY[[i > ]]; end loop; raise notice '%', 'aa'; end$$ language plpgsql; Yeah, that's array_cat(), which I've not done anything with.

Re: [HACKERS] Parallel Seq Scan

2015-05-06 Thread Robert Haas
On Wed, May 6, 2015 at 7:55 AM, Amit Kapila wrote: >>> - I believe the separation of concerns between ExecFunnel() and >>> ExecEndFunnel() is not quite right. If the scan is shut down before >>> it runs to completion (e.g. because of LIMIT), then I think we'll call >>> ExecEndFunnel() before Exec

Re: [HACKERS] is possible to upgrade from 9.2 to 9.4 with pg_upgrade

2015-05-06 Thread Pavel Stehule
2015-05-06 15:15 GMT+02:00 Alvaro Herrera : > Pavel Stehule wrote: > > Hi > > > > I am working on preparation the migration from 9.2 to 9.4 > > > > pg_upgrade fails > > > > pg_upgrade -b /usr/lib64/pgsql/postgresql-9.2/bin -B /usr/bin/ -d > > /mnt/ebs/pgsql/data -D /mnt/ebs/pgsql/data94 -k' > > P

Re: [HACKERS] Where are the detoast function called in select * from table_name case?

2015-05-06 Thread Pavel Stehule
Hi Depends on usage, but often times the detoasting is called from DatumGet* macros The values are detoasted only when it is required #define PG_GETARG_TEXT_PP(n) DatumGetTextPP(PG_GETARG_DATUM(n)) #define DatumGetTextPP(X) ((text *) PG_DETOAST_DATUM_PACKED(X)) #define PG_DETO

Re: [HACKERS] is possible to upgrade from 9.2 to 9.4 with pg_upgrade

2015-05-06 Thread Alvaro Herrera
Pavel Stehule wrote: > Hi > > I am working on preparation the migration from 9.2 to 9.4 > > pg_upgrade fails > > pg_upgrade -b /usr/lib64/pgsql/postgresql-9.2/bin -B /usr/bin/ -d > /mnt/ebs/pgsql/data -D /mnt/ebs/pgsql/data94 -k' > Performing Consistency Checks > - >

Re: [HACKERS] Where are the detoast function called in select * from table_name case?

2015-05-06 Thread Heikki Linnakangas
On 05/06/2015 04:00 PM, Rui Hai Jiang wrote: I've been reading the PostgreSQL code for weeks to figure out how TOAST works. I couldn't find where are the TOAST function called to detoast a tuple comes from a select query, for example, select * from table_name. Does anyone know this? Can you give

[HACKERS] Where are the detoast function called in select * from table_name case?

2015-05-06 Thread Rui Hai Jiang
Hello, I've been reading the PostgreSQL code for weeks to figure out how TOAST works. I couldn't find where are the TOAST function called to detoast a tuple comes from a select query, for example, select * from table_name. Does anyone know this? Can you give me some help? And any help would sa

Re: [HACKERS] Parallel Seq Scan

2015-05-06 Thread Amit Kapila
On Tue, Apr 28, 2015 at 5:37 PM, Robert Haas wrote: > > On Fri, Apr 24, 2015 at 8:32 AM, Amit Kapila wrote: >> - I believe the separation of concerns between ExecFunnel() and >> ExecEndFunnel() is not quite right. If the scan is shut down before >> it runs to completion (e.g. because of LIMIT),

[HACKERS] is possible to upgrade from 9.2 to 9.4 with pg_upgrade

2015-05-06 Thread Pavel Stehule
Hi I am working on preparation the migration from 9.2 to 9.4 pg_upgrade fails pg_upgrade -b /usr/lib64/pgsql/postgresql-9.2/bin -B /usr/bin/ -d /mnt/ebs/pgsql/data -D /mnt/ebs/pgsql/data94 -k' Performing Consistency Checks - Checking cluster versions

Re: [HACKERS] Disabling trust/ident authentication configure option

2015-05-06 Thread Volker Aßmann
On Tue, May 5, 2015 at 10:39 PM, Robert Haas wrote: > On Tue, May 5, 2015 at 8:05 AM, Volker Aßmann > wrote: > > Changing the password to something simple is immediately obvious as a > > security flaw for most people who may come across database > configurations, > > but for the TRUST mode you a

Re: [HACKERS] BRIN range operator class

2015-05-06 Thread Emre Hasegeli
>> Looking at patch 04, it seems to me that it would be better to have >> the OpcInfo struct carry the typecache struct rather than the type OID, >> so that we can avoid repeated typecache lookups in brin_deform_tuple; > > Here's the patch. Looks better to me. I will incorporate with this patch.

Re: [HACKERS] BRIN range operator class

2015-05-06 Thread Emre Hasegeli
> Can you please explain what is the purpose of patch 07? I'm not sure I > understand; are we trying to avoid having to add pg_amproc entries for > these operators and instead piggy-back on btree opclass definitions? > Not too much in love with that idea; I see that there is less tedium in > that

Re: [HACKERS] Manipulating complex types as non-contiguous structures in-memory

2015-05-06 Thread Pavel Stehule
2015-05-06 0:50 GMT+02:00 Tom Lane : > I wrote: > > Pavel Stehule writes: > >> Significant slowdown is on following test: > > >> do $$ declare a int[] := '{}'; begin for i in 1..9 loop a := a || > 10; > >> end loop; end$$ language plpgsql; > >> do $$ declare a numeric[] := '{}'; begin for i i