Re: [HACKERS] patch: make_timestamp function

2013-12-18 Thread Pavel Stehule
Hello updated patch - optional time zone is as text. postgres=# SELECT make_timetz(8, 15, 55.333) = '8:15:55.333'::timetz; ?column? -- t (1 row) postgres=# SELECT make_timetz(8, 15, 55.333, 'HKT') = '8:15:55.333 HKT'::timetz; ?column? -- t (1 row) postgres=# SELECT

Re: [HACKERS] 9.3 reference constraint regression

2013-12-18 Thread Andres Freund
Hi, On 2013-12-17 18:27:26 -0300, Alvaro Herrera wrote: Well, it would help if those cases weren't dead code. Neither heap_update nor heap_delete are ever called in the no wait case at all. Yea, that sucks, maybe we ought to change that in HEAD. But there very well might be external

Re: [HACKERS] row security roadmap proposal

2013-12-18 Thread Craig Ringer
On 12/18/2013 02:21 AM, Simon Riggs wrote: On 16 December 2013 14:36, Craig Ringer cr...@2ndquadrant.com wrote: - Decide on and implement a structure for row-security functionality its self. I'm persuaded by Robert's comments here, that whatever we expose must be significantly more usable

Re: [HACKERS] Patch: show xid and xmin in pg_stat_activity and pg_stat_replication

2013-12-18 Thread Christian Kruse
Hi, On 17/12/13 12:08, Robert Haas wrote: Please add your patch here so we don't lose track of it: https://commitfest.postgresql.org/action/commitfest_view/open Thanks. I nearly forgot that. Regards, -- Christian Kruse http://www.2ndQuadrant.com/ PostgreSQL Development,

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2013-12-18 Thread Tatsuo Ishii
Hi, I have looked into this because it's marked as ready for committer. On Tue, Nov 19, 2013 at 2:06 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 19 November 2013 09:59 Amit Kapila wrote: On Mon, Nov 18, 2013 at 8:31 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 18

Re: [HACKERS] [v9.4] row level security

2013-12-18 Thread Yeb Havinga
On 2013-12-14 05:40, Craig Ringer wrote: I find the heirachical and non-heirachical label security model used in Teradata to be extremely interesting and worthy of study. The concept is that there are heirachical label policies (think classified, unclassified, etc) or non-heirachical (mutually

Re: [HACKERS] Proposal: variant of regclass

2013-12-18 Thread Tatsuo Ishii
For the pgpool-II use case, I'm happy to follow you because pgpool-II always does a grammatical check (using raw parser) on a query first and such syntax problem will be pointed out, thus never reaches to the state where calling toregclass. One concern is, other users expect toregclass to

Re: [HACKERS] Problem with displaying wide tables in psql

2013-12-18 Thread Sergey Muraviov
Hello 2013/12/18 Sameer Thakur samthaku...@gmail.com On Wed, Dec 11, 2013 at 11:13 PM, Sergey Muraviov sergey.k.murav...@gmail.com wrote: Hi. I've improved the patch. It works in expanded mode when either format option is set to wrapped (\pset format wrapped), or we have no pager,

Re: [HACKERS] stats for network traffic WIP

2013-12-18 Thread Craig Ringer
On 12/12/2013 02:51 AM, Tom Lane wrote: The thing that I'm wondering is why the database would be the right place to be measuring it at all. If you've got a network usage problem, aggregate usage across everything on the server is probably what you need to be worried about, and PG can't tell

SQL objects UNITs (was: [HACKERS] Extension Templates S03E11)

2013-12-18 Thread Dimitri Fontaine
Simon Riggs si...@2ndquadrant.com writes: On 17 December 2013 23:42, Tom Lane t...@sss.pgh.pa.us wrote: We aim to have the simplest implementation that meets the stated need and reasonable extrapolations of that. Text in a catalog table is the simplest implementation. That is not a reason to

[HACKERS] hstore ng index for @ ?

2013-12-18 Thread Kaare Rasmussen
Hi In many ways the new hstore (and perhaps json) format looks very exciting. But will there ever be (GIN/GIST) index support for @ ? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Extension Templates S03E11

2013-12-18 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: I keep telling you this, and it keeps not sinking in. How can you say that? I've been spending a couple of years on designing and implementing and arguing for a complete feature set where dealing with modules is avoided at all costs. The problem we have now

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Heikki Linnakangas
On 12/18/2013 02:59 AM, Josh Berkus wrote: On 12/17/2013 01:42 PM, Kevin Grittner wrote: Josh Berkus j...@agliodbs.com wrote: Going back over this patch, I haven't seen any further discussion of the point Heikki raises above, which seems like a bit of a showstopper. Heikki, did you have

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Heikki Linnakangas
On 12/18/2013 02:59 AM, Josh Berkus wrote: On 12/17/2013 01:42 PM, Kevin Grittner wrote: It works fine as long as you set default_transaction_isolation = 'serializable' and never override that. :-) Of course, it sure would be nice to have a way to prohibit overrides, but that's another issue.

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Andres Freund
On 2013-12-18 13:07:51 +0200, Heikki Linnakangas wrote: Here's another idea that doesn't involve SSI: At COMMIT, take a new snapshot and check that the assertion still passes with that snapshot. Now, there's a race condition, if another transaction is committing at the same time, and

Re: [HACKERS] Example query causing param_info to be set in plain rel path

2013-12-18 Thread Ashutosh Bapat
I got an example where paths for plain rel require param_info i.e. plain rel scans require to take care of the lateral references. Here's the example from PG regression explain verbose select v.* from (int8_tbl x left join (select q1,(select coalesce(q2,0)) q2 from int8_tbl) y on x.q2 = y.q1)

Re: [HACKERS] GIN improvements part 1: additional information

2013-12-18 Thread Alexander Korotkov
On Tue, Dec 17, 2013 at 2:49 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 12/17/2013 12:22 AM, Alexander Korotkov wrote: On Mon, Dec 16, 2013 at 3:30 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 12/12/2013 06:44 PM, Alexander Korotkov wrote: When values are

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Heikki Linnakangas
On 12/18/2013 01:39 PM, Andres Freund wrote: On 2013-12-18 13:07:51 +0200, Heikki Linnakangas wrote: Here's another idea that doesn't involve SSI: At COMMIT, take a new snapshot and check that the assertion still passes with that snapshot. Now, there's a race condition, if another transaction

Re: [HACKERS] hstore ng index for @ ?

2013-12-18 Thread Alexander Korotkov
Hi! On Wed, Dec 18, 2013 at 2:35 PM, Kaare Rasmussen ka...@jasonic.dk wrote: In many ways the new hstore (and perhaps json) format looks very exciting. But will there ever be (GIN/GIST) index support for @ ? It looks not hard to do with GiST. About GIN I don't have promising ideas: likely we

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Andres Freund
On 2013-12-18 13:46:59 +0200, Heikki Linnakangas wrote: On 12/18/2013 01:39 PM, Andres Freund wrote: On 2013-12-18 13:07:51 +0200, Heikki Linnakangas wrote: Here's another idea that doesn't involve SSI: At COMMIT, take a new snapshot and check that the assertion still passes with that

Re: [HACKERS] PoC: Partial sort

2013-12-18 Thread Alexander Korotkov
On Sat, Dec 14, 2013 at 6:39 PM, Martijn van Oosterhout klep...@svana.orgwrote: On Sat, Dec 14, 2013 at 06:21:18PM +0400, Alexander Korotkov wrote: Is that actually all that beneficial when sorting with a bog standard qsort() since that doesn't generally benefit from data being

Re: [HACKERS] PoC: Partial sort

2013-12-18 Thread Alexander Korotkov
On Sat, Dec 14, 2013 at 7:04 PM, Andres Freund and...@2ndquadrant.comwrote: Hi, Limit (cost=69214.06..69214.08 rows=10 width=16) (actual time=0.097..0.099 rows=10 loops=1) - Sort (cost=69214.06..71714.06 rows=100 width=16) (actual time=0.096..0.097 rows=10 loops=1)

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Heikki Linnakangas
On 12/18/2013 01:50 PM, Andres Freund wrote: On 2013-12-18 13:46:59 +0200, Heikki Linnakangas wrote: On 12/18/2013 01:39 PM, Andres Freund wrote: On 2013-12-18 13:07:51 +0200, Heikki Linnakangas wrote: Here's another idea that doesn't involve SSI: At COMMIT, take a new snapshot and check

Re: [HACKERS] PoC: Partial sort

2013-12-18 Thread Alexander Korotkov
On Sat, Dec 14, 2013 at 11:47 PM, Andreas Karlsson andr...@proxel.sewrote: On 12/14/2013 10:59 AM, Alexander Korotkov wrote: This patch allows to use index for order-by if order-by clause and index has non-empty common prefix. So, index gives right ordering for first n order-by columns. In

Re: [HACKERS] commit fest 2013-11 final report

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 7:14 PM, Peter Eisentraut pete...@gmx.net wrote: On 12/17/13, 10:19 AM, Tom Lane wrote: Perhaps we should just move all the Needs Review and RFC patches forward to the next fest, so we don't forget about them? This was done the last few times, but it has caused some

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2013-12-18 Thread Amit Kapila
On Wed, Dec 18, 2013 at 2:05 PM, Tatsuo Ishii is...@postgresql.org wrote: Hi, I have looked into this because it's marked as ready for committer. Thank you. It looks like working as advertised. Great! However I have noticed a few minor issues. 1) validate_conf_option +/* + * Validates

[HACKERS] sepgsql: label regression test failed

2013-12-18 Thread Sergey Muraviov
Hi I've tried to test postgres 9.3.2 and 9.4devel with selinux on Fedora 20 and met with a label regression test failure. PS I've got some warning during build process. -- Best regards, Sergey Muraviov regression.out Description: Binary data regression.diffs Description: Binary data

Re: [HACKERS] sepgsql: label regression test failed

2013-12-18 Thread Kohei KaiGai
Could you show me semodule -l on your environment? I believe security policy has not been changed between F19 and F20... Thanks, 2013/12/18 Sergey Muraviov sergey.k.murav...@gmail.com: Hi I've tried to test postgres 9.3.2 and 9.4devel with selinux on Fedora 20 and met with a label regression

Re: [HACKERS] sepgsql: label regression test failed

2013-12-18 Thread Sergey Muraviov
# semodule -l | grep sepgslq sepgsql-regtest 1.07 Full list of modules is in attachment. 2013/12/18 Kohei KaiGai kai...@kaigai.gr.jp Could you show me semodule -l on your environment? I believe security policy has not been changed between F19 and F20... Thanks, 2013/12/18 Sergey Muraviov

Re: [HACKERS] stats for network traffic WIP

2013-12-18 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: On 12/12/2013 02:51 AM, Tom Lane wrote: The thing that I'm wondering is why the database would be the right place to be measuring it at all. If you've got a network usage problem, aggregate usage across everything on the server is probably what

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2013-12-18 Thread Andrew Dunstan
On 12/18/2013 03:35 AM, Tatsuo Ishii wrote: 3) initdb.c It seems the memory allocated for autoconflines[0] and autoconflines[1] by pg_strdup is never freed. (I think there's similar problem with conflines as well, though it was not introduced by the patch). Why would we care? initdb

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Andrew Dunstan
On 12/18/2013 06:00 AM, Heikki Linnakangas wrote: If you don't force everything to run in SSI mode, then you have to somehow figure out what parts do need to run in SSI mode to enforce the assertion. For example, if the assertion refers tables A and B, perhaps you can get away without

Re: SQL objects UNITs (was: [HACKERS] Extension Templates S03E11)

2013-12-18 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Here's my attempt: # Inline Extension, Extension Templates The problem with *Inline Extension* is the dump and restore policy. The contents of an extensions are not be found in a `pg_dump` script, ever. You keep coming back to this and I

Re: [HACKERS] GIN improvements part 1: additional information

2013-12-18 Thread Heikki Linnakangas
On 12/18/2013 01:45 PM, Alexander Korotkov wrote: On Tue, Dec 17, 2013 at 2:49 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 12/17/2013 12:22 AM, Alexander Korotkov wrote: 2) Storage would be easily extendable to hold additional information as well. Better compression shouldn't

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2013-12-18 Thread Tatsuo Ishii
Is there any reason for the function returns int as it always returns 0 or 1. Maybe returns bool is better? No, return type should be bool, I have changed the same in attached patch. Confirmed. 2) initdb.c + strcpy(tempautobuf, # Do not edit this file manually! \n); +

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 12:35 PM, Jeff Janes jeff.ja...@gmail.com wrote: Since it doesn't use directIO, you can't warm the PG buffers without also warming FS cache as a side effect. That is why I like 'buffer' as the default--if the data fits in shared_buffers, it warm those, otherwise it at

Re: SQL objects UNITs (was: [HACKERS] Extension Templates S03E11)

2013-12-18 Thread Alvaro Herrera
Stephen Frost escribió: * Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Basically with building `UNIT` we realise with hindsight that we failed to build a proper `EXTENSION` system, and we send that message to our users. Little difficult to draw conclusions about what out 'hindsight'

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 9:03 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: (2013/12/18 5:33), Robert Haas wrote: Sounds like it might be worth dusting the patch off again... I'd like to request you to add all_index option and usage_count option. When all_index option is selected,

Re: [HACKERS] row security roadmap proposal

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 1:27 PM, Simon Riggs si...@2ndquadrant.com wrote: Not sure I'd say required, but its certainly desirable to have updateable security barrier views in themselves. And it comes across to me as a cleaner and potentially more performant way of doing the security checks for

Re: [HACKERS] row security roadmap proposal

2013-12-18 Thread Robert Haas
On Wed, Dec 18, 2013 at 3:30 AM, Craig Ringer cr...@2ndquadrant.com wrote: In my view the proposed patch doesn't offer a significant improvement in declarative security, beyond what we can get by just adding update support to s.b. views and using search_path to control whether a user sees the

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 10:22 PM, Amit Kapila amit.kapil...@gmail.com wrote: Me either; at the very least, it's short an underscore: wal_log_hint_bits would be more readable. But how about just wal_log_hints? +1 for wal_log_hints, it sounds better. +1. -- Robert Haas EnterpriseDB:

Re: [HACKERS] 9.3 reference constraint regression

2013-12-18 Thread Alvaro Herrera
Andres Freund wrote: I have to say, it makes me really uncomfortable to take such shortcuts. In other branches we are doing liveliness checks with MultiXactIdIsRunning() et al. Why isn't that necessary here? And how likely is that this won't cause breakage somewhere down the line because

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Kevin Grittner
Heikki Linnakangas hlinnakan...@vmware.com wrote: On 12/18/2013 01:39 PM, Andres Freund wrote: On 2013-12-18 13:07:51 +0200, Heikki Linnakangas wrote: Here's another idea that doesn't involve SSI: At COMMIT, take a new snapshot and check that the assertion still passes with that snapshot.

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-18 Thread Sawada Masahiko
2013/12/14 0:14 Tom Lane t...@sss.pgh.pa.us: Heikki Linnakangas hlinnakan...@vmware.com writes: I'm not totally satisfied with the name of the GUC, wal_log_hintbits. Me either; at the very least, it's short an underscore: wal_log_hint_bits would be more readable. But how about just

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Kevin Grittner
Josh Berkus j...@agliodbs.com wrote: Serializable or not, *what* do we lock for assertions?  It's not rows.  Tables?  Which tables?  What if the assertion is an interpreted language function?  Does the SSI reference counter really take care of all of this? The simple answer is that, without

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Alvaro Herrera
Heikki Linnakangas wrote: Ah, I see. You don't need to block anyone else from modifying the table, you just need to block anyone else from committing a transaction that had modified the table. So the locks shouldn't interfere with regular table locks. A ShareUpdateExclusiveLock on the

Re: [HACKERS] RFC: Async query processing

2013-12-18 Thread Florian Weimer
On 11/04/2013 02:51 AM, Claudio Freire wrote: On Sun, Nov 3, 2013 at 3:58 PM, Florian Weimer fwei...@redhat.com wrote: I would like to add truly asynchronous query processing to libpq, enabling command pipelining. The idea is to to allow applications to auto-tune to the bandwidth-delay product

Re: [HACKERS] 9.3 reference constraint regression

2013-12-18 Thread Alvaro Herrera
Alvaro Herrera wrote: Andres Freund wrote: I have to say, it makes me really uncomfortable to take such shortcuts. In other branches we are doing liveliness checks with MultiXactIdIsRunning() et al. Why isn't that necessary here? And how likely is that this won't cause breakage

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Cédric Villemain
Le mardi 17 décembre 2013 17:45:51, Robert Haas a écrit : On Tue, Dec 17, 2013 at 11:02 AM, Jim Nasby j...@nasby.net wrote: On 12/17/13, 8:34 AM, Robert Haas wrote: On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila amit.kapil...@gmail.com wrote: I have used pg_prewarm during some of work

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Cédric Villemain
Le mardi 17 décembre 2013 21:14:44, Josh Berkus a écrit : On 12/17/2013 06:34 AM, Robert Haas wrote: On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila amit.kapil...@gmail.com wrote: I have used pg_prewarm during some of work related to Buffer Management and other performance related work. It

Re: [HACKERS] Dynamic Shared Memory stuff

2013-12-18 Thread Robert Haas
On Tue, Dec 10, 2013 at 6:26 PM, Tom Lane t...@sss.pgh.pa.us wrote: Noah Misch n...@leadboat.com writes: On Tue, Dec 10, 2013 at 07:50:20PM +0200, Heikki Linnakangas wrote: Let's not add more cases like that, if we can avoid it. Only if we can avoid it for a modicum of effort and feature

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 7:05 PM, Cédric Villemain ced...@2ndquadrant.fr wrote: Le mardi 17 décembre 2013 21:14:44, Josh Berkus a écrit : On 12/17/2013 06:34 AM, Robert Haas wrote: On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila amit.kapil...@gmail.com wrote: I have used pg_prewarm during some

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Josh Berkus
On 12/18/2013 08:44 AM, Alvaro Herrera wrote: Another thought: at the initial run of the assertion, note which tables it locked, and record this as an OID array in the catalog row for the assertion; consider running the assertion only when those tables are touched. This doesn't work if the

[HACKERS] 9.3 regression with dbt2

2013-12-18 Thread Dong Ye
Hi, We recently observed ~15% performance regression with dbt2 from PG 9.3. We narrowed down on testing master between 9.2 cut and 9.3 cut. It seems that 0ac5ad5134f2769ccbaefec73844f8504c4d6182 is the culprit commit. We did several runs and perf profiling comparing it against its parent

Re: [HACKERS] 9.3 regression with dbt2

2013-12-18 Thread Andres Freund
Hello, On 2013-12-18 10:24:56 -0800, Dong Ye wrote: It seems that 0ac5ad5134f2769ccbaefec73844f8504c4d6182 is the culprit commit. How long does a run take to verify the problem? Could you retry with the patch attached to

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Andres Freund
On 2013-12-18 13:44:15 -0300, Alvaro Herrera wrote: Heikki Linnakangas wrote: Ah, I see. You don't need to block anyone else from modifying the table, you just need to block anyone else from committing a transaction that had modified the table. So the locks shouldn't interfere with

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Jim Nasby
On 12/18/13, 11:57 AM, Josh Berkus wrote: On 12/18/2013 08:44 AM, Alvaro Herrera wrote: Another thought: at the initial run of the assertion, note which tables it locked, and record this as an OID array in the catalog row for the assertion; consider running the assertion only when those tables

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Jim Nasby
On 12/18/13, 10:44 AM, Alvaro Herrera wrote: It might prove useful to note that any given assertion involves tables A, B, C. If a transaction doesn't modify any of those tables then there's no need to run the assertion when the transaction commits, skipping acquisition of the assertion lock.

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Alvaro Herrera
Andres Freund wrote: On 2013-12-18 13:44:15 -0300, Alvaro Herrera wrote: Heikki Linnakangas wrote: Ah, I see. You don't need to block anyone else from modifying the table, you just need to block anyone else from committing a transaction that had modified the table. So the locks

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Kevin Grittner
Jim Nasby j...@nasby.net wrote: This is another case where it would be very useful to restrict what relations a transaction (or in this case, a substransaction) can access. If we had the ability to make that restriction then we could force assertions that aren't plain SQL to explicitly

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Andres Freund
On 2013-12-18 16:39:58 -0300, Alvaro Herrera wrote: Andres Freund wrote: It would only force serialization for transactions that modify tables covered by the assert, that doesn't seem to bad. Anything covered by an assert shoulnd't be modified frequently, otherwise you'll run into major

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Jim Nasby
On 12/18/13, 1:42 PM, Kevin Grittner wrote: Jim Nasby j...@nasby.net wrote: This is another case where it would be very useful to restrict what relations a transaction (or in this case, a substransaction) can access. If we had the ability to make that restriction then we could force assertions

Re: [HACKERS] 9.3 regression with dbt2

2013-12-18 Thread Dong Ye
~20 minutes each run with binary. Try your patch now.. You are right I used -g in perf record. But what I reported was flat (meant as a start). Expand GetMultiXactIdMembers: 3.82% postgres postgres [.] GetMultiXactIdMembers |

Re: [HACKERS] 9.3 regression with dbt2

2013-12-18 Thread Andres Freund
Hi, On 2013-12-18 14:59:45 -0500, Dong Ye wrote: ~20 minutes each run with binary. Try your patch now.. You are right I used -g in perf record. But what I reported was flat (meant as a start). Expand GetMultiXactIdMembers: 3.82% postgres postgres [.]

Re: [HACKERS] 9.3 regression with dbt2

2013-12-18 Thread Dong Ye
HI On Wed, Dec 18, 2013 at 3:03 PM, Andres Freund and...@2ndquadrant.comwrote: That looks like a postgres compiled without -fno-omit-frame-pointer. Without that hierarchical profiles are meaningless. Very new perfs can also do it using dwarf, but it's unusabl slow... Let me complete

Re: [HACKERS] SQL objects UNITs

2013-12-18 Thread Jim Nasby
On 12/18/13, 4:22 AM, Dimitri Fontaine wrote: ALTER UNIT name SET SCHEMA new schema; FWIW, with the units that we've developed we use schemas to differentiate between public objects and internal (private or protected) objects. So single-schema stuff becomes a PITA. Of course, since

Re: [HACKERS] 9.3 regression with dbt2

2013-12-18 Thread Dong Ye
Hi, Applied your patch (but not using -fno-omit-frame-pointer). It seems recover the perf loss: 55659.72 notpm. FWIW, the profile is below. I will do a run with the flag.. Samples: 598K of event 'cycles', Event count (approx.): 6568957160059 + 4.03% postgres postgres

[HACKERS] array_length(anyarray)

2013-12-18 Thread Marko Tiikkaja
Hi, Attached is a patch to add support for array_length(anyarray), which only works for one-dimensional arrays, returns 0 for empty arrays and complains if the array's lower bound isn't 1. In other words, does the right thing when used with the arrays people use 99% of the time. I'll add

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Kevin Grittner
Jim Nasby j...@nasby.net wrote: On 12/18/13, 1:42 PM, Kevin Grittner wrote: Jim Nasby j...@nasby.net wrote: This is another case where it would be very useful to restrict what relations a transaction (or in this case, a substransaction) can access. If we had the ability to make that

Re: [HACKERS] stats for network traffic WIP

2013-12-18 Thread Robert Haas
On Wed, Dec 18, 2013 at 8:47 AM, Stephen Frost sfr...@snowman.net wrote: Agreed. My other thought on this is that there's a lot to be said for having everything you need available through one tool- kinda like how Emacs users rarely go outside of it.. :) And then there's also the

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Andrew Dunstan
On 12/18/2013 02:45 PM, Andres Freund wrote: On 2013-12-18 16:39:58 -0300, Alvaro Herrera wrote: Andres Freund wrote: It would only force serialization for transactions that modify tables covered by the assert, that doesn't seem to bad. Anything covered by an assert shoulnd't be modified

Re: [HACKERS] stats for network traffic WIP

2013-12-18 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Wed, Dec 18, 2013 at 8:47 AM, Stephen Frost sfr...@snowman.net wrote: Agreed. My other thought on this is that there's a lot to be said for having everything you need available through one tool- kinda like how Emacs users rarely go outside of

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Heikki Linnakangas
On 12/18/2013 11:04 PM, Andrew Dunstan wrote: On 12/18/2013 02:45 PM, Andres Freund wrote: On 2013-12-18 16:39:58 -0300, Alvaro Herrera wrote: Andres Freund wrote: It would only force serialization for transactions that modify tables covered by the assert, that doesn't seem to bad. Anything

Re: [HACKERS] array_length(anyarray)

2013-12-18 Thread Andrew Dunstan
On 12/18/2013 03:27 PM, Marko Tiikkaja wrote: Hi, Attached is a patch to add support for array_length(anyarray), which only works for one-dimensional arrays, returns 0 for empty arrays and complains if the array's lower bound isn't 1. In other words, does the right thing when used with the

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Andrew Dunstan
On 12/18/2013 04:09 PM, Heikki Linnakangas wrote: On 12/18/2013 11:04 PM, Andrew Dunstan wrote: On 12/18/2013 02:45 PM, Andres Freund wrote: On 2013-12-18 16:39:58 -0300, Alvaro Herrera wrote: Andres Freund wrote: It would only force serialization for transactions that modify tables

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Josh Berkus
On 12/18/2013 11:26 AM, Jim Nasby wrote: The flip-side is that now you can get serialization failures, and I think there's a ton of software that has no clue how to deal with that. So now you don't get to use assertions at all unless you re-engineer your application (but see below). Well, the

Re: [HACKERS] array_length(anyarray)

2013-12-18 Thread Marko Tiikkaja
On 2013-12-18 22:13, Andrew Dunstan wrote: On 12/18/2013 03:27 PM, Marko Tiikkaja wrote: Attached is a patch to add support for array_length(anyarray), which only works for one-dimensional arrays, returns 0 for empty arrays and complains if the array's lower bound isn't 1. In other words, does

Re: [HACKERS] array_length(anyarray)

2013-12-18 Thread Marko Tiikkaja
On 2013-12-18 22:19, I wrote: On 2013-12-18 22:13, Andrew Dunstan wrote: On 12/18/2013 03:27 PM, Marko Tiikkaja wrote: Attached is a patch to add support for array_length(anyarray), which only works for one-dimensional arrays, returns 0 for empty arrays and complains if the array's lower bound

Re: [HACKERS] array_length(anyarray)

2013-12-18 Thread Andrew Dunstan
On 12/18/2013 04:19 PM, Marko Tiikkaja wrote: On 2013-12-18 22:13, Andrew Dunstan wrote: On 12/18/2013 03:27 PM, Marko Tiikkaja wrote: Attached is a patch to add support for array_length(anyarray), which only works for one-dimensional arrays, returns 0 for empty arrays and complains if the

Re: [HACKERS] array_length(anyarray)

2013-12-18 Thread Marko Tiikkaja
On 2013-12-18 22:32, Andrew Dunstan wrote: You're not really free to assume it - you'll need an exception handler for the other-than-1 case, or your code might blow up. This seems to be codifying a bad pattern, which should be using array_lower() and array_upper() instead. That's the entire

Re: [HACKERS] Extension Templates S03E11

2013-12-18 Thread Greg Stark
Yeah I think this whole discussion is happening at the wrong level. The problem you're having, despite appearances, is not that people disagree about the best way to accomplish your goals. The problem is that not everyone is convinced your goals are a good idea. Either they just don't understand

Re: [HACKERS] 9.3 regression with dbt2

2013-12-18 Thread Dong Ye
Applying your patch plus adding -fno-omit-frame-pointer, I got 54526.90 notpm. The profile (part) below: # Samples: 610K of event 'cycles' # Event count (approx.): 6686532056450 # # Overhead Command Shared Object Symbol # .. .

Re: [HACKERS] shared memory message queues

2013-12-18 Thread Andres Freund
On 2013-12-18 15:23:23 -0500, Robert Haas wrote: It sounds like most people who have looked at this stuff are broadly happy with it, so I'd like to push on toward commit soon, but it'd be helpful, Andres, if you could review the comment additions to shm-mq-v2.patch and see whether those

Re: [HACKERS] array_length(anyarray)

2013-12-18 Thread David Johnston
Marko Tiikkaja-4 wrote On 2013-12-18 22:32, Andrew Dunstan wrote: You're not really free to assume it - you'll need an exception handler for the other-than-1 case, or your code might blow up. This seems to be codifying a bad pattern, which should be using array_lower() and array_upper()

Re: [HACKERS] preserving forensic information when we freeze

2013-12-18 Thread Andres Freund
Hi, On 2013-12-17 16:00:14 -0500, Robert Haas wrote: @@ -5874,19 +5858,27 @@ heap_prepare_freeze_tuple(HeapTupleHeader tuple, TransactionId cutoff_xid, void heap_execute_freeze_tuple(HeapTupleHeader tuple, xl_heap_freeze_tuple *frz) { + tuple-t_infomask = frz-t_infomask; +

Re: [HACKERS] Assertion failure in base backup code path

2013-12-18 Thread Andres Freund
Hi Magnus, It looks to me like the path to do_pg_start_backup() outside of a transaction context comes from your initial commit of the base backup facility. The problem is that you're not allowed to do anything leading to a syscache/catcache lookup in those contexts. Greetings, Andres Freund

Re: [HACKERS] stats for network traffic WIP

2013-12-18 Thread Bruce Momjian
On Wed, Dec 18, 2013 at 03:41:24PM -0500, Robert Haas wrote: On the other hand, there's not much value in adding monitoring features that are going to materially harm performance, and a lot of the monitoring features that get proposed die on the vine for exactly that reason. I think the root

Re: [HACKERS] array_length(anyarray)

2013-12-18 Thread Marko Tiikkaja
On 12/19/13, 12:01 AM, David Johnston wrote: Marko Tiikkaja-4 wrote On 2013-12-18 22:32, Andrew Dunstan wrote: You're not really free to assume it - you'll need an exception handler for the other-than-1 case, or your code might blow up. This seems to be codifying a bad pattern, which should

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Kevin Grittner
Josh Berkus j...@agliodbs.com wrote: On 12/18/2013 11:26 AM, Jim Nasby wrote: Another possibility is to allow for two different types of assertions, one based on SSI and one based on locking. The locking version would have to pretty much lock on a table basis (or even a whole-database

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-12-18 Thread Bruce Momjian
On Wed, Dec 11, 2013 at 10:22:32AM +, Haribabu kommi wrote: The make_absolute_path() function moving to port is changed in similar way as Bruce Momjian approach. The psprintf is used to store the error string which Occurred in the function. But psprintf is not used for storing the absolute

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Cédric Villemain
Le mercredi 18 décembre 2013 18:40:09, Robert Haas a écrit : Now that we have dynamic background workers, I've been thinking that it might be possible to write a background worker to do asynchronous prefetch on systems where we don't have OS support. We could store a small ring buffer in

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Alvaro Herrera
Robert Haas escribió: I'm not inclined to wait for the next CommitFest to commit this, because it's a very simple patch and has already had a lot more field testing than most patches get before they're committed. And it's just a contrib module, so the damage it can do if there is in fact a

Re: [HACKERS] PoC: Partial sort

2013-12-18 Thread Andreas Karlsson
On 12/18/2013 01:02 PM, Alexander Korotkov wrote: My idea for a solution was to modify tuplesort to allow storing the already sorted keys in either memtuples or the sort result file, but setting a field so it does not sort thee already sorted tuples again. This would allow the

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Robert Haas
On Wed, Dec 18, 2013 at 6:07 PM, Cédric Villemain ced...@2ndquadrant.fr wrote: In the case of effective_io_concurrency, however, this may not work as well as expected, IIRC it is used to prefetch heap blocks, hopefully the requested blocks are contiguous but if there are too much holes it is

Re: [HACKERS] Autoconf 2.69 update

2013-12-18 Thread Peter Eisentraut
On Thu, 2013-11-14 at 22:00 -0500, Peter Eisentraut wrote: I'm proposing that we upgrade our Autoconf to 2.69 This has been done. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] preserving forensic information when we freeze

2013-12-18 Thread Robert Haas
On Wed, Dec 18, 2013 at 5:54 PM, Andres Freund and...@2ndquadrant.com wrote: if (frz-frzflags XLH_FREEZE_XVAC) + { HeapTupleHeaderSetXvac(tuple, FrozenTransactionId); + /* If we somehow haven't hinted the tuple previously, do it now. */ +

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2013-12-18 Thread Amit Kapila
On Wed, Dec 18, 2013 at 8:25 PM, Tatsuo Ishii is...@postgresql.org wrote: Is there any reason for the function returns int as it always returns 0 or 1. Maybe returns bool is better? I have committed your patches. Thanks. Thank you very much. With Regards, Amit Kapila. EnterpriseDB:

Re: [HACKERS] clang's -Wmissing-variable-declarations shows some shoddy programming

2013-12-18 Thread Bruce Momjian
On Sat, Dec 14, 2013 at 04:52:28PM +0100, Andres Freund wrote: Hi, Compiling postgres with said option in CFLAGS really gives an astounding number of warnings. Except some bison/flex generated ones, none of them looks acceptable to me. Most are just file local variables with a missing

Re: [HACKERS] row security roadmap proposal

2013-12-18 Thread Craig Ringer
On 12/18/2013 11:14 PM, Robert Haas wrote: To be clear, I wasn't advocating for a declarative approach; I think predicates are fine. There are usability issues to worry about either way, and my concern is that we address those. A declarative approach would certainly be valuable in that, for

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-18 Thread Amit Kapila
On Wed, Dec 18, 2013 at 11:30 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Dec 18, 2013 at 11:22 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Fri, Dec 13, 2013 at 7:57 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Thanks, committed with some minor changes: Should

  1   2   >