Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Alvaro Herrera
Satoshi Nagayasu wrote: > But, for now, I think we should have a real index for the > statistics data because we already have several index storages, > and it will allow us to minimize read/write operations. > > BTW, what kind of index would be preferred for this purpose? > btree or hash? I find

Re: [HACKERS] Is it necessary to rewrite table while increasing the scale of datatype numeric?

2013-09-04 Thread wangshuo
于 2013-09-04 23:41, Jeff Janes 回复: On Tue, Sep 3, 2013 at 9:08 PM, wrote: Hi, Hackers! I find that it takes a long time when I increase the scale of a numeric datatype. By checking the code, I found that's because it needs to rewrite that table's file. After checking that table's data fil

Re: [HACKERS] [9.4] Make full_page_writes only settable on server start?

2013-09-04 Thread Fujii Masao
On Thu, Sep 5, 2013 at 1:55 AM, Andres Freund wrote: > On 2013-09-04 09:23:20 -0700, Jeff Davis wrote: >> On Wed, 2013-09-04 at 11:32 -0400, Tom Lane wrote: >> > Jeff Davis writes: >> > > I think code complexity matters quite a lot. If we can eliminate some >> > > complex code in a complex area,

Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Atri Sharma
Sent from my iPad On 05-Sep-2013, at 8:58, Satoshi Nagayasu wrote: > (2013/09/05 3:59), Alvaro Herrera wrote: >> Tomas Vondra wrote: >> >>> My idea was to keep the per-database stats, but allow some sort of >>> "random" access - updating / deleting the records in place, adding >>> records etc

Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Satoshi Nagayasu
(2013/09/05 3:59), Alvaro Herrera wrote: Tomas Vondra wrote: My idea was to keep the per-database stats, but allow some sort of "random" access - updating / deleting the records in place, adding records etc. The simplest way I could think of was adding a simple "index" - a mapping of OID to pos

Re: [HACKERS] ENABLE/DISABLE CONSTRAINT NAME

2013-09-04 Thread wangshuo
于 2013-09-05 01:56, Robert Haas 回复: On Tue, Sep 3, 2013 at 3:13 AM, wrote: 于 2013-09-03 08:15, David Johnston 回复: Jeff Davis-8 wrote Is there any semantic difference between marking a constraint as DISABLED and simply dropping it? Or does it just make it easier to re-add it later? Davi

Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Satoshi Nagayasu
(2013/09/05 3:50), Pavel Stehule wrote: > we very successfully use a tmpfs volume for pgstat files (use a backport > of multiple statfiles from 9.3 to 9.1 It works quite well as long as you have the objects (tables, indexes, functions) spread across multiple databases. Once

[HACKERS] Re: Is it necessary to rewrite table while increasing the scale of datatype numeric???

2013-09-04 Thread wangshuo
于 2013-09-04 19:30, Noah Misch 回复: On Wed, Sep 04, 2013 at 12:08:48PM +0800, wangs...@highgo.com.cn wrote: I find that it takes a long time when I increase the scale of a numeric datatype. By checking the code, I found that's because it needs to rewrite that table's file. After checking that

Re: [HACKERS] [tiny doc fix] statistics are not retained across immediate shutdown

2013-09-04 Thread Fujii Masao
On Wed, Sep 4, 2013 at 11:56 PM, Andres Freund wrote: > On 2013-09-04 10:53:19 -0400, Tom Lane wrote: >> "Tsunakawa, Takayuki" writes: >> > I propose a tiny fix to clarify this. Please find the attached patch. >> >> That's not an accurate description of what happens, though. >> AFAIR, we do not

Re: [HACKERS] improve Chinese locale performance

2013-09-04 Thread Quan Zongliang
On 07/23/2013 09:42 PM, Craig Ringer wrote: (Replying on phone, please forgive bad quoting) Isn't this pretty much what adopting ICU is supposed to give us? OS-independent collations? I'd be interested in seeing the rest data for this performance report, partly as I'd like to see how ICU coll

Re: [HACKERS] strange IS NULL behaviour

2013-09-04 Thread Bruce Momjian
On Tue, Sep 3, 2013 at 09:32:44PM -0400, Bruce Momjian wrote: > In this test, SELECT NULL (which internally would produce SELECT > ROW(NULL)), returns TRUE, while SELECT ROW(NULL) and further nesting > returns false. > > This has made me adjust my goal and change it so SELECT ROW(NULL) IS > NULL

Re: [HACKERS] Frontend/backend protocol improvements proposal (request).

2013-09-04 Thread Noah Misch
On Fri, Jun 21, 2013 at 12:37:32PM +0400, Dmitriy Igrishin wrote: > 2013/6/21 Albe Laurenz > > Dmitriy Igrishin wrote: > > > While developing a C++ client library for Postgres I felt lack of extra > > > information in command tags in the CommandComplete (B) message > > > for the following commands

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-09-04 Thread Bruce Momjian
On Tue, Jan 8, 2013 at 08:40:44PM -0500, Andrew Dunstan wrote: > > On 01/08/2013 08:08 PM, Tom Lane wrote: > >Robert Haas writes: > >>On Tue, Jan 8, 2013 at 7:17 PM, Tom Lane wrote: > >>>... And I don't especially like the idea of trying to > >>>make it depend directly on the box's physical RA

Re: [HACKERS] INSERT...ON DUPLICATE KEY IGNORE

2013-09-04 Thread Gavin Flower
On 05/09/13 08:26, Robert Haas wrote: On Sat, Aug 31, 2013 at 2:34 PM, Andres Freund wrote: After some thinking I don't think any solution primarily based on holding page level locks across other index operations is going to scale ok. I'd like to chime in with a large +1 for this sentiment and

Re: [HACKERS] INSERT...ON DUPLICATE KEY IGNORE

2013-09-04 Thread Peter Geoghegan
On Wed, Sep 4, 2013 at 5:08 PM, Andres Freund wrote: > I don't want "my" idea to win, I want a idea to win. I know. I want the same thing. > You're the patch author here whose plans are laid open to be scrutinized ;). > If you think my idea has merit, use and adapt it to reality. If not, find

Re: [HACKERS] INSERT...ON DUPLICATE KEY IGNORE

2013-09-04 Thread Andres Freund
Hi, We seem to be miscommunication a bit. You've proposed an framework and algorithm for something I'd really, really like to see. I don't think that it can work explicitly as you proposed, so I roughly sketched out a solution I can see. I don't want "my" idea to win, I want a idea to win. I h

Re: [HACKERS] INSERT...ON DUPLICATE KEY IGNORE

2013-09-04 Thread Peter Geoghegan
On Wed, Sep 4, 2013 at 3:39 PM, Andres Freund wrote: > Sorry to be harsh here, but I don't think I need to do that. I've > explained most of the reasons I see that that approach won't work out > and so far I don't see those refuted. And to me those issues seem to be > fatal for the approach. If yo

Re: [HACKERS] INSERT...ON DUPLICATE KEY IGNORE

2013-09-04 Thread Andres Freund
On 2013-09-04 15:01:57 -0700, Peter Geoghegan wrote: > On Wed, Sep 4, 2013 at 1:26 PM, Robert Haas wrote: > > Concurrent readers will block in a non-interruptible wait if they try > > to access a buffer, and that's a situation that will be intolerable > > if, for example, it can persist across a d

Re: [HACKERS] 9.4 regression

2013-09-04 Thread Robert Haas
On Wed, Sep 4, 2013 at 11:26 AM, Tom Lane wrote: > Stephen Frost writes: >> * Andres Freund (and...@2ndquadrant.com) wrote: >>> I'd vote for adding zeroing *after* the fallocate() first. That's what's >>> suggested by kernel hackers and what several other large applications >>> do. As it looks li

Re: [HACKERS] Analysis on backend-private memory usage (and a patch)

2013-09-04 Thread Tom Lane
Heikki Linnakangas writes: > One fairly simple thing we could do is to teach catcache.c to resize the > caches. Then we could make the initial size of all the syscaches much > smaller. I think this is attractive for the *other* reason you mention, namely preserving reasonable performance when a

Re: [HACKERS] dynamic shared memory

2013-09-04 Thread Jim Nasby
On 8/31/13 7:17 AM, Robert Haas wrote: On Thu, Aug 29, 2013 at 8:12 PM, Jim Nasby wrote: On 8/13/13 8:09 PM, Robert Haas wrote: is removed, the segment automatically goes away (we could allow for server-lifespan segments as well with only trivial changes, but I'm not sure whether there are com

[HACKERS] Analysis on backend-private memory usage (and a patch)

2013-09-04 Thread Heikki Linnakangas
I received a complaint that each backend consumes a lot of backend-private memory, even if it's completely idle. "a lot" is of course very subjective and how much memory is actually used depends heavily on the application. In this case, the database is fairly small, but they have 250 connection

Re: [HACKERS] INSERT...ON DUPLICATE KEY IGNORE

2013-09-04 Thread Peter Geoghegan
On Wed, Sep 4, 2013 at 1:26 PM, Robert Haas wrote: > Concurrent readers will block in a non-interruptible wait if they try > to access a buffer, and that's a situation that will be intolerable > if, for example, it can persist across a disk I/O. And I don't see > any way to avoid that. Then I ha

Re: [HACKERS] [PERFORM] encouraging index-only scans

2013-09-04 Thread Bruce Momjian
On Thu, Dec 13, 2012 at 03:31:06PM +, Peter Geoghegan wrote: > On 13 December 2012 03:51, Tom Lane wrote: > > ANALYZE does not set that value, and is not going to start doing so, > > because it doesn't scan enough of the table to derive a trustworthy > > value. > > I'm slightly surprised by y

Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Tomas Vondra
On 4.9.2013 14:13, Stephen Frost wrote: > * Satoshi Nagayasu (sn...@uptime.jp) wrote: > >> Yes, we have already split single pgstat.stat file into several >> pieces. >> >> However, we still need to read/write large amount of statistics >> data when we have a large number of tables in single data

Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Tomas Vondra
On 4.9.2013 20:59, Alvaro Herrera wrote: > Tomas Vondra wrote: > >> My idea was to keep the per-database stats, but allow some sort of >> "random" access - updating / deleting the records in place, adding >> records etc. The simplest way I could think of was adding a simple >> "index" - a mapping

[HACKERS] De-normalization optimizer research project

2013-09-04 Thread arthernan
I am working on a research project that looks at an entire database access layer written against a normalized database and suggests de-normalizations to be done to the database structure. In effect looking at de-normalizations as a kind of cache. I am looking for any code I can leverage to do this

Re: [HACKERS] lcr v5 - introduction of InvalidCommandId

2013-09-04 Thread Andres Freund
On 2013-09-03 11:40:57 -0400, Robert Haas wrote: > > 0002 wal_decoding: Introduce InvalidCommandId and declare that to be the > > new maximum for CommandCounterIncrement > > I'm still unconvinced we want this. Ok, so the reason for the existance of this patch is that currently there is no way to

Re: [HACKERS] Improving avg performance for numeric

2013-09-04 Thread Peter Eisentraut
On 9/4/13 2:26 PM, Pavel Stehule wrote: > > > > 2013/9/4 Peter Eisentraut mailto:pete...@gmx.net>> > > On 7/8/13 10:05 AM, Pavel Stehule wrote: > > I am testing your code, and It increase speed of sum about 24% faster > > then original implementation. > > This patch needs to be

Re: [HACKERS] INSERT...ON DUPLICATE KEY IGNORE

2013-09-04 Thread Robert Haas
On Sat, Aug 31, 2013 at 2:34 PM, Andres Freund wrote: > After some thinking I don't think any solution primarily based on > holding page level locks across other index operations is going to scale > ok. I'd like to chime in with a large +1 for this sentiment and pretty much everything else Andres

Re: [HACKERS] Further XLogInsert scaling tweaking

2013-09-04 Thread Heikki Linnakangas
On 03.09.2013 16:22, Merlin Moncure wrote: On Mon, Sep 2, 2013 at 10:32 PM, Bruce Momjian wrote: On Mon, Sep 2, 2013 at 10:14:03AM +0300, Heikki Linnakangas wrote: diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 39c58d0..28e62ea 100644 --- a/src/backe

Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Alvaro Herrera
Tomas Vondra wrote: > My idea was to keep the per-database stats, but allow some sort of > "random" access - updating / deleting the records in place, adding > records etc. The simplest way I could think of was adding a simple > "index" - a mapping of OID to position in the stat file. > > I.e. a

Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Pavel Stehule
> > > > > we very successfully use a tmpfs volume for pgstat files (use a backport > > of multiple statfiles from 9.3 to 9.1 > > It works quite well as long as you have the objects (tables, indexes, > functions) spread across multiple databases. Once you have one database > with very large number o

Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Tomas Vondra
On 4.9.2013 14:43, Pavel Stehule wrote: > > > > 2013/9/4 Atri Sharma mailto:atri.j...@gmail.com>> > > > > Sent from my iPad > > On 04-Sep-2013, at 15:39, Satoshi Nagayasu > wrote: > > > (2013/09/04 15:23), Atri Sharma wrote: > >> > >> > >

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Tom Lane
Kohei KaiGai writes: > 2013/9/4 Tom Lane : >> And for insert/update/delete, we do what exactly? > Regarding to update/delete, this patch also enhanced to allow update or > delete mechanism allows to take a sub-query on top of the table scan plan. > So, its explain output shows as follows: > pos

Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Tomas Vondra
On 4.9.2013 07:24, Satoshi Nagayasu wrote: > Hi, > > (2013/09/04 12:52), Atri Sharma wrote: >> On Wed, Sep 4, 2013 at 6:40 AM, Satoshi Nagayasu >> wrote: >>> Hi, >>> >>> I'm considering overhauling pgstat.stat, and would like to know >>> how many people are interested in this topic. >>> >>> A

Re: [HACKERS] [9.4] Make full_page_writes only settable on server start?

2013-09-04 Thread Andres Freund
On 2013-09-04 09:23:20 -0700, Jeff Davis wrote: > On Wed, 2013-09-04 at 11:32 -0400, Tom Lane wrote: > > Jeff Davis writes: > > > I think code complexity matters quite a lot. If we can eliminate some > > > complex code in a complex area, and all we give up is a feature with > > > essentially no us

Re: [HACKERS] Improving avg performance for numeric

2013-09-04 Thread Pavel Stehule
2013/9/4 Peter Eisentraut > On 7/8/13 10:05 AM, Pavel Stehule wrote: > > I am testing your code, and It increase speed of sum about 24% faster > > then original implementation. > > This patch needs to be rebased (and/or the later version registered in > the commit fest). > > I updated a commit fe

Re: [HACKERS] Eliminating pg_catalog.pg_rewrite.ev_attr

2013-09-04 Thread Kevin Grittner
Tom Lane wrote: > Shouldn't attribute_used() be removed from rewriteManip.h? Yeah, I don't know how I missed that.  Thanks. > I was a bit surprised by your removal of the > rangeTableEntry_used() test in the hunk at > rewriteHandler.c:1273ff.  That's probably all right, but it takes > this out

Re: [HACKERS] ENABLE/DISABLE CONSTRAINT NAME

2013-09-04 Thread Robert Haas
On Tue, Sep 3, 2013 at 3:13 AM, wrote: > 于 2013-09-03 08:15, David Johnston 回复: > >> Jeff Davis-8 wrote >>> >>> Is there any semantic difference between marking a constraint as >>> DISABLED and simply dropping it? Or does it just make it easier to >>> re-add it later? >> >> > David Johnston wrote

Re: [HACKERS] Improving avg performance for numeric

2013-09-04 Thread Peter Eisentraut
On 7/8/13 10:05 AM, Pavel Stehule wrote: > I am testing your code, and It increase speed of sum about 24% faster > then original implementation. This patch needs to be rebased (and/or the later version registered in the commit fest). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Robert Haas
On Wed, Sep 4, 2013 at 11:22 AM, Tom Lane wrote: >> The same mechanism will prevent UPDATE and DELETE from seeing any rows >> the user shouldn't be able to touch. > > No, it won't, because we don't support direct update/delete on views > (and if you look, you'll notice the auto-updatable-view stuf

Re: [HACKERS] Eliminating pg_catalog.pg_rewrite.ev_attr

2013-09-04 Thread Tom Lane
Kevin Grittner writes: > Unless someone has an objection or thinks this needs to go through > the CF process, I will commit it tomorrow, with a catversion bump. Shouldn't attribute_used() be removed from rewriteManip.h? I was a bit surprised by your removal of the rangeTableEntry_used() test in

Re: [HACKERS] [9.4] Make full_page_writes only settable on server start?

2013-09-04 Thread Jeff Davis
On Wed, 2013-09-04 at 17:00 +0200, Andres Freund wrote: > On 2013-09-04 07:57:15 -0700, Jeff Davis wrote: > > XLogSaveBufferForHint() calls XLogCheckBuffer() but doesn't also look at > > the full page writes setting (like in XLogInsert()). That means, if > > checksums are enabled and full_page_writ

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Kohei KaiGai
2013/9/4 Tom Lane : > Robert Haas writes: >> On Wed, Sep 4, 2013 at 10:50 AM, Tom Lane wrote: >>> Robert Haas writes: Right. IMHO, this new feature should be similarly simple: when an unprivileged user references a table, treat that as a reference to a leakproof view over the tab

Re: [HACKERS] Re: Proposal/design feedback needed: WITHIN GROUP (sql standard ordered set aggregate functions)

2013-09-04 Thread Atri Sharma
Sent from my iPad On 04-Sep-2013, at 21:38, Peter Eisentraut wrote: > On 7/19/13 11:59 AM, Atri Sharma wrote: >> Hi all, >> >> This is our current work-in-progress patch for WITHIN GROUP. > > This patch needs to be rebased. > This version of patch is quite old.We will be sending an updated

[HACKERS] Eliminating pg_catalog.pg_rewrite.ev_attr

2013-09-04 Thread Kevin Grittner
This was previously discussed here: http://www.postgresql.org/message-id/flat/24836.1370713...@sss.pgh.pa.us#24836.1370713...@sss.pgh.pa.us The attached patch implements what I think we agreed on. To recap, ev_attr was present in pg_rewrite at the point that Postgres95 version 1.01 source code w

Re: [HACKERS] [9.4] Make full_page_writes only settable on server start?

2013-09-04 Thread Jeff Davis
On Wed, 2013-09-04 at 11:32 -0400, Tom Lane wrote: > Jeff Davis writes: > > I think code complexity matters quite a lot. If we can eliminate some > > complex code in a complex area, and all we give up is a feature with > > essentially no use case, that sounds like we're moving in the right > > dir

Re: [HACKERS] Eliminating pg_catalog.pg_rewrite.ev_attr

2013-09-04 Thread Alvaro Herrera
Is this transformation correct? If I read this correctly, you're missing the rangeTableEntry_used() condition, no? > *** a/src/backend/rewrite/rewriteHandler.c > --- b/src/backend/rewrite/rewriteHandler.c > *** > *** 1273,1287 matchLocks(CmdType event, > }

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Kohei KaiGai
2013/9/4 Kevin Grittner : > Robert Haas wrote: >> Tom Lane wrote: >>> Robert Haas writes: > IMHO, this new feature should be similarly simple: when an unprivileged user references a table, treat that as a reference to a leakproof view over the table, with the RLS qual injected >>>

Re: [HACKERS] Re: Proposal/design feedback needed: WITHIN GROUP (sql standard ordered set aggregate functions)

2013-09-04 Thread Peter Eisentraut
On 7/19/13 11:59 AM, Atri Sharma wrote: > Hi all, > > This is our current work-in-progress patch for WITHIN GROUP. This patch needs to be rebased. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgs

[HACKERS] Re: [HACKERS] Is it necessary to rewrite table while increasing the scale of datatype numeric?

2013-09-04 Thread Jeff Janes
On Tue, Sep 3, 2013 at 9:08 PM, wrote: > Hi, Hackers! > > I find that it takes a long time when I increase the scale of a numeric > datatype. > By checking the code, I found that's because it needs to rewrite that > table's file. > After checking that table's data file, I found only parameter n_h

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Kohei KaiGai
2013/9/4 Tom Lane : > Robert Haas writes: >> On Wed, Sep 4, 2013 at 10:45 AM, Tom Lane wrote: >>> Well, the security-barrier view stuff did not present itself as a 100% >>> solution. But perhaps more to the point, it was conceptually simple to >>> implement, ie don't flatten views if they have t

Re: [HACKERS] [9.4] Make full_page_writes only settable on server start?

2013-09-04 Thread Tom Lane
Jeff Davis writes: > On Tue, 2013-09-03 at 15:42 -0400, Robert Haas wrote: >> Although this is true, the administrator's estimate of whether that >> guarantee is or is not provided might not be as consistent as the >> hardware behavior itself. I am generally of the feeling that having >> to resta

Re: [HACKERS] 9.4 regression

2013-09-04 Thread Tom Lane
Stephen Frost writes: > * Andres Freund (and...@2ndquadrant.com) wrote: >> I'd vote for adding zeroing *after* the fallocate() first. That's what's >> suggested by kernel hackers and what several other large applications >> do. As it looks like it's what we would have to do if we ever get to use >

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Tom Lane
Robert Haas writes: > On Wed, Sep 4, 2013 at 10:45 AM, Tom Lane wrote: >> Well, the security-barrier view stuff did not present itself as a 100% >> solution. But perhaps more to the point, it was conceptually simple to >> implement, ie don't flatten views if they have this bit set, and don't >>

Re: [HACKERS] [tiny doc fix] statistics are not retained across immediate shutdown

2013-09-04 Thread Tom Lane
"Tsunakawa, Takayuki" writes: > I propose a tiny fix to clarify this. Please find the attached patch. That's not an accurate description of what happens, though. AFAIR, we do not throw away pg_stats files as a result of recovery. regards, tom lane -- Sent via pgsql-ha

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Kevin Grittner
Robert Haas wrote: > Tom Lane wrote: >> Robert Haas writes: >>> IMHO, this new feature should be similarly simple: when an >>> unprivileged user references a table, treat that as a reference >>> to a leakproof view over the table, with the RLS qual injected >>> into the view. >> >> And for inse

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Sun, Sep 1, 2013 at 11:47 PM, Greg Smith wrote: > > And if someone can INSERT values that they can't actually see once they're > > committed, that's a similarly bad we should describe. > > This is desirable in some cases but not others. If the go

Re: [HACKERS] 9.4 regression

2013-09-04 Thread Andres Freund
Hi, On 2013-09-04 11:15:37 -0400, Stephen Frost wrote: > * Andres Freund (and...@2ndquadrant.com) wrote: > > I'd vote for adding zeroing *after* the fallocate() first. That's what's > > suggested by kernel hackers and what several other large applications > > do. As it looks like it's what we woul

Re: [HACKERS] getting rid of maintainer-check

2013-09-04 Thread Robert Haas
On Tue, Sep 3, 2013 at 10:41 PM, Peter Eisentraut wrote: > The maintainer-check target never really caught on, I think. Most > people don't run it, and that in turn annoys those who do. Also, it > doesn't provide much functionality. > > I propose that we get rid of it and roll the functionality

Re: [HACKERS] getting rid of maintainer-check

2013-09-04 Thread Andres Freund
On 2013-09-03 22:41:17 -0400, Peter Eisentraut wrote: > The maintainer-check target never really caught on, I think. Most > people don't run it, and that in turn annoys those who do. Also, it > doesn't provide much functionality. > > I propose that we get rid of it and roll the functionality int

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Tom Lane
Robert Haas writes: > On Wed, Sep 4, 2013 at 10:50 AM, Tom Lane wrote: >> Robert Haas writes: >>> Right. IMHO, this new feature should be similarly simple: when an >>> unprivileged user references a table, treat that as a reference to a >>> leakproof view over the table, with the RLS qual injec

Re: [HACKERS] 9.4 regression

2013-09-04 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > I'd vote for adding zeroing *after* the fallocate() first. That's what's > suggested by kernel hackers and what several other large applications > do. As it looks like it's what we would have to do if we ever get to use > fallocate for relation exte

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Robert Haas
On Fri, Aug 30, 2013 at 3:43 PM, Tom Lane wrote: > I think it's entirely sensible to question whether we should reject (not > "hold up") RLS if it has major covert-channel problems. We've already had this argument before, about the security_barrier view stuff, and that code got committed and is a

Re: [HACKERS] 9.4 regression

2013-09-04 Thread Jeff Davis
On Thu, 2013-08-08 at 17:42 -0400, Tom Lane wrote: > Jon Nelson writes: > > At this point I'm convinced that the issue is a pathological case in > > ext4. The performance impact disappears as soon as the unwritten > > extent(s) are written to with real data. Thus, even though allocating > > files

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Kohei KaiGai
2013/9/3 Bruce Momjian : > On Sun, Sep 1, 2013 at 11:05:58AM -0700, Josh Berkus wrote: >> > Security community also concludes it is not avoidable nature as long >> > as human can observe system behavior and estimate something, thus, >> > security evaluation criteria does not require eliminate cove

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Robert Haas
On Sun, Sep 1, 2013 at 11:47 PM, Greg Smith wrote: > And if someone can INSERT values that they can't actually see once they're > committed, that's a similarly bad we should describe. This is desirable in some cases but not others. If the goal is compartmentalization, then it's sensible to preve

Re: [HACKERS] [9.4] Make full_page_writes only settable on server start?

2013-09-04 Thread Andres Freund
On 2013-09-04 07:57:15 -0700, Jeff Davis wrote: > XLogSaveBufferForHint() calls XLogCheckBuffer() but doesn't also look at > the full page writes setting (like in XLogInsert()). That means, if > checksums are enabled and full_page_writes is off, we'll still write > some full page images for checksu

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Tom Lane
Robert Haas writes: > On Fri, Aug 30, 2013 at 3:43 PM, Tom Lane wrote: >> I think it's entirely sensible to question whether we should reject (not >> "hold up") RLS if it has major covert-channel problems. > We've already had this argument before, about the security_barrier > view stuff, and tha

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Robert Haas
On Wed, Sep 4, 2013 at 10:50 AM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Sep 4, 2013 at 10:45 AM, Tom Lane wrote: >>> Well, the security-barrier view stuff did not present itself as a 100% >>> solution. But perhaps more to the point, it was conceptually simple to >>> implement, ie don'

Re: [HACKERS] 9.4 regression

2013-09-04 Thread Andres Freund
On 2013-09-04 08:01:30 -0700, Jeff Davis wrote: > On Thu, 2013-08-08 at 17:42 -0400, Tom Lane wrote: > > Jon Nelson writes: > > > At this point I'm convinced that the issue is a pathological case in > > > ext4. The performance impact disappears as soon as the unwritten > > > extent(s) are written

Re: [HACKERS] logical changeset generation v5

2013-09-04 Thread Robert Haas
On Tue, Sep 3, 2013 at 7:10 PM, Andres Freund wrote: >> I don't think it particularly needs to be configurable, but I wonder >> if we can't be a bit smarter about when we do it. For example, >> suppose we logged it every 15 s but only until we log a non-overflowed >> snapshot. > > There's actuall

Re: [HACKERS] UTF8 national character data type support WIP patch and list of open issues.

2013-09-04 Thread Tom Lane
"Boguk, Maksym" writes: > Hi, my task is implementing ANSI NATIONAL character string types as > part of PostgreSQL core. No, that's not a given. You have a problem to solve, ie store some UTF8 strings in a database that's mostly just 1-byte data. It is not clear that NATIONAL CHARACTER is the

Re: [HACKERS] [tiny doc fix] statistics are not retained across immediate shutdown

2013-09-04 Thread Andres Freund
On 2013-09-04 10:53:19 -0400, Tom Lane wrote: > "Tsunakawa, Takayuki" writes: > > I propose a tiny fix to clarify this. Please find the attached patch. > > That's not an accurate description of what happens, though. > AFAIR, we do not throw away pg_stats files as a result of recovery. StartupXL

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Robert Haas
On Wed, Sep 4, 2013 at 10:45 AM, Tom Lane wrote: > Well, the security-barrier view stuff did not present itself as a 100% > solution. But perhaps more to the point, it was conceptually simple to > implement, ie don't flatten views if they have this bit set, and don't > push down quals into such s

Re: [HACKERS] [9.4] Make full_page_writes only settable on server start?

2013-09-04 Thread Jeff Davis
On Tue, 2013-09-03 at 15:42 -0400, Robert Haas wrote: > Although this is true, the administrator's estimate of whether that > guarantee is or is not provided might not be as consistent as the > hardware behavior itself. I am generally of the feeling that having > to restart the server to change se

Re: [HACKERS] [v9.4] row level security

2013-09-04 Thread Kohei KaiGai
2013/9/1 Greg Stark : > On Sun, Sep 1, 2013 at 8:31 PM, Kohei KaiGai wrote: >> Or, any other criteria even though? >> >> My (current) preference is plan (c: we will be able to fix up *this* >> cover-channel with reasonable efforts on explain code. probably, >> we can close it if we don't print fil

Re: [HACKERS] The PostgreSQL License requires "LICENSE" file?

2013-09-04 Thread Josh Berkus
On 09/03/2013 11:20 PM, Tomonari Katsumata wrote: > I understands that "the PostgreSQL license" does not require > any specified file name and missing "LICENSE" is not problem. > > I'm Sorry for the stupid question. No, thank *you* for pointing it out! None of us had actually looked at that page

Re: [HACKERS] logical changeset generation v5

2013-09-04 Thread Andres Freund
On 2013-09-04 10:02:05 -0400, Robert Haas wrote: > On Tue, Sep 3, 2013 at 7:10 PM, Andres Freund wrote: > >> I don't think it particularly needs to be configurable, but I wonder > >> if we can't be a bit smarter about when we do it. For example, > >> suppose we logged it every 15 s but only until

Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Pavel Stehule
2013/9/4 Atri Sharma > > > Sent from my iPad > > On 04-Sep-2013, at 15:39, Satoshi Nagayasu wrote: > > > (2013/09/04 15:23), Atri Sharma wrote: > >> > >> > >> Sent from my iPad > >> > >> On 04-Sep-2013, at 10:54, Satoshi Nagayasu wrote: > >> > >>> Hi, > >>> > >>> (2013/09/04 12:52), Atri Sharma

Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Pavel Stehule
2013/9/4 Atri Sharma > > > Sent from my iPad > > On 04-Sep-2013, at 15:39, Satoshi Nagayasu wrote: > > > (2013/09/04 15:23), Atri Sharma wrote: > >> > >> > >> Sent from my iPad > >> > >> On 04-Sep-2013, at 10:54, Satoshi Nagayasu wrote: > >> > >>> Hi, > >>> > >>> (2013/09/04 12:52), Atri Sharma

Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Atri Sharma
Sent from my iPad On 04-Sep-2013, at 15:39, Satoshi Nagayasu wrote: > (2013/09/04 15:23), Atri Sharma wrote: >> >> >> Sent from my iPad >> >> On 04-Sep-2013, at 10:54, Satoshi Nagayasu wrote: >> >>> Hi, >>> >>> (2013/09/04 12:52), Atri Sharma wrote: On Wed, Sep 4, 2013 at 6:40 AM, S

Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Stephen Frost
Satoshi, * Satoshi Nagayasu (sn...@uptime.jp) wrote: > (2013/09/04 13:07), Alvaro Herrera wrote: > >We already changed it: > > > > commit 187492b6c2e8cafc5b39063ca3b67846e8155d24 > > Author: Alvaro Herrera > > Date: Mon Feb 18 17:56:08 2013 -0300 > > > > Split pgstat file in smaller pieces

[HACKERS] Re: Is it necessary to rewrite table while increasing the scale of datatype numeric???

2013-09-04 Thread Noah Misch
On Wed, Sep 04, 2013 at 12:08:48PM +0800, wangs...@highgo.com.cn wrote: > I find that it takes a long time when I increase the scale of a numeric > datatype. > By checking the code, I found that's because it needs to rewrite that > table's file. > After checking that table's data file, I found

Re: [HACKERS] [rfc] overhauling pgstat.stat

2013-09-04 Thread Satoshi Nagayasu
(2013/09/04 15:23), Atri Sharma wrote: Sent from my iPad On 04-Sep-2013, at 10:54, Satoshi Nagayasu wrote: Hi, (2013/09/04 12:52), Atri Sharma wrote: On Wed, Sep 4, 2013 at 6:40 AM, Satoshi Nagayasu wrote: Hi, I'm considering overhauling pgstat.stat, and would like to know how many peo

Re: [HACKERS] [9.3 doc fix] clarification of Solaris versions

2013-09-04 Thread MauMau
From: "Robert Haas" OK, patch committed and back-patched to 9.3. The patch file turned out to be sorta garbled. I'm not sure if a broken version of diff was used to generate this or whether MauMau hand-edited it after the fact, but the number of lines that were indicated in the control lines d