Re: [HACKERS] Row Level Security Bug ?

2017-11-12 Thread Joe Conway
On 11/12/2017 10:17 AM, Andrea Adami wrote: > if i do: > > SET ROLE 'manage...@scuola-1.it ' [SELECT from table] > i see only one row (as expected) > > but when i do: [SELECT from VIEWs] > I see all the rows always > > this way i lack all the row level security

Re: [HACKERS] Potential hot-standby bug around xacts committed but in xl_running_xacts

2017-05-02 Thread Simon Riggs
On 2 May 2017 at 18:06, Andres Freund wrote: >> What I suggest is that with logical decoding in mind we do this >> 1. Inject a new record XLOG_SNAPSHOT_START at the start of >> LogStandbySnapshot(). We start logical decoding from there. >> 2. Record any transactions that end

Re: [HACKERS] Potential hot-standby bug around xacts committed but in xl_running_xacts

2017-05-02 Thread Craig Ringer
On 2 May 2017 at 13:12, Simon Riggs wrote: > What I suggest is that with logical decoding in mind we do this > 1. Inject a new record XLOG_SNAPSHOT_START at the start of > LogStandbySnapshot(). We start logical decoding from there. > 2. Record any transactions that end >

Re: [HACKERS] Potential hot-standby bug around xacts committed but in xl_running_xacts

2017-05-01 Thread Simon Riggs
On 1 May 2017 at 22:38, Andres Freund wrote: > The thread below > http://archives.postgresql.org/message-id/f37e975c-908f-858e-707f-058d3b1eb214%402ndquadrant.com > describes an issue in logical decoding that arises because > xl_running_xacts' contents aren't necessarily

Re: [HACKERS] Latch reset ordering bug in condition_variable.c

2017-02-09 Thread Robert Haas
On Thu, Feb 9, 2017 at 6:01 AM, Thomas Munro wrote: > ConditionVariablePrepareToSleep() has a race that can leave you > hanging, introduced by me in the v4 patch. The problem is that that > function resets our latch after adding our process to the wakeup list. >

Re: [HACKERS] _hash_addovflpage has a bug

2017-01-10 Thread Robert Haas
On Mon, Jan 9, 2017 at 10:46 PM, Amit Kapila wrote: > Yeah, we can write code that way, but then it is better to rely just > on retain_pin variable in the function and add an Assert for bucket > page whenever we are retaining pin. How about doing something like >

Re: [HACKERS] _hash_addovflpage has a bug

2017-01-09 Thread Amit Kapila
On Mon, Jan 9, 2017 at 11:59 PM, Robert Haas wrote: > On Fri, Jan 6, 2017 at 11:32 PM, Amit Kapila wrote: >> On Sat, Jan 7, 2017 at 2:33 AM, Robert Haas wrote: >>> It looks to to me like the recent hash index changes have

Re: [HACKERS] _hash_addovflpage has a bug

2017-01-09 Thread Robert Haas
On Fri, Jan 6, 2017 at 11:32 PM, Amit Kapila wrote: > On Sat, Jan 7, 2017 at 2:33 AM, Robert Haas wrote: >> It looks to to me like the recent hash index changes have left >> _hash_addovflpage slightly broken. I think that if that function >>

Re: [HACKERS] _hash_addovflpage has a bug

2017-01-06 Thread Amit Kapila
On Sat, Jan 7, 2017 at 2:33 AM, Robert Haas wrote: > It looks to to me like the recent hash index changes have left > _hash_addovflpage slightly broken. I think that if that function > reaches the point where it calls _hash_getbuf() to fetch the next page > in the bucket

Re: [HACKERS] Re: [BUGS][PATCH] BUG #14486: Inserting and selecting interval have different constraints

2017-01-05 Thread Vitaly Burovoy
On 1/5/17, Tom Lane wrote: > Vitaly Burovoy writes: >> On 1/5/17, Tom Lane wrote: >>> My point is that ideally, any value that can physically fit into struct >>> Interval ought to be considered valid. The fact that interval_out

Re: [HACKERS] Re: [BUGS][PATCH] BUG #14486: Inserting and selecting interval have different constraints

2017-01-05 Thread Tom Lane
Vitaly Burovoy writes: > On 1/5/17, Tom Lane wrote: >> My point is that ideally, any value that can physically fit into struct >> Interval ought to be considered valid. The fact that interval_out can't >> cope is a bug in interval_out, which ideally

Re: [HACKERS] Re: [BUGS][PATCH] BUG #14486: Inserting and selecting interval have different constraints

2017-01-05 Thread Vitaly Burovoy
On 1/5/17, Tom Lane wrote: > Vitaly Burovoy writes: >> On 1/5/17, Tom Lane wrote: >>> We could think about replacing interval2tm's output format with some >>> other struct that uses a TimeOffset for hours and so cannot overflow.

Re: [HACKERS] Re: [BUGS][PATCH] BUG #14486: Inserting and selecting interval have different constraints

2017-01-05 Thread Tom Lane
Vitaly Burovoy writes: > On 1/5/17, Tom Lane wrote: >> We could think about replacing interval2tm's output format with some >> other struct that uses a TimeOffset for hours and so cannot overflow. >> I'm not sure though how far the effects would

Re: [HACKERS] Re: [BUGS][PATCH] BUG #14486: Inserting and selecting interval have different constraints

2017-01-05 Thread Vitaly Burovoy
On 1/5/17, Tom Lane wrote: > Vitaly Burovoy writes: >>> I've written a patch which fixes that bug (in attachment). >>> Should it be registered in the CF? > >> Oops. Forgot to attach the patch. Fixed. > > I suspect that many of these SAMESIGN() tests

Re: [HACKERS] Re: [BUGS][PATCH] BUG #14486: Inserting and selecting interval have different constraints

2017-01-05 Thread Tom Lane
Vitaly Burovoy writes: >> I've written a patch which fixes that bug (in attachment). >> Should it be registered in the CF? > Oops. Forgot to attach the patch. Fixed. I suspect that many of these SAMESIGN() tests you've added are not actually adequate/useful. That's

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-28 Thread Teodor Sigaev
The more I think about it, the more I think gin is just an innocent bystander, for which I just happen to have a particularly demanding test. I think something about snapshots and wrap-around may be broken. After 10 hours of running I've got 1587 XX000 2016-04-28 05:57:09.964 MSK:ERROR:

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-28 Thread Noah Misch
On Tue, Apr 26, 2016 at 08:22:03PM +0300, Teodor Sigaev wrote: > >>Check my reasoning: In version 4 I added a remebering of tail of pending > >>list into blknoFinish variable. And when we read page which was a tail on > >>cleanup start then we sets cleanupFinish variable and after cleaning that >

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-26 Thread Teodor Sigaev
Check my reasoning: In version 4 I added a remebering of tail of pending list into blknoFinish variable. And when we read page which was a tail on cleanup start then we sets cleanupFinish variable and after cleaning that page we will stop further cleanup. Any insert caused during cleanup will be

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-23 Thread Noah Misch
On Fri, Apr 22, 2016 at 02:03:01PM -0700, Jeff Janes wrote: > On Thu, Apr 21, 2016 at 11:00 PM, Noah Misch wrote: > > Could you describe the test case in sufficient detail for Teodor to > > reproduce > > your results? > > [detailed description and attachments] Thanks. > The

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-22 Thread Jeff Janes
On Thu, Apr 21, 2016 at 11:00 PM, Noah Misch wrote: > On Mon, Apr 18, 2016 at 05:48:17PM +0300, Teodor Sigaev wrote: >> >>Added, see attached patch (based on v3.1) >> > >> >With this applied, I am getting a couple errors I have not seen before >> >after extensive crash recovery

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-22 Thread Robert Haas
On Fri, Apr 22, 2016 at 2:20 PM, Jeff Janes wrote: >> Check my reasoning: In version 4 I added a remebering of tail of pending >> list into blknoFinish variable. And when we read page which was a tail on >> cleanup start then we sets cleanupFinish variable and after cleaning

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-22 Thread Jeff Janes
On Mon, Apr 18, 2016 at 7:48 AM, Teodor Sigaev wrote: >>> Added, see attached patch (based on v3.1) >> >> >> With this applied, I am getting a couple errors I have not seen before >> after extensive crash recovery testing: >> ERROR: attempted to delete invisible tuple >> ERROR:

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-22 Thread Peter Geoghegan
On Thu, Nov 5, 2015 at 2:44 PM, Jeff Janes wrote: > The bug theoretically exists in 9.5, but it wasn't until 9.6 (commit > e95680832854cf300e64c) that free pages were recycled aggressively > enough that it actually becomes likely to be hit. In other words: The bug could be

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-22 Thread Noah Misch
On Mon, Apr 18, 2016 at 05:48:17PM +0300, Teodor Sigaev wrote: > >>Added, see attached patch (based on v3.1) > > > >With this applied, I am getting a couple errors I have not seen before > >after extensive crash recovery testing: > >ERROR: attempted to delete invisible tuple > >ERROR: unexpected

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-18 Thread Teodor Sigaev
Added, see attached patch (based on v3.1) With this applied, I am getting a couple errors I have not seen before after extensive crash recovery testing: ERROR: attempted to delete invisible tuple ERROR: unexpected chunk number 1 (expected 2) for toast value 100338365 in pg_toast_16425 Huh,

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-17 Thread Jeff Janes
On Tue, Apr 12, 2016 at 9:53 AM, Teodor Sigaev wrote: > > With pending cleanup patch backend will try to get lock on metapage with > ConditionalLockPage. Will it interrupt autovacum worker? Correct, ConditionalLockPage should not interrupt the autovacuum worker. >> >>

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-15 Thread Teodor Sigaev
Alvaro's recommendation, to let the cleaner off the hook once it passes the page which was the tail page at the time it started, would prevent any process from getting pinned down indefinitely, but would Added, see attached patch (based on v3.1) If there is no objections I will aplly it at

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-12 Thread Teodor Sigaev
There are only 3 fundamental options I see, the cleaner can wait, "help", or move on. "Helping" is what it does now and is dangerous. Moving on gives the above-discussed unthrottling problem. Waiting has two problems. The act of waiting will cause autovacuums to be canceled, unless ugly hacks

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-12 Thread Teodor Sigaev
This restricts the memory used by ordinary backends when doing the cleanup to be work_mem. Shouldn't we let them use maintenance_work_mem? Only one backend can be doing this clean up of a given index at any given time, so we don't need to worry about many concurrent allocations of

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-12 Thread Noah Misch
On Thu, Apr 07, 2016 at 05:53:54PM -0700, Jeff Janes wrote: > On Thu, Apr 7, 2016 at 4:33 PM, Tom Lane wrote: > > Jeff Janes writes: > >> To summarize the behavior change: > > > >> In the released code, an inserting backend that violates the pending > >>

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-07 Thread Jeff Janes
On Thu, Apr 7, 2016 at 4:33 PM, Tom Lane wrote: > Jeff Janes writes: >> To summarize the behavior change: > >> In the released code, an inserting backend that violates the pending >> list limit will try to clean the list, even if it is already being >>

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-07 Thread Tom Lane
Jeff Janes writes: > To summarize the behavior change: > In the released code, an inserting backend that violates the pending > list limit will try to clean the list, even if it is already being > cleaned. It won't accomplish anything useful, but will go through the >

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-07 Thread Alvaro Herrera
Jeff Janes wrote: > The proposed change removes that throttle, so that inserters will > immediately see there is already a cleaner and just go back about > their business. Due to that, unthrottled backends could add to the > pending list faster than the cleaner can clean it, leading to >

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-07 Thread Jeff Janes
On Wed, Apr 6, 2016 at 9:52 AM, Teodor Sigaev wrote: > I'm inclining to push v3.1 as one of two winners by size/performance and, > unlike to pending lock patch, it doesn't change an internal logic of lock > machinery. This restricts the memory used by ordinary backends when

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-06 Thread Teodor Sigaev
I've tested the v2, v3 and v3.1 of the patch, to see if there are any differences. The v2 no longer applies, so I tested it on ee943004. The following table shows the total duration of the data load, and also sizes of the two GIN indexes. duration (sec) subject body

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-05 Thread Tomas Vondra
Hi, On 04/04/2016 02:25 PM, Tomas Vondra wrote: On 04/04/2016 02:06 PM, Teodor Sigaev wrote: The above-described topic is currently a PostgreSQL 9.6 open item. Teodor, since you committed the patch believed to have created it, you own this open item. If that responsibility lies elsewhere,

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-04 Thread Tomas Vondra
On 04/04/2016 02:06 PM, Teodor Sigaev wrote: The above-described topic is currently a PostgreSQL 9.6 open item. Teodor, since you committed the patch believed to have created it, you own this open item. If that responsibility lies elsewhere, please let us know whose responsibility it is to fix

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-04 Thread Teodor Sigaev
The above-described topic is currently a PostgreSQL 9.6 open item. Teodor, since you committed the patch believed to have created it, you own this open item. If that responsibility lies elsewhere, please let us know whose responsibility it is to fix this. Since new open items may be discovered

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-04 Thread Noah Misch
On Thu, Feb 25, 2016 at 11:19:20AM -0800, Jeff Janes wrote: > On Wed, Feb 24, 2016 at 8:51 AM, Teodor Sigaev wrote: > > Thank you for remembering this problem, at least for me. > > > >>> Well, turns out there's a quite significant difference, actually. The > >>> index sizes I

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-02-25 Thread Jeff Janes
On Wed, Feb 24, 2016 at 8:51 AM, Teodor Sigaev wrote: > Thank you for remembering this problem, at least for me. > >>> Well, turns out there's a quite significant difference, actually. The >>> index sizes I get (quite stable after multiple runs): >>> >>> 9.5 : 2428 MB >>>

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-02-25 Thread Tomas Vondra
Hi, On 02/25/2016 05:32 PM, Teodor Sigaev wrote: Well, turns out there's a quite significant difference, actually. The index sizes I get (quite stable after multiple runs): 9.5 : 2428 MB 9.6 + alone cleanup : 730 MB 9.6 + pending lock : 488 MB In attach modified alone_cleanup

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-02-25 Thread Teodor Sigaev
Well, turns out there's a quite significant difference, actually. The index sizes I get (quite stable after multiple runs): 9.5 : 2428 MB 9.6 + alone cleanup : 730 MB 9.6 + pending lock : 488 MB In attach modified alone_cleanup patch which doesn't break cleanup process as it does

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-02-24 Thread Teodor Sigaev
Thank you for remembering this problem, at least for me. Well, turns out there's a quite significant difference, actually. The index sizes I get (quite stable after multiple runs): 9.5 : 2428 MB 9.6 + alone cleanup : 730 MB 9.6 + pending lock : 488 MB Interesting, I don't see why

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-02-24 Thread Tomas Vondra
On 02/24/2016 06:56 AM, Robert Haas wrote: On Wed, Feb 24, 2016 at 9:17 AM, Tomas Vondra wrote: ... Are we going to anything about this? While the bug is present in 9.5 (and possibly other versions), fixing it before 9.6 gets out seems important because

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-02-23 Thread Robert Haas
On Wed, Feb 24, 2016 at 9:17 AM, Tomas Vondra wrote: >> Well, turns out there's a quite significant difference, actually. The >> index sizes I get (quite stable after multiple runs): >> >> 9.5 : 2428 MB >> 9.6 + alone cleanup : 730 MB >> 9.6 + pending

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-02-23 Thread Tomas Vondra
Hi, On 01/05/2016 10:38 AM, Tomas Vondra wrote: Hi, ... There shouldn't be a difference between the two approaches (although I guess there could be if one left a larger pending list than the other, as pending lists is very space inefficient), but since you included 9.5 in your test I

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-01-05 Thread Tomas Vondra
Hi, On 12/23/2015 09:33 PM, Jeff Janes wrote: On Mon, Dec 21, 2015 at 11:51 AM, Tomas Vondra wrote: On 12/21/2015 07:41 PM, Jeff Janes wrote: On Sat, Dec 19, 2015 at 3:19 PM, Tomas Vondra wrote: ... So both patches seem to

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2015-12-23 Thread Jeff Janes
On Mon, Dec 21, 2015 at 11:51 AM, Tomas Vondra wrote: > > > On 12/21/2015 07:41 PM, Jeff Janes wrote: >> >> On Sat, Dec 19, 2015 at 3:19 PM, Tomas Vondra >> wrote: > > > ... > >>> So both patches seem to do the trick, but (2) is faster.

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2015-12-21 Thread Tomas Vondra
On 12/21/2015 07:41 PM, Jeff Janes wrote: On Sat, Dec 19, 2015 at 3:19 PM, Tomas Vondra wrote: ... So both patches seem to do the trick, but (2) is faster. Not sure if this is expected. (BTW all the results are without asserts enabled). Do you know what the

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2015-12-21 Thread Jeff Janes
On Sat, Dec 19, 2015 at 3:19 PM, Tomas Vondra wrote: > Hi, > > On 11/06/2015 02:09 AM, Tomas Vondra wrote: >> >> Hi, >> >> On 11/06/2015 01:05 AM, Jeff Janes wrote: >>> >>> On Thu, Nov 5, 2015 at 3:50 PM, Tomas Vondra >>> wrote: >> >>

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2015-12-19 Thread Tomas Vondra
Hi, On 11/06/2015 02:09 AM, Tomas Vondra wrote: Hi, On 11/06/2015 01:05 AM, Jeff Janes wrote: On Thu, Nov 5, 2015 at 3:50 PM, Tomas Vondra wrote: ... I can do that - I see there are three patches in the two threads: 1) gin_pending_lwlock.patch (Jeff

Re: [HACKERS] psql tab completion bug for ALL IN TABLESPACE

2015-12-14 Thread Andres Freund
On 2015-12-14 12:18:27 +0100, Andres Freund wrote: > On 2015-12-12 21:04:31 +0900, Michael Paquier wrote: > > Hi all, > > > > I just bumped into the following thing while looking again at Thomas' > > patch for psql tab completion: > > --- a/src/bin/psql/tab-complete.c > > +++

Re: [HACKERS] psql tab completion bug for ALL IN TABLESPACE

2015-12-14 Thread Michael Paquier
On Mon, Dec 14, 2015 at 8:18 PM, Andres Freund wrote: > On 2015-12-12 21:04:31 +0900, Michael Paquier wrote: >> Hi all, >> >> I just bumped into the following thing while looking again at Thomas' >> patch for psql tab completion: >> --- a/src/bin/psql/tab-complete.c >> +++

Re: [HACKERS] psql tab completion bug for ALL IN TABLESPACE

2015-12-14 Thread Andres Freund
On 2015-12-14 20:58:21 +0900, Michael Paquier wrote: > On Mon, Dec 14, 2015 at 8:49 PM, Andres Freund wrote: > > On 2015-12-14 20:44:20 +0900, Michael Paquier wrote: > >> + /* > >> + * ALTER TABLE,INDEX,MATERIALIZED VIEW ALL IN TABLESPACE xxx OWNED > >> BY xxx > >> +

Re: [HACKERS] psql tab completion bug for ALL IN TABLESPACE

2015-12-14 Thread Michael Paquier
On Mon, Dec 14, 2015 at 9:08 PM, Andres Freund wrote: > ALTER TABLE ALL IN TABLESPACE pg_default OWNED BY andres SET TABLESPACE > works, because of Missed that. - /* If we have TABLE SET TABLESPACE provide a list of tablespaces */ - else if

Re: [HACKERS] psql tab completion bug for ALL IN TABLESPACE

2015-12-14 Thread Michael Paquier
On Mon, Dec 14, 2015 at 8:49 PM, Andres Freund wrote: > On 2015-12-14 20:44:20 +0900, Michael Paquier wrote: >> + /* >> + * ALTER TABLE,INDEX,MATERIALIZED VIEW ALL IN TABLESPACE xxx OWNED BY >> xxx >> + * SET TABLESPACE. >> + */ >> + else if

Re: [HACKERS] psql tab completion bug for ALL IN TABLESPACE

2015-12-14 Thread Michael Paquier
On Mon, Dec 14, 2015 at 8:36 PM, Andres Freund wrote: > On 2015-12-14 12:18:27 +0100, Andres Freund wrote: >> On 2015-12-12 21:04:31 +0900, Michael Paquier wrote: >> > Hi all, >> > >> > I just bumped into the following thing while looking again at Thomas' >> > patch for psql

Re: [HACKERS] psql tab completion bug for ALL IN TABLESPACE

2015-12-14 Thread Andres Freund
On 2015-12-12 21:04:31 +0900, Michael Paquier wrote: > Hi all, > > I just bumped into the following thing while looking again at Thomas' > patch for psql tab completion: > --- a/src/bin/psql/tab-complete.c > +++ b/src/bin/psql/tab-complete.c > @@ -1040,7 +1040,7 @@ psql_completion(const char

Re: [HACKERS] psql tab completion bug for ALL IN TABLESPACE

2015-12-14 Thread Andres Freund
On 2015-12-14 20:44:20 +0900, Michael Paquier wrote: > + /* > + * ALTER TABLE,INDEX,MATERIALIZED VIEW ALL IN TABLESPACE xxx OWNED BY > xxx > + * SET TABLESPACE. > + */ > + else if (pg_strcasecmp(prev9_wd, "ALL") == 0 && > + pg_strcasecmp(prev8_wd, "IN")

Re: [HACKERS] Within CF app, "Bug Fixes" should be "Bug Fixes/Refactoring"

2015-11-07 Thread Peter Eisentraut
On 11/6/15 11:34 AM, Robert Haas wrote: > On Fri, Nov 6, 2015 at 12:52 AM, Michael Paquier > wrote: >>> I guess I'm wondering whether there's really enough of this to need >>> its own category. >> >> We have a category "Code comments" as well. Let's give it a shot so I

Re: [HACKERS] Within CF app, "Bug Fixes" should be "Bug Fixes/Refactoring"

2015-11-06 Thread Robert Haas
On Fri, Nov 6, 2015 at 12:52 AM, Michael Paquier wrote: >> I guess I'm wondering whether there's really enough of this to need >> its own category. > > We have a category "Code comments" as well. Let's give it a shot so I > am adding it. We could always remove it later

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2015-11-05 Thread Jeff Janes
On Thu, Nov 5, 2015 at 2:18 PM, Tomas Vondra wrote: > Hi, > > while repeating some full-text benchmarks on master, I've discovered > that there's a data corruption bug somewhere. What happens is that while > loading data into a table with GIN indexes (using multiple

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2015-11-05 Thread Tomas Vondra
On 11/05/2015 11:44 PM, Jeff Janes wrote: > This looks like it is probably the same bug discussed here: http://www.postgresql.org/message-id/CAMkU=1xalflhuuohfp5v33rzedlvb5aknnujceum9knbkrb...@mail.gmail.com And here: http://www.postgresql.org/message-id/56041b26.2040...@sigaev.ru The bug

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2015-11-05 Thread Jeff Janes
On Thu, Nov 5, 2015 at 3:50 PM, Tomas Vondra wrote: > > > On 11/05/2015 11:44 PM, Jeff Janes wrote: >> >> >> This looks like it is probably the same bug discussed here: >> >> >>

Re: [HACKERS] Within CF app, "Bug Fixes" should be "Bug Fixes/Refactoring"

2015-11-05 Thread Peter Geoghegan
On Thu, Oct 29, 2015 at 1:29 PM, Peter Geoghegan wrote: >> "Refactoring" seems rather a narrow definition of what might show up >> in such a category, btw. Maybe "Code Beautification" would be a >> suitable title? I'm bikeshedding though. > > I think that there is value in

Re: [HACKERS] Within CF app, "Bug Fixes" should be "Bug Fixes/Refactoring"

2015-11-05 Thread Robert Haas
On Thu, Nov 5, 2015 at 4:53 PM, Peter Geoghegan wrote: > On Thu, Oct 29, 2015 at 1:29 PM, Peter Geoghegan wrote: >>> "Refactoring" seems rather a narrow definition of what might show up >>> in such a category, btw. Maybe "Code Beautification" would be a >>>

Re: [HACKERS] Within CF app, "Bug Fixes" should be "Bug Fixes/Refactoring"

2015-11-05 Thread Michael Paquier
On Fri, Nov 6, 2015 at 1:47 PM, Robert Haas wrote: > On Thu, Nov 5, 2015 at 4:53 PM, Peter Geoghegan wrote: >> On Thu, Oct 29, 2015 at 1:29 PM, Peter Geoghegan wrote: "Refactoring" seems rather a narrow definition of what might show

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2015-11-05 Thread Tomas Vondra
Hi, On 11/06/2015 01:05 AM, Jeff Janes wrote: On Thu, Nov 5, 2015 at 3:50 PM, Tomas Vondra wrote: ... I can do that - I see there are three patches in the two threads: 1) gin_pending_lwlock.patch (Jeff Janes) 2) gin_pending_pagelock.patch (Jeff Janes)

Re: [HACKERS] Within CF app, "Bug Fixes" should be "Bug Fixes/Refactoring"

2015-10-29 Thread Michael Paquier
On Thu, Oct 29, 2015 at 8:33 PM, Peter Geoghegan wrote: > I think that within the CF app, we should either rename the patch > topic "Bug Fixes" to "Bug Fixes/Refactoring", or introduce a new > "Refactoring" topic. I prefer the first approach. I would vote for the second

Re: [HACKERS] Within CF app, "Bug Fixes" should be "Bug Fixes/Refactoring"

2015-10-29 Thread Mike Blackwell
​ On Thu, Oct 29, 2015 at 2:45 PM, Michael Paquier wrote: > On Thu, Oct 29, 2015 at 8:33 PM, Peter Geoghegan wrote: > > I think that within the CF app, we should either rename the patch > > topic "Bug Fixes" to "Bug Fixes/Refactoring", or introduce

Re: [HACKERS] Within CF app, "Bug Fixes" should be "Bug Fixes/Refactoring"

2015-10-29 Thread Tom Lane
Michael Paquier writes: > On Thu, Oct 29, 2015 at 8:33 PM, Peter Geoghegan wrote: >> I think that within the CF app, we should either rename the patch >> topic "Bug Fixes" to "Bug Fixes/Refactoring", or introduce a new >> "Refactoring" topic. I prefer

Re: [HACKERS] Within CF app, "Bug Fixes" should be "Bug Fixes/Refactoring"

2015-10-29 Thread Peter Geoghegan
On Thu, Oct 29, 2015 at 1:10 PM, Tom Lane wrote: > Ditto. Bug fixes are not at all like refactoring --- in particular, we'd > usually not consider refactoring as fit material for back-patching. > > "Refactoring" seems rather a narrow definition of what might show up > in such

Re: [HACKERS] Within CF app, "Bug Fixes" should be "Bug Fixes/Refactoring"

2015-10-29 Thread Josh Berkus
On 10/29/2015 01:10 PM, Tom Lane wrote: > Michael Paquier writes: >> On Thu, Oct 29, 2015 at 8:33 PM, Peter Geoghegan wrote: >>> I think that within the CF app, we should either rename the patch >>> topic "Bug Fixes" to "Bug Fixes/Refactoring", or

Re: [HACKERS] Potential GIN vacuum bug

2015-10-04 Thread Jeff Janes
On Thu, Sep 3, 2015 at 10:42 AM, Jeff Janes wrote: > On Mon, Aug 31, 2015 at 12:10 AM, Jeff Janes wrote: > >> On Sun, Aug 30, 2015 at 3:57 PM, Tom Lane wrote: >> >>> Jeff Janes writes: >>> > On Sun, Aug 30,

Re: [HACKERS] Potential GIN vacuum bug

2015-10-02 Thread Robert Haas
On Thu, Oct 1, 2015 at 4:44 PM, Jeff Janes wrote: > Is the locking around indexes summarized someplace? Not to my knowledge. :-( -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Potential GIN vacuum bug

2015-10-01 Thread Jeff Janes
On Tue, Sep 1, 2015 at 8:05 AM, Robert Haas wrote: > On Sun, Aug 30, 2015 at 6:57 PM, Tom Lane wrote: > >> But we would still have to deal with the > >> fact that unconditional acquire attempt by the backends will cause a > vacuum > >> to cancel

Re: [HACKERS] Arguable RLS security bug, EvalPlanQual() paranoia

2015-09-29 Thread Stephen Frost
* Adam Brightwell (adam.brightw...@crunchydatasolutions.com) wrote: > On Mon, Aug 3, 2015 at 6:21 PM, Peter Geoghegan wrote: > > On Mon, Aug 3, 2015 at 3:07 PM, Stephen Frost wrote: > >> Thoughts? Trying to keep it straight-forward and provide a simple > >>

Re: [HACKERS] Arguable RLS security bug, EvalPlanQual() paranoia

2015-09-29 Thread Adam Brightwell
On Mon, Aug 3, 2015 at 6:21 PM, Peter Geoghegan wrote: > On Mon, Aug 3, 2015 at 3:07 PM, Stephen Frost wrote: >> Thoughts? Trying to keep it straight-forward and provide a simple >> solution for users to be able to address the issue, if they're worried >>

Re: [HACKERS] Arguable RLS security bug, EvalPlanQual() paranoia

2015-09-29 Thread Adam Brightwell
> I'm not convinced this is the right place, but at a minimum it should be > referenced from the RLS documentation. Further, it should be noted that > users who have direct SQL access can control what the isolation level > is for their transaction. I agree that it should be referenced by the RLS

Re: [HACKERS] Is this a bug?

2015-09-05 Thread Bruce Momjian
On Fri, Sep 4, 2015 at 09:40:10AM +0900, Michael Paquier wrote: > Robert Haas wrote: > > On Wed, Aug 26, 2015 at 3:31 PM, Alvaro Herrera > > wrote: > > > Fabrízio de Royes Mello wrote: > > > > > >> Why this patch was reverted one day after

Re: [HACKERS] Is this a bug?

2015-09-03 Thread Robert Haas
On Wed, Aug 26, 2015 at 3:31 PM, Alvaro Herrera wrote: > Fabrízio de Royes Mello wrote: > >> Why this patch was reverted one day after applied [1]? I didn't see any >> discussion around it. > > Contributors whose patches are getting committed should really subscribe > to

Re: [HACKERS] Is this a bug?

2015-09-03 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Aug 26, 2015 at 3:31 PM, Alvaro Herrera > wrote: > > Fabrízio de Royes Mello wrote: > > > >> Why this patch was reverted one day after applied [1]? I didn't see any > >> discussion around it. > > > > Contributors whose patches are getting

Re: [HACKERS] Potential GIN vacuum bug

2015-09-03 Thread Jeff Janes
On Mon, Aug 31, 2015 at 12:10 AM, Jeff Janes wrote: > On Sun, Aug 30, 2015 at 3:57 PM, Tom Lane wrote: > >> Jeff Janes writes: >> > On Sun, Aug 30, 2015 at 11:11 AM, Tom Lane wrote: >> > > But we would still

Re: [HACKERS] Is this a bug?

2015-09-03 Thread Michael Paquier
On Fri, Sep 4, 2015 at 3:52 AM, Alvaro Herrera wrote: > Robert Haas wrote: > > On Wed, Aug 26, 2015 at 3:31 PM, Alvaro Herrera > > wrote: > > > Fabrízio de Royes Mello wrote: > > > > > >> Why this patch was reverted one day after applied [1]?

Re: [HACKERS] Potential GIN vacuum bug

2015-09-01 Thread Robert Haas
On Sun, Aug 30, 2015 at 6:57 PM, Tom Lane wrote: >> But we would still have to deal with the >> fact that unconditional acquire attempt by the backends will cause a vacuum >> to cancel itself, which is undesirable. > > Good point. > >> If we define a new namespace for >> this

Re: [HACKERS] Potential GIN vacuum bug

2015-08-31 Thread Jeff Janes
On Sun, Aug 30, 2015 at 3:57 PM, Tom Lane wrote: > Jeff Janes writes: > > On Sun, Aug 30, 2015 at 11:11 AM, Tom Lane wrote: > >> Your earlier point about how the current design throttles insertions to > >> keep the pending list from

Re: [HACKERS] Potential GIN vacuum bug

2015-08-30 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: Attached is a patch to deal with this without the heavyweight locks. I realized that using the clean up lock on the meta page, rather than the pending head page, would be easier to implement as a pin was already held on the meta page throughout, and the

Re: [HACKERS] Potential GIN vacuum bug

2015-08-30 Thread Jeff Janes
On Sat, Aug 22, 2015 at 11:25 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Tue, Aug 18, 2015 at 8:59 AM, Robert Haas robertmh...@gmail.com wrote: On Mon, Aug 17, 2015 at 5:41 PM, Jeff Janes jeff.ja...@gmail.com wrote: User backends attempt to take the lock conditionally, because otherwise

Re: [HACKERS] Potential GIN vacuum bug

2015-08-30 Thread Jeff Janes
On Sun, Aug 30, 2015 at 11:11 AM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: Your earlier point about how the current design throttles insertions to keep the pending list from growing without bound seems like a bigger deal to worry about. I think we'd like to

Re: [HACKERS] Potential GIN vacuum bug

2015-08-30 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: On Sun, Aug 30, 2015 at 11:11 AM, Tom Lane t...@sss.pgh.pa.us wrote: Your earlier point about how the current design throttles insertions to keep the pending list from growing without bound seems like a bigger deal to worry about. I think we'd like to

Re: [HACKERS] Is this a bug?

2015-08-26 Thread Fabrízio de Royes Mello
On Mon, Aug 25, 2014 at 6:07 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Aug 22, 2014 at 10:04:50PM -0400, Bruce Momjian wrote: On Fri, Aug 22, 2014 at 03:12:47PM -0400, Robert Haas wrote: On Fri, Aug 22, 2014 at 2:33 PM, Bruce Momjian br...@momjian.us wrote: Yes, you remember

Re: [HACKERS] Is this a bug?

2015-08-26 Thread Thom Brown
On 26 August 2015 at 20:24, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: On Mon, Aug 25, 2014 at 6:07 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Aug 22, 2014 at 10:04:50PM -0400, Bruce Momjian wrote: On Fri, Aug 22, 2014 at 03:12:47PM -0400, Robert Haas wrote: On

Re: [HACKERS] Is this a bug?

2015-08-26 Thread Alvaro Herrera
Fabrízio de Royes Mello wrote: Why this patch was reverted one day after applied [1]? I didn't see any discussion around it. Contributors whose patches are getting committed should really subscribe to pgsql-committers. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] Is this a bug?

2015-08-26 Thread Fabrízio de Royes Mello
On Wed, Aug 26, 2015 at 4:30 PM, Andres Freund and...@anarazel.de wrote: On 2015-08-26 16:24:31 -0300, Fabrízio de Royes Mello wrote: Why this patch was reverted one day after applied [1]? I didn't see any discussion around it.

Re: [HACKERS] Is this a bug?

2015-08-26 Thread Andres Freund
On 2015-08-26 16:24:31 -0300, Fabrízio de Royes Mello wrote: Why this patch was reverted one day after applied [1]? I didn't see any discussion around it. http://www.postgresql.org/message-id/23918.1409010...@sss.pgh.pa.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] Is this a bug?

2015-08-26 Thread Fabrízio de Royes Mello
On Wed, Aug 26, 2015 at 4:31 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Fabrízio de Royes Mello wrote: Why this patch was reverted one day after applied [1]? I didn't see any discussion around it. Contributors whose patches are getting committed should really subscribe to

Re: [HACKERS] Potential GIN vacuum bug

2015-08-22 Thread Jeff Janes
On Tue, Aug 18, 2015 at 8:59 AM, Robert Haas robertmh...@gmail.com wrote: On Mon, Aug 17, 2015 at 5:41 PM, Jeff Janes jeff.ja...@gmail.com wrote: User backends attempt to take the lock conditionally, because otherwise they would cause an autovacuum already holding the lock to cancel itself,

Re: [HACKERS] Potential GIN vacuum bug

2015-08-18 Thread Robert Haas
On Mon, Aug 17, 2015 at 5:41 PM, Jeff Janes jeff.ja...@gmail.com wrote: User backends attempt to take the lock conditionally, because otherwise they would cause an autovacuum already holding the lock to cancel itself, which seems quite bad. Not that this a substantial behavior change, in that

Re: [HACKERS] Potential GIN vacuum bug

2015-08-17 Thread Jeff Janes
On Mon, Aug 17, 2015 at 6:23 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Aug 16, 2015 11:49 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 08/16/2015 12:58 AM, Jeff Janes wrote: When ginbulkdelete gets called for the first time in a VACUUM(i.e. stats == NULL), one of the first

Re: [HACKERS] Potential GIN vacuum bug

2015-08-17 Thread Alvaro Herrera
Jeff Janes wrote: The attached patch takes a ShareUpdateExclusiveLock lock on the index in order to clean the pending list. Does it take a lock on the table also? Because if not ... One potential problem is how it will interact with create index concurrently. ... then I don't understand

  1   2   3   4   5   >