Hi,
As part of our monitoring work for our customers, we stumbled upon an issue
with our customers' servers who have a wal_keep_segments setting higher
than 0.
We have a monitoring script that checks the number of WAL files in the
pg_xlog directory, according to the setting of three parameters
(c
On Thu, Aug 7, 2014 at 5:24 PM, wrote:
>> Okay, applied the patch.
>>
>> I heavily modified your patch based on the master that the refactoring
>> patch has been applied. Attached is that modified version. Could you
>> review that?
>
> Thank you for the patch.
> I did a review of the patch.
>
> N
On Wed, 2014-08-06 at 11:43 -0400, Robert Haas wrote:
> Comparing the median times, that's about a 3% regression. For this
> particular case, we might be able to recapture that by replacing the
> bespoke memory-tracking logic in tuplesort.c with use of this new
> facility. I'm not sure whether th
I think there's a race condition in mminsert, if two backends insert a
tuple to the same heap page range concurrently. mminsert does this:
1. Fetch the MMtuple for the page range
2. Check if any of the stored datums need updating
3. Unlock the page.
4. Lock the page again in exclusive mode.
5. U
(2014/08/06 20:43), Etsuro Fujita wrote:
>> (2014/06/30 22:48), Tom Lane wrote:
>>> I wonder whether it isn't time to change that. It was coded like that
>>> originally only because calculating the values would've been a waste of
>>> cycles at the time. But this is at least the third place where
On Fri, Aug 8, 2014 at 4:31 AM, Pavel Stehule wrote:
>
>
>
> 2014-08-07 7:10 GMT+02:00 Fujii Masao :
>
>> On Thu, Aug 7, 2014 at 6:26 AM, Pavel Stehule
>> wrote:
>> > Hello
>> >
>> > updated version patch in attachment
>>
>> Thanks! But ISTM you forgot to attached the patch.
>
>
> grr .. I am sor
On Thu, Aug 7, 2014 at 12:07 PM, Abhijit Menon-Sen wrote:
> At 2014-08-07 23:22:43 +0900, masao.fu...@gmail.com wrote:
>> That is, we log replication commands only when log_statement is set to
>> all. Neither new parameter is introduced nor log_statement is
>> redefined as a list.
>
> That sounds
On Thu, Aug 7, 2014 at 5:52 PM, Peter Geoghegan wrote:
> On Thu, Aug 7, 2014 at 2:34 PM, Rod Taylor wrote:
>> This one is frequently sorted as batch operations against the files are
>> performed in alphabetical order to reduce conflict issues that a random
>> ordering may cause between jobs.
>
>
Hi,
Currently there's no way to generate or extract armor headers from the
PGP armored format in pgcrypto. I've written a patch to add the
support. For example:
local:marko=#* select armor('zooka', array['Version', 'Comment'],
array['Created by pgcrypto', 'PostgreSQL, the database']);
Another race condition:
If a new tuple is inserted to the range while summarization runs, it's
possible that the new tuple isn't included in the tuple that the
summarization calculated, nor does the insertion itself udpate it.
1. There is no index tuple for page range 1-10
2. Summarization be
* Robert Haas (robertmh...@gmail.com) wrote:
> On Thu, Aug 7, 2014 at 5:52 PM, Peter Geoghegan wrote:
> > On Thu, Aug 7, 2014 at 2:34 PM, Rod Taylor wrote:
> >> This one is frequently sorted as batch operations against the files are
> >> performed in alphabetical order to reduce conflict issues t
On 08/07/2014 11:17 PM, Tom Lane wrote:
I looked into the issue reported in bug #11109. The problem appears to be
that jsonb's on-disk format is designed in such a way that the leading
portion of any JSON array or object will be fairly incompressible, because
it consists mostly of a strictly-in
I've tracked down the real root cause. The fix is very simple. Please
check the attached one-liner patch.
The cause is that the temporary relations are truncated unconditionally
regardless of whether they are accessed in the transaction or not. That is,
the following sequence of steps resul
On Mon, Apr 14, 2014 at 11:40 PM, Tom Lane wrote:
> Fujii Masao writes:
>> On Tue, Apr 1, 2014 at 1:41 AM, Robert Haas wrote:
>>> Should we try to install some hack around fastupdate for 9.4? I fear
>>> the divergence between reasonable values of work_mem and reasonable
>>> sizes for that list
On Fri, Aug 8, 2014 at 11:43 PM, Fujii Masao wrote:
> On Mon, Apr 14, 2014 at 11:40 PM, Tom Lane wrote:
>> Fujii Masao writes:
>>> On Tue, Apr 1, 2014 at 1:41 AM, Robert Haas wrote:
Should we try to install some hack around fastupdate for 9.4? I fear
the divergence between reasonable
On Fri, Aug 8, 2014 at 11:43 PM, Fujii Masao wrote:
> The attached patch introduces...
A patch perhaps? :)
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Aug 8, 2014 at 08:51:13AM -0400, Robert Haas wrote:
> On Thu, Aug 7, 2014 at 12:07 PM, Abhijit Menon-Sen
> wrote:
> > At 2014-08-07 23:22:43 +0900, masao.fu...@gmail.com wrote:
> >> That is, we log replication commands only when log_statement is set to
> >> all. Neither new parameter is
Stephen Frost writes:
> * Tom Lane (t...@sss.pgh.pa.us) wrote:
>> I looked into the issue reported in bug #11109. The problem appears to be
>> that jsonb's on-disk format is designed in such a way that the leading
>> portion of any JSON array or object will be fairly incompressible, because
>> it
Andrew Dunstan writes:
> On 08/07/2014 11:17 PM, Tom Lane wrote:
>> I looked into the issue reported in bug #11109. The problem appears to be
>> that jsonb's on-disk format is designed in such a way that the leading
>> portion of any JSON array or object will be fairly incompressible, because
>>
On Fri, Aug 8, 2014 at 11:00 AM, Bruce Momjian wrote:
> On Fri, Aug 8, 2014 at 08:51:13AM -0400, Robert Haas wrote:
>> On Thu, Aug 7, 2014 at 12:07 PM, Abhijit Menon-Sen
>> wrote:
>> > At 2014-08-07 23:22:43 +0900, masao.fu...@gmail.com wrote:
>> >> That is, we log replication commands only whe
On 08/08/2014 11:18 AM, Tom Lane wrote:
Andrew Dunstan writes:
On 08/07/2014 11:17 PM, Tom Lane wrote:
I looked into the issue reported in bug #11109. The problem appears to be
that jsonb's on-disk format is designed in such a way that the leading
portion of any JSON array or object will be
In master I've been testing some new code that I'm working on around
foreign keys.
I wasn't quite sure if it was possible to include the same column twice in
a foreign key, so I tested
create table r1 (a int);
create table r2 (b int);
create unique index on r2(b,b);
alter table r1 add constrai
Andrew Dunstan writes:
> On 08/08/2014 11:18 AM, Tom Lane wrote:
>> That's not really the issue here, I think. The problem is that a
>> relatively minor aspect of the representation, namely the choice to store
>> a series of offsets rather than a series of lengths, produces
>> nonrepetitive data
On Fri, Aug 8, 2014 at 8:02 AM, Tom Lane wrote:
> Stephen Frost writes:
> > * Tom Lane (t...@sss.pgh.pa.us) wrote:
>
> > I'm rather disinclined to change the on-disk format because of this
> > specific test, that feels a bit like the tail wagging the dog to me,
> > especially as I do hope that s
David Rowley writes:
> I wasn't quite sure if it was possible to include the same column twice in
> a foreign key, so I tested
> create table r1 (a int);
> create table r2 (b int);
> create unique index on r2(b,b);
> alter table r1 add constraint r2_b_fkey foreign key (a,a) references r2
> (b
John W Higgins writes:
> Would an answer be to switch the location of the jsonb "header" data to the
> end of the field as opposed to the beginning of the field? That would allow
> pglz to see what it wants to see early on and go to work when possible?
Hm, might work. Seems a bit odd, but it wou
On 08/08/2014 12:04 PM, John W Higgins wrote:
Would an answer be to switch the location of the jsonb "header" data
to the end of the field as opposed to the beginning of the field? That
would allow pglz to see what it wants to see early on and go to work
when possible?
Add an offset at th
On 08/08/2014 11:54 AM, Tom Lane wrote:
Andrew Dunstan writes:
On 08/08/2014 11:18 AM, Tom Lane wrote:
That's not really the issue here, I think. The problem is that a
relatively minor aspect of the representation, namely the choice to store
a series of offsets rather than a series of length
On Fri, Aug 8, 2014 at 5:54 AM, Robert Haas wrote:
> Well, I'm not sure why you're having a hard time imagining it.
> Presorted input is a common case in general; that's why we have a
> check for it. That check adds overhead in the non-pre-sorted case to
> improve the pre-sorted case, and nobody'
On Sat, Aug 9, 2014 at 12:29 AM, Robert Haas wrote:
> On Fri, Aug 8, 2014 at 11:00 AM, Bruce Momjian wrote:
>> On Fri, Aug 8, 2014 at 08:51:13AM -0400, Robert Haas wrote:
>>> On Thu, Aug 7, 2014 at 12:07 PM, Abhijit Menon-Sen
>>> wrote:
>>> > At 2014-08-07 23:22:43 +0900, masao.fu...@gmail.com
value-to-be-compressed. (first_success_by is 1024 in the default set of
compression parameters.)
Curious idea: we could swap JEntry array and values: values in the
begining of type will be catched by pg_lzcompress. But we will need to
know offset of JEntry array, so header will grow up till 8
On 08/07/2014 10:12 PM, Stephen Frost wrote:
If you want the specific version, update your deb line. Don't complain
because you used the Debian repo that follows the Debian guidelines and
didn't like what you got- that's not going to change.
May I have a link? Because I would argue that the
Curious idea: we could swap JEntry array and values: values in the
begining of type will be catched by pg_lzcompress. But we will need to
know offset of JEntry array, so header will grow up till 8 bytes
(actually, it will be a varlena header!)
May be I wasn't clear:jsonb type will start from str
On Fri, Aug 8, 2014 at 11:02:26AM -0400, Tom Lane wrote:
> 2. Are we going to ship 9.4 without fixing this? I definitely don't see
> replacing pg_lzcompress as being on the agenda for 9.4, whereas changing
> jsonb is still within the bounds of reason.
FYI, pg_upgrade could be taught to refuse to
On 08/07/2014 08:32 PM, Fujii Masao wrote:
> This is not user-friendly. I'd like to propose the attached patch which
> introduces the infrastructure which allows us to specify the unit when
> setting INTEGER storage parameter like autovacuum_vacuum_cost_delay.
> Comment? Review?
No review, but tha
On 08/08/2014 08:02 AM, Tom Lane wrote:
> 2. Are we going to ship 9.4 without fixing this? I definitely don't see
> replacing pg_lzcompress as being on the agenda for 9.4, whereas changing
> jsonb is still within the bounds of reason.
>
> Considering all the hype that's built up around jsonb, shi
Hi
2014-08-08 13:58 GMT+02:00 Fujii Masao :
> On Fri, Aug 8, 2014 at 4:31 AM, Pavel Stehule
> wrote:
> >
> >
> >
> > 2014-08-07 7:10 GMT+02:00 Fujii Masao :
> >
> >> On Thu, Aug 7, 2014 at 6:26 AM, Pavel Stehule
> >> wrote:
> >> > Hello
> >> >
> >> > updated version patch in attachment
> >>
>
On Fri, Aug 8, 2014 at 9:14 PM, Teodor Sigaev wrote:
> Curious idea: we could swap JEntry array and values: values in the
>> begining of type will be catched by pg_lzcompress. But we will need to
>> know offset of JEntry array, so header will grow up till 8 bytes
>> (actually, it will be a varlen
On Fri, Aug 8, 2014 at 7:35 PM, Andrew Dunstan wrote:
>> I took a quick look and saw that this wouldn't be that easy to get around.
>> As I'd suspected upthread, there are places that do random access into a
>> JEntry array, such as the binary search in findJsonbValueFromContainer().
>> If we have
On 08/08/2014 06:17 AM, Tom Lane wrote:
> I looked into the issue reported in bug #11109. The problem appears to be
> that jsonb's on-disk format is designed in such a way that the leading
> portion of any JSON array or object will be fairly incompressible, because
> it consists mostly of a strict
On Fri, Aug 8, 2014 at 12:41 PM, Ants Aasma wrote:
> I don't think binary search is the main problem here. Objects are
> usually reasonably sized, while arrays are more likely to be huge. To
> make matters worse, jsonb -> int goes from O(1) to O(n).
I don't think it's true that arrays are more li
On Fri, Aug 8, 2014 at 12:06 PM, Josh Berkus wrote:
> One we ship 9.4, many users are going to load 100's of GB into JSONB
> fields. Even if we fix the compressability issue in 9.5, those users
> won't be able to fix the compression without rewriting all their data,
> which could be prohibitive.
I was not complaining; I think JSONB is awesome.
But I am one of those people who would like to put 100's of GB (or more)
JSON files into Postgres and I am concerned about file size and possible
future changes to the format.
On Fri, Aug 8, 2014 at 7:10 PM, Peter Geoghegan wrote:
> On Fri, Aug
On Sun, Jul 27, 2014 at 12:00 AM, Peter Geoghegan wrote:
> Robert pointed out a case where varying character case of an English
> word did not alter the primary level bytes (and thus the poor man's
> normalized key was the same). He acknowledged that most of the entropy
> of the first 8 bytes of t
I wrote:
> David Rowley writes:
>> The attached seems to fix the problem, but the whole thing makes me wonder
>> if this is even meant to be allowed? I was thinking that this might be a
>> good time to disallow this altogether, since it's already broken and looks
>> like it has been for about 11 y
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Stephen Frost writes:
> > * Tom Lane (t...@sss.pgh.pa.us) wrote:
> >> I looked into the issue reported in bug #11109. The problem appears to be
> >> that jsonb's on-disk format is designed in such a way that the leading
> >> portion of any JSON array or ob
* Joshua D. Drake (j...@commandprompt.com) wrote:
> On 08/07/2014 10:12 PM, Stephen Frost wrote:
> >If you want the specific version, update your deb line. Don't complain
> >because you used the Debian repo that follows the Debian guidelines and
> >didn't like what you got- that's not going to cha
* Bruce Momjian (br...@momjian.us) wrote:
> On Fri, Aug 8, 2014 at 11:02:26AM -0400, Tom Lane wrote:
> > 2. Are we going to ship 9.4 without fixing this? I definitely don't see
> > replacing pg_lzcompress as being on the agenda for 9.4, whereas changing
> > jsonb is still within the bounds of rea
* Josh Berkus (j...@agliodbs.com) wrote:
> On 08/08/2014 08:02 AM, Tom Lane wrote:
> > 2. Are we going to ship 9.4 without fixing this? I definitely don't see
> > replacing pg_lzcompress as being on the agenda for 9.4, whereas changing
> > jsonb is still within the bounds of reason.
> >
> > Consi
Stephen Frost writes:
> What about considering how large the object is when we are analyzing if
> it compresses well overall?
Hmm, yeah, that's a possibility: we could redefine the limit at which
we bail out in terms of a fraction of the object size instead of a fixed
limit. However, that risks
On 08/08/2014 08:45 PM, Tom Lane wrote:
Perhaps another options would be a new storage type which basically says
"just compress it, no matter what"? We'd be able to make that the
default for jsonb columns too, no?
Meh. We could do that, but it would still require adding arguments to
toast_com
Stephen Frost writes:
> * Joshua D. Drake (j...@commandprompt.com) wrote:
>> On 08/07/2014 10:12 PM, Stephen Frost wrote:
>>> If you want the specific version, update your deb line. Don't complain
>>> because you used the Debian repo that follows the Debian guidelines and
>>> didn't like what you
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Stephen Frost writes:
> > What about considering how large the object is when we are analyzing if
> > it compresses well overall?
>
> Hmm, yeah, that's a possibility: we could redefine the limit at which
> we bail out in terms of a fraction of the object s
Stephen Frost writes:
> I agree that we need to avoid changing jsonb's on-disk representation.
... post-release, I assume you mean.
> Have I missed where a good suggestion has been made about how to do that
> which preserves the binary-search capabilities and doesn't make the code
> much more di
This is 9.3:
peter=# \a
Output format is unaligned.
peter=# \a
Output format is aligned.
peter=# \x
Expanded display is on.
peter=# \x
Expanded display is off.
This is new in 9.4:
peter=# \a
Output format (format) is unaligned.
peter=# \a
Output format (format) is aligned.
peter=# \x
Expanded di
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> We only bump the SO version when we make a low-level ABI break; but even
> without doing that we could potentially have changed library behavior in
> ways that could negatively impact some applications.
We should definitely be paying attention for such ca
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Stephen Frost writes:
> > I agree that we need to avoid changing jsonb's on-disk representation.
>
> ... post-release, I assume you mean.
Yes.
> > Have I missed where a good suggestion has been made about how to do that
> > which preserves the binary-sea
9.3 pg_restore --help output:
-I, --index=NAME restore named index
-n, --schema=NAMErestore only objects in this schema
-P, --function=NAME(args)restore named function
-t, --table=NAME restore named table(s)
-T, --trigger=NAME restore nam
On Sat, Aug 9, 2014 at 10:44 AM, Peter Eisentraut wrote:
> -I, --index=NAME restore named indexes (repeat for multiple)
A single --index entry restores only one index, so what about
something like that:
-I, --index=NAME restore named index (repeat for multiple entries)
>
On Sat, Aug 9, 2014 at 6:15 AM, Tom Lane wrote:
>
> Stephen Frost writes:
> > What about considering how large the object is when we are analyzing if
> > it compresses well overall?
>
> Hmm, yeah, that's a possibility: we could redefine the limit at which
> we bail out in terms of a fraction of t
On Fri, Aug 8, 2014 at 08:25:04PM -0400, Stephen Frost wrote:
> * Bruce Momjian (br...@momjian.us) wrote:
> > On Fri, Aug 8, 2014 at 11:02:26AM -0400, Tom Lane wrote:
> > > 2. Are we going to ship 9.4 without fixing this? I definitely don't see
> > > replacing pg_lzcompress as being on the agend
On Sat, Aug 9, 2014 at 12:13 PM, Tom Lane wrote:
>
> So I'm thinking you're right: we should rewrite this code so that only
> indexes having all columns distinct can match, thereby ensuring that there
> is only one possible interpretation of the equality semantics for the
> foreign key.
>
>
I've
David Rowley writes:
> On Sat, Aug 9, 2014 at 12:13 PM, Tom Lane wrote:
>> So I'm thinking you're right: we should rewrite this code so that only
>> indexes having all columns distinct can match, thereby ensuring that there
>> is only one possible interpretation of the equality semantics for the
Thanks to all for the great info. We are new to postgresql and this discussion
has both instructed us and increased our respect for the database and the
community.
I am seeing a behavior that I don’t understand and hopefully you guys can clear
it up.
I am using AWS postgresql db.m3.2xlarge an
On Sat, Aug 9, 2014 at 3:34 PM, Tom Lane wrote:
> David Rowley writes:
> > On Sat, Aug 9, 2014 at 12:13 PM, Tom Lane wrote:
> >> So I'm thinking you're right: we should rewrite this code so that only
> >> indexes having all columns distinct can match, thereby ensuring that
> there
> >> is only
akapila wrote
> On Sat, Aug 9, 2014 at 6:15 AM, Tom Lane <
> tgl@.pa
> > wrote:
>>
>> Stephen Frost <
> sfrost@
> > writes:
>> > What about considering how large the object is when we are analyzing if
>> > it compresses well overall?
>>
>> Hmm, yeah, that's a possibility: we could redefine the
Peter Eisentraut-2 wrote
> 9.3 pg_restore --help output:
>
> -I, --index=NAME restore named index
> -n, --schema=NAMErestore only objects in this schema
> -P, --function=NAME(args)restore named function
> -t, --table=NAME restore named table(s)
> -
Hi all,
Please find attached a patch to add support of synchronous replication
for multiple standby servers. This is controlled by the addition of a
new GUC parameter called synchronous_standby_num, that makes server
wait for transaction commit on the first N standbys defined in
synchronous_standb
68 matches
Mail list logo
