On Fri, Oct 7, 2016 at 11:34 PM, Alfred Perlstein wrote:
>
> Are the batched writes all done before fsync is called?
>
In most cases, but if while writing, the wal segment (size - 16M) is
finished, we do fsync and then do the remaining writes and at end
again perform fsync.
> Are you sure that A
On Fri, Oct 7, 2016 at 3:02 PM, Tomas Vondra
wrote:
>
> I got access to a large machine with 72/144 cores (thanks to Oleg and
> Alexander from Postgres Professional), and I'm running the tests on that
> machine too.
>
> Results from Dilip's workload (with scale 300, unlogged tables) look like
> th
On Sun, Sep 11, 2016 at 11:05 AM, Peter Geoghegan wrote:
> So, while there are still a few loose ends with this revision (it
> should still certainly be considered WIP), I wanted to get a revision
> out quickly because V1 has been left to bitrot for too long now, and
> my schedule is very full for
Robert Haas writes:
> On Fri, Oct 7, 2016 at 7:14 PM, Tom Lane wrote:
>> BTW, OS X hasn't got libintl AFAICT:
>> What are you using to get past that?
> MacPorts.
OK, I'll take a look.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.
On 2016-10-07 17:12:45 -0400, Tom Lane wrote:
> Andres Freund writes:
> > It's not quite there yet, unfortunately. At the moment
> > pg_atomic_write_u32() is used for local buffers - and we explicitly
> > don't want that to be locking for temp buffers
> > (c.f. 6b93fcd149329d4ee7319561b30fc15a573c
On Fri, Oct 7, 2016 at 7:14 PM, Tom Lane wrote:
> I wrote:
>> Robert Haas writes:
>>> This broke the build for me. OS X Yosemite, 10.10.5.
>
>> Hm, probably means we need an explicit reference to -lintl when
>> linking libpqwalreceiver.so.
>
> BTW, OS X hasn't got libintl AFAICT:
>
> # ./configu
I wrote:
> Robert Haas writes:
>> This broke the build for me. OS X Yosemite, 10.10.5.
> Hm, probably means we need an explicit reference to -lintl when
> linking libpqwalreceiver.so.
BTW, OS X hasn't got libintl AFAICT:
# ./configure --enable-nls
...
checking for library containing bind_textd
Robert Haas writes:
> On Wed, Oct 5, 2016 at 11:04 PM, Tom Lane wrote:
>> Remove -Wl,-undefined,dynamic_lookup in macOS build.
> This broke the build for me. OS X Yosemite, 10.10.5.
Hm, probably means we need an explicit reference to -lintl when
linking libpqwalreceiver.so.
On Wed, Oct 5, 2016 at 11:04 PM, Tom Lane wrote:
> Remove -Wl,-undefined,dynamic_lookup in macOS build.
>
> We don't need this anymore, and it prevents build-time error checking
> that's usually good to have, so remove it. Undoes one change of commit
> cac765820.
>
> Unfortunately, it's much hard
On Fri, Oct 7, 2016 at 5:09 PM, Tom Lane wrote:
> What seems like a saner answer to me is to change the backend so that it
> will accept these ALTER commands in either order, with the same end state.
> The reason it throws an error now, IMO, is just so that blindly issuing
> the same ALTER ADD CO
On 10/07/2016 06:55 PM, Robert Haas wrote:
On Fri, Oct 7, 2016 at 6:46 AM, Heikki Linnakangas wrote:
Ouch. We should find and document an authoritative source for all the
mappings we have...
I think the next steps here are:
1. Find an authoritative source for all the existing mappings.
2. Gen
Andres Freund writes:
> It's not quite there yet, unfortunately. At the moment
> pg_atomic_write_u32() is used for local buffers - and we explicitly
> don't want that to be locking for temp buffers
> (c.f. 6b93fcd149329d4ee7319561b30fc15a573c6307).
Hm.
> Don't really have a great idea about addr
I've been looking into the misbehavior complained of here:
https://www.postgresql.org/message-id/CADbMkNPT-Jz5PRSQ4RbUASYAjocV_KHUWapR%2Bg8fNvhUAyRpxA%40mail.gmail.com
I originally thought this was pg_dump's fault, but I now think it's not,
or at least that changing the backend's behavior would be
On 09/07/2016 09:50 AM, Kyotaro HORIGUCHI wrote:
Hi,
I found an useless entry in utf8_to_sjis.map
{0xc19c, 0x815f},
which is apparently illegal as UTF-8 which postgresql
deliberately refuses. So it should be removed and the attached
patch does that. 0x815f(SJIS) is also mapped from 0xefbcbc
On Fri, Oct 7, 2016 at 11:51 AM, Jeff Janes wrote:
> What happens if you turn fsync off? Once a xlog file is fully written, it
> is immediately fsynced, even if the backend is holding WALWriteLock or
> wal_insert (or both) at the time, and even if synchrounous_commit is off.
> Assuming this machi
On Fri, Oct 7, 2016 at 1:39 PM, Andres Freund wrote:
> On 2016-10-06 14:38:56 -0400, Robert Haas wrote:
>> Obviously, there's a vast increase in TPS, and the backends seem to
>> spend most of their time actually doing work. ClientRead is now the
>> overwhelmingly dominant wait event, although wal
On 09/12/2016 11:08 AM, Michael Paquier wrote:
On Fri, Sep 9, 2016 at 6:49 AM, Andres Freund wrote:
On 2016-09-08 18:13:06 -0300, Alvaro Herrera wrote:
I can't vouch for the windows stuff, and
the invocations indeed look vulnerable. I'm not sure if the fix actually
matters on windows, given .
On 2016-10-06 00:06:33 -0400, Tom Lane wrote:
> Andres Freund writes:
> > Hm. After a long battle of head vs. wall I think I see what the problem
> > is. For the fallback atomics implementation I somehow had assumed that
> > pg_atomic_write_u32() doesn't need to lock, as it's just an unlocked
> >
This thread gets me thinking about the definition of "support." While
support in practice seems to primarily relate to fixes/updates to the
supported version itself it could just as well apply to interoperability
support by newer versions.
Given that the standard PostgreSQL upgrade process involve
On 10/7/16 10:42 AM, Andres Freund wrote:
Hi,
On 2016-10-06 20:52:22 -0700, Alfred Perlstein wrote:
This contention on WAL reminds me of another scenario I've heard about that
was similar.
To fix things what happened was that anyone that the first person to block
would be responsible for wri
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Realistically though, how much would code coverage info have helped?
> Code coverage on a back branch would not have told you about whether
> it leaves blobs behind in the final regression DB state. Code coverage
> on HEAD might have helped you notice some
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote:
> Stephen Frost wrote:
> > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote:
> > > Stephen Frost wrote:
> > >
> > > > What would be really nice would be code coverage information for the
> > > > back-branches also, as that would allow us to figur
Alvaro Herrera writes:
> Stephen Frost wrote:
>> I wasn't sure how far back it went, but if it's only to 9.1, then yes,
>> farther than that. Specifically, to as far back as we wish to provide
>> support for pg_dump, assuming it's reasonable to do so.
> Do you really want to go back to applying
Hi,
On 2016-10-06 20:52:22 -0700, Alfred Perlstein wrote:
> This contention on WAL reminds me of another scenario I've heard about that
> was similar.
>
> To fix things what happened was that anyone that the first person to block
> would be responsible for writing out all buffers for anyone block
Hi,
On 2016-10-06 14:38:56 -0400, Robert Haas wrote:
> Obviously, there's a vast increase in TPS, and the backends seem to
> spend most of their time actually doing work. ClientRead is now the
> overwhelmingly dominant wait event, although wal_insert and
> WALWriteLock contention is clearly still
Hi,
On 2016-10-06 18:15:58 -0400, Robert Haas wrote:
> That's pretty tight, especially since I now notice Andres also left a
> postmaster running on this machine back in April, with
> shared_buffers=8GB.
Oops, sorry for that.
- Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@post
Stephen Frost writes:
> * Tom Lane (t...@sss.pgh.pa.us) wrote:
>> It might be a good idea to retroactively modify 9.1-9.3 so that there
>> are some blobs in the final state, for purposes of testing pg_dump and
>> pg_upgrade.
> I certainly think that would be a good idea. I thought we had been
>
Stephen Frost wrote:
> * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote:
> > Stephen Frost wrote:
> >
> > > What would be really nice would be code coverage information for the
> > > back-branches also, as that would allow us to figure out what we're
> > > missing coverage for. I realize that we
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote:
> Stephen Frost wrote:
>
> > What would be really nice would be code coverage information for the
> > back-branches also, as that would allow us to figure out what we're
> > missing coverage for. I realize that we don't like adding new things to
Stephen Frost wrote:
> What would be really nice would be code coverage information for the
> back-branches also, as that would allow us to figure out what we're
> missing coverage for. I realize that we don't like adding new things to
> back-branches as those changes could impact packagers, but
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Stephen Frost writes:
> > Ugh. Thanks for fixing. I had tested back to 7.4 with the regression
> > tests but either those didn't include blobs or something got changed
> > after my testing and I didn't re-test all the way back when I should
> > have.
>
>
Robert Haas wrote:
> On Wed, Oct 5, 2016 at 10:58 AM, Francisco Olarte
> wrote:
> > On Tue, Oct 4, 2016 at 7:50 PM, Robert Haas wrote:
> >> On Mon, Oct 3, 2016 at 5:44 PM, Alvaro Herrera
> >> wrote:
> > ...
> >>> I wonder if the real answer isn't just to disallow -f with parallel
> >>> vacuumin
Le 03/10/2016 à 16:09, Christoph Berg a écrit :
> Hi Gilles,
>
> I've just tried v4 of the patch. The OID you picked for
> pg_current_logfile doesn't work anymore, but after increasing it
> randomly by 1, it compiles. I like the added functionality,
> especially that "select pg_read_file(pg_cur
Robert Haas writes:
> On Fri, Oct 7, 2016 at 6:46 AM, Heikki Linnakangas wrote:
>> Ouch. We should find and document an authoritative source for all the
>> mappings we have...
>>
>> I think the next steps here are:
>>
>> 1. Find an authoritative source for all the existing mappings.
>> 2. Gener
Robert Haas writes:
> On Fri, Oct 7, 2016 at 11:34 AM, Tom Lane wrote:
>> Greg Stark writes:
>>> For another there may be binary-only applications or drivers out there
>>> that are using V2 for whatever reason.
>> The problem with letting it just sit there is that we're not, in fact,
>> testing
On Fri, Oct 7, 2016 at 8:24 AM, Anastasia Lubennikova <
a.lubennik...@postgrespro.ru> wrote:
> Hi, hackers!
>
> I am examining various global variables in ShmemVariableCache,
> pg_control and pg_controldata. To force and debug xid wraparound,
> I've implemented a function, that allows to set next
On Fri, Oct 7, 2016 at 6:46 AM, Heikki Linnakangas wrote:
> Ouch. We should find and document an authoritative source for all the
> mappings we have...
>
> I think the next steps here are:
>
> 1. Find an authoritative source for all the existing mappings.
> 2. Generate the radix tree files directl
On 10/07/2016 03:05 PM, Michael Paquier wrote:
On Fri, Oct 7, 2016 at 8:54 PM, Heikki Linnakangas wrote:
I'm a bit surprised no-one else has reported this yet. Have I missed a
report?
Andres has reported that a couple of weeks back, on Debian testing like you:
https://www.postgresql.org/messa
On Thu, Oct 6, 2016 at 11:38 AM, Robert Haas wrote:
>
> Next, I tried lowering the scale factor to something that fits in
> shared buffers. Here are the results at scale factor 300:
>
> 14 Lock| tuple
> 22 LWLockTranche | lock_manager
> 39 LWLockNamed | WALBuf
Anastasia Lubennikova writes:
> What is the purpose of having CheckPoint stored in control file then?
IIRC, we use some but not all of the fields. It seemed prudent (and
simpler) to keep a copy of the whole thing.
regards, tom lane
--
Sent via pgsql-hackers mailing li
On Fri, Oct 7, 2016 at 11:34 AM, Tom Lane wrote:
> Greg Stark writes:
>> On Fri, Oct 7, 2016 at 3:06 PM, Tom Lane wrote:
>>> In the same line, maybe we should kill libpq's support for V2 protocol
>>> (which would make the cutoff 7.4). And maybe the server's support too,
>>> though that wouldn't
Stephen Frost writes:
> Ugh. Thanks for fixing. I had tested back to 7.4 with the regression
> tests but either those didn't include blobs or something got changed
> after my testing and I didn't re-test all the way back when I should
> have.
It looks like the final state of the regression test
Robert:
On Fri, Oct 7, 2016 at 3:20 PM, Robert Haas wrote:
> On Wed, Oct 5, 2016 at 10:58 AM, Francisco Olarte
> wrote:
>> On Tue, Oct 4, 2016 at 7:50 PM, Robert Haas wrote:
>>> On Mon, Oct 3, 2016 at 5:44 PM, Alvaro Herrera
>>> wrote:
>> ...
I wonder if the real answer isn't just to dis
Greg Stark writes:
> On Fri, Oct 7, 2016 at 3:06 PM, Tom Lane wrote:
>> In the same line, maybe we should kill libpq's support for V2 protocol
>> (which would make the cutoff 7.4). And maybe the server's support too,
>> though that wouldn't save very much code. The argument for cutting this
>>
Hi, hackers!
I am examining various global variables in ShmemVariableCache,
pg_control and pg_controldata. To force and debug xid wraparound,
I've implemented a function, that allows to set nextXid value in control
file manually, but it doesn't work as expected.
After an investigation I found t
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Amit Langote writes:
> > Just noticed that the getBlobs() query does not work for a 7.3 server
> > (maybe <= 7.3) due to the following change in commit 23f34fa4 [1]:
>
> Ugh.
>
> > I could fix that using the attached patch.
>
> There's more wrong than th
On Fri, Oct 7, 2016 at 3:06 PM, Tom Lane wrote:
> pg_dump alleges support for dumping from servers back to 7.0. Would v10
> be a good time to remove some of that code? It's getting harder and
> harder to even compile those ancient branches, let alone get people to
> test against them (cf. 4806f2
On Fri, Oct 7, 2016 at 10:06 AM, Tom Lane wrote:
> pg_dump alleges support for dumping from servers back to 7.0. Would v10
> be a good time to remove some of that code? It's getting harder and
> harder to even compile those ancient branches, let alone get people to
> test against them (cf. 4806f
06.10.2016 20:59, Pavan Deolasee:
I investigated a bug report from one of our customers and it looked
very similar to previous bug reports here [1], [2], [3] (and probably
more). In these reports, the error looks something like this:
ERROR: could not read block 28991 in file "base/16390/57202
On Fri, Oct 7, 2016 at 4:06 PM, Tom Lane wrote:
> pg_dump alleges support for dumping from servers back to 7.0. Would v10
> be a good time to remove some of that code? It's getting harder and
> harder to even compile those ancient branches, let alone get people to
> test against them (cf. 4806f
Robert Haas writes:
> On Fri, Oct 7, 2016 at 9:39 AM, Tom Lane wrote:
>> There's more wrong than that, as you'd notice if you tried dumping
>> a DB that actually had some LOs in it :-(. This obviously wasn't
>> tested on anything older than 9.0.
> Back in 2014, we talked about removing support
On Fri, Oct 7, 2016 at 9:39 AM, Tom Lane wrote:
> Amit Langote writes:
>> Just noticed that the getBlobs() query does not work for a 7.3 server
>> (maybe <= 7.3) due to the following change in commit 23f34fa4 [1]:
>
> Ugh.
>
>> I could fix that using the attached patch.
>
> There's more wrong tha
pg_dump alleges support for dumping from servers back to 7.0. Would v10
be a good time to remove some of that code? It's getting harder and
harder to even compile those ancient branches, let alone get people to
test against them (cf. 4806f26f9). My initial thought is to cut support
for pre-7.3 o
On Thu, Oct 6, 2016 at 5:07 PM, Tom Lane wrote:
> Robert Haas writes:
>> Alternatively, get a bigger box. :-)
>
> So what's it take to get access to hydra?
Send me a private email with your .ssh key.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Amit Langote writes:
> Just noticed that the getBlobs() query does not work for a 7.3 server
> (maybe <= 7.3) due to the following change in commit 23f34fa4 [1]:
Ugh.
> I could fix that using the attached patch.
There's more wrong than that, as you'd notice if you tried dumping
a DB that actual
On Wed, Oct 5, 2016 at 2:09 PM, Andres Freund wrote:
> On 2016-10-04 21:40:29 -0400, Tom Lane wrote:
>> Andres Freund writes:
>> > On 2016-10-05 09:38:15 +0900, Michael Paquier wrote:
>> >> The existing interface of MemoryContextAlloc do not care much about
>> >> anything except Size, so I'd just
On Wed, Oct 5, 2016 at 10:58 AM, Francisco Olarte
wrote:
> On Tue, Oct 4, 2016 at 7:50 PM, Robert Haas wrote:
>> On Mon, Oct 3, 2016 at 5:44 PM, Alvaro Herrera
>> wrote:
> ...
>>> I wonder if the real answer isn't just to disallow -f with parallel
>>> vacuuming.
>> Seems like we should figure o
On Thu, Oct 6, 2016 at 8:40 PM, Jeff Janes wrote:
> In commit 37484ad2aacef5ec7, you changed the way that frozen tuples were
> represented, so that we could make freezing more aggressive without losing
> forensic evidence. But I don't think we ever did anything to actually make
> the freezing mor
Robert,
This contention on WAL reminds me of another scenario I've heard about
that was similar.
To fix things what happened was that anyone that the first person to
block would be responsible for writing out all buffers for anyone
blocked behind "him".
The for example if you have many thr
This was my first test which had help Gerdan.
I did some tests and found nothing special. The stated resource is implemented
correctly.
He passes all regression tests and enables the use of the new features
specified.
The test was initially performed to verify that the features exist, however
The following review has been posted through the commitfest application:
make installcheck-world: tested, passed
Implements feature: tested, passed
Spec compliant: tested, passed
Documentation:not tested
This was my first test which had help Gerdan.
I did some tests a
On Fri, Oct 7, 2016 at 8:54 PM, Heikki Linnakangas wrote:
> I think we should fix this by the attached. Any better ideas?
Updating PG_PROVE_FLAGS in src/Makefile.global.in, because any test
suite having its own .pm file will fail. It is as well necessary to
refresh $ENV{PERL5LIB} in vcregress.pl.
On my system running Debian stretch (= testing):
~/git-sandbox-pgsql/96stable/src/bin/pg_rewind (REL9_6_STABLE)$ make check
rm -rf '/home/heikki/git-sandbox-pgsql/96stable'/tmp_install
/bin/mkdir -p '/home/heikki/git-sandbox-pgsql/96stable'/tmp_install/log
make -C '../../..'
DESTDIR='/home/heikk
On 2016/10/07 18:27, Ashutosh Bapat wrote:
> It's allowed to specify an non-default opclass in partition by clause,
> but I do not see any testcase testing the same. Can you please add
> one.
OK, I will add some tests related to that.
Thanks,
Amit
--
Sent via pgsql-hackers mailing list (pgsq
On 10/07/2016 01:34 PM, Michael Banck wrote:
Hi,
ISTM that pg_rewind's --source-pgdata and --source-server options are
mutually exclusive, and the synopsis in the documentation seems to
indicate that as well:
|pg_rewind [option...] {-D | --target-pgdata} directory
|{--source-pgdata=directory |
On 10/07/2016 11:36 AM, Kyotaro HORIGUCHI wrote:
The radix conversion function and map conversion script became
more generic than the previous state. So I could easily added
radix conversion of EUC_JP in addition to SjiftJIS.
nm -S said that the size of radix tree data for sjis->utf8
conversion
Hi,
ISTM that pg_rewind's --source-pgdata and --source-server options are
mutually exclusive, and the synopsis in the documentation seems to
indicate that as well:
|pg_rewind [option...] {-D | --target-pgdata} directory
|{--source-pgdata=directory | --source-server=connstr}
However, there is no
Hello,
I've faced multiple times a painful limitation with pg_stat_statements.
Queries are normalized based on the textual SQL statements, and the
queryid is computed using objects' oids. So for instance with different
search_path settings, we can end up with the same normalized query but
differen
On 10/05/2016 10:03 AM, Amit Kapila wrote:
On Wed, Oct 5, 2016 at 12:05 PM, Tomas Vondra
wrote:
Hi,
After collecting a lot more results from multiple kernel versions, I can
confirm that I see a significant improvement with 128 and 192 clients,
roughly by 30%:
64
It's allowed to specify an non-default opclass in partition by clause,
but I do not see any testcase testing the same. Can you please add
one.
On Fri, Oct 7, 2016 at 2:50 PM, Amit Langote
wrote:
> On 2016/10/07 17:33, Rajkumar Raghuwanshi wrote:
>> On Thu, Oct 6, 2016 at 12:44 PM, Amit Langote wr
On 10/06/2016 07:36 AM, Pavan Deolasee wrote:
On Wed, Oct 5, 2016 at 1:43 PM, Tomas Vondra
mailto:tomas.von...@2ndquadrant.com>> wrote:
...
I can confirm the significant speedup, often by more than 75%
(depending on number of indexes, whether the data set fits into RAM,
etc.). S
On Fri, Oct 7, 2016 at 6:46 AM, Robert Haas wrote:
> On Thu, Oct 6, 2016 at 3:56 PM, Jeff Janes wrote:
>>> Sure, I could handle each case separately, but the goal of this patch
>>> (as hinted at in the Subject) is to generalize all the different tasks
>>> we've been giving to VACUUM. The only mi
On Thu, Oct 6, 2016 at 12:44 PM, Amit Langote wrote:
>
> Attached revised patches. Also, includes a fix for an issue reported by
> Rajkumar Raghuwanshi [1] which turned out to be a bug in one of the later
> patches. I will now move on to addressing the comments on patch 0003.
>
> Thanks a lot f
On Fri, Oct 7, 2016 at 9:40 AM, Jeff Janes wrote:
> On Thu, Oct 6, 2016 at 2:46 PM, Robert Haas wrote:
>>
>>
>> > Also, I think that doing more counts which get amalgamated into the same
>> > threshold, rather than introducing another parameter, is a bad thing. I
>> > have insert-mostly, most of
On Thu, Oct 6, 2016 at 2:52 PM, Amit Langote
wrote:
> On 2016/10/06 17:45, Ashutosh Bapat wrote:
>> On Thu, Oct 6, 2016 at 1:34 PM, Masahiko Sawada
>> wrote:
>>> On Thu, Oct 6, 2016 at 1:41 PM, Ashutosh Bapat
>>> wrote:
> My understanding is that basically the local server can not return
>
75 matches
Mail list logo