Re: [HACKERS] OS scheduler bugs affecting high-concurrency contention
Hi, On 04/16/2016 04:15 PM, Kevin Grittner wrote: There is a paper that any one interested in performance at high concurrency, especially in Linux, should read[1]. While doing other work, a group of researchers saw behavior that they suspected was due to scheduler bugs in Linux. There were no tools that made proving that practical, so they developed such a tool set and used it to find four bugs in the Linux kernel which were introduced in these releases, have not yet been fixed, and have this following maximum impact when running NAS benchmarks, based on running with and without the researchers' fixes for the bugs: 2.6.32: 22% 2,6.38: 13x 3.9: 27x 3.19: 138x That's right -- one of these OS scheduler bugs in production versions of Linux can make one of NASA's benchmarks run for 138 times as long as it does without the bug. I don't feel that I can interpret the results of any high-concurrency benchmarks in a meaningful way without knowing which of these bugs were present in the OS used for the benchmark. Just as an example, it is helpful to know that the benchmarks Andres presented were run on 3.16, so it would have three of these OS bugs affecting results, but not the most severe one. I encourage you to read the paper an draw your own conclusions. Anyway, please don't confuse this thread with the one on the "snapshot too old" patch -- I am still working on that and will post results there when they are ready. Thanks for the link, appreciated. On slightly related topic, Jens Axboe proposed a patchset [1] to improve the performance of background buffered writeback. On Lwn.net an article about the issue at hand has been recently published [2]. Maybe this work could somewhat solve the problem experienced by PostgreSQL users while checkpoint process flushes all pending changes to disk and recycles the transaction logs. -- Andrea Suisani suis...@opinioni.net Demetra opinioni.net srl [1] "[PATCHSET v3][RFC] Make background writeback not suck" http://thread.gmane.org/gmane.linux.kernel/2186732 [2] "Toward less-annoying background writeback" https://lwn.net/SubscriberLink/682582/93d9e5b6bed03a32/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery inconsistencies, standby much larger than primary
Hi all, On 02/12/2014 08:27 PM, Greg Stark wrote: On Wed, Feb 12, 2014 at 6:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Greg Stark st...@mit.edu writes: For what it's worth I've confirmed the bug in wal-e caused the initial problem. Huh? Bug in wal-e? What bug? WAL-E actually didn't restore a whole 1GB file due to a transient S3 problem, in fact a bunch of them. It's remarkable that Postgres kept going with that much data missing. But the arithmetic worked out on the case I checked it on, which was the last one that I just sent the xlog record for last night. In that case there was precisely one segment missing and the relation was extended by the number of segments you would expect if it filled in that missing segment and then jumped to the end of the relation. sorry for interrupting, but did we already notify wal-e's maintainer? Andrea ps cc:ed Daniel -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] logical changeset generation v3
Il 16/11/2012 05:34, Michael Paquier ha scritto: Do you have a git repository or something where all the 14 patches are applied? I would like to test the feature globally. Sorry I recall that you put a link somewhere but I cannot remember its email... http://archives.postgresql.org/pgsql-hackers/2012-11/msg00686.php Andrea -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] swapcache-style cache?
On 02/28/2012 08:54 AM, Andrea Suisani wrote: On 02/28/2012 04:52 AM, Rob Wultsch wrote: On Wed, Feb 22, 2012 at 2:31 PM, jamesja...@mansionfamily.plus.com wrote: Has anyone considered managing a system like the DragonFLY swapcache for a DBMS like PostgreSQL? https://www.facebook.com/note.php?note_id=388112370932 in the same vein: http://bcache.evilpiepirate.org/ [cut] it was submitted to linux kernel mailing list a bunch of time, the last one: https://lkml.org/lkml/2011/9/10/13 forgot to mention another good write-up https://lwn.net/Articles/394672/ Andrea -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] swapcache-style cache?
On 02/28/2012 04:52 AM, Rob Wultsch wrote: On Wed, Feb 22, 2012 at 2:31 PM, jamesja...@mansionfamily.plus.com wrote: Has anyone considered managing a system like the DragonFLY swapcache for a DBMS like PostgreSQL? https://www.facebook.com/note.php?note_id=388112370932 in the same vein: http://bcache.evilpiepirate.org/ from the main page: Bcache is a patch for the Linux kernel to use SSDs to cache other block devices. It's analogous to L2Arc for ZFS, but Bcache also does writeback caching, and it's filesystem agnostic. It's designed to be switched on with a minimum of effort, and to work well without configuration on any setup. By default it won't cache sequential IO, just the random reads and writes that SSDs excel at. It's meant to be suitable for desktops, servers, high end storage arrays, and perhaps even embedded. it was submitted to linux kernel mailing list a bunch of time, the last one: https://lkml.org/lkml/2011/9/10/13 Andrea -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [OT?] Time-zone database down [was: Re: timezone buglet?]
On 10/05/2011 07:37 AM, Tom Lane wrote: davegda...@sonic.net writes: Postgresql 9.0.4 has the timezone: America/Blanc-Sablon However other sources seem to spell this with an underscore instead of dash: America/Blanc_Sablon I don't know what other sources you're consulting, but Blanc-Sablon is the way it appears in the Olson timezone database, and that's what we follow. Speaking of Olson tz database, I've just stumbled across this post and I thought it would be worthy to report it here: http://blog.joda.org/2011/10/today-time-zone-database-was-closed.html We're not going to get into the business of editorializing on their information. If you want to fool with it locally, look into the .../share/timezone/ directory. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] fstat vs. lseek
hi On 08/08/2011 07:50 PM, Robert Haas wrote: On Mon, Aug 8, 2011 at 1:31 PM, Andres Freundand...@anarazel.de wrote: If its ok I will write a mail to lkml referencing this thread and your numbers inline (with attribution obviously). That would be great. Please go ahead. I've just stumbled across this thread on lkml [1] Improve lseek scalability v3. and I thought to ping pgsql hackers list just in case, more to the point they're asking are there any real workloads which care [Make generic lseek lockless safe] maybe I've got it wrong but it seems somewhat related to what has been discussed here and also in Robert Haas's Linux and glibc Scalability blog post [1]. [cut] Andrea [1] https://lkml.org/lkml/2011/9/15/399 [2] http://rhaas.blogspot.com/2011/08/linux-and-glibc-scalability.html -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Debian readline/libedit breakage
On 02/10/2011 11:34 PM, Joshua D. Drake wrote: Hello, Per: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=607109 It seems we may have a problem to consider. As far as I know, we are the only major platform that supports libedit but our default is readline. Unfortunately readline is not compatible with OpenSSL (apparently?) licensing. This seems that it may be a problem for us considering the pre-package builds we do. What does everyone think? Should we work on getting libedit up to snuff? JD on lwn.net there's a nice summary about this debate you can read it at (*): http://lwn.net/SubscriberLink/428111/36b6b26832f4309d/ Andrea -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] limiting hint bit I/O
On 01/18/2011 06:44 PM, Robert Haas wrote: On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncuremmonc...@gmail.com wrote: a few weeks back I hacked an experimental patch that removed the hint bit action completely. the results were very premature and/or incorrect, but my initial findings suggested that hint bits might not be worth the cost from performance standpoint. i'd like to see some more investigation in this direction before going with a complex application mechanism (although that would be beneficial vs the status quo). I think it's not very responsible to allege that hint bits aren't providing a benefit without providing the patch that you used and the tests that you ran. maybe I'm wrong but it seems it did post an experimental patch and also a tests used, see: http://archives.postgresql.org/pgsql-hackers/2010-12/msg01897.php This is a topic that needs careful analysis, and I think that saying hint bits don't provide a benefit... maybe... doesn't do anything but confuse the issue. How about doing some tests with the patch from my OP and posting the results? If removing hint bits entirely doesn't degrade performance, then surely the less-drastic approach I've taken here ought to be OK too. But in my testing, it didn't look too good. Andrea -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] limiting hint bit I/O
On 01/19/2011 09:03 AM, Andrea Suisani wrote: On 01/18/2011 06:44 PM, Robert Haas wrote: On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncuremmonc...@gmail.com wrote: a few weeks back I hacked an experimental patch that removed the hint bit action completely. the results were very premature and/or incorrect, but my initial findings suggested that hint bits might not be worth the cost from performance standpoint. i'd like to see some more investigation in this direction before going with a complex application mechanism (although that would be beneficial vs the status quo). I think it's not very responsible to allege that hint bits aren't providing a benefit without providing the patch that you used and the tests that you ran. maybe I'm wrong but it seems it did post an experimental patch and also ^^ he a tests used, see: ^^ the sorry for the typos (not enough caffeine I suppose :) Andrea -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PG 9.1 tentative timeline
On 06/11/2010 02:25 PM, Devrim GÜNDÜZ wrote: On Fri, 2010-06-11 at 19:56 +0800, Pr, Solaiyappan (NSN - IN/Bangalore) wrote: Also, is there any synchronous replication patch planned for the PostgreSQL 9.0 version? Cybertec announced new version of Cybercluster, which includes sync replication -- I haven't tested it though. look at this thread for more info http://archives.postgresql.org/message-id/b058e23346b050bd0fa2d3981af6da58.squir...@internal.cybertec.at Andrea -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Testing of parallel restore with current snapshot
On 27/02/2010 07:52, Gokulakannan Somasundaram wrote: Tom, I just took the patch, but it seems to be in binary format. Can you send me the patch to me? gunzip shuould do the trick Thanks, Gokul. On Sat, May 30, 2009 at 3:12 AM, Tom Lanet...@sss.pgh.pa.us wrote: Josh Berkusj...@agliodbs.com writes: Tom, Is anyone interested enough to try it if I code it? If you're patient for results, sure. I seem to be doing a customer migration or upgrade every week now, so it wouldn't take me long to have a test subject with a fairly complex database. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Mysql.whynot or PG vs MySQL comparison table?
Greg Stark wrote: Before i duplicate work does anyone have a MySQL.whynot or Postgres versus MySQL comparison table? http://sql-info.de/ not so up-to-date still it contanins some kind of comparisons... between mysql and postgresql andrea -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
