Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-22 Thread Jan Kara
> mind waiting an almost arbitrarily long time for fsync to complete if > other processes could still get their I/O requests serviced in a > reasonable amount of time in the meanwhile. As I wrote in some other email in this thread, using IO priorities for data file checkpoint might be

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-22 Thread Jan Kara
On Wed 22-01-14 09:07:19, Dave Chinner wrote: > On Tue, Jan 21, 2014 at 09:20:52PM +0100, Jan Kara wrote: > > > If we're forcing the WAL out to disk because of transaction commit or > > > because we need to write the buffer protected by a certain WAL record > &g

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-16 Thread Jan Kara
On Wed 15-01-14 10:12:38, Robert Haas wrote: > On Wed, Jan 15, 2014 at 4:35 AM, Jan Kara wrote: > > Filesystems could in theory provide facility like atomic write (at least up > > to a certain size say in MB range) but it's not so easy and when there are > > no str

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-16 Thread Jan Kara
On Wed 15-01-14 21:37:16, Robert Haas wrote: > On Wed, Jan 15, 2014 at 8:41 PM, Jan Kara wrote: > > On Wed 15-01-14 10:12:38, Robert Haas wrote: > >> On Wed, Jan 15, 2014 at 4:35 AM, Jan Kara wrote: > >> > Filesystems could in theory provide facility like ato

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-15 Thread Jan Kara
On Wed 15-01-14 14:38:44, Hannu Krosing wrote: > On 01/15/2014 02:01 PM, Jan Kara wrote: > > On Wed 15-01-14 12:16:50, Hannu Krosing wrote: > >> On 01/14/2014 06:12 PM, Robert Haas wrote: > >>> This would be pretty similar to copy-on-write, except > >>

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-15 Thread Jan Kara
is running...). So I'm somewhat skeptical you will be able to tune dirty_background_bytes frequently in a useful way. Honza -- Jan Kara SUSE Labs, CR -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-15 Thread Jan Kara
plex unnecessarily. OTOH without widespread atomic write support I understand application developers have similar stance. So it's kind of chicken and egg problem. BTW, e.g. ext3/4 has quite a bit of the infrastructure in place due to its data=journal mode so if someone on the PostgreSQL side wanted

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-14 Thread Jan Kara
On Tue 14-01-14 10:04:16, Robert Haas wrote: > On Tue, Jan 14, 2014 at 5:00 AM, Jan Kara wrote: > > I thought that instead of injecting pages into pagecache for aging as you > > describe in 3), you would mark pages as volatile (i.e. for reclaim by > > kernel) through vrange

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-14 Thread Jan Kara
On Tue 14-01-14 06:42:43, Kevin Grittner wrote: > First off, I want to give a +1 on everything in the recent posts > from Heikki and Hannu. > > Jan Kara wrote: > > > Now the aging of pages marked as volatile as it is currently > > implemented needn't be perfect f

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-14 Thread Jan Kara
great gain...). However, if you really issue the IO from the thread with low priority, it will have low priority. So specifically if you call fsync() from a thread with low IO priority, the flushing done by fsync() will have this low IO priority. Similarly if you called sync_file_range() once

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-14 Thread Jan Kara
currently implemented needn't be perfect for your needs but you still have time to influence what gets implemented... Actually developers of the vrange() syscall were specifically looking for some ideas what to base aging on. Currently I think it is first marked - first evicted. Honza -- Jan Kara SUSE Labs, CR -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Jan Kara
trying to achieve. Note to postgres guys: I think you should have a look at the proposed 'vrange' system call. The latest posting is here: http://www.spinics.net/lists/linux-mm/msg67328.html. It contains a rather detailed description of the feature. And if the feature looks good to you, you can add your 'me to' plus if anyone would be willing to try that out with postgress that would be most welcome (although I understand you might not want to burn your time on experimental kernel feature). Honza -- Jan Kara SUSE Labs, CR -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Jan Kara
ut of free IO requests, POSIX_FADV_WILLNEED can block for a significant amount of time. Honza -- Jan Kara SUSE Labs, CR -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.p