Re: pread() and pwrite()

2018-11-07 Thread Thomas Munro
On Thu, Nov 8, 2018 at 4:27 AM Andrew Dunstan wrote: > On 11/7/18 10:05 AM, Jesper Pedersen wrote: > > On 11/7/18 9:30 AM, Tom Lane wrote: > >> I'm confused by this. Surely the pwrite-based code is writing > >> exactly the > >> same data as before. Do we have to conclude that valgrind is > >>

Re: pread() and pwrite()

2018-11-07 Thread Andrew Dunstan
On 11/7/18 10:05 AM, Jesper Pedersen wrote: Hi Tom, On 11/7/18 9:30 AM, Tom Lane wrote: I'm confused by this.  Surely the pwrite-based code is writing exactly the same data as before.  Do we have to conclude that valgrind is complaining about passing uninitialized data to pwrite() when it

Re: pread() and pwrite()

2018-11-07 Thread Jesper Pedersen
Hi Tom, On 11/7/18 9:30 AM, Tom Lane wrote: I'm confused by this. Surely the pwrite-based code is writing exactly the same data as before. Do we have to conclude that valgrind is complaining about passing uninitialized data to pwrite() when it did not complain about exactly the same thing for

Re: pread() and pwrite()

2018-11-07 Thread Andrew Dunstan
On 11/7/18 9:30 AM, Tom Lane wrote: Andrew Dunstan writes: On 11/7/18 7:26 AM, Jesper Pedersen wrote: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink=2018-11-07%2001%3A01%3A01 And lousyjack, which uses a slightly different way of calling valgrind, and thus got past initdb,

Re: pread() and pwrite()

2018-11-07 Thread Tom Lane
Andrew Dunstan writes: > On 11/7/18 7:26 AM, Jesper Pedersen wrote: >> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink=2018-11-07%2001%3A01%3A01 > And lousyjack, which uses a slightly different way of calling valgrind, > and thus got past initdb, found a bunch more: >

Re: pread() and pwrite()

2018-11-07 Thread Jesper Pedersen
Hi, On 11/7/18 7:26 AM, Jesper Pedersen wrote: On 11/6/18 4:04 PM, Thomas Munro wrote: On Wed, Nov 7, 2018 at 4:42 AM Jesper Pedersen Thanks!  Pushed.  I'll keep an eye on the build farm to see if anything breaks on Cygwin or some other frankenOS. There is [1] on Andres' skink setup.

Re: pread() and pwrite()

2018-11-07 Thread Andrew Dunstan
On 11/7/18 7:26 AM, Jesper Pedersen wrote: Hi Thomas, On 11/6/18 4:04 PM, Thomas Munro wrote: On Wed, Nov 7, 2018 at 4:42 AM Jesper Pedersen Thanks!  Pushed.  I'll keep an eye on the build farm to see if anything breaks on Cygwin or some other frankenOS. There is [1] on Andres' skink

Re: pread() and pwrite()

2018-11-07 Thread Jesper Pedersen
Hi Thomas, On 11/6/18 4:04 PM, Thomas Munro wrote: On Wed, Nov 7, 2018 at 4:42 AM Jesper Pedersen Thanks! Pushed. I'll keep an eye on the build farm to see if anything breaks on Cygwin or some other frankenOS. There is [1] on Andres' skink setup. Looking. [1]

Re: pread() and pwrite()

2018-11-06 Thread Thomas Munro
On Wed, Nov 7, 2018 at 4:42 AM Jesper Pedersen wrote: > Passes check-world, and includes the feedback on this thread. > > New status: Ready for Committer Thanks! Pushed. I'll keep an eye on the build farm to see if anything breaks on Cygwin or some other frankenOS. -- Thomas Munro

Re: pread() and pwrite()

2018-11-06 Thread Jesper Pedersen
: Alvaro Herrera writes: On 2018-Nov-04, Thomas Munro wrote: Here's a patch to add Windows support by supplying src/backend/port/win32/pread.c. Thoughts? Hmm, so how easy is to detect that somebody runs read/write on fds where pread/pwrite have occurred? I guess for data files it's easy

Re: pread() and pwrite()

2018-11-06 Thread Tom Lane
Thomas Munro writes: > On Tue, Nov 6, 2018 at 6:23 AM Tom Lane wrote: >> What I suggest is that we *not* try to make this a completely transparent >> substitute. Instead, make the functions exported by src/port/ be >> "pg_pread" and "pg_pwrite", ... > OK. But since we're using this from both

Re: pread() and pwrite()

2018-11-05 Thread Thomas Munro
2018-Nov-04, Thomas Munro wrote: > >> Here's a patch to add Windows support by supplying > >> src/backend/port/win32/pread.c. Thoughts? > > > Hmm, so how easy is to detect that somebody runs read/write on fds where > > pread/pwrite have occurred? I guess for d

Re: pread() and pwrite()

2018-11-05 Thread Tom Lane
Alvaro Herrera writes: > On 2018-Nov-04, Thomas Munro wrote: >> Here's a patch to add Windows support by supplying >> src/backend/port/win32/pread.c. Thoughts? > Hmm, so how easy is to detect that somebody runs read/write on fds where > pread/pwrite have occurred? I gues

Re: pread() and pwrite()

2018-11-05 Thread Alvaro Herrera
Please remove Tell from line 18 in fd.h. To Küssnacht with him! -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: pread() and pwrite()

2018-11-05 Thread Alvaro Herrera
On 2018-Nov-04, Thomas Munro wrote: > Here's a patch to add Windows support by supplying > src/backend/port/win32/pread.c. Thoughts? Hmm, so how easy is to detect that somebody runs read/write on fds where pread/pwrite have occurred? I guess for data files it's easy to detect since

Re: pread() and pwrite()

2018-11-05 Thread Jesper Pedersen
read Heikki's comment [1] as it would be ok to commit benefiting all platforms that has pread/pwrite. Here's a patch to add Windows support by supplying src/backend/port/win32/pread.c. Thoughts? If we do that, I suppose we might as well supply implementations for HP-UX 10.20 as well, and then we

Re: pread() and pwrite()

2018-11-05 Thread Thomas Munro
ikki's comment [1] as it would be ok to commit > > benefiting all platforms that has pread/pwrite. > > Here's a patch to add Windows support by supplying > src/backend/port/win32/pread.c. Thoughts? If we do that, I suppose we might as well supply implementations for HP-UX 10.20 as wel

Re: pread() and pwrite()

2018-11-03 Thread Thomas Munro
On Sat, Nov 3, 2018 at 2:07 AM Jesper Pedersen wrote: > This still applies, and passes make check-world. > > I wonder what the commit policy is on this, if the Windows part isn't > included. I read Heikki's comment [1] as it would be ok to commit > benefiting all platforms that h

Re: pread() and pwrite()

2018-11-02 Thread Jesper Pedersen
platforms that has pread/pwrite. The functions in [2] could be a follow-up patch as well. [1] https://www.postgresql.org/message-id/6cc7c8dd-29f9-7d75-d18a-99f19c076d10%40iki.fi [2] https://www.postgresql.org/message-id/c2f56d0a-cadd-3df1-ae48-b84dc8128c37%40redhat.com Best regards, Jesper

Re: pread() and pwrite()

2018-10-09 Thread Thomas Munro
course. Thanks, much nicer. Rebased. -- Thomas Munro http://www.enterprisedb.com 0001-Use-pread-pwrite-instead-of-lseek-read-write-v7.patch Description: Binary data

Re: pread() and pwrite()

2018-10-09 Thread Andrew Dunstan
On 10/09/2018 02:37 PM, Andres Freund wrote: On 2018-10-09 14:32:29 -0400, Andrew Dunstan wrote: On 10/08/2018 09:55 PM, Tom Lane wrote: Thomas Munro writes: Rebased again. Patches that touch AC_CHECK_FUNCS are fun like that! Yeah, I've been burnt by that too recently. It occurs to me

Re: pread() and pwrite()

2018-10-09 Thread Andres Freund
On 2018-10-09 14:32:29 -0400, Andrew Dunstan wrote: > > > On 10/08/2018 09:55 PM, Tom Lane wrote: > > Thomas Munro writes: > > > Rebased again. Patches that touch AC_CHECK_FUNCS are fun like that! > > Yeah, I've been burnt by that too recently. It occurs to me we could make > > that at least

Re: pread() and pwrite()

2018-10-09 Thread Andrew Dunstan
On 10/08/2018 09:55 PM, Tom Lane wrote: Thomas Munro writes: Rebased again. Patches that touch AC_CHECK_FUNCS are fun like that! Yeah, I've been burnt by that too recently. It occurs to me we could make that at least a little less painful if we formatted the macro with one line per

Re: pread() and pwrite()

2018-10-08 Thread Tom Lane
I wrote: > Thomas Munro writes: >> On Tue, Oct 9, 2018 at 2:55 PM Tom Lane wrote: >>> Yeah, I've been burnt by that too recently. It occurs to me we could make >>> that at least a little less painful if we formatted the macro with one >>> line per function name: >> +1, was about to suggest the

Re: pread() and pwrite()

2018-10-08 Thread Tom Lane
Thomas Munro writes: > On Tue, Oct 9, 2018 at 2:55 PM Tom Lane wrote: >> Yeah, I've been burnt by that too recently. It occurs to me we could make >> that at least a little less painful if we formatted the macro with one >> line per function name: >> >> AC_CHECK_FUNCS([ >> cbrt >>

Re: pread() and pwrite()

2018-10-08 Thread Thomas Munro
On Tue, Oct 9, 2018 at 2:55 PM Tom Lane wrote: > Thomas Munro writes: > > Rebased again. Patches that touch AC_CHECK_FUNCS are fun like that! > > Yeah, I've been burnt by that too recently. It occurs to me we could make > that at least a little less painful if we formatted the macro with one >

Re: pread() and pwrite()

2018-10-08 Thread Tom Lane
Thomas Munro writes: > Rebased again. Patches that touch AC_CHECK_FUNCS are fun like that! Yeah, I've been burnt by that too recently. It occurs to me we could make that at least a little less painful if we formatted the macro with one line per function name: AC_CHECK_FUNCS([ cbrt

Re: pread() and pwrite()

2018-10-08 Thread Thomas Munro
On Fri, Sep 28, 2018 at 2:03 AM Jesper Pedersen wrote: > Thanks for v5 too. Rebased again. Patches that touch AC_CHECK_FUNCS are fun like that! -- Thomas Munro http://www.enterprisedb.com 0001-Use-pread-pwrite-instead-of-lseek-read-write-v6.patch Description: Binary data

Re: pread() and pwrite()

2018-09-27 Thread Jesper Pedersen
Hi Thomas, On 9/18/18 9:48 PM, Thomas Munro wrote: It certainly wouldn't hurt... but more pressing to get this committed would be Windows support IMHO. I think the thing to do is to open files with the FILE_FLAG_OVERLAPPED flag, and then use ReadFile() and WriteFile() with an LPOVERLAPPED

Re: pread() and pwrite()

2018-09-27 Thread Thomas Munro
On Wed, Sep 19, 2018 at 1:48 PM Thomas Munro wrote: > On Fri, Sep 7, 2018 at 2:17 AM Jesper Pedersen > wrote: > > > This needs a rebase again. And again, due to the conflict with ppoll in AC_CHECK_FUNCS. -- Thomas Munro http://www.enterprisedb.com 0001-Use-pread-pwrite-instea

Re: pread() and pwrite()

2018-09-18 Thread Thomas Munro
h I'm not yet sure about). Then pgbench's 7-round-trip transaction makes only the strictly necessary 18 syscalls (every one an explainable network message, disk page or sync). Unpatched master has 5 extra lseek()s. -- Thomas Munro http://www.enterprisedb.com 0001-Use-pread-pwrite-instead-of-lseek-read-write-v4.patch Description: Binary data

Re: pread() and pwrite()

2018-09-06 Thread Jesper Pedersen
Hi, On 09/05/2018 02:42 PM, Jesper Pedersen wrote: On 07/26/2018 10:04 PM, Thomas Munro wrote: Done.  Rebased. This needs a rebase again. Would it be of benefit to update these call sites * slru.c - SlruPhysicalReadPage - SlruPhysicalWritePage * xlogutils.c - XLogRead *

Re: pread() and pwrite()

2018-09-01 Thread Thomas Munro
D fallback altogether. Are there any > > other platforms out there that don't have pread/pwrite that we care about? > > AFAICT, macOS has them as far back as we care about (prairiedog does). > HPUX 10.20 (gaur/pademelon) does not, so personally I'd like to keep > the lseek+read workaroun

Re: pread() and pwrite()

2018-07-26 Thread Thomas Munro
> Now that they're gone I agree that we could just drop the "At" suffix; > "at" suffix is also used by various POSIX functions to operate in a > specific directory which may just add to confusion. Done. Rebased. -- Thomas Munro http://www.enterprisedb.com 0001-Use-pread-pwrite-instead-of-lseek-read-write-v3.patch Description: Binary data

Re: pread() and pwrite()

2018-07-20 Thread Daniel Gustafsson
k altogether. Are there any >> other platforms out there that don't have pread/pwrite that we care about? > > AFAICT, macOS has them as far back as we care about (prairiedog does). > HPUX 10.20 (gaur/pademelon) does not, so personally I'd like to keep > the lseek+read workaround.

Re: pread() and pwrite()

2018-07-20 Thread Oskari Saarenmaa
On Thu, Jul 12, 2018 at 01:55:31PM +1200, Thomas Munro wrote: > A couple of years ago, Oskari Saarenmaa proposed a patch[1] to adopt > $SUBJECT. Last year, Tobias Oberstein argued again that we should do > that[2]. At the end of that thread there was a +1 from multiple > committers in support of

Re: pread() and pwrite()

2018-07-20 Thread Heikki Linnakangas
sets errno on error, then set up the macros so that Windows can use them as pread(), pwrite(). It might also be necessary to open all files with FILE_FLAG_OVERLAPPED. Does any Windows hacker have a bettter idea, and/or want to try to write that patch? Otherwise I'll eventually try to do some long distan

Re: pread() and pwrite()

2018-07-19 Thread Thomas Munro
; returnCode = read(vfdP->fd, buffer, amount); > +#endif > pgstat_report_wait_end(); > > This obviously lacks error handling for lseek(). Fixed. Updated the main WAL IO routines to use pread()/pwrite() too. Not super heavily tested yet. An idea for how to handle Wi

pread() and pwrite()

2018-07-11 Thread Thomas Munro
more work before the festivities begin. [1] https://www.postgresql.org/message-id/flat/a86bd200-ebbe-d829-e3ca-0c4474b2fcb7%40ohmu.fi [2] https://www.postgresql.org/message-id/flat/b8748d39-0b19-0514-a1b9-4e5a28e6a208%40gmail.com -- Thomas Munro http://www.enterprisedb.com 0001-Use-pread-pwrite-