On Wed, Nov 28, 2018 at 06:36:25AM +0900, Michael Paquier wrote:
> Yes, let's do so rather sooner than later if there are no objections
> from others. I am going to ping the other thread about what's discussed
> here additionally in case folks missed what you reported. Fixing the
> temp file
On Tue, Nov 27, 2018 at 04:26:45PM +0100, Michael Banck wrote:
> Oh, I kinda followed that thread a bit, but I think that patch fixes
> things more by matter of moving code around, at least I haven't noticed
> that tablespaces were explicitly claimed to be fixed in that thread.
That may have been
Hi,
Am Dienstag, den 27.11.2018, 22:52 +0900 schrieb Michael Paquier:
> On Tue, Nov 27, 2018 at 02:09:05PM +0100, Michael Banck wrote:
> > I had a quick look at fixing this but did not manage to immediately come
> > up with a solution, so posting here for now.
>
> If you look at another thread,
On Tue, Nov 27, 2018 at 02:09:05PM +0100, Michael Banck wrote:
> I had a quick look at fixing this but did not manage to immediately come
> up with a solution, so posting here for now.
If you look at another thread, the patch posted on the top would
actually solve this issue:
Hi,
Am Freitag, den 19.10.2018, 22:50 +0900 schrieb Michael Paquier:
> On Wed, Oct 17, 2018 at 05:30:05PM -0400, Andrew Dunstan wrote:
> Thanks. This is now committed after some tweaks to the comments, a bit
> earlier than I thought first.
I found an issue with this [d55241af7, "Use whitelist
On Mon, Nov 19, 2018 at 07:11:19PM +0100, Michael Banck wrote:
> First off, I think those fail_corrupt files should have different
> filenames than the empty ones above (I left `9_vm.123' as an
> example of a duplicated file).
A comment about that is a good idea. So added.
> So either we
Hi,
On Sun, Oct 14, 2018 at 10:59:02AM +0900, Michael Paquier wrote:
> On Sat, Oct 13, 2018 at 05:53:00PM -0400, Andrew Dunstan wrote:
> > It occurred to me that a pretty simple fix could just be to blacklist
> > everything that didn't start with a digit. The whitelist approach is
> > probably
On Sat, Oct 20, 2018 at 02:03:32AM -0400, Stephen Frost wrote:
> On Sat, Oct 20, 2018 at 01:11 Andres Freund wrote:
>> or the new checksum validation logic complaining about them, and such
>>> when doing backups and I wonder if that is because people simply don’t
>>> use the two together much,
Greetings,
On Sat, Oct 20, 2018 at 01:11 Andres Freund wrote:
> Hi,
>
> On 2018-10-20 01:07:43 -0400, Stephen Frost wrote:
> > I have to say that I can’t recall hearing much in the way of complaints
> > about pg_basebackup copying all the random cstore files
>
> Why would somebody complain
Greetings,
On Sat, Oct 20, 2018 at 01:16 Andres Freund wrote:
> Hi,
>
> On 2018-10-20 00:25:19 -0400, Stephen Frost wrote:
> > If we are going to decide that we only deal with files in our directories
> > matching these whitelisted patterns, then shouldn’t we apply similar
> logic
> > in things
Hi,
On 2018-10-20 00:25:19 -0400, Stephen Frost wrote:
> If we are going to decide that we only deal with files in our directories
> matching these whitelisted patterns, then shouldn’t we apply similar logic
> in things like DROP DATABASE and any other cases where we perform actions
> in a
Hi,
On 2018-10-20 01:07:43 -0400, Stephen Frost wrote:
> I have to say that I can’t recall hearing much in the way of complaints
> about pg_basebackup copying all the random cstore files
Why would somebody complain about that? It's usually desirable.
> or the new checksum validation logic
Greetings,
On Sat, Oct 20, 2018 at 00:58 Michael Paquier wrote:
> On Sat, Oct 20, 2018 at 12:41:04AM -0400, Stephen Frost wrote:
> > I’d also like to give David Steele a chance to comment on the specific
> API,
> > and any other backup tools authors, which I don’t think we should be
> > rushing
On Sat, Oct 20, 2018 at 12:41:04AM -0400, Stephen Frost wrote:
> I’d also like to give David Steele a chance to comment on the specific API,
> and any other backup tools authors, which I don’t think we should be
> rushing into anyway and I would think we’d only put into master..
By the way, we
Greetings,
On Sat, Oct 20, 2018 at 00:43 Michael Paquier wrote:
> On Sat, Oct 20, 2018 at 12:25:19AM -0400, Stephen Frost wrote:
> > On Fri, Oct 19, 2018 at 23:40 Michael Paquier
> wrote:
> >> - I agree with not doing a simple revert to not turn the buildfarm red
> >> again. This is annoying
On Sat, Oct 20, 2018 at 12:41:04AM -0400, Stephen Frost wrote:
> I’d also like to give David Steele a chance to comment on the specific API,
> and any other backup tools authors, which I don’t think we should be
> rushing into anyway and I would think we’d only put into master..
Getting David
On Sat, Oct 20, 2018 at 12:25:19AM -0400, Stephen Frost wrote:
> On Fri, Oct 19, 2018 at 23:40 Michael Paquier wrote:
>> - I agree with not doing a simple revert to not turn the buildfarm red
>> again. This is annoying for animal maintainers. Andrew has done a very
>> nice work in disabling
Greetings,
On Sat, Oct 20, 2018 at 00:31 Michael Paquier wrote:
> On Fri, Oct 19, 2018 at 08:50:04PM -0700, Andres Freund wrote:
> > On 2018-10-20 12:39:55 +0900, Michael Paquier wrote:
> >> So what I think we ought to do is the following:
> >> - Start a new thread, this one about TAP tests is
Hi,
On 2018-10-20 13:30:46 +0900, Michael Paquier wrote:
> On Fri, Oct 19, 2018 at 08:50:04PM -0700, Andres Freund wrote:
> > On 2018-10-20 12:39:55 +0900, Michael Paquier wrote:
> >> So what I think we ought to do is the following:
> >> - Start a new thread, this one about TAP tests is not
On Fri, Oct 19, 2018 at 08:50:04PM -0700, Andres Freund wrote:
> On 2018-10-20 12:39:55 +0900, Michael Paquier wrote:
>> So what I think we ought to do is the following:
>> - Start a new thread, this one about TAP tests is not adapted.
>> - Add in src/common/relpath.c the API from d55241af called
Greetings,
On Fri, Oct 19, 2018 at 23:40 Michael Paquier wrote:
> On Fri, Oct 19, 2018 at 06:43:18PM -0400, Tom Lane wrote:
> > Andrew Dunstan writes:
> >> I don't think just reverting it is really acceptable.
> >
> > +several. I do not mind somebody writing and installing a better fix.
> > I
Hi,
On 2018-10-20 12:39:55 +0900, Michael Paquier wrote:
> - I agree with Stephen's point that we should decide if a file has
> checksums or not in a single place, and that we should use the same
> logic for base backups and pg_verify_checksums.
To be clear, I wholeheartedly agree on that. We
On Fri, Oct 19, 2018 at 06:43:18PM -0400, Tom Lane wrote:
> Andrew Dunstan writes:
>> I don't think just reverting it is really acceptable.
>
> +several. I do not mind somebody writing and installing a better fix.
> I do object to turning the buildfarm red again.
I did not expect this thread
Andrew Dunstan writes:
> I don't think just reverting it is really acceptable.
+several. I do not mind somebody writing and installing a better fix.
I do object to turning the buildfarm red again.
regards, tom lane
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2018-10-19 17:32:58 -0400, Stephen Frost wrote:
> > As you pointed out previously, the current code *doesn't* work, before
> > or after this change, and we clearly need to rework this to move things
> > into libpgcommon and also fix
On 10/19/2018 05:32 PM, Stephen Frost wrote:
As you pointed out previously, the current code *doesn't* work, before
or after this change, and we clearly need to rework this to move things
into libpgcommon and also fix pg_basebackup. Reverting this would at
least get us back to having
Hi,
On 2018-10-19 17:32:58 -0400, Stephen Frost wrote:
> As you pointed out previously, the current code *doesn't* work, before
> or after this change, and we clearly need to rework this to move things
> into libpgcommon and also fix pg_basebackup. Reverting this would at
> least get us back to
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2018-10-19 16:35:46 -0400, Stephen Frost wrote:
> > The only justification for *not* doing this is that some extension
> > author might have dumped files into our tablespace directory, something
> > we've never claimed to support nor
Hi,
On 2018-10-19 16:35:46 -0400, Stephen Frost wrote:
> The only justification for *not* doing this is that some extension
> author might have dumped files into our tablespace directory, something
> we've never claimed to support nor generally worried about in all the
> time that I can recall
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2018-10-19 15:57:28 -0400, Stephen Frost wrote:
> > Perhaps it doesn't today but surely one goal of pg_verify_checksum is to
> > be able to run it on a running cluster eventually.
>
> I was saying *precisely* that above. I give up.
On 2018-10-19 15:57:28 -0400, Stephen Frost wrote:
> > > > > Of course- the same is true with a crash/restart case, so I'm not sure
> > > > > what you're getting at here.
> > > >
> > > > pg_verify_checksum doesn't support running on a crashed cluster, and I'm
> > > > not sure it'd make sense to
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2018-10-19 14:47:51 -0400, Stephen Frost wrote:
> > * Andres Freund (and...@anarazel.de) wrote:
> > > Oh, for crying out loud. Yes, obviously you can create conflicts, nobody
> > > ever doubted that. How on earth is that a useful point
Hi,
On 2018-10-19 14:47:51 -0400, Stephen Frost wrote:
> * Andres Freund (and...@anarazel.de) wrote:
> > On 2018-10-19 14:11:03 -0400, Stephen Frost wrote:
> > > * Andres Freund (and...@anarazel.de) wrote:
> > > > cstore e.g. does this, and it's already out there. So yes, we should
> > > >
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2018-10-19 14:11:03 -0400, Stephen Frost wrote:
> > * Andres Freund (and...@anarazel.de) wrote:
> > > cstore e.g. does this, and it's already out there. So yes, we should
> > > provide better infrastructure. But for now, we gotta deal
On 2018-10-19 14:11:03 -0400, Stephen Frost wrote:
> Greetings,
>
> * Andres Freund (and...@anarazel.de) wrote:
> > On 2018-10-19 13:52:28 -0400, Stephen Frost wrote:
> > > > > I also categorically disagree with the notion that it's ok for
> > > > > extensions to dump files into our tablespace
On 2018-10-19 14:08:19 -0400, J Chapman Flack wrote:
> On 10/19/2018 01:17 PM, Andres Freund wrote:
> > On 2018-10-19 10:36:59 -0400, Stephen Frost wrote:
> >
> >> I also categorically disagree with the notion that it's ok for
> >> extensions to dump files into our tablespace diretories
> >
On 10/19/2018 01:17 PM, Andres Freund wrote:
> On 2018-10-19 10:36:59 -0400, Stephen Frost wrote:
>
>> I also categorically disagree with the notion that it's ok for
>> extensions to dump files into our tablespace diretories
>
> I'm unconvinced. There already are extensions doing so, and
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2018-10-19 13:52:28 -0400, Stephen Frost wrote:
> > > > I also categorically disagree with the notion that it's ok for
> > > > extensions to dump files into our tablespace diretories or that we
> > > > should try to work around random
Hi,
On 2018-10-19 13:52:28 -0400, Stephen Frost wrote:
> > > I also categorically disagree with the notion that it's ok for
> > > extensions to dump files into our tablespace diretories or that we
> > > should try to work around random code dumping extra files in the
> > > directories which we
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2018-10-19 10:36:59 -0400, Stephen Frost wrote:
> > * Michael Paquier (mich...@paquier.xyz) wrote:
> > > On Wed, Oct 17, 2018 at 05:30:05PM -0400, Andrew Dunstan wrote:
> > > > Fine by me.
> > >
> > > Thanks. This is now committed
Hi,
On 2018-10-19 10:36:59 -0400, Stephen Frost wrote:
> * Michael Paquier (mich...@paquier.xyz) wrote:
> > On Wed, Oct 17, 2018 at 05:30:05PM -0400, Andrew Dunstan wrote:
> > > Fine by me.
> >
> > Thanks. This is now committed after some tweaks to the comments, a bit
> > earlier than I thought
Greetings,
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Stephen Frost writes:
> > * Tom Lane (t...@sss.pgh.pa.us) wrote:
> >> Seems like a job for a new(?) module in src/common/.
>
> > Yeah, that would be nice... Even nicer would be a way for non-core PG
> > tools to be able to use it (we have
Stephen Frost writes:
> * Tom Lane (t...@sss.pgh.pa.us) wrote:
>> Seems like a job for a new(?) module in src/common/.
> Yeah, that would be nice... Even nicer would be a way for non-core PG
> tools to be able to use it (we have basically the same thing in
> pgbackrest, unsurprisingly), though
Greetings,
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Michael Banck writes:
> > Am Freitag, den 19.10.2018, 10:36 -0400 schrieb Stephen Frost:
> >> I just saw this committed and I'm trying to figure out why we are
> >> creating yet-another-list when it comes to deciding what should be
> >>
Michael Banck writes:
> Am Freitag, den 19.10.2018, 10:36 -0400 schrieb Stephen Frost:
>> I just saw this committed and I'm trying to figure out why we are
>> creating yet-another-list when it comes to deciding what should be
>> checksum'd and what shouldn't be.
> To be fair, the list in
Hi,
Am Freitag, den 19.10.2018, 10:36 -0400 schrieb Stephen Frost:
> Greetings,
>
> * Michael Paquier (mich...@paquier.xyz) wrote:
> > On Wed, Oct 17, 2018 at 05:30:05PM -0400, Andrew Dunstan wrote:
> > > Fine by me.
> >
> > Thanks. This is now committed after some tweaks to the comments, a
Greetings,
* Michael Paquier (mich...@paquier.xyz) wrote:
> On Wed, Oct 17, 2018 at 05:30:05PM -0400, Andrew Dunstan wrote:
> > Fine by me.
>
> Thanks. This is now committed after some tweaks to the comments, a bit
> earlier than I thought first.
I just saw this committed and I'm trying to
On Wed, Oct 17, 2018 at 05:30:05PM -0400, Andrew Dunstan wrote:
> Fine by me.
Thanks. This is now committed after some tweaks to the comments, a bit
earlier than I thought first.
--
Michael
signature.asc
Description: PGP signature
On 10/16/2018 06:09 PM, Michael Paquier wrote:
On Mon, Oct 15, 2018 at 11:20:28AM -0400, Andrew Dunstan wrote:
On 10/15/2018 11:05 AM, Tom Lane wrote:
Andrew Dunstan writes:
We should fix this in PG11 rather than ship a broken utility. If
everyone is happy, I can apply this.
At this
On October 16, 2018 3:47:55 PM PDT, Tom Lane wrote:
>Michael Paquier writes:
>> On Mon, Oct 15, 2018 at 11:20:28AM -0400, Andrew Dunstan wrote:
>>> OK. In that case should we note that the utility is broken on
>Windows?
>
>> Not sure if that's worth adding in the documentation, something in
Michael Paquier writes:
> On Mon, Oct 15, 2018 at 11:20:28AM -0400, Andrew Dunstan wrote:
>> OK. In that case should we note that the utility is broken on Windows?
> Not sure if that's worth adding in the documentation, something in the
> press release would be more adapted?
I can't get too
On Mon, Oct 15, 2018 at 11:20:28AM -0400, Andrew Dunstan wrote:
> On 10/15/2018 11:05 AM, Tom Lane wrote:
>> Andrew Dunstan writes:
>>> We should fix this in PG11 rather than ship a broken utility. If
>>> everyone is happy, I can apply this.
>>
>> At this point I'd be happier if you waited till
On 10/15/2018 11:05 AM, Tom Lane wrote:
Andrew Dunstan writes:
We should fix this in PG11 rather than ship a broken utility. If
everyone is happy, I can apply this.
At this point I'd be happier if you waited till after 11.0 wraps...
there is no margin for error today.
Andrew Dunstan writes:
> We should fix this in PG11 rather than ship a broken utility. If
> everyone is happy, I can apply this.
At this point I'd be happier if you waited till after 11.0 wraps...
there is no margin for error today.
regards, tom lane
On 10/14/2018 09:11 PM, Andrew Dunstan wrote:
What do you think about the updated version attached?
Looks good to me.
We should fix this in PG11 rather than ship a broken utility. If
everyone is happy, I can apply this.
cheers
andrew
--
Andrew Dunstan
On 10/14/2018 06:41 PM, Michael Paquier wrote:
On Sun, Oct 14, 2018 at 10:24:55AM -0400, Andrew Dunstan wrote:
[snip]
Thanks a lot for the review, Andrew!
This code now seems redundant:
if (strcmp(fn, ".") == 0 ||
strcmp(fn, "..") == 0)
return true;
On Sun, Oct 14, 2018 at 10:24:55AM -0400, Andrew Dunstan wrote:
> [snip]
Thanks a lot for the review, Andrew!
> This code now seems redundant:
>
> if (strcmp(fn, ".") == 0 ||
> strcmp(fn, "..") == 0)
> return true;
Indeed. I have renamed skipfile() to
On 10/13/2018 09:59 PM, Michael Paquier wrote:
On Sat, Oct 13, 2018 at 05:53:00PM -0400, Andrew Dunstan wrote:
It occurred to me that a pretty simple fix could just be to blacklist
everything that didn't start with a digit. The whitelist approach is
probably preferable... depends how urgent
On Sat, Oct 13, 2018 at 05:53:00PM -0400, Andrew Dunstan wrote:
> It occurred to me that a pretty simple fix could just be to blacklist
> everything that didn't start with a digit. The whitelist approach is
> probably preferable... depends how urgent we see this as.
Yeah, possibly, still that's
more than a couple of
minutes in a row for a couple of days at least, and I don't like to keep
the buildfarm unhappy for the time being. There are three options:
1) Revert the TAP tests of pg_verify_checksums.
2) Push the patch which adds new entries for EXEC_BACKEND files in the
skip list. That
r a couple of days at least, and I don't like to keep
the buildfarm unhappy for the time being. There are three options:
1) Revert the TAP tests of pg_verify_checksums.
2) Push the patch which adds new entries for EXEC_BACKEND files in the
skip list. That's a short workaround, and that would all
vents me to stay in
front of a computer or to look at a screen for more than a couple of
minutes in a row for a couple of days at least, and I don't like to keep
the buildfarm unhappy for the time being. There are three options:
1) Revert the TAP tests of pg_verify_checksums.
2) Push the patch wh
Hi,
On Fri, Oct 12, 2018 at 03:05:43PM +0900, Michael Paquier wrote:
> On Fri, Oct 12, 2018 at 12:11:58PM +0900, Michael Paquier wrote:
> > Agreed. I am just working on a patch for v11- which uses a
> > whitelist-based method instead of what is present now. Reverting the
> > tests to put the
On Fri, Oct 12, 2018 at 12:11:58PM +0900, Michael Paquier wrote:
> Agreed. I am just working on a patch for v11- which uses a
> whitelist-based method instead of what is present now. Reverting the
> tests to put the buildfarm to green could be done, but that's not the
> root of the problem.
So,
On Thu, Oct 11, 2018 at 07:41:18PM -0700, Andres Freund wrote:
> We only remove temp dirs on startup, and I'm pretty sure there at least
> not too long ago were a few error cases where we can leak temp files in
> individual sessions. And there's talk about allowing
> pg_verify_checksums when
Hi,
On 2018-10-12 11:07:58 +0900, Michael Paquier wrote:
> On Thu, Oct 11, 2018 at 06:53:19PM -0700, Andres Freund wrote:
> > Hm. Maybe I'm confused, but how is it correct to use a blacklisting
> > approach here? It's far from absurd to have files inside a tablespacs
> > when using
On Thu, Oct 11, 2018 at 06:53:19PM -0700, Andres Freund wrote:
> On 2018-10-12 10:39:18 +0900, Michael Paquier wrote:
>> I have been able to reproduce the problem, and that's a bug within
>> pg_verify_checksums as it fails to consider that config_exec_params is
>> not a file it should scan when
Hi,
On 2018-10-12 10:39:18 +0900, Michael Paquier wrote:
> On Thu, Oct 11, 2018 at 06:04:11PM -0700, Andres Freund wrote:
> > culicidae tests EXEC_BACKEND, so there's an explanation as to why it
> > sometimes behaves differently. But here I don't immediately see how
> > that'd matter. Probably
On Thu, Oct 11, 2018 at 06:04:11PM -0700, Andres Freund wrote:
> culicidae tests EXEC_BACKEND, so there's an explanation as to why it
> sometimes behaves differently. But here I don't immediately see how
> that'd matter. Probably still worth verifying that it's not somehow
> caused by that.
Hi,
On 2018-10-12 09:56:14 +0900, Michael Paquier wrote:
> On Fri, Oct 12, 2018 at 12:17:57AM +, Michael Paquier wrote:
> > Add TAP tests for pg_verify_checksums
> >
> > All options available in the utility get coverage:
> > - Tests with disabled page checksums.
&g
On Fri, Oct 12, 2018 at 12:17:57AM +, Michael Paquier wrote:
> Add TAP tests for pg_verify_checksums
>
> All options available in the utility get coverage:
> - Tests with disabled page checksums.
> - Tests with enabled test checksums.
> - Emulation of corruption an
On Wed, Oct 10, 2018 at 10:50:02AM +0900, Michael Paquier wrote:
> The resulting patch is attached. Does that look good?
And committed. Thanks all for taking the time to review.
--
Michael
signature.asc
Description: PGP signature
On Tue, Oct 09, 2018 at 05:14:50PM +0200, Michael Banck wrote:
> Am Dienstag, den 09.10.2018, 16:54 +0200 schrieb Peter Eisentraut:
>> On 06/10/2018 13:46, Michael Paquier wrote:
>
>>> +# Time to create a corruption
>
> That looks a bit weird, maybe "some corruption"? Or maybe it's just me
> not
Hi,
Am Dienstag, den 09.10.2018, 16:54 +0200 schrieb Peter Eisentraut:
> On 06/10/2018 13:46, Michael Paquier wrote:
> > What do you think about the updated version attached?
> > +# Time to create a corruption
That looks a bit weird, maybe "some corupption"? Or maybe it's just me
not being a
On 06/10/2018 13:46, Michael Paquier wrote:
> On Fri, Oct 05, 2018 at 01:38:05PM +0200, Michael Banck wrote:
>> It's too late for v11 though at this point I guess?
>
> Unfortunately yes.
>
>> I think it would be easy to also test the -r command-line option, as we
>> already create a table.
>
>
On Fri, Oct 05, 2018 at 01:38:05PM +0200, Michael Banck wrote:
> It's too late for v11 though at this point I guess?
Unfortunately yes.
> I think it would be easy to also test the -r command-line option, as we
> already create a table.
Good idea. Let's add this test.
> That comment should
Hi,
On Fri, Oct 05, 2018 at 10:26:45AM +0900, Michael Paquier wrote:
> The topic of $subject has been discussed a bit times, resulting in a
> couple of patches on the way:
> https://www.postgresql.org/message-id/20180830200258.gg15...@paquier.xyz
>
Hi all,
The topic of $subject has been discussed a bit times, resulting in a
couple of patches on the way:
https://www.postgresql.org/message-id/20180830200258.gg15...@paquier.xyz
https://www.postgresql.org/message-id/cabuevezekrwpegk2o-ov4z2mjft6cu8clfa-v1s1j4z8x7w...@mail.gmail.com
However
78 matches
Mail list logo
