Invalid byte sequence errors on DB restore

2020-03-15 Thread Samuel Smith
My current DB backup routine is just to call pg_dump and pipe to gzip. We recently started to get a failure to restore (which is basically just using psql -f on the pg_dump file) with the following errors: invalid byte sequence for encoding "UTF8": 0xa0 and invalid byte sequence for encoding

Re: Order by and timestamp

2020-03-15 Thread Tom Lane
Adrian Klaver writes: > On 3/15/20 2:33 PM, Björn Lundin wrote: >> I then did ’select * from AMARKETS order by STARTTS’ > Is amarkets in more then one schema? Yeah, it's hard to think of any explanation other than "the query used a corrupt index on startts to produce the ordering". But your \d

Re: Order by and timestamp

2020-03-15 Thread Adrian Klaver
On 3/15/20 2:33 PM, Björn Lundin wrote: Hi! I have an old database that behaves a bit strange. I keeps horse races in UK/IE. I have a program that continuously* adds record into a market table , described as below. *continuously means ’after each race’ which is ca 12:00 --> 23:00. I then did

Re: Order by and timestamp

2020-03-15 Thread Adrian Klaver
On 3/15/20 2:48 PM, Steven Lembark wrote: On Sun, 15 Mar 2020 22:33:35 +0100:wq Björn Lundin wrote: And to my surprise i get a result like this (note the order of column STARTTS) (1) Suggest using "pastebin.com" for this kind of data. It may not look very pretty -- or readable at all

Re: unexpected chunk number 2 (expected 0) for toast value ... in pg_toast_18536

2020-03-15 Thread Tom Lane
Karsten Hilbert writes: >>> According to >>> http://www.databasesoup.com/2013/10/de-corrupting-toast-tables.html >>> an UPDATE of the row is recommended -- should that work >>> better than a DELETE ? > OK, got that. What I now don't understand is how the UPDATE > won't have to touch the TOAST

Re: unexpected chunk number 2 (expected 0) for toast value ... in pg_toast_18536

2020-03-15 Thread Karsten Hilbert
On Sun, Mar 15, 2020 at 02:35:39PM -0700, Adrian Klaver wrote: > On 3/15/20 1:21 PM, Karsten Hilbert wrote: > > On Sun, Mar 15, 2020 at 12:58:53PM -0700, Adrian Klaver wrote: > > > > > > > We then tried to DELETE the offending row > > > > > > > > > > delete from blobs.doc_obj where pk =

Re: Order by and timestamp

2020-03-15 Thread Steven Lembark
On Sun, 15 Mar 2020 22:33:35 +0100:wq Björn Lundin wrote: > And to my surprise i get a result like this (note the order of > column STARTTS) (1) Suggest using "pastebin.com" for this kind of data. It may not look very pretty -- or readable at all -- on the viewer's end depending on

Re: unexpected chunk number 2 (expected 0) for toast value ... in pg_toast_18536

2020-03-15 Thread Adrian Klaver
On 3/15/20 1:21 PM, Karsten Hilbert wrote: On Sun, Mar 15, 2020 at 12:58:53PM -0700, Adrian Klaver wrote: We then tried to DELETE the offending row delete from blobs.doc_obj where pk = 82224; but that, again, shows the "unexpected chunk" problem. According to

Order by and timestamp

2020-03-15 Thread Björn Lundin
Hi! I have an old database that behaves a bit strange. I keeps horse races in UK/IE. I have a program that continuously* adds record into a market table , described as below. *continuously means ’after each race’ which is ca 12:00 --> 23:00. I then did ’select * from AMARKETS order by STARTTS’

Re: unexpected chunk number 2 (expected 0) for toast value ... in pg_toast_18536

2020-03-15 Thread Karsten Hilbert
On Sun, Mar 15, 2020 at 05:04:06PM -0400, Jan Wieck wrote: > Have you tried to reindex the table? Toast internally forces an index scan, > so missing index tuples or an otherwise corrupted toast index would have > the same symptoms as toast chunks actually missing. We sure did, but thanks for

Re: unexpected chunk number 2 (expected 0) for toast value ... in pg_toast_18536

2020-03-15 Thread Jan Wieck
Have you tried to reindex the table? Toast internally forces an index scan, so missing index tuples or an otherwise corrupted toast index would have the same symptoms as toast chunks actually missing. Regards, Jan On Sun, Mar 15, 2020, 16:21 Karsten Hilbert wrote: > On Sun, Mar 15, 2020 at

Re: unexpected chunk number 2 (expected 0) for toast value ... in pg_toast_18536

2020-03-15 Thread Karsten Hilbert
On Sun, Mar 15, 2020 at 12:58:53PM -0700, Adrian Klaver wrote: > > > We then tried to DELETE the offending row > > > > > > delete from blobs.doc_obj where pk = 82224; > > > > > > but that, again, shows the "unexpected chunk" problem. > > > > According to > > > >

Re: unexpected chunk number 2 (expected 0) for toast value ... in pg_toast_18536

2020-03-15 Thread Adrian Klaver
On 3/15/20 12:20 PM, Karsten Hilbert wrote: On Sun, Mar 15, 2020 at 07:23:49PM +0100, Karsten Hilbert wrote: We then tried to DELETE the offending row delete from blobs.doc_obj where pk = 82224; but that, again, shows the "unexpected chunk" problem. According to

Re: unexpected chunk number 2 (expected 0) for toast value ... in pg_toast_18536

2020-03-15 Thread Karsten Hilbert
On Sun, Mar 15, 2020 at 07:23:49PM +0100, Karsten Hilbert wrote: > We then tried to DELETE the offending row > > delete from blobs.doc_obj where pk = 82224; > > but that, again, shows the "unexpected chunk" problem. According to

unexpected chunk number 2 (expected 0) for toast value ... in pg_toast_18536

2020-03-15 Thread Karsten Hilbert
Dear community, we are seeing the below error on PG 9.6.16 on Debian: (different issue from the psycopg2 one recently posted by me) > /home/marc# pg_dump -p 5432 --username=gm-dbo --dbname=gnumed_v22 > --compress=0 --no-sync --format=custom --file=/dev/null > pg_dump: Ausgabe des Inhalts der

Re: Exclude logging certain connections?

2020-03-15 Thread Paul Förster
Hi Adrian, > Nothing from what I see: > > From here: > > https://www.postgresql.org/docs/12/config-setting.html#id-1.6.6.4.5 > > env PGOPTIONS="-c log_connections=off" psql -d test -U aklaver > > psql: error: could not connect to server: FATAL: permission denied to set > parameter

Re: Exclude logging certain connections?

2020-03-15 Thread Adrian Klaver
On 3/15/20 3:09 AM, Paul Förster wrote: Hi, is there a way to exclude certain connections, for example recurring monitoring connections, from the postgres.log? I found this from in old post on stackexchange.com: postgres=# create role mon login; CREATE ROLE postgres=# alter role mon set

Exclude logging certain connections?

2020-03-15 Thread Paul Förster
Hi, is there a way to exclude certain connections, for example recurring monitoring connections, from the postgres.log? I found this from in old post on stackexchange.com: postgres=# create role mon login; CREATE ROLE postgres=# alter role mon set log_connections=off; ERROR: parameter