Re: Harry Ambrose 2017-08-25
> Hi All,
>
> Sorry to open this can of worms again. However, we are still struggling
> with this issue across quite a large amount of our estate.
Hi,
we've just seen exactly this error on a customer database running
9.5.3 (postgresql95.x86_64 9.5.3-2PGDG.rhel6). L
Hi All,
Sorry to open this can of worms again. However, we are still struggling
with this issue across quite a large amount of our estate.
>From doing some further research I stumbled across the following which
seems to sum up what we are seeing quite well...
http://grokbase.com/t/postgresql/pgs
On Wed, Aug 9, 2017 at 6:27 AM, ADSJ (Adam Sjøgren) wrote:
> On 2017-06-21 Adam Sjøgren wrote:
>
>> Adam Sjøgren wrote:
>
>>> Meanwhile, I can report that I have upgraded from 9.3.14 to 9.3.17 and
>>> the errors keep appearing the log.
>
> Just to close this, for the record: We haven't seen the
Unfortunately we still see it frequently :(
On 9 August 2017 at 14:29, Achilleas Mantzios
wrote:
> On 09/08/2017 15:27, ADSJ (Adam Sjøgren) wrote:
>
>> On 2017-06-21 Adam Sjøgren wrote:
>>
>> Adam Sjøgren wrote:
>>>
Meanwhile, I can report that I have upgraded from 9.3.14 to 9.3.17 and
>>
On 09/08/2017 15:27, ADSJ (Adam Sjøgren) wrote:
On 2017-06-21 Adam Sjøgren wrote:
Adam Sjøgren wrote:
Meanwhile, I can report that I have upgraded from 9.3.14 to 9.3.17 and
the errors keep appearing the log.
Just to close this, for the record: We haven't seen the errors since
2017-06-30. We
On 2017-06-21 Adam Sjøgren wrote:
> Adam Sjøgren wrote:
>> Meanwhile, I can report that I have upgraded from 9.3.14 to 9.3.17 and
>> the errors keep appearing the log.
Just to close this, for the record: We haven't seen the errors since
2017-06-30. We upgraded to 9.3.17 (latest 9.3 point-relea
Hi Everyone,
Still trying to fathom this one. I have added quite a few log lines to a
copy of 9.4.12 and compiled it hoping to find the fault.
Below is from the log (at DEBUG5). Apologies for my name in the log lines,
it was the easiest way to grep them specifically I also apologise that its
a bi
Adam Sjøgren wrote:
> Meanwhile, I can report that I have upgraded from 9.3.14 to 9.3.17 and
> the errors keep appearing the log.
Just a quick update with more observations:
All the errors in the postgres.log from one of the tables are triggered
by a stored procedure that gathers data to put in
Harry Ambrose writes:
> Not sure whether its relevant or not, however upon adding an ANALYSE before
> the second vacuum the issue has not presented when testing. I have managed
> 95 cycles thus far.
I'm still unable to reproduce :-( --- I ran about two dozen cycles
overnight with no sign of troub
Hi,
Not sure whether its relevant or not, however upon adding an ANALYSE before
the second vacuum the issue has not presented when testing. I have managed
95 cycles thus far.
BEGIN;
CREATE TABLE x (id BIGSERIAL PRIMARY KEY, payload1 VARCHAR, payload2
VARCHAR, payload3 VARCHAR, payload4 BIGINT, pa
Hi,
BTW, how do you get that jar to make the test table on a non-default
> tablespace? Or are you just putting the whole test DB on a tablespace?
>
> regards, tom lane
>
I have been putting the whole database on a tablespace. It seemed easier
than modifying the jar.
Many
Harry Ambrose writes:
> - Custom tablespaces (where the errors occur) sat on 4 disk RAID-10 (ext3
> filesystem).
BTW, how do you get that jar to make the test table on a non-default
tablespace? Or are you just putting the whole test DB on a tablespace?
regards, tom lane
Achilleas Mantzios writes:
> After 2 full attempts, (and after bringing my poor - old workstation to its
> knees) it still does not produce the supposed ERROR :
Yeah, I've had little luck reproducing it either. However, I noticed a
few messages back that Harry is testing against a master/slave
It seems to be very hit and miss...
The below is from the machine described in this thread running PostgreSQL
9.4.10:
update 10
update 12
update 14
update 16
update 18
Update all
Vacuum
org.postgresql.util.PSQLException: ERROR: unexpected chunk number 2285
(expected 0)
Hi,
pls tell me, I am currently running 2nd run in my box, (New attempt 2), and
> its in the "Attempting vacuum" phase.
> What is it supposed to do next?
> I got no errors , it has gotten my machine to its knees.
>
The jar has an endless while loop. Thus please kill the PID when you are
done test
On 12/06/2017 10:46, Harry Ambrose wrote:
Hi,
> Their suggestion is to upload to Google Drive. That or use a third party
site, like Dropbox.
I have uploaded the jar to dropbox, link below (please let me know if you have
any issues downloading):
https://www.dropbox.com/s/96vm465i7rwhcf8/toast
Hi,
> Their suggestion is to upload to Google Drive. That or use a third party
site, like Dropbox.
I have uploaded the jar to dropbox, link below (please let me know if you
have any issues downloading):
https://www.dropbox.com/s/96vm465i7rwhcf8/toast-corrupter-aio.jar?dl=0
> So I guess you run
Alvaro Herrera wrote:
> ADSJ (Adam Sjøgren) wrote:
>
>> Our database has started reporting errors like this:
>>
>> 2017-05-31 13:48:10 CEST ERROR: unexpected chunk number 0 (expected 1)
>> for toast value 14242189 in pg_toast_10919630
> Does the problem still reproduce if you revert commit
On 09/06/2017 19:02, Harry Ambrose wrote:
Hi,
No error messages found.
- is your RAM ECC? Did you run any memtest?
Yes, memory is ECC. No error messages found.
So I guess you run memtest86+ and it reported that your memory is indeed ECC
and also that it is working properly?
Best wishes,
On 06/11/2017 08:34 AM, Tom Lane wrote:
Alvaro Herrera writes:
I'm unable to run this file. Maybe it was corrupted in transit, given
that it was considered to be text.
My copy came through fine, so I tried to forward it to you off-list,
but gmail rejected it as a possible security hazard. D
Alvaro Herrera writes:
> I'm unable to run this file. Maybe it was corrupted in transit, given
> that it was considered to be text.
My copy came through fine, so I tried to forward it to you off-list,
but gmail rejected it as a possible security hazard. Do you know
which binary mime types they
Harry Ambrose wrote:
> Hi,
>
> Please find the jar attached (renamed with a .txt extension as I know some
> email services deem jars a security issue).
>
> The jar accepts the following arguments:
>
> $1 = host
> $2 = database
> $3 = username
> $4 = password
> $5 = port
>
> It returns its loggi
ADSJ (Adam Sjøgren) wrote:
> Our database has started reporting errors like this:
>
> 2017-05-31 13:48:10 CEST ERROR: unexpected chunk number 0 (expected 1) for
> toast value 14242189 in pg_toast_10919630
Does the problem still reproduce if you revert commit
6c243f90ab6904f27fa990f1f3261e1d09
Hi,
Please find the jar attached (renamed with a .txt extension as I know some
email services deem jars a security issue).
The jar accepts the following arguments:
$1 = host
$2 = database
$3 = username
$4 = password
$5 = port
It returns its logging to STDOUT. Please let me know if you require f
Hi,
Please find the jar attached (renamed with a .txt extension as I know some
email services deem jars a security issue).
The jar accepts the following arguments:
$1 = host
$2 = database
$3 = username
$4 = password
$5 = port
It returns its logging to STDOUT. Please let me know if you require f
Robert Lakes writes:
> I am new to postgress and I am trying to write my first function to insert,
> update or delete and trap errors as a result of the table not existing ,
> the columns not exist or if any other error simply pass back the sqlstate
Please do not hijack an existing thread to ask
Ha guys,
I am new to postgress and I am trying to write my first function to insert,
update or delete and trap errors as a result of the table not existing ,
the columns not exist or if any other error simply pass back the sqlstate
here's my code can you help
CREATE OR REPLACE FUNCTION listings_aud
Hi,
Maybe you could give some info on :
> - your ext3 mkfs and mount options (journal, barriers, etc)
>
/etc/fstab details below:
LABEL=/var/lib/pgsql/var/lib/pgsql ext3defaults
1 2
LABEL=/tablespace1 /tablespace1ext3defaults
1 2
LABEL=/tabl
On 09/06/2017 14:44, Harry Ambrose wrote:
Hi Tom,
Thanks for attempting to replicate the issue.
Anyway, the bad news is I couldn't reproduce the problem then and I can't
now. I don't know if it's a timing issue or if there's something critical
about configuration that I'm not dupli
Hi Tom,
Thanks for attempting to replicate the issue.
Anyway, the bad news is I couldn't reproduce the problem then and I can't
> now. I don't know if it's a timing issue or if there's something critical
> about configuration that I'm not duplicating. Can you explain what sort
> of platform you
Harry Ambrose writes:
> Please find the jar attached (renamed with a .txt extension as I know some
> email services deem jars a security issue).
Hmm, the output from this script reminds me quite a lot of one I was
sent in connection with bug #1 awhile back:
https://www.postgresql.org/message-
On 06/08/2017 08:13 AM, ADSJ (Adam Sjøgren) wrote:
Achilleas writes:
Anyone has a handy little script lying around?
http://www.databasesoup.com/2013/10/de-corrupting-toast-tables.html
Thanks for the response!
Adam
--
Adrian Klaver
adrian.kla...@aklaver.com
--
Sent via pg
Harry writes:
> The second vacuum causes an ERROR identical to that you are reporting
> below (unexpected chunk number n (expected n) for toast value...).
> However it may take up to ten attempts to replicate it.
Interesting.
> Out of interest, are you using any tablespaces other than pg_default
Achilleas writes:
> First try to find which tables those toast relations refer to :
> select 10919630::regclass , 10920100::regclass ;
> Are those critical tables? Can you restore them somehow?
They are our two big tables, containing the bulk of our data (one with
168M rows, the other with 320M r
Harry Ambrose writes:
> Tom - I can provide a jar that I have been using to replicate the issue.
> Whats the best transport method to send it over?
If it's not enormous, just send it as an email attachment.
regards, tom lane
--
Sent via pgsql-general mailing list (pgs
Hi,
Thanks for the responses.
> "One lesson I learned from the BSD camp when dealing with random freezes and
> panics : when all else fails to give an answer it is time to start blaming my
> hardware. Are those tablespaces on any cheap SSD's ?”
The tablespaces are not sat on SSD’s. Something I
Harry Ambrose writes:
> I have been following the updates to the 9.4 branch hoping a fix will appear,
> but sadly no luck yet. I have manually replicated the issue on 9.4.4, 9.4.10
> and 9.4.12. My replication steps are:
This is a very interesting report, but you didn't actually provide a
repro
On 07/06/2017 17:49, Harry Ambrose wrote:
Hi,
Out of interest, are you using any tablespaces other than pg_default? I can
only replicate the issue when using separately mounted tablespaces.
One lesson I learned from the BSD camp when dealing with random freezes and panics : when all else fails t
Hi,
I too have been experiencing this with a busy PostgreSQL instance.
I have been following the updates to the 9.4 branch hoping a fix will appear,
but sadly no luck yet. I have manually replicated the issue on 9.4.4, 9.4.10
and 9.4.12. My replication steps are:
BEGIN;
CREATE TABLE x (id BIGS
On 07/06/2017 16:33, ADSJ (Adam Sjøgren) wrote:
Our database has started reporting errors like this:
2017-05-31 13:48:10 CEST ERROR: unexpected chunk number 0 (expected 1) for
toast value 14242189 in pg_toast_10919630
...
2017-06-01 11:06:56 CEST ERROR: unexpected chunk number 0
Our database has started reporting errors like this:
2017-05-31 13:48:10 CEST ERROR: unexpected chunk number 0 (expected 1) for
toast value 14242189 in pg_toast_10919630
...
2017-06-01 11:06:56 CEST ERROR: unexpected chunk number 0 (expected 1) for
toast value 19573520 in pg_toast_10
41 matches
Mail list logo