Re: [HACKERS] Errors in our encoding conversion tables

2015-12-02 Thread Tom Lane
Robert Haas writes: > On Fri, Nov 27, 2015 at 8:54 PM, Tatsuo Ishii wrote: >> In short, there are number of reasons we cannot simply import the >> consortium's mapping regarding SJIS (and EUC_JP). > I haven't seen a response to this point, but it seems important. I'll defer to Tatsuo-san concer

Re: [HACKERS] Errors in our encoding conversion tables

2015-12-02 Thread Robert Haas
On Fri, Nov 27, 2015 at 8:54 PM, Tatsuo Ishii wrote: > I explain why the manual editing is necessary. > > One of the most famous problems with Unicode is "wave dash" > (U+301C). According the Unicode consortium's Unicode/SJIS map, it > corresponds to 0x8160 of Shift_JIS. Unfortunately this was a m

Re: [HACKERS] Errors in our encoding conversion tables

2015-11-28 Thread Tom Lane
I wrote: > There's a discussion over at > http://www.postgresql.org/message-id/flat/2sa.dhu5.1hk1yrptnfy.1ml...@seznam.cz > of an apparent error in our WIN1250 -> LATIN2 conversion. Attached is an updated patch (against today's HEAD) showing proposed changes to bring cyrillic_and_mic.c and latin2_

Re: [HACKERS] Errors in our encoding conversion tables

2015-11-27 Thread Tatsuo Ishii
> I wrote: >> I have not attempted to reverify the files in utils/mb/Unicode against the >> original Unicode Consortium data, but maybe we ought to do that before >> taking any further steps here. > > I downloaded the mapping files from unicode.org and attempted to verify > that the Unicode/*.map

Re: [HACKERS] Errors in our encoding conversion tables

2015-11-27 Thread Tom Lane
I wrote: > gb18030_to_utf8.map utf8_to_gb18030.map > Could not find the reference file gb-18030-2000.xml, whose origin is > unstated anyway. Ah, scratch that complaint; digging in our git history turned up the origin of that file, so I double-checked it and then updated the script with a comment

Re: [HACKERS] Errors in our encoding conversion tables

2015-11-27 Thread Tom Lane
I wrote: > I have not attempted to reverify the files in utils/mb/Unicode against the > original Unicode Consortium data, but maybe we ought to do that before > taking any further steps here. I downloaded the mapping files from unicode.org and attempted to verify that the Unicode/*.map files could

Re: [HACKERS] Errors in our encoding conversion tables

2015-11-27 Thread Tom Lane
Albe Laurenz writes: > I agree with your proposed fix, the only thing that makes me feel > uncomfortable > is that you get error messages like: > ERROR: character with byte sequence 0x96 in encoding "WIN1250" has no > equivalent in encoding "MULE_INTERNAL" Hm, yeah. It's pretty silly that t

Re: [HACKERS] Errors in our encoding conversion tables

2015-11-27 Thread Albe Laurenz
Tom Lane wrote: > There's a discussion over at > http://www.postgresql.org/message-id/flat/2sa.dhu5.1hk1yrptnfy.1ml...@seznam.cz > of an apparent error in our WIN1250 -> LATIN2 conversion. I looked into this > and found that indeed, the code will happily translate certain characters > for which th

Re: [HACKERS] Errors in our encoding conversion tables

2015-11-26 Thread Tom Lane
Tatsuo Ishii writes: > I have started to looking into it. I wonder how do you create the part > of your patch: The code I used is below. > In the above you seem to disable the conversion from 0x96 of win1250 > to ISO-8859-2 by using the Unicode mapping files in > src/backend/utils/mb/Unicode. Bu

Re: [HACKERS] Errors in our encoding conversion tables

2015-11-26 Thread Tatsuo Ishii
> There's a discussion over at > http://www.postgresql.org/message-id/flat/2sa.dhu5.1hk1yrptnfy.1ml...@seznam.cz > of an apparent error in our WIN1250 -> LATIN2 conversion. I looked into this > and found that indeed, the code will happily translate certain characters > for which there seems to be

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-12-10 Thread Andres Freund
On 2013-12-10 11:12:03 -0800, Josh Berkus wrote: > On 12/10/2013 10:48 AM, Andres Freund wrote: > > On 2013-12-10 10:44:30 -0800, Josh Berkus wrote: > >> On 12/10/2013 10:39 AM, Andres Freund wrote: > >>> Hi, > >>> > >>> On 2013-12-10 10:38:32 -0800, Josh Berkus wrote: > We've just run across

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-12-10 Thread Josh Berkus
On 12/10/2013 10:48 AM, Andres Freund wrote: > On 2013-12-10 10:44:30 -0800, Josh Berkus wrote: >> On 12/10/2013 10:39 AM, Andres Freund wrote: >>> Hi, >>> >>> On 2013-12-10 10:38:32 -0800, Josh Berkus wrote: We've just run across a case of this exact issue on 9.2.4. I thought it was sup

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-12-10 Thread Andres Freund
On 2013-12-10 10:44:30 -0800, Josh Berkus wrote: > On 12/10/2013 10:39 AM, Andres Freund wrote: > > Hi, > > > > On 2013-12-10 10:38:32 -0800, Josh Berkus wrote: > >> We've just run across a case of this exact issue on 9.2.4. I thought it > >> was supposed to be 9.3-only? > > > > Could you please

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-12-10 Thread Josh Berkus
On 12/10/2013 10:39 AM, Andres Freund wrote: > Hi, > > On 2013-12-10 10:38:32 -0800, Josh Berkus wrote: >> We've just run across a case of this exact issue on 9.2.4. I thought it >> was supposed to be 9.3-only? > > Could you please describe "this exact issue"? Fatal errors due to missing pg_sub

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-12-10 Thread Andres Freund
Hi, On 2013-12-10 10:38:32 -0800, Josh Berkus wrote: > We've just run across a case of this exact issue on 9.2.4. I thought it > was supposed to be 9.3-only? Could you please describe "this exact issue"? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.co

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-12-10 Thread Josh Berkus
Andres, all: We've just run across a case of this exact issue on 9.2.4. I thought it was supposed to be 9.3-only? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-12-05 Thread J Smith
On Thu, Nov 28, 2013 at 5:15 AM, Andres Freund wrote: > Hi, > > Do you still have the core file around? If so could you 'p > *ShmemVariableCache' and 'p *ControlFile'? > So sorry, I didn't see this message until just today. Seems it was accidentally archived before hitting my eyeballs. I see tha

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-28 Thread Andres Freund
Hi, On 2013-11-24 16:56:26 -0500, J Smith wrote: > coredumper worked like a charm. Useful tool, that is... although as a > bit of advice, I'd try not to run it on Postgres if your various > memory settings are tweaked towards production use -- the core dump > that was captured on my server weighed

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-27 Thread Andres Freund
On 2013-11-27 13:57:52 -0300, Alvaro Herrera wrote: > Per bug report by J Smith in > cadfupgc5bmtv-yg9znxv-vcfkb+jprqs7m2oesqxam_4z1j...@mail.gmail.com > diagnosed by Andres Freund. Alvaro, do you see a way this could actually have caused J.'s problems? I thought about a few, but each

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-27 Thread Alvaro Herrera
Alvaro Herrera escribió: > Andres Freund escribió: > This seems simple to handle by adding the check you propose to the loop. > Basically if the xmax doesn't match the xmin, we reached the end, > there's nothing more to lock and we can return success without any > further work: As mentioned in th

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-25 Thread Andres Freund
On 2013-11-25 18:06:30 -0300, Alvaro Herrera wrote: > > I mean that in the !KEYS_UPDATED case we don't need to abort if we're > > only acquiring a key share... > > Hm, I think that's correct -- we don't need to abort. But we still need > to wait until the updater completes. So this proposed patc

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-25 Thread Alvaro Herrera
Andres Freund escribió: > On 2013-11-25 17:10:39 -0300, Alvaro Herrera wrote: > > Let me point out that this is exactly the same code that would be > > affected by my proposed fix for #8434, which would have this check the > > updateXid in all cases, not only in KEYS_UPDATED as currently. > > Hm.

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-25 Thread Andres Freund
On 2013-11-25 17:10:39 -0300, Alvaro Herrera wrote: > > I am not sure whether that's the origin of the problem but at the very > > least it seems to me that heap_lock_updated_tuple_rec() is missing > > several important pieces: > > a) do the priorXmax==xmin dance to check we're still following the

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-25 Thread Alvaro Herrera
Andres Freund escribió: > Ok, this is helpful. Do you rather longrunning transactions? The > transaction that does foreign key checks has an xid of 10260613, while > the row that's getting checked has 13514992. Thanks for the analysis. > #4 0x00635dc7 in XactLockTableWait (xid=13514992)

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-25 Thread J Smith
On Mon, Nov 25, 2013 at 11:46 AM, Alvaro Herrera wrote: > J Smith escribió: > >> We did have some long-running transactions, yes. We refactored a bit >> and removed them and the problem ceased on our end. We ended up >> reverting our changes for the sake of running this experiment over the >> week

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-25 Thread Alvaro Herrera
J Smith escribió: > We did have some long-running transactions, yes. We refactored a bit > and removed them and the problem ceased on our end. We ended up > reverting our changes for the sake of running this experiment over the > weekend and the errors returned. We've since restored our fix and >

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-25 Thread J Smith
On Mon, Nov 25, 2013 at 6:47 AM, Andres Freund wrote: > Hi, > > On 2013-11-24 16:56:26 -0500, J Smith wrote: > >> Nov 23 14:38:32 dev postgres[23810]: [4-1] user=dev,db=dev ERROR: could not >> access status of transaction 13514992 >> Nov 23 14:38:32 dev postgres[23810]: [4-2] user=dev,db=dev DET

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-25 Thread Andres Freund
Hi, On 2013-11-24 16:56:26 -0500, J Smith wrote: > coredumper worked like a charm. Useful tool, that is... although as a > bit of advice, I'd try not to run it on Postgres if your various > memory settings are tweaked towards production use -- the core dump > that was captured on my server weighed

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-24 Thread J Smith
coredumper worked like a charm. Useful tool, that is... although as a bit of advice, I'd try not to run it on Postgres if your various memory settings are tweaked towards production use -- the core dump that was captured on my server weighed in at 16 GB. Anyways, I've attached both the log entries

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-23 Thread J Smith
On Tue, Nov 19, 2013 at 10:16 AM, J Smith wrote: > Alright, we'll look into doing that heading into the weekend. > Interestingly, we haven't experienced the issue since our main Java > developer made some modifications to our backend system. I'm not > entirely sure what the changes entail except t

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-19 Thread J Smith
Alright, we'll look into doing that heading into the weekend. Interestingly, we haven't experienced the issue since our main Java developer made some modifications to our backend system. I'm not entirely sure what the changes entail except that it's a one-liner that involves re-SELECTing a table du

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-19 Thread Robert Haas
On Fri, Nov 15, 2013 at 4:01 PM, J Smith wrote: > On Fri, Nov 15, 2013 at 3:21 PM, Robert Haas wrote: >> I think what would help the most is if you could arrange to obtain a >> stack backtrace at the point when the error is thrown. Maybe put a >> long sleep call in just before the error happens,

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-15 Thread J Smith
On Fri, Nov 15, 2013 at 3:21 PM, Robert Haas wrote: > > I think what would help the most is if you could arrange to obtain a > stack backtrace at the point when the error is thrown. Maybe put a > long sleep call in just before the error happens, and when it gets > stuck there, attach gdb and run

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-15 Thread Robert Haas
On Wed, Nov 13, 2013 at 12:29 PM, J Smith wrote: > Looks like we got another set of errors overnight. Here's the log file > from the errors. (Log file scrubbed slightly to remove private data, > but still representative of the problem I believe.) > > Nov 13 05:34:34 dev postgres[6084]: [4-1] user=

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-13 Thread J Smith
Looks like we got another set of errors overnight. Here's the log file from the errors. (Log file scrubbed slightly to remove private data, but still representative of the problem I believe.) Nov 13 05:34:34 dev postgres[6084]: [4-1] user=dev,db=dev ERROR: could not access status of transaction 63

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-12 Thread J Smith
On Tue, Nov 12, 2013 at 11:55 AM, Andres Freund wrote: > Hi, > > On 2013-11-12 11:46:19 -0500, J Smith wrote: >> > * Does SELECT count(*) FROM pg_prepared_xacts; return 0? >> >> Yes it does. > > Could you show the output? Do you actually use prepared xacts actively? jay:dev@jagger=# select * from

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-12 Thread J Smith
On Tue, Nov 12, 2013 at 11:54 AM, Stephen Frost wrote: > > Did you also upgrade to PostGIS 2.x as part of this..? Seems like it'd > be unrelated, but one never knows. Any chance you could distill this > down into a small test case which exhibits the problem? I'm guessing > 'no', but figured I'd

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-12 Thread Andres Freund
Hi, On 2013-11-12 11:46:19 -0500, J Smith wrote: > > * Does SELECT count(*) FROM pg_prepared_xacts; return 0? > > Yes it does. Could you show the output? Do you actually use prepared xacts actively? Do you actively use row level locking? Is there high concurrency in that environment? In short,

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-12 Thread Stephen Frost
* J Smith (dark.panda+li...@gmail.com) wrote: > We haven't been able to use pg_upgrade as we rely heavily on PostGIS > and do hard upgrades via pg_dump and postgis_restore.pl when we > upgrade. Did you also upgrade to PostGIS 2.x as part of this..? Seems like it'd be unrelated, but one never know

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-12 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > He referred to using pg_dumpall/pg_dump. But that bug was erroring out > on pg_clog, not pg_subtrans, right? Yeah, that was pg_clog. Obviously responded before really looking at it. :) > My gut feeling is thats it's related to foreign key locks d

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-12 Thread J Smith
G'day Andres. On Tue, Nov 12, 2013 at 11:13 AM, Andres Freund wrote: > Hi, > > On 2013-11-12 10:56:55 -0500, J Smith wrote: >> G'day list. Didn't get any interest in pgsql-general, thought I'd try >> my luck here, which perhaps would be more fitting in case I've >> stumbled upon an edge case issu

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-12 Thread J Smith
On Tue, Nov 12, 2013 at 11:25 AM, Stephen Frost wrote: > > How was this upgrade done? If you used pg_upgrade, what version of the > pg_upgrade code did you use? As I recall, there was a bug in older > versions which could exhibit in this way.. > > http://wiki.postgresql.org/wiki/20110408pg_upgra

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-12 Thread Andres Freund
On 2013-11-12 11:25:03 -0500, Stephen Frost wrote: > * J Smith (dark.panda+li...@gmail.com) wrote: > > I've recently upgraded a number of servers from PostgreSQL 9.2.5 to > > 9.3.1 and have started getting the following errors every couple of > > hours along with some failed transactions. > > How

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-12 Thread Stephen Frost
* J Smith (dark.panda+li...@gmail.com) wrote: > I've recently upgraded a number of servers from PostgreSQL 9.2.5 to > 9.3.1 and have started getting the following errors every couple of > hours along with some failed transactions. How was this upgrade done? If you used pg_upgrade, what version of

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-12 Thread Andres Freund
Hi, On 2013-11-12 10:56:55 -0500, J Smith wrote: > G'day list. Didn't get any interest in pgsql-general, thought I'd try > my luck here, which perhaps would be more fitting in case I've > stumbled upon an edge case issue or something... Normally the bug report for/the -bugs mailing list is the ri

Re: [HACKERS] Errors with run_build.pl - 8.3RC2

2008-01-22 Thread Andrew Dunstan
cinu wrote: Hi All, I was running the run_Build.pl script that is specific to Buildfarm and encountered errors. I am listing out the names of the logfiles and the errors that I have seen. Can anyone give me some clarity on these errors? Even though these errors are existing, at the end the l

Re: [HACKERS] errors

2003-07-07 Thread Christopher Kings-Lynne
Hi Ivan, 7.4CVS already supports this. Regards, Chris - Original Message - From: "ivan" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Monday, July 07, 2003 2:58 PM Subject: [HACKERS] errors > > hi, > > when be meet error send string to fe. Is possible to be will send error > no

Re: [HACKERS] Errors compiling hba.c in current CVS

2003-06-17 Thread Bruno Wolff III
On Mon, Jun 16, 2003 at 17:21:01 -0400, Bruce Momjian <[EMAIL PROTECTED]> wrote: > > Does the following patch fix the problem? It doesn't use sa_family_t > anymore. I tried current CVS and current CVS with the patch you attached and it still didn't work. ---(end of bro

Re: [HACKERS] Errors compiling hba.c in current CVS

2003-06-17 Thread Kurt Roeckx
On Mon, Jun 16, 2003 at 05:21:01PM -0400, Bruce Momjian wrote: > > Does the following patch fix the problem? It doesn't use sa_family_t > anymore. > > ! sa_family_t ss_family; /* address family */ [...] > ! char dummy_sa_family[SIZEOF_SOCKADDR_FAMILY]; That is NOT going to work

Re: [HACKERS] Errors compiling hba.c in current CVS

2003-06-17 Thread Bruno Wolff III
On Mon, Jun 16, 2003 at 17:21:01 -0400, Bruce Momjian <[EMAIL PROTECTED]> wrote: > > Does the following patch fix the problem? It doesn't use sa_family_t > anymore. I tried using the pacth and it didn't help. I am going to get a fresh CVS copy and see if that works. --

Re: [HACKERS] Errors compiling hba.c in current CVS

2003-06-16 Thread Bruce Momjian
Does the following patch fix the problem? It doesn't use sa_family_t anymore. --- Kurt Roeckx wrote: > On Mon, Jun 16, 2003 at 02:23:31PM -0500, Bruno Wolff III wrote: > > On Mon, Jun 16, 2003 at 11:47:58 -0500, > > Brun

Re: [HACKERS] Errors compiling hba.c in current CVS

2003-06-16 Thread Kurt Roeckx
On Mon, Jun 16, 2003 at 02:23:31PM -0500, Bruno Wolff III wrote: > On Mon, Jun 16, 2003 at 11:47:58 -0500, > Bruno Wolff III <[EMAIL PROTECTED]> wrote: > > I get the errors below when compiling on a RH 6.1 system. > > I used the following config paramters: > > ./configure --prefix=/usr/local/pgsq

Re: [HACKERS] Errors compiling hba.c in current CVS

2003-06-16 Thread Kurt Roeckx
On Mon, Jun 16, 2003 at 03:36:55PM -0400, Bruce Momjian wrote: > > I am working on this now. The missing typedef for sa_family_t is really > just used for structure alignment, so I am working on a fix to define a > char array and #define to be the same length as the native ss_family, > because on

Re: [HACKERS] Errors compiling hba.c in current CVS

2003-06-16 Thread Bruce Momjian
I am working on this now. The missing typedef for sa_family_t is really just used for structure alignment, so I am working on a fix to define a char array and #define to be the same length as the native ss_family, because on my system sa_family_t is: sys/sockettypes.h:11:typedef u_char s

Re: [HACKERS] Errors compiling hba.c in current CVS

2003-06-16 Thread Bruno Wolff III
On Mon, Jun 16, 2003 at 11:47:58 -0500, Bruno Wolff III <[EMAIL PROTECTED]> wrote: > I get the errors below when compiling on a RH 6.1 system. > I used the following config paramters: > ./configure --prefix=/usr/local/pgsql --enable-integer-datetimes --with-pgport=5433 > > hba.c: In function `pa