Re: [HACKERS] Upcoming PG re-releases

2005-12-10 Thread Robert Treat
Was thinking if someone could summarize this all it would make a really good FAQ entry. Robert Treat On Friday 09 December 2005 13:28, Martijn van Oosterhout wrote: On Fri, Dec 09, 2005 at 12:38:21PM -0500, Bruce Momjian wrote: This means someone who is planning on upgrading to 8.1 in two

Re: [HACKERS] Upcoming PG re-releases

2005-12-10 Thread Bruce Momjian
I don't see it asked very often, and I think our 8.1 releae note addition (plus a mention in the 8.1.1 notes) will complete this. --- Robert Treat wrote: Was thinking if someone could summarize this all it would make a

Re: [HACKERS] Upcoming PG re-releases

2005-12-10 Thread Joshua D. Drake
Bruce Momjian wrote: I don't see it asked very often, and I think our 8.1 releae note addition (plus a mention in the 8.1.1 notes) will complete this. Actually a upgrade FAQ is probably a good idea. Something that says what really happens when foo changes in 8.1 or how foo is different

Re: [HACKERS] Upcoming PG re-releases

2005-12-09 Thread Martijn van Oosterhout
On Thu, Dec 08, 2005 at 05:54:35PM -0500, Gregory Maxwell wrote: No, what is needed for people who care about fixing their data is a loadable strip_invalid_utf8() that works in older versions.. then just select * from bar where foo != strip_invalid_utf8(foo); The function would be useful in

Re: [HACKERS] Upcoming PG re-releases

2005-12-09 Thread Bruce Momjian
Martijn van Oosterhout wrote: -- Start of PGP signed section. On Thu, Dec 08, 2005 at 05:54:35PM -0500, Gregory Maxwell wrote: No, what is needed for people who care about fixing their data is a loadable strip_invalid_utf8() that works in older versions.. then just select * from bar where

Re: [HACKERS] Upcoming PG re-releases

2005-12-09 Thread Martijn van Oosterhout
On Fri, Dec 09, 2005 at 11:34:22AM -0500, Bruce Momjian wrote: I think the problem with any kind of function-call detection is that the data has to get into the database first, and it isn't clear how someone loading a failed dump would do that aside from modifying the column to bytea in the

Re: [HACKERS] Upcoming PG re-releases

2005-12-09 Thread Bruce Momjian
Martijn van Oosterhout wrote: -- Start of PGP signed section. On Fri, Dec 09, 2005 at 11:34:22AM -0500, Bruce Momjian wrote: I think the problem with any kind of function-call detection is that the data has to get into the database first, and it isn't clear how someone loading a failed dump

Re: [HACKERS] Upcoming PG re-releases

2005-12-09 Thread Martijn van Oosterhout
On Fri, Dec 09, 2005 at 12:38:21PM -0500, Bruce Momjian wrote: This means someone who is planning on upgrading to 8.1 in two months can use this function now to weed out the bad data before the upgrade even starts. Oh, so you back-load it into the old database. Interesting. I assume to

Re: [HACKERS] Upcoming PG re-releases

2005-12-08 Thread Gavin Sherry
On Tue, 6 Dec 2005, Bruce Momjian wrote: Exactly what does vim do that iconv does not? Fuzzy encoding sounds scary to me. Right. It actually makes assumptions about the source encoding. People who care about their data need, unfortunately, to spend a bit of time on this problem. I've been

Re: [HACKERS] Upcoming PG re-releases

2005-12-08 Thread Bruce Momjian
Gavin Sherry wrote: On Tue, 6 Dec 2005, Bruce Momjian wrote: Exactly what does vim do that iconv does not? Fuzzy encoding sounds scary to me. Right. It actually makes assumptions about the source encoding. People who care about their data need, unfortunately, to spend a bit of time

Re: [HACKERS] Upcoming PG re-releases

2005-12-08 Thread Gregory Maxwell
On 12/8/05, Bruce Momjian pgman@candle.pha.pa.us wrote: A script which identifies non-utf-8 characters and provides some context, line numbers, etc, will greatly speed up the process of remedying the situation. I think the best we can do is the iconv -c with the diff

Re: [HACKERS] Upcoming PG re-releases

2005-12-06 Thread Bruce Momjian
I have added your suggestions to the 8.1.X release notes. --- Paul Lindner wrote: -- Start of PGP signed section. On Sat, Dec 03, 2005 at 10:54:08AM -0500, Bruce Momjian wrote: Neil Conway wrote: On Wed, 2005-11-30 at

Re: [HACKERS] Upcoming PG re-releases

2005-12-06 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes: I have added your suggestions to the 8.1.X release notes. Did you read the followup discussion? Recommending -c without a large warning seems a very bad idea. regards, tom lane ---(end of

Re: [HACKERS] Upcoming PG re-releases

2005-12-06 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: I have added your suggestions to the 8.1.X release notes. Did you read the followup discussion? Recommending -c without a large warning seems a very bad idea. Well, I said it would remove invalid sequences. What else should we

Re: [HACKERS] Upcoming PG re-releases

2005-12-06 Thread Bruce Momjian
Bruce Momjian wrote: Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: I have added your suggestions to the 8.1.X release notes. Did you read the followup discussion? Recommending -c without a large warning seems a very bad idea. Well, I said it would remove invalid

Re: [HACKERS] Upcoming PG re-releases

2005-12-06 Thread Peter Eisentraut
Bruce Momjian wrote: One nice solution would be if iconv would report the lines with errors and you could correct them, but I see no way to do that. The only thing you could do is to diff the old and new files to see the problems. Is that helpful? Here is new text I have used: I think this

Re: [HACKERS] Upcoming PG re-releases

2005-12-06 Thread Bruce Momjian
Nice, updated. --- Peter Eisentraut wrote: Bruce Momjian wrote: One nice solution would be if iconv would report the lines with errors and you could correct them, but I see no way to do that. The only thing you

Re: [HACKERS] Upcoming PG re-releases

2005-12-06 Thread Gavin Sherry
Hi, On Tue, 6 Dec 2005, Bruce Momjian wrote: Nice, updated. --- I think my suggestion from the other day is useful also. --- Omar Kilani and I have spent a few hours looking at the problem. For situations where there

Re: [HACKERS] Upcoming PG re-releases

2005-12-06 Thread Bruce Momjian
Exactly what does vim do that iconv does not? Fuzzy encoding sounds scary to me. --- Gavin Sherry wrote: Hi, On Tue, 6 Dec 2005, Bruce Momjian wrote: Nice, updated.

Re: [HACKERS] Upcoming PG re-releases

2005-12-04 Thread Paul Lindner
On Sat, Dec 03, 2005 at 10:54:08AM -0500, Bruce Momjian wrote: Neil Conway wrote: On Wed, 2005-11-30 at 10:56 -0500, Tom Lane wrote: It's been about a month since 8.1.0 was released, and we've found about the usual number of bugs for a new release, so it seems like it's time for 8.1.1.

Re: [HACKERS] Upcoming PG re-releases

2005-12-04 Thread Tom Lane
Paul Lindner [EMAIL PROTECTED] writes: To convert your pre-8.1 database to 8.1 you may have to remove and/or fix the offending characters. One simple way to fix the problem is to run your pg_dump output through the iconv command like this: iconv -c -f UTF8 -t UTF8 -o fixed.sql dump.sql Is

Re: [HACKERS] Upcoming PG re-releases

2005-12-04 Thread Paul Lindner
On Sun, Dec 04, 2005 at 11:34:16AM -0500, Tom Lane wrote: Paul Lindner [EMAIL PROTECTED] writes: To convert your pre-8.1 database to 8.1 you may have to remove and/or fix the offending characters. One simple way to fix the problem is to run your pg_dump output through the iconv command

Re: [HACKERS] Upcoming PG re-releases

2005-12-04 Thread Tom Lane
Paul Lindner [EMAIL PROTECTED] writes: On Sun, Dec 04, 2005 at 11:34:16AM -0500, Tom Lane wrote: Paul Lindner [EMAIL PROTECTED] writes: iconv -c -f UTF8 -t UTF8 -o fixed.sql dump.sql Is that really a one-size-fits-all solution? Especially with -c? I'd say yes, and the -c flag is needed so

Re: [HACKERS] Upcoming PG re-releases

2005-12-04 Thread Martijn van Oosterhout
On Sun, Dec 04, 2005 at 12:19:32PM -0500, Gregory Maxwell wrote: That's exactly what's bothering me about it. If we recommend that we had better put a large THIS WILL DESTROY YOUR DATA warning first. The problem is that the data is not invalid from the user's point of view --- more

Re: [HACKERS] Upcoming PG re-releases

2005-12-04 Thread Gavin Sherry
Hi all, On Sun, 4 Dec 2005, Tom Lane wrote: Paul Lindner [EMAIL PROTECTED] writes: To convert your pre-8.1 database to 8.1 you may have to remove and/or fix the offending characters. One simple way to fix the problem is to run your pg_dump output through the iconv command like this:

Re: [HACKERS] Upcoming PG re-releases

2005-12-03 Thread Neil Conway
On Wed, 2005-11-30 at 10:56 -0500, Tom Lane wrote: It's been about a month since 8.1.0 was released, and we've found about the usual number of bugs for a new release, so it seems like it's time for 8.1.1. I think one fix that should be made in time for 8.1.1 is adding a note to the version

Re: [HACKERS] Upcoming PG re-releases

2005-12-03 Thread Bruce Momjian
Neil Conway wrote: On Wed, 2005-11-30 at 10:56 -0500, Tom Lane wrote: It's been about a month since 8.1.0 was released, and we've found about the usual number of bugs for a new release, so it seems like it's time for 8.1.1. I think one fix that should be made in time for 8.1.1 is adding

Re: [HACKERS] Upcoming PG re-releases

2005-11-30 Thread Andrew Dunstan
Tom Lane wrote: We will at the same time be making new dot-releases in the 7.3, 7.4, and 8.0 branches, principally to fix the SLRU race condition reported by Jim Nasby and Robert Creager. Was there a conclusion out of the recent discussion on EOL policy? The consensus seemed to be

Re: [HACKERS] Upcoming PG re-releases

2005-11-30 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: Have we actually officially stopped supporting the 7.2 series? Yeah, we have. It reached the too difficult to support point already (the VACUUM/ctid bug back in August --- the patch used in the later branches wouldn't apply at all, IIRC). All this

Re: [HACKERS] Upcoming PG re-releases

2005-11-30 Thread Robert Treat
On Wednesday 30 November 2005 11:40, Tom Lane wrote: Personally I expect to keep supporting 7.3 for a long while, because Red Hat pays me to ;-) ... and the EOL date for RHEL3 is a long way away yet. The PG community may stop bothering with 7.3 releases before that. But I think Marc and Bruce

Re: [HACKERS] Upcoming PG re-releases

2005-11-30 Thread Andrew Dunstan
Tom Lane said: We hashed all this out in the pghackers list back in August, but I agree there ought to be something about it on the website. The reason I asked again is that, notwithstanding the recent discussion, I have observed confusion about the matter (including Jan telling me he didn't

Re: [HACKERS] Upcoming PG re-releases

2005-11-30 Thread David Fetter
On Wed, Nov 30, 2005 at 01:23:38PM -0500, Robert Treat wrote: On Wednesday 30 November 2005 11:40, Tom Lane wrote: Personally I expect to keep supporting 7.3 for a long while, because Red Hat pays me to ;-) ... and the EOL date for RHEL3 is a long way away yet. The PG community may stop