On 12/8/05, Bruce Momjian <pgman@candle.pha.pa.us> wrote: > > A script which identifies non-utf-8 characters and provides some > > context, line numbers, etc, will greatly speed up the process of > > remedying the situation. > > I think the best we can do is the "iconv -c with the diff" idea, which > is already in the release notes. I suppose we could merge the iconv and > diff into a single command, but I don't see a portable way to output the > iconv output to stdout., /dev/stdin not being portable.
No, what is needed for people who care about fixing their data is a loadable strip_invalid_utf8() that works in older versions.. then just select * from bar where foo != strip_invalid_utf8(foo); The function would be useful in general, for example, if you have an application which doesn't already have much utf8 logic, you want to use a text field, and stripping is the behaviour you want. For example, lots of simple web applications. ---------------------------(end of broadcast)--------------------------- TIP 5: don't forget to increase your free space map settings
