Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Heikki Linnakangas
Committed. Tom Lane wrote: * You should try to get rid of LOCALE_NAME_BUFLEN altogether. Definitely the comment about it in pg_control.h is now obsolete. Yep. I removed LOCALE_NAME_BUFLEN. The real max length of a locale name is now NAMEDATALEN, because it's stored in a name field in

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Martijn van Oosterhout
On Mon, Sep 22, 2008 at 10:22:35AM +0300, Heikki Linnakangas wrote: BTW, the original patch didn't have any provision for creating rows in pg_collation reflecting the locales available in the OS, but I think we'd need that. Otherwise the DBA would need to manually run CREATE COLLATION for

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Dave Page
On Tue, Sep 23, 2008 at 10:20 AM, Heikki Linnakangas [EMAIL PROTECTED] wrote: Committed. *adds yet another item to his pgAdmin todo list* :-( Tom Lane wrote: * You should try to get rid of LOCALE_NAME_BUFLEN altogether. Definitely the comment about it in pg_control.h is now obsolete.

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Martijn van Oosterhout
On Mon, Sep 22, 2008 at 06:11:04PM +0300, Heikki Linnakangas wrote: This patch should allow to use both system catalog and ICU. Not without another patch that actually introduces ICU support. What that would look like, how that would be stored in the catalogs, and whether we want that is

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Heikki Linnakangas
Martijn van Oosterhout wrote: On Mon, Sep 22, 2008 at 10:22:35AM +0300, Heikki Linnakangas wrote: BTW, the original patch didn't have any provision for creating rows in pg_collation reflecting the locales available in the OS, but I think we'd need that. Otherwise the DBA would need to manually

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Martijn van Oosterhout
On Tue, Sep 23, 2008 at 01:32:38PM +0300, Heikki Linnakangas wrote: Martijn van Oosterhout wrote: You're assuming collations are denumerable. They're not. There is no way to find the list of available collations/locales. You may be able to guess a few but certainly not all of them. locale

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Zdenek Kotala
Martijn van Oosterhout napsal(a): On Mon, Sep 22, 2008 at 06:11:04PM +0300, Heikki Linnakangas wrote: This patch should allow to use both system catalog and ICU. Not without another patch that actually introduces ICU support. What that would look like, how that would be stored in the catalogs,

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Heikki Linnakangas
Zdenek Kotala wrote: pg_collation catalog is also important for pg_dump, because system collation names are not compatible over OS and pg_dump output should be portable. pg_collation adds abstract layer which solve this problem. That's a valid point. We'll still need a way to map OS locale to

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Tom Lane
Martijn van Oosterhout [EMAIL PROTECTED] writes: On Tue, Sep 23, 2008 at 01:32:38PM +0300, Heikki Linnakangas wrote: locale -a manages to do it somehow... Sure, by (on glibc) opening the binary archive and parsing it and then trying to reverse lookup the alias list. We could ofcourse program

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Magnus Hagander
Tom Lane wrote: Martijn van Oosterhout [EMAIL PROTECTED] writes: On Tue, Sep 23, 2008 at 01:32:38PM +0300, Heikki Linnakangas wrote: locale -a manages to do it somehow... Sure, by (on glibc) opening the binary archive and parsing it and then trying to reverse lookup the alias list. We could

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: pg_collation catalog is also important for pg_dump, because system collation names are not compatible over OS and pg_dump output should be portable. pg_collation adds abstract layer which solve this problem. That's a valid point. We'll still

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Petr Jelinek
Magnus Hagander wrote: exec(locale -a) ... I suppose we'd need something else for Windows, but I'm sure there's a way. IIRC, the data is in the registry. Should be enumerable somehow - we'll have to do it platform specific of course, but it's not the first time we'd do that for windows...

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Magnus Hagander
Petr Jelinek wrote: Magnus Hagander wrote: exec(locale -a) ... I suppose we'd need something else for Windows, but I'm sure there's a way. IIRC, the data is in the registry. Should be enumerable somehow - we'll have to do it platform specific of course, but it's not the first time we'd do

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Heikki Linnakangas
Martijn van Oosterhout wrote: On Fri, Sep 19, 2008 at 10:13:43AM +0300, Heikki Linnakangas wrote: It's not like the patch is going to disappear from planet Earth if it doesn't get committed for 8.4. It's still valuable and available when the new catalogs are needed. I just prefer it as it

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: The first thing that we see is that the COLLATE keyword means different things, so it's probably best to change that into: CREATE DATABASE name WITH LC_COLLATE=locale name LC_CTYPE=locale name in the stripped-down version. Then we need a way to

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Heikki Linnakangas
Tom Lane wrote: What might be sensible to ask is whether it is ever actually reasonable for LC_COLLATE and LC_CTYPE to have different settings. If we were willing to enforce that they be the same, we could reduce this to just the standard syntax COLLATE=something and be done with it. Not being

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): For anyone counting, Firebird added support for ICU more than three years ago. ICU is orthogonal to this patch. This patch didn't provide ICU support, and we could start using ICU without the catalog changes. This patch should allow to use both system catalog

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Heikki Linnakangas
Zdenek Kotala wrote: Heikki Linnakangas napsal(a): For anyone counting, Firebird added support for ICU more than three years ago. ICU is orthogonal to this patch. This patch didn't provide ICU support, and we could start using ICU without the catalog changes. This patch should allow to

Re: [HACKERS] WIP patch: Collation support

2008-09-19 Thread Martijn van Oosterhout
On Thu, Sep 18, 2008 at 05:41:16PM +0300, Heikki Linnakangas wrote: Attached is an updated version of the stripped-down patch. I've cleaned it up a bit, and added more sanity checks. Documentation is still missing and I haven't test it much. FWIW, I disagree with the stripping down and think

Re: [HACKERS] WIP patch: Collation support

2008-09-19 Thread Heikki Linnakangas
Martijn van Oosterhout wrote: FWIW, I disagree with the stripping down and think we should aim for the whole patch as submitted. Historically, the hardest part of getting collation support into postgres has been the catalog changes and easier the support for something other than OS locales. I

Re: [HACKERS] WIP patch: Collation support

2008-09-19 Thread Heikki Linnakangas
Here's an updated version of the stripped-down patch, now with documentation changes, plus a couple of minor bug fixes. I think this is looking pretty good now, and I would appreciate review from others before I go ahead committing this. Is there any more places in the documentation that

Re: [HACKERS] WIP patch: Collation support

2008-09-19 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Here's an updated version of the stripped-down patch, now with documentation changes, plus a couple of minor bug fixes. I think this is looking pretty good now, and I would appreciate review from others before I go ahead committing this. I guess

Re: [HACKERS] WIP patch: Collation support

2008-09-19 Thread Heikki Linnakangas
Heikki Linnakangas wrote: Here's an updated version of the stripped-down patch, now with documentation changes, plus a couple of minor bug fixes. Another update, marching towards committing. Now with pg_dump/pg_dumpall support, and collation/ctype is also shown in psql \l output. I wonder

Re: [HACKERS] WIP patch: Collation support

2008-09-19 Thread Martijn van Oosterhout
On Fri, Sep 19, 2008 at 10:13:43AM +0300, Heikki Linnakangas wrote: In addition to catalog changes, for finer grained collation you need changes in the planner and executor as well. In planner, to provide support for the concept of a sort order using a specific collation, and track that

Re: [HACKERS] WIP patch: Collation support

2008-09-18 Thread Heikki Linnakangas
Gregory Stark wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Well, I proposed disallowing using a different collation than the source database, except for using template0 as the source. That's pretty limited, but is trivial to implement and still let's you have databases with different

Re: [HACKERS] WIP patch: Collation support

2008-09-16 Thread Heikki Linnakangas
Martijn van Oosterhout wrote: On Wed, Sep 10, 2008 at 12:51:02PM +0300, Heikki Linnakangas wrote: Since the set of collations isn't exactly denumerable, we need some way to allow the user to specify the collation they want. The only collation PostgreSQL knows about is the C collation. Anything

Re: [HACKERS] WIP patch: Collation support

2008-09-16 Thread Gregory Stark
Heikki Linnakangas [EMAIL PROTECTED] writes: Well, I proposed disallowing using a different collation than the source database, except for using template0 as the source. That's pretty limited, but is trivial to implement and still let's you have databases with different collations in the same

Re: [HACKERS] WIP patch: Collation support

2008-09-16 Thread Zdenek Kotala
Gregory Stark napsal(a): Heikki Linnakangas [EMAIL PROTECTED] writes: snip AFAIK we can't easily connect to the new database and do some fiddling with it, can we? If we could we could check if there are any non-empty indexes which depend on the collation and only print the warning if we

Re: [HACKERS] WIP patch: Collation support

2008-09-16 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Martijn van Oosterhout wrote: On Wed, Sep 10, 2008 at 12:51:02PM +0300, Heikki Linnakangas wrote: Since the set of collations isn't exactly denumerable, we need some way to allow the user to specify the collation they want. The only collation PostgreSQL knows

Re: [HACKERS] WIP patch: Collation support

2008-09-16 Thread Alvaro Herrera
Zdenek Kotala escribió: Gregory Stark napsal(a): AFAIK we can't easily connect to the new database and do some fiddling with it, can we? If we could we could check if there are any non-empty indexes which depend on the collation and only print the warning if we find any (and even mark them

Re: [HACKERS] WIP patch: Collation support

2008-09-10 Thread Heikki Linnakangas
Radek Strnad wrote: Progress so far: - created catalogs pg_collation a pg_charset which are filled with three standard collations - initdb changes rows called DEFAULT in both catalogs during the bki bootstrap phase with current system LC_COLLATE and LC_CTYPE or those set by command line. - new

Re: [HACKERS] WIP patch: Collation support

2008-09-10 Thread Martijn van Oosterhout
On Wed, Sep 10, 2008 at 11:29:14AM +0300, Heikki Linnakangas wrote: Radek Strnad wrote: - because of pg_collation and pg_charset are catalogs individual for each database, if you want to create a database with collation other than specified, create it in template1 and then create database I

Re: [HACKERS] WIP patch: Collation support

2008-09-10 Thread Heikki Linnakangas
Martijn van Oosterhout wrote: On Wed, Sep 10, 2008 at 11:29:14AM +0300, Heikki Linnakangas wrote: Radek Strnad wrote: - because of pg_collation and pg_charset are catalogs individual for each database, if you want to create a database with collation other than specified, create it in template1

Re: [HACKERS] WIP patch: Collation support

2008-09-10 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Design functionality changes left: - move retrieveing collation from pg_database to pg_type The problem there is that pg_collation is local catalog, but pg_database is global catalog. IIRC, It was discussed during last commitfest. I think it is bad idea to

Re: [HACKERS] WIP patch: Collation support

2008-09-10 Thread Martijn van Oosterhout
On Wed, Sep 10, 2008 at 12:51:02PM +0300, Heikki Linnakangas wrote: Since the set of collations isn't exactly denumerable, we need some way to allow the user to specify the collation they want. The only collation PostgreSQL knows about is the C collation. Anything else is user-defined.

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Peter Eisentraut
Radek Strnad wrote: - new collations can be defined with command CREATE COLLATION collation name FOR character set specification FROM existing collation name [STRCOLFN fn name] [ pad characteristic ] [ case sensitive ] [ LCCOLLATE lc_collate ] [ LCCTYPE lc_ctype ] How do you plan to make a

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Martijn van Oosterhout
On Tue, Sep 02, 2008 at 02:50:47PM +0300, Peter Eisentraut wrote: Radek Strnad wrote: - new collations can be defined with command CREATE COLLATION collation name FOR character set specification FROM existing collation name [STRCOLFN fn name] [ pad characteristic ] [ case sensitive ] [

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Radek Strnad
I think at least case sensitivity can be done by comparing two strings converted to upper case with toupper() function. Regards Radek Strnad On Tue, Sep 2, 2008 at 2:00 PM, Martijn van Oosterhout [EMAIL PROTECTED]wrote: On Tue, Sep 02, 2008 at 02:50:47PM +0300, Peter Eisentraut wrote:

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Peter Eisentraut
Martijn van Oosterhout wrote: On Tue, Sep 02, 2008 at 02:50:47PM +0300, Peter Eisentraut wrote: Radek Strnad wrote: - new collations can be defined with command CREATE COLLATION collation name FOR character set specification FROM existing collation name [STRCOLFN fn name] [ pad

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Martijn van Oosterhout
On Tue, Sep 02, 2008 at 04:46:16PM +0300, Peter Eisentraut wrote: While it's true POSIX locales don't handle this, other collation libraries do and we should support them if the user wants. Do they handle exactly those two attributes specifically? Can you point out references? Or do you

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Gregory Stark
Martijn van Oosterhout [EMAIL PROTECTED] writes: On Tue, Sep 02, 2008 at 04:46:16PM +0300, Peter Eisentraut wrote: While it's true POSIX locales don't handle this, other collation libraries do and we should support them if the user wants. I think that's backwards. We have to go with the

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Martijn van Oosterhout
On Tue, Sep 02, 2008 at 05:42:13PM +0100, Gregory Stark wrote: Martijn van Oosterhout [EMAIL PROTECTED] writes: On Tue, Sep 02, 2008 at 04:46:16PM +0300, Peter Eisentraut wrote: While it's true POSIX locales don't handle this, other collation libraries do and we should support them if the

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Alvaro Herrera
Radek Strnad escribió: Ok, so do you suggest to leave it with a notice reindex database or start to solve it somehow? I don't know. If there are two tasks that need the same treatment, it seems a safe conclusion that they need a common solution. -- Alvaro Herrera

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Tom Lane
Martijn van Oosterhout [EMAIL PROTECTED] writes: And I think that's backwards. Why can we only use a feature once every OS out there implements it? We still run on systems that don't have SSL support. LC_TYPE settings are not portable between systems, yet that doesn't bother anyone. Why should

Re: [HACKERS] WIP patch: Collation support

2008-09-01 Thread Radek Strnad
Ok, so do you suggest to leave it with a notice reindex database or start to solve it somehow? Regards Radek Strnad On Mon, Sep 1, 2008 at 12:08 AM, Alvaro Herrera [EMAIL PROTECTED]wrote: Radek Strnad escribió: - when creating database with different collation than database cluster, the

Re: [HACKERS] WIP patch: Collation support

2008-08-31 Thread Alvaro Herrera
Radek Strnad escribió: - when creating database with different collation than database cluster, the database has to be reindexed. Any idea how to do it? Function ReindexDatabase works only when database is opened. We have this Todo item: Set proper permissions on non-system schemas during