[HACKERS] Are postgresql-9.4 binaries available on Windows XP?
Hi Dave and Andrew, I recently noticed the thread [BUGS] BUG #11039: installation fails when trying to install C++ redistributable . Unfortunately I have no XP machine at hand and can't test the installer by myself. Looking at the binaries in the package, they seem to be built using Visual Studio 2013. I'm suspicious if the binaries are available on Windows XP. If I recognize correctly, Visual Studio 2012 or later doesn't support Windows XP by default and Platform Toolset v120_xp (or v110_xp) must be specified so as to build binaries guaranteed to be avaiable on Windows XP. However MSBuildProject.pm seems to specify v120 (or v110) as the PlarformToolset property. Is it intentional? regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New parameter RollbackError to control rollback behavior on error
Hi Michael, Isn't it an ODBC issue? regards, Hiroshi Inoue (2014/03/26 15:39), Michael Paquier wrote: Hi all, The behavior of rollback when an error occurs on an handle is controlled by extending Protocol with $PROTNUM-[0|1|2] where: - 0 = let the application handle rollbacks - 1 = rollback whole transaction when an error occurs - 2 = rollback only statement that failed Using such an extension is somewhat awkward as a single string is used for two settings... The proposed attached patch adds a new parameter called RollbackError that allows to control the behavior of rollback on error with a different parameter. For backward-compatibility purposes, this patch does not break the old grammar of Protocol: it just gives the priority to RollbackError if both Protocol and RollbackError are set for a connection. Regression tests to test RollbackError and combinations of RollbackError/Protocol are added in the patch in the existing test error-rollback (which has needed some refactoring, older tests are not impacted). Docs are included as well. I thought first about including that in my cleanup work for 9.4, but as this actually does not break the driver it may be worth adding it directly to master, explaining the patch attached here. Comments welcome. Note that if there are objections I do not mind adding that for the work that would be merged later to 9.4 builds. Regards, -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/18 0:02), Dave Page wrote: On Mon, Feb 17, 2014 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: On Fri, Feb 14, 2014 at 5:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: (BTW, narwhal is evidently not trying to build plpython. I wonder why not?) Not sure - it's certainly installed on the box. I've enabled it for now, and will see what happens. Sigh ... stop the presses. In 9.3, narwhal is *still* showing a PGDLLIMPORT-type failure that no other Windows critter is unhappy about: dlltool --export-all --output-def worker_spi.def worker_spi.o dllwrap -o worker_spi.dll --def worker_spi.def worker_spi.o -L../../src/port -L../../src/common -Wl,--allow-multiple-definition -L/mingw/lib -Wl,--as-needed -L../../src/backend -lpostgres Info: resolving _MyBgworkerEntry by linking to __imp__MyBgworkerEntry (auto-import) fu01.o(.idata$3+0xc): undefined reference to `libpostgres_a_iname' fu02.o(.idata$3+0xc): undefined reference to `libpostgres_a_iname' nmth00.o(.idata$4+0x0): undefined reference to `_nm__MyBgworkerEntry' collect2: ld returned 1 exit status So we are back to square one AFAICS: we still have no idea why narwhal is pickier than everything else. (BTW, to save people the trouble of looking: MyBgworkerEntry is marked PGDLLIMPORT in HEAD but not 9.3.) Also, in HEAD narwhal is building things OK, but then seems to be dumping core in the dblink regression test, leaving one with not a very warm feeling about whether the contrib executables it's building are any good. Well, as we know, Narwhal is really quite old now. I think I built it seven+ years ago. There is a difference between narwhal and my pretty old machine (Windows Vista gcc 3.4.5). Linking perl58.lib or tcl84.lib works on my machine but neither works on narwhal. Therefore dlltool still remains in Makefiles of plperl, pltcl and plpython (linking pythonxx.lib doesn't work even on a newer machine). I tried another way which links the dll directly instead of msvc import library and got successful results on both old and newer machines except for plpython on an old machine which fails with a compilation error before linkage phase. I attached a patch which eliminates pexports and dlltool from makefile of plperl. The patch links mingw gcc import library if it exists, otherwise links the dll directly. When it works on narwhal, I can provide similar patches for pltcl and plpython. regards, Hiroshi Inoue diff --git a/src/pl/plperl/GNUmakefile b/src/pl/plperl/GNUmakefile index 1e800a3..55fe9b9 100644 --- a/src/pl/plperl/GNUmakefile +++ b/src/pl/plperl/GNUmakefile @@ -36,23 +36,17 @@ DATA = plperl.control plperl--1.0.sql plperl--unpackaged--1.0.sql \ PERLCHUNKS = plc_perlboot.pl plc_trusted.pl -# Perl on win32 ships with import libraries only for Microsoft Visual C++, -# which are not compatible with mingw gcc. Therefore we need to build a -# new import library to link with. +# When Perl on win32 doesn't ship with mingw import libraries, link to the DLL +# directly. ifeq ($(PORTNAME), win32) +perllibdir := $(subst -L,,$(filter -L%, $(perl_embed_ldflags))) perlwithver := $(subst -l,,$(filter -l%, $(perl_embed_ldflags))) -PERLDLL := $(dir $(subst ',,$(PERL)))$(perlwithver).dll -# we no longer want to include the original -l spec in SHLIB_LINK -override perl_embed_ldflags := - -OBJS += lib$(perlwithver).a - -lib$(perlwithver).a: $(perlwithver).def - dlltool --dllname $(perlwithver).dll --def $(perlwithver).def --output-lib lib$(perlwithver).a - -$(perlwithver).def: $(PERLDLL) - pexports $^ $@ +# Link to the perl DLL directly in the bin directory unless the mingw import +# library exists. +ifeq (, $(shell find $(perllibdir) -maxdepth 1 -name lib$(perlwithver).a)) +override perl_embed_ldflags := -L$(dir $(PERL))' -l$(perlwithver) +endif endif # win32 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/20 10:32), Tom Lane wrote: Hiroshi Inoue in...@tpf.co.jp writes: Attached is a patch to remove dllwarp from pgevent/Makefile. Actually, it looks like this patch doesn't work at all: Strangely enough it works here though I see double EXPORTS lines in libpgeventdll.def. http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jacanadt=2014-02-20%2001%3A00%3A53 Did I fat-finger the commit somehow? I made a couple of cosmetic changes, or at least I thought they were cosmetic. Seems EXPORTS line in exports.txt should be removed. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/12 15:31), Inoue, Hiroshi wrote: (2014/02/12 3:03), Tom Lane wrote: Hiroshi Inoue in...@tpf.co.jp writes: (2014/02/09 8:06), Andrew Dunstan wrote: Yeah. Incidentally, we didn't quite get rid of dlltool for Cygwin. We did get rid of dllwrap. But I agree this is worth trying for Mingw. I tried MINGW port with the attached change and successfully built src and contrib and all pararell regression tests were OK. I cleaned this up a bit (the if-nesting in Makefile.shlib was making my head hurt, not to mention that it left a bunch of dead code) and committed it. Thanks. By my count, the only remaining usage of dlltool is in plpython's Makefile. Can we get rid of that? Maybe this is one of the few use cases of dlltool. Because python doesn't ship with its MINGW import library, the Makefile uses dlltool to generate an import library from the python DLL. Also, the only remaining usage of dllwrap is in src/bin/pgevent/Makefile. Do we need that either? Maybe this can be removed. I would make a patch later. Sorry for the late reply. Attached is a patch to remove dllwarp from pgevent/Makefile. regards, Hiroshi Inoue diff --git a/src/bin/pgevent/Makefile b/src/bin/pgevent/Makefile index 1d90276..0a7c16d 100644 --- a/src/bin/pgevent/Makefile +++ b/src/bin/pgevent/Makefile @@ -17,30 +17,21 @@ include $(top_builddir)/src/Makefile.global ifeq ($(PORTNAME), win32) OBJS=pgevent.o pgmsgevent.o -NAME=pgevent.dll +NAME=pgevent +SHLIB_LINK = +SHLIB_EXPORTS = exports.txt -all: $(NAME) +all: all-lib -install: all install-lib +include $(top_srcdir)/src/Makefile.shlib -pgevent.dll: pgevent.def $(OBJS) - $(DLLWRAP) --def $ -o $(NAME) $(OBJS) +install: all install-lib pgmsgevent.o: pgmsgevent.rc win32ver.rc $(WINDRES) $ -o $@ --include-dir=$(top_builddir)/src/include --include-dir=$(top_srcdir)/src/include --include-dir=$(srcdir) --include-dir=. -all-lib: $(NAME) - -install-lib: $(NAME) - $(INSTALL_STLIB) $ '$(DESTDIR)$(libdir)/$' - -uninstall-lib: - rm -f '$(DESTDIR)$(libdir)/$(NAME)' - -clean distclean: - rm -f $(OBJS) $(NAME) win32ver.rc -clean-lib: - rm -f $(NAME) +clean distclean: clean-lib + rm -f $(OBJS) win32ver.rc $(DLL_DEFFILE) endif diff --git a/src/bin/pgevent/exports.txt b/src/bin/pgevent/exports.txt new file mode 100644 index 000..70dcd25 --- /dev/null +++ b/src/bin/pgevent/exports.txt @@ -0,0 +1,5 @@ +; dlltool --output-def pgevent.def pgevent.o pgmsgevent.o +EXPORTS + DllUnregisterServer@0 ; + DllRegisterServer@0 ; + DllInstall ; -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/15 11:42), Tom Lane wrote: Hiroshi Inoue in...@tpf.co.jp writes: (2014/02/15 2:32), Tom Lane wrote: (BTW, narwhal is evidently not trying to build plpython. I wonder why not?) Due to *initializer element is not constant* error which also can be see on my old machine. Hmm, isn't that one of the symptoms of inadequacies in --enable-auto-import? Wonder if the disable change fixed it. The error is a compilation error not a linkage one. gcc on narwhal or my old machine is too old unfortunately. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/15 2:32), Tom Lane wrote: I wrote: Hiroshi Inoue in...@tpf.co.jp writes: One thing I'm wondering about is that plperl is linking perlxx.lib not libperlxx.a. I made a patch following plpython and it also works here. Is it worth trying? I hadn't noticed that part of plpython's Makefile before. Man, that's an ugly technique :-(. Still, there's little about this platform that isn't ugly. Let's try it and see what happens. And what happens is this: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=narwhaldt=2014-02-14%2017%3A00%3A02 namely, it gets through plperl now and then chokes with the same symptoms on pltcl. So I guess we need the same hack in pltcl. The fun never stops ... Pltcl still fails. tclxx.dll lives in bin directory not in lib directory. The attached patch would fix the problem. regards, Hiroshi Inoue diff --git a/src/pl/tcl/Makefile b/src/pl/tcl/Makefile index e0fb13e..2ab2a27 100644 --- a/src/pl/tcl/Makefile +++ b/src/pl/tcl/Makefile @@ -53,7 +53,7 @@ PSQLDIR = $(bindir) ifeq ($(PORTNAME), win32) tclwithver = $(subst -l,,$(filter -l%, $(TCL_LIB_SPEC))) -TCLDLL = $(subst -L,,$(filter -L%, $(TCL_LIB_SPEC)))/$(tclwithver).dll +TCLDLL = $(dir $(TCLSH))/$(tclwithver).dll OBJS += lib$(tclwithver).a -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/12 8:30), Tom Lane wrote: I wrote: Hiroshi Inoue in...@tpf.co.jp writes: I tried MINGW port with the attached change and successfully built src and contrib and all pararell regression tests were OK. I cleaned this up a bit (the if-nesting in Makefile.shlib was making my head hurt, not to mention that it left a bunch of dead code) and committed it. Hm ... according to buildfarm member narwhal, this doesn't work so well for plperl: gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -g -Wno-comment -shared -o plperl.dll plperl.o SPI.o Util.o -L../../../src/port -L../../../src/common -Wl,--allow-multiple-definition -L/mingw/lib -Wl,--as-needed -LC:/Perl/lib/CORE -lperl58 -L../../../src/backend -lpostgres -lpgcommon -lpgport -lintl -lxslt -lxml2 -lssleay32 -leay32 -lz -lm -lws2_32 -lshfolder -Wl,--export-all-symbols -Wl,--out-implib=libplperl.a Cannot export .idata$4: symbol not found Cannot export .idata$5: symbol not found Cannot export .idata$6: symbol not found Cannot export .text: symbol not found Cannot export perl58_NULL_THUNK_DATA: symbol not found Creating library file: libplperl.a collect2: ld returned 1 exit status make[3]: *** [plperl.dll] Error 1 Not very clear what's going on there; could this be a problem in narwhal's admittedly-ancient toolchain? The error doesn't occur here (un)fortunately. One thing I'm wondering about is that plperl is linking perlxx.lib not libperlxx.a. I made a patch following plpython and it also works here. Is it worth trying? regards, Hiroshi Inoue diff --git a/src/pl/plperl/GNUmakefile b/src/pl/plperl/GNUmakefile index e0e31ec..065c5d3 100644 --- a/src/pl/plperl/GNUmakefile +++ b/src/pl/plperl/GNUmakefile @@ -16,6 +16,8 @@ endif ifeq ($(shared_libperl),yes) ifeq ($(PORTNAME), win32) +generate_perl_implib = yes + override CPPFLAGS += -DPLPERL_HAVE_UID_GID # Perl on win32 contains /* within comment all over the header file, # so disable this warning. @@ -36,7 +38,21 @@ DATA = plperl.control plperl--1.0.sql plperl--unpackaged--1.0.sql \ PERLCHUNKS = plc_perlboot.pl plc_trusted.pl +# Perl on win32 ships with import libraries only for Microsoft Visual C++, +# which may not be handled with mingw gcc. Therefore we choose to build a +# new import library to link with. +ifeq ($(generate_perl_implib), yes) + perlwithver = $(subst -l,,$(filter -l%, $(perl_embed_ldflags))) + OBJS += lib$(perlwithver).a +lib$(perlwithver).a: $(perlwithver).def + dlltool --dllname $(perlwithver).dll --def $(perlwithver).def --output-lib lib$(perlwithver).a +$(perlwithver).def: + pexports $(dir $(subst ',,$(PERL)))$(perlwithver).dll $(perlwithver).def + +SHLIB_LINK = +else SHLIB_LINK = $(perl_embed_ldflags) +endif REGRESS_OPTS = --dbname=$(PL_TESTDB) --load-extension=plperl --load-extension=plperlu REGRESS = plperl plperl_lc plperl_trigger plperl_shared plperl_elog plperl_util plperl_init plperlu plperl_array @@ -105,6 +121,9 @@ submake: clean distclean maintainer-clean: clean-lib rm -f SPI.c Util.c $(OBJS) perlchunks.h plperl_opmask.h rm -rf $(pg_regress_clean_files) +ifeq (yes, $(generate_perl_implib)) + rm -rf $(perlwithver).def +endif else # can't build -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/15 2:32), Tom Lane wrote: I wrote: Hiroshi Inoue in...@tpf.co.jp writes: One thing I'm wondering about is that plperl is linking perlxx.lib not libperlxx.a. I made a patch following plpython and it also works here. Is it worth trying? I hadn't noticed that part of plpython's Makefile before. Man, that's an ugly technique :-(. Still, there's little about this platform that isn't ugly. Let's try it and see what happens. And what happens is this: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=narwhaldt=2014-02-14%2017%3A00%3A02 namely, it gets through plperl now and then chokes with the same symptoms on pltcl. So I guess we need the same hack in pltcl. The fun never stops ... There seems some risk to link msvc import library. Linking perlxx.lib or tcl.lib worls here but linking pythonxx.lib doesn't work even in newer machine. (BTW, narwhal is evidently not trying to build plpython. I wonder why not?) Due to *initializer element is not constant* error which also can be see on my old machine. http://www.postgresql.org/message-id/CA+OCxozr=0wkQDF7kfd2n-bJQOwdSUs0Myohg29pA_U5=2p...@mail.gmail.com regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/13 9:51), Hiroshi Inoue wrote: (2014/02/12 12:28), Inoue, Hiroshi wrote: (2014/02/12 8:30), Tom Lane wrote: I wrote: Hiroshi Inoue in...@tpf.co.jp writes: I tried MINGW port with the attached change and successfully built src and contrib and all pararell regression tests were OK. I cleaned this up a bit (the if-nesting in Makefile.shlib was making my head hurt, not to mention that it left a bunch of dead code) and committed it. Hm ... according to buildfarm member narwhal, this doesn't work so well for plperl: gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -g -Wno-comment -shared -o plperl.dll plperl.o SPI.o Util.o -L../../../src/port -L../../../src/common -Wl,--allow-multiple-definition -L/mingw/lib -Wl,--as-needed -LC:/Perl/lib/CORE -lperl58 -L../../../src/backend -lpostgres -lpgcommon -lpgport -lintl -lxslt -lxml2 -lssleay32 -leay32 -lz -lm -lws2_32 -lshfolder -Wl,--export-all-symbols -Wl,--out-implib=libplperl.a Cannot export .idata$4: symbol not found Cannot export .idata$5: symbol not found Cannot export .idata$6: symbol not found Cannot export .text: symbol not found Cannot export perl58_NULL_THUNK_DATA: symbol not found Creating library file: libplperl.a collect2: ld returned 1 exit status make[3]: *** [plperl.dll] Error 1 Oops I forgot to inclule plperl, tcl or python, sorry. I would retry build operations with them. Unfortunately it would take pretty long time because build operations are pretty (or veeery in an old machine) slow. Not very clear what's going on there; could this be a problem in narwhal's admittedly-ancient toolchain? As for build, plperl and pltcl are OK on both Windows7+gcc4.6.1 machine and Windows Vista+gcc3.4.5 machine. plpython is OK on gcc4.6.1 machine but causes a *initializer element is not constant* error on gcc3.4.5 machine. I've not run regression test yet. Rebuild with --disable-auto-import causes errors in contrib on both machines. Errors occur in pg_buffercache, pg_stat_statements, postgres_fdw and test_shm_mq. --enable-auto-import cures all of them. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/12 12:28), Inoue, Hiroshi wrote: (2014/02/12 8:30), Tom Lane wrote: I wrote: Hiroshi Inoue in...@tpf.co.jp writes: I tried MINGW port with the attached change and successfully built src and contrib and all pararell regression tests were OK. I cleaned this up a bit (the if-nesting in Makefile.shlib was making my head hurt, not to mention that it left a bunch of dead code) and committed it. Hm ... according to buildfarm member narwhal, this doesn't work so well for plperl: gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -g -Wno-comment -shared -o plperl.dll plperl.o SPI.o Util.o -L../../../src/port -L../../../src/common -Wl,--allow-multiple-definition -L/mingw/lib -Wl,--as-needed -LC:/Perl/lib/CORE -lperl58 -L../../../src/backend -lpostgres -lpgcommon -lpgport -lintl -lxslt -lxml2 -lssleay32 -leay32 -lz -lm -lws2_32 -lshfolder -Wl,--export-all-symbols -Wl,--out-implib=libplperl.a Cannot export .idata$4: symbol not found Cannot export .idata$5: symbol not found Cannot export .idata$6: symbol not found Cannot export .text: symbol not found Cannot export perl58_NULL_THUNK_DATA: symbol not found Creating library file: libplperl.a collect2: ld returned 1 exit status make[3]: *** [plperl.dll] Error 1 Oops I forgot to inclule plperl, tcl or python, sorry. I would retry build operations with them. Unfortunately it would take pretty long time because build operations are pretty (or veeery in an old machine) slow. Not very clear what's going on there; could this be a problem in narwhal's admittedly-ancient toolchain? As for build, plperl and pltcl are OK on both Windows7+gcc4.6.1 machine and Windows Vista+gcc3.4.5 machine. plpython is OK on gcc4.6.1 machine but causes a *initializer element is not constant* error on gcc3.4.5 machine. I've not run regression test yet. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/13 8:21), Tom Lane wrote: Craig Ringer cr...@2ndquadrant.com writes: On 02/12/2014 02:31 PM, Inoue, Hiroshi wrote: Maybe this is one of the few use cases of dlltool. Because python doesn't ship with its MINGW import library, the Makefile uses dlltool to generate an import library from the python DLL. Wow. Has anyone been in touch with the Python package maintainers to see if they can fix that and bundle the .lib ? Indeed somebody should pester them about that, but it's not clear how much it'd help us even if they fixed it tomorrow. We're going to have to be able to build with existing Python distributions for a long time to come. Though the comment in src/pl/plpython/Makefile refers to it, I see libpython27.a in Python27. I can't find it in Python25. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/09 8:06), Andrew Dunstan wrote: On 02/08/2014 05:34 PM, Tom Lane wrote: Hiroshi Inoue in...@tpf.co.jp writes: Though I'm not a MINGW expert at all, I know dllwrap is a deprecated tool and dlltool is almost a deprecated tool. Cygwin port is removing the use of dllwrap and dlltool now. Isn't it better for MINGW port to follow it? Only way to make that happen is to prepare and test a patch ... Yeah. Incidentally, we didn't quite get rid of dlltool for Cygwin. We did get rid of dllwrap. But I agree this is worth trying for Mingw. I tried MINGW port with the attached change and successfully built src and contrib and all pararell regression tests were OK. regards, Hiroshi Inoue diff --git a/src/Makefile.shlib b/src/Makefile.shlib index a95e4d6..367f585 100644 --- a/src/Makefile.shlib +++ b/src/Makefile.shlib @@ -371,7 +371,20 @@ $(stlib): $(OBJS) | $(SHLIB_PREREQS) $(RANLIB) $@ +else #cygwin +ifeq ($(PORTNAME), win32) +ifeq (,$(SHLIB_EXPORTS)) +$(shlib) $(stlib): $(OBJS) | $(SHLIB_PREREQS) + $(CC) $(CFLAGS) -shared -o $@ $(OBJS) $(LDFLAGS) $(LDFLAGS_SL) $(SHLIB_LINK) $(LIBS) -Wl,--out-implib=$(stlib) -Wl,--export-all-symbols + else +DLL_DEFFILE = lib$(NAME)dll.def +$(shlib) $(stlib): $(OBJS) $(DLL_DEFFILE) | $(SHLIB_PREREQS) + $(CC) $(CFLAGS) -shared -o $@ $(OBJS) $(LDFLAGS) $(LDFLAGS_SL) $(SHLIB_LINK) $(LIBS) $(DLL_DEFFILE) -Wl,--out-implib=$(stlib) +endif + + +else #win32 ifeq (,$(SHLIB_EXPORTS)) DLL_DEFFILE = lib$(NAME)dll.def exports_file = $(DLL_DEFFILE) @@ -388,6 +401,7 @@ $(shlib): $(OBJS) $(DLL_DEFFILE) | $(SHLIB_PREREQS) $(stlib): $(shlib) $(DLL_DEFFILE) | $(SHLIB_PREREQS) $(DLLTOOL) --dllname $(shlib) $(DLLTOOL_LIBFLAGS) --def $(DLL_DEFFILE) --output-lib $@ +endif # PORTNAME == win32 endif # PORTNAME == cgywin endif # PORTNAME == cygwin || PORTNAME == win32 diff --git a/src/backend/Makefile b/src/backend/Makefile index 356890d..41a4e41 100644 --- a/src/backend/Makefile +++ b/src/backend/Makefile @@ -80,18 +80,8 @@ endif # cygwin ifeq ($(PORTNAME), win32) LIBS += -lsecur32 -postgres: $(OBJS) postgres.def libpostgres.a $(WIN32RES) - $(DLLTOOL) --dllname $@$(X) --output-exp $@.exp --def postgres.def - $(CC) $(CFLAGS) $(LDFLAGS) $(LDFLAGS_EX) -o $@$(X) -Wl,--base-file,$@.base $@.exp $(call expand_subsys,$(OBJS)) $(WIN32RES) $(LIBS) - $(DLLTOOL) --dllname $@$(X) --base-file $@.base --output-exp $@.exp --def postgres.def - $(CC) $(CFLAGS) $(LDFLAGS) $(LDFLAGS_EX) -Wl,--stack=$(WIN32_STACK_RLIMIT) -o $@$(X) $@.exp $(call expand_subsys,$(OBJS)) $(WIN32RES) $(LIBS) - rm -f $@.exp $@.base - -postgres.def: $(OBJS) - $(DLLTOOL) --export-all --output-def $@ $(call expand_subsys,$^) - -libpostgres.a: postgres.def - $(DLLTOOL) --dllname postgres.exe --def postgres.def --output-lib $@ +postgres: $(OBJS) $(WIN32RES) + $(CC) $(CFLAGS) $(LDFLAGS) $(LDFLAGS_EX) -Wl,--stack=$(WIN32_STACK_RLIMIT) -o $@$(X) -Wl,--out-implib=libpostgres.a -Wl,--export-all-symbols $(call expand_subsys,$(OBJS)) $(WIN32RES) $(LIBS) endif # win32 diff --git a/src/makefiles/Makefile.win32 b/src/makefiles/Makefile.win32 index 1aae9e9..b18621b 100644 --- a/src/makefiles/Makefile.win32 +++ b/src/makefiles/Makefile.win32 @@ -72,6 +72,4 @@ win32ver.o: win32ver.rc # Rule for building a shared library from a single .o file %.dll: %.o - $(DLLTOOL) --export-all --output-def $*.def $ - $(DLLWRAP) -o $@ --def $*.def $ $(LDFLAGS) $(LDFLAGS_SL) $(BE_DLLLIBS) - rm -f $*.def + $(CC) $(CFLAGS) -shared -o $@ $ -Wl,--export-all-symbols $(LDFLAGS) $(LDFLAGS_SL) $(BE_DLLLIBS) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/05 14:52), Tom Lane wrote: Craig Ringer cr...@2ndquadrant.com writes: On 02/05/2014 06:29 AM, Tom Lane wrote: I had been okay with the manual PGDLLIMPORT-sprinkling approach (not happy with it, of course, but prepared to tolerate it) as long as I believed the buildfarm would reliably tell us of the need for it. That assumption has now been conclusively disproven, though. I'm kind of horrified that the dynamic linker doesn't throw its toys when it sees this. Indeed :-(. The truly strange part of this is that it seems that the one Windows buildfarm member that's telling the truth (or most nearly so, anyway) is narwhal, which appears to have the oldest and cruftiest toolchain of the lot. I'd really like to come out the other end of this investigation with a clear understanding of why the newer toolchains are failing to report a link problem, and yet not building working executables. Is it a linkage error? Could you please show me the error message concretely? regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
OK I can see the error message at PostgreSQL Build Farm Log. I see a similar problem in a simple test case. gcc -DCALLPG=\import1\ -c export.c -o export1.o dlltool --export-all --output-def export1.def export1.o dlltool --dllname export1.exe --def export1.def --output-lib export1.a dlltool --dllname export1.exe --output-exp export1.exp --def export1.def gcc -o export1.exe -Wl,--base-file,export1.base export1.exp export1.o dlltool --dllname export1.exe --base-file export1.base --output-exp export1.exp --def export1.def gcc -Wl,--stack=65536 -o export1.exe export1.exp export1.o rm -f export1.exp export1.base gcc -DCALLPG=\import1\ -c import.c -o import1.o dlltool --export-all --output-def import1.def import1.o dllwrap -o import1.dll --dllname import1.dll import1.o export1.a --def import1.def Info: resolving _intnum by linking to __imp__intnum (auto-import) fu01.o:(.idata$2+0xc): undefined reference to `export1_a_iname' nmth00.o:(.idata$4+0x0): undefined reference to `_nm__intnum' collect2: ld returned 1 exit status Adding __declspec(dllimport) fixes the problem. Using gcc only build also fixes the problem. gcc -DCALLPG=\import2\ -c export.c -o export2.o gcc -Wl,--export-all-symbols -Wl,--out-implib=export2.a -o export2 export2.o Creating library file: export2.a gcc -DCALLPG=\import2\ -c import.c -o import2.o gcc -shared -o import2.dll import2.o export2.a -Wl,--out-implib=import2.dll.a Info: resolving _intnum by linking to __imp__intnum (auto-import) Creating library file: import2.dll.a Though I'm not a MINGW expert at all, I know dllwrap is a deprecated tool and dlltool is almost a deprecated tool. Cygwin port is removing the use of dllwrap and dlltool now. Isn't it better for MINGW port to follow it? Changing the machine from the one using gcc 3.4.5 to another one using 4.6.1 also fixes the problem. regards, Hiroshi Inoue (2014/02/07 12:25), Hiroshi Inoue wrote: (2014/02/05 14:52), Tom Lane wrote: Craig Ringer cr...@2ndquadrant.com writes: On 02/05/2014 06:29 AM, Tom Lane wrote: I had been okay with the manual PGDLLIMPORT-sprinkling approach (not happy with it, of course, but prepared to tolerate it) as long as I believed the buildfarm would reliably tell us of the need for it. That assumption has now been conclusively disproven, though. I'm kind of horrified that the dynamic linker doesn't throw its toys when it sees this. Indeed :-(. The truly strange part of this is that it seems that the one Windows buildfarm member that's telling the truth (or most nearly so, anyway) is narwhal, which appears to have the oldest and cruftiest toolchain of the lot. I'd really like to come out the other end of this investigation with a clear understanding of why the newer toolchains are failing to report a link problem, and yet not building working executables. Is it a linkage error? Could you please show me the error message concretely? regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [ODBC] getting rid of SnapshotNow
(2013/07/19 22:03), Andres Freund wrote: On 2013-07-19 08:57:01 +0900, Inoue, Hiroshi wrote: I had the idea they were used for a client-side implementation of WHERE CURRENT OF. Perhaps that's dead code and could be removed entirely? It's been reported that ODBC still uses them. Though PostgreSQL's TID is similar to Orale's ROWID, it is transient and changed after update operations unfortunately. I implemented the currtid_xx functions to supplement the difference. For example currtid(relname, original tid) (hopefully) returns the current tid of the original row when it is updated. That is only guaranteed to work though when you're in a transaction old enough to prevent removal of the old or intermediate row versions. E.g. Yes it's what I meant by (hopefully). At the time when I implemented currtid(), I was able to use TIDs in combination with OIDs. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] getting rid of SnapshotNow
(2013/07/18 21:46), Robert Haas wrote: There seems to be a consensus that we should try to get rid of SnapshotNow entirely now that we have MVCC catalog scans, so I'm attaching two patches that together come close to achieving that goal: ... With that done, the only remaining uses of SnapshotNow in our code base will be in currtid_byreloid() and currtid_byrelname(). So far no one on this list has been able to understand clearly what the purpose of those functions is, so I'm copying this email to pgsql-odbc in case someone there can provide more insight. If I were a betting man, I'd bet that they are used in contexts where the difference between SnapshotNow and SnapshotSelf wouldn't matter there, either. Using SnapshotSelf instead of SnapshotNow for currtid_ () wouldn't matter. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [ODBC] [HACKERS] getting rid of SnapshotNow
(2013/07/20 1:11), Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-07-20 00:49:11 +0900, Hiroshi Inoue wrote: Using SnapshotSelf instead of SnapshotNow for currtid_ () wouldn't matter. I think it actually might. You could get into dicey situations if you use currtid_ in a query performing updates or inserts because it would see the to-be-inserted tuple... I'm pretty sure Hiroshi-san was only opining about whether it would matter for ODBC's usage. IIUC, ODBC is using this function to re-fetch rows that it inserted, updated, or at least selected-for-update in a previous command of the current transaction, so actually any snapshot would do fine. In any case, since I moved the goalposts by suggesting that SnapshotSelf is just as dangerous as SnapshotNow, what we need to know is whether it'd be all right to change this code to use a fresh MVCC snapshot; and if not, why not. It's pretty hard to see a reason why client-side code would want to make use of the results of a non-MVCC snapshot. OK I agree to replace SnapshotNow for currtid_xx() by a MVCC-snapshot. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: setlocale() on Windows doesn't work correctly if the locale name
(2011/04/20 15:30), Heikki Linnakangas wrote:
On 20.04.2011 06:48, Hiroshi Inoue wrote:
I can find no concrete reference to problems about locale
names containing dots. Is the following an example?
Yes.
In my environment (Windows Vista using VC8)
setlocale(LC_, Chinese (Traditional)_MCO.950);
works and
setlocale(LC_, NULL);
returns
Chinese (Traditional)_Macao S.A.R..950
but
setlocale(LC_, Chinese (Traditional)_Macao S.A.R..950);
fails.
I see another issue for the behavior.
For example, the following code in src/backend/utis/adt/pg_locale.c
won't work as expected in case the current locale is Hong Kong, Macao or
UAE because the last setlocale() in the code would fail. I can
find such save restore operations of locales in several places.
bool
check_locale(int category, const char *value)
{
char *save;
boolret;
save = setlocale(category, NULL);
if (!save)
return false; /* won't happen, we hope */
/* save may be pointing at a modifiable scratch variable, see above */
save = pstrdup(save);
/* set the locale with setlocale, to see if it accepts it. */
ret = (setlocale(category, value) != NULL);
setlocale(category, save); /* assume this won't fail */
pfree(save);
return ret;
}
regards,
Hiroshi Inoue
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: setlocale() on Windows doesn't work correctly if the locale name
(2011/04/20 22:08), Tom Lane wrote: Hiroshi Inouein...@tpf.co.jp writes: In my environment (Windows Vista using VC8) setlocale(LC_, Chinese (Traditional)_MCO.950); works and setlocale(LC_, NULL); returns Chinese (Traditional)_Macao S.A.R..950 but setlocale(LC_, Chinese (Traditional)_Macao S.A.R..950); fails. Interesting. This example suggests that maybe Windows' setlocale can only cope with dot as introducing a codepage number. ACP or OCP as well as codepage number seem to be allowed. Are there any cases where a dot works as part of the basic locale name? Unfortunately I don't know any explanation how dots are allowed. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: setlocale() on Windows doesn't work correctly if the locale name
(2011/04/20 15:30), Heikki Linnakangas wrote: On 20.04.2011 06:48, Hiroshi Inoue wrote: I can find no concrete reference to problems about locale names containing dots. Is the following an example? Yes. In my environment (Windows Vista using VC8) setlocale(LC_, Chinese (Traditional)_MCO.950); works and setlocale(LC_, NULL); returns Chinese (Traditional)_Macao S.A.R..950 Interesting. According to Microsoft's documentation, the codes are three-letter country codes specified by ISO-3166 (http://msdn.microsoft.com/en-us/library/cdax410z%28v=VS.100%29.aspx). However, according to Wikipedia, MCO stands for Monaco, not Macau (https://secure.wikimedia.org/wikipedia/en/wiki/ISO_3166-1_alpha-3). Hmm Windows locale system seems to have an inconsistency and the same country code (MCO) corresponds to different countries. ZHM_MCO corresponds to Chinese (Traditional)_Macao S.A.R..950 whereas FRM_MCO corresponds to French_Principality of Monaco. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [COMMITTERS] pgsql: setlocale() on Windows doesn't work correctly if the locale name
(2011/04/16 2:56), Heikki Linnakangas wrote: setlocale() on Windows doesn't work correctly if the locale name contains apostrophes or dots. As for apostrophes, isn't the cause that initdb loses the single quote of locale? ([BUGS] BUG #5818: initdb lose the single quote of locale) As the bug reporter mentions, initdb loses the single quote in reality. Concretely speaking, scanstr() called from bootscanner.l loses it. I'm not sure if it's suitable for the bootstrap code to call scanstr(). regards, Hiroshi Inoue There isn't much hope of Microsoft fixing it any time soon, it's been like that for ages, so we better work around it. So, map a few common Windows locale names known to cause problems to aliases that work. Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/d5a7bf8c11c8b66c822bbb1a6c90e1a14425bd6e Modified Files -- src/bin/initdb/initdb.c | 89 +++ 1 files changed, 82 insertions(+), 7 deletions(-) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: setlocale() on Windows doesn't work correctly if the locale name
(2011/04/20 9:22), Tom Lane wrote: Hiroshi Inouein...@tpf.co.jp writes: (2011/04/16 2:56), Heikki Linnakangas wrote: setlocale() on Windows doesn't work correctly if the locale name contains apostrophes or dots. As for apostrophes, isn't the cause that initdb loses the single quote of locale? ([BUGS] BUG #5818: initdb lose the single quote of locale) As the bug reporter mentions, initdb loses the single quote in reality. Concretely speaking, scanstr() called from bootscanner.l loses it. I'm not sure if it's suitable for the bootstrap code to call scanstr(). Huh? Bootstrap mode just deals with the data found in src/include/catalog/*.h. The locale names found by initdb.c are stuck in there afterwards, using regular SQL commands. bootstrap_template1() in initdb runs the BKI script in bootstrap mode to create template1. Some symbols (LC_COLLATE, LC_CTYPE in pg_database etc) in the BKI script are substituted by actual values using replace_token(). Isn't it correct? ISTM replace_token() takes care of nothing about single quotes in its input values but the comment in scanstr() says /* * Note: if scanner is working right, unescaped quotes can only * appear in pairs, so there should be another character. */ regards, Hiroshi Inoue I don't know where the problem really comes from, but I doubt the connection you're trying to make above. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: setlocale() on Windows doesn't work correctly if the locale name
(2011/04/20 12:25), Andrew Dunstan wrote: On 04/19/2011 09:42 PM, Hiroshi Inoue wrote: bootstrap_template1() in initdb runs the BKI script in bootstrap mode to create template1. Some symbols (LC_COLLATE, LC_CTYPE in pg_database etc) in the BKI script are substituted by actual values using replace_token(). Isn't it correct? ISTM replace_token() takes care of nothing about single quotes in its input values but the comment in scanstr() says /* * Note: if scanner is working right, unescaped quotes can only * appear in pairs, so there should be another character. */ That's perfectly true, but only one of the replaced locale names contains a single quote mark. So clearly there's more going on here than just the bug you're referring to. Heikki's commit message specifically refers to dots in locale names, which shouldn't cause a problem of that type, I believe. Yes it's completely another issue as for dots. I can find no concrete reference to problems about locale names containing dots. Is the following an example? In my environment (Windows Vista using VC8) setlocale(LC_, Chinese (Traditional)_MCO.950); works and setlocale(LC_, NULL); returns Chinese (Traditional)_Macao S.A.R..950 but setlocale(LC_, Chinese (Traditional)_Macao S.A.R..950); fails. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] setlocale and gettext in Postgres
Hi all, I see now the following lines in libintl.h of version 0.18.1.1 which didn't exist in 0.17 version. /* Support for the locale chosen by the user. */ #if (defined __APPLE__ defined __MACH__) || defined _WIN32 || defined __WIN32__ || defined __CYGWIN__ #undef setlocale #define setlocale libintl_setlocale extern char *setlocale (int, const char *); . . The macro may cause a trouble especially on Windows. Though libintl_setlocale() calls setlocale() internally, what the Mingw version of libintl library calls is the one in msvcrt.dll not in the libraries which main programs link. Unfortunately locale-aware functions like printf() calls in main programs would see C locales not the locales set by libintl_setlocale(). Attached is a patch to disable the macro on Windows. regards, Hiroshi Inoue *** port.h.orig 2011-01-15 05:29:13.43600 +0900 --- port.h 2011-01-23 21:56:19.55400 +0900 *** *** 176,181 --- 176,188 #ifdef printf #undef printf #endif + /* + * Versions of libintl = 0.18? try to replace setlocale() with a macro + * to their own versions. Disable the macro, if it exists. + */ + #if defined(setlocale) defined(WIN32) + #undef setlocale + #endif extern intpg_vsnprintf(char *str, size_t count, const char *fmt, va_list args); extern int -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
(2010/11/01 19:24), Itagaki Takahiro wrote: I compiled the source with mingw gcc 4.5.0, that has been released recently. The compile was succeeded and worked well at least for simple queries, but there were many warnings during the compile. 1. warning: 'symbol' redeclared without dllimport attribute: previous dllimport ignored 2. warning: unknown conversion type character 'm' in format 3. warning: unknown conversion type character 'l' in format 1 is easy to fix with the attached patch. Is it safe to put back the patch you applied in http://archives.postgresql.org/pgsql-committers/2010-05/msg00338.php in the case __GNUC__ =4? regards, Hiroshi Inoue I wonder why mingw gcc 4.5 can build codes without the fix... *** a/src/include/port/win32.h --- b/src/include/port/win32.h *** *** 58,64 #define PGDLLIMPORT __declspec (dllimport) #endWindows 7 64bit diff --git a/src/include/port/win32.h b/src/include/port/win32.h indexif ! #ifdef _MSC_VER #define PGDLLEXPORT __declspec (dllexport) #else #define PGDLLEXPORT __declspec (dllimport) --- 58,64 #define PGDLLIMPORT __declspec (dllimport) #endif ! #if defined(_MSC_VER) || __GNUC__= 4 #define PGDLLEXPORT __declspec (dllexport) #else #define PGDLLEXPORT __declspec (dllimport) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
(2010/11/02 8:31), Itagaki Takahiro wrote: On Tue, Nov 2, 2010 at 6:02 AM, Hiroshi Inouein...@tpf.co.jp wrote: 1. warning: 'symbol' redeclared without dllimport attribute: previous dllimport ignored Is it safe to put back the patch you applied in http://archives.postgresql.org/pgsql-committers/2010-05/msg00338.php in the case __GNUC__=4? Hmmm, I didn't test compiling with gcc version between 3.5 and 4.4. Does anyone test them? If it works only on gcc 4.5, I'll also add _GNUC_MINOR__= 5 for the check. The problem which was fixed by your old patch is at runtime not at compilation time. Is it fixed with gcc 4.5? regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [COMMITTERS] pgsql: PGDLLEXPORT is __declspec (dllexport) only on MSVC, but is
井上です。 ご苦労様です。 このスレッド気になっていたのですが、ようやく少し 余裕ができたのでテストなどしてみました。 Takahiro Itagaki wrote: Log Message: --- PGDLLEXPORT is __declspec (dllexport) only on MSVC, but is __declspec (dllimport) on other compilers 私が知る限りdlimportがexportの引きがねになることは ないのでこの部分にはかなり違和感を感じていました。 実際__declspec(..)をすっぱり取り除いてmingwでbuild してみましたが少なくともinitdbでエラーになることは ありません。この場合の__declspec(dllimport)指定は 意味があるようには思えませんし運よく害がないだけだ という気がします。 because cygwin and mingw don't like dllexport. 一方__declspec(dllexport)指定は運悪く害があるのです が原因は単純ではなさそうです。エラーが発生している 場所はLoadLibrary()でありGetProcAddress()ではない ので関数がexportされないという原因ではなさそうです。 ネットを見て回ったところではdllwrap(及び？dlltool) を使っているのがまずいのかなという感触はありますが 明確な所はわかりません。又、それを避けるにはどうし たらよいのかがわかるほどmingwに詳しくはありません。 以上 参考まで Modified Files: -- pgsql/src/include/port: cygwin.h (r1.8 - r1.9) (http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/port/cygwin.h?r1=1.8r2=1.9) win32.h (r1.94 - r1.95) (http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/port/win32.h?r1=1.94r2=1.95) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] trouble with to_char('L')
Bruce Momjian wrote: Takahiro Itagaki wrote: Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp wrote: Revised patch attached. Please test it. I applied this version of the patch. Please check wheter the bug is fixed and any buildfarm failures. Great. I have merged in my C comments into the code with the attached patch so we remember why the code is setup as it is. One thing I am confused about is that, for Win32, our numeric/monetary handling sets lc_ctype to match numeric/monetary, while our time code in the same file uses that method _and_ uses wcsftime() to return the value in wide characters. So, why do we do both for time? Is there any value to that? Unfortunately wcsftime() is a halfway conveniece function which uses ANSI version of functionalities internally. AFAIC the only way to remove the dependency to LC_CTYPE is to call GeLocaleInfoW() directly. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] trouble with to_char('L')
Takahiro Itagaki wrote: Magnus Hagander mag...@hagander.net wrote: 1. setlocale(LC_CTYPE, lc_monetary) 2. setlocale(LC_MONETARY, lc_monetary) 3. lc = localeconv() 4. pg_do_encoding_conversion(lc-xxx, FROM pg_get_encoding_from_locale(lc_monetary), TO GetDatabaseEncoding()) 5. Revert LC_CTYPE and LC_MONETARY. A patch attached for the above straightforwardly. Does this work? I have 2 questions about this patch. 1. How does it work when LC_MONETARY and LC_NUMERIC are different? 2. Calling db_encoding_strdup() for lconv-grouping is appropriate? regards, Hiroshi Inoue Note that #ifdef WIN32 parts in the patch are harmless on other platforms even if they are enabled. Let's work off what we have now to start with at least. Bruce, can you comment on that thing about the extra parameter? And UTF8? Regards, --- Takahiro Itagaki NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] trouble with to_char('L')
Bruce Momjian wrote: Hiroshi Inoue wrote: Bruce Momjian wrote: Bruce Momjian wrote: Hiroshi Inoue wrote: Bruce Momjian wrote: Hiroshi Inoue wrote: Bruce Momjian wrote: Where are we on this issue? Oops I forgot it completely. I have a little improved version and would post it tonight. Ah, very good. Thanks. Attached is an improved version. I spent many hours on this patch and am attaching an updated version. I have restructured the code and added many comments, but this is the main one: * Ideally, the server encoding and locale settings would * always match. Unfortunately, WIN32 does not support UTF-8 * values for setlocale(), even though PostgreSQL runs fine with * a UTF-8 encoding on Windows: * * http://msdn.microsoft.com/en-us/library/x99tb11d.aspx * * Therefore, we must set LC_CTYPE to match LC_NUMERIC and * LC_MONETARY, call localeconv(), and use mbstowcs() to * convert the locale-aware string, e.g. Euro symbol, which * is not in UTF-8 to the server encoding. I need someone with WIN32 experience to review and test this patch. I don't understand why cache_locale_time() works on Windows. It sets the LC_CTYPE but does not do any encoding coversion. Doesn't strftime_win32 do the conversion? Oh, I now see strftime is redefined as a macro in that C files. Thanks. Do month and day-of-week names not work either, or do they work and the encoding conversion for numeric/money, e.g. Euro, it not necessary? db_strdup does the conversion. Should we pull the encoding conversion into a separate function and have strftime_win32() and db_strdup() both call it? We may be able to pull the conversion WideChars = UTF8 = a PG encoding into an function. BTW both PGLC_localeconv() and cache_locale_time() save the current LC_CTYPE first and restore them just before returning the functions. I'm suspicious if it's OK when errors occur in middle of the functions. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] trouble with to_char('L')
Bruce Momjian wrote: Bruce Momjian wrote: Hiroshi Inoue wrote: Bruce Momjian wrote: Hiroshi Inoue wrote: Bruce Momjian wrote: Where are we on this issue? Oops I forgot it completely. I have a little improved version and would post it tonight. Ah, very good. Thanks. Attached is an improved version. I spent many hours on this patch and am attaching an updated version. I have restructured the code and added many comments, but this is the main one: * Ideally, the server encoding and locale settings would * always match. Unfortunately, WIN32 does not support UTF-8 * values for setlocale(), even though PostgreSQL runs fine with * a UTF-8 encoding on Windows: * * http://msdn.microsoft.com/en-us/library/x99tb11d.aspx * * Therefore, we must set LC_CTYPE to match LC_NUMERIC and * LC_MONETARY, call localeconv(), and use mbstowcs() to * convert the locale-aware string, e.g. Euro symbol, which * is not in UTF-8 to the server encoding. I need someone with WIN32 experience to review and test this patch. I don't understand why cache_locale_time() works on Windows. It sets the LC_CTYPE but does not do any encoding coversion. Doesn't strftime_win32 do the conversion? Do month and day-of-week names not work either, or do they work and the encoding conversion for numeric/money, e.g. Euro, it not necessary? db_strdup does the conversion. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq naming on Win64
Dave Page wrote: On Thu, Jan 7, 2010 at 3:58 AM, Hiroshi Inoue in...@tpf.co.jp wrote: Maybe I'm missing the point and have a question. For example, do 32bit psql and the 64bit one have the same name? If so, where will they be installed? I'm only talking about libpq. I see no reason to have 32 64 bit versions of other utilities installed in parallel. Well what about the libraries (e.g openssl) libpq links? regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq naming on Win64
Dave Page wrote: On Tue, Jan 5, 2010 at 3:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: After chatting with Magnus, we feel that a good solution would be to rename libpq on Win64 to libpq64.dll to distinguish it from the 32 bit equivalent. Isn't that going to break applications? Where by break I mean have to explicitly link with 'libpq64', thereby rendering them unportable to any other platform. I'm really not concerned about that - a build rule to link with the right library based on pointer size is trivial. I would have thought Microsoft would have a better solution than this for managing 64-bit libraries. Or am I too optimistic about Redmond's competence? They have two separate installation directories for 32 and 64 bit packages. With PostgreSQL though, we'll quite possibly be shipping both 32 and 64 bit components in the same installer, and thus going into the same installation directory. We may have no choice about that, as we can't force all the dependent libraries to add 64 bit support when we need it. Maybe I'm missing the point and have a question. For example, do 32bit psql and the 64bit one have the same name? If so, where will they be installed? regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] krb_server_keyfile setting doesn't work on Windows
Magnus Hagander wrote: On Thu, Dec 31, 2009 at 00:57, Magnus Hagander mag...@hagander.net wrote: 2009/12/31 Hiroshi Inoue in...@tpf.co.jp: Magnus Hagander wrote: 2009/12/30 Hiroshi Inoue in...@tpf.co.jp: Hi, As far as I tested, the krb_server_keyfile setting in postgres.conf doesn't work on Windows. Because gssapi32.dll(krb5_32.dll) seems to call getenv(KRB5_KTNAME) in msvcr71, postgres.exe should call putenv(KRB5_KTNAME=...) in msvcr71. The attached patch fixes the problem in my test case. Isn't the main backend linked with msvcr71.dll anyway? What main backend directly links is msvcr80 or msvcr90 according to the msvc build environment. Arrgh. My bad, I thought msvcr71 was vs2005. Now that you put it like this, I know it's vs2003. Then the regular putenv should put it in th eenv of msvcr71.dll, and the stuff that's wrapped through src/port/win32env.c will put it in the regular msvcr file. I wonder if you're possibly being hit with the bug I patched the other day, but didn't backpatch. (http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=f8bcd7220b1166f7c037ceaf0a53958cbc6a7630). Can you see if that fix solves your problem as well? (Either directly or by testing HEAD) I'm testing using the current cvs. If not, the fix should still go in win32env.c, not directly in auth.c I don't object to it. Possibly we would have to add msvcr80 or msvcr90 as well in the future. To be safe, yes, we should have that. Do you want to work on such a complete solution, or should I look at it? Mail to you seem not to be properly delivered atm :-( Hopefully you can read this through the list. Sorry for the delay - I've been on new year holiday in Japan. I've applied a patch that should fix this problem, by always updating *all* available msvcrt libraries. Please check that it solves your problem as well. I checked the behavior using the current cvs and it works well. Thanks. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] krb_server_keyfile setting doesn't work on Windows
Hi,
As far as I tested, the krb_server_keyfile setting
in postgres.conf doesn't work on Windows.
Because gssapi32.dll(krb5_32.dll) seems to call
getenv(KRB5_KTNAME) in msvcr71, postgres.exe
should call putenv(KRB5_KTNAME=...) in msvcr71.
The attached patch fixes the problem in my test
case.
regards,
Hiroshi Inoue
Index: auth.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/libpq/auth.c,v
retrieving revision 1.188
diff -c -r1.188 auth.c
*** auth.c 12 Dec 2009 21:35:21 - 1.188
--- auth.c 30 Dec 2009 15:03:51 -
***
*** 877,882
--- 877,905
errdetail(%s: %s, msg_major, msg_minor)));
}
+ #ifdefWIN32
+ static void
+ msvc_krb5_ktname(const char *kt_path)
+ {
+ typedef int (_cdecl * PUTENVPROC) (const char *);
+ const char *msvcrdll = msvcr71;
+ static boolinitialized = false;
+ HMODULE hmodule;
+ static PUTENVPROC putenvFunc = NULL;
+
+ if (initialized)
+ {
+ if (!putenvFunc)
+ return;
+ }
+ else if (hmodule = GetModuleHandle(msvcrdll), hmodule != NULL)
+ putenvFunc = (PUTENVPROC) GetProcAddress(hmodule, _putenv);
+ initialized = true;
+ if (putenvFunc != NULL)
+ putenvFunc(kt_path);
+ }
+ #endif /* WIN32 */
+
static int
pg_GSS_recvauth(Port *port)
{
***
*** 923,928
--- 946,954
return STATUS_ERROR;
}
snprintf(kt_path, kt_len, KRB5_KTNAME=%s,
pg_krb_server_keyfile);
+ #ifdefWIN32
+ msvc_krb5_ktname(kt_path);
+ #endif /* WIN32 */
putenv(kt_path);
}
}
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] krb_server_keyfile setting doesn't work on Windows
Tom Lane wrote: Hiroshi Inoue in...@tpf.co.jp writes: Because gssapi32.dll(krb5_32.dll) seems to call getenv(KRB5_KTNAME) in msvcr71, postgres.exe should call putenv(KRB5_KTNAME=...) in msvcr71. The attached patch fixes the problem in my test case. Don't we already have something like that in our src/port putenv substitute? Yes, pgwin32_putenv() calls putenv() in msvcrt which libintl or other MINGW dlls use. I'm not sure if it's appropriate to add msvcr71's putenv call to pgwin32_putenv() (src/port/win32env.c). regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] krb_server_keyfile setting doesn't work on Windows
Magnus Hagander wrote: 2009/12/30 Hiroshi Inoue in...@tpf.co.jp: Hi, As far as I tested, the krb_server_keyfile setting in postgres.conf doesn't work on Windows. Because gssapi32.dll(krb5_32.dll) seems to call getenv(KRB5_KTNAME) in msvcr71, postgres.exe should call putenv(KRB5_KTNAME=...) in msvcr71. The attached patch fixes the problem in my test case. Isn't the main backend linked with msvcr71.dll anyway? What main backend directly links is msvcr80 or msvcr90 according to the msvc build environment. Then the regular putenv should put it in th eenv of msvcr71.dll, and the stuff that's wrapped through src/port/win32env.c will put it in the regular msvcr file. I wonder if you're possibly being hit with the bug I patched the other day, but didn't backpatch. (http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=f8bcd7220b1166f7c037ceaf0a53958cbc6a7630). Can you see if that fix solves your problem as well? (Either directly or by testing HEAD) I'm testing using the current cvs. If not, the fix should still go in win32env.c, not directly in auth.c I don't object to it. Possibly we would have to add msvcr80 or msvcr90 as well in the future. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] msvc build fails in Japanese environment
Hi,
I try to build msvc version of postgres using the current cvs and
get the following error.
Unable to determine vcbuild version from first line of output! at
src/tools/msvc/Solution.pm line 70.
The error comes from the following output of vcbuild /? in my
environment.
Microsoft(R) Visual C++ Project Builder - コマンド ライン バージョン
8.00.50727
The Command Line Version part is localized.
In addtion there's no space between Mircrosoft and (R).
The attahced patch fixes the error in my environment.
regards,
Hiroshi Inoue
Index: Solution.pm
===
RCS file: /projects/cvsroot/pgsql/src/tools/msvc/Solution.pm,v
retrieving revision 1.49
diff -c -r1.49 Solution.pm
*** Solution.pm 23 Dec 2009 13:27:04 - 1.49
--- Solution.pm 29 Dec 2009 23:47:43 -
***
*** 66,72
open(P,vcbuild /? |) || die vcbuild command not found;
my $line = P;
close(P);
! if ($line !~ /^Microsoft \(R\) Visual C\+\+ Project Builder - Command
Line Version (\d+)\.00\.\d+/) {
die Unable to determine vcbuild version from first line of output!;
}
if ($1 == 8) { $self-{vcver} = '8.00' }
--- 66,72
open(P,vcbuild /? |) || die vcbuild command not found;
my $line = P;
close(P);
! if ($line !~ /^Microsoft\s*\(R\) Visual C\+\+ Project Builder -
\D+(\d+)\.00\.\d+/) {
die Unable to determine vcbuild version from first line of output!;
}
if ($1 == 8) { $self-{vcver} = '8.00' }
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] trouble with to_char('L')
Tom Lane wrote: Hiroshi Inoue in...@tpf.co.jp writes: Tom Lane wrote: * This seems to be assuming that the user has set LC_MONETARY and LC_NUMERIC the same. What if they're different? Strictky speaking they should be handled individually. I thought about this some more, and I wonder why you did it like this at all. The patch claimed to be copying the LC_TIME code, but the LC_TIME code isn't trying to temporarily change any locale settings. LC_TIME and LC_CTYPE (on Windows) settings are changed temporarily in cache_locale_time() in pg_locale.c. What we are doing in that code is assuming that the system will give us back the localized strings in the encoding identified by CP_ACP; AFAIK it's not right. LC_TIME, LC_MONETARY or LC_NUMERIC related output is encoded using LC_CTYPE setting. so all we have to do is convert CP_ACP to wide chars and then to UTF8. Can't we use a similar approach for the output of localeconv? What LC_CTIME code and my patch intend is setting LC_CTYPE to an appropriate value so that related output is converted correctly. If we can set LC_CTYPE to xxx_xxx.65001(UTF8), we can eliminate two steps but it causes an error on Windows. regards, HIroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] trouble with to_char('L')
Tom Lane wrote: Hiroshi Inoue in...@tpf.co.jp writes: Tom Lane wrote: I think what this suggests is that there probably needs to be some encoding conversion logic near the places we examine localeconv() output. Attached is a patch to the current CVS. It uses a similar way like LC_TIME stuff does. I'm not really in a position to test/commit this, since I don't have a Windows machine. However, since no one else is stepping up to deal with it, here's a quick review: Thanks for the review. I've forgotten the patch because Japanese doesn't have trouble with this issue (the currency symbol is ascii \). If this is really expected to be fixed, I would update the patch according to your suggestion. * This seems to be assuming that the user has set LC_MONETARY and LC_NUMERIC the same. What if they're different? Strictky speaking they should be handled individually. * What if the selected locale corresponds to Unicode (ie UTF16) encoding? As far as I tested set_locale(LC_MONETARY, xxx.65001) causes an error. * #define'ing strdup() to do something rather different from strdup seems pretty horrid from the standpoint of code readability and maintainability, especially with nary a comment explaining it. Maybe using a function instead of strdup() which calls dbstr_win32() in case of Windows would be better. BTW grouping and money_grouping seem to be out of encoding conversion. Are they guaranteed to be null terminated? * Code will dump core on malloc failure. I can take care of it. * Since this code is surely not performance critical, I wouldn't bother with trying to optimize it; hence drop the special case for all-ASCII. I can take care of it. * Surely we already have a symbol somewhere that can be used in place of this: #defineMAX_BYTES_PER_CHARACTER 4 I can't find it. max(pg_encoding_max_length(encoding), pg_encoding_max_length(PG_UTF8)) may be better. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Solution of the file name problem of copy on windows.
Tom Lane wrote: Hiroshi Saito z-sa...@guitar.ocn.ne.jp writes: I want to solve one problem before the release of 8.4. However, since it also seems to be the new feature, if not enough for 8.4, you may suggest that it is 8.5. I'm not too clear on what this is really supposed to accomplish, but we are hardly going to put code like that into every single file access in Postgres, which is what seems to be the logical implication. Shouldn't we just tell people to use a database encoding that matches their system environment? Unfortunately (as usual) under Japanese Windows there's no database encoding that matches the system environment. As for the file name in COPY command, there's little meaning to convert it to the server encoding because the file name is irrelevant to the database. Because Windows is Unicode(UTF-16) based, it seems natural to convert the file name to wide characters once. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] More message encoding woes
Heikki Linnakangas wrote: Hiroshi Inoue wrote: Heikki Linnakangas wrote: I just tried that, and it seems that gettext() does transliteration, so any characters that have no counterpart in the database encoding will be replaced with something similar, or question marks. Assuming that's universal across platforms, and I think it is, using the empty string should work. It also means that you can use lc_messages='ja' with server_encoding='latin1', but it will be unreadable because all the non-ascii characters are replaced with question marks. For something like lc_messages='es_ES' and server_encoding='koi8-r', it will still look quite nice. Attached is a patch I've been testing. Seems to work quite well. It would be nice if someone could test it on Windows, which seems to be a bit special in this regard. Unfortunately it doesn't seem to work on Windows. First any combination of valid lc_messages and non-existent encoding passes the test strcmp(gettext(), ) != 0 . Now that's strange. Can you check what gettext() returns in that case then? Translated but not converted string. I'm not sure if it's a bug or not. I can see no description what should be returned in such case. Second for example the combination of ja(lc_messages) and ISO-8859-1 passes the the test but the test fails after I changed the last_trans lator part of ja message catalog to contain Japanese kanji characters. Yeah, the inconsistency is not nice. In practice, though, if you try to use an encoding that can't represent kanji characters with Japanese, you're better off falling back to English than displaying strings full of question marks. The same goes for all other languages as well, IMHO. If you're going to fall back to English for some translations (and in practice some is a pretty high percentage) because the encoding is missing a character and transliteration is not working, you might as well not bother translating at all. What is wrong with checking if the codeset is valid using iconv_open()? regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] More message encoding woes
Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Hiroshi Inoue wrote: What is wrong with checking if the codeset is valid using iconv_open()? That would probably work as well. We'd have to decide what we'd try to convert from with iconv_open(). The problem I have with that is that you are now guessing at *two* platform-specific encoding names not one, plus hoping there is a conversion between the two. AFAIK iconv_open() supports all combinations of the valid encoding values. Or we may be able to check it using the same encoding for both from and to. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] More message encoding woes
Heikki Linnakangas wrote: Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: Maybe use a special string Translate Me First that doesn't actually need to be end-user-visible, just so no one sweats over getting it right in context. Yep, something like that. There seems to be a magic empty string translation at the beginning of every po file that returns the meta-information about the translation, like translation author and date. Assuming that works reliably, I'll use that. At first that sounded like an ideal answer, but I can see a gotcha: suppose the translation's author's name contains some characters that don't convert to the database encoding. I suppose that would result in failure, when we'd prefer it not to. A single-purpose string could be documented as whatever you translate this to should be pure ASCII, never mind if it's sensible. I just tried that, and it seems that gettext() does transliteration, so any characters that have no counterpart in the database encoding will be replaced with something similar, or question marks. Assuming that's universal across platforms, and I think it is, using the empty string should work. It also means that you can use lc_messages='ja' with server_encoding='latin1', but it will be unreadable because all the non-ascii characters are replaced with question marks. For something like lc_messages='es_ES' and server_encoding='koi8-r', it will still look quite nice. Attached is a patch I've been testing. Seems to work quite well. It would be nice if someone could test it on Windows, which seems to be a bit special in this regard. Unfortunately it doesn't seem to work on Windows. First any combination of valid lc_messages and non-existent encoding passes the test strcmp(gettext(), ) != 0 . Second for example the combination of ja(lc_messages) and ISO-8859-1 passes the the test but the test fails after I changed the last_trans lator part of ja message catalog to contain Japanese kanji characters. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] More message encoding woes
Hiroshi Inoue wrote: Heikki Linnakangas wrote: Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: Maybe use a special string Translate Me First that doesn't actually need to be end-user-visible, just so no one sweats over getting it right in context. Yep, something like that. There seems to be a magic empty string translation at the beginning of every po file that returns the meta-information about the translation, like translation author and date. Assuming that works reliably, I'll use that. At first that sounded like an ideal answer, but I can see a gotcha: suppose the translation's author's name contains some characters that don't convert to the database encoding. I suppose that would result in failure, when we'd prefer it not to. A single-purpose string could be documented as whatever you translate this to should be pure ASCII, never mind if it's sensible. I just tried that, and it seems that gettext() does transliteration, so any characters that have no counterpart in the database encoding will be replaced with something similar, or question marks. Assuming that's universal across platforms, and I think it is, using the empty string should work. It also means that you can use lc_messages='ja' with server_encoding='latin1', but it will be unreadable because all the non-ascii characters are replaced with question marks. For something like lc_messages='es_ES' and server_encoding='koi8-r', it will still look quite nice. Attached is a patch I've been testing. Seems to work quite well. It would be nice if someone could test it on Windows, which seems to be a bit special in this regard. Unfortunately it doesn't seem to work on Windows. Is it unappropriate to call iconv_open() to check if the codeset is valid for bind_textdomain_codeset()? regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] More message encoding woes
Heikki Linnakangas wrote: Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: Maybe use a special string Translate Me First that doesn't actually need to be end-user-visible, just so no one sweats over getting it right in context. Yep, something like that. There seems to be a magic empty string translation at the beginning of every po file that returns the meta-information about the translation, like translation author and date. Assuming that works reliably, I'll use that. At first that sounded like an ideal answer, but I can see a gotcha: suppose the translation's author's name contains some characters that don't convert to the database encoding. I suppose that would result in failure, when we'd prefer it not to. A single-purpose string could be documented as whatever you translate this to should be pure ASCII, never mind if it's sensible. I just tried that, and it seems that gettext() does transliteration, so any characters that have no counterpart in the database encoding will be replaced with something similar, or question marks. Assuming that's universal across platforms, and I think it is, using the empty string should work. It also means that you can use lc_messages='ja' with server_encoding='latin1', but it will be unreadable because all the non-ascii characters are replaced with question marks. It doesn't occur in the current Windows environment. As for Windows gnu gettext which we are using, we would see the original msgid when iconv can't convert the msgstr to the target codeset. set client_encoding to utf_8; SET show server_encoding; server_encoding - LATIN1 (1 row) show lc_messages; lc_messages Japanese_Japan.932 (1 row) 1; ERROR: syntax error at or near 1 LINE 1: 1; OTOH when the sever encoding is utf8 then set client_encoding to utf_8; SET show server_encoding; server_encoding - UTF8 (1 row) show lc_messages; lc_messages Japanese_Japan.932 (1 row) 1; ERROR: 1またはその近辺で構文エラー LINE 1: 1; ^ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] More message encoding woes
Tom Lane wrote: Hiroshi Inoue in...@tpf.co.jp writes: Heikki Linnakangas wrote: I just tried that, and it seems that gettext() does transliteration, so any characters that have no counterpart in the database encoding will be replaced with something similar, or question marks. It doesn't occur in the current Windows environment. As for Windows gnu gettext which we are using, we would see the original msgid when iconv can't convert the msgstr to the target codeset. Well, if iconv has no conversion to the codeset at all then there is no point in selecting that particular codeset setting anyway. The question was about whether we can distinguish no conversion available from conversion available, but the test string has some unconvertible characters. What I meant is we would see no '?' when we use Windows gnu gettext. Whether conversion available or not depends on individual msgids. For example, when the Japanese msgstr corresponding to a msgid has no characters other than ASCII accidentally, Windows gnu gettext will use the msgstr not the original msgid. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] regression test crashes at tsearch
Teodor Sigaev wrote: I think that Mr. Inoue's patch is right. why isn't it taken into consideration yet? I can't check that patch because I don't have a Windows box. But I did some investigations. As I understand, the patch prevents from calling of wcstombs/mbstowcs with C locale and I checked trace for that. But wcstombs/mbstowcs doesn't called at all with C-locale. With C locale char2wchar calls pg_mb2wchar_with_len(), pg_mb2wchar_with_len() converts server encoded strings to pg_wchar strings. But pg_wchar is typedef'd as unsigned int which is not the same as wchar_t at least on Windows (unsigned short). If pg_mb2wchar_with_len() works for wchar_t on Windows, we had better call it in all cases on Windows to implement char2wchar(). and char2wchar is called from single place: TParserInit. wchar2char isn't called at all. I modified it corresponding to the change in char2wchar() so that wchar2char(char2wchar(x)) becomes x. Though I'm not sure if it is called or not, isn't it an principle? If there's an effective function like pg_wchar2mb_with_len() which converts wchar_t strings to server encoded strings, we had better simply call it for char2wchar(). regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] regression test crashes at tsearch
Hi,
I see a regression test failure in my mingw-vista port
when I invoke the command
make check MULTIBYTE=euc_jp NO_LOCALE=yes
.
It causes a crash at tsearch.
The crash seems to occur when the server encoding isn't
UTF-8 with no locale.
The attached is a patch to avoid the crash.
regards,
Hiroshi Inoue
Index: backend/utils/mb/mbutils.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/utils/mb/mbutils.c,v
retrieving revision 1.78
diff -c -r1.78 mbutils.c
*** backend/utils/mb/mbutils.c 22 Jan 2009 10:09:48 - 1.78
--- backend/utils/mb/mbutils.c 17 Feb 2009 21:59:26 -
***
*** 575,580
--- 575,584
wchar2char(char *to, const wchar_t *from, size_t tolen)
{
size_t result;
+ #ifdefWIN32
+ int encoding = GetDatabaseEncoding();
+ booluseWcstombs = !(encoding == PG_UTF8 || lc_ctype_is_c());
+ #endif
if (tolen == 0)
return 0;
***
*** 584,602
* On Windows, the Unicode locales assume UTF16 not UTF8 encoding,
* and for some reason mbstowcs and wcstombs won't do this for us,
* so we use MultiByteToWideChar().
*/
! if (GetDatabaseEncoding() == PG_UTF8)
{
! result = WideCharToMultiByte(CP_UTF8, 0, from, -1, to, tolen,
NULL, NULL);
/* A zero return is failure */
! if (result = 0)
result = -1;
else
{
- Assert(result = tolen);
/* Microsoft counts the zero terminator in the result */
! result--;
}
}
else
--- 588,624
* On Windows, the Unicode locales assume UTF16 not UTF8 encoding,
* and for some reason mbstowcs and wcstombs won't do this for us,
* so we use MultiByteToWideChar().
+* Also note wcstombs/mbstowcs is unavailable when LC_CTYPE is C.
*/
! if (!useWcstombs)
{
! int utf8len = tolen;
! char *utf8str = to;
!
! if (encoding != PG_UTF8)
! {
! utf8len = pg_encoding_max_length(PG_UTF8) * tolen;
! utf8str = palloc(utf8len + 1);
! }
! utf8len = WideCharToMultiByte(CP_UTF8, 0, from, -1, utf8str,
utf8len,
NULL, NULL);
/* A zero return is failure */
! if (utf8len = 0)
result = -1;
else
{
/* Microsoft counts the zero terminator in the result */
! result = utf8len - 1;
! if (encoding != PG_UTF8)
! {
! char *mbstr =
pg_do_encoding_conversion((unsigned char *) utf8str, result, PG_UTF8, encoding);
! result = strlcpy(to, mbstr, tolen);
! if (utf8str != to)
! pfree(utf8str);
! if (mbstr != utf8str)
! pfree(mbstr);
! }
! Assert(result = tolen);
}
}
else
***
*** 618,637
char2wchar(wchar_t *to, size_t tolen, const char *from, size_t fromlen)
{
size_t result;
if (tolen == 0)
return 0;
#ifdef WIN32
! /* See WIN32 Unicode comment above */
! if (GetDatabaseEncoding() == PG_UTF8)
{
/* Win32 API does not work for zero-length input */
! if (fromlen == 0)
result = 0;
else
{
! result = MultiByteToWideChar(CP_UTF8, 0, from, fromlen,
to, tolen - 1);
/* A zero return is failure */
if (result == 0)
result = -1;
--- 640,672
char2wchar(wchar_t *to, size_t tolen, const char *from, size_t fromlen)
{
size_t result;
+ #ifdefWIN32
+ int encoding = GetDatabaseEncoding();
+ booluseMbstowcs = !(encoding == PG_UTF8 || lc_ctype_is_c());
+ #endif
if (tolen == 0)
return 0;
#ifdef WIN32
! if (!useMbstowcs)
{
+ int utf8len = fromlen;
+ unsigned char *utf8str = (unsigned char *) from;
+
+ if (encoding != PG_UTF8)
+ {
+ utf8str = pg_do_encoding_conversion(from, fromlen,
encoding, PG_UTF8);
+ if (utf8str != from)
+ utf8len
Re: [HACKERS] mingw check hung
Andrew Dunstan wrote: Andrew Dunstan wrote: Magnus Hagander wrote: Andrew Dunstan wrote: Magnus Hagander wrote: Are we *sure*, btw, that this is actually a mingw issue, and not something else in the environment? Could you try a MSVC compiled binary on the same machine? My MSVC buildfarm animal runs on the same machine, and does not suffer the same problem. Meh. Stupid mingw :-) So how about we #ifdef out that NULL setting based on WIN32_ONLY_COMPILER, does that seem reasonable? The odd thing is that it doesn't seem to affect Vista, only XP. Anyway, yes, I think that would be OK. How do we then test to see if the original problem is still fixed? Further proof that this is a Windows version issue: I took the problem build from my XP and put it on my Vista box: the same build that causes a problem on XP runs perfectly on Vista. Go figure. Maybe we need a version check at runtime? That would be icky. Eventually does the crash come from the call SetEnvironemntVariable (.., NULL) on mingw-XP(or older?)? I'm also interested in this issue and want to know the cause. However is it necessary to call SetEnvironmentVariable() in the first place? My original patch doesn't contain SetEnvironmentVariable call in pg_unsetenv() because _putenv() seems to call SetEnvironmentVariable internally. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Explicitly bind gettext() to the UTF8 locale when in use.
Magnus Hagander wrote: Peter Eisentraut wrote: Magnus Hagander wrote: However, one question: The comment currently says it's harmless to do this on non-windows platforms. Does this still hold true? Yes, the non-WIN32 code path appears to be the same, still. But the ifdef WIN32 part we don't want, because that presumes something about the spelling of encoding names in the local iconv library. If we do keep the thing win32 only, I think we should just wrap the whole thing in #ifdef WIN32 and no longer do the codeset stuff at all on Unix - that'll make for cleaner code. Yes, that would be much better. Something like this then? It seems OK to me. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #4186: set lc_messages does not work
Magnus Hagander wrote: Hiroshi Inoue wrote: Magnus Hagander wrote: There still needs to be some error checking added in IsoLocaleName(), but this is a start. Can someone please test this? :-) OK I would check it tonight. Thanks. OK seems to works here. The attached is a test case using lc_messages C, fr, de, es and sv. regards, Hiroshi Inoue SET lc_messages - c (1 row) ERROR: syntax error at or near 1 LINE 1: 1; ^ SET lc_messages - fr (1 row) ERREUR: erreur de syntaxe sur ou près de « 1 » LINE 1: 1; ^ SET lc_messages - de (1 row) FEHLER: Fehler »Syntaxfehler« bei »1« LINE 1: 1; ^ SET lc_messages - es (1 row) ERROR: error de sintaxis en o cerca de «1» LINE 1: 1; ^ SET lc_messages - sv (1 row) FEL: syntaxfel vid eller nära 1 LINE 1: 1; ^ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Explicitly bind gettext() to the UTF8 locale when in use.
Hiroshi Inoue wrote:
Magnus Hagander wrote:
Hiroshi Inoue wrote:
Hiroshi Inoue wrote:
Bruce Momjian wrote:
Hiroshi, is this patch still needed?
Yes though it should be slightly changed now.
In what way should it be changed?
One is already committed by you.
[COMMITTERS] pgsql: Use the new text domain names
Another is to bind the codeset EUC-JP for
PG_EUC_JIS_2004 server encoding.
The attached is an updated patch.
regards,
Hiroshi Inoue
Index: mbutils.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/utils/mb/mbutils.c,v
retrieving revision 1.77
diff -c -c -r1.77 mbutils.c
*** mbutils.c 19 Jan 2009 15:34:23 - 1.77
--- mbutils.c 20 Jan 2009 12:54:33 -
***
*** 837,842
--- 837,881
return clen;
}
+ #ifdefWIN32
+ static const struct codeset_map {
+ int encoding;
+ const char *codeset;
+ } codeset_map_array[] = {
+ {PG_UTF8, UTF-8},
+ {PG_LATIN1, LATIN1},
+ {PG_LATIN2, LATIN2},
+ {PG_LATIN3, LATIN3},
+ {PG_LATIN4, LATIN4},
+ {PG_ISO_8859_5, ISO-8859-5},
+ {PG_ISO_8859_6, ISO_8859-6},
+ {PG_ISO_8859_7, ISO-8859-7},
+ {PG_ISO_8859_8, ISO-8859-8},
+ {PG_LATIN5, LATIN5},
+ {PG_LATIN6, LATIN6},
+ {PG_LATIN7, LATIN7},
+ {PG_LATIN8, LATIN8},
+ {PG_LATIN9, LATIN-9},
+ {PG_LATIN10, LATIN10},
+ {PG_KOI8R, KOI8-R},
+ {PG_WIN1250, CP1250},
+ {PG_WIN1251, CP1251},
+ {PG_WIN1252, CP1252},
+ {PG_WIN1253, CP1253},
+ {PG_WIN1254, CP1254},
+ {PG_WIN1255, CP1255},
+ {PG_WIN1256, CP1256},
+ {PG_WIN1257, CP1257},
+ {PG_WIN1258, CP1258},
+ {PG_WIN866, CP866},
+ {PG_WIN874, CP874},
+ {PG_EUC_CN, EUC-CN},
+ {PG_EUC_JP, EUC-JP},
+ {PG_EUC_KR, EUC-KR},
+ {PG_EUC_TW, EUC-TW},
+ {PG_EUC_JIS_2004, EUC-JP}};
+ #endif /* WIN32 */
+
/* mbcliplen for any single-byte encoding */
static int
cliplen(const char *str, int len, int limit)
***
*** 852,857
--- 891,898
void
SetDatabaseEncoding(int encoding)
{
+ const char *target_codeset = NULL;
+
if (!PG_VALID_BE_ENCODING(encoding))
elog(ERROR, invalid database encoding: %d, encoding);
***
*** 873,879
*/
#ifdef ENABLE_NLS
if (encoding == PG_UTF8)
! if (bind_textdomain_codeset(textdomain(NULL), UTF-8) == NULL)
elog(LOG, bind_textdomain_codeset failed);
#endif
}
--- 914,935
*/
#ifdef ENABLE_NLS
if (encoding == PG_UTF8)
! target_codeset = UTF-8;
! #ifdefWIN32
! else
! {
! int i;
!
! for (i = 0; i sizeof(codeset_map_array) / sizeof(struct
codeset_map); i++)
! if (codeset_map_array[i].encoding == encoding)
! {
! target_codeset = codeset_map_array[i].codeset;
! break;
! }
! }
! #endif /* WIN32 */
! if (target_codeset != NULL)
! if (bind_textdomain_codeset(textdomain(NULL), target_codeset)
== NULL)
elog(LOG, bind_textdomain_codeset failed);
#endif
}
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Explicitly bind gettext() to the UTF8 locale when in use.
Magnus Hagander wrote: Hiroshi Inoue wrote: Hiroshi Inoue wrote: Bruce Momjian wrote: Hiroshi, is this patch still needed? Yes though it should be slightly changed now. In what way should it be changed? One is already committed by you. [COMMITTERS] pgsql: Use the new text domain names Another is to bind the codeset EUC-JP for PG_EUC_JIS_2004 server encoding. Though EUC_JP and EUC_JIS_2004 aren't completely compatible, it seems OK in most cases. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #4186: set lc_messages does not work
Magnus Hagander wrote:
Hiroshi Inoue wrote:
Hiroshi Inoue wrote:
Magnus Hagander wrote:
Do you want to send an updated patch for it, or do you want me to look
at it?
I would send a new patch to which I added a simple ISO style check for
locale names.
Attached is a new patch.
I added a simple ISO style locale name check.
Avoided codings like *NULL == somthing*.
It also includes the changes to mbutils.c and elog.c which fix
recently introduced bug by the domain name change from postgres
to postgres-8.4.
Attached is a further updated version of this patch. Changes include:
Thanks.
* Actually avoid NULL==something style coding everywhere (I think)
* Avoid coding like if (ret = putenvFunc(envval), 0 != ret)
* Per discussion, remove pg_locale.c specific coding, put it in
src/port, and do a redefine so we *always* use these new functions
* Some further minor cleanups
* Removed the change to mbutils.c and elog.c - those are a separate
issue, will deal with those as a separate patch.
There still needs to be some error checking added in IsoLocaleName(),
but this is a start.
Can someone please test this? :-)
OK I would check it tonight.
Please note the patch relies on the proper gettext module.
The header, lib and dlls are placed on Saito-san's site
http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/libintl_check/
.
Also note the current source tries to find the message catalog
postgres-8.4.mo not postgres.mo.
The attached is an example patch to change the name of mo files
at installation time. I'm not good at perl and there may be
cleaner way.
regards,
Hiroshi Inoue
*** Install_orig.pm Wed Sep 17 13:31:08 2008
--- Install.pm Sat Jan 10 21:15:59 2009
***
*** 460,465
--- 460,478
print Installing NLS files...;
EnsureDirectories($target, share/locale);
+
+ my $majorver;
+ open(F, src/include/pg_config.h) || confess Could not open file
pg_config.h\n;
+ while (F)
+ {
+ if (/^#define\s+PG_MAJORVERSION\s+([^]+)/)
+ {
+ $majorver = $1;
+ }
+ }
+ print majorver=$majorver\n;
+ close(F);
+
my @flist;
File::Find::find({wanted =
sub { /^nls\.mk\z/s
***
*** 481,487
EnsureDirectories($target, share/locale/$lang,
share/locale/$lang/LC_MESSAGES);
system(
! \$nlspath\\bin\\msgfmt\ -o
\$target\\share\\locale\\$lang\\LC_MESSAGES\\$prgm.mo\ $_
)
croak(Could not run msgfmt on $dir\\$_);
print .;
--- 494,500
EnsureDirectories($target, share/locale/$lang,
share/locale/$lang/LC_MESSAGES);
system(
! \$nlspath\\bin\\msgfmt\ -o
\$target\\share\\locale\\$lang\\LC_MESSAGES\\$prgm-$majorver.mo\ $_
)
croak(Could not run msgfmt on $dir\\$_);
print .;
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Explicitly bind gettext() to the UTF8 locale when in use.
Hiroshi Inoue wrote: Bruce Momjian wrote: Hiroshi, is this patch still needed? Yes though it should be slightly changed now. *set lc_messages does not work* issue isn't directly related to this issue. Though I'm not sure how we can test it, I can provide test results like the attached one. The attached is a test result in case LC_MESSAGES=fr causing the following 3 errors in databases with various encodings. select a; == column a does not exist 1; == syntax error at or near 1 select * from a; == relation a does not exist Comments? Please note the encoding of the attached file is utf-8. regards, Hiroshi Inoue SET current_database -- utf8 (1 row) server_encoding - UTF8 (1 row) SET lc_messages - fr (1 row) ERREUR: la colonne « a » n'existe pas LINE 1: select a; ^ ERREUR: erreur de syntaxe sur ou près de « 1 » LINE 1: 1; ^ ERREUR: la relation « a » n'existe pas LINE 1: select * from a; ^ SET current_database -- eucjp (1 row) server_encoding - EUC_JP (1 row) SET lc_messages - fr (1 row) ERREUR: la colonne a n'existe pas LINE 1: select a; ^ ERREUR: erreur de syntaxe sur ou près de 1 LINE 1: 1; ^ ERREUR: la relation a n'existe pas LINE 1: select * from a; ^ SET current_database -- euc-jis-2004 (1 row) server_encoding - EUC_JIS_2004 (1 row) SET lc_messages - fr (1 row) ERREUR: la colonne a n'existe pas LINE 1: select a; ^ ERROR: character 0x8fabb2 of encoding EUC_JIS_2004 has no equivalent in UTF8 ERREUR: la relation a n'existe pas LINE 1: select * from a; ^ SET current_database -- euccn (1 row) server_encoding - EUC_CN (1 row) SET lc_messages - fr (1 row) ERREUR: la colonne a n'existe pas LINE 1: select a; ^ ERREUR: erreur de syntaxe sur ou près de 1 LINE 1: 1; ^ ERREUR: la relation a n'existe pas LINE 1: select * from a; ^ SET current_database -- euctw (1 row) server_encoding - EUC_TW (1 row) SET lc_messages - fr (1 row) ERREUR: la colonne a n'existe pas LINE 1: select a; ^ ERREUR: erreur de syntaxe sur ou pr`es de 1 LINE 1: 1; ^ ERREUR: la relation a n'existe pas LINE 1: select * from a; ^ SET current_database -- euckr (1 row) server_encoding - EUC_KR (1 row) SET lc_messages - fr (1 row) ERREUR: la colonne a n'existe pas LINE 1: select a; ^ ERREUR: erreur de syntaxe sur ou pr`es de 1 LINE 1: 1; ^ ERREUR: la relation a n'existe pas LINE 1: select * from a; ^ SET current_database -- latin1 (1 row) server_encoding - LATIN1 (1 row) SET lc_messages - fr (1 row) ERREUR: la colonne « a » n'existe pas LINE 1: select a; ^ ERREUR: erreur de syntaxe sur ou près de « 1 » LINE 1: 1; ^ ERREUR: la relation « a » n'existe pas LINE 1: select * from a; ^ SET current_database -- latin2 (1 row) server_encoding - LATIN2 (1 row) SET lc_messages - fr (1 row) ERREUR: la colonne a n'existe pas LINE 1: select a; ^ ERREUR: erreur de syntaxe sur ou pr`es de 1 LINE 1: 1; ^ ERREUR: la relation a n'existe pas LINE 1: select * from a; ^ SET current_database -- latin3 (1 row) server_encoding - LATIN3 (1 row) SET lc_messages - fr (1 row) ERREUR: la colonne a n'existe pas LINE 1: select a; ^ ERREUR: erreur de syntaxe sur ou près de 1 LINE 1: 1; ^ ERREUR: la relation a n'existe pas LINE 1: select * from a; ^ SET current_database -- latin4 (1 row) server_encoding - LATIN4 (1 row) SET lc_messages - fr (1 row) ERREUR: la colonne a n'existe pas LINE 1: select a; ^ ERREUR: erreur de syntaxe sur ou pr`es de 1 LINE 1: 1; ^ ERREUR: la relation a n'existe pas LINE 1: select * from a; ^ SET current_database -- latin5 (1 row) server_encoding - LATIN5 (1 row) SET lc_messages - fr (1 row) ERREUR: la colonne « a » n'existe pas LINE 1: select a; ^ ERREUR: erreur de syntaxe sur ou près de « 1 » LINE 1: 1; ^ ERREUR: la relation « a » n'existe pas LINE 1: select * from a; ^ SET current_database -- latin6 (1 row) server_encoding
Re: [HACKERS] Re: [COMMITTERS] pgsql: Explicitly bind gettext() to the UTF8 locale when in use.
Bruce Momjian wrote: Hiroshi, is this patch still needed? Yes though it should be slightly changed now. *set lc_messages does not work* issue isn't directly related to this issue. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #4186: set lc_messages does not work
Hiroshi Inoue wrote:
Magnus Hagander wrote:
Do you want to send an updated patch for it, or do you want me to look
at it?
I would send a new patch to which I added a simple ISO style check for
locale names.
Attached is a new patch.
I added a simple ISO style locale name check.
Avoided codings like *NULL == somthing*.
It also includes the changes to mbutils.c and elog.c which fix
recently introduced bug by the domain name change from postgres
to postgres-8.4.
regards,
Hiroshi Inoue
Index: backend/utils/adt/pg_locale.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/utils/adt/pg_locale.c,v
retrieving revision 1.43
diff -c -c -r1.43 pg_locale.c
*** backend/utils/adt/pg_locale.c 1 Jan 2009 17:23:49 - 1.43
--- backend/utils/adt/pg_locale.c 8 Jan 2009 19:26:02 -
***
*** 88,93
--- 88,102
static char lc_numeric_envbuf[LC_ENV_BUFSIZE];
static char lc_time_envbuf[LC_ENV_BUFSIZE];
+ /*
+ *The following 2 functions are ajusted ones for Windows MSVC.
+ */
+ static int pg_putenv(const char *); /* putenv() adjusted for MSVC */
+ extern void pg_unsetenv(const char *);/* unsetenv() adjusted for MSVC
*/
+ /*The following function is available under MSVC8.0 or later */
+ #ifdef WIN32
+ static char *IsoLocaleName(const char *); /* MSVC specific */
+ #endif
/*
* pg_perm_setlocale
***
*** 147,152
--- 156,167
case LC_MESSAGES:
envvar = LC_MESSAGES;
envbuf = lc_messages_envbuf;
+ /* Refresh msgid pool maintained by gettext? */
+ // textdomain(textdomain(NULL));
+ #ifdef WIN32
+ if (result = IsoLocaleName(locale), result == NULL)
+ return NULL;
+ #endif /* WIN32 */
break;
#endif
case LC_MONETARY:
***
*** 165,171
elog(FATAL, unrecognized LC category: %d, category);
envvar = NULL; /* keep compiler quiet */
envbuf = NULL;
! break;
}
snprintf(envbuf, LC_ENV_BUFSIZE - 1, %s=%s, envvar, result);
--- 180,186
elog(FATAL, unrecognized LC category: %d, category);
envvar = NULL; /* keep compiler quiet */
envbuf = NULL;
! return NULL;
}
snprintf(envbuf, LC_ENV_BUFSIZE - 1, %s=%s, envvar, result);
***
*** 181,189
*/
if (!SetEnvironmentVariable(envvar, result))
return NULL;
! if (_putenv(envbuf))
! return NULL;
! #endif
return result;
}
--- 196,204
*/
if (!SetEnvironmentVariable(envvar, result))
return NULL;
! if (pg_putenv(envbuf))
! result = NULL;
! #endif /* WIN32 */
return result;
}
***
*** 214,219
--- 229,352
return ret;
}
+ /*
+ *Ajusted version of putenv() for MSVC.
+ */
+ static
+ int pg_putenv(const char *envval)
+ {
+ #if defined(_MSC_VER) (_MSC_VER = 1300) /* VC7.0 or later */
+ /*
+ *Each MSVC version has its own _putenv() in its runtime library
+ *msvcrXX.dll. So we have to call _putenv() in msvcrt.dll so as
+ *to be referenced by GnuWin32 library.
+ */
+ typedef int (_cdecl *PUTENVPROC)(const char *);
+ HMODULE hmodule;
+ static PUTENVPROC putenvFunc = NULL;
+ int ret;
+
+ if (putenvFunc == NULL)
+ {
+ if (hmodule = GetModuleHandle(msvcrt), hmodule == NULL)
+ return 1;
+ putenvFunc = (PUTENVPROC) GetProcAddress(hmodule, _putenv);
+ if (putenvFunc == NULL)
+ return 1;
+ }
+ if (ret = putenvFunc(envval), ret != 0)
+ return ret;
+ #endif /* _MSC_VER = 1300 */
+ return putenv(envval);
+ }
+
+ /*
+ *Adjusted version of unsetenv() for MSVC.
+ */
+ void pg_unsetenv(const char *name)
+ {
+ #ifdefWIN32
+ char *envbuf;
+
+ if (envbuf = (char *) malloc(strlen(name) + 2))
+ {
+ sprintf(envbuf, %s=, name);
+ pg_putenv(envbuf);
+ free(envbuf);
+ }
+ #else
+ unsetenv(name);
+ #endif /* WIN32 */
+ }
+
+ #ifdef WIN32
+ /*
+ *Convert Windows locale name to the ISO's one if possible.
+ *
+ *This function returns NULL if conversion is impossible
+ *and the format style isn't ISO, otherwise returns the
+ *ISO formatted locale name.
+ */
+ static
+ char *IsoLocaleName(const char *winlocname)
+ {
+ #if (_MSC_VER = 1400) /* VC8.0 or later */
+ static char iso_lc_messages[32];
+ int usecategory
Re: [HACKERS] Solve a problem of LC_TIME of windows.
ITAGAKI Takahiro wrote: Magnus Hagander mag...@hagander.net wrote: ITAGAKI Takahiro wrote: Ok, wcsftime() requries both LC_TIME and LC_CTYPE are the same setting (at least encoding) on Windows. Hmm. Is this actually cleaner than using the original method as suggested? Because if I understand things right, that version did *not* require the setting of LC_CTYPE? (this is the version that uses strftime and does two conversions) No, we can't. I found strftime doesn't work if the database is initialized with non-C locale. It works only when lc_ctype = C and the encoding of lc_time is matched with Windows' one. On the other hand, wcsftime seems to be more robust. (The reason might come from mysterious msvcrt implementation.) The setting of LC_CTYPE acts like a catalyst for wcsftime() to work safely. I checked the code around the patch and found the central function cache_locale_time() is rarely called (at most once per LC_TIME change). The function cache_locale_time() calls strftime() only 38(=7*2+12*2) times. Now I fundamentally agree with Itagaki-san's patch. Though it may not be the most effective code, it looks like the clearest one. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #4186: set lc_messages does not work
Magnus Hagander wrote: Hiroshi Inoue wrote: Hiroshi Inoue wrote: Where are we on this? AFAICS there are 2 causes. 1. MSVC version of postgres is using a bad gettext module. 2. getenv() in mingw cannot see the result of putenv() in MSVC8.0. As for 1, we have to use another gettext module. I can provide it if requested. Yes, I think that'll be needed. Exactly what is wrong and needs to be changed? (Copying DAve in on this since he builds the MSI) As far as I see, the libintl3.dll(version 0.14.4.1952) at http://sourceforge.net/project/showfiles.php?group_id=23617package_id=17255release_id=325693 was built under a bad configuration. In fact I can see HAVE_LC_MESSAGES #defined in src\gettext\0.14.4\gettext-0.14.4\gettext-runtime\config.h. Probably due to this mistake, the gettext module libint3.dll doesn't see the environment variables LC_MESSAGES etc at all and tries to call setlocale() instead. Is it possible to build this one with the same version of MSVC? If it is, then that should remove the need for #2, right? As for 2, pg_putenv() or pg_unsetenv() in the attachced patch calls corresponding putenv() whose result can be referenced by getenv() in mingw. Oh, yuck. This must be because msvcrt.dll (used by mingw) caches the values in the environment. That's a rather ugly solution, but I guess if we have no other choice.. Does it make a difference if you try to set the value using SetEnvironmentVariable()? SetEnvironmentVariable() is called from the first in pg_perm_setlocale(). It would definitely work if the gettext stuff uses GetEnvironmentVariable(). Yes I think so. I doubt it does though, but it might work anyway... In addtion the patch provides a functionality to Windows locale name to ISO formatted locale name. Comments ? I wonder if it's cleaner to use this load msvcrt version of setenv() *always*. Or to cover all bases, perhaps we should always do *both* - that is, both set in current runtime and msvcrt.dll... We don't do this in a lot of places today, but we might use more in the future. And as long as it's not in a performance critical path, doing it twice is almost for free... Agreed. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Solve a problem of LC_TIME of windows.
ITAGAKI Takahiro wrote: Hiroshi Inoue in...@tpf.co.jp wrote: Seems LC_CTYPE and LC_TIME should be convertible even though we use wcsftime (which internally calls strftime?). Ok, wcsftime() requries both LC_TIME and LC_CTYPE are the same setting (at least encoding) on Windows. The attached patch is an updated version to fix cache_locale_time(). Now it sets LC_TIME and LC_CTYPE to the specified locale and restore them at end of the function. I tested the patch on Windows XP Japanese Edition (SJIS) with UTF-8 and EUCJP databases, and worked expectedly. I've also thought a similar implementation but there seems a problem of efficiency. As far as I see wcsftime() is almost = strftime() + mbstowcs() and so using strftime() is effective at least for the following cases. 1) LC_CTIME is C. 2) LC_CTYPE != C and the database encoding != UTF-8. In this case the current restriction of PostgreSQL requires that the database encoding matches the encoding of the LC_CTYPE. We seem to be able to call strftime() directly in above cases. Comments ? regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #4186: set lc_messages does not work
Magnus Hagander wrote: Hiroshi Inoue wrote: AFAICS there are 2 causes. 1. MSVC version of postgres is using a bad gettext module. 2. getenv() in mingw cannot see the result of putenv() in MSVC8.0. As for 1, we have to use another gettext module. I can provide it if requested. Yes, I think that'll be needed. Exactly what is wrong and needs to be changed? (Copying DAve in on this since he builds the MSI) Is it possible to build this one with the same version of MSVC? If it is, then that should remove the need for #2, right? Maybe so but what I built was gettext-0.17 and the README.woe32 says This file explains how to create binaries for the mingw execution environment. MS Visual C/C++ with nmake is no longer supported. BTW gettext-0.14.4 has both README.mingw and README.woe32. REAME.woe32 seems to be for MSVC build. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Solve a problem of LC_TIME of windows.
Alvaro Herrera wrote: ITAGAKI Takahiro wrote: Hiroshi Inoue in...@tpf.co.jp wrote: Seems LC_CTYPE and LC_TIME should be convertible even though we use wcsftime (which internally calls strftime?). Ok, wcsftime() requries both LC_TIME and LC_CTYPE are the same setting (at least encoding) on Windows. shouldn't this use LC_TIME? + setlocale(LC_CTYPE, locale_time); AFAIK the results of strftime() is determined by LC_TIME and LC_CTYPE. Date/time representations by LC_TIME language are converted to LC_CTYPE codeset. In fact I can see the Chinese strftome() representaions by Japanese codeset fortunately. Our goal is to convert LC_CTIME representaions to the database encoding. Though we expected that wcsftime() doesn't rely on LC_CTYPE, wcsftime() relies on LC_CTYPE because it is almost = strftime() + mbstowcs() and a problem occured e.g. when the LC_CTYPE is set to C. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #4186: set lc_messages does not work
Magnus Hagander wrote: Hiroshi Inoue wrote: Magnus Hagander wrote: Hiroshi Inoue wrote: AFAICS there are 2 causes. 1. MSVC version of postgres is using a bad gettext module. 2. getenv() in mingw cannot see the result of putenv() in MSVC8.0. As for 1, we have to use another gettext module. I can provide it if requested. Yes, I think that'll be needed. Exactly what is wrong and needs to be changed? (Copying DAve in on this since he builds the MSI) Is it possible to build this one with the same version of MSVC? If it is, then that should remove the need for #2, right? Maybe so but what I built was gettext-0.17 and the README.woe32 says This file explains how to create binaries for the mingw execution environment. MS Visual C/C++ with nmake is no longer supported. BTW gettext-0.14.4 has both README.mingw and README.woe32. REAME.woe32 seems to be for MSVC build. They are so annoying... Anyway, if they don't support it, we shuldn't try to force it. Let's go down the path that they will have different ones, and make getenv() and friends update both. Do you want to send an updated patch for it, or do you want me to look at it? I would send a new patch to which I added a simple ISO style check for locale names. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #4186: set lc_messages does not work
Hi, I posted a patch 18 days ago but have got no responce. Anyway I've simplified the patch by using an appropriate gettext module. Hiroshi Inoue wrote: Bruce Momjian wrote: Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: Thomas H. wrote: so at least that explains the changed behaviour. nevertheless, LC_MESSAGES seems to be defunct - with the locale folder present, pg always picks the os' language and ignores the lc_message value. This looks like I can reproduce though, at least on cvs head. Did this work for you in previous versions? Maybe we were using a different build of gettext in the previous releases, one that didn't look at the same info as the current code? Where are we on this? AFAICS there are 2 causes. 1. MSVC version of postgres is using a bad gettext module. 2. getenv() in mingw cannot see the result of putenv() in MSVC8.0. As for 1, we have to use another gettext module. I can provide it if requested. As for 2, pg_putenv() or pg_unsetenv() in the attachced patch calls corresponding putenv() whose result can be referenced by getenv() in mingw. In addtion the patch provides a functionality to Windows locale name to ISO formatted locale name. Comments ? regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #4186: set lc_messages does not work
Oops, I forgot to attach the patch, sorry.
Hiroshi Inoue wrote:
Hi,
I posted a patch 18 days ago but have got no responce.
Anyway I've simplified the patch by using an appropriate
gettext module.
Hiroshi Inoue wrote:
Bruce Momjian wrote:
Tom Lane wrote:
Magnus Hagander mag...@hagander.net writes:
Thomas H. wrote:
so at least that explains the changed behaviour. nevertheless,
LC_MESSAGES seems to be defunct - with the locale folder present,
pg always picks the os' language and ignores the lc_message value.
This looks like I can reproduce though, at least on cvs head. Did this
work for you in previous versions?
Maybe we were using a different build of gettext in the previous
releases, one that didn't look at the same info as the current code?
Where are we on this?
AFAICS there are 2 causes.
1. MSVC version of postgres is using a bad gettext module.
2. getenv() in mingw cannot see the result of putenv() in MSVC8.0.
As for 1, we have to use another gettext module. I can provide it
if requested.
As for 2, pg_putenv() or pg_unsetenv() in the attachced patch calls
corresponding putenv() whose result can be referenced by getenv() in
mingw.
In addtion the patch provides a functionality to Windows locale
name to ISO formatted locale name.
Comments ?
regards,
Hiroshi Inoue
Index: backend/main/main.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/main/main.c,v
retrieving revision 1.111
diff -c -c -r1.111 main.c
*** backend/main/main.c 11 Dec 2008 07:34:07 - 1.111
--- backend/main/main.c 26 Dec 2008 11:54:30 -
***
*** 132,138
* environment, remove any LC_ALL setting, so that the environment
* variables installed by pg_perm_setlocale have force.
*/
! unsetenv(LC_ALL);
/*
* Catch standard options before doing much else
--- 132,138
* environment, remove any LC_ALL setting, so that the environment
* variables installed by pg_perm_setlocale have force.
*/
! pg_unsetenv(LC_ALL);
/*
* Catch standard options before doing much else
Index: backend/utils/adt/pg_locale.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/utils/adt/pg_locale.c,v
retrieving revision 1.42
diff -c -c -r1.42 pg_locale.c
*** backend/utils/adt/pg_locale.c 23 Sep 2008 09:20:36 - 1.42
--- backend/utils/adt/pg_locale.c 26 Dec 2008 11:54:31 -
***
*** 88,93
--- 88,102
static char lc_numeric_envbuf[LC_ENV_BUFSIZE];
static char lc_time_envbuf[LC_ENV_BUFSIZE];
+ /*
+ *The following 2 functions are ajusted ones for Windows MSVC.
+ */
+ static int pg_putenv(const char *); /* putenv() adjusted for MSVC */
+ extern void pg_unsetenv(const char *);/* unsetenv() adjusted for MSVC
*/
+ /*The following function is available under MSVC8.0 or later */
+ #ifdef WIN32
+ static char *IsoLocaleName(const char *); /* MSVC specific */
+ #endif
/*
* pg_perm_setlocale
***
*** 147,152
--- 156,167
case LC_MESSAGES:
envvar = LC_MESSAGES;
envbuf = lc_messages_envbuf;
+ #ifdef WIN32
+ if (result = IsoLocaleName(locale), NULL == result)
+ result = locale;
+ #endif /* WIN32 */
break;
#endif
case LC_MONETARY:
***
*** 165,170
--- 180,186
elog(FATAL, unrecognized LC category: %d, category);
envvar = NULL; /* keep compiler quiet */
envbuf = NULL;
+ return NULL;
break;
}
***
*** 181,189
*/
if (!SetEnvironmentVariable(envvar, result))
return NULL;
! if (_putenv(envbuf))
! return NULL;
! #endif
return result;
}
--- 197,205
*/
if (!SetEnvironmentVariable(envvar, result))
return NULL;
! if (pg_putenv(envbuf))
! result = NULL;
! #endif /* WIN32 */
return result;
}
***
*** 214,219
--- 230,335
return ret;
}
+ /*
+ *Ajusted version of putenv() for MSVC.
+ */
+ static
+ int pg_putenv(const char *envval)
+ {
+ #if defined(_MSC_VER) (_MSC_VER = 1300) /* VC7.0 or later */
+ /*
+ *Each MSVC version has its own _putenv() in its runtime library
+ *msvcrXX.dll. So we have to call _putenv() in msvcrt.dll so as
+ *to be referenced by GnuWin32 library.
+ */
+ typedef int (_cdecl *PUTENVPROC)(const char *);
+ HMODULE hmodule;
+ static PUTENVPROC putenvFunc = NULL;
+ int ret;
+
+ if (NULL == putenvFunc)
+ {
+ if (hmodule
Re: [HACKERS] upper()/lower() truncates the result under Japanese Windows
Tom Lane wrote: Hiroshi Inoue in...@tpf.co.jp writes: Upper(), lower() or initcap() function truncates the result under Japanese Windows with e.g. the server encoding=UTF-8 and the LC_CTYPE setting Japanese_japan.932 . Hmm, I guess that makes sense, since the LC_CTYPE implies an encoding other than UTF-8; MB_CUR_MAX should be set according to LC_CTYPE. The proposed patch seems pretty ugly though. Why don't we just stop using MB_CUR_MAX altogether? These three functions are the only references to it AFAICS. Although it looks ugly, it only follows what wchar2char() does. Though I don't like to use MB_CUR_MAX, it seems safe as long as wchar2char() calls wcstombs(). regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] upper()/lower() truncates the result under Japanese Windows
Hi,
Upper(), lower() or initcap() function truncates the result
under Japanese Windows with e.g. the server encoding=UTF-8
and the LC_CTYPE setting Japanese_japan.932 .
Below is an example.
$ psql
psql (8.4devel)
Type help for help.
inoue=# \encoding sjis
inoue=# show server_encoding;
server_encoding
-
UTF8
(1 行)
inoue=# show LC_CTYPE;
lc_ctype
Japanese_Japan.932
(1 行)
inoue=# \set jpnstr '''カタカナ'''
inoue=# select char_length(:jpnstr);
char_length
-
4
(1 行)
inoue=# select upper(:jpnstr);
upper
カタカ
(1 行)
inoue=# select char_length(upper(:jpnstr));
char_length
-
3
(1 行)
The output of the last command should be 4 not 3.
Attached is a patch to fix the bug.
After applying the patch the result is
inoue=# select upper(:jpnstr);
upper
--
カタカナ
(1 行)
inoue=# select char_length(upper(:jpnstr));
char_length
-
4
(1 行)
regards,
Hiroshi Inoue
Index: formatting.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/utils/adt/formatting.c,v
retrieving revision 1.151
diff -c -c -r1.151 formatting.c
*** formatting.c1 Dec 2008 17:11:18 - 1.151
--- formatting.c14 Dec 2008 09:09:00 -
***
*** 1462,1467
--- 1462,1468
{
wchar_t *workspace;
int curr_char = 0;
+ size_t max_len, alloc_size;
/* Output workspace cannot have more codes than input bytes */
workspace = (wchar_t *) palloc((nbytes + 1) * sizeof(wchar_t));
***
*** 1472,1480
workspace[curr_char] = towlower(workspace[curr_char]);
/* Make result large enough; case change might change number of
bytes */
! result = palloc(curr_char * MB_CUR_MAX + 1);
! wchar2char(result, workspace, curr_char * MB_CUR_MAX + 1);
pfree(workspace);
}
else
--- 1473,1489
workspace[curr_char] = towlower(workspace[curr_char]);
/* Make result large enough; case change might change number of
bytes */
! #ifdefWIN32
! max_len = pg_database_encoding_max_length();
! if (MB_CUR_MAX max_len)
! max_len = MB_CUR_MAX;
! #else
! max_len = MB_CUR_MAX;
! #endif
! alloc_size = curr_char * max_len + 1;
! result = palloc(alloc_size);
! wchar2char(result, workspace, alloc_size);
pfree(workspace);
}
else
***
*** 1510,1515
--- 1519,1525
{
wchar_t *workspace;
int curr_char = 0;
+ size_t max_len, alloc_size;
/* Output workspace cannot have more codes than input bytes */
workspace = (wchar_t *) palloc((nbytes + 1) * sizeof(wchar_t));
***
*** 1520,1528
workspace[curr_char] = towupper(workspace[curr_char]);
/* Make result large enough; case change might change number of
bytes */
! result = palloc(curr_char * MB_CUR_MAX + 1);
! wchar2char(result, workspace, curr_char * MB_CUR_MAX + 1);
pfree(workspace);
}
else
--- 1530,1546
workspace[curr_char] = towupper(workspace[curr_char]);
/* Make result large enough; case change might change number of
bytes */
! #ifdefWIN32
! max_len = pg_database_encoding_max_length();
! if (MB_CUR_MAX max_len)
! max_len = MB_CUR_MAX;
! #else
! max_len = MB_CUR_MAX;
! #endif
! alloc_size = curr_char * max_len + 1;
! result = palloc(alloc_size);
! wchar2char(result, workspace, alloc_size);
pfree(workspace);
}
else
***
*** 1559,1564
--- 1577,1583
{
wchar_t *workspace;
int curr_char = 0;
+ size_t max_len, alloc_size;
/* Output workspace cannot have more codes than input bytes */
workspace = (wchar_t *) palloc((nbytes + 1) * sizeof(wchar_t));
***
*** 1575,1583
}
/* Make result large enough; case change might change number of
bytes */
! result = palloc(curr_char * MB_CUR_MAX + 1);
! wchar2char(result, workspace, curr_char * MB_CUR_MAX + 1);
pfree(workspace);
}
else
--- 1594,1610
}
/* Make result large enough; case change might change number
Re: [HACKERS] Re: [COMMITTERS] pgsql: Explicitly bind gettext() to the UTF8 locale when in use.
Magnus Hagander wrote: Hiroshi Inoue wrote: I think the thing us that as long as the encodings are compatible (latin1 with different names for example) it worked fine. In any case I think the problem is that gettext is looking at a setting that is not what we are looking at. Particularly with the 8.4 changes to allow per-database locale settings, this has got to be fixed in a bulletproof way. Attached is a new patch to apply bind_textdomain_codeset() to most server encodings. Exceptions are PG_SQL_ASCII, PG_MULE_INTERNAL and PG_EUC_JIS_2004. EUC-JP may be OK for EUC_JIS_2004. Unfortunately it's hard for Saito-san and me to check encodings other than EUC-JP. In principle this looks good, I think, but I'm a bit worried around the lack of testing. Thanks and I agree with you. I can do some testing under LATIN1 which is what we use in Sweden (just need to get gettext working *at all* in my dev environment again - I've somehow managed to break it), and perhaps we can find someone to do a test in an eastern-european locale to get some more datapoints? Can you outline the steps one needs to go through to show the problem, so we can confirm it's fixed in these locales? Saito-san and I have been working on another related problem about changing LC_MESSAGES locale properly under Windows and would be able to provide a patch in a few days. It seems preferable for us to apply the patch also so as to change the message catalog easily. Attached is an example in which LC_MESSAGES is cht_twn(zh_TW) and the server encoding is EUC-TW. You can see it as a UTF-8 text because the client_encoding is set to UTF-8 first. BTW you can see another problem at line 4 in the text. At the point the LC_MESSAGES is still japanese and postgres fails to convert a Japanese error message to EUC_TW encoding. There's no wonder but it doesn't seem preferable. regards, Hiroshi Inoue set client_encoding to utf_8; SET 1; psql:cmd/euctw.sql:2: ERROR: character 0xb9e6 of encoding EUC_TW has no equivalent in UTF8 select current_database(); current_database -- euctw (1 s) show server_encoding; server_encoding - EUC_TW (1 s) show lc_messages; lc_messages Japanese_Japan.932 (1 s) set lc_messages to cht; SET select a; psql:cmd/euctw.sql:7: é¯èª¤: æ¬ä½aä¸åå¨ LINE 1: select a; ^ 1; psql:cmd/euctw.sql:8: é¯èª¤: å¨èªæ³é¯èª¤éè¿ç¼ç 1 LINE 1: 1; ^ select * from a; psql:cmd/euctw.sql:9: é¯èª¤: relation aä¸åå¨ LINE 1: select * from a; ^ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Explicitly bind gettext() to the UTF8 locale when in use.
Magnus Hagander wrote:
On 25 nov 2008, at 05.00, Tom Lane [EMAIL PROTECTED] wrote:
Hiroshi Inoue [EMAIL PROTECTED] writes:
Tom Lane wrote:
If that's true then this code is presently broken for *every* locale
under Windows, not only Japanese.
Maybe there are a few languages/countires where 2 encodings are
widely used.
UTF8 vs Latin-N?
We already special-cases utf8...
I think the thing us that as long as the encodings are compatible
(latin1 with different names for example) it worked fine.
In any case I think the problem is that gettext is
looking at a setting that is not what we are looking at. Particularly
with the 8.4 changes to allow per-database locale settings, this has
got to be fixed in a bulletproof way.
Attached is a new patch to apply bind_textdomain_codeset() to most
server encodings. Exceptions are PG_SQL_ASCII, PG_MULE_INTERNAL
and PG_EUC_JIS_2004. EUC-JP may be OK for EUC_JIS_2004.
Unfortunately it's hard for Saito-san and me to check encodings
other than EUC-JP.
regards,
Hiroshi Inoue
*** mbutils.c.orig Sun Nov 23 08:42:57 2008
--- mbutils.c Wed Nov 26 12:17:12 2008
***
*** 822,830
--- 822,870
return clen;
}
+ #ifdefWIN32
+ static const struct codeset_map {
+ int encoding;
+ const char *codeset;
+ } codeset_map_array[] = {
+ {PG_UTF8, UTF-8},
+ {PG_LATIN1, LATIN1},
+ {PG_LATIN2, LATIN2},
+ {PG_LATIN3, LATIN3},
+ {PG_LATIN4, LATIN4},
+ {PG_ISO_8859_5, ISO-8859-5},
+ {PG_ISO_8859_6, ISO_8859-6},
+ {PG_ISO_8859_7, ISO-8859-7},
+ {PG_ISO_8859_8, ISO-8859-8},
+ {PG_LATIN5, LATIN5},
+ {PG_LATIN6, LATIN6},
+ {PG_LATIN7, LATIN7},
+ {PG_LATIN8, LATIN8},
+ {PG_LATIN9, LATIN-9},
+ {PG_LATIN10, LATIN10},
+ {PG_KOI8R, KOI8-R},
+ {PG_WIN1250, CP1250},
+ {PG_WIN1251, CP1251},
+ {PG_WIN1252, CP1252},
+ {PG_WIN1253, CP1253},
+ {PG_WIN1254, CP1254},
+ {PG_WIN1255, CP1255},
+ {PG_WIN1256, CP1256},
+ {PG_WIN1257, CP1257},
+ {PG_WIN1258, CP1258},
+ {PG_WIN866, CP866},
+ {PG_WIN874, CP874},
+ {PG_EUC_CN, EUC-CN},
+ {PG_EUC_JP, EUC-JP},
+ {PG_EUC_KR, EUC-KR},
+ {PG_EUC_TW, EUC-TW}};
+ #endif /* WIN32 */
+
void
SetDatabaseEncoding(int encoding)
{
+ const char *target_codeset = NULL;
+
if (!PG_VALID_BE_ENCODING(encoding))
elog(ERROR, invalid database encoding: %d, encoding);
***
*** 846,852
*/
#ifdef ENABLE_NLS
if (encoding == PG_UTF8)
! if (bind_textdomain_codeset(postgres, UTF-8) == NULL)
elog(LOG, bind_textdomain_codeset failed);
#endif
}
--- 886,907
*/
#ifdef ENABLE_NLS
if (encoding == PG_UTF8)
! target_codeset = UTF-8;
! #ifdefWIN32
! else
! {
! int i;
!
! for (i = 0; i sizeof(codeset_map_array) / sizeof(struct
codeset_map); i++)
! if (codeset_map_array[i].encoding == encoding)
! {
! target_codeset = codeset_map_array[i].codeset;
! break;
! }
! }
! #endif /* WIN32 */
! if (target_codeset != NULL)
! if (bind_textdomain_codeset(postgres, target_codeset) == NULL)
elog(LOG, bind_textdomain_codeset failed);
#endif
}
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] Solve a problem of LC_TIME of windows.
ITAGAKI Takahiro wrote: Hiroshi Saito [EMAIL PROTECTED] wrote: I think that MinGW does not have a direct relation. #define_UNICODE is required for wcsftime. Probably, ITAGAKI-san has only forgotten it.:-) No, definition of _UNICODE is independent from wcsftime (and other wcs functions). It affects only _tcs functions (_tcsftime). Wednesday in japanese should be the following sequences in unicode: wcs = 6c34 66dc 65e5 -- sui yo bi I rebuild my test on VC++2005 SP1, but it has the same bug. So, we cannot use wcsftime in Windows unless we build binaries in VC++2008 at least (and the bug is fixed there). Please call setlocale(LC_CTYPE/LC_ALL, ) first. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] Solve a problem of LC_TIME of windows.
ITAGAKI Takahiro wrote: Hiroshi Inoue [EMAIL PROTECTED] wrote: Please call setlocale(LC_CTYPE/LC_ALL, ) first. Ah, it works! But setlocale(*, ) means that we always use platform locale (Japanese and SJIS in Japan). Maybe you can call setlocale(LC_CTYPE, .20932) instead and you would get CP20932 encoding. The encoding of LC_TIME or LC_MESSAGES has little meaning. It could be different from server encoding and locale in postgres. Is it acceptable? I think we need to set LC_CTYPE and other LC_* independently... Seems LC_CTYPE and LC_TIME should be convertible even though we use wcsftime (which internally calls strftime?). As for gettext(LC_MESSAGES) on Windows we can set LC_CTYPE independently because it is unrelated to the output encoding. In addition we can call bind_textdomain_codeset to change the output encoding. I'm providing a patch to adjust the output encoding of Windows gettext. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] Solve a problem of LC_TIME of windows.
井上です。 ITAGAKI Takahiro wrote: Hiroshi Saito [EMAIL PROTECTED] wrote: Um, It was not supported. http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/LC_TIME_PATCH/ITAGAKI_PATCH.txt Hmm... the implementation of wcsftime() in msvcrt seems to be completely broken. I ran the attached test (localetest.c) and got the following results. The point is that wcsftime() returns the same character codes as strftime() i.e, the result is not an unicode string :-( The bug might be fixed in recently msvcrts in VC2005 or VC2008, but at least mingw uses the old broken C runtime. 私はmingwを使っていないのでよくわからないのですが、ランタイムも一緒に 提供しているのですか？ We'd better to use strftime() and multiple conversions to avoid the Microsoft's bug. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [COMMITTERS] pgsql: Explicitly bind gettext() to the UTF8 locale when in use.
Magnus Hagander wrote: Hiroshi Inoue wrote: Hi Magnus and all, Magnus Hagander wrote: Log Message: --- Explicitly bind gettext() to the UTF8 locale when in use. This is required on Windows due to the special locale handling for UTF8 that doesn't change the full environment. Thanks to this change UTF-8 case was solved but Japanese users are still unhappy with Windows databases with EUC_JP encoding. Shift_JIS which is a Japanese encoding under Windows doesn't match any server encoding and causes a crash with the use of gettext. So Saito-san removed ja message catalog just before the 8.3 release. Attached is a simple patch to avoid the crash and enable the use of Japanese message catalog. Please apply the patch if there's no problem. Hi! It will clearly also need an update to the comment, but I can take care of that. I assume you have tested this? Though I myself didn't test it, Saito-san tested it. The comment says that it works because we are handling UTF8 on a special way on Windows, ISTM UTF-8 isn't a special case. In fact the comment also mentions the following. * In future we might want to call bind_textdomain_codeset * unconditionally, but that If your database is in EUC_JP, I don't see why gettext() isn't picking it up properly in the first place.. In Japan 2 encodings (EUC_JP and Shift_JIS) are often used. EUC_JP is mainly used on *nix and on Windows Shift_JIS is used. We use EUC_JP not Shift_JIS as the server encoding And why do we need that on Windows only, and not on other platforms? because Shift_JIS isn't allowed as a server encoding. So the Japanese Windows native message encoding Shift_JIS never matches the server encoding EUC_JP and a conversion between Shitt_jis and EUC_JP is necessarily needed. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Explicitly bind gettext() to the UTF8 locale when in use.
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Hiroshi Inoue wrote: because Shift_JIS isn't allowed as a server encoding. So the Japanese Windows native message encoding Shift_JIS never matches the server encoding EUC_JP and a conversion between Shitt_jis and EUC_JP is necessarily needed. Ah, so we're basically hardcoding that information? The system will go up in SJIS, but since we can't deal with it, we switch it to EUC_JP? I'm not following this either. If the patch is really necessary then it seems it must be working around a bug in the Windows version of gettext, ie failure to distinguish CP932 from CP20932. Is that correct? I'm afraid I don't understand what you mean exactly. AFAIK the output encoding of Windows gettext is detemined by the ANSI system code page which is usualy CP932(Shift_JIS) in Japan and unrelated to the locale settings. In addition CP20932 is rarely used in Japan IIRC. I've never used it and don't know what it is correctly (maybe a kind of EUC_JP). regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Explicitly bind gettext() to the UTF8 locale when in use.
Tom Lane wrote: Hiroshi Inoue [EMAIL PROTECTED] writes: Tom Lane wrote: I'm not following this either. If the patch is really necessary then it seems it must be working around a bug in the Windows version of gettext, ie failure to distinguish CP932 from CP20932. Is that correct? I'm afraid I don't understand what you mean exactly. AFAIK the output encoding of Windows gettext is detemined by the ANSI system code page which is usualy CP932(Shift_JIS) in Japan and unrelated to the locale settings. If that's true then this code is presently broken for *every* locale under Windows, not only Japanese. Maybe there are a few languages/countires where 2 encodings are widely used. To my mind the really correct thing to be doing here would be to call bind_textdomain_codeset in all cases, rather than trusting gettext to guess correctly about which encoding we want. As the comment notes, we have not attempted that because the codeset names aren't well standardized. But it seems to me that we could certainly find out what codeset names are used on Windows, and apply bind_textdomain_codeset all the time on Windows. That would make a lot more sense than ad-hoc treatment of UTF-8 and EUC-JP if you ask me ... I fundamentally agree with you. What we hope is to enable the use of Japanese message catalog which we gave up in 8.3 Windows-version release. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Concurrently updating an updatable view
Heikki Linnakangas wrote: Hiroshi Inoue wrote: Concurrently updating an updatable view seems to cause an unexpected result. Is it a known issue? Looks right to me. What did you expect? Shouldn't the last response (session-2) UPDATE 1 be (seesion-2) UPDATE 0 ? regards, Hiroshi Inoue ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Concurrently updating an updatable view
Richard Huxton wrote: Heikki Linnakangas wrote: The problem is that the new tuple version is checked only against the condition in the update rule, id=OLD.id, but not the condition in the original update-claus, dt='a'. Yeah, that's confusing :(. Bit more than just normal rule confusion I'd say. Try the following two statements in parallel (assuming you've just run the previous): UPDATE test SET dt='c'; UPDATE test SET dt='x' FROM test t2 WHERE test.id=t2.id AND t2.dt='b'; This isn't a problem with the view mechanism - it's a problem with re-checking clauses involving subqueries or joins I'd guess. I don't understand the PostgreSQL specific *FROM* clause correctly. Currently the relations in the *FROM* clause seem to be read only and UPDATE operations seem to acquire no tuple level lock on them. regards, Hiroshi Inoue ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Concurrently updating an updatable view
Florian G. Pflug wrote: Richard Huxton wrote: Richard Huxton wrote: Heikki Linnakangas wrote: snip Bit more than just normal rule confusion I'd say. Try the following two statements in parallel (assuming you've just run the previous): UPDATE test SET dt='c'; UPDATE test SET dt='x' FROM test t2 WHERE test.id=t2.id AND t2.dt='b'; This isn't a problem with the view mechanism - it's a problem with re-checking clauses involving subqueries or joins I'd guess. I'm trying to decide if it's unexpected or just plain wrong, and I think I'd have to argue wrong. Or perhaps I'd not argue that :-/ Well, src/backend/executor/README agrees with you that it's wrong.. Thanks for the pointer. Note a fundamental bogosity of this approach: if the relation containing the original tuple is being used in a self-join, the other instance(s) of the relation will be treated as still containing the original tuple, whereas logical consistency would demand that the modified tuple appear in them too. Is the above description about UPDATE or DELETE operations? AFAIR SELECT FOR UPDATE operations avoided the incosistency from the first for joins though I'm not sure about subqueries. Or I may be misunderstanding something? But we'd have to actually substitute the modified tuple for the original, while still returning all the rest of the relation, to ensure consistent answers. Implementing this correctly is a task for future work. snip I think there should be a big, fat warning that self-referential updates have highly non-obvious behaviour in read-committed mode, and should be avoided. It seems pretty difficult for PostgreSQL rule system to avoid such kind of updates. I'm suspicious if UPDATABLE VIEWS can be implemented using the rule system. regards, Hiroshi Inoue ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[HACKERS] Concurrently updating an updatable view
Hi developers, Concurrently updating an updatable view seems to cause an unexpected result. Is it a known issue? = select version(); version --- PostgreSQL 8.2.1 on i686-pc-mingw32, compiled by GCC gcc.exe (GCC) 3.4.4 (mingw special) (1 row) = create table test (id int4 primary key, dt text) NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index test_pkey for table test CREATE TABLE = insert into test values (1, 'a'); INSERT 0 1 = create view test_v as select * from test; CREATE VIEW = create rule test_upd as on update to test_v do instead update test set dt=NEW.dt where id=OLD.id; CREATE RULE The result of concurrently running the same query update test_v set dt='b' where dt='a' is as follows. session-1 = begin; BEGIN session-1 = update test_v set dt='b' where dt='a'; UPDATE 1 session-2 = begin; BEGIN session-2 = update test_v set dt='b' where dt='a'; (blocked) session-1 = commit; COMMIT (session-2) UPDATE 1 *Explain* shows the following plan for the query here. = explain update test_v set dt='b' where dt='a'; QUERY PLAN - Hash Join (cost=24.57..50.59 rows=6 width=10) Hash Cond: (public.test.id = public.test.id) - Seq Scan on test (cost=0.00..21.60 rows=1160 width=10) - Hash (cost=24.50..24.50 rows=6 width=4) - Seq Scan on test (cost=0.00..24.50 rows=6 width=4) Filter: (dt = 'a'::text) (6 rows) regards, Hiroshi Inoue ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Re: [COMMITTERS] psqlodbc - psqlodbc: Put Autotools-generated files into subdirectory
Tom Lane wrote: Hiroshi Inoue [EMAIL PROTECTED] writes: Robert Treat wrote: It's generally a very bad idea for a BSD licensed project to include lgpl licensed code Psqlodbc package is LGPL licensed and seems to have little problem to include copy of BSD licensed code as a part of it. Right, that direction is fine, it's the other way around that's problematic (because adding some BSD code adds no new restrictions on what users can do with an overall-LGPL project, whereas having some LGPL components in a supposedly BSD project does limit what they can do with it). I don't see any reason why you shouldn't include those PG autoconf macros in psqlodbc. You just need to document that they have a BSD license, in case someone wants to use them by themselves. Yes I am in that direction. You may see some confusing words in my other posts but they are all I once thought. Thanks for your clarification. regards, Hiroshi Inoue ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Re: [COMMITTERS] psqlodbc - psqlodbc: Put Autotools-generated files into subdirectory
Alvaro Herrera wrote: Hiroshi Inoue wrote: Alvaro Herrera wrote: Robert Treat wrote: On Monday 07 May 2007 15:52, Joshua D. Drake wrote: Andrew Dunstan wrote: Hiroshi Inoue wrote: snip Of course, the developer who owns the LGPL-licensed copyright is free to relicense his work under a different license, so if the ODBC developers want to contribute code to Postgres they can give their work under the Postgres license. (They must obtain permission from all the involved developers, obviously). There are no original developers in the project now and I don't know where or how they are now. I personally am not so eager to change the license to BSD because it has been LGPL too long. Yes, that is a problem for releasing old code whose developers are long gone. (What I was thinking was copying *new* code from psqlodbc into Postgres). What do you mean by *new* code? New line?, word? or other kind of classification? Oppositely I thought we can implement the BSD licensed autoconf macros by ourselves but I'm not sure how it can be considered as *not derived*. ISTM it would be necessary to get legal advice to be sure that it would be considered not derived, but one would think that that's too much hassle for something that can be done much more simply by including the differently-licensed files in the first place, which is legal anyway. OK I understand. Thanks. regards, Hiroshi Inoue ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Re: [COMMITTERS] psqlodbc - psqlodbc: Put Autotools-generated files into subdirectory
Robert Treat wrote: On Monday 07 May 2007 15:52, Joshua D. Drake wrote: Andrew Dunstan wrote: Hiroshi Inoue wrote: Maybe it's BSD which is different from the license of psqlodbc (LGPL). Is there no problem with their coexistence ? Or is it possible for psqlodbc to be LGPL entirely ? I am having difficulty in understanding what the problem is. My understanding is that using BSD licensed code is ok in an LGPL project, but (probably) not vice versa. To my knowledge you can do it either way, as long as you remember that any changes to the lgpl code have to be released. It's generally a very bad idea for a BSD licensed project to include lgpl licensed code because people who try and use your work in thier own projects, under the assumption that it really is bsd licensed, get bitten when they find out that they have now illegally included code that is licensed via some other license. Psqlodbc package is LGPL licensed and seems to have little problem to include copy of BSD licensed code as a part of it. It's what I understand now from other developers' teachings. I'm still afraid of my misunderstanding a little. Thanks. Hiroshi Inoue ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Re: [COMMITTERS] psqlodbc - psqlodbc: Put Autotools-generated files into subdirectory
Alvaro Herrera wrote: Robert Treat wrote: On Monday 07 May 2007 15:52, Joshua D. Drake wrote: Andrew Dunstan wrote: Hiroshi Inoue wrote: Maybe it's BSD which is different from the license of psqlodbc (LGPL). Is there no problem with their coexistence ? Or is it possible for psqlodbc to be LGPL entirely ? I am having difficulty in understanding what the problem is. My understanding is that using BSD licensed code is ok in an LGPL project, but (probably) not vice versa. To my knowledge you can do it either way, as long as you remember that any changes to the lgpl code have to be released. It's generally a very bad idea for a BSD licensed project to include lgpl licensed code because people who try and use your work in thier own projects, under the assumption that it really is bsd licensed, get bitten when they find out that they have now illegally included code that is licensed via some other license. Of course, the developer who owns the LGPL-licensed copyright is free to relicense his work under a different license, so if the ODBC developers want to contribute code to Postgres they can give their work under the Postgres license. (They must obtain permission from all the involved developers, obviously). There are no original developers in the project now and I don't know where or how they are now. I personally am not so eager to change the license to BSD because it has been LGPL too long. Oppositely I thought we can implement the BSD licensed autoconf macros by ourselves but I'm not sure how it can be considered as *not derived*. Thanks. regards, Hiroshi Inoue ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] Re: [COMMITTERS] psqlodbc - psqlodbc: Put Autotools-generated files into subdirectory
Peter Eisentraut wrote: Am Dienstag, 8. Mai 2007 01:45 schrieb Hiroshi Inoue: Must I mail them directly to you in the first place ? Yes. Oh I seem to have been apart from the community too long. Could you please tell me where I can find the rule ? regards, HIroshi Inoue ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] Re: [COMMITTERS] psqlodbc - psqlodbc: Put Autotools-generated files into subdirectory
Peter Eisentraut wrote: Am Dienstag, 8. Mai 2007 15:12 schrieb Hiroshi Inoue: Oh I seem to have been apart from the community too long. Could you please tell me where I can find the rule ? The only rule there is is that if you want to talk to person X, you write to person X. That rule is as old as communications. Hmmm have I misunderstood mailing lists ? Is the rule right in case of communications via mailing lists ? For example I know a ML which doesn't use direct reply mais at all. As for PostgreSQL a few years ago AFAIR it was left to each member whether he sends direct reply mails as well. I don't know if it is stll so currently. regards, Hiroshi Inoue ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] Re: [COMMITTERS] psqlodbc - psqlodbc: Put Autotools-generated files into subdirectory
I've not seen your reply yet. Do you have a mind to cooperate with us ? I wrote: Peter Eisentraut wrote: Hiroshi Inoue wrote: Hiroshi Inoue wrote: User Petere wrote: Log Message: --- Put Autotools-generated files into subdirectory config/; add macro files used from PostgreSQL there so you don't need a PostgreSQL source tree to bootstrap the code. snip Added Files: --- psqlodbc/config: general.m4 (r1.1) Under what license is this file distributed ? Could you please reply to my question ? It's from the PostgreSQL source tree, so whatever license that has. So what's it ? Could you please take account of developers in the psqlodbc project a little more ? ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] Re: [COMMITTERS] psqlodbc - psqlodbc: Put Autotools-generated files into subdirectory
Maybe it's BSD which is different from the license of psqlodbc (LGPL). Is there no problem with their coexistence ? Or is it possible for psqlodbc to be LGPL entirely ? Thanks a lot. Hiroshi Inoue Jim Nasby wrote: On May 6, 2007, at 4:38 PM, Hiroshi Inoue wrote: Under what license is this file distributed ? Could you please reply to my question ? It's from the PostgreSQL source tree, so whatever license that has. So what's it ? Could you please take account of developers in the psqlodbc project a little more ? [EMAIL PROTECTED]:59]~/pgsql/HEAD:61%cat COPYRIGHT PostgreSQL Database Management System (formerly known as Postgres, then as Postgres95) Portions Copyright (c) 1996-2007, PostgreSQL Global Development Group Portions Copyright (c) 1994, The Regents of the University of California Permission to use, copy, modify, and distribute this software and its documentation for any purpose, without fee, and without a written agreement is hereby granted, provided that the above copyright notice and this paragraph and the following two paragraphs appear in all copies. IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN AS IS BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. [EMAIL PROTECTED]:59]~/pgsql/HEAD:62% -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] Re: [COMMITTERS] psqlodbc - psqlodbc: Put Autotools-generated files into subdirectory
Peter Eisentraut wrote: Hiroshi Inoue wrote: I've not seen your reply yet. You keep sending your emails to randomly invented addresses, so I don't get them. Must I mail them directly to you in the first place ? I'm sending the emails to pgsql-committes and pgsql-hackers also. Please note that the question is to your recent sudden change to psqlodbc. regards, Hiroshi Inoue ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Re: [COMMITTERS] psqlodbc - psqlodbc: Put Autotools-generated files into subdirectory
Andrew Dunstan wrote: Hiroshi Inoue wrote: Maybe it's BSD which is different from the license of psqlodbc (LGPL). Is there no problem with their coexistence ? Or is it possible for psqlodbc to be LGPL entirely ? I am having difficulty in understanding what the problem is. My understanding is that using BSD licensed code is ok in an LGPL project, but (probably) not vice versa. Thanks. I'm not an expert about licence at all. Though it's impossible to release (L)GPL code as BSD one, it seems possible to release BSD code as (L)GPL one. I feel strange a little that there's no *give and take*. regards, Hiroshi Inoue ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Re: [COMMITTERS] psqlodbc - psqlodbc: Put Autotools-generated files into subdirectory
Joshua D. Drake wrote: Hiroshi Inoue wrote: Joshua D. Drake wrote: Andrew Dunstan wrote: Hiroshi Inoue wrote: Maybe it's BSD which is different from the license of psqlodbc (LGPL). Is there no problem with their coexistence ? Or is it possible for psqlodbc to be LGPL entirely ? I am having difficulty in understanding what the problem is. My understanding is that using BSD licensed code is ok in an LGPL project, but (probably) not vice versa. To my knowledge you can do it either way, as long as you remember that any changes to the lgpl code have to be released. Could you tell me a recommendation ? I feel completely LGPL one is simpler. If you are asking me, which code you should release under, I would suggest the LGPL as the psqlodbc driver is licensed under the LGPL. Thanks a lot. Could someone confirm the following my recognition ? The LPGL package could add and release a copy of some Postgres BSD licensed code as LGPL ones together with the current LGPL code and then the package is still entirely LGPL. When some changes are made on the LGPL but originally BSD licencsed code, we can't reflect the changes to the orignal Postgres code. regards, Hiroshi Inoue ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Re: [COMMITTERS] psqlodbc - psqlodbc: Put Autotools-generated files into subdirectory
Tom Lane wrote: Hiroshi Inoue [EMAIL PROTECTED] writes: Could someone confirm the following my recognition ? The LPGL package could add and release a copy of some Postgres BSD licensed code as LGPL ones together with the current LGPL code and then the package is still entirely LGPL. No, the files you borrowed from Postgres remain BSD --- you cannot unilaterally relicense someone else's code. Oops I've completely misunderstood it. Probably I misread some articles about the coexistence. Many thanks. This does not actually affect anything, since the BSD license terms are more liberal than LGPL: you can distribute the combined package just as easily as if it were all-LGPL, and anyone using the package is still effectively bound by the LGPL terms for the useful parts. But it would be appropriate to note that files X,Y,Z are distributed under a different license, namely insert PG terms here. Thanks for your clear explanation. regards, Hiroshi Inoue ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[HACKERS] Re: [COMMITTERS] psqlodbc - psqlodbc: Put Autotools-generated files into subdirectory
Peter Eisentraut wrote: Hiroshi Inoue wrote: Hiroshi Inoue wrote: User Petere wrote: Log Message: --- Put Autotools-generated files into subdirectory config/; add macro files used from PostgreSQL there so you don't need a PostgreSQL source tree to bootstrap the code. snip Added Files: --- psqlodbc/config: general.m4 (r1.1) Under what license is this file distributed ? Could you please reply to my question ? It's from the PostgreSQL source tree, so whatever license that has. So what's it ? Could you please take account of developers in the psqlodbc project a little more ? ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Practical impediment to supporting multiple SSL libraries
Martijn van Oosterhout wrote: On Thu, Apr 13, 2006 at 09:00:10PM -0400, Tom Lane wrote: Probably there would need to be at least three callbacks involved: one for setup, called just after the tuple descriptor info has been received; one for per-field data receipt, and one for per-tuple operations (called after all the fields of the current tuple have been passed to the per-field callback). Maybe you'd want a shutdown callback too, although that's probably not strictly necessary since whatever you might need it to do could be done equally well in the app after PQgetResult returns. (You still want to return a PGresult to carry command success/failure info, and probably the tuple descriptor info, even though use of the callbacks would leave it containing none of the data.) Sounds really good. The only thing now is that the main author of the wire-protocol code in psqlODBC has not yet made any comment on any of this. So we dont want to set anything in stone until we know it would solve their problem... Unfortunately I don't have so much time to examine it. Though the double_copying issue may be the biggest one, I'm pretty sure it's not the unqiue one. We are happy if we would be able to replace the current code by libpq API one by one but it's impossible because the driver can't go back to libpq mode once after it went into hi-jacking mode. As for hi-jacking mode used in the driver it's better to be able to use encapsulated recv/send than getting the pointer to SSL or socket. regards, Hiroshi Inoue ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Practical impediment to supporting multiple SSL libraries
Martijn van Oosterhout wrote: On Fri, Apr 14, 2006 at 04:53:53PM +0200, Martijn van Oosterhout wrote: Sounds really good. snip There's a message on the pgsql-odbc mailing list[1] with some reasons for not using libpq: 1. The driver sets some session default parameters(DateStyle, client_encoding etc) using start-up message. As far as I can see it only does this when the environment variables are set. Which IMHO is the correct behaviour. IMHO if libpq is to be a generic library it should first provide exactly what it can do using the protocol. *Environment varibales* are not appropriate for per application/datasource settings at all. 3. Quote: I don't know what libraries the libpq would need in the future but it's quite unpleasant for me if the psqlodbc driver can't be loaded with the lack of needeless librairies. It's a reason, just not a good one IMHO. If the user has installed libpq with a number of libraries, then that's what the user wants. I'm not sure why psqlODBC is worried about that. It's very important to clarify for what the libraries are needed and my basic policy is to provide appropriate bindings(linkage) between the libraries for the current dependency relation. As for SSL mode it is only a mere extra for the current enhanced driver. My main purpose was to finish up my unfinished work before 7.4 using V3 protocol, holdable cursors etc. The current driver under Windows is available without the existence of libpq. regards, Hiroshi Inoue ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Practical impediment to supporting multiple SSL libraries
Stephen Frost wrote: * Martijn van Oosterhout (kleptog@svana.org) wrote: On Thu, Apr 13, 2006 at 08:48:54AM +0100, Dave Page wrote: Well, we had a pure custom implementation of the protocol, had a pure libpq based version and after much discussion decided that the best version of all was the hybrid as it allowed us to hijack features like SSL, Kerberos, pgpass et al, yet not be constrained by the limitations of libpq, or copy query results about so much. Right. Would you see value in a more formal libpq hijack-me interface that would support making the initial connection and then handing off the rest to something else? I'm wondering because obviously with the current setup, if libpq is compiled with SSL support, psqlODBC must also be. Are there any points where you have to fight libpq over control of the socket? [...] Is there anything else you might need? Instead of having it hijack the libpq connection and implement the wireline protocol itself, why don't we work on fixing the problems (such as the double-copying that libpq requires) in libpq to allow the driver (and others!) to use it in the 'orthodox' way? I would have spoken up on the ODBC list if I understood that 'hybrid' really meant 'just using libpq for connection/authentication'. I really think it's a bad idea to have the ODBC driver reimplement the wireline protocol because that protocol does change from time to time and someone using libpq will hopefully have fewer changes (and thus makes the code easier to maintain) than someone implementing the wireline protocol themselves (just causing more busy-work that, at least we saw in the past with the ODBC driver, may end up taking *forever* for someone to be able to commit the extra required time to implement). Libpq and the psqlodbc driver have walked on another road for a very long time. In 6.3 or before, there wasn't a libpq library under Windows. In 6.4 we had the libpq library under Windows but it wasn't able to talk to 6.3 or before unfortunately At last in 7.4 the libpq was able to speak both protocol v3 and protocol v2 but it is a pretty hard work at least for me to tranfer all the accummulated works to libpq based version. I'm not sure what kind of functionalities required for libpq to make the tranfer easy. Of cource double-copying issue is big one of them. regards, Hiroshi Inue ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Practical impediment to supporting multiple SSL libraries
Martijn van Oosterhout wrote: On Wed, Apr 12, 2006 at 05:03:32PM +0100, Dave Page wrote: about the declaration of PQgetssl The next version of psqlODBC (that has just gone into CVS tip after months of work and debate) uses it, and would break almost completely should it be removed, therefore any backwards incompatible change should be avoided imho. And 2 or 4 could cause chaos for Windows users if different DLL builds get mixed up. Hmm, may I ask what it uses it for? Just to get information, or something more substantial? In case of SSL mode, the driver gets the communication path using PQsocket() or PQgetssl() after calling PQconnectdb(). The driver comunicates with the server by itself using the path. In case of non-SSL mode, the driver never calls libpq API at all. regards, Hiroshi Inoue ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Why isn't DECLARE CURSOR ... FOR UPDATE supported?
-Original Message- From: Tom Lane Is there any good reason for this restriction? regression=# begin; BEGIN regression=# declare c cursor for select * from tenk1 for update; ERROR: DECLARE CURSOR ... FOR UPDATE is not supported DETAIL: Cursors must be READ ONLY. Because we haven't supported updatable cursors yet. regards, Hiroshi Inoue ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] 2-phase commit
I seem to have misunderstood the problem completely. (BI apologize to you all(especially Tom) for disturbing (Bthis thread. (B (BI wonder if there might be such a nice solution when (Bsome of the systems or communications are dead. (BAnd as many people already mentioned, there's not so (Bmuch allowance if we only adopt XA-based protocol. (B (Bregards, (BHiroshi Inoue (Bhttp://www.geocities.jp/inocchichichi/psqlodbc/ (B (BTom Lane wrote: (B (B Hiroshi Inoue [EMAIL PROTECTED] writes: (B The simplest senario(though there could be varations) is (B (B [At participant(master)'s side] (BBecause the commit operations is done, does nothing. (B (B [At coordinator(slave)' side] (B 1) After a while (B 2) re-establish the communication path between the (Bpartcipant(master)'s TM. (B 3) resend the "commit requeset" to the participant's TM. (B1)2)3) would be repeated until the coordinator receives (Bthe "commit ok" message from the partcipant. (B (B [ scratches head ] I think you are using the terms "master" and "slave" (B oppositely than I would. But in any case, this is not an answer to the (B concern I had. You're assuming that the "coordinator(slave)" side is (B willing to resend a request indefinitely, and also that the (B "participant(master)" side is willing to retain per-transaction commit (B state indefinitely so that it can correctly answer belated questions (B from the other side. What I was complaining about was that I don't (B think either side can afford to remember per-transaction state (B indefinitely. 2PC in the abstract is a useless academic abstraction --- (B where the rubber meets the road is defining how you cope with failures (B in the commit protocol. (B (B regards, tom lane (B (B---(end of broadcast)--- (BTIP 7: don't forget to increase your free space map settings
