Re: [HACKERS] SunOS4
I've been working to make PostgreSQL run on SunOS4 (again). I have committed massive changes for SunOS4 port. Tested on: SunOS 4.1.4 Vine Linux 2.1 (variant of RedHat Linux 6.2J) FreeBSD 4.2-RELEASE Please let me know if I have broken something. -- Tatsuo Ishii
[HACKERS] Re: SOLVED: COPY doesn't works when containing ' ' or ' ' characters on db
On Mon, 26 Feb 2001 22:16:35 -0500 Tom Lane [EMAIL PROTECTED] wrote: Jaume Teixi [EMAIL PROTECTED] writes: I finally percated that when data contains '' or '' it's impossible to parse trought: COPY products FROM '/var/lib/postgres/dadesi.txt' USING DELIMITERS '|' \g it causes: SELECT edicion FROM products; edicion - Espaa|Nacional ---puts on the same cell either there's an '|' in the middle!!! I finally, thanks to Oliver Elphick, managed to create database with: CREATE DATABASE "demo" WITH ENCODING = 'SQL_ASCII' and data was imported OK, great, thanks!
Re: [HACKERS] Idea for reducing planning time
Bruce Momjian [EMAIL PROTECTED] writes: If you like I'll post the patch, but it strikes me as a waste of list bandwidth --- anyone who is likely to actually review it is perfectly capable of doing cvs diff for themselves ... Posting patch is only useful if you want people to review it. They are more likely to if you send it to patches, already diff'ed. In fact, how do you pull out a patch set from CVS? You have to use -D and specify a date/time range, right? That's a good point --- there doesn't seem to be any real easy way of extracting a set of changes to different files except to use -D. And even that doesn't do anything to separate unrelated patches applied at about the same time. For the record, you can get a diff of this kind with a command like cvs diff -c -D '2000-12-14 17:00' -D '2000-12-14 18:00' executed in the top level of the tree you want to search. regards, tom lane Tom, do you have a plan to make a back patch for 7.0.3? I got a bug report from a user with a script to reproduce the problem. Seems the backend consumes infinite memory. I don't want to recommend her to use 7.0.2 since it has a merge join problem... -- Tatsuo Ishii join.tar.gz
AW: [HACKERS] CommitDelay performance improvement
I agree that 30k looks like the magic delay, and probably 30/5 would be a good conservative choice. But now I think about the choice of number, I think it must vary with the speed of the machine and length of the transactions; at 20tps, each TX is completing in around 50ms. I think disk speed should probably be the main factor. After the first run 30k/5 also seemed the best here, but running the test again shows, that the results are only reproducible after a new initdb. Anybody else see reproducible results without previous initdb ? One thing I noticed is, that WAL_FILES needs to be at least 4, because one run fills up to 3 logfiles, and we don't want to measure WAL formating. Andreas
Re: AW: [HACKERS] CommitDelay performance improvement
At 10:56 27/02/01 +0100, Zeugswetter Andreas SB wrote: I agree that 30k looks like the magic delay, and probably 30/5 would be a good conservative choice. But now I think about the choice of number, I think it must vary with the speed of the machine and length of the transactions; at 20tps, each TX is completing in around 50ms. I think disk speed should probably be the main factor. After the first run 30k/5 also seemed the best here, but running the test again shows, that the results are only reproducible after a new initdb. Anybody else see reproducible results without previous initdb ? I think we want something that reflects the chance of a time-saving as a result of a wait, which is why I suggested having each backend monitor commits/sec, then basing the delay on some % of that number. eg. if commits/sec = 1, then it's either low-load, or long tx's, in either case CommitDelay won't help. Similarly, if we have 1000 commits/sec, then we have a very fast system and/or disk, and CommitDelay of 10ms is clearly glacial. AFAICS, dynamically monitoring commits/sec (or a similar statistic) is TOWTG, but in all cases we need to set a max on CommitDelay to prevent individual TXs getting too long (although I am unsure if the latter is *really* necessary, it is far better to be safe). Note: commits/sec need to be kept for each backend so we can remove the contribution of the backend that is considering waiting. Philip Warner| __---_ Albatross Consulting Pty. Ltd. |/ - \ (A.B.N. 75 008 659 498) | /(@) __---_ Tel: (+61) 0500 83 82 81 | _ \ Fax: (+61) 0500 83 82 82 | ___ | Http://www.rhyme.com.au |/ \| |---- PGP key available upon request, | / and from pgp5.ai.mit.edu:11371 |/
[HACKERS] Query precompilation?
Hi, I have an application which has an queue of data it has to insert into a table in a local database. the insert-queries syntax is all the same, and the values are the only thing that differs. The insert-query looks like this: INSERT INTO "table" VALUES(a, b, c, d, e, f, g, h) ...but I cannot insert more than 200/sec, and that is much too slow for me. Are there ways to precompile a sqlquery or do other tricks to get the *fastest* insertion-rate, since the data-queue is growing faster than 200/sec... I don't care about integrity etc! I'm using PostgreSQL 7.0.3, RH 6.2 Linux 2.2.4, and the pq library with gcc. Regards, Steffen E. Thorkildsen (PS! Please reply to my e-mail aswell.)
Re[4]: [HACKERS] Re: [PATCHES] A patch for xlog.c
On Tue, 27 Feb 2001, Xu Yifeng wrote: Hello Tom, Tuesday, February 27, 2001, 12:45:18 PM, you wrote: TL Bruce Momjian [EMAIL PROTECTED] writes: I know it is easy for you, but the number of reports and problems we hear about shows it is an issue for some. TL We hear some reports, but not a lot. We have no idea whatever what TL problems might ensue if we used mmap instead. I'm dubious that SysV TL shmem creates enough problems to justify replacing it with a solution TL of essentially unknown portability characteristics... TL regards, tom lane could anyone investigate mmap() in many modern UNIX systems to prove that mmap() is so un-portable? it seems mmap() is a portable problem like you said, but I think SYSV shmem for PGSQL is a installation problem. you push some difficults to end user, and take easy taskes for yourself. Considering that, so far as I can tell, both you and Bruce are the only ones that are really heavy on moving away from SysV ... how many ppl are actually finding it to be that much more difficult? :)
Re: [HACKERS] RE: Re: [ADMIN] v7.1b4 bad performance
So 7.0.3 is twice as fast only with fsync off. Are there FK updates/deletes in pgbench' tests? Remember how SELECT FOR UPDATE in FK triggers affects performance... Also, 5 clients is small number. Vadim P.S. Sorry for delays with my replies - internet connection is pain here: it takes 5-10 minutes to read each message -:((( --- FREE! The World's Best Email Address @email.com Reserve your name now at http://www.email.com
Re: [HACKERS] WAL does not recover gracefully from out-of-disk-space
Regardless of whether this particular behavior is fixable, this brings up something that I think we *must* do before 7.1 release: create a utility that blows away a corrupted logfile to allow the system to restart with whatever is in the datafiles. Otherwise, there is no recovery technique for WAL restart failures, short of initdb and restore from last backup. I'd rather be able to get at data of questionable up-to-dateness than not have any chance of recovery at all. I've asked 2 or 3 times how to recover from recovery failure but got no answer. We should some recipi for the failure before 7.1 release. And I answered 2 or 3 times with fixes for each reported recovery failure -:) (And asked to help with testing...) Seems to me that "fixing" is the only "answer" you would get asking the same question to Oracle, Informix or any other system with transaction log. Does anybody know how "big boys" deal with this issue? Vadim --- FREE! The World's Best Email Address @email.com Reserve your name now at http://www.email.com
Re: [HACKERS] Release in 2 weeks ...
The Hermit Hacker wrote: Morning all ... Are there any major outstandings that ppl have on their plates, that should prevent a release? I'd like to put out an RC1 by Friday this week, with a full release schedualed for March 15th ... this would give Thomas his two weeks for the docs freeze ... Basically, RC1 would say to ppl that we're ready to release, there will be no more core changes that will require an initdb ... feel comfortable using this version in production, with the only major changes between now and release being docs related ... Does this work? Or is there something earth-shattering that still has to be done? Yep ! As of beta4, the ODBC driver is still seriously broken (the original libpsqlodbc.so.0.26 doesn't even connect. A version patched by Nick Gorham allows some connectivity (you can query the DB), but still has some serious breakage (i. e. no "obvious" ways to see views from StarOffice or MS-Access)). And I have not yet had any opportunity to test the JDBC driver. [ Explanation : I follow the Debian packages prepared by Oliver Elphick, I'm not versed enough in Debian to recreate those packages myself, and I do *not* want to break Debian dependencies by installing Postgres "The Wrong Way (TM)". Hence, I'm stuck with beta4, a broken ODBC and no JDBC. Unless some kind soul can send me a JD. 1.1 .jar file ... Furthermore, I've had some serious hardware troubles (a dying IDE disk). I wasn't even able to fulfill Tom Lane's suggestion to try to add -d2 to my postmaster to debug the ODBC connection. I'll try to do that Real Soon Now (TM, again), but not for now : my day-work backlog is ... impressive. ] These issues might seem small change to you die-hard plpgsql hackers. To a lmot of people using Postgres for everyday office work through "nice" interface, it's bread-and-butter, and these issues *should* be fixed *before* release ... [ crawling back under my rock ... ] Emmanuel Charpentier
[HACKERS] Re: Query precompilation?
On Tue, 27 Feb 2001, you wrote: Hi, I have an application which has an queue of data it has to insert into a table in a local database. the insert-queries syntax is all the same, and the values are the only thing that differs. The insert-query looks like this: INSERT INTO "table" VALUES(a, b, c, d, e, f, g, h) ...but I cannot insert more than 200/sec, and that is much too slow for mme. Are there ways to precompile a sqlquery or do other tricks to get the *fastest* insertion-rate, since the data-queue is growing faster than 200/sec... I don't care about integrity etc! You should !-) You can find some valueable tips in the documentation: http://www.de.postgresql.org/users-lounge/docs/7.0/user/c4929.htm I'm using PostgreSQL 7.0.3, RH 6.2 Linux 2.2.4, and the pq library with gcc. Regards, Steffen E. Thorkildsen (PS! Please reply to my e-mail aswell.)
[HACKERS] Re: [PATCHES] A patch for xlog.c
I don't know about other 'commercial OSs', but I'd be shocked if a Linux admin never does any kernel config cleanup befor egoing production *shrug* oops... - Thomas
Re: [HACKERS] SunOS4
Tatsuo Ishii writes: I've been working to make PostgreSQL run on SunOS4 (again). I have committed massive changes for SunOS4 port. Tested on: SunOS 4.1.4 Vine Linux 2.1 (variant of RedHat Linux 6.2J) FreeBSD 4.2-RELEASE Please let me know if I have broken something. I think the OPTARG_DECL is not necessary. We've gotten by since the beginning of time with always declaring them explicitly. And you're not even using it consistently. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
Re: [HACKERS] Idea for reducing planning time
Tatsuo Ishii [EMAIL PROTECTED] writes: Tom, do you have a plan to make a back patch for 7.0.3? No, I don't. No time for it now. I got a bug report from a user with a script to reproduce the problem. Seems the backend consumes infinite memory. Not infinite, surely ;-) ... but possibly more than her kernel will allow. As a workaround, I'd suggest setting geqo_threshold smaller, perhaps 8 or 9. IIRC, the way to do that in 7.0 is set geqo='on=8'; Not sure if it's possible to set up a system-wide default for that in 7.0. BTW, the main reason planning this join in 7.0 is so slow is that all the options look exactly alike and so the planner has no reason to discard any paths. As soon as you create some indexes, load up some data and VACUUM, the symmetry would be broken and performance should improve (geqo or not). Or at least it'd usually be broken. Is it possible that all her tables are exactly the same size? regards, tom lane
Re: [HACKERS] Query precompilation?
Steffen Emil Thorkildsen [EMAIL PROTECTED] writes: I have an application which has an queue of data it has to insert into a table in a local database. the insert-queries syntax is all the same, and the values are the only thing that differs. The insert-query looks like this: INSERT INTO "table" VALUES(a, b, c, d, e, f, g, h) ...but I cannot insert more than 200/sec, and that is much too slow for me. Consider using COPY FROM STDIN instead ... regards, tom lane
Re: [HACKERS] 'postgres -Q' in test/bench
Marko Kreen writes:
test/bench/{create|runtest}.sh uses switch '-Q' for
postgres, but postgres gives error on it. Otherwise
seems working, only lots of debug output is seen.
Replace it with '-d 0'. I'm not sure these benchmark tools are maintained
at all. contrib/pgbench seems to be preferred.
--
Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
[HACKERS] Re: Query precompilation?
Steffen Emil Thorkildsen [EMAIL PROTECTED] writes: me. Are there ways to precompile a sqlquery or do other tricks to get the *fastest* insertion-rate, since the data-queue is growing faster than 200/sec... I don't care about integrity etc! I'm using PostgreSQL 7.0.3, RH 6.2 Linux 2.2.4, and the pq library with gcc. Apart from the COPY mentioned by Tom Lane, you should also fo through the obvious checklist: use -F to disable fsync, drop indexes(if possible), use several connections(could help if you have multiprossessor system)
Re: Re[3]: [HACKERS] Re: [PATCHES] A patch for xlog.c
The Hermit Hacker writes: I don't ... I personally admin FreeBSD and Solaris boxen ... FreeBSD, first step is to always recompile the kernel after an install, to get rid of crud and add Shared Memory ... the Solaris boxes, you add a couple of lines to /etc/system and reboot, and you have Shared Memory ... I don't know about other 'commercial OSs', but I'd be shocked if a Linux admin never does any kernel config cleanup befor egoing production *shrug* Linux allows you to load and unload kernel modules, while the system is running, to add and remove stuff as you need it. But this is moot because Linux also allows you to increase shared memory (up to the total addressable memory) while the system is running. Recompiling Linux kernels is a thing of the past with modern distributions. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
Re: [ADMIN] COPY doesn't works when containing ' ' or ' ' characters on db
Tom Lane wrote: Jaume Teixi [EMAIL PROTECTED] writes: I finally percated that when data contains '' or '' it's impossible to parse trought: COPY products FROM '/var/lib/postgres/dadesi.txt' USING DELIMITERS '|' \g it causes: SELECT edicion FROM products; edicion - Espaa|Nacional ---puts on the same cell either there's an '|' in the middle!!! Very odd. What LOCALE and multibyte encodings are you using, if any? This seems like it must be a multibyte issue, but I can't guess what. Also, which Postgres version are you running? If you said, I missed it. I think this happens when the front-end encoding is SQL_ASCII and the database is using UNICODE. Then, there are misunderstandings between front-end and back-end, so that a single character with the eighth bit set may be sent by the front-end and interpreted by the back-end as the first half of a UNICODE two-byte character. -- Oliver Elphick[EMAIL PROTECTED] Isle of Wight http://www.lfix.co.uk/oliver PGP: 1024R/32B8FAA1: 97 EA 1D 47 72 3F 28 47 6B 7E 39 CC 56 E4 C1 47 GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C "If we confess our sins, he is faithful and just to forgive us our sins, and to cleanse us from all unrighteousness." I John 1:9
Re: [ADMIN] COPY doesn't works when containing ' ' or ' ' characters on db
"Oliver Elphick" [EMAIL PROTECTED] writes: I think this happens when the front-end encoding is SQL_ASCII and the database is using UNICODE. Then, there are misunderstandings between front-end and back-end, so that a single character with the eighth bit set may be sent by the front-end and interpreted by the back-end as the first half of a UNICODE two-byte character. I wondered about that, but his examples had one or more characters between the eighth-bit-set character and the '|', so this doesn't seem to explain the problem. Still, if it went away after moving to ASCII encoding, it clearly is a multibyte issue of some sort. regards, tom lane
Re: [HACKERS] Re: Query precompilation?
(...) I don't care about integrity etc! You should !-) You can find some valueable tips in the documentation: http://www.de.postgresql.org/users-lounge/docs/7.0/user/c4929.htm In the docs there is this paragraph: Disable Auto-commit Turn off auto-commit and just do one commit at the end. Otherwise Postgres is doing a lot of work for each record added. In general when you are doing bulk inserts, you want to turn off some of the database features to gain speed. This sounds nice, but I've read a lot of postgres documents and still do not know how to disable autocommit. Is this possible? And how? Mario Weilguni -- === Mario Weilguni KPNQwest Austria GmbH Senior Engineer Web Solutions Nikolaiplatz 4 tel: +43-316-8138248020 graz, austria fax: +43-316-813824-26 http://www.kpnqwest.at e-mail: [EMAIL PROTECTED] ===
Re: [HACKERS] SunOS4
Tatsuo Ishii [EMAIL PROTECTED] writes: I have committed massive changes for SunOS4 port. Tested on: SunOS 4.1.4 Vine Linux 2.1 (variant of RedHat Linux 6.2J) FreeBSD 4.2-RELEASE Please let me know if I have broken something. Everything still builds and passes regression on HPUX, but I concur with Peter that the HAVE_OPTARG configure stuff must be unnecessary. Please observe that src/backend/bootstrap/bootstrap.c src/backend/postmaster/postmaster.c src/backend/tcop/postgres.c src/bin/pg_dump/pg_dump.c src/bin/psql/startup.c src/interfaces/ecpg/preproc/ecpg.c all seem to be getting along fine with no configure test. There are also a bunch of contrib modules that use optarg, and would also need to be changed if you want to apply a configure test. I suggest reverting the configure and config.h changes and instead making pg_restore and pg_id follow the coding practices used in the above-mentioned files for optarg. regards, tom lane
[HACKERS] beta5 and unixware 711
Hi all, I've been trying to play with beta5 today on unixware 711. I have 2 problems: 1) enabling --with-tcl yields to link errors on bin/pgtclsh and interfaces/pl/tcl because Makefile insists on linking with libtcl7.6.0 instead on libtcl7.6 2) enabling --with-openssl causes a compilation error on src/backend/libpq/crypt.c because of multiply defined symbol des_encrypt Regards, -- Olivier PRENANT Tel:+33-5-61-50-97-00 (Work) Quartier d'Harraud Turrou +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: [EMAIL PROTECTED] -- Make your life a dream, make your dream a reality. (St Exupery)
Re: [HACKERS] Release in 2 weeks ...
Emmanuel Charpentier [EMAIL PROTECTED] writes: Yep ! As of beta4, the ODBC driver is still seriously broken (the original libpsqlodbc.so.0.26 doesn't even connect. A version patched by Nick Gorham allows some connectivity (you can query the DB), but still has some serious breakage (i. e. no "obvious" ways to see views from StarOffice or MS-Access)). I'd be willing to work harder on ODBC if I had any way to test it ;-). I have a copy of OpenOffice for LinuxPPC but have not figured out how to tell it to connect to Postgres. If someone can slip me a clue on how to configure it and do simple database stuff with it, I'll try to clean up the most pressing ODBC problems before we release. regards, tom lane
Re: [HACKERS] regression.out and regression.diff
Vince Vielhaber writes:
is regression.out and/or regression.diff deleted if the tests pass? I've
never seen all tests pass so I don't know but I just had someone tell me
that it does.
We've come to a point where all tests should pass all the time on all
supported platforms. If they don't, that's a bug and should be posted to
a mailing list.
The test output should come from 'gmake check', not the test against an
already installed server ('installcheck').
--
Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
Re: [HACKERS] regression.out and regression.diff
On Tue, 27 Feb 2001, Peter Eisentraut wrote:
Vince Vielhaber writes:
is regression.out and/or regression.diff deleted if the tests pass? I've
never seen all tests pass so I don't know but I just had someone tell me
that it does.
We've come to a point where all tests should pass all the time on all
supported platforms. If they don't, that's a bug and should be posted to
a mailing list.
The test output should come from 'gmake check', not the test against an
already installed server ('installcheck').
It was a snapshot - no idea what kind of shape it was in and I don't
recall how I did the check, but it may have been installcheck.
Either way my question still needs to be answered. If there are no
errors, is regression.out and regression.diff deleted? I can see the
diff file being empty, but deleted?
Vince.
--
==
Vince Vielhaber -- KA8CSHemail: [EMAIL PROTECTED]http://www.pop4.net
128K ISDN from $22.00/mo - 56K Dialup from $16.00/mo at Pop4 Networking
Online Campground Directoryhttp://www.camping-usa.com
Online Giftshop Superstorehttp://www.cloudninegifts.com
==
Re: [HACKERS] beta5 and unixware 711
* Olivier PRENANT [EMAIL PROTECTED] [010227 13:30]: Hi all, I've been trying to play with beta5 today on unixware 711. I have 2 problems: 1) enabling --with-tcl yields to link errors on bin/pgtclsh and interfaces/pl/tcl because Makefile insists on linking with libtcl7.6.0 instead on libtcl7.6 With the Skunkware packages installed, it's fine here. 2) enabling --with-openssl causes a compilation error on src/backend/libpq/crypt.c because of multiply defined symbol des_encrypt This is an OpenSSL error. We need to get THEIR attention LER Regards, -- Olivier PRENANT Tel:+33-5-61-50-97-00 (Work) Quartier d'Harraud Turrou +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: [EMAIL PROTECTED] -- Make your life a dream, make your dream a reality. (St Exupery) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Re: [HACKERS] regression.out and regression.diff
Vince Vielhaber writes: It was a snapshot - no idea what kind of shape it was in and I don't recall how I did the check, but it may have been installcheck. Even snapshots should work all the time. If not, it should be reported, not posted to a web page. Either way my question still needs to be answered. If there are no errors, is regression.out and regression.diff deleted? I can see the diff file being empty, but deleted? The diff file always exists; if all tests passed, it's empty. The .out file always contains a copy of what you see on the screen (i.e., the ok/FAIL). Which one of these two you want depends on what you are trying to achieve, but the .out file is probably less necessary, because if there are failed tests, you will see in the diff file. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
Re: [HACKERS] regression.out and regression.diff
On Tue, 27 Feb 2001, Vince Vielhaber wrote:
On Tue, 27 Feb 2001, Peter Eisentraut wrote:
Vince Vielhaber writes:
is regression.out and/or regression.diff deleted if the tests pass? I've
never seen all tests pass so I don't know but I just had someone tell me
that it does.
We've come to a point where all tests should pass all the time on all
supported platforms. If they don't, that's a bug and should be posted to
a mailing list.
The test output should come from 'gmake check', not the test against an
already installed server ('installcheck').
It was a snapshot - no idea what kind of shape it was in and I don't
recall how I did the check, but it may have been installcheck.
Make that "must have been installcheck" 'cuze I just reran it with
gmake check and it passed all but the ignored random test.
Vince.
--
==
Vince Vielhaber -- KA8CSHemail: [EMAIL PROTECTED]http://www.pop4.net
128K ISDN from $22.00/mo - 56K Dialup from $16.00/mo at Pop4 Networking
Online Campground Directoryhttp://www.camping-usa.com
Online Giftshop Superstorehttp://www.cloudninegifts.com
==
Re: [HACKERS] beta5 and unixware 711
On Tue, 27 Feb 2001, Larry Rosenman wrote: * Olivier PRENANT [EMAIL PROTECTED] [010227 13:30]: Hi all, I've been trying to play with beta5 today on unixware 711. I have 2 problems: 1) enabling --with-tcl yields to link errors on bin/pgtclsh and interfaces/pl/tcl because Makefile insists on linking with libtcl7.6.0 instead on libtcl7.6 With the Skunkware packages installed, it's fine here. You mean V802?? I had the same problems... 2) enabling --with-openssl causes a compilation error on src/backend/libpq/crypt.c because of multiply defined symbol des_encrypt This is an OpenSSL error. We need to get THEIR attention In the mean time, is there something I can do (apart from disbling ssl)? Regards, LER Regards, -- Olivier PRENANT Tel:+33-5-61-50-97-00 (Work) Quartier d'Harraud Turrou +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: [EMAIL PROTECTED] -- Make your life a dream, make your dream a reality. (St Exupery) -- Olivier PRENANT Tel:+33-5-61-50-97-00 (Work) Quartier d'Harraud Turrou +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: [EMAIL PROTECTED] -- Make your life a dream, make your dream a reality. (St Exupery)
Re: [HACKERS] beta5 and unixware 711
* Olivier PRENANT [EMAIL PROTECTED] [010227 15:00]: On Tue, 27 Feb 2001, Larry Rosenman wrote: * Olivier PRENANT [EMAIL PROTECTED] [010227 13:30]: Hi all, I've been trying to play with beta5 today on unixware 711. I have 2 problems: 1) enabling --with-tcl yields to link errors on bin/pgtclsh and interfaces/pl/tcl because Makefile insists on linking with libtcl7.6.0 instead on libtcl7.6 With the Skunkware packages installed, it's fine here. You mean V802?? I had the same problems... lerami% ldd ~postgres/bin/pgtclsh /usr/local/pgsql/bin/pgtclsh needs: libpgtcl.so.2 = /usr/local/pgsql/lib/libpgtcl.so.2 libpq.so.2 = /usr/local/pgsql/lib/libpq.so.2 libtcl8.2.so = /usr/local/lib/libtcl8.2.so /usr/lib/libdl.so.1 /usr/lib/libsocket.so.2 /usr/lib/libm.so.1 /usr/local/lib/libz.so.1 /usr/lib/libresolv.so.2 /usr/lib/libnsl.so.1 /usr/local/lib/libreadline.so.3 /usr/lib/libc.so.1 lerami% Here is my config input: CC=cc CXX=CC ./configure --prefix=/usr/local/pgsql --enable-syslog \ --with-CXX --with-perl --enable-multibyte --enable-cassert \ --with-includes=/usr/local/include --with-libs=/usr/local/lib \ --with-tcl --with-tclconfig=/usr/local/lib \ --with-tkconfig=/usr/local/lib --enable-locale --with-python 2) enabling --with-openssl causes a compilation error on src/backend/libpq/crypt.c because of multiply defined symbol des_encrypt This is an OpenSSL error. We need to get THEIR attention In the mean time, is there something I can do (apart from disbling ssl)? I would remove the des_encrypt from openSSL's des.h. Let me know if that works (I haven't tried). LER Regards, LER Regards, -- Olivier PRENANT Tel:+33-5-61-50-97-00 (Work) Quartier d'Harraud Turrou +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: [EMAIL PROTECTED] -- Make your life a dream, make your dream a reality. (St Exupery) -- Olivier PRENANT Tel:+33-5-61-50-97-00 (Work) Quartier d'Harraud Turrou +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: [EMAIL PROTECTED] -- Make your life a dream, make your dream a reality. (St Exupery) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
[HACKERS] Database Internals Presentation
I have completed a database internals presentation. The PDF is at: http://candle.pha.pa.us/main/writings/internals.pdf I am interested in any comments. I need to add text to it. FYI, you will find a system catalog chart in the presentation. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] regression.out and regression.diff
Peter Eisentraut [EMAIL PROTECTED] writes: Either way my question still needs to be answered. If there are no errors, is regression.out and regression.diff deleted? I can see the diff file being empty, but deleted? The diff file always exists; if all tests passed, it's empty. Forgot your own code already, Peter? if [ -s "$diff_file" ]; then echo "The differences that caused some tests to fail can be viewed in the" echo "file \`$diff_file'. A copy of the test summary that you see" echo "above is saved in the file \`$result_summary_file'." echo else rm -f "$diff_file" "$result_summary_file" fi regards, tom lane
Re: [HACKERS] regression.out and regression.diff
Vince Vielhaber [EMAIL PROTECTED] writes:
The test output should come from 'gmake check', not the test against an
already installed server ('installcheck').
It was a snapshot - no idea what kind of shape it was in and I don't
recall how I did the check, but it may have been installcheck.
Make that "must have been installcheck" 'cuze I just reran it with
gmake check and it passed all but the ignored random test.
I have no idea why Peter thinks 'make installcheck' should be less
reliable than 'make check'. If installcheck fails for you, let's
see that too.
regards, tom lane
[HACKERS] Re: mmap for zeroing WAL log
Matthew Kirkwood [EMAIL PROTECTED] writes: I had assumed that the overhead would come from synchronous metadata incurring writes of at least the inode, block bitmap and probably an indirect block for each syscall. No Unix that I've ever heard of forces metadata to disk after each "write" call; anyone who tried it would have abysmal performance. That's what fsync and the syncer daemon are for. regards, tom lane
[HACKERS] Re: mmap for zeroing WAL log
On Sat, 24 Feb 2001, Tom Lane wrote: I am confused why mmap() is better than writing to a real file. It isn't, except that it allows to initialise the logfile in one syscall, without first allocating and zeroing (and hence dirtying) 16Mb of memory. Uh, the existing code does not zero 16Mb of memory... it zeroes 8K and then writes that block repeatedly. See the "one syscall" bit above. It's possible that the overhead of a syscall for each 8K block is significant, I had assumed that the overhead would come from synchronous metadata incurring writes of at least the inode, block bitmap and probably an indirect block for each syscall. but on the other hand writing a block at a time is a heavily used and heavily optimized path in all Unixen. It's at least as plausible that the mmap-as-source-of-zeroes path will be slower! Results: On Linux/ext2, it appears good for a gain of 3-5% for log creations (via a fairly minimal test program). On FreeBSD 4.1-RELEASE/ffs (with all of sync/async/softupdates) it is a couple of percent worse in elapsed time, but consumes around a third more system CPU time (12sec vs 9sec on one test system). I am awaiting numbers from reiserfs but, for now, it looks like I am far from vindicated. Matthew.
Re: [ODBC] Re: [HACKERS] Release in 2 weeks ...
Tom Lane wrote: Emmanuel Charpentier [EMAIL PROTECTED] writes: Yep ! As of beta4, the ODBC driver is still seriously broken (the original libpsqlodbc.so.0.26 doesn't even connect. A version patched by Nick Gorham allows some connectivity (you can query the DB), but still has some serious breakage (i. e. no "obvious" ways to see views from StarOffice or MS-Access)). I think I've fixed this bug at least for MS-Access. You could get the latest win32 driver from ftp://ftp.greatbridge.org/pub/pgadmin/stable/psqlodbc.zip . Please try it. However I'm not sure about unixODBC. Regards, Hiroshi Inoue
Re: [HACKERS] Re: [ADMIN] COPY doesn't works when containing ' 'or ' ' characters on db
"Oliver Elphick" [EMAIL PROTECTED] writes: I think this happens when the front-end encoding is SQL_ASCII and the database is using UNICODE. Then, there are misunderstandings between front-end and back-end, so that a single character with the eighth bit set may be sent by the front-end and interpreted by the back-end as the first half of a UNICODE two-byte character. I wondered about that, but his examples had one or more characters between the eighth-bit-set character and the '|', so this doesn't seem to explain the problem. No. From Jaume's example: SELECT edicion FROM products; edicion - España|Nacional ---puts on the same cell either there's an '|' in the middle!!! \361 == 0xf1. UTF-8 assumes that: if (the first byte) 0xe0 == 0xe0, then the letter consists of 3 bytes. So PostgreSQL believes that "ña|" is one UTF-8 letter and eat up '|'. My guess is Jaume made an UNICODE database but provided it ISO 8859-1 or that kind of single-byte latin encoding data. I'm wondering why so many people are using UTF-8 database even he does not understand what UTF-8 is:-) I hope 7.1 would solve this kind of confusion by enabling an automatic encoding conversion between UTF-8 and others. -- Tatsuo Ishii
[HACKERS] RE: [BUGS] Problem with 7.0.3 dump - 7.1b4 restore
Attached is a single INSERT that shows the problem. The character after the word "Fiber" truncates the text when using JDBC. NOTE, the text IS in the database, that is, the dump/restore seems ok, the problem is when trying to read the text later. The database is UTF8 and I just tested with beta 5. Oh, BTW, if I try to set (INSERT) this same character via JDBC and then retreive it again then everything is fine. Thanks. I'll dig into it. -- Tatsuo Ishii
[HACKERS] RE: [BUGS] Problem with 7.0.3 dump - 7.1b4 restore
Attached is a single INSERT that shows the problem. The character after the word "Fiber" truncates the text when using JDBC. NOTE, the text IS in the database, that is, the dump/restore seems ok, the problem is when trying to read the text later. The database is UTF8 and I just tested with beta 5. Oh, BTW, if I try to set (INSERT) this same character via JDBC and then retreive it again then everything is fine. I have tested your data using psql: unicode=# create table pr_prop_info(i1 int, i2 int, i3 int, t text); CREATE unicode=# \encoding LATIN1 unicode=# \i example.sql INSERT 2378114 1 unicode=# select * from pr_prop_info; The character after the word "Fiber" looks like "Optic Cable". So as long as the server/client encoding set correctly, it looks ok. I guess we have some problems with JDBC driver. Unfortunately I am not a Java guru at all. Can anyone look into our JDBC driver regarding this problem? -- Tatsuo Ishii
[HACKERS] Re: Release in 2 weeks ...
Are there any major outstandings that ppl have on their plates, that should prevent a release? I'd like to put out an RC1 by Friday this week, with a full release schedualed for March 15th ... this would give Thomas his two weeks for the docs freeze ... I'm interested to know what exactly takes two weeks with the docs and what could be done to speed it up. The "official" version of the story is that it takes ~10-20 hours for me to work through the docs to format them for hardcopy with ApplixWare, primarily because something in the jade RTF tickles a bug in the page formatting with Applix. (This round, I'll resort even to M$Word to avoid that time sink, since I just don't have the time.) The reality is that it is a two week quiet time for us to get the last bugs out and to get the last platform-specific reports. At this moment we have not started the "report now or risk having a broken platform" threats that help iron out the last problems. Scrappy has proposed that we start that period now. Were the concerns about WAL etc enough to hold off on that, or are we counting down from now? - Thomas
[HACKERS] Re: Query precompilation?
Mario Weilguni wrote: (...) I don't care about integrity etc! You should !-) You can find some valueable tips in the documentation: http://www.de.postgresql.org/users-lounge/docs/7.0/user/c4929.htm In the docs there is this paragraph: Disable Auto-commit Turn off auto-commit and just do one commit at the end. Otherwise Postgres is doing a lot of work for each record added. In general when you are doing bulk inserts, you want to turn off some of the database features to gain speed. This sounds nice, but I've read a lot of postgres documents and still do not know how to disable autocommit. Is this possible? And how? At the moment, use a BEGIN/COMMIT block around a set of insert statements. Someday we'll likely have an explicit command to affect the behavior. - Thomas
Re: [ODBC] Re: [HACKERS] Release in 2 weeks ...
I have a copy of OpenOffice for LinuxPPC but have not figured out how to tell it to connect to Postgres. If someone can slip me a clue on how to configure it and do simple database stuff with it, I'll try to clean up the most pressing ODBC problems before we release. I've got a clue for ApplixWare, if you happen to have that package (US$90). - Thomas
[HACKERS] BLCKSZ 0?
Any suggestion why postmaster/postgres would think it had been compiled with BLCKSZ 0? : root@blue:/usr/local/pgsql# su postgres -c "bin/postmaster -D /usr/local/pgsql/data " DEBUG: Data Base System is starting up at Tue Feb 27 22:31:51 2001 FATAL 2: database was initialized with BLCKSZ 0, but the backend was compiled with BLCKSZ 8192. looks like you need to initdb. FATAL 2: database was initialized with BLCKSZ 0, but the backend was compiled with BLCKSZ 8192. looks like you need to initdb. Startup failed - abort -- Dominic J. Eidson "Baruk Khazad! Khazad ai-menu!" - Gimli --- http://www.the-infinite.org/ http://www.the-infinite.org/~dominic/
Re: [HACKERS] Re: Release in 2 weeks ...
Thomas Lockhart [EMAIL PROTECTED] writes: I'm interested to know what exactly takes two weeks with the docs and what could be done to speed it up. The "official" version of the story is that it takes ~10-20 hours for me to work through the docs to format them for hardcopy with ApplixWare, primarily because something in the jade RTF tickles a bug in the page formatting with Applix. I'm sure anything that could be done to eliminate this formatting make-work would be just fine with Thomas ;-). However, it probably wouldn't really change the release scheduling much, since as he points out it's partially an excuse for clamping down: The reality is that it is a two week quiet time for us to get the last bugs out and to get the last platform-specific reports. In short, now is our "okay people, let's get *serious*" phase. No features, no trivial stuff, just get the critical bugs out. Scrappy has proposed that we start that period now. Were the concerns about WAL etc enough to hold off on that, or are we counting down from now? I'm pretty concerned about WAL, but have no good reason not to start the release countdown. regards, tom lane
[GENERAL] Re: [HACKERS] BLCKSZ 0?
"Dominic J. Eidson" [EMAIL PROTECTED] writes: Any suggestion why postmaster/postgres would think it had been compiled with BLCKSZ 0? : root@blue:/usr/local/pgsql# su postgres -c "bin/postmaster -D /usr/local/pgsql/data " DEBUG: Data Base System is starting up at Tue Feb 27 22:31:51 2001 FATAL 2: database was initialized with BLCKSZ 0, but the backend was compiled with BLCKSZ 8192. looks like you need to initdb. Read that again --- it did *not* say it was compiled with BLCKSZ 0. It said (or meant, anyway) it found zero in the pg_control field that indicates the BLCKSZ in use in the database. Something's broken with your pg_control file ... care to give more details? regards, tom lane
Re: [ODBC] Re: [HACKERS] Release in 2 weeks ...
Thomas Lockhart wrote: I have a copy of OpenOffice for LinuxPPC but have not figured out how to tell it to connect to Postgres. If someone can slip me a clue on how to configure it and do simple database stuff with it, I'll try to clean up the most pressing ODBC problems before we release. I've got a clue for ApplixWare, if you happen to have that package (US$90). Please post it, Thomas. I got nowhere following their instructions. -- Oliver Elphick[EMAIL PROTECTED] Isle of Wight http://www.lfix.co.uk/oliver PGP: 1024R/32B8FAA1: 97 EA 1D 47 72 3F 28 47 6B 7E 39 CC 56 E4 C1 47 GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C "These things have I written unto you that believe on the name of the Son of God; that ye may know that ye have eternal life, and that ye may believe on the name of the Son of God."I John 5:13
Re: [ODBC] Re: [HACKERS] Release in 2 weeks ...
I've got a clue for ApplixWare, if you happen to have that package (US$90). Please post it, Thomas. I got nowhere following their instructions. Uh, who's instructions? We have a writeup on Applix and ODBC in the docs. Have you found those, or are those falling short of helpful? - Thomas
