Re: [Pgcluster-general] pgcluster patch for postgresql 9.0
hi johnson! how to build? 1. download postgresql-9.0.2 source package 2. extract postgresql-9.0.2 source package 3. mv postgresql-9.0.2 to pgcluster 4. cd pgcluster 5. patch -p 1 (path_to_patch_file)/postgresql-9.0.2-to-pgcluster-9.0.2.1.patch 6. patch -p 0 -f (path_to_patch_file)/pgcluster-9.0.2.1-to-pgcluster-9.0.2.2.patch 7. mkdir build (or whatever you want) 8. cd build (or the directory you created before) 9. ../configure make maintainer-clean 10. ../configure make this works fine for me! if you still have problems please post the complete compile output! regards, markus Am Montag, den 14.02.2011, 12:49 +0800 schrieb johnson: I patch postgresql src with your patch postgresql-9.0.2-to-pgcluster-9.0.2.1.patch then configure --enable-replication-loadbalance then make it show the error /tool/postgresql-9.0.2/obj/../src/pgcluster/pglb/main.c:120: error Log_error_verbosity /tool/postgresql-9.0.2/obj/../src/include/utils/elog.h: Log_error_verbosity I fix it and it show another error... Thanks for your reply quickly, but I don't know how to use this new patch, and I do not fine pgcluster.creator.user the path file refered. Best regards Johnson 於 2011/2/11 上午 09:42, johnson 提到: Hi! I've tried it, and I found there are many errors when compile with pglb enable. Best Regards Johnson ___ Pgcluster-general mailing list Pgcluster-general@pgfoundry.org http://pgfoundry.org/mailman/listinfo/pgcluster-general ___ Pgcluster-general mailing list Pgcluster-general@pgfoundry.org http://pgfoundry.org/mailman/listinfo/pgcluster-general ___ Pgcluster-general mailing list Pgcluster-general@pgfoundry.org http://pgfoundry.org/mailman/listinfo/pgcluster-general
Re: [Pgcluster-general] pgcluster patch for postgresql 9.0
Hi! I've tried it, and I found there are many errors when compile with pglb enable. Best Regards Johnson ___ Pgcluster-general mailing list Pgcluster-general@pgfoundry.org http://pgfoundry.org/mailman/listinfo/pgcluster-general
Re: [Pgcluster-general] pgcluster patch
many open files LOCATION: replicate_main, /root/pgcluster/obj/src/../../src/pgcluster/pgrp/main.c:685 ERROR: Failed to connect to 1.1.1.14:2000 (error=0/could not create socket: Too many open files ) LOCATION: PGRcreateConn, /root/pgcluster/obj/src/../../src/pgcluster/pgrp/replicate.c:296 ERROR: [1.1.1.12:2000] may be down LOCATION: setTransactionTbl, /root/pgcluster/obj/src/../../src/pgcluster/pgrp/replicate.c:457 ERROR: accept failed: Too many open files LOCATION: replicate_main, /root/pgcluster/obj/src/../../src/pgcluster/pgrp/main.c:685 ERROR: accept failed: Too many open files LOCATION: replicate_main, /root/pgcluster/obj/src/../../src/pgcluster/pgrp/main.c:685 # cut here # LOCATION: setTransactionTbl, /root/pgcluster/obj/src/../../src/pgcluster/pgrp/replicate.c:457 ERROR: Failed to connect to 1.1.1.14:2000 (error=0/could not create socket: Too many open files ) LOCATION: PGRcreateConn, /root/pgcluster/obj/src/../../src/pgcluster/pgrp/replicate.c:296 # cut here # LOCATION: setTransactionTbl, /root/pgcluster/obj/src/../../src/pgcluster/pgrp/replicate.c:457 ERROR: [1.1.1.12:2000] may be down LOCATION: setTransactionTbl, /root/pgcluster/obj/src/../../src/pgcluster/pgrp/replicate.c:457 ERROR: [1.1.1.11:2000] may be down LOCATION: setTransactionTbl, /root/pgcluster/obj/src/../../src/pgcluster/pgrp/replicate.c:457 ERROR: Failed to connect to 1.1.1.14:2000 (error=0/could not create socket: Too many open files ) LOCATION: PGRcreateConn, /root/pgcluster/obj/src/../../src/pgcluster/pgrp/replicate.c:296 ERROR: Failed to connect to 1.1.1.12:2000 (error=0/FATAL: database user= does not exist ) LOCATION: PGRcreateConn, /root/pgcluster/obj/src/../../src/pgcluster/pgrp/replicate.c:296 # cut here # What can I do? Thanks. Best wishes, -Original Message- From: pgcluster-general-boun...@pgfoundry.org [mailto:pgcluster-general-boun...@pgfoundry.org] On Behalf Of Markus Schmidleitner Sent: 2010年6月8日 21:33 To: pgcluster-general@pgfoundry.org Cc: Alexander Raudner Subject: [Pgcluster-general] pgcluster patch hi list, hi mitani! @all here's another patch based on the patches i sent in before. the current patch is against postgresql 8.3.11. i hope anyone will give it a try. comments/suggestions are highly appreciated! @mitani maybe you can do some code review/testing on it? would be nice ;-) i also attached a small script and config files which i use for regression testing. you may need to change some paths etc. how to build? 1. download postgresql-8.3.11 source package 2. extract postgresql-8.3.11 source package 3. mv postgresql-8.3.11 to pgcluster 4. cd pgcluster 5. patch -p1 (path_to_patch_file)/postgresql-8.3.11-to-pgcluster-1.9.15.3.patch 7. mkdir obj 8. cd obj 9. ../configure make maintainer-clean 10. ../configure make ... what's new? - fixed bugs in cold recovery (works again now) - fixed bugs in lifecheck process shutdown (was running forever) - fixed bugs in pglb (compiles again now) - some other improvements/fixes/changes changes from earlier patches: - implemented sequence replication (numbers are generated by replication-server. NOTE: generating sequence numbers while a copy operation is in progress is currently not supported!) - reworked copy operation - reworked replication (now the work is done locally first and then replicated - so queries which will fail anyway don't get replicated any more) - reworked hot-recovery - reworked PREPAREd statement handling - include SAVEPOINT handling - reject queries on cluster if it is used for recovery - fixed accept bug in pgreplicate - fixed bugs in pgc_compare - handle errors from remote host's so that query will fail on source too - treat processes lifecheck, recovery provider and recovery as auxilary processes (autorestart, shutdown, ...) - start lifecheck, recovery provider ... only if not in standalone mode - implemented cluster connect/shutdown message - made regression tests working again (in replication mode) - default regression tests (make test) now run in standalone-mode omitting all replication code - implemented cluster-compare utility (pgc_compare) - reworked replication of copy commands - fixed bug in scan_terminate - fixed some other bugs/issues that prevented regression tests to run sucessfully - usage of transactions in conjunction with AutoLockTable now works - some refactorings and code cleanups - initdb now works as expected (failed in previous patches) - added postgresql startup parameter '-a' to set the cluster in full standalone mode, bypassing all replication operations (mainly for regression testing and an operational mode which we need here) - added a few configuration parameters (see current cluster.conf and pgreplicate.conf) - now you can specify a comma separated list of hostnames or addresses in Host_Name-Parameter (currently this is limited to a set of 2 - see cluster.conf and pgreplicate.conf
Re: [Pgcluster-general] pgcluster patch
hi markus! thank you for your answer. now i have an other problem with pgcluster build with your patch: if i run a db and start an other in cold recovery it do not work as usual. /usr/local/pgsql/bin/pg_ctl -D /usr/local/pgsql/data -o -i -R start server starting LOG: Start in recovery mode! LOG: Please wait until synchronization with Master DB finished ... LOG: find replication server for recovery LOG: start recovery LOG: 1st recovery step of [global] directory... LOG: OK LOG: 1st recovery step of [base] directory... LOG: OK LOG: 1st recovery step of [pg_clog] directory... LOG: OK LOG: 1st recovery step of [pg_xlog] directory... LOG: OK LOG: ERROR: in use for recovery LOG: 1st sync_table_space LOG: FAILED FATAL: Synchronization with Master DB failed (cold recovery). now when i start it without the -R option: server starting LOG: database system was interrupted; last known up at 2010-06-02 11:39:12 CEST LOG: database system was not properly shut down; automatic recovery in progress LOG: recovery provider started (pid 17961) LOG: record with zero length at 0/4CB908 LOG: redo is not required LOG: autovacuum launcher started LOG: database system is ready to accept connections and the recovery is done as i can see on the dbs can you imagine what is going wrong in that case? i do not really understand why i get LOG: ERROR: in use for recovery regards Am Dienstag, den 01.06.2010, 15:52 +0200 schrieb Markus Schmidleitner: hi alexander! edit src/Makefile.global.in, uncomment the line with ENABLE_LOADBALANCE ... and rebuild pgcluster (clean, configure, ...). i have to mention that i didn't test the loadbalance feature because we have our own loadbalancing solution here and don't use pg_lb. so there are likely some problems related to this feature. regards markus Am Dienstag, den 01.06.2010, 15:00 +0200 schrieb Alexander Raudner: hi! @markus i have tied to build pgcluster with your newest patch. so far it works, postmaster and pg_replicate are made, but i can't find any pg_lb in the bin-folder. i build it on an 64-bit debian lenny exactly the way you describe. can you imagine what's going wrong? regards ___ Pgcluster-general mailing list Pgcluster-general@pgfoundry.org http://pgfoundry.org/mailman/listinfo/pgcluster-general ___ Pgcluster-general mailing list Pgcluster-general@pgfoundry.org http://pgfoundry.org/mailman/listinfo/pgcluster-general
[Pgcluster-general] pgcluster patch
hi! @markus i have tied to build pgcluster with your newest patch. so far it works, postmaster and pg_replicate are made, but i can't find any pg_lb in the bin-folder. i build it on an 64-bit debian lenny exactly the way you describe. can you imagine what's going wrong? regards ___ Pgcluster-general mailing list Pgcluster-general@pgfoundry.org http://pgfoundry.org/mailman/listinfo/pgcluster-general
Re: [Pgcluster-general] pgcluster patch
You might want to, considering the work you are putting into pgcluster ... the list revolves around the various clustering techs, and hooks/APIs in core to better integrate them ... not *hugely* active list, so you won't be inundated with emails as a result ... http://archives.postgresql.org/pgsql-cluster-hackers/ On Thu, 8 Apr 2010, Markus Schmidleitner wrote: marc, unfortunately i'm not involved with / on the pgsql-hackers-cluster mailing list! why do you ask? Am Mittwoch, den 07.04.2010, 12:22 -0300 schrieb Marc G. Fournier: Markus, are you involved with / on the pgsql-hackers-cluster mailing list? On Wed, 7 Apr 2010, Markus Schmidleitner wrote: hi list, hi mitani! @all here's another patch based on the patches i sent in before. the current patch is against postgresql 8.3.10. i hope anyone will give it a try. comments/suggestions are highly appreciated! @mitani maybe you can do some code review/testing on it? would be nice ;-) i also attached a small script and config files which i use for regression testing. you may need to change some paths etc. how to build? 1. download postgresql-8.3.10.tar.gz 2. extract postgresql-8.3.10.tar.gz 3. mv postgresql-8.3.10 to pgcluster 4. cd pgcluster 5. patch -p1 (path_to_patch_file)/postgresql-8.3.10-to-pgcluster-1.9.14.patch 7. mkdir obj 8. cd obj 9. ../configure make maintainer-clean 10. ../configure make ... what's new? - reworked replication (now the work is done locally first and then replicated - so queries which will fail anyway don't get replicated any more) - reworked hot-recovery - reworked PREPAREd statement handling - include SAVEPOINT handling - reject queries on cluster if it is used for recovery - fixed accept bug in pgreplicate - fixed bugs in pgc_compare - handle errors from remote host's so that query will fail on source too - treat processes lifecheck, recovery provider and recovery as auxilary processes (autorestart, shutdown, ...) - start lifecheck, recovery provider ... only if not in standalone mode - implemented cluster connect/shutdown message - some other improvements/fixes/changes changes from earlier patches: - made regression tests working again (in replication mode) - default regression tests (make test) now run in standalone-mode omitting all replication code - implemented cluster-compare utility (pgc_compare) - reworked replication of copy commands - fixed bug in scan_terminate - fixed some other bugs/issues that prevented regression tests to run sucessfully - usage of transactions in conjunction with AutoLockTable now works - some refactorings and code cleanups - initdb now works as expected (failed in previous patches) - added postgresql startup parameter '-a' to set the cluster in full standalone mode, bypassing all replication operations (mainly for regression testing and an operational mode which we need here) - added a few configuration parameters (see current cluster.conf and pgreplicate.conf) - now you can specify a comma separated list of hostnames or addresses in Host_Name-Parameter (currently this is limited to a set of 2 - see cluster.conf and pgreplicate.conf) - pgreplicate and postgresql recovery processes now listen only on those interfaces that are specified - set default-value of auto_lock_table to false - reworked locking code and deadlock notification - some refinements on networking-code - use modified elog instead of own logging-code (mainly for logging in common code used by pgreplicate and postgresql - currently logging from pgreplicate to a logfile is not implemented) - some more general reworks, dead code eliminations ... - replication server will be noticed to all clusters - cluster will be noticed to all replication servers - standby replication server send cascade info to cluster * cluster will automatically switch to primary replication server - primary replication server requests replication servers from cluster * replication server will automatically setup cascade * clusters automatically switch back to primary replication server - added threading-support in pgreplicate (mainly backported from cybercluster) - code refactoring * implemented functions for non-blocking i/o which are used by entire pgcluster * all net-i/o operations are non-blocking * earse dead code * remove needless memcpy * other minor reworks - implemented a cluster-notice message when cascading is used * it informs the upper and lower cascade that cluster finished recovery and can be used now best regards, Markus Schmidleitner Marc G. FournierHub.Org Hosting Solutions S.A. scra...@hub.org http://www.hub.org Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org Marc G. FournierHub.Org Hosting Solutions S.A. scra...@hub.org http://www.hub.org Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org
Re: [Pgcluster-general] pgcluster patch
Okay, after a couple of false starts, I appear to have everything install and have bene working through the install doc ... seems very straightforward, and I think its all configured right ... start pgreplicate - check start cluster DB - check start pglb - check connect to pglb works fine, and I can list DBs,but as soon as I try and CREATE, I get: LOG: all replication servers down STATEMENT: create database test; WARNING: no replication server found LOG: all replication servers down STATEMENT: create database test; WARNING: no replication server found cluster.conf has: Replicate_Server_Info Host_Name192.0.2.23/Host_Name Port8001/Port Recovery_Port8101/Recovery_Port /Replicate_Server_Info And it appears to connect if I telnet to the port, although not sure what I should type for getting debug info: %telnet 192.0.2.23 8001 Trying 192.0.2.23... Connected to pgsql-lb. Escape character is '^]'. pgreplication.sts shows: Tue Feb 16 21:29:04 2010 host 192.0.2.24:5432 initialize Tue Feb 16 21:29:04 2010 host 192.0.2.25:5432 initialize Tue Feb 16 21:29:04 2010 cascade 192.0.2.23:8001 start use Tue Feb 16 21:29:10 2010 host 192.0.2.24:5432 start use Tue Feb 16 21:29:10 2010 host 192.0.2.25:5432 start use In my pgreplication.conf file, I have my replication server defined as: Host_Name192.0.2.23/Host_Name Replication_Port8001/Replication_Port Recovery_Port8101/Recovery_Port RLOG_Port8301/RLOG_Port Response_Modenormal/Response_Mode Use_Replication_Logyes/Use_Replication_Log Connect_Timeout3s/Connect_Timeout Login_Timeout5s/Connect_Timeout Replication_Timeout1min/Replication_Timeout Cascade_Timeout1min/Cascade_Timeout Recovery_Timeout10min/Recovery_Timeout LifeCheck_Timeout3s/LifeCheck_Timeout LifeCheck_Interval15s/LifeCheck_Interval So ... what am I missing? The above errors seem to all I can squeeze out, and even though I have 'Use_Replication_Log' set to yes, am not getting anything in /tmp/pgreplication.log (which is defined) ... On Wed, 21 Oct 2009, Markus Schmidleitner wrote: hi list, hi mitani! @all here's another patch based on the patches i sent in before. the current patch is against postgresql 8.3.7. i hope anyone will give it a try. comments/suggestions are highly appreciated! @mitani maybe you can do some code review/testing on it? would be nice ;-) i also attached a small script and config files which i use for regression testing. you may need to change some paths etc. how to build? 1. download postgresql-8.3.7.tar.gz 2. extract postgresql-8.3.7.tar.gz 3. mv postgresql-8.3.7 to pgcluster 4. cd pgcluster 5. patch -p1 (path_to_patch_file)/postgresql-8.3.7-to-pgcluster-1.9.12.patch 7. mkdir obj 8. cd obj 9. ../configure make maintainer-clean 10. ../configure make ... what's new? - made regression tests working again (in replication mode) - default regression tests (make test) now run in standalone-mode omitting all replication code - implemented cluster-compare utility (pgc_compare) - reworked replication of copy commands - fixed bug in scan_terminate - fixed some other bugs/issues that prevented regression tests to run sucessfully - usage of transactions in conjunction with AutoLockTable now works - some refactorings and code cleanups changes from earlier patches: - initdb now works as expected (failed in previous patches) - added postgresql startup parameter '-a' to set the cluster in full standalone mode, bypassing all replication operations (mainly for regression testing and an operational mode which we need here) - added a few configuration parameters (see current cluster.conf and pgreplicate.conf) - now you can specify a comma separated list of hostnames or addresses in Host_Name-Parameter (currently this is limited to a set of 2 - see cluster.conf and pgreplicate.conf) - pgreplicate and postgresql recovery processes now listen only on those interfaces that are specified - set default-value of auto_lock_table to false - reworked locking code and deadlock notification - some refinements on networking-code - use modified elog instead of own logging-code (mainly for logging in common code used by pgreplicate and postgresql - currently logging from pgreplicate to a logfile is not implemented) - some more general reworks, dead code eliminations ... - replication server will be noticed to all clusters - cluster will be noticed to all replication servers - standby replication server send cascade info to cluster * cluster will automatically switch to primary replication server - primary replication server requests replication servers from cluster * replication server will automatically setup cascade * clusters automatically switch back to primary replication server - added threading-support in pgreplicate (mainly backported from cybercluster) - code refactoring * implemented functions for non-blocking i/o which are used by entire pgcluster * all net-i/o operations are non-blocking * earse dead code * remove needless
Re: [Pgcluster-general] pgcluster patch
Hi Markus, Thank you for sending nice patch. I'd like to check it during this holiday. This week has the longest holiday season in Japan. Five days! (And it's including Sat. and Sun.) Oh my gawd, should I back to The Netherlands ? Regards, --- At.Mitani On Thu, 30 Apr 2009 17:00:00 +0200 Markus Schmidleitner markus.schmidleit...@ocilion.com wrote: hello list, hello mitani! @mitani great, you're back!! thank's a lot for this awsome piece of software - great work - keep it up! @all maybe someone recognized the 'cybercluster-backport and more'-patch and the 'replication-server-failover'-patch i sent in a while ago. here's another patch-set based on the 2 patches before. the current patch-set is against postgresql 8.3.7. i hope anyone will give it a try. comments/suggestions are highly appreciated! @mitani maybe you can do some code review/testing on it? would be nice ;-) changes regarding to previous patches: - initdb now works as expected (failed in previous patches) - added postgresql startup parameter '-a' to set the cluster in full standalone mode, bypassing all replication operations (mainly for regression testing and an operational mode which we need here) - added a few configuration parameters (see current cluster.conf and pgreplicate.conf) - now you can specify a comma separated list of hostnames or addresses in Host_Name-Parameter (currently this is limited to a set of 2 - see cluster.conf and pgreplicate.conf) - pgreplicate and postgresql recovery processes now listen only on those interfaces that are specified - set default-value of auto_lock_table to false - reworked locking code and deadlock notification - some refinements on networking-code - use modified elog instead of own logging-code (mainly for logging in common code used by pgreplicate and postgresql - currently logging from pgreplicate to a logfile is not implemented) - some more general reworks, dead code eliminations ... changes from earlier patches: - replication server will be noticed to all clusters - cluster will be noticed to all replication servers - standby replication server send cascade info to cluster * cluster will automatically switch to primary replication server - primary replication server requests replication servers from cluster * replication server will automatically setup cascade * clusters automatically switch back to primary replication server - added threading-support in pgreplicate (mainly backported from cybercluster) - code refactoring * implemented functions for non-blocking i/o which are used by entire pgcluster * all net-i/o operations are non-blocking * earse dead code * remove needless memcpy * other minor reworks - implemented a cluster-notice message when cascading is used * it informs the upper and lower cascade that cluster finished recovery and can be used now best regards, Markus Schmidleitner __ Information from ESET NOD32 Antivirus, version of virus signature database 3784 (20090121) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com -- mitani mit...@sraw.co.jp ___ Pgcluster-general mailing list Pgcluster-general@pgfoundry.org http://pgfoundry.org/mailman/listinfo/pgcluster-general
