Le 05/25/2011 03:52 PM, Tatsuo Ishii a écrit :
>>> We are about to release pgpool-II 3.0.4, the lastest version of
>>> pgpool-II 3.0 stable tree. Currently release date is scheduled on May
>>> 30, Monday.
>>>
>>> If you would like add fixes to this release please propose.
>>
>> See "Detaching, then attaching node results in weird status". I'm still
>> waiting for your answer on this one.
>
> I have been looking into it. Here are log when pcp_detach_node executed:
>
> 2011-05-25 22:39:37 LOG: pid 17063: starting degeneration. shutdown host
> /tmp(5433)
> 2011-05-25 22:39:37 ERROR: pid 17063: failover_handler: no valid DB node found
> 2011-05-25 22:39:37 LOG: pid 17063: Restart all children
> 2011-05-25 22:39:37 LOG: pid 17063: execute command:
> /usr/local/etc/failover.sh 0 "/tmp" 5433 /usr/local/pgsql/data -1 0 "" 0
> 2011-05-25 22:39:37 LOG: pid 17063: find_primary_node_repeatedly: waiting
> for finding a primary node
>
> The log stopped here while. After 60 seconds, you see below.
>
> 2011-05-25 22:40:37 LOG: pid 17063: failover: set new primary node: -1
> 2011-05-25 22:40:37 LOG: pid 17694: do_child: failback event found. restart
> myself.
> 2011-05-25 22:40:37 LOG: pid 17695: do_child: failback event found. restart
> myself.
> 2011-05-25 22:40:37 LOG: pid 17696: do_child: failback event found. restart
> myself.
> 2011-05-25 22:40:37 LOG: pid 17063: failover done. shutdown host /tmp(5433)
>
> If I do not wait for 60 seconds and execute pcp_attach_node, the
> problem happens. However if I wait for 60 seconds and see "failover
> done." message, the problem does not occur.
>
> So it seems we need to wait for find_primary_node_repeatedly finish
> before we issue pcp_attach_node. This suggests that your fix might not
> be appropreate because your fix does not corresponds to this "timing"
> issue.
>
> I'm going to keep on looking into this...
That's actually not the issue I'm talking about. I'm in V3.0, with a
single backend, no failover script. See my config in attachment.
When I do the pcp_detach_node, I have this:
2011-05-25 20:24:12 LOG: pid 31861: notice_backend_error: 0 fail over
request from pid 31861
2011-05-25 20:24:12 LOG: pid 31828: starting degeneration. shutdown
host localhost(5432)
2011-05-25 20:24:12 ERROR: pid 31828: failover_handler: no valid DB node
found
2011-05-25 20:24:12 LOG: pid 31828: failover done. shutdown host
localhost(5432)
Which seems fine to me. Then I do the pcp_attach_node, and I got this:
2011-05-25 20:25:23 LOG: pid 31861: send_failback_request: fail back 0
th node request from pid 31861
2011-05-25 20:25:23 ERROR: pid 31861: send_failback_request: node 0 is
alive.
I was mistaken on the "node 0 is alive" message. I thought it means that
node 0 is NOW up. What it really means is that pgpool thought it was
ALREADY alive (hence the ERROR message level on the
send_failback_request function). Digging harder on this issue, I finally
found that the VALID_BACKEND macro returns true when it should return
false. Actually, there is already this comment in get_next_master_node():
/*
* Do not use VALID_BACKEND macro in raw mode.
* VALID_BACKEND return true only if the argument is master
* node id. In other words, standby nodes are false. So need
* to check backend status without VALID_BACKEND.
*/
And I'm actually in raw mode. VALID_BACKEND is used so much it would be
really dangerous to change it. So, I'm not sure what we really should do
here. I've got a patch that fixes my issue cleanly, not sure it's the
best way to do this. See the patch in attachment.
BTW, when I do a pcp_attach_node, I have the status 2, but it didn't
check if there was a PostgreSQL backend available. Not sure we want to
do something on this too. Why doesn't it check if the backend is
available? it doesn't do at startup too. I find this really weird, but
I'm sure there is a reason.
--
Guillaume
http://www.postgresql.fr
http://dalibo.com
#
# pgpool-II configuration file sample
# $Header: /cvsroot/pgpool/pgpool-II/pgpool.conf.sample,v 1.39.2.2 2011/02/22
06:36:49 kitagawa Exp $
# Host name or IP address to listen on: '*' for all, '' for no TCP/IP
# connections
listen_addresses = 'localhost'
# Port number for pgpool
port = 9999
# Port number for pgpool communication manager
pcp_port = 9898
# Unix domain socket path. (The Debian package defaults to
# /var/run/postgresql.)
socket_dir = '/tmp'
# Unix domain socket path for pgpool communication manager.
# (Debian package defaults to /var/run/postgresql)
pcp_socket_dir = '/tmp'
# Unix domain socket path for the backend. Debian package defaults to
/var/run/postgresql!
backend_socket_dir = '/tmp'
# pgpool communication manager timeout. 0 means no timeout. This parameter is
ignored now.
pcp_timeout = 10
# number of pre-forked child process
num_init_children = 32
# Number of connection pools allowed for a child process
max_pool = 4
# If idle for this many seconds, child exits. 0 means no timeout.
child_life_time = 300
# If idle for this many seconds, connection to PostgreSQL closes.
# 0 means no timeout.
connection_life_time = 0
# If child_max_connections connections were received, child exits.
# 0 means no exit.
child_max_connections = 0
# If client_idle_limit is n (n > 0), the client is forced to be
# disconnected whenever after n seconds idle (even inside an explicit
# transactions!)
# 0 means no disconnect.
client_idle_limit = 0
# Maximum time in seconds to complete client authentication.
# 0 means no timeout.
authentication_timeout = 60
# Logging directory
logdir = '/tmp'
# pid file name
pid_file_name = 'pgpool.pid'
# Replication mode
replication_mode = false
# Load balancing mode, i.e., all SELECTs are load balanced.
load_balance_mode = false
# If there's a disagreement with the packet kind sent from backend,
# then degenrate the node which is most likely "minority". If false,
# just force to exit this session.
replication_stop_on_mismatch = false
# If there's a disagreement with the number of affected tuples in
# UPDATE/DELETE, then degenrate the node which is most likely
# "minority".
# If false, just abort the transaction to keep the consistency.
failover_if_affected_tuples_mismatch = false
# If true, replicate SELECT statement when replication_mode or parallel_mode is
enabled.
# A priority of replicate_select is higher than load_balance_mode.
replicate_select = false
# Semicolon separated list of queries to be issued at the end of a
# session
reset_query_list = 'ABORT; DISCARD ALL'
# for 8.2 or older this should be as follows.
#reset_query_list = 'ABORT; RESET ALL; SET SESSION AUTHORIZATION DEFAULT'
# white_function_list is a comma separated list of function names
# those do not write to database. Any functions not listed here
# are regarded to write to database and SELECTs including such
# writer-functions will be executed on master(primary) in master/slave
# mode, or executed on all DB nodes in replication mode.
#
# black_function_list is a comma separated list of function names
# those write to database. Any functions not listed here
# are regarded not to write to database and SELECTs including such
# read-only-functions will be executed on any DB nodes.
#
# You cannot make full both white_function_list and
# black_function_list at the same time. If you specify something in
# one of them, you should make empty other.
#
# Pre 3.0 pgpool-II recognizes nextval and setval in hard coded
# way. Following setting will do the same as the previous version.
# white_function_list = ''
# black_function_list = 'nextval,setval'
white_function_list = ''
black_function_list = 'nextval,setval'
# If true print timestamp on each log line.
print_timestamp = true
# If true, operate in master/slave mode.
master_slave_mode = false
# Master/slave sub mode. either 'slony' or 'stream'. Default is 'slony'.
master_slave_sub_mode = 'slony'
# If the standby server delays more than delay_threshold,
# any query goes to the primary only. The unit is in bytes.
# 0 disables the check. Default is 0.
# Note that health_check_period required to be greater than 0
# to enable the functionality.
delay_threshold = 0
# 'always' logs the standby delay whenever health check runs.
# 'if_over_threshold' logs only if the delay exceeds delay_threshold.
# 'none' disables the delay log.
log_standby_delay = 'none'
# If true, cache connection pool.
connection_cache = true
# Health check timeout. 0 means no timeout.
health_check_timeout = 20
# Health check period. 0 means no health check.
health_check_period = 0
# Health check user
health_check_user = 'nobody'
# Execute command by failover.
# special values: %d = node id
# %h = host name
# %p = port number
# %D = database cluster path
# %m = new master node id
# %H = hostname of the new master node
# %M = old master node id
# %P = old primary node id
# %% = '%' character
#
failover_command = ''
# Execute command by failback.
# special values: %d = node id
# %h = host name
# %p = port number
# %D = database cluster path
# %m = new master node id
# %H = hostname of the new master node
# %M = old master node id
# %P = old primary node id
# %% = '%' character
#
failback_command = ''
# If true, trigger fail over when writing to the backend communication
# socket fails. This is the same behavior of pgpool-II 2.2.x or
# earlier. If set to false, pgpool will report an error and disconnect
# the session.
fail_over_on_backend_error = true
# If true, automatically locks a table with INSERT statements to keep
# SERIAL data consistency. If the data does not have SERIAL data
# type, no lock will be issued. An /*INSERT LOCK*/ comment has the
# same effect. A /*NO INSERT LOCK*/ comment disables the effect.
insert_lock = true
# If true, ignore leading white spaces of each query while pgpool judges
# whether the query is a SELECT so that it can be load balanced. This
# is useful for certain APIs such as DBI/DBD which is known to adding an
# extra leading white space.
ignore_leading_white_space = true
# If true, print all statements to the log. Like the log_statement option
# to PostgreSQL, this allows for observing queries without engaging in full
# debugging.
log_statement = false
# If true, print all statements to the log. Similar to log_statement except
# that prints DB node id and backend process id info.
log_per_node_statement = false
# If true, incoming connections will be printed to the log.
log_connections = false
# If true, hostname will be shown in ps status. Also shown in
# connection log if log_connections = true.
# Be warned that this feature will add overhead to look up hostname.
log_hostname = false
# if non 0, run in parallel query mode
parallel_mode = false
# if non 0, use query cache
enable_query_cache = false
#set pgpool2 hostname
pgpool2_hostname = ''
# system DB info
system_db_hostname = 'localhost'
system_db_port = 5432
system_db_dbname = 'pgpool'
system_db_schema = 'pgpool_catalog'
system_db_user = 'pgpool'
system_db_password = ''
# backend_hostname, backend_port, backend_weight
# here are examples
backend_hostname0 = 'localhost'
backend_port0 = 5432
#backend_weight0 = 1
#backend_data_directory0 = '/data'
#backend_hostname1 = 'host2'
#backend_port1 = 5433
#backend_weight1 = 1
#backend_data_directory1 = '/data1'
# - HBA -
# If true, use pool_hba.conf for client authentication.
enable_pool_hba = false
# - online recovery -
# online recovery user
recovery_user = 'nobody'
# online recovery password
recovery_password = ''
# execute a command in first stage.
recovery_1st_stage_command = ''
# execute a command in second stage.
recovery_2nd_stage_command = ''
# maximum time in seconds to wait for the recovering node's postmaster
# start-up. 0 means no wait.
# this is also used as a timer waiting for clients disconnected before
# starting 2nd stage
recovery_timeout = 90
# If client_idle_limit_in_recovery is n (n > 0), the client is forced
# to be disconnected whenever after n seconds idle (even inside an
# explicit transactions!) in the second stage of online recovery.
# n = -1 forces clients to be disconnected immediately.
# 0 disables this functionality(wait forever).
# This parameter only takes effect in recovery 2nd stage.
client_idle_limit_in_recovery = 0
# Specify table name to lock. This is used when rewriting lo_creat
# command in replication mode. The table must exist and has writable
# permission to public. If the table name is '', no rewriting occurs.
lobj_lock_table = ''
# If true, enable SSL support for both frontend and backend connections.
# note that you must also set ssl_key and ssl_cert for SSL to work in
# the frontend connections.
ssl = false
# path to the SSL private key file
#ssl_key = './server.key'
# path to the SSL public certificate file
#ssl_cert = './server.cert'
# If either ssl_ca_cert or ssl_ca_cert_dir is set, then certificate
# verification will be performed to establish the authenticity of the
# certificate. If neither is set to a nonempty string then no such
# verification takes place. ssl_ca_cert should be a path to a single
# PEM format file containing CA root certificate(s), whereas ssl_ca_cert_dir
# should be a directory containing such files. These are analagous to the
# -CAfile and -CApath options to openssl verify(1), respectively.
#ssl_ca_cert = ''
#ssl_ca_cert_dir = ''
# Debug message verbosity level. 0: no message, 1 <= : more verbose
debug_level = 0
Index: main.c
===================================================================
RCS file: /cvsroot/pgpool/pgpool-II/main.c,v
retrieving revision 1.85.2.9
diff -c -p -r1.85.2.9 main.c
*** main.c 25 May 2011 17:13:16 -0000 1.85.2.9
--- main.c 25 May 2011 20:09:34 -0000
*************** void send_failback_request(int node_id)
*** 1281,1287 ****
Req_info->kind = NODE_UP_REQUEST;
Req_info->node_id[0] = node_id;
! if (node_id < 0 || node_id >= MAX_NUM_BACKENDS || VALID_BACKEND(node_id))
{
pool_error("send_failback_request: node %d is alive.", node_id);
return;
--- 1281,1288 ----
Req_info->kind = NODE_UP_REQUEST;
Req_info->node_id[0] = node_id;
! if (node_id < 0 || node_id >= MAX_NUM_BACKENDS ||
! (RAW_MODE && BACKEND_INFO(node_id).backend_status != CON_DOWN && VALID_BACKEND(node_id)))
{
pool_error("send_failback_request: node %d is alive.", node_id);
return;
*************** static void failover(void)
*** 1459,1465 ****
if (Req_info->kind == NODE_UP_REQUEST)
{
if (node_id >= MAX_NUM_BACKENDS ||
! (Req_info->kind == NODE_UP_REQUEST && VALID_BACKEND(node_id)) ||
(Req_info->kind == NODE_DOWN_REQUEST && !VALID_BACKEND(node_id)))
{
pool_semaphore_unlock(REQUEST_INFO_SEM);
--- 1460,1467 ----
if (Req_info->kind == NODE_UP_REQUEST)
{
if (node_id >= MAX_NUM_BACKENDS ||
! (Req_info->kind == NODE_UP_REQUEST && !(RAW_MODE &&
! BACKEND_INFO(i).backend_status == CON_DOWN) && VALID_BACKEND(node_id)) ||
(Req_info->kind == NODE_DOWN_REQUEST && !VALID_BACKEND(node_id)))
{
pool_semaphore_unlock(REQUEST_INFO_SEM);
_______________________________________________
Pgpool-hackers mailing list
[email protected]
http://pgfoundry.org/mailman/listinfo/pgpool-hackers
