On Feb 13, 2008 12:21 PM, Zoltan Boszormenyi <[EMAIL PROTECTED]> wrote: > > Serge Dubrouski írta: > > On Feb 13, 2008 6:41 AM, Zoltan Boszormenyi <[EMAIL PROTECTED]> wrote: > > > >> Serge Dubrouski írta: > >> > >> > >>> On Feb 13, 2008 4:29 AM, Zoltan Boszormenyi <[EMAIL PROTECTED]> wrote: > >>> > >>> > >>>> Andrew Beekhof írta: > >>>> > >>>> > >>>>> On Feb 12, 2008, at 8:57 PM, Zoltan Boszormenyi wrote: > >>>>> > >>>>> > >>>>> > >>>>>> Andrew Beekhof írta: > >>>>>> > >>>>>> > >>>>>>> On Feb 12, 2008, at 4:59 PM, Zoltan Boszormenyi wrote: > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> Serge Dubrouski írta: > >>>>>>>> > >>>>>>>> > >>>>>>>>> pgsql OCF RA doesn't support multistate configuration so I don't > >>>>>>>>> think > >>>>>>>>> that creating a clone would be a good idea. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> Thanks for the information. > >>>>>>>> > >>>>>>>> Some other questions. > >>>>>>>> > >>>>>>>> According to http://linux-ha.org/v2/faq/resource_too_active > >>>>>>>> the monitor action should return 0 for running, 7 ($OCF_NOT_RUNNING) > >>>>>>>> for downed resources and anything else for failed ones. > >>>>>>>> Either this documentation is buggy, > >>>>>>>> > >>>>>>>> > >>>>>>> no > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>> or heartbeat doesn't conform to its own docs. > >>>>>>>> > >>>>>>>> > >>>>>>> also no > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>> Here's the scenario: londiste creates a pidfile and deletes it when > >>>>>>>> it quits correctly. > >>>>>>>> However, if I kill it manually then the pidfile stays. What should > >>>>>>>> my script return > >>>>>>>> when it detects that the process with the indicated PID is no > >>>>>>>> longer there? > >>>>>>>> It's not a "downed" resource, it's a failed one. So I returned > >>>>>>>> $OCF_ERR_GENERIC. > >>>>>>>> But after some time heartbeat says that my resource became > >>>>>>>> "unmanaged". > >>>>>>>> > >>>>>>>> > >>>>>>> i'm guessing (because you've not included anything on which to > >>>>>>> comment properly) that the stop action failed > >>>>>>> > >>>>>>> > >>>>>> It shouldn't have failed, stop action always returns $OCF_SUCCESS. > >>>>>> > >>>>>> > >>>>>> > >>>>>>>> In contrast to this, the pgsql OCF RA does it differently. It > >>>>>>>> always returns 7 > >>>>>>>> when it finds that there's no postmaster process. Which is the > >>>>>>>> right behaviour? > >>>>>>>> > >>>>>>>> > >>>>>>> it depends what you want to happen. > >>>>>>> if you want a stop to be sent, use OCF_ERR_GENERIC. > >>>>>>> if the resource is stateless and doesnt need any cleaning up, use > >>>>>>> OCF_NOT_RUNNING > >>>>>>> > >>>>>>> > >>>>>> It's quite an important detail. Shouldn't this be documented at > >>>>>> http://linux-ha.org/OCFResourceAgent ? > >>>>>> > >>>>>> > >>>>> yep. but its a wiki so anyone can do that :) > >>>>> > >>>>> > >>>> I see. It's an excuse because no one did it yet. :-) > >>>> > >>>> Yesterday another problem popped up and I don't understand why > >>>> didn't it happen before. I upgraded to heartbeat 2.1.3 using the > >>>> SuSe build service packages at > >>>> http://download.opensuse.org/repositories/server:/ha-clustering/ > >>>> but the problem seems persisting. I have two pgsql resources, > >>>> using the stock install on my Fedora 6, i.e. pgdata=/var/lib/pgsql/data. > >>>> Both are tied to their respective nodes, symmetric_cluster on, > >>>> the constraints' score is -INFINITY for running them on the wrong node. > >>>> The documentation of heartbeat said that for a symmetric cluster > >>>> it's the way to bind a resource to a node (or to a set of nodes). > >>>> The problem is that after the first pgsql resource is started > >>>> successfully > >>>> on the first node then the second pgsql resource is checked whether > >>>> it's running on the first node - surprise, surprise, the system indicates > >>>> that it does. As a consequence, it's marked as startup failed and > >>>> heartbeat doesn't try to start it on the second node. Doing a cleanup > >>>> on the failed second pgsql resource makes it start but now the first > >>>> pgsql resource is marked failed. I guess because of the cleanup, > >>>> the second pgsql is thuoght to be running on node1 and is stopped. > >>>> The monitor action of the first resource notices that is's dead. > >>>> Catch 22? > >>>> > >>>> Turning the configuration upside down (symmetric_cluster off > >>>> and using +INFINITY rsc_location scores for binding to the correct > >>>> node) didn't help. > >>>> > >>>> How can I solve this besides using a different PGDATA directory > >>>> on the second node? The two machines is supposed to be configured > >>>> identically regarding PostgreSQL. > >>>> > >>>> > >>> Attached is a patch for pgsql that supposedly fixes this issue. Please > >>> test it and let me know the results. > >>> > >>> > >> Thanks, but why not simply use pg_ctl status? > >> > >> --- old_ocf/pgsql 2008-01-25 17:16:52.000000000 +0100 > >> +++ pgsql 2008-02-11 06:15:28.000000000 +0100 > >> @@ -267,7 +267,7 @@ > >> # > >> > >> pgsql_status() { > >> - pgrep -u $OCF_RESKEY_pgdba "postmaster|postgres" >/dev/null 2>&1 > >> + runasowner "$OCF_RESKEY_pgctl -D $OCF_RESKEY_pgdata status > >> >/dev/null 2>&1" > >> } > >> > >> # > >> > >> This one above solved the problem for me. > >> But it requires that I use different PGDATA on different nodes. > >> > > > > Why would you need to have the same PGDATA for different instances of > > PostgreSQL? Imagine what would happen at the failover time. You'll > > have 2 instances of PostgreSQL trying to start on the same PG_DATA > > which, I'm affraid isn't a good idea. > > > > > >> It seems your patch isn't different in this regard. Your mods: > >> 1. if $PIDFILE is readable, look at the PID it has > >> 2. if not, look at the processes > >> > > > > The problem is that by default PG_DATA isn't readable by everybody, > > but OCF spec requires that "status" command should work for any user, > > also "stop" command should report service down for for stopped service > > for any user as well. That's why I have to use this double logic. > > > > I see. But for an identically setup two-node PG cluster may still produce > a false positive, causing either multi-running state (either or all of the > PG instances are considered running everywhere), or the issue that > when one of them starts the other doesn't get started because it's > considered already running. I experienced both now... > > >> It still have problems with identical PGDATA directories. > >> Do you have an idea how to distinguish in that case? > >> > > > > Still don't understand why do you need to have the same PG_DATA? Are > > you trying to build a cluster with 2 instances of PostgreSQL each of > > each is tied to particular node? > > > > Yes, we set up a two-node PostgreSQL active-active (with SkyTools > Londiste) or active-passive (with SkyTools WalMgr) replication system. > The virtual IP is the service IP for the PostgreSQL cluster. > When the service IP is migrated to the secondary node: > - in the active-active case, pg_hba.conf is switched so connections > from the primary machine are rejected, existing replication > connections are killed; > - in the active-passive case, the standby PostgreSQL is woken up > and it starts listening with the last known good transaction state > Otherwise the two machine's PostgreSQL setup should be identical.
Ok, I see. In this case you configure as many identical pgsql resouces as you need BUT you have to use a set of rsc_location rules to make sure that they each of them is tied to a particular node and there never will be two postgresses running at the same node concurrently. Having identical PGDATA for all of them shouldn't create any problems. You'll also have to create your own RA tied to virtual IP that will be responsible of switching PostgeSQL from passive mode to active mode. For active/active mode you probably won't have create a such RA, just switch an IP. Unfortunately I've never worked with SkyTools myself, so I'm not a really good adviser here. There were a couple of other guys who tried to write a multistate RA for pgsql but they didn't post any results here. > > > >>>> And a question about 2.1.3. After the upgrade, haclient couldn't connect > >>>> because mgmtd wasn't started. I needed to add these two lines to ha.cf: > >>>> > >>>> apiauth mgmtd uid=root > >>>> respawn root /usr/lib64/heartbeat/mgmtd -v > >>>> > >>>> Is it really needed? It wasn't for 2.0.8 and the docs say that > >>>> it's not necessary since 2.0.5. Documentation got outdated again, > >>>> or something broke? > >>>> > > -- > ---------------------------------- > Zoltán Böszörményi > Cybertec Schönig & Schönig GmbH > http://www.postgresql.at/ > > > _______________________________________________ > Linux-HA mailing list > [email protected] > http://lists.linux-ha.org/mailman/listinfo/linux-ha > See also: http://linux-ha.org/ReportingProblems > -- Serge Dubrouski. _______________________________________________ Linux-HA mailing list [email protected] http://lists.linux-ha.org/mailman/listinfo/linux-ha See also: http://linux-ha.org/ReportingProblems
