Hi Klaus, Thank you. I created another thread about it. Please look at that.
On Monday, March 22, 2021, 02:18:38 PM GMT+4:30, Klaus Wenninger <kwenn...@redhat.com> wrote: On 3/18/21 3:45 PM, Ken Gaillot wrote: > "Clusters from Scratch" is exactly that. The currently published one is > based on CentOS 7: > > https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/2.0/html-single/Clusters_from_Scratch/ > > A new one based on CentOS Stream 8 will be published with the next > release. A preview is available: > > http://people.redhat.com/~kgaillot/pacemaker/doc/2.0/Clusters_from_Scratch/html/ > > You only need to do through Chapter 7 to get a clustered web server > working. After that, you need some way to synchronize the web > documents; the walk-through demonstrates DRBD and GFS, but you can use > any means available to you. As I understood it just the proxy should be clustered. So I'd guess a synchronization of the webserver-data isn't needed as the actual webserver is running outside the cluster on node3. Maybe Jason you can ack that view. Regards, Klaus > > On Thu, 2021-03-18 at 14:31 +0000, Jason Long wrote: >> Thank you, but please forget node3. I want to make a cluster with two >> nodes. Both nodes has Apache web server and when a node stopped, then >> another node work. >> Can you show me a good tutorial about it? Or can you write it here? >> >> >> >> >> >> >> On Thursday, March 18, 2021, 01:40:22 AM GMT+3:30, Ken Gaillot < >> kgail...@redhat.com> wrote: >> >> >> >> >> >> On Wed, 2021-03-17 at 20:37 +0000, Jason Long wrote: >>> The 192.168.1.4 is my secondary VM. >>> I want to follow " >>> https://www.digitalocean.com/community/tutorials/how-to-set-up-an-apache-active-passive-cluster-using-pacemaker-on-centos-7 >>> " tutorial. >>> At "Step 8 — Adding the Apache Resource", I got below problems: >> Verify that you did Step 2 on node3. With the cluster stopped, start >> the web server manually, and use curl or wget to verify that you can >> successfully get the /server-status URL from the local host. >> >> Step 6 is a bad idea. You should configure and test fencing instead, >> before adding resources. >> >>> 1- In the tutorial example, you will see: >>> >>> >>> Full list of resources: >>> * Cluster_VIP (ocf::heartbeat:IPaddr2): Started webnode01 >>> * WebServer (ocf::heartbeat:apache): Started webnode02 >>> >>> >>> But mine: >>> >>> >>> Full List of Resources: >>> * Cluster_VIP (ocf::heartbeat:IPaddr2): Started node1 >>> * WebServer (ocf::heartbeat:apache): Starting node2 >>> >>> >>> Please be careful about "Started" and "Starting" words. >>> >>> >>> 2- I wanted to restart the Apache resource, but: >>> >>> >>> [root@node1 log]# sudo pcs resource restart WebServer >>> Error: crm_resource: Error performing operation: Timer expired >>> Set 'WebServer' option: id=WebServer-meta_attributes-target-role >>> set=WebServer-meta_attributes name=target-role value=stopped >>> Waiting for 1 resources to stop: >>> * WebServer >>> Deleted 'WebServer' option: id=WebServer-meta_attributes-target- >>> role >>> name=target-role >>> Waiting for 1 resources to start again: >>> * WebServer >>> Could not complete restart of WebServer, 1 resources remaining >>> * WebServer >>> >>> >>> >>> Logs are: >>> https://paste.ubuntu.com/p/nHfTRFh4RD/ >>> >>> >>> >>> Why? >>> >>> >>> >>> >>> >>> On Wednesday, March 17, 2021, 11:42:11 PM GMT+3:30, Jason Long < >>> hack3r...@yahoo.com> wrote: >>> >>> >>> >>> >>> >>> Yes, I want Apache always on node3, and then the reverse proxy can >>> move between node1 and node2. >>> Please see my new efforts. >>> >>> >>> >>> >>> >>> >>> On Wednesday, March 17, 2021, 11:38:01 PM GMT+3:30, Ken Gaillot < >>> kgail...@redhat.com> wrote: >>> >>> >>> >>> >>> >>> It sounds to me like your intent is to have apache always on node3, >>> and >>> then the reverse proxy can move between node1 and node2. The >>> floating >>> IP address, which is what users use to contact the site, is >>> associated >>> with the reverse proxy. >>> >>> In any case, the floating IP should be an *additional* IP that is >>> not >>> the primary IP address of any host. The cluster will associate this >>> IP >>> with whichever node is running the proxy. >>> >>> If the above scenario is what you want, then you can ban the web >>> server >>> from node1 and node2, create a group consisting of the floating IP >>> and >>> the reverse proxy, and ban the group from node3. >>> >>> On Wed, 2021-03-17 at 18:51 +0000, Jason Long wrote: >>>> Hello, >>>> I changed "IP" to my Apache web server: >>>> >>>> $ sudo pcs resource update floating_ip ocf:heartbeat:IPaddr2 >>>> ip=192.168.1.4 cidr_netmask=24 op monitor interval=5s >>>> >>>> And did: >>>> >>>> $ sudo pcs status >>>> Cluster name: mycluster >>>> Cluster Summary: >>>> * Stack: corosync >>>> * Current DC: node1 (version 2.0.5-10.fc33-ba59be7122) - >>>> partition >>>> with quorum >>>> * Last updated: Wed Mar 17 21:55:58 2021 >>>> * Last change: Wed Mar 17 21:55:02 2021 by root via cibadmin >>>> on >>>> node1 >>>> * 2 nodes configured >>>> * 2 resource instances configured >>>> >>>> Node List: >>>> * Online: [ node1 node2 ] >>>> Full List of Resources: >>>> * floating_ip (ocf::heartbeat:IPaddr2): Started node1 >>>> * http_server (ocf::heartbeat:apache): Stopped >>>> >>>> Failed Resource Actions: >>>> * http_server_start_0 on node1 'error' (1): call=10, >>>> status='Timed >>>> Out', exitreason='', last-rc-change='2021-03-17 21:50:31 +03:30', >>>> queued=0ms, exec=40002ms >>>> * http_server_start_0 on node2 'error' (1): call=11, >>>> status='Timed >>>> Out', exitreason='', last-rc-change='2021-03-17 21:51:11 +03:30', >>>> queued=0ms, exec=40002ms >>>> >>>> Daemon Status: >>>> corosync: active/enabled >>>> pacemaker: active/enabled >>>> pcsd: active/enabled >>>> >>>> >>>> Why "http_server (ocf::heartbeat:apache): Stopped" ? >>>> >>>> I think you misunderstand my goal, please examine " >>>> https://paste.ubuntu.com/p/Nx2ptqZjFg/". I just have one Apache >>>> server and two Reverse Proxy servers, when a Reverse Proxy server >>>> stopped then another one work. >>>> In this scenario, is group resources mandatory? >>>> >>>> >>>> >>>> >>>> >>>> On Wednesday, March 17, 2021, 01:50:35 AM GMT+3:30, Reid Wahl < >>>> nw...@redhat.com> wrote: >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Tue, Mar 16, 2021 at 3:13 PM Jason Long <hack3r...@yahoo.com> >>>> wrote: >>>>> I'm using CentOS. >>>> Ah okay. I think I had made an assumption based on the pastebins >>>> URLs. >>>> >>>>> Thus, I must use my Apache web server IP instead of node2? >>>> Yes, it's never a good idea to configure a node's constant IP >>>> address >>>> within an IPaddr2 resource. That will almost inevitably result in >>>> Pacemaker taking down the IP address at some point. >>>> >>>> For an IPaddr2 resource, you configure the IP address that's free >>>> to >>>> move around the cluster. In this case, that's the Apache web >>>> server >>>> IP. Node2's IP address isn't free to move to node1. >>>> >>>>> About resource group, is you mean " >>>>> https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/2.0/html-single/Pacemaker_Explained/index.html#group-resources >>>>> " ? >>>> Yes, that's correct. And if you have access to the Red Hat docs, >>>> you >>>> can also refer to the following: >>>> - Chapter 5. Configuring an active/passive Apache HTTP server >>>> in >>>> a >>>> Red Hat High Availability cluster ( >>>> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_high_availability_clusters/assembly_configuring-active-passive-http-server-in-a-cluster-configuring-and-managing-high-availability-clusters >>>> ) >>>> - Configuring resource groups ( >>>> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_high_availability_clusters/assembly_configuring-cluster-resources-configuring-and-managing-high-availability-clusters#assembly_resource-groups-configuring-cluster-resources >>>> ) >>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Wednesday, March 17, 2021, 01:10:33 AM GMT+3:30, Reid Wahl < >>>>> nw...@redhat.com> wrote: >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Tue, Mar 16, 2021 at 1:47 PM Jason Long <hack3r...@yahoo.com >>>>> wrote: >>>>>> Thanks. >>>>>> Excuse me, did you read how did I set my cluster up? Please >>>>>> look >>>>>> at: https://paste.ubuntu.com/p/Nx2ptqZjFg/ >>>>>> Which part of my configuration is wrong? >>>>> 1. You configured the IPaddr2 resource to use node2's IP >>>>> address >>>>> (192.168.1.10) instead of the floating IP address >>>>> (192.168.1.4). >>>>> 2. You didn't configure the resources into a resource group. As >>>>> a >>>>> result, the floating IP may end up on a different node compared >>>>> to >>>>> the web server. >>>>> >>>>> Both of these are explained in more detail in previous emails >>>>> :) >>>>> >>>>> I also thought that Ubuntu used /etc/apache2 instead of >>>>> /etc/httpd, >>>>> but maybe not. >>>>> >>>>>> Both of the main and secondary servers are an Apache >>>>>> Reverse >>>>>> Proxy Server. I want when main server failed, then the >>>>>> secondary >>>>>> server handle the requests. >>>>>> How can I achieve this goal? >>>>> I don't know anything about reverse proxies, sorry. I can only >>>>> really comment on general principles here, like "an IPaddr2 >>>>> resource shouldn't manage an IP address that's expected to stay >>>>> on >>>>> one particular node" and "if two resources need to run on the >>>>> same >>>>> node and start in a particular order, they need to be grouped." >>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tuesday, March 16, 2021, 11:57:13 PM GMT+3:30, Reid Wahl < >>>>>> nw...@redhat.com> wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Mar 16, 2021 at 1:03 PM Jason Long < >>>>>> hack3r...@yahoo.com >>>>>> wrote: >>>>>>> Thanks. >>>>>>> I changed it to the IP address of node2 and I can see my >>>>>>> Apache >>>>>>> Web Server. >>>>>> Like I said, you don't want to do that. You should change it >>>>>> to >>>>>> an IP address that you want the cluster to manage. If you set >>>>>> it >>>>>> to node2's IP address, Pacemaker will try to remove node2's >>>>>> IP >>>>>> address and assign it to node1 if the resource fails over to >>>>>> node1. If node2 is using that address for anything else >>>>>> (e.g., >>>>>> corosync communication), then that would be a big problem. >>>>>> >>>>>> The managed floating IP address should be an IP address >>>>>> dedicated >>>>>> to the web server, that can move between cluster nodes as >>>>>> needed. >>>>>> >>>>>>> >>>>>>> $ sudo pcs resource update floating_ip >>>>>>> ocf:heartbeat:IPaddr2 >>>>>>> ip=192.168.1.10 cidr_netmask=24 op monitor interval=5s >>>>>>> >>>>>>> Now, I want to test my cluster and stop node1. On node1 I >>>>>>> did: >>>>>>> >>>>>>> # pcs cluster stop http_server >>>>>>> Error: nodes 'http_server' do not appear to exist in >>>>>>> configuration >>>>>>> >>>>>>> Why? >>>>>> The `pcs cluster stop` command stops pacemaker and corosync >>>>>> services on a particular node (the local node if you don't >>>>>> specify one). You've specified `http_server`, so the command >>>>>> is >>>>>> trying to connect to a node called "http_server" and stop >>>>>> services there. >>>>>> >>>>>> If you want to stop node1, then run `pcs cluster stop node1`. >>>>>> >>>>>> If you want to prevent the http_server resource from running >>>>>> anywhere, then run `pcs resource disable http_server`. >>>>>> >>>>>> If you want to prevent the http_server resource from running >>>>>> on >>>>>> node2, then run `pcs resource ban http_server node2`. If you >>>>>> want >>>>>> to remove that constraint later and allow it to run on node2 >>>>>> again, run `pcs resource clear http_server`. >>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tuesday, March 16, 2021, 11:05:48 PM GMT+3:30, Reid Wahl >>>>>>> < >>>>>>> nw...@redhat.com> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Mar 16, 2021 at 12:11 PM Jason Long < >>>>>>> hack3r...@yahoo.com> wrote: >>>>>>>> Thank you so much. >>>>>>>> I forgot to ask a question. In below command, what should >>>>>>>> be >>>>>>>> the ip="IP" value? Is it the IP address of my Apache or >>>>>>>> node2? >>>>>>>> >>>>>>>> $ sudo pcs resource create floating_ip >>>>>>>> ocf:heartbeat:IPaddr2 >>>>>>>> ip="IP" cidr_netmask=24 op monitor interval=5s >>>>>>> It's the IP address that you want the cluster to manage. >>>>>>> That >>>>>>> sounds like it would be your web server IP address. You >>>>>>> definitely don't want to set the ip option to some IP >>>>>>> address >>>>>>> that resides statically on one of the nodes. An IP managed >>>>>>> by >>>>>>> an IPaddr2 resource can be moved around the cluster. >>>>>>> >>>>>>> If that's your web server IP address, you'll also want to >>>>>>> put >>>>>>> it in a resource group with your apache resource. >>>>>>> Otherwise, >>>>>>> the floating IP may end up on a different node from your >>>>>>> web >>>>>>> server, which renders the IP address useless. >>>>>>> >>>>>>> For resources that already exist, you can use the `pcs >>>>>>> resource >>>>>>> group add` command. For example: `pcs resource group add >>>>>>> apache_group floating_ip http_server`. >>>>>>> >>>>>>> For resources that you're newly creating, you can use the ` >>>>>>> --group` option of `pcs resource create`. For example, `pcs >>>>>>> resource create new_IP IPaddr2 <options> --group >>>>>>> apache_group`. >>>>>>> That adds the new resource to the end of the group. >>>>>>> >>>>>>> The pcs help outputs have more details on these options. >>>>>>> >>>>>>> If you're new to resource groups, you can check them out >>>>>>> here: >>>>>>> https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/2.0/html-single/Pacemaker_Explained/index.html#group-resources >>>>>>> >>>>>>>> >>>>>>>> Logs are: >>>>>>>> https://paste.ubuntu.com/p/86YHRX6rdC/ >>>>>>>> https://paste.ubuntu.com/p/HHVzNvhRM2/ >>>>>>>> https://paste.ubuntu.com/p/kNxynhfyc2/ >>>>>>>> >>>>>>>> I have not any "status.conf" file: >>>>>>>> >>>>>>>> # cat /etc/httpd/conf.d/status.conf >>>>>>>> cat: /etc/httpd/conf.d/status.conf: No such file or >>>>>>>> directory >>>>>>>> >>>>>>> If you're using Ubuntu, I believe it's in a different >>>>>>> location >>>>>>> -- somewhere in /etc/apache2 if memory serves. >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tuesday, March 16, 2021, 07:20:32 PM GMT+3:30, Klaus >>>>>>>> Wenninger <kwenn...@redhat.com> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 3/16/21 3:18 PM, Ken Gaillot wrote: >>>>>>>>> On Tue, 2021-03-16 at 09:42 +0000, Jason Long wrote: >>>>>>>>>> Hello, >>>>>>>>>> I want to launch a Clustering for my Apache Web >>>>>>>>>> Server. >>>>>>>>>> I >>>>>>>>>> have three >>>>>>>>>> servers: >>>>>>>>>> >>>>>>>>>> 1- Main server that acts as a Reverse Proxy >>>>>>>>>> 2- The secondary server that when my main server >>>>>>>>>> stopped, >>>>>>>>>> work as a >>>>>>>>>> Reverse Proxy >>>>>>>>>> 3- Apache Web Server >>>>>>>>>> >>>>>>>>>> The client ---> Reverse Proxy Server ---> Apache Web >>>>>>>>>> Server >>>>>>>>>> >>>>>>>>>> The IP addresses are: >>>>>>>>>> Main Server (node1) : 192.168.1.3 >>>>>>>>>> Secondary Server (node2) : 192.168.1.10 >>>>>>>>>> Apache Web Server (node3) : 192.168.1.4 >>>>>>>>>> >>>>>>>>>> On the main and secondary servers, I installed and >>>>>>>>>> configured Apache >>>>>>>>>> as a Reverse Proxy Server. I created a Virtual Host >>>>>>>>>> and >>>>>>>>>> my Reverse >>>>>>>>>> Configuration is: >>>>>>>>>> >>>>>>>>>> <VirtualHost *:80> >>>>>>>>>> ProxyPreserveHost On >>>>>>>>>> ProxyPass / http://192.168.1.4/ >>>>>>>>>> ProxyPassReverse / http://192.168.1.4/ >>>>>>>>>> </VirtualHost> >>>>>>>>>> >>>>>>>>>> As you see, it forward all requests to the Apache Web >>>>>>>>>> Server. >>>>>>>>>> >>>>>>>>>> I installed and configured Corosync and Pacemaker as >>>>>>>>>> below: >>>>>>>>>> >>>>>>>>>> On the main and secondary servers, I opened >>>>>>>>>> "/etc/hosts" >>>>>>>>>> files and >>>>>>>>>> added my servers IP addresses and host names: >>>>>>>>>> >>>>>>>>>> 192.168.1.3 node1 >>>>>>>>>> 192.168.1.10 node2 >>>>>>>>>> >>>>>>>>>> Then installed Pacemaker, Corosync, and Pcs packages >>>>>>>>>> on >>>>>>>>>> both servers >>>>>>>>>> and started its service: >>>>>>>>>> >>>>>>>>>> $ sudo yum install corosync pacemaker pcs >>>>>>>>>> $ sudo systemctl enable pcsd >>>>>>>>>> $ sudo systemctl start pcsd >>>>>>>>>> $ sudo systemctl status pcsd >>>>>>>>>> >>>>>>>>>> Then Configured the firewall on both servers as >>>>>>>>>> below: >>>>>>>>>> >>>>>>>>>> $ sudo firewall-cmd --permanent --add-service=http >>>>>>>>>> $ sudo firewall-cmd --permanent --add-service=high- >>>>>>>>>> availability >>>>>>>>>> $ sudo firewall-cmd --reload >>>>>>>>>> >>>>>>>>>> After it, on both servers, I created a password for >>>>>>>>>> the >>>>>>>>>> "hacluster" >>>>>>>>>> user, then on the main server: >>>>>>>>>> >>>>>>>>>> $ sudo pcs host auth node1 node2 -u hacluster -p >>>>>>>>>> password >>>>>>>>>> node1: Authorized >>>>>>>>>> node2: Authorized >>>>>>>>>> >>>>>>>>>> Then: >>>>>>>>>> $ sudo pcs cluster setup mycluster node1 node2 -- >>>>>>>>>> start >>>>>>>>>> -- >>>>>>>>>> enable >>>>>>>>>> $ sudo pcs cluster enable --all >>>>>>>>>> node1: Cluster Enabled >>>>>>>>>> node2: Cluster Enabled >>>>>>>>>> >>>>>>>>>> After it: >>>>>>>>>> $ sudo pcs cluster start --all >>>>>>>>>> node1: Starting Cluster... >>>>>>>>>> node2: Starting Cluster... >>>>>>>>>> >>>>>>>>>> I checked my clusters with below command and they are >>>>>>>>>> up >>>>>>>>>> and running: >>>>>>>>>> $ sudo pcs status >>>>>>>>>> ... >>>>>>>>>> Node List: >>>>>>>>>> * Online: [ node1 node2 ] >>>>>>>>>> .... >>>>>>>>>> >>>>>>>>>> And finally, I tried to add a resource: >>>>>>>>>> $ sudo pcs resource create floating_ip >>>>>>>>>> ocf:heartbeat:IPaddr2 >>>>>>>>>> ip=192.168.1.4 cidr_netmask=24 op monitor interval=5s >>>>>>>> Shouldn't the virtual-IP moved between node1 & node2 be >>>>>>>> different from the IP used for the web-server on node3? >>>>>>>> And having just one instance of the reverse-proxy that >>>>>>>> should probably be colocated with the virtual-IP - right? >>>>>>>> >>>>>>>> Klaus >>>>>>>> >>>>>>>>>> $ sudo pcs resource create http_server >>>>>>>>>> ocf:heartbeat:apache >>>>>>>>>> configfile="/etc/httpd/conf.d/VirtualHost.conf" op >>>>>>>>>> monitor >>>>>>>>>> timeout="5s" interval="5s" >>>>>>>>>> >>>>>>>>>> On both servers (Main and Secondary), pcsd service is >>>>>>>>>> enabled, but >>>>>>>>>> when I want to see my Apache Web Server then it show >>>>>>>>>> me >>>>>>>>>> below error: >>>>>>>>>> >>>>>>>>>> Proxy Error >>>>>>>>>> The proxy server received an invalid response from an >>>>>>>>>> upstream >>>>>>>>>> server. >>>>>>>>>> The proxy server could not handle the request >>>>>>>>>> Reason: Error reading from remote server >>>>>>>>>> >>>>>>>>>> Why? Which parts of my configuration is wrong? >>>>>>>>>> The output of "sudo pcs status" command is: >>>>>>>>>> https://paste.ubuntu.com/p/V9KvHKwKtC/ >>>>>>>>>> >>>>>>>>>> Thank you. >>>>>>>>> The thing to investigate is: >>>>>>>>> >>>>>>>>> Failed Resource Actions: >>>>>>>>> * http_server_start_0 on node2 'error' (1): >>>>>>>>> call=12, >>>>>>>>> status='Timed Out', exitreason='', last-rc- >>>>>>>>> change='2021- >>>>>>>>> 03- >>>>>>>>> 16 12:28:14 +03:30', queued=0ms, exec=40004ms >>>>>>>>> * http_server_start_0 on node1 'error' (1): >>>>>>>>> call=14, >>>>>>>>> status='Timed Out', exitreason='', last-rc- >>>>>>>>> change='2021- >>>>>>>>> 03- >>>>>>>>> 16 12:28:52 +03:30', queued=0ms, exec=40002ms >>>>>>>>> >>>>>>>>> The web server start timed out. Check the system, >>>>>>>>> pacemaker >>>>>>>>> and apache >>>>>>>>> logs around those times for any hints. >>>>>>>>> >>>>>>>>> Did you enable and test the status URL? The >>>>>>>>> ocf:heartbeat:apache agent >>>>>>>>> checks the status as part of its monitor (which is also >>>>>>>>> done for >>>>>>>>> start). It would be something like: >>>>>>>>> >>>>>>>>> cat <<-END >/etc/httpd/conf.d/status.conf >>>>>>>>> <Location /server-status> >>>>>>>>> SetHandler server-status >>>>>>>>> Require local >>>>>>>>> </Location> >>>>>>>>> END >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Manage your subscription: >>>>>>>> https://lists.clusterlabs.org/mailman/listinfo/users >>>>>>>> >>>>>>>> ClusterLabs home: https://www.clusterlabs.org/ >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Manage your subscription: >>>>>>>> https://lists.clusterlabs.org/mailman/listinfo/users >>>>>>>> >>>>>>>> ClusterLabs home: https://www.clusterlabs.org/ >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> >>>>>>> Reid Wahl, RHCA >>>>>>> Senior Software Maintenance Engineer, Red Hat >>>>>>> CEE - Platform Support Delivery - ClusterHA >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Manage your subscription: >>>>>>> https://lists.clusterlabs.org/mailman/listinfo/users >>>>>>> >>>>>>> ClusterLabs home: https://www.clusterlabs.org/ >>>>>>> _______________________________________________ >>>>>>> Manage your subscription: >>>>>>> https://lists.clusterlabs.org/mailman/listinfo/users >>>>>>> >>>>>>> ClusterLabs home: https://www.clusterlabs.org/ >>>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> >>>>>> Reid Wahl, RHCA >>>>>> Senior Software Maintenance Engineer, Red Hat >>>>>> CEE - Platform Support Delivery - ClusterHA >>>>>> >>>>>> _______________________________________________ >>>>>> Manage your subscription: >>>>>> https://lists.clusterlabs.org/mailman/listinfo/users >>>>>> >>>>>> ClusterLabs home: https://www.clusterlabs.org/ >>>>>> _______________________________________________ >>>>>> Manage your subscription: >>>>>> https://lists.clusterlabs.org/mailman/listinfo/users >>>>>> >>>>>> ClusterLabs home: https://www.clusterlabs.org/ >>>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> >>>>> Reid Wahl, RHCA >>>>> Senior Software Maintenance Engineer, Red Hat >>>>> CEE - Platform Support Delivery - ClusterHA >>>>> >>>>> _______________________________________________ >>>>> Manage your subscription: >>>>> https://lists.clusterlabs.org/mailman/listinfo/users >>>>> >>>>> ClusterLabs home: https://www.clusterlabs.org/ >>>>> _______________________________________________ >>>>> Manage your subscription: >>>>> https://lists.clusterlabs.org/mailman/listinfo/users >>>>> >>>>> ClusterLabs home: https://www.clusterlabs.org/ >>>>> >>>> _______________________________________________ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/ _______________________________________________ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/