On 3/27/2012 01:18 AM, Andreas Krurz wrote:
> Mountpoint created on both nodes, defined correct device and valid file
> system? What happens after a cleanup? ... crm resource cleanup p_fs_vol01 ...
> grep for "Filesystem" in your logs to get the error output from the resource
> agent.
>
> For more ... please share current drbd state/configuration and your cluster
> configuration.
>
> Regards,
> Andreas
* Pardon me if I'm not replying correctly, I'm trying to learn the mailing list
usage. I'll see how this goes. Look out, I'm a noob!
Andreas,
Thank you for your reply.
Mountpoints are done using LVM2 (as mentioned in the LinBit guide; the DRBD
resource is the used as the physical volume for the LV Group) and are all
showing available on ds01, status is NOT available on ds02 at this time. I
formatted them with ext4 and specified that difference when going through the
LinBit guide (for the Pacemaker config; they mention ext3 in their guide).
I had previously run the cleanup, and it did not appear to make a different.
This time I first ran crm_mon -1 and the lvm resource was Started, but the
Filesystems were not. That is how it has been.
Then, and maybe I shouldn't have worried about this yet, but I noticed in my
global_common.conf file that I hadn't included any wait-connect in the Startup
section (see below for my additions.
After doing so, though I have not restarted anything yet, I ran crm resource
cleanup p_fs_vol01 , then I saw the lvm resource say "FAILED" and I am now
getting the following from crm_mon -1:
Stack: Heartbeat
Current DC: ds02 (8a61ab9e-da93-4b4d-8f37-9523436b5f14) - partition with quorum
Version: 1.1.5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f
2 Nodes configured, unknown expected votes
4 Resources configured.
============
Online: [ ds01 ds02 ]
Master/Slave Set: ms_drbd_nfs [p_drbd_nfs]
p_drbd_nfs:0 (ocf::linbit:drbd): Slave ds01 (unmanaged) FAILED
p_drbd_nfs:1 (ocf::linbit:drbd): Slave ds02 (unmanaged) FAILED
Clone Set: cl_lsb_nfsserver [p_lsb_nfsserver]
Started: [ ds01 ds02 ]
Failed actions:
p_drbd_nfs:0_demote_0 (node=ds01, call=75, rc=5, status=complete): not
installed
p_drbd_nfs:0_stop_0 (node=ds01, call=79, rc=5, status=complete): not
installed
p_drbd_nfs:1_monitor_30000 (node=ds02, call=447, rc=5, status=complete):
not installed
p_drbd_nfs:1_stop_0 (node=ds02, call=458, rc=5, status=complete): not
installed
Grep for "Filesystem" in /var/log/syslog on ds01 shows the following for every
volume repeatedly:
Mar 29 11:06:19 ds01 pengine: [27987]: notice: native_print:
p_fs_vol01#011(ocf::heartbeat:Filesystem):#011Stopped
On ds02, I receive the same in the syslog file, with the addition of this
message after the above messages:
Mar 29 11:09:26 ds02 Filesystem[2000]: [2021]: WARNING: Couldn't find device
[/dev/nfs/vol01]. Expected /dev/??? to exist
DRBD State from ds01 (Before restarting ds02): Connected and UpToDate with ds01
as the Primary.
DRBD State from ds02 (After restarting ds02; interesting; Pacemaker?): cat:
/proc/drbd: No such file or directory
DRBD State from ds01 (After restarting ds02): WFConnection with ds02 as unknown.
---- Configuration below here ---
:::DRBD Resource Config:::
resource nfs {
device /dev/drbd0;
disk /dev/sda1;
meta-disk internal;
on ds01 {
address 192.168.1.11:7790;
}
on ds02 {
address 192.168.1.12:7790;
}
:::DRBD Global_common.conf:::
global {
usage-count yes;
}
common {
protocol C;
handlers {
pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh;
/usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot
-f";
pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh;
/usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot
-f";
local-io-error "/usr/lib/drbd/notify-io-error.sh;
/usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt
-f";
}
startup {
# Just included before reply on mailing list. 3/29/2012, Please
reply with comment to this if I am mistaken for adding these.
wfc-timeout 120
degr-wfc-timeout 120
outdated-wfc-timeout 120
wait-after-sb 180
}
disk {
on-io-error detach;
}
net {
after-sb-0pri disconnect;
after-sb-1pri disconnect;
after-sb-2pri disconnect;
}
syncer {
rate 100M;
al-extents 257;
}
}
:::Heartbeat ha.cf:::
autojoin none
mcast bond0 239.0.0.1 694 1 0
bcast bond1
keepalive 2
deadtime 15
warntime 5
initdead 60
node ds01
node ds02
pacemaker respawn
:::Pacemaker CIB.XML:::
<cib epoch="60" num_updates="0" admin_epoch="0" validate-with="pacemaker-1.2"
crm_feature_set="3.0.5" have-quorum="1" cib-last-written="Thu Mar 29 10:39:59
2012" dc-uuid="8a61ab9e-da93-4b4d-8f37-9523436b5f14">
<configuration>
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<nvpair id="cib-bootstrap-options-dc-version" name="dc-version"
value="1.1.5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f"/>
<nvpair id="cib-bootstrap-options-cluster-infrastructure"
name="cluster-infrastructure" value="Heartbeat"/>
<nvpair id="cib-bootstrap-options-stonith-enabled"
name="stonith-enabled" value="false"/>
<nvpair id="cib-bootstrap-options-last-lrm-refresh"
name="last-lrm-refresh" value="1333042796"/>
<nvpair id="cib-bootstrap-options-no-quorum-policy"
name="no-quorum-policy" value="ignore"/>
</cluster_property_set>
</crm_config>
<nodes>
<node id="b0dff0ec-073e-475b-b7b9-167ae122e5e0" type="normal"
uname="ds01"/>
<node id="8a61ab9e-da93-4b4d-8f37-9523436b5f14" type="normal"
uname="ds02"/>
</nodes>
<resources>
<primitive class="ocf" id="failover-ip" provider="heartbeat"
type="IPaddr">
<instance_attributes id="failover-ip-instance_attributes">
<nvpair id="failover-ip-instance_attributes-ip" name="ip"
value="192.168.2.10"/>
</instance_attributes>
<operations>
<op id="failover-ip-monitor-10s" interval="10s" name="monitor"/>
</operations>
<meta_attributes id="failover-ip-meta_attributes">
<nvpair id="failover-ip-meta_attributes-target-role"
name="target-role" value="Stopped"/>
</meta_attributes>
</primitive>
<master id="ms_drbd_nfs">
<meta_attributes id="ms_drbd_nfs-meta_attributes">
<nvpair id="ms_drbd_nfs-meta_attributes-master-max" name="master-max"
value="1"/>
<nvpair id="ms_drbd_nfs-meta_attributes-master-node-max"
name="master-node-max" value="1"/>
<nvpair id="ms_drbd_nfs-meta_attributes-clone-max" name="clone-max"
value="2"/>
<nvpair id="ms_drbd_nfs-meta_attributes-clone-node-max"
name="clone-node-max" value="1"/>
<nvpair id="ms_drbd_nfs-meta_attributes-notify" name="notify"
value="true"/>
</meta_attributes>
<primitive class="ocf" id="p_drbd_nfs" provider="linbit" type="drbd">
<instance_attributes id="p_drbd_nfs-instance_attributes">
<nvpair id="p_drbd_nfs-instance_attributes-drbd_resource"
name="drbd_resource" value="nfs"/>
</instance_attributes>
<operations>
<op id="p_drbd_nfs-monitor-15" interval="15" name="monitor"
role="Master"/>
<op id="p_drbd_nfs-monitor-30" interval="30" name="monitor"
role="Slave"/>
</operations>
</primitive>
</master>
<clone id="cl_lsb_nfsserver">
<primitive class="lsb" id="p_lsb_nfsserver" type="nfs-kernel-server">
<operations>
<op id="p_lsb_nfsserver-monitor-30s" interval="30s" name="monitor"/>
</operations>
</primitive>
</clone>
<group id="g_nfs">
<primitive class="ocf" id="p_lvm_nfs" provider="heartbeat" type="LVM">
<instance_attributes id="p_lvm_nfs-instance_attributes">
<nvpair id="p_lvm_nfs-instance_attributes-volgrpname"
name="volgrpname" value="nfs"/>
</instance_attributes>
<operations>
<op id="p_lvm_nfs-monitor-30s" interval="30s" name="monitor"/>
</operations>
<meta_attributes id="p_lvm_nfs-meta_attributes">
<nvpair id="p_lvm_nfs-meta_attributes-is-managed" name="is-managed"
value="true"/>
</meta_attributes>
</primitive>
<primitive class="ocf" id="p_fs_vol01" provider="heartbeat"
type="Filesystem">
<instance_attributes id="p_fs_vol01-instance_attributes">
<nvpair id="p_fs_vol01-instance_attributes-device" name="device"
value="/dev/nfs/vol01"/>
<nvpair id="p_fs_vol01-instance_attributes-directory"
name="directory" value="/srv/nfs/vol01"/>
<nvpair id="p_fs_vol01-instance_attributes-fstype" name="fstype"
value="ext4"/>
</instance_attributes>
<operations>
<op id="p_fs_vol01-monitor-10s" interval="10s" name="monitor"/>
</operations>
<meta_attributes id="p_fs_vol01-meta_attributes">
</meta_attributes>
</primitive>
<primitive class="ocf" id="p_fs_vol02" provider="heartbeat"
type="Filesystem">
<instance_attributes id="p_fs_vol02-instance_attributes">
<nvpair id="p_fs_vol02-instance_attributes-device" name="device"
value="/dev/nfs/vol02"/>
<nvpair id="p_fs_vol02-instance_attributes-directory"
name="directory" value="/srv/nfs/vol02"/>
<nvpair id="p_fs_vol02-instance_attributes-fstype" name="fstype"
value="ext4"/>
</instance_attributes>
<operations>
<op id="p_fs_vol02-monitor-10s" interval="10s" name="monitor"/>
</operations>
</primitive>
<primitive class="ocf" id="p_fs_vol03" provider="heartbeat"
type="Filesystem">
<instance_attributes id="p_fs_vol03-instance_attributes">
<nvpair id="p_fs_vol03-instance_attributes-device" name="device"
value="/dev/nfs/vol03"/>
<nvpair id="p_fs_vol03-instance_attributes-directory"
name="directory" value="/srv/nfs/vol03"/>
<nvpair id="p_fs_vol03-instance_attributes-fstype" name="fstype"
value="ext4"/>
</instance_attributes>
<operations>
<op id="p_fs_vol03-monitor-10s" interval="10s" name="monitor"/>
</operations>
</primitive>
<primitive class="ocf" id="p_fs_vol04" provider="heartbeat"
type="Filesystem">
<instance_attributes id="p_fs_vol04-instance_attributes">
<nvpair id="p_fs_vol04-instance_attributes-device" name="device"
value="/dev/nfs/vol04"/>
<nvpair id="p_fs_vol04-instance_attributes-directory"
name="directory" value="/srv/nfs/vol04"/>
<nvpair id="p_fs_vol04-instance_attributes-fstype" name="fstype"
value="ext4"/>
</instance_attributes>
<operations>
<op id="p_fs_vol04-monitor-10s" interval="10s" name="monitor"/>
</operations>
</primitive>
<primitive class="ocf" id="p_fs_vol05" provider="heartbeat"
type="Filesystem">
<instance_attributes id="p_fs_vol05-instance_attributes">
<nvpair id="p_fs_vol05-instance_attributes-device" name="device"
value="/dev/nfs/vol05"/>
<nvpair id="p_fs_vol05-instance_attributes-directory"
name="directory" value="/srv/nfs/vol05"/>
<nvpair id="p_fs_vol05-instance_attributes-fstype" name="fstype"
value="ext4"/>
</instance_attributes>
<operations>
<op id="p_fs_vol05-monitor-10s" interval="10s" name="monitor"/>
</operations>
</primitive>
<primitive class="ocf" id="p_fs_vol06" provider="heartbeat"
type="Filesystem">
<instance_attributes id="p_fs_vol06-instance_attributes">
<nvpair id="p_fs_vol06-instance_attributes-device" name="device"
value="/dev/nfs/vol06"/>
<nvpair id="p_fs_vol06-instance_attributes-directory"
name="directory" value="/srv/nfs/vol06"/>
<nvpair id="p_fs_vol06-instance_attributes-fstype" name="fstype"
value="ext4"/>
</instance_attributes>
<operations>
<op id="p_fs_vol06-monitor-10s" interval="10s" name="monitor"/>
</operations>
</primitive>
<primitive class="ocf" id="p_fs_vol07" provider="heartbeat"
type="Filesystem">
<instance_attributes id="p_fs_vol07-instance_attributes">
<nvpair id="p_fs_vol07-instance_attributes-device" name="device"
value="/dev/nfs/vol07"/>
<nvpair id="p_fs_vol07-instance_attributes-directory"
name="directory" value="/srv/nfs/vol07"/>
<nvpair id="p_fs_vol07-instance_attributes-fstype" name="fstype"
value="ext4"/>
</instance_attributes>
<operations>
<op id="p_fs_vol07-monitor-10s" interval="10s" name="monitor"/>
</operations>
</primitive>
<primitive class="ocf" id="p_fs_vol08" provider="heartbeat"
type="Filesystem">
<instance_attributes id="p_fs_vol08-instance_attributes">
<nvpair id="p_fs_vol08-instance_attributes-device" name="device"
value="/dev/nfs/vol08"/>
<nvpair id="p_fs_vol08-instance_attributes-directory"
name="directory" value="/srv/nfs/vol08"/>
<nvpair id="p_fs_vol08-instance_attributes-fstype" name="fstype"
value="ext4"/>
</instance_attributes>
<operations>
<op id="p_fs_vol08-monitor-10s" interval="10s" name="monitor"/>
</operations>
</primitive>
<primitive class="ocf" id="p_fs_vol09" provider="heartbeat"
type="Filesystem">
<instance_attributes id="p_fs_vol09-instance_attributes">
<nvpair id="p_fs_vol09-instance_attributes-device" name="device"
value="/dev/nfs/vol09"/>
<nvpair id="p_fs_vol09-instance_attributes-directory"
name="directory" value="/srv/nfs/vol09"/>
<nvpair id="p_fs_vol09-instance_attributes-fstype" name="fstype"
value="ext4"/>
</instance_attributes>
<operations>
<op id="p_fs_vol09-monitor-10s" interval="10s" name="monitor"/>
</operations>
</primitive>
<primitive class="ocf" id="p_fs_vol10" provider="heartbeat"
type="Filesystem">
<instance_attributes id="p_fs_vol10-instance_attributes">
<nvpair id="p_fs_vol10-instance_attributes-device" name="device"
value="/dev/nfs/vol10"/>
<nvpair id="p_fs_vol10-instance_attributes-directory"
name="directory" value="/srv/nfs/vol10"/>
<nvpair id="p_fs_vol10-instance_attributes-fstype" name="fstype"
value="ext4"/>
</instance_attributes>
<operations>
<op id="p_fs_vol10-monitor-10s" interval="10s" name="monitor"/>
</operations>
</primitive>
<primitive class="ocf" id="p_fs_dc1" provider="heartbeat"
type="Filesystem">
<instance_attributes id="p_fs_dc1-instance_attributes">
<nvpair id="p_fs_dc1-instance_attributes-device" name="device"
value="/dev/nfs/dc1"/>
<nvpair id="p_fs_dc1-instance_attributes-directory"
name="directory" value="/srv/nfs/dc1"/>
<nvpair id="p_fs_dc1-instance_attributes-fstype" name="fstype"
value="ext4"/>
</instance_attributes>
<operations>
<op id="p_fs_dc1-monitor-10s" interval="10s" name="monitor"/>
</operations>
</primitive>
<meta_attributes id="g_nfs-meta_attributes">
<nvpair id="g_nfs-meta_attributes-target-role" name="target-role"
value="Started"/>
</meta_attributes>
</group>
</resources>
<constraints>
<rsc_order first="ms_drbd_nfs" first-action="promote"
id="o_drbd_before_nfs" score="INFINITY" then="g_nfs" then-action="start"/>
<rsc_colocation id="c_nfs_on_drbd" rsc="g_nfs" score="INFINITY"
with-rsc="ms_drbd_nfs" with-rsc-role="Master"/>
<rsc_location id="cli-prefer-failover-ip" rsc="failover-ip">
<rule id="cli-prefer-rule-failover-ip" score="INFINITY"
boolean-op="and">
<expression id="cli-prefer-expr-failover-ip" attribute="#uname"
operation="eq" value="ds01" type="string"/>
</rule>
</rsc_location>
</constraints>
<rsc_defaults>
<meta_attributes id="rsc-options">
<nvpair id="rsc-options-resource-stickiness" name="resource-stickiness"
value="200"/>
</meta_attributes>
</rsc_defaults>
</configuration>
</cib>
Thank you,
Robert
_______________________________________________
drbd-user mailing list
[email protected]
http://lists.linbit.com/mailman/listinfo/drbd-user
