Re: [ovirt-users] Fwd: RE: FC QLogic question

2014-09-12 Thread Bill Dossett
Hi,

Sorry, I got waylaid at the end of yesterday - catching up...  (sorry for 
length of thread now as well, but would like to get to the bottom of this)

Have seen posts about ovirt-node... I personally like the idea and hope that it 
continues to develop.  Having to load a Linux distro up and then put ovirt 
hypervisor on it is more work and then you need to worry about upgrading the 
linux distro and ovirt and suddenly ovirt doesn't run on the distro you have 
and you start over.  IMHO anyway.

All of the LUN names are listed in /dev/mapper

/etc/multipath.conf is thus

# RHEV REVISION 1.0

defaults {
polling_interval5
getuid_callout  /sbin/scsi_id --whitelisted --replace-whitespace 
--device=/dev/%n
no_path_retry   fail
user_friendly_names no
flush_on_last_del   yes
fast_io_fail_tmo5
dev_loss_tmo30
max_fds 4096
}

devices {
device {
vendor  HITACHI
product DF.*
getuid_callout  /sbin/scsi_id --whitelisted --replace-whitespace 
--device=/dev/%n
}
device {
vendor  COMPELNT
product Compellent Vol
no_path_retry   fail
}
}

But on the multipath -ll round robin is enabled...

I can't see how the firmware missing can be a problem if all my disks are 
appearing on node.  Some cards don't need to be flashed, they can be if needed 
but if my LUNs are appearing, then I assume it is working.

Sorry, I am very new to node and not much experience with FC under Linux, so 
trying to get up to speed on commands...

Which ovirt am Iusing?  Not sure what you want back,  uname -a yields 
2.6.32-431.3.1.el6.x86_6

The iso I used is ovirt-node-iso-3.4-20140423.0.el6.iso   from the 
releases/stable/iso folder - I am fairly sure, there are two ISOs there.  
Unfortunately I am not in the office today and can't get to the desktop where I 
burned the ISOs from...

Thanks
Bill

-Original Message-
From: Fabian Deutsch [mailto:fdeut...@redhat.com]
Sent: 11 September 2014 13:40
To: Daniel Helgenberger; Bill Dossett
Cc: users@ovirt.org
Subject: Re: [ovirt-users] Fwd: RE: FC QLogic question

- Original Message -

 On 11.09.2014 14:23, Fabian Deutsch wrote:
  Hey Daniel,
 
  - Original Message -
  On 11.09.2014 14:09, Fabian Deutsch wrote:
  Hey Bill and Daniel,
 
  it would indeed be interesting - mandatory - to know if it's qla*
  or qxl (qxl is IMHO related to graphics).
  Once we know that I'll see if the bug below fixes it.
  Hello Fabian,
  indeed this was my mistake; while typing the mail I was reading on
  the spice website - so qla is the correct module of course! I
  already replied this to Bill but forgot to cc the list also ...
 
  Also, for the node iso it would make little sense to have the qlx
  module loaded. For completeness, I attach my answer to Bill.
  Thanks :)
 
  So to summarize:
  The adapter is recognized on Node?
 
  But it does not show up in Engine?

 Bill has to tell, but I think so. I suggested to him this might be

Ah, I mixed you an Bill up in the thread.

 because of dm-multipath, multipath.conf might blacklist here. My nodes
 are plain EL6 and I have multipath enabled - but I do not know if
 ovirt-node filters for dm-mp devnodes only by default (this would make
 sense because of iscsi-multipathing in ovirt 3.4).

Well, we did some multipath changes lately. Let#s see what Bill says:

Bill,
what node are you using?
What does multipath -l say?
How does /dev/mapper/* and /dev/multipath.conf look like?

And is the disk appearing on Node and/or Engine?

Thanks!
fabian

 Maybe you can help out on this topic?

 Thanks!
 
  Thanks
  - fabian
 
  I'd need your help to verify that we are also including all
  relevant dependencies.
 
  Thanks
  - fabian
 
  - Original Message -
  On 11.09.2014 13:05, Bill Dossett wrote:
 
  Hi Daniel,
 
  I did a lsmod | grep qxl  and it returned nothing...  did you
  perhaps mean qla?
 
  [root@ovirt141 ~]# lsmod | grep qla
  qla2xxx   466573  56
 
  I think that is the one
  Indeed, sorry for that.
 
  lsscsi does list all of the LUNs...  so that's working!
  Great!
 
  But nothing in ovirt engine when I try to add a storage domain.
  Please check back in ovirt engine, with Hosts / Hardware info. You
  should see the adapter there, along with  WWN's ect.
 
  I am not quite sure, but AFAIK the node uses dm-multipath by
  default (for iscsi multipathing). So what does:
  # multipath -ll
  list?
 
  Thanks again, I guess I will try and get a bit further with the
  installs I have...
 
  Bill
 
 
 
 
 
 
 
 
 
 
  Hi Daniel,
 
  I did a lsmod | grep qxl  and it returned nothing...  did you
  perhaps mean qla?
 
  [root@ovirt141 ~]# lsmod | grep qla
  qla2xxx   466573  56
 
  I think that is the one
 
  lsscsi does list all of the LUNs...  so that's working!
 
  But nothing in ovirt engine when I try to add a storage domain

Re: [ovirt-users] Fwd: RE: FC QLogic question

2014-09-12 Thread Sander Grendelman
This could also be an selinux issue with FC on the node, I've hit that one
before.
Especially since the luns do show up with multipath -ll
Could be as simple as flipping a sanlock related selinux boolean.

A quick test would be to boot in permissive mode.
See http://www.ovirt.org/Node_Troubleshooting#SELinux for some more
information.
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] Fwd: RE: FC QLogic question

2014-09-12 Thread Bill Dossett
Hmmm… I’ve just gone to add a storage domain and in the admin portal and bingo 
bango, all my LUNs appeared.

Just wiping out everything from my last lab now.

Thanks everyone, I am not quite sure why it started working, but knowing that 
the ovirt-node was seeing the LUNs prompted me to continue.

Thanks all for the help.

Bill


From: sander.grendel...@gmail.com [mailto:sander.grendel...@gmail.com] On 
Behalf Of Sander Grendelman
Sent: 12 September 2014 08:28
To: Bill Dossett
Cc: users@ovirt.org
Subject: Re: [ovirt-users] Fwd: RE: FC QLogic question

This could also be an selinux issue with FC on the node, I've hit that one 
before.
Especially since the luns do show up with multipath -ll
Could be as simple as flipping a sanlock related selinux boolean.

A quick test would be to boot in permissive mode.
See http://www.ovirt.org/Node_Troubleshooting#SELinux for some more information.



___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] Fwd: RE: FC QLogic question

2014-09-12 Thread Bill Dossett
Bugger, spoke to soon…. One host sees the LUNs the other doesn’t.  Same builds 
of ovirt-node.  Both dell 2950s, and both qlogic cards…  but one says


Sep  9 08:12:23 localhost kernel: qla2xxx [:0c:00.0]-00fb:1: QLogic QLE2460 
- QLogic 4GB FC Single-Port PCI-E HBA for IBM System x.

And further down the log this  (this one does not find the LUNS)

Sep  9 08:12:24 localhost kernel: qla2xxx [:0c:00.0]-d001:1: Firmware dump 
saved to temp buffer (1/c90013d9e000).
Sep  9 08:12:24 localhost kernel: qla2xxx [:0c:00.0]-101e:1: Mailbox cmd 
timeout occurred, cmd=0x69, mb[0]=0x69. Scheduling ISP abort
Sep  9 08:12:24 localhost kernel: qla2xxx [:0c:00.0]-00af:1: Performing ISP 
error recovery - ha=88086ce53000.
Sep  9 08:12:24 localhost kernel: qla2xxx :0c:00.0: firmware: requesting 
ql2400_fw.bin
Sep  9 08:12:24 localhost kernel: qla2xxx [:0c:00.0]-0063:1: Failed to load 
firmware image (ql2400_fw.bin).
Sep  9 08:12:24 localhost kernel: qla2xxx [:0c:00.0]-0090:1: Fimware image 
unavailable.
Sep  9 08:12:24 localhost kernel: qla2xxx [:0c:00.0]-0091:1: Firmware 
images can be retrieved from: http://ldriver.qlogic.com/firmware/.
Sep  9 08:12:24 localhost firmware.sh[4794]: Cannot find  firmware file 
'ql2400_fw.bin'
Sep  9 08:12:26 localhost kernel: qla2xxx [:0c:00.0]-505f:1: Link is 
operational (4 Gbps).
Sep  9 08:12:26 localhost kernel: qla2xxx [:0c:00.0]-1020:1:  Failed 
mbx[0]=69, mb[1]=0, mb[2]=0, mb[3]=0, cmd=69 .
Sep  9 08:12:32 localhost kdump: mkdumprd: failed to make kdump initrd
Sep  9 08:12:32 localhost lldpad: config file failed to load,

And the one that does find the LUNS is

Sep 11 08:33:39 localhost kernel: qla2xxx [:0a:00.0]-00fb:0: QLogic QLE2460 
- SG-(X)PCIE1FC-QF4, Sun StorageTek 4 Gb FC Enterprise PCI-Express Single 
Channel.

And a little further down it discovers the LUNs

Sep 11 08:33:39 localhost kernel: qla2xxx [:0e:00.0]-505f:1: Link is 
operational (4 Gbps).
Sep 11 08:33:39 localhost kernel: scsi 1:0:0:0: Direct-Access DGC  LUNZ 
0216 PQ: 0 ANSI: 4
Sep 11 08:33:39 localhost kernel: scsi 1:0:0:1: Direct-Access DGC  RAID 
5   0216 PQ: 0 ANSI: 4
Sep 11 08:33:39 localhost kernel: scsi 1:0:0:2: Direct-Access DGC  RAID 
5   0216 PQ: 0 ANSI: 4
Sep 11 08:33:39 localhost kernel: scsi 1:0:0:3: Direct-Access DGC  RAID 
5   0216 PQ: 0 ANSI: 4
Sep 11 08:33:39 localhost kernel: scsi 1:0:0:5: Direct-Access DGC  RAID 
5   0216 PQ: 0 ANSI: 4
Sep 11 08:33:39 localhost kernel: scsi 1:0:0:11: Direct-Access DGC  
RAID 1   0216 PQ: 0 ANSI: 4
Sep 11 08:33:39 localhost kernel: scsi 1:0:0:51: Direct-Access DGC  
RAID 1   0216 PQ: 0 ANSI: 4


So, I am thinking maybe different firmware levels on these cards and the first 
one for IBM systems has different firmware?  As it seems to be throwing more 
errors.   Should I try to add the firmware?  And if so I know it goes in 
/lib/firmware but then the boot image needs to be updated I think and I don’t 
know how to do that… or am I barking up the wrong tree.

Thanks again for the help.



From: users-boun...@ovirt.org [mailto:users-boun...@ovirt.org] On Behalf Of 
Bill Dossett
Sent: 12 September 2014 08:35
To: Sander Grendelman
Cc: users@ovirt.org
Subject: Re: [ovirt-users] Fwd: RE: FC QLogic question

Hmmm… I’ve just gone to add a storage domain and in the admin portal and bingo 
bango, all my LUNs appeared.

Just wiping out everything from my last lab now.

Thanks everyone, I am not quite sure why it started working, but knowing that 
the ovirt-node was seeing the LUNs prompted me to continue.

Thanks all for the help.

Bill


From: sander.grendel...@gmail.commailto:sander.grendel...@gmail.com 
[mailto:sander.grendel...@gmail.com] On Behalf Of Sander Grendelman
Sent: 12 September 2014 08:28
To: Bill Dossett
Cc: users@ovirt.orgmailto:users@ovirt.org
Subject: Re: [ovirt-users] Fwd: RE: FC QLogic question

This could also be an selinux issue with FC on the node, I've hit that one 
before.
Especially since the luns do show up with multipath -ll
Could be as simple as flipping a sanlock related selinux boolean.

A quick test would be to boot in permissive mode.
See http://www.ovirt.org/Node_Troubleshooting#SELinux for some more information.






___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


[ovirt-users] Fwd: RE: FC QLogic question

2014-09-11 Thread Daniel Helgenberger











Hi Daniel,

I did a lsmod | grep qxl  and it returned nothing...  did you perhaps mean qla?

[root@ovirt141 ~]# lsmod | grep qla
qla2xxx   466573  56

I think that is the one

lsscsi does list all of the LUNs...  so that's working!

But nothing in ovirt engine when I try to add a storage domain.

Thanks again, I guess I will try and get a bit further with the installs I 
have...

Bill



-Original Message-
From: Daniel Helgenberger [mailto:daniel.helgenber...@m-box.de]
Sent: 11 September 2014 11:53
To: Bill Dossett
Subject: Re: [ovirt-users] FC QLogic question

On 11.09.2014 12:17, Bill Dossett wrote:
 Thanks for responding Daniel.
Never mind!
  I was wondering and thought the message may be irrelevant.  But yes,
 it doesn't appear to be working.  When I try to add a FC storage
 domain, no hosts are showing in the drop down box of hosts.  I have
 loaded the latests ovirtnode (3.4.something...) on two hosts now and
 neither are working.
Lets first try to isolate your FC issure. So you are using the latest 
ovirt-node? EL6? Keep in mind, the kernel is standard Linux and therefore 
should support Qlogic adapters out of the box (IMHO they are the best supported 
HBAs in Linux).
Please try a:
# lsmod  | grep qxl
to see whatever the qxl module is loaded. Keep in mind you can run into 
problems with other conrollers in your host, though I only had this with LSI 
SAS / FC controllers so far.

Further check on the FC switch if the N port is logged in and the zoning is 
correct.

Finally, check with
# lsscsi
if you can see the luns.

 Also having a lot of issues with configuring the network.  I configure
 an interface I can ping it, but it shows as unconfigured... on one host
 I have added it to the manager and I see the interfaces anyway, but on
 the second host I configure the eth0, I can ping it, it added to the
 ovirt engine but it shows no NICs.  Should I be using an earlier version 
 maybe?

 thanks again and best regards.


 *Bill Dossett*
 *Virtualisation Architect*
 *Pitney Bowes - Tech Central***
 6 Hercules Way, 600 Leavesden Park
 Watford, WD25 7GS
 Phone(+44) (0)1923 279353 | VOIP: 54739353
 Mobile(+44) (0)7775 908612
 bill.doss...@pb.com mailto:bill.doss...@pb.com www.pb.com
 http://www.pb.com/ *In Engineering?**Raise a ticket via Remedy
 Anywhere [HERE
 http://teamsite.inside.pb.com/sites/TCPPM/GES/Lists/Create%20Incident
 /newIncident.aspx]takes
 less than a minute*

 *ManageIQ User Guide available* [*HERE
 https://dl.dropbox.com/u/6219441/ManageIQ%20-%20User%20Manual%20PB%20
 v4.pdf*] *Check out the ManageIQ Blog for recent enhancements and
 information [HERE]
 http://teamsite.inside.pb.com/sites/TCPPM/GES/manageiq/default.aspx*

 On Sep 11, 2014, at 10:52 AM, Daniel Helgenberger
 daniel.helgenber...@m-box.de mailto:daniel.helgenber...@m-box.de wrote:

  Hello Bill,
 
  what are your problems acually? Is fibre channel not working (as expected)?
 
  The log snippet you provided is completly normal and the messages
  can safely be ignored. This is also unrelated to oVirt but rather
  CentOS (or the distro you are running) I suppose.
 
  Cheers!
 
  On 11.09.2014 10:51, Bill Dossett wrote:
  Hi,
 
  I seem to have run into a problem setting up ovirt with Fibre Channel 
  storage.
 
  I have a 2MBs SAN and qlogic cards in the nodes.  I used this same
  setup with RHEV about 6 months ago and it just worked, storage was
  there no problems.  Now booting with ovirtnode distro, I am seeing
 
  Sep  9 08:01:27 localhost kernel: qla2xxx [:0c:00.0]-0091:1:
  Firmware images can be retrieved from: http://ldriver.qlogic.com/firmware/.
 
  Sep  9 08:01:27 localhost kernel: scsi1 : qla2xxx
 
  Sep  9 08:01:27 localhost kernel: qla2xxx [:0c:00.0]-00fb:1:
  QLogic QLE2460
  - QLogic 4GB FC Single-Port PCI-E HBA for IBM System x.
 
  Sep  9 08:01:27 localhost kernel: qla2xxx [:0c:00.0]-00fc:1:
  ISP2432: PCIe (2.5GT/s x4) @ :0c:00.0 hdma+ host#=1 fw=4.00.26 (2).
 
  Sep  9 08:01:27 localhost kernel: ata_piix :00:1f.1: PCI INT A
  - GSI 16 (level, low) - IRQ 16
 
  I have googled the firmware error and it says some cards need a
  firmware flash and some don't.  These are pretty standard HBAs so I
  would think they would be supported out of the box, but maybe not.
  Looking at trying to add the firmware looked kind of complex - if I
  need to do this, does anyone know of a nice and easy howto guide on
  this.  I'm just getting started with ovirt and appreciate any help
 
 
  Thanks
 
  *Bill Dossett*
 
  *Virtualisation Architect*
 
  *Pitney Bowes - Tech Central***
 
  6 Hercules Way, 600 Leavesden Park
 
  Watford, WD25 7GS
 
  Phone(+44) (0)1923 279353 | VOIP: 54739353
 
  Mobile(+44) (0)7775 908612
 
  bill.doss...@pb.com mailto:bill.doss...@pb.com
  mailto:bill.doss...@pb.com
 
  www.pb.com http://www.pb.com http://www.pb.com/
 
  *In Engineering?**Raise a ticket via Remedy Anywhere [HERE
  

Re: [ovirt-users] Fwd: RE: FC QLogic question

2014-09-11 Thread Fabian Deutsch
Hey Bill and Daniel,

it would indeed be interesting - mandatory - to know if it's qla* or qxl (qxl 
is IMHO related to graphics).
Once we know that I'll see if the bug below fixes it.

I'd need your help to verify that we are also including all relevant 
dependencies.

Thanks
- fabian

- Original Message -
 
 
   
 
   
 
   
 
   
 
 
 Hi Daniel,
 
 I did a lsmod | grep qxl  and it returned nothing...  did you perhaps mean
 qla?
 
 [root@ovirt141 ~]# lsmod | grep qla
 qla2xxx   466573  56
 
 I think that is the one
 
 lsscsi does list all of the LUNs...  so that's working!
 
 But nothing in ovirt engine when I try to add a storage domain.
 
 Thanks again, I guess I will try and get a bit further with the installs I
 have...
 
 Bill
 
 
 
 -Original Message-
 From: Daniel Helgenberger [mailto:daniel.helgenber...@m-box.de]
 Sent: 11 September 2014 11:53
 To: Bill Dossett
 Subject: Re: [ovirt-users] FC QLogic question
 
 On 11.09.2014 12:17, Bill Dossett wrote:
  Thanks for responding Daniel.
 Never mind!
   I was wondering and thought the message may be irrelevant.  But yes,
  it doesn't appear to be working.  When I try to add a FC storage
  domain, no hosts are showing in the drop down box of hosts.  I have
  loaded the latests ovirtnode (3.4.something...) on two hosts now and
  neither are working.
 Lets first try to isolate your FC issure. So you are using the latest
 ovirt-node? EL6? Keep in mind, the kernel is standard Linux and therefore
 should support Qlogic adapters out of the box (IMHO they are the best
 supported HBAs in Linux).
 Please try a:
 # lsmod  | grep qxl
 to see whatever the qxl module is loaded. Keep in mind you can run into
 problems with other conrollers in your host, though I only had this with LSI
 SAS / FC controllers so far.
 
 Further check on the FC switch if the N port is logged in and the zoning is
 correct.
 
 Finally, check with
 # lsscsi
 if you can see the luns.
 
  Also having a lot of issues with configuring the network.  I configure
  an interface I can ping it, but it shows as unconfigured... on one host
  I have added it to the manager and I see the interfaces anyway, but on
  the second host I configure the eth0, I can ping it, it added to the
  ovirt engine but it shows no NICs.  Should I be using an earlier version
  maybe?
 
  thanks again and best regards.
 
 
  *Bill Dossett*
  *Virtualisation Architect*
  *Pitney Bowes - Tech Central***
  6 Hercules Way, 600 Leavesden Park
  Watford, WD25 7GS
  Phone(+44) (0)1923 279353 | VOIP: 54739353
  Mobile(+44) (0)7775 908612
  bill.doss...@pb.com mailto:bill.doss...@pb.com www.pb.com
  http://www.pb.com/ *In Engineering?**Raise a ticket via Remedy
  Anywhere [HERE
  http://teamsite.inside.pb.com/sites/TCPPM/GES/Lists/Create%20Incident
  /newIncident.aspx]takes
  less than a minute*
 
  *ManageIQ User Guide available* [*HERE
  https://dl.dropbox.com/u/6219441/ManageIQ%20-%20User%20Manual%20PB%20
  v4.pdf*] *Check out the ManageIQ Blog for recent enhancements and
  information [HERE]
  http://teamsite.inside.pb.com/sites/TCPPM/GES/manageiq/default.aspx*
 
  On Sep 11, 2014, at 10:52 AM, Daniel Helgenberger
  daniel.helgenber...@m-box.de mailto:daniel.helgenber...@m-box.de wrote:
 
   Hello Bill,
  
   what are your problems acually? Is fibre channel not working (as
   expected)?
  
   The log snippet you provided is completly normal and the messages
   can safely be ignored. This is also unrelated to oVirt but rather
   CentOS (or the distro you are running) I suppose.
  
   Cheers!
  
   On 11.09.2014 10:51, Bill Dossett wrote:
   Hi,
  
   I seem to have run into a problem setting up ovirt with Fibre Channel
   storage.
  
   I have a 2MBs SAN and qlogic cards in the nodes.  I used this same
   setup with RHEV about 6 months ago and it just worked, storage was
   there no problems.  Now booting with ovirtnode distro, I am seeing
  
   Sep  9 08:01:27 localhost kernel: qla2xxx [:0c:00.0]-0091:1:
   Firmware images can be retrieved from:
   http://ldriver.qlogic.com/firmware/.
  
   Sep  9 08:01:27 localhost kernel: scsi1 : qla2xxx
  
   Sep  9 08:01:27 localhost kernel: qla2xxx [:0c:00.0]-00fb:1:
   QLogic QLE2460
   - QLogic 4GB FC Single-Port PCI-E HBA for IBM System x.
  
   Sep  9 08:01:27 localhost kernel: qla2xxx [:0c:00.0]-00fc:1:
   ISP2432: PCIe (2.5GT/s x4) @ :0c:00.0 hdma+ host#=1 fw=4.00.26 (2).
  
   Sep  9 08:01:27 localhost kernel: ata_piix :00:1f.1: PCI INT A
   - GSI 16 (level, low) - IRQ 16
  
   I have googled the firmware error and it says some cards need a
   firmware flash and some don't.  These are pretty standard HBAs so I
   would think they would be supported out of the box, but maybe not.
   Looking at trying to add the firmware looked kind of complex - if I
   need to do this, does anyone know of a nice and easy howto guide on
   this.  I'm just getting started with ovirt and appreciate any help
  
  
   Thanks
  
   *Bill 

Re: [ovirt-users] Fwd: RE: FC QLogic question

2014-09-11 Thread Fabian Deutsch
Hey Daniel,

- Original Message -
 
 On 11.09.2014 14:09, Fabian Deutsch wrote:
  Hey Bill and Daniel,
 
  it would indeed be interesting - mandatory - to know if it's qla* or qxl
  (qxl is IMHO related to graphics).
  Once we know that I'll see if the bug below fixes it.
 Hello Fabian,
 indeed this was my mistake; while typing the mail I was reading on the
 spice website - so qla is the correct module of course! I already
 replied this to Bill but forgot to cc the list also ...
 
 Also, for the node iso it would make little sense to have the qlx module
 loaded. For completeness, I attach my answer to Bill.

Thanks :)

So to summarize: 
The adapter is recognized on Node?

But it does not show up in Engine?

Thanks
- fabian

 
  I'd need your help to verify that we are also including all relevant
  dependencies.
 
  Thanks
  - fabian
 
  - Original Message -
 
 On 11.09.2014 13:05, Bill Dossett wrote:
 
  Hi Daniel,
 
  I did a lsmod | grep qxl  and it returned nothing...  did you perhaps mean
  qla?
 
  [root@ovirt141 ~]# lsmod | grep qla
  qla2xxx   466573  56
 
  I think that is the one
 
 Indeed, sorry for that.
 
  lsscsi does list all of the LUNs...  so that's working!
 
 Great!
 
  But nothing in ovirt engine when I try to add a storage domain.
 
 Please check back in ovirt engine, with Hosts / Hardware info. You
 should see the adapter there, along with  WWN's ect.
 
 I am not quite sure, but AFAIK the node uses dm-multipath by default
 (for iscsi multipathing). So what does:
 # multipath -ll
 list?
 
 Thanks again, I guess I will try and get a bit further with the installs I
 have...
 
 Bill
 
 
 
 
 
 
 
 
 
 
 
  Hi Daniel,
 
  I did a lsmod | grep qxl  and it returned nothing...  did you perhaps mean
  qla?
 
  [root@ovirt141 ~]# lsmod | grep qla
  qla2xxx   466573  56
 
  I think that is the one
 
  lsscsi does list all of the LUNs...  so that's working!
 
  But nothing in ovirt engine when I try to add a storage domain.
 
  Thanks again, I guess I will try and get a bit further with the installs I
  have...
 
  Bill
 
 
 
  -Original Message-
  From: Daniel Helgenberger [mailto:daniel.helgenber...@m-box.de]
  Sent: 11 September 2014 11:53
  To: Bill Dossett
  Subject: Re: [ovirt-users] FC QLogic question
 
  On 11.09.2014 12:17, Bill Dossett wrote:
  Thanks for responding Daniel.
  Never mind!
   I was wondering and thought the message may be irrelevant.  But yes,
  it doesn't appear to be working.  When I try to add a FC storage
  domain, no hosts are showing in the drop down box of hosts.  I have
  loaded the latests ovirtnode (3.4.something...) on two hosts now and
  neither are working.
  Lets first try to isolate your FC issure. So you are using the latest
  ovirt-node? EL6? Keep in mind, the kernel is standard Linux and therefore
  should support Qlogic adapters out of the box (IMHO they are the best
  supported HBAs in Linux).
  Please try a:
  # lsmod  | grep qxl
  to see whatever the qxl module is loaded. Keep in mind you can run into
  problems with other conrollers in your host, though I only had this with
  LSI
  SAS / FC controllers so far.
 
  Further check on the FC switch if the N port is logged in and the zoning
  is
  correct.
 
  Finally, check with
  # lsscsi
  if you can see the luns.
 
  Also having a lot of issues with configuring the network.  I configure
  an interface I can ping it, but it shows as unconfigured... on one host
  I have added it to the manager and I see the interfaces anyway, but on
  the second host I configure the eth0, I can ping it, it added to the
  ovirt engine but it shows no NICs.  Should I be using an earlier version
  maybe?
 
  thanks again and best regards.
 
 
  *Bill Dossett*
  *Virtualisation Architect*
  *Pitney Bowes - Tech Central***
  6 Hercules Way, 600 Leavesden Park
  Watford, WD25 7GS
  Phone(+44) (0)1923 279353 | VOIP: 54739353
  Mobile(+44) (0)7775 908612
  bill.doss...@pb.com mailto:bill.doss...@pb.com www.pb.com
  http://www.pb.com/ *In Engineering?**Raise a ticket via Remedy
  Anywhere [HERE
  http://teamsite.inside.pb.com/sites/TCPPM/GES/Lists/Create%20Incident
  /newIncident.aspx]takes
  less than a minute*
 
  *ManageIQ User Guide available* [*HERE
  https://dl.dropbox.com/u/6219441/ManageIQ%20-%20User%20Manual%20PB%20
  v4.pdf*] *Check out the ManageIQ Blog for recent enhancements and
  information [HERE]
  http://teamsite.inside.pb.com/sites/TCPPM/GES/manageiq/default.aspx*
 
  On Sep 11, 2014, at 10:52 AM, Daniel Helgenberger
  daniel.helgenber...@m-box.de mailto:daniel.helgenber...@m-box.de
  wrote:
 
  Hello Bill,
 
  what are your problems acually? Is fibre channel not working (as
  expected)?
 
  The log snippet you provided is completly normal and the messages
  can safely be ignored. This is also unrelated to oVirt but rather
  CentOS (or the distro you are running) I suppose.
 
  Cheers!
 
  On 11.09.2014 10:51, Bill Dossett wrote:
  Hi,
 
  I seem 

Re: [ovirt-users] Fwd: RE: FC QLogic question

2014-09-11 Thread Daniel Helgenberger

On 11.09.2014 14:23, Fabian Deutsch wrote:
 Hey Daniel,

 - Original Message -
 On 11.09.2014 14:09, Fabian Deutsch wrote:
 Hey Bill and Daniel,

 it would indeed be interesting - mandatory - to know if it's qla* or qxl
 (qxl is IMHO related to graphics).
 Once we know that I'll see if the bug below fixes it.
 Hello Fabian,
 indeed this was my mistake; while typing the mail I was reading on the
 spice website - so qla is the correct module of course! I already
 replied this to Bill but forgot to cc the list also ...

 Also, for the node iso it would make little sense to have the qlx module
 loaded. For completeness, I attach my answer to Bill.
 Thanks :)

 So to summarize: 
 The adapter is recognized on Node?

 But it does not show up in Engine?
Bill has to tell, but I think so. I suggested to him this might be
because of dm-multipath, multipath.conf might blacklist here. My nodes
are plain EL6 and I have multipath enabled - but I do not know if
ovirt-node filters for dm-mp devnodes only by default (this would make
sense because of iscsi-multipathing in ovirt 3.4).

Maybe you can help out on this topic?

Thanks!

 Thanks
 - fabian

 I'd need your help to verify that we are also including all relevant
 dependencies.

 Thanks
 - fabian

 - Original Message -
 On 11.09.2014 13:05, Bill Dossett wrote:

 Hi Daniel,

 I did a lsmod | grep qxl  and it returned nothing...  did you perhaps mean
 qla?

 [root@ovirt141 ~]# lsmod | grep qla
 qla2xxx   466573  56

 I think that is the one
 Indeed, sorry for that.

 lsscsi does list all of the LUNs...  so that's working!
 Great!

 But nothing in ovirt engine when I try to add a storage domain.
 Please check back in ovirt engine, with Hosts / Hardware info. You
 should see the adapter there, along with  WWN's ect.

 I am not quite sure, but AFAIK the node uses dm-multipath by default
 (for iscsi multipathing). So what does:
 # multipath -ll
 list?

 Thanks again, I guess I will try and get a bit further with the installs I
 have...

 Bill










 Hi Daniel,

 I did a lsmod | grep qxl  and it returned nothing...  did you perhaps mean
 qla?

 [root@ovirt141 ~]# lsmod | grep qla
 qla2xxx   466573  56

 I think that is the one

 lsscsi does list all of the LUNs...  so that's working!

 But nothing in ovirt engine when I try to add a storage domain.

 Thanks again, I guess I will try and get a bit further with the installs I
 have...

 Bill



 -Original Message-
 From: Daniel Helgenberger [mailto:daniel.helgenber...@m-box.de]
 Sent: 11 September 2014 11:53
 To: Bill Dossett
 Subject: Re: [ovirt-users] FC QLogic question

 On 11.09.2014 12:17, Bill Dossett wrote:
 Thanks for responding Daniel.
 Never mind!
  I was wondering and thought the message may be irrelevant.  But yes,
 it doesn't appear to be working.  When I try to add a FC storage
 domain, no hosts are showing in the drop down box of hosts.  I have
 loaded the latests ovirtnode (3.4.something...) on two hosts now and
 neither are working.
 Lets first try to isolate your FC issure. So you are using the latest
 ovirt-node? EL6? Keep in mind, the kernel is standard Linux and therefore
 should support Qlogic adapters out of the box (IMHO they are the best
 supported HBAs in Linux).
 Please try a:
 # lsmod  | grep qxl
 to see whatever the qxl module is loaded. Keep in mind you can run into
 problems with other conrollers in your host, though I only had this with
 LSI
 SAS / FC controllers so far.

 Further check on the FC switch if the N port is logged in and the zoning
 is
 correct.

 Finally, check with
 # lsscsi
 if you can see the luns.

 Also having a lot of issues with configuring the network.  I configure
 an interface I can ping it, but it shows as unconfigured... on one host
 I have added it to the manager and I see the interfaces anyway, but on
 the second host I configure the eth0, I can ping it, it added to the
 ovirt engine but it shows no NICs.  Should I be using an earlier version
 maybe?

 thanks again and best regards.


 *Bill Dossett*
 *Virtualisation Architect*
 *Pitney Bowes - Tech Central***
 6 Hercules Way, 600 Leavesden Park
 Watford, WD25 7GS
 Phone(+44) (0)1923 279353 | VOIP: 54739353
 Mobile(+44) (0)7775 908612
 bill.doss...@pb.com mailto:bill.doss...@pb.com www.pb.com
 http://www.pb.com/ *In Engineering?**Raise a ticket via Remedy
 Anywhere [HERE
 http://teamsite.inside.pb.com/sites/TCPPM/GES/Lists/Create%20Incident
 /newIncident.aspx]takes
 less than a minute*

 *ManageIQ User Guide available* [*HERE
 https://dl.dropbox.com/u/6219441/ManageIQ%20-%20User%20Manual%20PB%20
 v4.pdf*] *Check out the ManageIQ Blog for recent enhancements and
 information [HERE]
 http://teamsite.inside.pb.com/sites/TCPPM/GES/manageiq/default.aspx*

 On Sep 11, 2014, at 10:52 AM, Daniel Helgenberger
 daniel.helgenber...@m-box.de mailto:daniel.helgenber...@m-box.de
 wrote:

 Hello Bill,

 what are your problems acually? Is fibre channel not working 

Re: [ovirt-users] Fwd: RE: FC QLogic question

2014-09-11 Thread Fabian Deutsch
- Original Message -
 
 On 11.09.2014 14:23, Fabian Deutsch wrote:
  Hey Daniel,
 
  - Original Message -
  On 11.09.2014 14:09, Fabian Deutsch wrote:
  Hey Bill and Daniel,
 
  it would indeed be interesting - mandatory - to know if it's qla* or qxl
  (qxl is IMHO related to graphics).
  Once we know that I'll see if the bug below fixes it.
  Hello Fabian,
  indeed this was my mistake; while typing the mail I was reading on the
  spice website - so qla is the correct module of course! I already
  replied this to Bill but forgot to cc the list also ...
 
  Also, for the node iso it would make little sense to have the qlx module
  loaded. For completeness, I attach my answer to Bill.
  Thanks :)
 
  So to summarize:
  The adapter is recognized on Node?
 
  But it does not show up in Engine?

 Bill has to tell, but I think so. I suggested to him this might be

Ah, I mixed you an Bill up in the thread.

 because of dm-multipath, multipath.conf might blacklist here. My nodes
 are plain EL6 and I have multipath enabled - but I do not know if
 ovirt-node filters for dm-mp devnodes only by default (this would make
 sense because of iscsi-multipathing in ovirt 3.4).

Well, we did some multipath changes lately. Let#s see what Bill says:

Bill,
what node are you using?
What does multipath -l say?
How does /dev/mapper/* and /dev/multipath.conf look like?

And is the disk appearing on Node and/or Engine?

Thanks!
fabian

 Maybe you can help out on this topic?
 
 Thanks!
 
  Thanks
  - fabian
 
  I'd need your help to verify that we are also including all relevant
  dependencies.
 
  Thanks
  - fabian
 
  - Original Message -
  On 11.09.2014 13:05, Bill Dossett wrote:
 
  Hi Daniel,
 
  I did a lsmod | grep qxl  and it returned nothing...  did you perhaps
  mean
  qla?
 
  [root@ovirt141 ~]# lsmod | grep qla
  qla2xxx   466573  56
 
  I think that is the one
  Indeed, sorry for that.
 
  lsscsi does list all of the LUNs...  so that's working!
  Great!
 
  But nothing in ovirt engine when I try to add a storage domain.
  Please check back in ovirt engine, with Hosts / Hardware info. You
  should see the adapter there, along with  WWN's ect.
 
  I am not quite sure, but AFAIK the node uses dm-multipath by default
  (for iscsi multipathing). So what does:
  # multipath -ll
  list?
 
  Thanks again, I guess I will try and get a bit further with the installs I
  have...
 
  Bill
 
   
 
   
 
   
 
   
 
 
  Hi Daniel,
 
  I did a lsmod | grep qxl  and it returned nothing...  did you perhaps
  mean
  qla?
 
  [root@ovirt141 ~]# lsmod | grep qla
  qla2xxx   466573  56
 
  I think that is the one
 
  lsscsi does list all of the LUNs...  so that's working!
 
  But nothing in ovirt engine when I try to add a storage domain.
 
  Thanks again, I guess I will try and get a bit further with the installs
  I
  have...
 
  Bill
 
 
 
  -Original Message-
  From: Daniel Helgenberger [mailto:daniel.helgenber...@m-box.de]
  Sent: 11 September 2014 11:53
  To: Bill Dossett
  Subject: Re: [ovirt-users] FC QLogic question
 
  On 11.09.2014 12:17, Bill Dossett wrote:
  Thanks for responding Daniel.
  Never mind!
   I was wondering and thought the message may be irrelevant.  But yes,
  it doesn't appear to be working.  When I try to add a FC storage
  domain, no hosts are showing in the drop down box of hosts.  I have
  loaded the latests ovirtnode (3.4.something...) on two hosts now and
  neither are working.
  Lets first try to isolate your FC issure. So you are using the latest
  ovirt-node? EL6? Keep in mind, the kernel is standard Linux and
  therefore
  should support Qlogic adapters out of the box (IMHO they are the best
  supported HBAs in Linux).
  Please try a:
  # lsmod  | grep qxl
  to see whatever the qxl module is loaded. Keep in mind you can run into
  problems with other conrollers in your host, though I only had this with
  LSI
  SAS / FC controllers so far.
 
  Further check on the FC switch if the N port is logged in and the zoning
  is
  correct.
 
  Finally, check with
  # lsscsi
  if you can see the luns.
 
  Also having a lot of issues with configuring the network.  I configure
  an interface I can ping it, but it shows as unconfigured... on one host
  I have added it to the manager and I see the interfaces anyway, but on
  the second host I configure the eth0, I can ping it, it added to the
  ovirt engine but it shows no NICs.  Should I be using an earlier
  version
  maybe?
 
  thanks again and best regards.
 
 
  *Bill Dossett*
  *Virtualisation Architect*
  *Pitney Bowes - Tech Central***
  6 Hercules Way, 600 Leavesden Park
  Watford, WD25 7GS
  Phone(+44) (0)1923 279353 | VOIP: 54739353
  Mobile(+44) (0)7775 908612
  bill.doss...@pb.com mailto:bill.doss...@pb.com www.pb.com
  http://www.pb.com/ *In Engineering?**Raise a ticket via Remedy
  Anywhere [HERE
  http://teamsite.inside.pb.com/sites/TCPPM/GES/Lists/Create%20Incident
  /newIncident.aspx]takes
  less 

Re: [ovirt-users] Fwd: RE: FC QLogic question

2014-09-11 Thread Daniel Helgenberger
Hi Sven!

Thanks for the heads up on the BZ. 

One question remains;
if the firmware is missing in oVirt node then how come Bill does the module 
loaded and all the LUNs with lsscsi?

Cheers,

-- 
Daniel Helgenberger 
m box bewegtbild GmbH 

P: +49/30/2408781-22
F: +49/30/2408781-10
ACKERSTR. 19 
D-10115 BERLIN 
www.m-box.de  www.monkeymen.tv 

 Am 11.09.2014 um 17:29 schrieb Sven Kieske s.kie...@mittwald.de:
 
 Sorry for top posting, but this thread goes way too long:
 
 please take a look at this BZ:
 
 https://bugzilla.redhat.com/show_bug.cgi?id=1063001
 
 I think this is your issue! I already posted to the
 original thread but somehow nobody seems to read my mail.
 
 On 11/09/14 14:34, Daniel Helgenberger wrote:
 
 On 11.09.2014 14:23, Fabian Deutsch wrote:
 Hey Daniel,
 
 - Original Message -
 On 11.09.2014 14:09, Fabian Deutsch wrote:
 Hey Bill and Daniel,
 
 it would indeed be interesting - mandatory - to know if it's qla* or qxl
 (qxl is IMHO related to graphics).
 Once we know that I'll see if the bug below fixes it.
 Hello Fabian,
 indeed this was my mistake; while typing the mail I was reading on the
 spice website - so qla is the correct module of course! I already
 replied this to Bill but forgot to cc the list also ...
 
 Also, for the node iso it would make little sense to have the qlx module
 loaded. For completeness, I attach my answer to Bill.
 Thanks :)
 
 So to summarize: 
 The adapter is recognized on Node?
 
 But it does not show up in Engine?
 Bill has to tell, but I think so. I suggested to him this might be
 because of dm-multipath, multipath.conf might blacklist here. My nodes
 are plain EL6 and I have multipath enabled - but I do not know if
 ovirt-node filters for dm-mp devnodes only by default (this would make
 sense because of iscsi-multipathing in ovirt 3.4).
 
 Maybe you can help out on this topic?
 
 Thanks!
 
 Thanks
 - fabian
 
 I'd need your help to verify that we are also including all relevant
 dependencies.
 
 Thanks
 - fabian
 
 - Original Message -
 On 11.09.2014 13:05, Bill Dossett wrote:
 
 Hi Daniel,
 
 I did a lsmod | grep qxl  and it returned nothing...  did you perhaps mean
 qla?
 
 [root@ovirt141 ~]# lsmod | grep qla
 qla2xxx   466573  56
 
 I think that is the one
 Indeed, sorry for that.
 
 lsscsi does list all of the LUNs...  so that's working!
 Great!
 
 But nothing in ovirt engine when I try to add a storage domain.
 Please check back in ovirt engine, with Hosts / Hardware info. You
 should see the adapter there, along with  WWN's ect.
 
 I am not quite sure, but AFAIK the node uses dm-multipath by default
 (for iscsi multipathing). So what does:
 # multipath -ll
 list?
 
 Thanks again, I guess I will try and get a bit further with the installs I
 have...
 
 Bill
 

 

 

 

 
 
 Hi Daniel,
 
 I did a lsmod | grep qxl  and it returned nothing...  did you perhaps 
 mean
 qla?
 
 [root@ovirt141 ~]# lsmod | grep qla
 qla2xxx   466573  56
 
 I think that is the one
 
 lsscsi does list all of the LUNs...  so that's working!
 
 But nothing in ovirt engine when I try to add a storage domain.
 
 Thanks again, I guess I will try and get a bit further with the installs 
 I
 have...
 
 Bill
 
 
 
 -Original Message-
 From: Daniel Helgenberger [mailto:daniel.helgenber...@m-box.de]
 Sent: 11 September 2014 11:53
 To: Bill Dossett
 Subject: Re: [ovirt-users] FC QLogic question
 
 On 11.09.2014 12:17, Bill Dossett wrote:
 Thanks for responding Daniel.
 Never mind!
 I was wondering and thought the message may be irrelevant.  But yes,
 it doesn't appear to be working.  When I try to add a FC storage
 domain, no hosts are showing in the drop down box of hosts.  I have
 loaded the latests ovirtnode (3.4.something...) on two hosts now and
 neither are working.
 Lets first try to isolate your FC issure. So you are using the latest
 ovirt-node? EL6? Keep in mind, the kernel is standard Linux and therefore
 should support Qlogic adapters out of the box (IMHO they are the best
 supported HBAs in Linux).
 Please try a:
 # lsmod  | grep qxl
 to see whatever the qxl module is loaded. Keep in mind you can run into
 problems with other conrollers in your host, though I only had this with
 LSI
 SAS / FC controllers so far.
 
 Further check on the FC switch if the N port is logged in and the zoning
 is
 correct.
 
 Finally, check with
 # lsscsi
 if you can see the luns.
 
 Also having a lot of issues with configuring the network.  I configure
 an interface I can ping it, but it shows as unconfigured... on one host
 I have added it to the manager and I see the interfaces anyway, but on
 the second host I configure the eth0, I can ping it, it added to the
 ovirt engine but it shows no NICs.  Should I be using an earlier version
 maybe?
 
 thanks again and best regards.
 
 
 *Bill Dossett*
 *Virtualisation Architect*
 *Pitney Bowes - Tech Central***
 6 Hercules Way, 600 Leavesden Park
 Watford, WD25 7GS
 Phone(+44) (0)1923 

Re: [ovirt-users] Fwd: RE: FC QLogic question

2014-09-11 Thread Sven Kieske
I really don't know, maybe it got silently fixed and the BZ was not
updated.

Just another reason _not_ to use node (sorry fabian) imho.

On 11/09/14 18:16, Daniel Helgenberger wrote:
 Hi Sven!
 
 Thanks for the heads up on the BZ. 
 
 One question remains;
 if the firmware is missing in oVirt node then how come Bill does the module 
 loaded and all the LUNs with lsscsi?
 
 Cheers,
 

-- 
Mit freundlichen Grüßen / Regards

Sven Kieske

Systemadministrator
Mittwald CM Service GmbH  Co. KG
Königsberger Straße 6
32339 Espelkamp
T: +49-5772-293-100
F: +49-5772-293-333
https://www.mittwald.de
Geschäftsführer: Robert Meyer
St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] Fwd: RE: FC QLogic question

2014-09-11 Thread Sven Kieske
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 11.09.2014 19:20, Douglas Schilling Landgraf wrote:
 I do believe we are improving all components of oVirt release
 after release, including oVirt Node. Discouraging people to use it
 (or any other component) won't help the project. We are here to
 listen the feedback from users to improve the software and
 sometimes adding new features based in their requests and
 suggestions. I know you are a very active contributor in this
 project and thanks for that, if you still have issues in the node
 please keep the bugzillas coming.

Sorry for getting Off-Topic:
Well yeah, I see lots of improvement, also in the node project.

But when people ask about what do you recommend for production
I really can't call for node (yet).

Most deployments tend to need customization specific to the site
where the deployment takes place and in the past it was very
hard to alter node for your personal needs (that's also not the
projects main goal imho).

I don't want to discourage anyone from testing node, but I won't
recommend it for production, as I wouldn't recommend fedora
(and breakage has happened for people using fedora, just search this
mailing list).

I really appreciate all the work that is done and I see many
improvements, but when it comes to giving advice, some developers
are a little naive, I can even make a list:

someone has a problem in production:
yeah this is fixed in $BETA (or $ALPHA), just try it out (without
even mentioning that it's a beta release!)
you could easily backport the fix yourself, it's just a one liner
you need to alter config param $X (not mentioning this param
gets overwritten on upgrades)
and so on.

I know these are all just good intentions, but I think
some people seem to think those people asking are all just
running test environments, which is plainly wrong, or I don't
know what they think, maybe that it's already robust enough.

the project has matured so much, you should keep in mind
people might use it to run their business/store sensitive
data/host very critical machines on top of ovirt.

this is great, but this also comes with some kind of
responsibility on the side of a developer. At least
that's my opinion when I develop stuff (not such
huge projects ;) )

I know there are labels on every open source project
and in all software licenses like use at your own risk.

But there is some moral responsibility for stuff you do
you can't talk away with some legal paper saying your
not responsible for what you do (code) or advice you
give.

Maybe you should just ask yourself sometimes:

would I do myself what I just advocated
if I have really sensible data in my environment?

Anyway, keep up the good work! I know it's
all good intentions and of course every developer
wants his new features getting used. :)

Thanks

Sven

-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (MingW32)

iQGcBAEBAgAGBQJUEiDjAAoJEAq0kGAWDrqlNzsL/1EWhurbdEgt1jhBb/stoNhH
KnOANZyYCYUoUttGxG3h3KxM0wRA1mfpSO+yOtVYd/bfclu0M6dy5oRpRw4Q3m2U
VGVKqyDKsuQSaYX9CGOY90sYyO8DCt+KV04IN/XntsSpudF5Y+V7EqgBAaGqSS3N
ETNL9Jru91pdnJ+ThXtdEsp05DXg7pYYKTRFJ3mCEQOsD2AkcUlz91tQ2h7tC6ZF
KPjdRLK2iwQR/ssKduPdirlJ/jxW1K2JaBWUJx9OLLFLpm7D6Ea87beIQwGqX27S
jSk1lDm3MGJbK2cuKxZ8rDUnoJFa0DFkUi1pxp6l9IheOirXrvHcW57WkdUkZTb8
7fAeDMpFG3fny0+duAANneMI05NtA9hPl1Vnn6QBnp+nb7GET5JhcEpdgPe17h+W
hCBdgfN1OveLscN8E8hGrV9YKKUxkJkFAcXhgxOrTW23z6rwwYDG46t630xTSgIp
g401Zs7wyHZVb2FEZRZCuSBK07vcJI/b1nnG7OpS8A==
=A8rd
-END PGP SIGNATURE-
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users