BUG: My nfs server: centos 7.2, nfs-utils 1:1.3.0-0.21.el7
cloudstack error: 2016-01-20 09:35:47,951 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] (pool-56-thread-1:ctx-c962141a) (logid:e48a8f70) Faild to get url: http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2, due to java.io.IOException: access denied NFS error: Jan 20 16:07:07 rgw-lb01 nfsdcltrack[32306]: sqlite_insert_client: insert statement prepare failed: table clients has 2 columns but 3 values were supplied Jan 20 16:13:43 rgw-lb01 nfsdcltrack[32528]: sqlite_insert_client: insert statement prepare failed: table clients has 2 columns but 3 values were supplied Jan 20 16:22:49 rgw-lb01 nfsdcltrack[393]: sqlite_insert_client: insert statement prepare failed: table clients has 2 columns but 3 values were supplied https://bugzilla.redhat.com//show_bug.cgi?id=1285097 2016-01-20 10:47 GMT+03:00 Юрий Карпель <yu...@karpel.su>: > Test stend: cloudstack 4.7, kvm, ceph (centos7) > > > > Deploying two rgw civetweb + server nfs: > > > > [client.rgw.srv-rgw01] > > rgw print continue = false > > host = srv-rgw01 > > rgw frontends = civetweb port=8080 > > rgw_socket_path = /tmp/radosgw.sock > > > > [client.rgw.srv-rgw02] > > rgw print continue = false > > host = srv-rgw01 > > rgw frontends = civetweb port=8080 > > rgw_socket_path = /tmp/radosgw.sock > > > > swift test: > > [ceph@ceph-adm ~]$ swift -A http://ceph-rgw.test.bst.ru:8080/auth/v1.0/ -U > cloudstack:swift -K 'KBDbLt3DJ9hhMCVuPDfX1TwtLVywa2NVtO6ODBnu' list > > images > > [ceph@ceph-adm ~]$ > > [ceph@ceph-adm ~]$ swift upload images ceph.log > > ceph.log > > [ceph@ceph-adm ~]$ swift stat images > > Account: v1 > > Container: images > > Objects: 1 > > Bytes: 96355 > > Read ACL: > > Write ACL: > > Sync To: > > Sync Key: > > Accept-Ranges: bytes > > X-Storage-Policy: default-placement > > X-Container-Bytes-Used-Actual: 98304 > > X-Timestamp: 1453193667.00000 > > X-Trans-Id: > tx000000000000000000208-00569f2d4c-4395-default > > Content-Type: text/plain; charset=utf-8 > > [ceph@ceph-adm ~]$ > > > > > > Add to cloudstack: > > Name: Images > > Provider: Swift > > URL: http://ceph-rgw.test.bst.ru:8080/auth/v1.0/ > > Account:cloudstack > > Username: swift > > > > Log: > > > > 2016-01-20 09:35:37,688 DEBUG [c.c.a.ApiServlet] > (catalina-exec-20:ctx-72c3e20f ctx-320404d7) (logid:9e2762af) ===END=== > 192.168.7.29 -- GET > > command=addImageStore&response=json&name=Images&provider=Swift&url=http%3A%2F% > 2Fceph-rgw.test.bst.ru <http://2fceph-rgw.test.bst.ru/> > %3A8080%2Fauth%2Fv1.0%2F&details%5B0%5D.key=account&details%5B0%5D.value=cloudstack&details%5B1%5D.key=username&details%5B1%5D.value=swift&details%5B2%5D.key=key&details%5B2%5D.value=KBDbLt3DJ9hhMCVuPDfX1TwtLVywa2NVtO6ODBnu&_=1453271737545 > > 2016-01-20 09:35:43,684 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager > Timer:ctx-96868b17) (logid:73617e92) Resetting hosts suitable for reconnect > > 2016-01-20 09:35:43,689 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager > Timer:ctx-96868b17) (logid:73617e92) Completed resetting hosts suitable for > reconnect > > 2016-01-20 09:35:43,689 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager > Timer:ctx-96868b17) (logid:73617e92) Acquiring hosts for clusters already > owned by this management server > > 2016-01-20 09:35:43,690 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager > Timer:ctx-96868b17) (logid:73617e92) Completed acquiring hosts for clusters > already owned by this management server > > 2016-01-20 09:35:43,690 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager > Timer:ctx-96868b17) (logid:73617e92) Acquiring hosts for clusters not owned > by any management server > > 2016-01-20 09:35:43,692 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager > Timer:ctx-96868b17) (logid:73617e92) Completed acquiring hosts for clusters > not owned by any management server > > 2016-01-20 09:35:47,073 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > (AsyncJobMgr-Heartbeat-1:ctx-2c2a88d4) (logid:053e95d5) Begin cleanup > expired async-jobs > > 2016-01-20 09:35:47,100 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > (AsyncJobMgr-Heartbeat-1:ctx-2c2a88d4) (logid:053e95d5) End cleanup expired > async-jobs > > 2016-01-20 09:35:47,906 INFO [o.a.c.s.r.NfsSecondaryStorageResource] > (pool-56-thread-1:ctx-c962141a) (logid:e48a8f70) Determined host > rgw-lb01.test.bst.ru <http://rgw-lb01.cloud.bstelecom.ru/> corresponds to > IP 10.30.15.2 > > 2016-01-20 09:35:47,906 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] > (pool-56-thread-1:ctx-c962141a) (logid:e48a8f70) Mounting device with > nfs-style path of 10.30.15.2:/nfs > > 2016-01-20 09:35:47,906 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] > (pool-56-thread-1:ctx-c962141a) (logid:e48a8f70) making available > /var/cloudstack/mnt/secStorage/c6c692a0-265d-3109-93d4-f0f65f524d84 on > nfs://rgw-lb01.test.bst.ru/nfs > > 2016-01-20 09:35:47,906 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] > (pool-56-thread-1:ctx-c962141a) (logid:e48a8f70) local folder for mount > will be /var/cloudstack/mnt/secStorage/c6c692a0-265d-3109-93d4-f0f65f524d84 > > 2016-01-20 09:35:47,909 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] > (pool-56-thread-1:ctx-c962141a) (logid:e48a8f70) Executing: sudo mount > > 2016-01-20 09:35:47,940 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] > (pool-56-thread-1:ctx-c962141a) (logid:e48a8f70) Execution is successful. > > 2016-01-20 09:35:47,945 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] > (pool-56-thread-1:ctx-c962141a) (logid:e48a8f70) Some device already > mounted at > /var/cloudstack/mnt/secStorage/c6c692a0-265d-3109-93d4-f0f65f524d84, no > need to mount nfs://rgw-lb01test.bst.ru/nfs > > 2016-01-20 09:35:47,951 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] > (pool-56-thread-1:ctx-c962141a) (logid:e48a8f70) Faild to get url: > http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2, > due to java.io.IOException: access denied > > > > [c.c.s.StatsCollector] (StatsCollector-4:ctx-4e2b563c) (logid:0523bff9) > There is no secondary storage VM for secondary storage host Images > > > > Error: > > (logid:e48a8f70) Faild to get url: > http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2, > due to java.io.IOException: access denied > > > Management server It has access to the file > http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 > > > [root@srv-mgm01 ~]# wget > http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 > > Resolving cloudstack.apt-get.eu (cloudstack.apt-get.eu)... 185.27.175.74, > 2a00:f10:121:400:4b8:8ff:fe00:496 > > Connecting to cloudstack.apt-get.eu > (cloudstack.apt-get.eu)|185.27.175.74|:80... > connected. > > HTTP request sent, awaiting response... 200 OK > > 2016-01-19 21:27 GMT+03:00 ilya <ilya.mailing.li...@gmail.com>: > >> Юрий, вчем проблема? >> >> >> On 1/19/16 8:04 AM, Юрий Карпель wrote: >> > Cloudstack 4.7, kvm, centos7 >> > >> > Test cases: >> > >> > Swift (rados-gw): >> > [ceph@ceph-adm ~]$ swift -A http://10.30.15.15:8080/auth/v1.0/ -U >> > cloudstack:swift -K 'PiBNP6C4zRVNDGdM761APVMZm8Vozd20RE2tBm6d' list >> > images >> > [ceph@ceph-adm ~]$ >> > >> > >> > Add to cloudstack: >> > >> > 2016-01-19 19:01:06,146 INFO [o.a.c.s.d.l.SwiftImageStoreLifeCycleImpl] >> > (catalina-exec-22:ctx-44d59d94 ctx-0289da0c) (logid:fdf404d5) Trying to >> add >> > a swift store at http://10.30.15.15:8080/auth/v1.0/ in data center null >> > 2016-01-19 19:01:06,271 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl] >> > (catalina-exec-22:ctx-44d59d94 ctx-0289da0c) (logid:fdf404d5) template >> 3 is >> > already in store:82, type:Image >> > 2016-01-19 19:01:06,365 INFO [o.a.c.s.i.d.TemplateDataStoreDaoImpl] >> > (catalina-exec-22:ctx-44d59d94 ctx-0289da0c) (logid:fdf404d5) Duplicate >> 0 >> > template cache store records to region store >> > 2016-01-19 19:01:06,367 INFO [o.a.c.s.i.d.SnapshotDataStoreDaoImpl] >> > (catalina-exec-22:ctx-44d59d94 ctx-0289da0c) (logid:fdf404d5) Duplicate >> 0 >> > snapshot cache store records to region store >> > 2016-01-19 19:01:06,378 INFO [o.a.c.s.i.d.VolumeDataStoreDaoImpl] >> > (catalina-exec-22:ctx-44d59d94 ctx-0289da0c) (logid:fdf404d5) Duplicate >> 0 >> > volume cache store records to region store >> > 2016-01-19 19:01:06,383 DEBUG [c.c.a.ApiServlet] >> > (catalina-exec-22:ctx-44d59d94 ctx-0289da0c) (logid:fdf404d5) ===END=== >> > 192.168.7.29 -- GET >> > >> command=addImageStore&response=json&name=Images&provider=Swift&url=http%3A%2F%2F10.30.15.15%3A8080%2Fauth%2Fv1.0%2F&details%5B0%5D.key=account&details%5B0%5D.value=cloudstack%3Aswift&details%5B1%5D.key=username&details%5B1%5D.value=cloudstack%3Aswift&details%5B2%5D.key=key&details%5B2%5D.value=PiBNP6C4zRVNDGdM761APVMZm8Vozd20RE2tBm6d&_=1453219265296 >> > 2016-01-19 19:01:07,077 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] >> > (AsyncJobMgr-Heartbeat-1:ctx-2dfb716a) (logid:44c21614) Begin cleanup >> > expired async-jobs >> > 2016-01-19 19:01:07,102 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] >> > (AsyncJobMgr-Heartbeat-1:ctx-2dfb716a) (logid:44c21614) End cleanup >> expired >> > async-jobs >> > 2016-01-19 19:01:07,153 DEBUG >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] >> > (RouterStatusMonitor-1:ctx-aec40e3d) (logid:b3518114) Found 0 routers to >> > update status. >> > 2016-01-19 19:01:07,159 DEBUG >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] >> > (RouterStatusMonitor-1:ctx-aec40e3d) (logid:b3518114) Found 0 VPC >> networks >> > to update Redundant State. >> > 2016-01-19 19:01:07,165 DEBUG >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] >> > (RouterStatusMonitor-1:ctx-aec40e3d) (logid:b3518114) Found 0 networks >> to >> > update RvR status. >> > 2016-01-19 19:01:07,290 DEBUG >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] >> > (RouterStatusMonitor-1:ctx-dbe5f52c) (logid:86f8be40) Found 0 routers to >> > update status. >> > 2016-01-19 19:01:07,296 DEBUG >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] >> > (RouterStatusMonitor-1:ctx-dbe5f52c) (logid:86f8be40) Found 0 VPC >> networks >> > to update Redundant State. >> > 2016-01-19 19:01:07,301 DEBUG >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] >> > (RouterStatusMonitor-1:ctx-dbe5f52c) (logid:86f8be40) Found 0 networks >> to >> > update RvR status. >> > 2016-01-19 19:01:08,921 DEBUG [c.c.a.m.AgentManagerImpl] >> > (AgentManager-Handler-12:null) (logid:) Ping from 31( >> > srv-kvm01.cloud.bstelecom.ru) >> > 2016-01-19 19:01:08,921 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] >> > (AgentManager-Handler-12:null) (logid:) Process host VM state report >> from >> > ping process. host: 31 >> > 2016-01-19 19:01:08,921 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] >> > (AgentManager-Handler-12:null) (logid:) Process VM state report. host: >> 31, >> > number of records in report: 0 >> > 2016-01-19 19:01:08,930 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] >> > (AgentManager-Handler-12:null) (logid:) Done with process of VM state >> > report. host: 31 >> > 2016-01-19 19:01:13,569 DEBUG [c.c.h.d.HostDaoImpl] >> (ClusteredAgentManager >> > Timer:ctx-d77aa20b) (logid:efb2a365) Resetting hosts suitable for >> reconnect >> > 2016-01-19 19:01:13,575 DEBUG [c.c.h.d.HostDaoImpl] >> (ClusteredAgentManager >> > Timer:ctx-d77aa20b) (logid:efb2a365) Completed resetting hosts suitable >> for >> > reconnect >> > 2016-01-19 19:01:13,575 DEBUG [c.c.h.d.HostDaoImpl] >> (ClusteredAgentManager >> > Timer:ctx-d77aa20b) (logid:efb2a365) Acquiring hosts for clusters >> already >> > owned by this management server >> > 2016-01-19 19:01:13,576 DEBUG [c.c.h.d.HostDaoImpl] >> (ClusteredAgentManager >> > Timer:ctx-d77aa20b) (logid:efb2a365) Completed acquiring hosts for >> clusters >> > already owned by this management server >> > 2016-01-19 19:01:13,576 DEBUG [c.c.h.d.HostDaoImpl] >> (ClusteredAgentManager >> > Timer:ctx-d77aa20b) (logid:efb2a365) Acquiring hosts for clusters not >> owned >> > by any management server >> > 2016-01-19 19:01:13,578 DEBUG [c.c.h.d.HostDaoImpl] >> (ClusteredAgentManager >> > Timer:ctx-d77aa20b) (logid:efb2a365) Completed acquiring hosts for >> clusters >> > not owned by any management server >> > 2016-01-19 19:01:16,584 INFO [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) Determined host >> 10.30.15.2 >> > corresponds to IP 10.30.15.2 >> > 2016-01-19 19:01:16,584 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) Mounting device with >> > nfs-style path of 10.30.15.2:/nfs >> > 2016-01-19 19:01:16,584 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) making available >> > /var/cloudstack/mnt/secStorage/c6c692a0-265d-3109-93d4-f0f65f524d84 on >> > nfs://10.30.15.2/nfs >> > 2016-01-19 19:01:16,585 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) local folder for mount >> > will be >> /var/cloudstack/mnt/secStorage/c6c692a0-265d-3109-93d4-f0f65f524d84 >> > 2016-01-19 19:01:16,585 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) Executing: sudo mount >> > 2016-01-19 19:01:16,617 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) Execution is >> successful. >> > 2016-01-19 19:01:16,619 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) Make cmdline call to >> mount >> > 10.30.15.2:/nfs at >> > /var/cloudstack/mnt/secStorage/c6c692a0-265d-3109-93d4-f0f65f524d84 >> based >> > on uri nfs://10.30.15.2/nfs >> > 2016-01-19 19:01:16,620 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) Executing: sudo mount >> -t >> > nfs 10.30.15.2:/nfs >> > /var/cloudstack/mnt/secStorage/c6c692a0-265d-3109-93d4-f0f65f524d84 >> > 2016-01-19 19:01:16,686 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) Execution is >> successful. >> > 2016-01-19 19:01:16,689 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) Successfully mounted >> > 10.30.15.2:/nfs at >> > /var/cloudstack/mnt/secStorage/c6c692a0-265d-3109-93d4-f0f65f524d84 >> > 2016-01-19 19:01:16,689 DEBUG >> [o.a.c.s.r.LocalNfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) Executing: sudo chmod >> 777 >> > /var/cloudstack/mnt/secStorage/c6c692a0-265d-3109-93d4-f0f65f524d84 >> > 2016-01-19 19:01:16,708 DEBUG >> [o.a.c.s.r.LocalNfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) Execution is >> successful. >> > 2016-01-19 19:01:16,711 DEBUG >> [o.a.c.s.r.LocalNfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) Successfully set 777 >> > permission for >> > /var/cloudstack/mnt/secStorage/c6c692a0-265d-3109-93d4-f0f65f524d84 >> > 2016-01-19 19:01:16,714 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) snapshots already >> exists >> > on secondary storage, and is mounted at >> > /var/cloudstack/mnt/secStorage/c6c692a0-265d-3109-93d4-f0f65f524d84 >> > 2016-01-19 19:01:16,714 INFO [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) snapshots directory >> > created/exists on Secondary Storage. >> > 2016-01-19 19:01:16,715 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) volumes already exists >> on >> > secondary storage, and is mounted at >> > /var/cloudstack/mnt/secStorage/c6c692a0-265d-3109-93d4-f0f65f524d84 >> > 2016-01-19 19:01:16,715 INFO [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) volumes directory >> > created/exists on Secondary Storage. >> > 2016-01-19 19:01:16,720 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] >> > (pool-38-thread-1:ctx-b93abb9c) (logid:a42cd353) Faild to get url: >> > http://cloudstack.apt-* >> get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 >> > <http://get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2>, >> due to >> > java.io.IOException: access denied* >> > 2016-01-19 19:01:17,076 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] >> > (AsyncJobMgr-Heartbeat-1:ctx-9e72fbdd) (logid:6667a472) Begin cleanup >> > expired async-jobs >> > 2016-01-19 19:01:17,100 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] >> > (AsyncJobMgr-Heartbeat-1:ctx-9e72fbdd) (logid:6667a472) End cleanup >> expired >> > async-jobs >> > >> > >
