On Tue, Feb 9, 2016 at 9:45 AM, Paul Groeneweg | Pazion <p...@pazion.nl> wrote:
> I have added 2 logs from 2 hosts > > Both the hosts are just additional host: 2016-02-07 20:32:30 DEBUG otopi.plugins.otopi.dialog.human human.queryString:156 query OVEHOSTED_ADDITIONAL_HOST 2016-02-07 20:32:30 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:219 DIALOG:SEND The specified storage location already contains a data domain. Is this an additional host setup (Yes, No)[Yes]? 2016-02-07 20:32:37 INFO otopi.plugins.ovirt_hosted_engine_setup.storage.storage storage._handleHostId:249 Installing on additional host 2016-02-05 23:47:05 DEBUG otopi.plugins.otopi.dialog.human human.queryString:156 query OVEHOSTED_ADDITIONAL_HOST 2016-02-05 23:47:05 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:219 DIALOG:SEND The specified storage location already contains a data domain. Is this an additional host setup (Yes, No)[Yes]? 2016-02-05 23:47:07 INFO otopi.plugins.ovirt_hosted_engine_setup.storage.storage storage._handleHostId:123 Installing on additional host Did you redeployed also the engine VM on a fresh storage domain or di you just run 'hosted-engine --deploy' again on all the hosts? If it's just an upgrade you don't have to run 'hosted-engine --deploy' again. Please follow this guide: http://www.ovirt.org/Hosted_Engine_Howto#Upgrade_Hosted_Engine > > Op di 9 feb. 2016 om 09:07 schreef Simone Tiraboschi <stira...@redhat.com > >: > >> On Mon, Feb 8, 2016 at 11:25 PM, Paul Groeneweg | Pazion <p...@pazion.nl> >> wrote: >> >>> >>> I found vdsClient and ran a few commands. >>> >>> [root@geisha-1 ~]# vdsClient -s 0 getStorageDomainInfo >>> 88b69eba-ef4f-4dbe-ba53-20dadd424d0e >>> >>> uuid = 88b69eba-ef4f-4dbe-ba53-20dadd424d0e >>> >>> version = 3 >>> >>> role = Master >>> >>> remotePath = hostedstorage.pazion.nl:/opt/hosted-engine >>> >>> type = NFS >>> >>> class = Data >>> >>> pool = ['499b208c-9de9-4a2a-97de-30f410b4e6d4'] >>> >>> name = hostedengine_nfs >>> >>> >>> >> Can you please attach hosted-engine-setup logs? at the end of the setup >> procedure the hosted-engine setup shouldn't b attached to any storagePool. >> >> >>> [root@geisha-1 ~]# vdsClient -s 0 connectStoragePool >>> 499b208c-9de9-4a2a-97de-30f410b4e6d4 >>> >>> Error using command: list index out of range >>> >>> >>> ... >>> >>> >>> [root@geisha-1 ~]# vdsClient -s 0 getStoragePoolInfo >>> 499b208c-9de9-4a2a-97de-30f410b4e6d4 >>> >>> Unknown pool id, pool not connected: >>> ('499b208c-9de9-4a2a-97de-30f410b4e6d4',) >>> >>> >>> [root@geisha-1 ~]# vdsClient -s 0 getConnectedStoragePoolsList >>> >>> 00000002-0002-0002-0002-000000000385 >>> >>> >>> Can I manually create the missing storage pool which is missing with: >>> >>> vdsClient -s 0 createStoragePool 499b208c-9de9-4a2a-97de-30f410b4e6d4 >>> hostedengine_nfs 88b69eba-ef4f-4dbe-ba53-20dadd424d0e >>> >>> If this would be the solution, I still wonder why there is no existing >>> storagePool for the storageDomain. >>> >>> >>> >>> Op ma 8 feb. 2016 om 22:54 schreef Paul Groeneweg | Pazion < >>> p...@pazion.nl>: >>> >>>> I still keep getting >>>> >>>> The Hosted Engine Storage Domain doesn't exist. It should be imported >>>> into the setup. >>>> >>>> I went digging a little further and found this in the engine.log >>>> >>>> 2016-02-08 22:30:25,083 INFO >>>> >>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] >>>> (org.ovirt.thread.pool-8-thread-41) [] FINISH, >>>> HSMGetStorageDomainInfoVDSCommand, return: >>>> <StorageDomainStatic:{name='hostedengine_nfs', >>>> id='88b69eba-ef4f-4dbe-ba53-20dadd424d0e'}, >>>> 499b208c-9de9-4a2a-97de-30f410b4e6d4>, log id: 56723427 >>>> 2016-02-08 22:30:25,083 INFO >>>> [org.ovirt.engine.core.bll.storage.GetExistingStorageDomainListQuery] >>>> (org.ovirt.thread.pool-8-thread-41) [] FINISH, >>>> GetExistingStorageDomainListQuery, log id: 6ca437b1 >>>> 2016-02-08 22:30:25,084 INFO >>>> [org.ovirt.engine.core.bll.ImportHostedEngineStorageDomainCommand] >>>> (org.ovirt.thread.pool-8-thread-41) [364719ce] Lock Acquired to object >>>> 'EngineLock:{exclusiveLocks='[]', sharedLocks='null'}' >>>> 2016-02-08 22:30:25,102 WARN >>>> [org.ovirt.engine.core.bll.ImportHostedEngineStorageDomainCommand] >>>> (org.ovirt.thread.pool-8-thread-41) [364719ce] CanDoAction of action >>>> 'ImportHostedEngineStorageDomain' failed for user SYSTEM. Reasons: >>>> VAR__ACTION__ADD,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_NOT_EXIST >>>> >>>> So it looks like it can't find the storage domain. When I check >>>> vdsm.log I found an error: >>>> Thread-711902::ERROR::2016-02-08 >>>> 21:17:01,240::dispatcher::76::Storage.Dispatcher::(wrapper) {'status': >>>> {'message': "Unknown pool id, pool not connected: >>>> ('499b208c-9de9-4a2a-97de-30f410b4e6d4',)", 'code': 309}} >>>> >>>> It looks like 499b208c-9de9-4a2a-97de-30f410b4e6d4 is the id it is also >>>> trying to connect to in the engine.log. >>>> >>>> I did found https://bugzilla.redhat.com/show_bug.cgi?id=1269768 and >>>> https://bugzilla.redhat.com/show_bug.cgi?id=1294457 >>>> >>>> Can it be I have setup another DC with a master iscsi storage which is >>>> now offline? ( 1 DC with Cluster, VMs with hosted engine and master >>>> storage FC all up and running. ) >>>> >>>> Not sure where to go from here. Can I somehow restart this storage >>>> domain / do a new hosted-engine deploy with different settings? >>>> >>>> >>>> Op ma 8 feb. 2016 om 12:05 schreef Paul Groeneweg | Pazion < >>>> p...@pazion.nl>: >>>> >>>>> Op ma 8 feb. 2016 om 11:28 schreef Simone Tiraboschi < >>>>> stira...@redhat.com>: >>>>> >>>>>> On Mon, Feb 8, 2016 at 10:55 AM, Paul Groeneweg | Pazion < >>>>>> p...@pazion.nl> wrote: >>>>>> >>>>>>> Thanks for your response. >>>>>>> >>>>>>> I have reverted the commented entries and restarted ovirt-ha-agent. >>>>>>> Now status on the daemon gives the error again: >>>>>>> http://screencast.com/t/DCZo6QJM1 >>>>>>> >>>>>>> >>>>>> Here I'm a bit confused: is this a fresh hosted-engine deployment >>>>>> with 3.6 or an upgrade from an existing 3.5 hosted-engine deployment? >>>>>> >>>>> >>>>> I have reinstalled host ( complete OS reinstall) and ran >>>>> hosted-engine --deploy for redeployment on an existing oVirt ( >>>>> hosted-engine ). >>>>> >>>>> As described in previous mail, during the deployment I got errors >>>>> about a missing hostedengine.metadata symlink to /var/run/vdsd/storage/... >>>>> I created these by hand and then I ran deployment again and it worked. >>>>> >>>>> >>>>>> >>>>>>> The hosted engine domain is not shown in the web storage overview... >>>>>>> When I start import it shows a warning: >>>>>>> http://screencast.com/t/7HfJ9qWFwb2L >>>>>>> >>>>>>> Can I manually import the hostedengine storage? >>>>>>> ( Hosted engine is visible in web interface and I have running VMs >>>>>>> on another storage domain ). >>>>>>> >>>>>> >>>>>> It should auto-import it. >>>>>> Adding Roy here. >>>>>> >>>>>> >>>>> I believe this auto-import is failing... >>>>> Where can I find logs about this auto-import? >>>>> >>>>> >>>>> >>>>> >>>>>> >>>>>>> >>>>>>> Op ma 8 feb. 2016 om 10:36 schreef Simone Tiraboschi < >>>>>>> stira...@redhat.com>: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Feb 8, 2016 at 9:47 AM, Paul Groeneweg | Pazion < >>>>>>>> p...@pazion.nl> wrote: >>>>>>>> >>>>>>>>> Update: >>>>>>>>> >>>>>>>> >>>>>>>> The OVF_STORE volume will contain the engine VM description in a >>>>>>>> format that the engine can easily handle. >>>>>>>> It will be created by the engine when: >>>>>>>> - it imported the hosted-engine storage domain >>>>>>>> - the engineVM is visible >>>>>>>> - the datacenter is up (you need at least one additional regular >>>>>>>> storage domain) >>>>>>>> >>>>>>>> Please make sure that all these conditions are met. >>>>>>>> >>>>>>>> If the OVF_STORE is not there or it's not accessible, >>>>>>>> ovirt-ha-agent will fallback to the initial vm.conf from deployment >>>>>>>> time. >>>>>>>> You see a warning since we are failing back to an initial status and >>>>>>>> this >>>>>>>> can temporary override your latest customization to the engine VM >>>>>>>> configuration. >>>>>>>> >>>>>>>> >>>>>>>>> 1. I was able to remove the warnirngs about the not found ovf / >>>>>>>>> vm.conf by commenting the conf_volume_UUID, conf_image_UUID entries >>>>>>>>> in the >>>>>>>>> hosted-engine.conf. After a restart of the ovirt-ha-agent the errors >>>>>>>>> were >>>>>>>>> gone. ( I believe this is not a permanent solution ). >>>>>>>>> >>>>>>>> >>>>>>>> Please revert it. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> 2. This morning I noticed the warning in the event log. >>>>>>>>> "The Hosted Engine Storage Domain doesn't exist. It should be >>>>>>>>> imported into the setup." >>>>>>>>> >>>>>>>> >>>>>>>> It's just a side effect of your point 1, please revert it. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Any help would be appreciated. >>>>>>>>> >>>>>>>>> Best Regards, >>>>>>>>> >>>>>>>>> Paul Groeneweg >>>>>>>>> >>>>>>>>> Op zo 7 feb. 2016 om 22:14 schreef Paul Groeneweg | Pazion < >>>>>>>>> p...@pazion.nl>: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I have been upgrading our ovirt hosts to el7 to be able to run >>>>>>>>>> version 3.6. >>>>>>>>>> When deploying a new host have problems with the shared OVF >>>>>>>>>> storage config part. >>>>>>>>>> >>>>>>>>>> 1. I get an error during deploy: >>>>>>>>>> >>>>>>>>>> [ INFO ] Stage: Setup validation >>>>>>>>>> [ ERROR ] Failed to execute stage 'Setup validation': failed to >>>>>>>>>> read metadata: [Errno 2] No such file or directory: >>>>>>>>>> '/rhev/data-center/mnt/[NFS3_SHARE]/88b69eba-ef4f-4dbe-ba53-20dadd424d0e/ha_agent/hosted-engine.metadata' >>>>>>>>>> [ INFO ] Stage: Clean up >>>>>>>>>> >>>>>>>>>> It seems the symlink of hosted-engine.metadata is missing. >>>>>>>>>> So I added manually the /var/run/vdsm/storage folder with >>>>>>>>>> subfolder 88b69eba-ef4f-4dbe-ba53-20dadd424d0e and symlinks. >>>>>>>>>> Then I reran the hosted-engine deploy and this time all went well. >>>>>>>>>> >>>>>>>>>> 2. Now when the host is installed I notice the following in the >>>>>>>>>> logs >>>>>>>>>> >>>>>>>>>> .... >>>>>>>>>> Feb 5 22:33:08 geisha-2 ovirt-ha-agent: >>>>>>>>>> INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Preparing >>>>>>>>>> images >>>>>>>>>> Feb 5 22:33:08 geisha-2 ovirt-ha-agent: >>>>>>>>>> INFO:ovirt_hosted_engine_ha.lib.image.Image:Preparing images >>>>>>>>>> Feb 5 22:33:08 geisha-2 ovirt-ha-agent: >>>>>>>>>> INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Reloading >>>>>>>>>> vm.conf from the shared storage domain >>>>>>>>>> Feb 5 22:33:08 geisha-2 ovirt-ha-agent: >>>>>>>>>> INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config:Trying >>>>>>>>>> to get a fresher copy of vm configuration from the OVF_STORE >>>>>>>>>> Feb 5 22:33:08 geisha-2 ovirt-ha-agent: >>>>>>>>>> WARNING:ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore:Unable to >>>>>>>>>> find >>>>>>>>>> OVF_STORE >>>>>>>>>> Feb 5 22:33:08 geisha-2 journal: ovirt-ha-agent >>>>>>>>>> ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config ERROR >>>>>>>>>> Unable >>>>>>>>>> to get vm.conf from OVF_STORE, falling back to initial vm.conf >>>>>>>>>> Feb 5 22:33:08 geisha-2 ovirt-ha-agent: >>>>>>>>>> ERROR:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config:Unable >>>>>>>>>> to get vm.conf from OVF_STORE, falling back to initial vm.conf >>>>>>>>>> Feb 5 22:33:08 geisha-2 journal: ovirt-ha-agent >>>>>>>>>> ovirt_hosted_engine_ha.agent.agent.Agent ERROR Error: 'Path to volume >>>>>>>>>> 6a5b55c3-3f03-47cc-b322-d6d7562371be not found in >>>>>>>>>> /rhev/data-center/mnt' - >>>>>>>>>> trying to restart agent >>>>>>>>>> Feb 5 22:33:08 geisha-2 ovirt-ha-agent: >>>>>>>>>> ERROR:ovirt_hosted_engine_ha.agent.agent.Agent:Error: 'Path to volume >>>>>>>>>> 6a5b55c3-3f03-47cc-b322-d6d7562371be not found in >>>>>>>>>> /rhev/data-center/mnt' - >>>>>>>>>> trying to restart agent >>>>>>>>>> >>>>>>>>>> So at this point I was a bit stuck. >>>>>>>>>> I removed complete ovirt36 release and installed ovirt35 for the >>>>>>>>>> hosted engine deploy. >>>>>>>>>> >>>>>>>>>> This time all went without any problem directly. >>>>>>>>>> From there I retried upgrade and same problems again. >>>>>>>>>> >>>>>>>>>> I run ovirt hosted-engine on nfs3. I did found >>>>>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1159314 >>>>>>>>>> I added no_subtree_check and tried deploy again, but again >>>>>>>>>> without luck. >>>>>>>>>> >>>>>>>>>> I believe the hosted engine configs are not being migrated to the >>>>>>>>>> shared storage. And my problem has todo with >>>>>>>>>> http://www.ovirt.org/Features/Hosted_Engine_configuration_on_shared_storage >>>>>>>>>> >>>>>>>>>> And found /etc/ovirt-hosted-engine/hosted-engine.conf, >>>>>>>>>> conf_volume_UUID, conf_image_UUID >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hosts run el7 >>>>>>>>>> Hosted engine runs el6 >>>>>>>>>> Hosted engine storage is through NFS3 >>>>>>>>>> >>>>>>>>>> Can I import the hosted engine storage in to engine ( in web >>>>>>>>>> interface )? >>>>>>>>>> Or how can I create or trigger manually the OVF store the logs >>>>>>>>>> complain about? >>>>>>>>>> >>>>>>>>>> Best Regards, >>>>>>>>>> Paul Groeneweg >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Met vriendelijke groeten, >>>>>>>>>> >>>>>>>>>> Paul Groeneweg >>>>>>>>>> Pazion >>>>>>>>>> Webdevelopment - Hosting - Apps >>>>>>>>>> >>>>>>>>>> T +31 26 3020038 >>>>>>>>>> M +31 614 277 577 >>>>>>>>>> E p...@pazion.nl >>>>>>>>>> >>>>>>>>>> ***disclaimer*** >>>>>>>>>> "This e-mail and any attachments thereto may contain information >>>>>>>>>> which is confidential and/or protected by intellectual property >>>>>>>>>> rights and >>>>>>>>>> are intended for the sole use of the recipient(s) named above. Any >>>>>>>>>> use of >>>>>>>>>> the information contained herein (including, but not limited to, >>>>>>>>>> total or >>>>>>>>>> partial reproduction, communication or distribution in any form) by >>>>>>>>>> persons >>>>>>>>>> other than the designated recipient(s) is prohibited. If you have >>>>>>>>>> received >>>>>>>>>> this e-mail in error, please notify the sender either by telephone >>>>>>>>>> or by >>>>>>>>>> e-mail and delete the material from any computer. Thank you for your >>>>>>>>>> cooperation." >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> Met vriendelijke groeten, >>>>>>>>> >>>>>>>>> Paul Groeneweg >>>>>>>>> Pazion >>>>>>>>> Webdevelopment - Hosting - Apps >>>>>>>>> >>>>>>>>> T +31 26 3020038 >>>>>>>>> M +31 614 277 577 >>>>>>>>> E p...@pazion.nl >>>>>>>>> >>>>>>>>> ***disclaimer*** >>>>>>>>> "This e-mail and any attachments thereto may contain information >>>>>>>>> which is confidential and/or protected by intellectual property >>>>>>>>> rights and >>>>>>>>> are intended for the sole use of the recipient(s) named above. Any >>>>>>>>> use of >>>>>>>>> the information contained herein (including, but not limited to, >>>>>>>>> total or >>>>>>>>> partial reproduction, communication or distribution in any form) by >>>>>>>>> persons >>>>>>>>> other than the designated recipient(s) is prohibited. If you have >>>>>>>>> received >>>>>>>>> this e-mail in error, please notify the sender either by telephone or >>>>>>>>> by >>>>>>>>> e-mail and delete the material from any computer. Thank you for your >>>>>>>>> cooperation." >>>>>>>>> >>>>>>>> -- >>>>>>> Met vriendelijke groeten, >>>>>>> >>>>>>> Paul Groeneweg >>>>>>> Pazion >>>>>>> Webdevelopment - Hosting - Apps >>>>>>> >>>>>>> T +31 26 3020038 >>>>>>> M +31 614 277 577 >>>>>>> E p...@pazion.nl >>>>>>> >>>>>>> ***disclaimer*** >>>>>>> "This e-mail and any attachments thereto may contain information >>>>>>> which is confidential and/or protected by intellectual property rights >>>>>>> and >>>>>>> are intended for the sole use of the recipient(s) named above. Any use >>>>>>> of >>>>>>> the information contained herein (including, but not limited to, total >>>>>>> or >>>>>>> partial reproduction, communication or distribution in any form) by >>>>>>> persons >>>>>>> other than the designated recipient(s) is prohibited. If you have >>>>>>> received >>>>>>> this e-mail in error, please notify the sender either by telephone or by >>>>>>> e-mail and delete the material from any computer. Thank you for your >>>>>>> cooperation." >>>>>>> >>>>>> -- >>>>> Met vriendelijke groeten, >>>>> >>>>> Paul Groeneweg >>>>> Pazion >>>>> Webdevelopment - Hosting - Apps >>>>> >>>>> T +31 26 3020038 >>>>> M +31 614 277 577 >>>>> E p...@pazion.nl >>>>> >>>>> ***disclaimer*** >>>>> "This e-mail and any attachments thereto may contain information which >>>>> is confidential and/or protected by intellectual property rights and are >>>>> intended for the sole use of the recipient(s) named above. Any use of the >>>>> information contained herein (including, but not limited to, total or >>>>> partial reproduction, communication or distribution in any form) by >>>>> persons >>>>> other than the designated recipient(s) is prohibited. If you have received >>>>> this e-mail in error, please notify the sender either by telephone or by >>>>> e-mail and delete the material from any computer. Thank you for your >>>>> cooperation." >>>>> >>>> -- >>>> Met vriendelijke groeten, >>>> >>>> Paul Groeneweg >>>> Pazion >>>> Webdevelopment - Hosting - Apps >>>> >>>> T +31 26 3020038 >>>> M +31 614 277 577 >>>> E p...@pazion.nl >>>> >>>> ***disclaimer*** >>>> "This e-mail and any attachments thereto may contain information which >>>> is confidential and/or protected by intellectual property rights and are >>>> intended for the sole use of the recipient(s) named above. Any use of the >>>> information contained herein (including, but not limited to, total or >>>> partial reproduction, communication or distribution in any form) by persons >>>> other than the designated recipient(s) is prohibited. If you have received >>>> this e-mail in error, please notify the sender either by telephone or by >>>> e-mail and delete the material from any computer. Thank you for your >>>> cooperation." >>>> >>> -- >>> Met vriendelijke groeten, >>> >>> Paul Groeneweg >>> Pazion >>> Webdevelopment - Hosting - Apps >>> >>> T +31 26 3020038 >>> M +31 614 277 577 >>> E p...@pazion.nl >>> >>> ***disclaimer*** >>> "This e-mail and any attachments thereto may contain information which >>> is confidential and/or protected by intellectual property rights and are >>> intended for the sole use of the recipient(s) named above. Any use of the >>> information contained herein (including, but not limited to, total or >>> partial reproduction, communication or distribution in any form) by persons >>> other than the designated recipient(s) is prohibited. If you have received >>> this e-mail in error, please notify the sender either by telephone or by >>> e-mail and delete the material from any computer. Thank you for your >>> cooperation." >>> >> -- > Met vriendelijke groeten, > > Paul Groeneweg > Pazion > Webdevelopment - Hosting - Apps > > T +31 26 3020038 > M +31 614 277 577 > E p...@pazion.nl > > ***disclaimer*** > "This e-mail and any attachments thereto may contain information which is > confidential and/or protected by intellectual property rights and are > intended for the sole use of the recipient(s) named above. Any use of the > information contained herein (including, but not limited to, total or > partial reproduction, communication or distribution in any form) by persons > other than the designated recipient(s) is prohibited. If you have received > this e-mail in error, please notify the sender either by telephone or by > e-mail and delete the material from any computer. Thank you for your > cooperation." >
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users