Re: [ovirt-users] engine tries to allocate 11GB and aborts (was: hosted engine does not start correctly (CentOS 7.1, overt-engine 3.5.2))

2015-06-22 Thread Yedidyah Bar David
- Original Message -
 From: H. Haven Liu haven@ucla.edu
 To: Yedidyah Bar David d...@redhat.com
 Cc: oVirt Mailing List users@ovirt.org
 Sent: Tuesday, May 26, 2015 10:26:07 PM
 Subject: Re: engine tries to allocate 11GB and aborts (was: hosted engine 
 does not start correctly (CentOS 7.1,
 overt-engine 3.5.2))
 
 Didi,
 
 Both source and destination machines (assuming you are referring to the
 physical host source and virtual host destination of the migration) are
 running ovirt-engine 3.5.2.1-1. However, the source is on CentOS 6.6,
 whereas the destination is on CentOS 7.1. The source is running
 postgresql(/-server) 8.4.20, Java 1.7.0_79, httpd 2.2.15. Destination is
 running postgresql(/-server) 9.2.10, Java 1.7.0.79, jboss-httpd 2.4.6. I'm
 not sure which over package version is relevant.

Sorry for not replying earlier.

Now opened [1] to track such upgrades. Didn't try that myself.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1234257

 
 The failure occurs after engine-setup, and the web UI doesn't load, and the
 host reports the engine is not up (I'm not in front of the host, so I don't
 have the exact language).
 
 Our current setup has 2 physical hosts, ~15 active VMs. Each host has 64GB
 RAM. Even with all the VM on one host, the utilization is less than 50%. We
 have another physical storage server, that provides the NFS storage backend
 to the hosts and VMs. Prior to attempting the hosted-engine migration, I
 migrated all the VMs to host #1, where the engine was originally installed.
 I then removed host #2 from the host list via the UI, and performed
 hosted-engine --deploy on host #2. A few other quirks I encountered:
 
 1. I originally wanted to do hosted-engine --deploy on host #1, but that
 it was unsuccessful due to it complaining the storage connection was busy
 (or something).
 2. I then tried to do hosted-engine --deploy on host #2, using the same
 storage connection path as what I used for host #1 originally. The setup
 process asked if it was an additional host, I said no, but later it
 cannot activate the storage connection.
 3. Lastly I did hosted-engine --deploy on host #2, but pointing to a
 different storage connection path, still on the same storage server. This
 was successful until the I encountered the failure in question.

Not sure about the exact causes for these issues.

If still not resolved, please retry on a clean OS reinstalled.

Best,

 
 Thanks,
 
 Haven
 
 
 
 On Tue, May 26, 2015 at 2:20 AM, Yedidyah Bar David d...@redhat.com wrote:
 
  - Original Message -
   From: Haoyang Haven Liu haven@ucla.edu
   To: users@ovirt.org
   Sent: Tuesday, May 26, 2015 8:34:00 AM
   Subject: [ovirt-users] hosted engine does not start correctly (CentOS
  7.1,overt-engine 3.5.2)
  
   Hello,
  
   I tried to follow  http://www.ovirt.org/Migrate_to_Hosted_Engine  to
   migrate an existing ovirt environment. I managed to complete the
   “engine-backup —mode=restore…” step, and the following “engine-setup”
  step
   appear to have completed. However, the ovirt-engine service was not
  started
   successfully has errors:
  
   [root@ovirt ~]# systemctl status ovirt-engine
   ovirt-engine.service - oVirt Engine
   Loaded: loaded (/usr/lib/systemd/system/ovirt-engine.service; enabled)
   Active: failed (Result: exit-code) since Mon 2015-05-25 22:26:12 PDT;
  43s ago
   Main PID: 15931 (code=exited, status=1/FAILURE)
  
   May 25 22:26:11 ovirt.polysep.ucla.edu systemd[1]: Started oVirt Engine.
   May 25 22:26:12 ovirt.polysep.ucla.edu ovirt-engine.py[15931]:
  2015-05-25
   22:26:12,269 ovirt-engine: ERROR run:532 Error: process termina...code 1
   May 25 22:26:12 ovirt.polysep.ucla.edu systemd[1]:
  ovirt-engine.service: main
   process exited, code=exited, status=1/FAILURE
   May 25 22:26:12 ovirt.polysep.ucla.edu systemd[1]: Unit
  ovirt-engine.service
   entered failed state.
   Hint: Some lines were ellipsized, use -l to show in full.
  
   Additionally, /var/log/ovirt-engine/console.log complains “cannot
  allocate
   memory”. The VM engine has 4GB of memory, but ovirt-engine engine seems
  to
   want ~11GB of memory, is that right?
  
   [root@ovirt ~]# cat /var/log/ovirt-engine/console.log
   OpenJDK 64-Bit Server VM warning: INFO:
  os::commit_memory(0x00041218,
   11250696192, 0) failed; error='Cannot allocate memory' (errno=12)
   #
   # There is insufficient memory for the Java Runtime Environment to
  continue.
   # Native memory allocation (malloc) failed to allocate 11250696192 bytes
  for
   committing reserved memory.
   # An error report file with more information is saved as:
   # /tmp/jvm-15974/hs_error.log
   [root@ovirt ~]# free -m
   total used free shared buff/cache available
   Mem: 3791 123 3175 28 492 3428
   Swap: 2559 0 2559
   [root@ovirt ~]#
  
   The hs_error.log is attached.
  
   Any help is appreciated.
 
  Can you please compare versions of all installed packages between the
  source
  and destination machines?

[ovirt-users] engine tries to allocate 11GB and aborts (was: hosted engine does not start correctly (CentOS 7.1, overt-engine 3.5.2))

2015-05-26 Thread Yedidyah Bar David
- Original Message -
 From: Haoyang Haven Liu haven@ucla.edu
 To: users@ovirt.org
 Sent: Tuesday, May 26, 2015 8:34:00 AM
 Subject: [ovirt-users] hosted engine does not start correctly (CentOS 7.1,
 overt-engine 3.5.2)
 
 Hello,
 
 I tried to follow  http://www.ovirt.org/Migrate_to_Hosted_Engine  to
 migrate an existing ovirt environment. I managed to complete the
 “engine-backup —mode=restore…” step, and the following “engine-setup” step
 appear to have completed. However, the ovirt-engine service was not started
 successfully has errors:
 
 [root@ovirt ~]# systemctl status ovirt-engine
 ovirt-engine.service - oVirt Engine
 Loaded: loaded (/usr/lib/systemd/system/ovirt-engine.service; enabled)
 Active: failed (Result: exit-code) since Mon 2015-05-25 22:26:12 PDT; 43s ago
 Main PID: 15931 (code=exited, status=1/FAILURE)
 
 May 25 22:26:11 ovirt.polysep.ucla.edu systemd[1]: Started oVirt Engine.
 May 25 22:26:12 ovirt.polysep.ucla.edu ovirt-engine.py[15931]: 2015-05-25
 22:26:12,269 ovirt-engine: ERROR run:532 Error: process termina...code 1
 May 25 22:26:12 ovirt.polysep.ucla.edu systemd[1]: ovirt-engine.service: main
 process exited, code=exited, status=1/FAILURE
 May 25 22:26:12 ovirt.polysep.ucla.edu systemd[1]: Unit ovirt-engine.service
 entered failed state.
 Hint: Some lines were ellipsized, use -l to show in full.
 
 Additionally, /var/log/ovirt-engine/console.log complains “cannot allocate
 memory”. The VM engine has 4GB of memory, but ovirt-engine engine seems to
 want ~11GB of memory, is that right?
 
 [root@ovirt ~]# cat /var/log/ovirt-engine/console.log
 OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00041218,
 11250696192, 0) failed; error='Cannot allocate memory' (errno=12)
 #
 # There is insufficient memory for the Java Runtime Environment to continue.
 # Native memory allocation (malloc) failed to allocate 11250696192 bytes for
 committing reserved memory.
 # An error report file with more information is saved as:
 # /tmp/jvm-15974/hs_error.log
 [root@ovirt ~]# free -m
 total used free shared buff/cache available
 Mem: 3791 123 3175 28 492 3428
 Swap: 2559 0 2559
 [root@ovirt ~]#
 
 The hs_error.log is attached.
 
 Any help is appreciated.

Can you please compare versions of all installed packages between the source
and destination machines?

I assume the failure above is right after engine-setup, right? Before continuing
the interaction on hosted-engine side?

Did you try to restart the engine (systemctl restart ovirt-engine)? Did it fail
the same way?

How much memory do you have on the source machine? And how much does the engine
actually use there?

How large is your system (hosts, VMs etc)?

Best,
-- 
Didi
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users