The awsapi not getting up is not a problem; hence you are not using it. Why is not your CS app going up? Do you have any others log entries?
On Tue, Jul 15, 2014 at 5:24 PM, Adam <adam.scarce...@gmail.com> wrote: > I do not use awsapi either, and have never turned it on. If I look in the > database I can see that those features are turned off too: > > SELECT * FROM cloud.configuration where name = 'enable.ec2.api' or name = > 'enable.s3.api'; > > # category, instance, component, name, value, description, default_value, > updated, scope, is_dynamic > 'Advanced', 'DEFAULT', 'management-server', 'enable.ec2.api', 'false', > 'enable EC2 API on CloudStack', 'false', NULL, NULL, '0' > 'Advanced', 'DEFAULT', 'management-server', 'enable.s3.api', 'false', > 'enable Amazon S3 API on CloudStack', 'false', NULL, NULL, '0' > > > I found this though: https://issues.apache.org/jira/browse/CLOUDSTACK-2758 > > Not sure if it's related. > > I tried deleted everything from both the work and temp dirs, but it had no > affect. > > :( > > Best Regards, > > > > Adam Scarcella > > > On Tue, Jul 15, 2014 at 12:22 PM, Rafael Weingartner < > rafaelweingart...@gmail.com> wrote: > > > Have you tried to clean the directories work and temp of the tomcat > > installation that the CS is using? > > > > rm -rf /usr/share/cloudstack-management/temp/* > > > > rm -rf /usr/share/cloudstack-management/work/* > > > > > > In my CS installation I do not even have the awsapi files. > > > > I have just noticed that now, it has a soft link pointing to > > ../../cloudstack-bridge/webapps/awsapi, However, that folder does not > > exist. > > > > Are you using awsapi? > > > > I do not use it, so that is not a problem. > > > > > > On Tue, Jul 15, 2014 at 1:05 PM, Adam <adam.scarce...@gmail.com> wrote: > > > > > OK, so I have this issue HTTP Status 404 > > > > > > [image: Inline image 1] > > > > > > > > > ...except it must be different in some way because I've gone through > this > > > entire thread and even though I have the same original error mentioned > > and > > > the same behavior, I do not have the same "smoking gun" so to speak in > > the > > > localhost.log where my logging type was STATEMENT instead of MIXED. > > > > > > [root@cs-east-prod1 logs]# cat /etc/my.cnf > > > [mysqld] > > > datadir=/var/lib/mysql > > > sockeet=/var/lib/mysql/mysql.sock > > > user=mysql > > > # Disabling symbolic-links is recommended to prevent assorted security > > > risks > > > symbolic-links=0 > > > > > > > > > # > > > # BEGIN CloudStack 4.3 Requirements > > > # > > > innodb_rollback_on_timeout=1 > > > innodb_lock_wait_timeout=600 > > > max_connections=350 > > > log-bin=mysql-bin > > > binlog-format = 'ROW' > > > # > > > # END CloudStack 4.3 Requirements > > > # > > > > > > [mysqld_safe] > > > log-error=/var/log/mysqld.log > > > pid-file=/var/run/mysqld/mysqld.pid > > > [root@cs-east-prod1 logs]# > > > > > > > > > > > > I upgraded from 4.2.1 to 4.3 (CentOS 6.5 x86_64, KVM) and ever since > I've > > > not been able to start the cloudstack-management ui. I see these errors > > in > > > the catalina.out log: > > > > > > Jul 15, 2014 10:13:57 AM org.apache.axis2.deployment.ModuleDeployer > > deploy > > > SEVERE: The addressing-1.5.4.mar module, which is not valid, caused The > > > module.xml file cannot be found for the module: > > > > > > /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/addressing-1.5.4.mar > > > org.apache.axis2.deployment.DeploymentException: The module.xml file > > > cannot be found for the module: > > > > > > /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/addressing-1.5.4.mar > > > at > > > > > > org.apache.axis2.deployment.repository.util.ArchiveReader.readModuleArchive(ArchiveReader.java:529) > > > at > > > > org.apache.axis2.deployment.ModuleDeployer.deploy(ModuleDeployer.java:75) > > > at > > > > > > org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136) > > > at > > > > > > org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:661) > > > at > > > > > > org.apache.axis2.deployment.RepositoryListener.init(RepositoryListener.java:251) > > > at > > > > > > org.apache.axis2.deployment.RepositoryListener.init2(RepositoryListener.java:68) > > > at > > > > > > org.apache.axis2.deployment.RepositoryListener.<init>(RepositoryListener.java:63) > > > at > > > > > > org.apache.axis2.deployment.DeploymentEngine.loadRepository(DeploymentEngine.java:147) > > > at > > > > > > org.apache.axis2.deployment.WarBasedAxisConfigurator.getAxisConfiguration(WarBasedAxisConfigurator.java:229) > > > at > > > > > > org.apache.axis2.context.ConfigurationContextFactory.createConfigurationContext(ConfigurationContextFactory.java:64) > > > at > > > > > > org.apache.axis2.transport.http.AxisServlet.initConfigContext(AxisServlet.java:567) > > > at > > > org.apache.axis2.transport.http.AxisServlet.init(AxisServlet.java:452) > > > at > > > > > > org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1173) > > > at > > > org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:993) > > > at > > > > > > org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4187) > > > at > > > > org.apache.catalina.core.StandardContext.start(StandardContext.java:4496) > > > at > > > > > > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791) > > > at > > > org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771) > > > at > > > org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526) > > > at > > > > > > org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041) > > > at > > > > > > org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964) > > > at > > > org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502) > > > at > > > org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277) > > > at > > > > > > org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321) > > > at > > > > > > org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119) > > > at > > > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053) > > > at > > > org.apache.catalina.core.StandardHost.start(StandardHost.java:722) > > > at > > > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045) > > > at > > > org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443) > > > at > > > > org.apache.catalina.core.StandardService.start(StandardService.java:516) > > > at > > > org.apache.catalina.core.StandardServer.start(StandardServer.java:710) > > > at > org.apache.catalina.startup.Catalina.start(Catalina.java:593) > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > at > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > > at > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > at java.lang.reflect.Method.invoke(Method.java:606) > > > at > > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) > > > at > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) > > > Caused by: org.apache.axis2.deployment.DeploymentException: The > > module.xml > > > file cannot be found for the module: > > > > > > /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/addressing-1.5.4.mar > > > at > > > > > > org.apache.axis2.deployment.repository.util.ArchiveReader.readModuleArchive(ArchiveReader.java:523) > > > ... 37 more > > > > > > Jul 15, 2014 10:13:57 AM org.apache.axis2.deployment.ModuleDeployer > > deploy > > > INFO: Deploying module: addressing-1.5.6 - > > > > > > file:/usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/addressing-1.5.6.mar > > > Jul 15, 2014 10:13:57 AM org.apache.axis2.deployment.ModuleDeployer > > deploy > > > INFO: Deploying module: jaxws-1.5.6 - > > > > > > file:/usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/axis2-jaxws-mar-1.5.6.mar > > > Jul 15, 2014 10:13:57 AM org.apache.axis2.deployment.ModuleDeployer > > deploy > > > INFO: Deploying module: mtompolicy-1.5.6 - > > > > > > file:/usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/mtompolicy-1.5.6.mar > > > Jul 15, 2014 10:13:57 AM org.apache.axis2.deployment.ModuleDeployer > > deploy > > > INFO: Deploying module: addressing-1.5.6 - > > > > > > file:/usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/lib/axis2-1.5.6.jar > > > Jul 15, 2014 10:13:57 AM org.apache.catalina.loader.WebappClassLoader > > > loadClass > > > INFO: Illegal access: this web application instance has been stopped > > > already. Could not load > > > org.apache.cloudstack.managed.context.ManagedContextTimerTask$1. The > > > eventual following stack trace is caused by an error thrown for > debugging > > > purposes as well as to attempt to terminate the thread which caused the > > > illegal access, and has no functional impact. > > > java.lang.IllegalStateException > > > at > > > > > > org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1369) > > > at > > > > > > org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1329) > > > at > > > > > > org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(ManagedContextTimerTask.java:27) > > > at java.util.TimerThread.mainLoop(Timer.java:555) > > > at java.util.TimerThread.run(Timer.java:505) > > > > > > Exception in thread "Timer-2" java.lang.NoClassDefFoundError: > > > org/apache/cloudstack/managed/context/ManagedContextTimerTask$1 > > > at > > > > > > org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(ManagedContextTimerTask.java:27) > > > at java.util.TimerThread.mainLoop(Timer.java:555) > > > at java.util.TimerThread.run(Timer.java:505) > > > Caused by: java.lang.ClassNotFoundException: > > > org.apache.cloudstack.managed.context.ManagedContextTimerTask$1 > > > at > > > > > > org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1483) > > > at > > > > > > org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1329) > > > ... 3 more > > > > > > > > > I have absolutely no errors of any kind in the management-server.log. > > > > > > [root@cs-east-prod1 logs]# grep -i -E > > > 'exception|unable|fail|invalid|leak|warn|error' ./management-server.log > > > [root@cs-east-prod1 logs]# > > > > > > > > > The only thing I see in the access.log is: > > > > > > [root@cs-east-prod1 logs]# cat access_log.2014-07-15.txt > > > 10.97.34.115 - - [15/Jul/2014:10:27:15 -0500] "GET /client/ HTTP/1.1" > 404 > > > 952 > > > [root@cs-east-prod1 logs]# > > > > > > > > > ... and my localhost.log is entirely empty. > > > > > > > > > [root@cs-east-prod1 logs]# du -sh localhost.2014-07-15.log > > > 0 localhost.2014-07-15.log > > > > > > > > > And even more bizzare is the fact that I have another CloudStack 4.3 > > > server that is running on XenServer and it's perfectly fine. It was not > > an > > > upgrade though, it was a clean install of 4.3. I've compared all the > > files > > > associated with the addressing-1.5.4.mar but could not find anything > > > significant. > > > > > > Here is the output from my upgraded server (cs-east-prod1: broken with > > 404) > > > > > > [root@cs-east-prod1 modules]# pwd > > > /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules > > > [root@cs-east-prod1 modules]# md5sum addressing-1.5.4.mar > > > 1ef211c1a042e7d4b9b7e5f914222929 addressing-1.5.4.mar > > > [root@cs-east-prod1 modules]# cat modules.list > > > soapmonitor-1.5.6.mar > > > addressing-1.5.6.mar > > > mtompolicy-1.5.6.mar > > > mex-1.5.6.mar > > > ping-1.5.6.mar > > > axis2-jaxws-mar-1.5.6.mar > > > scripting-1.5.6.mar > > > [root@cs-east-prod1 modules]# ls -l > > > total 340 > > > -rwxrwxrwx 1 cloud cloud 126981 Mar 25 11:01 addressing-1.5.4.mar > > > -rwxrwxrwx 1 cloud cloud 34518 Mar 25 11:01 addressing-1.5.6.mar > > > -rwxrwxrwx 1 cloud cloud 14363 Mar 25 11:01 axis2-jaxws-mar-1.5.6.mar > > > -rwxrwxrwx 1 cloud cloud 38620 Mar 25 11:01 mex-1.5.6.mar > > > -rwxrwxrwx 1 cloud cloud 139 Mar 25 11:01 modules.list > > > -rwxrwxrwx 1 cloud cloud 18682 Mar 25 11:01 mtompolicy-1.5.6.mar > > > -rwxrwxrwx 1 cloud cloud 17647 Mar 25 11:01 ping-1.5.6.mar > > > -rwxrwxrwx 1 cloud cloud 9721 Mar 25 11:01 rahas-1.5.1.mar > > > -rwxrwxrwx 1 cloud cloud 9612 Mar 25 11:01 rampart-1.5.1.mar > > > -rwxrwxrwx 1 cloud cloud 33779 Mar 25 11:01 scripting-1.5.6.mar > > > -rwxrwxrwx 1 cloud cloud 15650 Mar 25 11:01 soapmonitor-1.5.6.mar > > > [root@cs-east-prod1 modules]# cd / > > > [root@cs-east-prod1 /]# find . -name "module.xml" > > > > > > > > > ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/s3/module.xml > > > > > > > > > ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/ec2/module.xml > > > [root@cs-east-prod1 /]# md5sum > > > > > > ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/s3/module.xml > > > baf5f63f5dd57484e9a88be6689939fc > > > > > > > ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/s3/module.xml > > > [root@cs-east-prod1 /]# md5sum > > > > > > ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/ec2/module.xml > > > 913b4561258980f7b1b4c7cb3de261ba > > > > > > > ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/ec2/module.xml > > > [root@cs-east-prod1 /]# > > > > > > > > > And here is the output from the exact same commands on my 4.3 server > > > that's working fine (na-nyc-bos-cs-poc2): > > > > > > [root@na-nyc-bos-cs-poc2 modules]# pwd > > > /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules > > > [root@na-nyc-bos-cs-poc2 modules]# md5sum addressing-1.5.4.mar > > > 1ef211c1a042e7d4b9b7e5f914222929 addressing-1.5.4.mar > > > [root@na-nyc-bos-cs-poc2 modules]# cat modules.list > > > soapmonitor-1.5.6.mar > > > addressing-1.5.6.mar > > > mtompolicy-1.5.6.mar > > > mex-1.5.6.mar > > > ping-1.5.6.mar > > > axis2-jaxws-mar-1.5.6.mar > > > scripting-1.5.6.mar > > > [root@na-nyc-bos-cs-poc2 modules]# ls -l > > > total 340 > > > -rw-r--r--. 1 cloud cloud 126981 Mar 25 11:01 addressing-1.5.4.mar > > > -rw-r--r--. 1 cloud cloud 34518 Mar 25 11:01 addressing-1.5.6.mar > > > -rw-r--r--. 1 cloud cloud 14363 Mar 25 11:01 axis2-jaxws-mar-1.5.6.mar > > > -rw-r--r--. 1 cloud cloud 38620 Mar 25 11:01 mex-1.5.6.mar > > > -rw-r--r--. 1 cloud cloud 139 Mar 25 11:01 modules.list > > > -rw-r--r--. 1 cloud cloud 18682 Mar 25 11:01 mtompolicy-1.5.6.mar > > > -rw-r--r--. 1 cloud cloud 17647 Mar 25 11:01 ping-1.5.6.mar > > > -rw-r--r--. 1 cloud cloud 9721 Mar 25 11:01 rahas-1.5.1.mar > > > -rw-r--r--. 1 cloud cloud 9612 Mar 25 11:01 rampart-1.5.1.mar > > > -rw-r--r--. 1 cloud cloud 33779 Mar 25 11:01 scripting-1.5.6.mar > > > -rw-r--r--. 1 cloud cloud 15650 Mar 25 11:01 soapmonitor-1.5.6.mar > > > [root@na-nyc-bos-cs-poc2 modules]# cd / > > > [root@na-nyc-bos-cs-poc2 /]# find . -name "module.xml" > > > > > > > > > ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/s3/module.xml > > > > > > > > > ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/ec2/module.xml > > > [root@na-nyc-bos-cs-poc2 /]# md5sum > > > > > > ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/s3/module.xml > > > baf5f63f5dd57484e9a88be6689939fc > > > > > > > ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/s3/module.xml > > > [root@na-nyc-bos-cs-poc2 /]# md5sum > > > > > > ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/ec2/module.xml > > > 913b4561258980f7b1b4c7cb3de261ba > > > > > > > ./usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/classes/com/cloud/bridge/auth/ec2/module.xml > > > [root@na-nyc-bos-cs-poc2 /]# > > > > > > > > > > > > *HELP!!!* > > > > > > I need to get my cloud back up and running. > > > > > > > > > > > > > > > Best Regards, > > > > > > > > > > > > Adam Scarcella > > > > > > > > > On Tue, Jun 10, 2014 at 8:41 PM, Rafael Weingartner < > > > rafaelweingart...@gmail.com> wrote: > > > > > >> Nice, no doubts that others will run into this. > > >> But when they run into that problem, it is going to be just a query at > > >> Google to uncover this thread ;) > > >> > > >> > > >> On Tue, Jun 10, 2014 at 8:39 PM, Steve Searles <ssear...@zimcom.net> > > >> wrote: > > >> > > >> > Rafael, > > >> > > > >> > Ok, I seem to have found the smoking gun in the localhost.log > > >> > > > >> > Caused by: java.sql.SQLException: Binary logging not possible. > > Message: > > >> > Transaction level 'READ-COMMITTED' in InnoDB is not safe for binlog > > mode > > >> > 'STATEMENT' > > >> > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1073) > > >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3597) > > >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3529) > > >> > at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1990) > > >> > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151) > > >> > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2625) > > >> > at > > >> > > > >> > > > com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2119) > > >> > at > > >> > > > >> > > > com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2415) > > >> > at > > >> > > > >> > > > com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2333) > > >> > at > > >> > > > >> > > > com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2318) > > >> > at > > >> > > > >> > > > org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105) > > >> > at > > >> > > > >> > > > org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105) > > >> > at com.cloud.utils.db.GenericDaoBase.update(GenericDaoBase.java:823) > > >> > ... 198 more > > >> > Jun 10, 2014 7:02:25 PM org.apache.catalina.core.ApplicationContext > > log > > >> > INFO: Shutting down log4j > > >> > > > >> > Oddly enough I thought the default logging type was MIXED not > > >> STATEMENT. > > >> > I am sure others will run into this. I am running (5.1.73) This is > > what > > >> > the MySQL website had to say. > > >> > > > >> > The default binary logging format depends on the version > of > > >> > MySQL you are using: > > >> > > > >> > - > > >> > > > >> > For MySQL 5.1.11 and earlier, and for MySQL 5.1.29 and later, > > >> > statement-based logging is used by default. > > >> > - > > >> > > > >> > For MySQL 5.1.12 through MySQL 5.1.28, mixed logging is used by > > >> default > > >> > > > >> > If you are using InnoDB > > >> > < > http://dev.mysql.com/doc/refman/5.1/en/innodb-storage-engine.html > > > > > >> > > >> > tables and the transaction isolation level is READ COMMITTED > > >> > < > > >> > > > http://dev.mysql.com/doc/refman/5.1/en/set-transaction.html#isolevel_read-committed > > >> > > > >> > or READ UNCOMMITTED > > >> > < > > >> > > > http://dev.mysql.com/doc/refman/5.1/en/set-transaction.html#isolevel_read-uncommitted > > >> >, > > >> > only row-based logging can be used. It is *possible* to change > the > > >> > > >> > logging format to STATEMENT, but doing so at runtime leads very > > >> > rapidly to errors because InnoDB can no longer perform inserts. > > >> > > > >> > Switching the replication format at runtime is not recommended > when > > >> > any temporary tables exist, because temporary tables are logged > > only > > >> when > > >> > using statement-based replication, whereas with row-based > > >> replication they > > >> > are not logged. With mixed replication, temporary tables are > > usually > > >> > logged; exceptions happen with user-defined functions (UDFs) and > > >> with the > > >> > UUID() > > >> > < > > >> > > > http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html#function_uuid > > >> > > > >> > > >> > function. > > >> > > > >> > With the binary log format set to ROW, many changes are written > to > > >> the > > >> > binary log using the row-based format. Some changes, however, > still > > >> use the > > >> > statement-based format. Examples include all DDL (data definition > > >> language) > > >> > statements such as CREATE TABLE > > >> > <http://dev.mysql.com/doc/refman/5.1/en/create-table.html>, > ALTER > > >> TABLE > > >> > <http://dev.mysql.com/doc/refman/5.1/en/alter-table.html>, or > DROP > > >> > TABLE <http://dev.mysql.com/doc/refman/5.1/en/drop-table.html>. > > >> > > >> > > > >> > > > >> > > > >> > Add > > >> > binlog-format=ROW > > >> > to /etc/my.cnf on the master DB server and you are good to go. > > >> > > > >> > > > >> > > > >> > Steven Searles, CTO | ssear...@zimcom.net > > >> > Zimcom Internet Solutions | www.zimcom.net > > >> > O: 513.231.9500 | D: 513.233.4130 > > >> > > > >> > > > >> > On Jun 10, 2014, at 6:18 PM, Rafael Weingartner < > > >> > rafaelweingart...@gmail.com> wrote: > > >> > > > >> > maybe, but, I actually do not care that much about the app that is > > used > > >> to > > >> > integrate CS with AWS, hence they are two separated apps, a problem > > with > > >> > AWS integration app should not affect the CS normal app. > > >> > > > >> > Could you please provide me again with the log file from > > >> > /var/log/cloudstack/management/management-server.log? There must be > > >> > something here. > > >> > > > >> > > > >> > On Tue, Jun 10, 2014 at 6:34 PM, Steve Searles <ssear...@zimcom.net > > > > >> > wrote: > > >> > > > >> > Is > > >> > > > >> > > > >> > > > http://mail-archives.apache.org/mod_mbox/cloudstack-issues/201401.mbox/%3CJIRA.12687852.1389186771873.61435.1389186832715@arcas%3E > > >> > relevant here? > > >> > > > >> > > > >> > > > >> > On Jun 10, 2014, at 5:31 PM, Steve Searles <ssear...@zimcom.net > > <mailto: > > >> > ssear...@zimcom.net>> wrote: > > >> > > > >> > Ok I was wrong, the addressing-1.5.4.mar is present. > > >> > > > >> > [root@csmgmt01 modules]# pwd > > >> > /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules > > >> > [root@csmgmt01 modules]# ls -lash > > >> > total 344K > > >> > 4.0K drwxr-xr-x. 2 cloud cloud 4.0K Mar 31 18:21 . > > >> > 4.0K drwxr-xr-x. 7 cloud cloud 4.0K Mar 31 18:21 .. > > >> > 124K -rw-r--r--. 1 cloud cloud 124K Mar 27 01:06 > addressing-1.5.4.mar > > >> > 36K -rw-r--r--. 1 cloud cloud 34K Mar 27 01:06 addressing-1.5.6.mar > > >> > 16K -rw-r--r--. 1 cloud cloud 15K Mar 27 01:06 > > >> axis2-jaxws-mar-1.5.6.mar > > >> > 40K -rw-r--r--. 1 cloud cloud 38K Mar 27 01:06 mex-1.5.6.mar > > >> > 4.0K -rw-r--r--. 1 cloud cloud 139 Mar 27 01:06 modules.list > > >> > 20K -rw-r--r--. 1 cloud cloud 19K Mar 27 01:06 mtompolicy-1.5.6.mar > > >> > 20K -rw-r--r--. 1 cloud cloud 18K Mar 27 01:06 ping-1.5.6.mar > > >> > 12K -rw-r--r--. 1 cloud cloud 9.5K Mar 27 01:06 rahas-1.5.1.mar > > >> > 12K -rw-r--r--. 1 cloud cloud 9.4K Mar 27 01:06 rampart-1.5.1.mar > > >> > 36K -rw-r--r--. 1 cloud cloud 33K Mar 27 01:06 scripting-1.5.6.mar > > >> > 16K -rw-r--r--. 1 cloud cloud 16K Mar 27 01:06 > soapmonitor-1.5.6.mar > > >> > [root@csmgmt01 modules]# > > >> > > > >> > > > >> > > > >> > The error is complaining about the module.xml file missing. > > >> > > > >> > Steve Searles > > >> > > > >> > On Jun 10, 2014, at 5:21 PM, Steve Searles <ssear...@zimcom.net > > <mailto: > > >> > ssear...@zimcom.net><mailto:ssear...@zimcom.net>> wrote: > > >> > > > >> > I agree with what you are saying. Yes. addressing-1.5.4.mar errors > I > > >> have > > >> > seen before. I have no webapps7080 directory so that is why that > > error > > >> is > > >> > being thrown. I still am wondering even if a dependency is missing > > why > > >> is > > >> > it only a dependency when sql replication(bin log) is on? Can you > tell > > >> me > > >> > how to get a stack trace from CS startup? > > >> > > > >> > That file is present in these locations on the management node. > > >> > > > >> > [root@csmgmt01 /]# find / -name addressing-1.5.4.mar > > >> > > > >> > > > /root/.m2/repository/org/apache/axis2/addressing/1.5.4/addressing-1.5.4.mar > > >> > > > >> > > > >> > > > >> > > > /root/apache-cloudstack-4.3.0-src/dist/rpmbuild/BUILD/cloudstack-4.3.0/awsapi/target/cloud-awsapi-4.3.0/WEB-INF/modules/addressing-1.5.4.mar > > >> > > > >> > > > >> > > > >> > > > /usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/modules/addressing-1.5.4.mar > > >> > [root@csmgmt01 /]# > > >> > > > >> > > > >> > > > >> > is it just a symlink that is missing? > > >> > > > >> > > > >> > > > >> > [root@csmgmt01 webapps]# ls -las > > >> > total 12 > > >> > 4 drwxr-xr-x. 3 root root 4096 Mar 31 18:21 . > > >> > 4 drwxr-xr-x. 4 root root 4096 Mar 31 18:21 .. > > >> > 4 drwxr-xr-x. 11 root root 4096 Mar 31 18:21 client > > >> > [root@csmgmt01 webapps]# cd client/ > > >> > [root@csmgmt01 client]# ls > > >> > css dictionary.jsp images index.jsp lib META-INF modules > > plugins > > >> > scripts tests WEB-INF > > >> > [root@csmgmt01 client]# ls -lash > > >> > total 264K > > >> > 4.0K drwxr-xr-x. 11 root root 4.0K Mar 31 18:21 . > > >> > 4.0K drwxr-xr-x. 3 root root 4.0K Mar 31 18:21 .. > > >> > 4.0K drwxr-xr-x. 2 root root 4.0K Mar 31 18:21 css > > >> > 116K -rw-r--r--. 1 root root 115K Mar 27 01:06 dictionary.jsp > > >> > 4.0K drwxr-xr-x. 2 root root 4.0K Mar 31 18:21 images > > >> > 104K -rw-r--r--. 1 root root 103K Mar 27 01:06 index.jsp > > >> > 4.0K drwxr-xr-x. 5 root root 4.0K Mar 31 18:21 lib > > >> > 4.0K drwxr-xr-x. 2 root root 4.0K Mar 27 01:06 META-INF > > >> > 4.0K drwxr-xr-x. 6 root root 4.0K Mar 31 18:21 modules > > >> > 4.0K drwxr-xr-x. 3 root root 4.0K Mar 31 18:21 plugins > > >> > 4.0K drwxr-xr-x. 4 root root 4.0K Mar 31 18:21 scripts > > >> > 4.0K drwxr-xr-x. 2 root root 4.0K Mar 31 18:21 tests > > >> > 4.0K drwxr-xr-x. 4 root root 4.0K Mar 31 18:21 WEB-INF > > >> > [root@csmgmt01 client]# pwd > > >> > /usr/share/cloudstack-management/webapps/client > > >> > [root@csmgmt01 client]# > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > Steve Searles > > >> > > > >> > > > >> > On Jun 10, 2014, at 3:04 PM, Rafael Weingartner < > > >> > rafaelweingart...@gmail.com<mailto:rafaelweingart...@gmail.com > > ><mailto: > > >> > rafaelweingart...@gmail.com><mailto:rafaelweingart...@gmail.com>> > > >> wrote: > > >> > > > >> > Look at this point: > > >> > "SEVERE: Error listenerStart > > >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.core.StandardContext > start > > >> > SEVERE: Context [/client] startup failed due to previous errors" > > >> > I could not find the stacktrace into the logs you provided. But, it > > has > > >> to > > >> > have some stack which would help to understand why the CS is not > > getting > > >> > up. > > >> > > > >> > This is a good point to look at: > > >> > "SEVERE: The addressing-1.5.4.mar module, which is not valid, caused > > The > > >> > module.xml file cannot be found for the module: > > >> > > > >> > > > >> > > > >> > > > /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/addressing-1.5.4.mar > > >> > org.apache.axis2.deployment.DeploymentException: The module.xml file > > >> cannot > > >> > be found for the module: > > >> > > > >> > > > >> > > > >> > > > /usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/modules/addressing-1.5.4.mar > > >> > at > > >> > > > >> > > > >> > > > >> > > > org.apache.axis2.deployment.repository.util.ArchiveReader.readModuleArchive(ArchiveReader.java:529) > > >> > at > > >> > > > >> > > org.apache.axis2.deployment.ModuleDeployer.deploy(ModuleDeployer.java:75) > > >> > at > > >> > > > >> > > > >> > > > >> > > > org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136) > > >> > at > > >> > > > >> > > > >> > > > >> > > > org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:661)" > > >> > > > >> > > > >> > > > >> > It seems there is a .xml file being missing. > > >> > But, still even though the awsapi is not working, the CS app should > > run > > >> > just fine, unless it depends on that service. > > >> > > > >> > This is also something interesting: > > >> > "Jun 10, 2014 2:12:58 PM > org.apache.catalina.loader.WebappClassLoader > > >> > loadClass > > >> > INFO: Illegal access: this web application instance has been stopped > > >> > already. Could not load > > >> > org.apache.cloudstack.managed.context.ManagedContextTimerTask$1. > The > > >> > eventual following stack trace is caused by an error thrown for > > >> debugging > > >> > purposes as well as to attempt to terminate the thread which caused > > the > > >> > illegal access, and has no functional impact. > > >> > java.lang.IllegalStateException > > >> > at > > >> > > > >> > > > >> > > > >> > > > org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1369)" > > >> > > > >> > it seems there is a missing dependency. > > >> > > > >> > BTW: out of curiosity, what do tou have in folder > > >> > "/usr/share/cloudstack-management/webapps" > > >> > > > >> > > > >> > On Tue, Jun 10, 2014 at 3:53 PM, Steve Searles <ssear...@zimcom.net > > >> > <mailto:ssear...@zimcom.net><mailto:ssear...@zimcom.net><mailto: > > >> > ssear...@zimcom.net>> wrote: > > >> > > > >> > Nothing interesting in the management server logs, you saw the > > complete > > >> > ones I sent yesterday. I have attached the complete catalina.out > > below. > > >> > This is starting to feel like a problem with the DB locking the > tables > > >> > during the startup integrity check. I am able to lock the tables > via > > >> the > > >> > command line with the bin log enabled without issue though. > > >> > > > >> > https://dl.dropboxusercontent.com/u/51895629/catalina.binlog.gz > > >> > > > >> > Steve Searles > > >> > > > >> > > > >> > On Jun 10, 2014, at 2:29 PM, Rafael Weingartner < > > >> > rafaelweingart...@gmail.com<mailto:rafaelweingart...@gmail.com>> > > wrote: > > >> > > > >> > this is interesting: > > >> > " > > >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.core.StandardContext > start > > >> > SEVERE: Context [/client] startup failed due to previous error" > > >> > > > >> > does it have any extra log before that message? > > >> > Did you find anything at /var/log/cloudstack/management-server.log? > > >> > > > >> > > > >> > On Tue, Jun 10, 2014 at 3:24 PM, Steve Searles <ssear...@zimcom.net > > >> > <mailto:ssear...@zimcom.net>> > > >> > wrote: > > >> > > > >> > I found the following differences in the catalina.out file with bin > > log > > >> > enabled and disabled. Can you point me in the right direction on > > >> debugging > > >> > the actual /client application further. Obviously the problem is > > >> > > > >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.core.StandardContext > start > > >> > SEVERE: Error listenerStart > > >> > > > >> > I could use a point in the right direction to get further debugging > > >> > info. > > >> > > > >> > > > >> > WITH BINLOG > > >> > > > >> > INFO [c.c.u.c.ComponentContext] (main:null) Setup Spring > Application > > >> > context > > >> > INFO [c.c.s.GsonHelper] (main:null) Default Builder inited. > > >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.core.StandardContext > start > > >> > SEVERE: Error listenerStart > > >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.core.StandardContext > start > > >> > SEVERE: Context [/client] startup failed due to previous errors > > >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.loader.WebappClassLoader > > >> > clearReferencesThreads > > >> > SEVERE: A web application appears to have started a thread named > > >> > [FileWatchdog] but has failed to stop it. This is very likely to > > create > > >> a > > >> > memory leak. > > >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.loader.WebappClassLoader > > >> > clearReferencesThreads > > >> > SEVERE: A web application appears to have started a thread named > > >> [Timer-0] > > >> > but has failed to stop it. This is very likely to create a memory > > leak. > > >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.loader.WebappClassLoader > > >> > clearReferencesThreads > > >> > SEVERE: A web application appears to have started a thread named > > >> > [ConnectionConcierge-1] but has failed to stop it. This is very > likely > > >> to > > >> > create a memory leak. > > >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.loader.WebappClassLoader > > >> > clearReferencesThreads > > >> > SEVERE: A web application appears to have started a thread named > > >> > [ClusteredAgentManager Timer] but has failed to stop it. This is > very > > >> > likely to create a memory leak. > > >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.loader.WebappClassLoader > > >> > clearReferencesThreads > > >> > SEVERE: A web application appears to have started a thread named > > >> [Timer-1] > > >> > but has failed to stop it. This is very likely to create a memory > > leak. > > >> > Jun 10, 2014 2:12:48 PM org.apache.catalina.loader.WebappClassLoader > > >> > clearThreadLocalMap > > >> > SEVERE: A web application created a ThreadLocal with key of type > > >> > [org.springframework.core.NamedThreadLocal] (value [Prototype beans > > >> > currently in creation]) and a value of type [null] (value [null]) > but > > >> > failed to remove it when the web application was stopped. To > prevent a > > >> > memory leak, the ThreadLocal has be > > >> > en forcibly removed. > > >> > > > >> > > > >> > WITHOUT BINLOG > > >> > > > >> > > > >> > INFO [c.c.u.c.ComponentContext] (main:null) Setup Spring > Application > > >> > context > > >> > INFO [c.c.s.GsonHelper] (main:null) Default Builder inited. > > >> > INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) Running > > system > > >> > integrity checker > com.cloud.upgrade.DatabaseIntegrityChecker@7f2af4a3 > > >> > INFO [c.c.u.DatabaseIntegrityChecker] (main:null) Grabbing lock to > > >> check > > >> > for database integrity. > > >> > INFO [c.c.u.DatabaseIntegrityChecker] (main:null) Performing > database > > >> > integrity check > > >> > INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) Running > > system > > >> > integrity checker > > >> > org.apache.cloudstack.utils.identity.ManagementServerNode@7c73b9ad > > >> > INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) > Configuring > > >> > CloudStack Components > > >> > INFO [c.c.u.d.GenericDaoBase] (main:null) Cache created: [ name = > > >> > UserDaoImpl status = STATUS_ALIVE eternal = false overflowToDisk = > > false > > >> > maxEntriesLocalHeap = 5000 maxEntriesLocalDisk = 0 > > >> > memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 300 > > >> timeToIdleSeconds = > > >> > 300 persistence = none diskExpiryThreadInt > > >> > ervalSeconds = 120 cacheEventListeners: > > >> > net.sf.ehcache.statistics.LiveCacheStatisticsWrapper hitCount = 0 > > >> > memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0 > > >> > missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false > > >> > maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ] > > >> > INFO [c.c.u.d.GenericDaoBase] (main:null) Cache created: [ name = > > >> > VMTemplateDaoImpl status = STATUS_ALIVE eternal = false > > overflowToDisk = > > >> > false maxEntriesLocalHeap = 100 maxEntriesLocalDisk = 0 > > >> > memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600 > > >> timeToIdleSeconds = > > >> > 300 persistence = none diskExpiryThre > > >> > adIntervalSeconds = 120 cacheEventListeners: > > >> > net.sf.ehcache.statistics.LiveCacheStatisticsWrapper hitCount = 0 > > >> > memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0 > > >> > missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false > > >> > maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = fals > > >> > e ] > > >> > INFO [c.c.u.d.GenericDaoBase] (main:null) Cache created: [ name = > > >> > ServiceOfferingDaoImpl status = STATUS_ALIVE eternal = false > > >> overflowToDisk > > >> > = false maxEntriesLocalHeap = 50 maxEntriesLocalDisk = 0 > > >> > memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600 > > >> timeToIdleSeconds = > > >> > 300 persistence = none diskExpiry > > >> > ThreadIntervalSeconds = 120 cacheEventListeners: > > >> > net.sf.ehcache.statistics.LiveCacheStatisticsWrapper hitCount = 0 > > >> > memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0 > > >> > missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false > > >> > maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = > > >> > false ] > > >> > INFO [c.c.u.d.GenericDaoBase] (main:null) Cache created: [ name = > > >> > DiskOfferingDaoImpl status = STATUS_ALIVE eternal = false > > >> overflowToDisk = > > >> > false maxEntriesLocalHeap = 50 maxEntriesLocalDisk = 0 > > >> > memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600 > > >> timeToIdleSeconds = > > >> > 300 persistence = none diskExpiryThr > > >> > eadIntervalSeconds = 120 cacheEventListeners: > > >> > net.sf.ehcache.statistics.LiveCacheStatisticsWrapper hitCount = 0 > > >> > memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0 > > >> > missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false > > >> > maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = fal > > >> > se ] > > >> > > > >> > > > >> > > > >> > > > >> > On Jun 10, 2014, at 10:30 AM, Rafael Weingartner < > > >> > rafaelweingart...@gmail.com<mailto:rafaelweingart...@gmail.com>> > > wrote: > > >> > > > >> > you do not need to debug into the mysql, you just need to find the > > stack > > >> > trace in the tomcat that is causing the app to shutdown during the > > >> start up > > >> > process > > >> > > > >> > > > >> > On Tue, Jun 10, 2014 at 11:26 AM, Steve Searles < > ssear...@zimcom.net > > >> > <mailto:ssear...@zimcom.net>> > > >> > wrote: > > >> > > > >> > Yes, I will see if I can get any more debugging info out of it. Yes > > >> that > > >> > is the URL we are using to access. In the condition where the bin > log > > >> is > > >> > on I get a 404 from tomcat meaning that the application has not > > >> started. I > > >> > will try some further debugging on the mysql side as well. This is > > not > > >> an > > >> > upgrade but rather a fresh install of 4.3. That is the problem I am > > >> seeing > > >> > as well, no useful debug information. > > >> > > > >> > Steve Searles > > >> > > > >> > On Jun 10, 2014, at 8:21 AM, Rafael Weingartner < > > >> > rafaelweingart...@gmail.com<mailto:rafaelweingart...@gmail.com>> > > wrote: > > >> > > > >> > I thought that it could be a missing dependency problem, because > with > > CS > > >> > 4.3 upgrade from 4.2.1 we had one. The mysql jdbc .jar was not in > the > > >> > classpath. So, we had to manually add it. > > >> > > > >> > I did not see any expressive tty in your log files. > > >> > Could you look for some more meaningful java stack traces, then we > > might > > >> > > > >> > be > > >> > > > >> > able to help you. > > >> > BTW, how are you trying to access the CS? > http://serverIp:8080/client > > ? > > >> > > > >> > > > >> > On Mon, Jun 9, 2014 at 11:31 PM, Steve Searles <ssear...@zimcom.net > > >> > <mailto:ssear...@zimcom.net>> > > >> > > > >> > wrote: > > >> > > > >> > > > >> > It is also worth noting that these are the last two DB queries that > > >> > > > >> > happen > > >> > > > >> > before the stop. So it may have something to do with cloudbridge? > > >> > > > >> > > > >> > > > >> > 140609 22:28:00 8 Connect cl...@db01.cvg.cs.zimcom.net > <mailto: > > >> > cl...@db01.cvg.cs.zimcom.net> on > > >> > cloudbridge > > >> > 8 Query /* mysql-connector-java-5.1.17-SNAPSHOT ( > > >> > Revision: ${bzr.revision-id} ) */SHOW VARIABLES WHERE Variable_name > > >> > ='language' OR Variable_name = 'net_write_timeout' OR Variable_name > = > > >> > 'interactive_timeout' OR Variable_name = 'wait_timeout' OR > > >> > > > >> > Variable_name = > > >> > > > >> > 'character_set_client' OR Variable_name = 'character_set_connection' > > OR > > >> > Variable_name = 'character_set' OR Variable_name = > > >> > > > >> > 'character_set_server' > > >> > > > >> > OR Variable_name = 'tx_isolation' OR Variable_name = > > >> > 'transaction_isolation' OR Variable_name = 'character_set_results' > OR > > >> > Variable_name = 'timezone' OR Variable_name = 'time_zone' OR > > >> > > > >> > Variable_name > > >> > > > >> > = 'system_time_zone' OR Variable_name = 'lower_case_table_names' OR > > >> > Variable_name = 'max_allowed_packet' OR Variable_name = > > >> > > > >> > 'net_buffer_length' > > >> > > > >> > OR Variable_name = 'sql_mode' OR Variable_name = 'query_cache_type' > OR > > >> > Variable_name = 'query_cache_size' OR Variable_name = 'init_connect' > > >> > 8 Query /* mysql-connector-java-5.1.17-SNAPSHOT ( > > >> > Revision: ${bzr.revision-id} ) */SELECT > > >> > > > >> > @@session.auto_increment_increment > > >> > > > >> > 8 Query SHOW COLLATION > > >> > 8 Query SET NAMES latin1 > > >> > 8 Query SET character_set_results = NULL > > >> > 8 Query SET autocommit=1 > > >> > 8 Query SET sql_mode='STRICT_TRANS_TABLES' > > >> > 8 Query SET autocommit=0 > > >> > 8 Query SET autocommit=0 > > >> > 8 Query SELECT offering_bundle.ID, > > >> > offering_bundle.AmazonEC2Offering, > offering_bundle.CloudStackOffering > > >> > > > >> > FROM > > >> > > > >> > offering_bundle > > >> > 8 Query commit > > >> > 8 Query rollback > > >> > 8 Query SET autocommit=1 > > >> > 8 Query SET autocommit=0 > > >> > 8 Query SET autocommit=0 > > >> > 8 Query SELECT mhost.ID, mhost.MHostKey, > > >> > mhost.Host, mhost.Version, mhost.LastHeartbeatTime FROM mhost WHERE > > >> > mhost.MHostKey = _binary'00:50:56:92:04:18' ORDER BY RAND() LIMIT 1 > > >> > 8 Query commit > > >> > 8 Query rollback > > >> > 8 Query SET autocommit=1 > > >> > 8 Query SET autocommit=0 > > >> > 8 Query SET autocommit=0 > > >> > 8 Query UPDATE mhost SET mhost.Host=_binary' > > >> > http://localhost:7080/awsapi' WHERE mhost.ID = 1 > > >> > 8 Query commit > > >> > 8 Query rollback > > >> > 8 Query SET autocommit=1 > > >> > 8 Query SET autocommit=0 > > >> > 8 Query SET autocommit=0 > > >> > 8 Query SELECT shost.ID, shost.Host, > > >> > shost.HostType, shost.MHostID, shost.ExportRoot, shost.UserOnHost, > > >> > shost.UserPassword FROM shost WHERE shost.MHostID = 1 AND > > >> > > > >> > shost.ExportRoot > > >> > > > >> > = _binary'/Users/john1/S3-Mount' ORDER BY RAND() LIMIT 1 > > >> > 8 Query commit > > >> > 8 Query rollback > > >> > 8 Query SET autocommit=1 > > >> > 9 Connect cl...@db01.cvg.cs.zimcom.net<mailto: > > >> > cl...@db01.cvg.cs.zimcom.net> on cloud > > >> > 9 Query /* mysql-connector-java-5.1.17-SNAPSHOT ( > > >> > Revision: ${bzr.revision-id} ) */SHOW VARIABLES WHERE Variable_name > > >> > ='language' OR Variable_name = 'net_write_timeout' OR Variable_name > = > > >> > 'interactive_timeout' OR Variable_name = 'wait_timeout' OR > > >> > > > >> > Variable_name = > > >> > > > >> > 'character_set_client' OR Variable_name = 'character_set_connection' > > OR > > >> > Variable_name = 'character_set' OR Variable_name = > > >> > > > >> > 'character_set_server' > > >> > > > >> > OR Variable_name = 'tx_isolation' OR Variable_name = > > >> > 'transaction_isolation' OR Variable_name = 'character_set_results' > OR > > >> > Variable_name = 'timezone' OR Variable_name = 'time_zone' OR > > >> > > > >> > Variable_name > > >> > > > >> > = 'system_time_zone' OR Variable_name = 'lower_case_table_names' OR > > >> > Variable_name = 'max_allowed_packet' OR Variable_name = > > >> > > > >> > 'net_buffer_length' > > >> > > > >> > OR Variable_name = 'sql_mode' OR Variable_name = 'query_cache_type' > OR > > >> > Variable_name = 'query_cache_size' OR Variable_name = 'init_connect' > > >> > 9 Query /* mysql-connector-java-5.1.17-SNAPSHOT ( > > >> > Revision: ${bzr.revision-id} ) */SELECT > > >> > > > >> > @@session.auto_increment_increment > > >> > > > >> > 9 Query SHOW COLLATION > > >> > 9 Query SET NAMES latin1 > > >> > 9 Query SET character_set_results = NULL > > >> > 9 Query SET autocommit=1 > > >> > 9 Query SET sql_mode='STRICT_TRANS_TABLES' > > >> > 9 Query SET autocommit=0 > > >> > 9 Query SELECT @@session.tx_isolation > > >> > 9 Query SET SESSION TRANSACTION ISOLATION LEVEL > > >> > READ COMMITTED > > >> > 9 Query SELECT 1 > > >> > 9 Query SET autocommit=0 > > >> > 9 Query SELECT configuration.name, > > >> > configuration.value FROM configuration WHERE configuration.name = > > >> > _binary'enable.ec2.api' ORDER BY RAND() LIMIT 1 > > >> > 9 Query commit > > >> > 9 Query rollback > > >> > 9 Query SET autocommit=1 > > >> > > > >> > > > >> > > > >> > Steve Searles > > >> > > > >> > > > >> > > > >> > -----Original Message----- > > >> > From: Steve Searles [mailto:ssear...@zimcom.net < > ssear...@zimcom.net > > >> > <mailto:ssear...@zimcom.net>> < > > >> > ssear...@zimcom.net<mailto:ssear...@zimcom.net>>] > > >> > Sent: Monday, June 9, 2014 10:19 PM > > >> > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> > > >> > Subject: RE: Management Server Startup Issue > > >> > > > >> > Rafael, Thank you for responding so quickly. Yes 4.3.0. I am using > a > > >> > dedicated DB servers and dedicated management servers. I will > provide > > >> > whatever log files you would find useful. It would be odd if it was > a > > >> > dependancy since CS really has no visibility to the sql server other > > >> > > > >> > than a > > >> > > > >> > connect string? Or so I thought; and the system is running fine with > > the > > >> > binlog off. With the bin log on I can run queries from the command > > line > > >> > and everything seems normal from the sql end, but the management > > server > > >> > will not run. Toggle binlog off CS works fine. The replication > slave > > >> > > > >> > has > > >> > > > >> > not even been started yet. Let me know what other info I can send > > over > > >> > that would be of help. There is nothing in the catalina logs of > > >> > > > >> > interest > > >> > > > >> > and my logging level is currently set to trace for the > > >> > management-server.log. I will turn on the sql query log and compare > > >> > > > >> > them > > >> > > > >> > between startups of the management nodes perhaps that will yeild > > >> > > > >> > something. > > >> > > > >> > I have included a dropbox link with the management server log from a > > >> > normal startup of CS and one with mysql binlog enabled. Any help > > would > > >> > > > >> > be > > >> > > > >> > greatly appriciated. > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > https://dl.dropboxusercontent.com/u/51895629/management-server.logs.tar.gz > > >> > > > >> > > > >> > > > >> > > > >> > Steve Searles > > >> > > > >> > > > >> > -----Original Message----- > > >> > From: Rafael Weingartner [mailto:rafaelweingart...@gmail.com] > > >> > Sent: Monday, June 9, 2014 6:57 PM > > >> > To: users@cloudstack.apache.org > > >> > Subject: Re: Management Server Startup Issue > > >> > > > >> > please give more logs. > > >> > It is the 4.3.0 right? I bet there is a problem with a missing > > >> > > > >> > dependency. > > >> > > > >> > > > >> > > > >> > On Mon, Jun 9, 2014 at 7:32 PM, Steve Searles <ssear...@zimcom.net> > > >> > > > >> > wrote: > > >> > > > >> > > > >> > Anyone have any idea why the management service would refuse to > start > > >> > if the DB server is configured as a master for replication? > > >> > > > >> > > > >> > > > >> > log_bin=mysql-bin > > >> > server_id=1 > > >> > > > >> > Adding the above the my.cnf file of my DB server, restarting mysql > > >> > then restarting cloudstack-managment will cause cloudstack to get > hung > > >> > on startup here. > > >> > > > >> > 2014-06-09 18:25:33,769 INFO [c.c.u.c.ComponentContext] (main:null) > > >> > Starting > > >> > > com.cloud.bridge.persist.dao.SHostDaoImpl_EnhancerByCloudStack_9da5499 > > >> > b > > >> > 2014-06-09 18:25:33,769 INFO [c.c.u.c.ComponentContext] (main:null) > > >> > Starting > > >> > > com.cloud.bridge.persist.dao.SObjectDaoImpl_EnhancerByCloudStack_847fb > > >> > 98c > > >> > 2014-06-09 18:25:33,769 INFO [c.c.u.c.ComponentContext] (main:null) > > >> > Starting > > >> > > com.cloud.bridge.persist.dao.CloudStackUserDaoImpl_EnhancerByCloudStac > > >> > k_7fce3ee6 > > >> > 2014-06-09 18:25:33,769 INFO [c.c.u.c.ComponentContext] (main:null) > > >> > Starting > > >> > > com.cloud.bridge.service.core.ec2.EC2Engine_EnhancerByCloudStack_d70c9 > > >> > e3c > > >> > 2014-06-09 18:25:33,769 INFO [c.c.u.c.ComponentContext] (main:null) > > >> > Starting > > >> > > com.cloud.bridge.service.controller.s3.ServiceProvider_EnhancerByCloud > > >> > Stack_23d38b1 > > >> > > > >> > That is the last few lines of the management-server.log and it will > > >> > hang there indefinitely. > > >> > > > >> > I noticed this on startup as well. > > >> > > > >> > 2014-06-09 18:25:16,841 INFO [c.c.u.d.T.Transaction] (main:null) Is > > >> > Data Base High Availiability enabled? Ans : false > > >> > 2014-06-09 18:25:17,164 DEBUG [c.c.u.d.ConnectionConcierge] > > >> > (main:null) Registering a database connection for LockMaster1 > > >> > 2014-06-09 18:25:17,164 INFO [c.c.u.d.Merovingian2] (main:null) > > >> > Cleaning up locks for 345049793560 > > >> > 2014-06-09 18:25:17,175 INFO [c.c.u.d.Merovingian2] (main:null) > > >> > Released > > >> > 0 locks for 345049793560 > > >> > 2014-06-09 18:25:17,206 INFO > [o.a.c.s.l.CloudStackExtendedLifeCycle] > > >> > (main:null) Running system integrity checker > > >> > com.cloud.upgrade.DatabaseUpgradeChecker@63c8e4ae > > >> > 2014-06-09 18:25:17,207 INFO [c.c.u.DatabaseUpgradeChecker] > > >> > (main:null) Grabbing lock to check for database upgrade. > > >> > 2014-06-09 18:25:17,254 DEBUG [c.c.u.d.VersionDaoImpl] (main:null) > > >> > Checking to see if the database is at a version before it was the > > >> > version table is created > > >> > 2014-06-09 18:25:17,295 INFO [c.c.u.DatabaseUpgradeChecker] > > >> > (main:null) DB version = 4.3.0 Code Version = 4.3.0 > > >> > 2014-06-09 18:25:17,296 INFO [c.c.u.DatabaseUpgradeChecker] > > >> > (main:null) DB version and code version matches so no upgrade > needed. > > >> > > > >> > The management server talking to the DB server at least enough to > > >> > check the version; and mysql from the command line on the management > > >> > server works fine as well. > > >> > > > >> > > > >> > This line caught my attention. > > >> > > > >> > > > >> > 2014-06-09 18:25:16,841 INFO [c.c.u.d.T.Transaction] (main:null) Is > > >> > Data Base High Availiability enabled? Ans : false > > >> > > > >> > Is there something I need to set when using mysql replication > specific > > >> > to cloudstack? I do not see anything in the documentation on this > > >> > other than setting up basic replication. > > >> > > > >> > > > >> > if I comment out the above two lines in my.cnf and restart the > > >> > management server it fires up just fine. I don’t get how enabling > bin > > >> > log would make a difference at all. > > >> > > > >> > Any help or explication for this strange issue would be appreciated. > > >> > > > >> > > > >> > - Steve Searles > > >> > > > >> > > > >> > > > >> > > > >> > -- > > >> > Rafael Weingärtner > > >> > > > >> > > > >> > > > >> > > > >> > -- > > >> > Rafael Weingärtner > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > -- > > >> > Rafael Weingärtner > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > -- > > >> > Rafael Weingärtner > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > -- > > >> > Rafael Weingärtner > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > -- > > >> > Rafael Weingärtner > > >> > > > >> > > > >> > > > >> > > >> > > >> -- > > >> Rafael Weingärtner > > >> > > > > > > > > > > > > -- > > Rafael Weingärtner > > > -- Rafael Weingärtner
