Hey Aleks, I don’t see anything in that log that would explain the behavior you’re observing. Markus did most of the work on deposits. Markus? Do you have any ideas?
Best Regards, Myrle On Fri 13. Jul 2018 at 07:06 Aleksandar Vidakovic <[email protected]> wrote: > @Myrle: with your proposed changes things look a bit better. I've disabled > Rhythm as you suggested and get past all microservice startups with a lot > less exceptions (one is coming repeatedly from Netty native transport, > because epoll is not available). > > The one exception (actually it happened on 2 occasions) that is still > there: > > [snip] > > 13:33:45.080 [main] INFO o.a.f.c.d.s.DepositAccountManagementApplication - > Started DepositAccountManagementApplication in 55.187 seconds (JVM running > for 62.772) > 13:34:45.730 [AsyncResolver-bootstrap-executor-0] INFO > c.n.d.s.r.aws.ConfigClusterResolver - Resolving eureka endpoints via > configuration > Resolving artifact > org.apache.fineract.cn.teller:service-boot:jar:0.1.0-BUILD-SNAPSHOT > Resolving metadata > > org.apache.fineract.cn.teller:service-boot:0.1.0-BUILD-SNAPSHOT/maven-metadata.xml > from /root/.m2/repository (enhanced) > Resolved metadata > > org.apache.fineract.cn.teller:service-boot:0.1.0-BUILD-SNAPSHOT/maven-metadata.xml > from /root/.m2/repository (enhanced) > Resolved artifact > org.apache.fineract.cn.teller:service-boot:jar:0.1.0-BUILD-SNAPSHOT from > /root/.m2/repository (enhanced) > 13:34:56.800 [DiscoveryClient-0] ERROR c.n.discovery.TimedSupervisorTask - > task supervisor timed out > java.util.concurrent.TimeoutException: null > at java.util.concurrent.FutureTask.get(FutureTask.java:205) > at > com.netflix.discovery.TimedSupervisorTask.run(TimedSupervisorTask.java:64) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > at > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > 13:34:56.801 [DiscoveryClient-1] ERROR c.n.discovery.TimedSupervisorTask - > task supervisor timed out > java.util.concurrent.TimeoutException: null > at java.util.concurrent.FutureTask.get(FutureTask.java:205) > at > com.netflix.discovery.TimedSupervisorTask.run(TimedSupervisorTask.java:64) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > at > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > 13:34:56.916 [DiscoveryClient-2] ERROR c.n.discovery.TimedSupervisorTask - > task supervisor timed out > java.util.concurrent.TimeoutException: null > at java.util.concurrent.FutureTask.get(FutureTask.java:205) > at > com.netflix.discovery.TimedSupervisorTask.run(TimedSupervisorTask.java:64) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > at > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > 13:35:08.586 [main] INFO o.s.c.a.AnnotationConfigApplicationContext - > Refreshing > > org.springframework.context.annotation.AnnotationConfigApplicationContext@7dc5e7b4 > : > startup date [Fri Jul 13 13:35:08 UTC 2018]; root of context hierarchy > 13:35:10.182 [main] INFO o.s.b.f.a.AutowiredAnnotationBeanPostProcessor - > JSR-330 'javax.inject.Inject' annotation found and supported for autowiring > 13:35:10.371 [main] INFO > o.s.c.s.PostProcessorRegistrationDelegate$BeanPostProcessorChecker - Bean > 'configurationPropertiesRebinderAutoConfiguration' of type [class > > org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration$$EnhancerBySpringCGLIB$$ffaf96f9] > is not eligible for getting processed by all BeanPostProcessors (for > example: not eligible for auto-proxying) > 13:35:10.435 [background-preinit] INFO o.h.validator.internal.util.Version > - HV000001: Hibernate Validator 5.3.0.Final > > [/snip] > > It looks like the whole startup procedure goes through and I can see some > kind of "summary" that lists all REST endpoints. But it stays in that state > for only a minute or so and then a cascade of shutdowns are triggered > again. The only thing that stays alive is Eureka which complains about the > inaccessible services. > > Any idea what else we could do other than just ordering an additional > machine? > > Cheers, > > Aleks > > On Mon, Jun 18, 2018 at 5:42 PM Myrle Krantz <[email protected]> wrote: > > > Hey Aleks, > > > > It looks like that exception is coming from Rhythm. I suspect it's not > the > > cause of your difficulties, since you say your other services are going > > down too. You can run most of the Fineract server without Rhythm. You > > could edit the demo-server script to leave it out for the purposes of > > testing. The consequence will be a lack of interest calculations for > > disbursed loans, but for finding out what the real problem is, it'd be > good > > to get this one out of the way. > > > > Best Regards, > > Myrle > > > > (FYI: I've removed the non-list subscribers from the to. Most of them > are > > already subscribed, and those who aren't, don't want to be.) > > > > On Wed, Jun 13, 2018 at 1:40 AM Aleksandar Vidakovic < > > [email protected]> wrote: > > > > > I managed to further improve the startup procedure (aka less exceptions > > > than before)... there are still some timeouts, but not as many as > before. > > > > > > The demo server still dies after this exception: > > > > > > 23:27:41.896 [AsyncResolver-bootstrap-executor-0] INFO > > > c.n.d.s.r.aws.ConfigClusterResolver - Resolving eureka endpoints via > > > configuration > > > 23:27:44.564 [AsyncResolver-bootstrap-executor-0] INFO > > > c.n.d.s.r.aws.ConfigClusterResolver - Resolving eureka endpoints via > > > configuration > > > 23:28:43.669 [AsyncResolver-bootstrap-executor-0] INFO > > > c.n.d.s.r.aws.ConfigClusterResolver - Resolving eureka endpoints via > > > configuration > > > 23:28:45.990 [pool-9-thread-1] ERROR > > o.s.s.s.TaskUtils$LoggingErrorHandler > > > - Unexpected error occurred in scheduled task. > > > org.springframework.transaction.CannotCreateTransactionException: Could > > not > > > open JPA EntityManager for transaction; nested exception is > > > javax.persistence.PersistenceException: > > > java.sql.SQLNonTransientConnectionException: Could not read resultset: > > > unexpected end of stream, read 0 bytes from 4 > > > Query is : set autocommit=0 > > > at > > > > > > > > > org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:431) > > > at > > > > > > > > > org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:373) > > > at > > > > > > > > > org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:430) > > > at > > > > > > > > > org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:276) > > > at > > > > > > > > > org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) > > > at > > > > > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) > > > at > > > > > > > > > org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) > > > at > > > > > > > > > org.apache.fineract.cn.rhythm.service.internal.service.Drummer$$EnhancerBySpringCGLIB$$70dfc8e3.checkForBeatsNeeded(<generated>) > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > at > > > > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > > at > > > > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > at java.lang.reflect.Method.invoke(Method.java:498) > > > at > > > > > > > > > org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) > > > at > > > > > > > > > org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) > > > at > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > > > at > > > > > > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > > > at > > > > > > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > > > at > > > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > > at > > > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > > at java.lang.Thread.run(Thread.java:748) > > > Caused by: javax.persistence.PersistenceException: > > > java.sql.SQLNonTransientConnectionException: Could not read resultset: > > > unexpected end of stream, read 0 bytes from 4 > > > Query is : set autocommit=0 > > > at > > > > > > > > > org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1692) > > > at > > > > > > > > > org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1602) > > > at > > > > > > > > > org.hibernate.jpa.spi.AbstractEntityManagerImpl.throwPersistenceException(AbstractEntityManagerImpl.java:1700) > > > at > > > > org.hibernate.jpa.internal.TransactionImpl.begin(TransactionImpl.java:48) > > > at > > > > > > > > > org.springframework.orm.jpa.vendor.HibernateJpaDialect.beginTransaction(HibernateJpaDialect.java:189) > > > at > > > > > > > > > org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:380) > > > ... 20 common frames omitted > > > Caused by: java.sql.SQLNonTransientConnectionException: Could not read > > > resultset: unexpected end of stream, read 0 bytes from 4 > > > Query is : set autocommit=0 > > > at > > > > > > > > > org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:123) > > > at > > > > > > > > > org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:69) > > > at > > > > > > > > > org.mariadb.jdbc.MariaDbStatement.executeQueryEpilog(MariaDbStatement.java:242) > > > at > > > > > > > > > org.mariadb.jdbc.MariaDbStatement.executeInternal(MariaDbStatement.java:270) > > > at > > > > > > org.mariadb.jdbc.MariaDbStatement.executeUpdate(MariaDbStatement.java:399) > > > at > > > > > > > > > org.mariadb.jdbc.MariaDbConnection.setAutoCommit(MariaDbConnection.java:584) > > > at > > > > > > > > > com.jolbox.bonecp.ConnectionHandle.setAutoCommit(ConnectionHandle.java:1292) > > > at > > > > > > > > > org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.begin(AbstractLogicalConnectionImplementor.java:67) > > > at > > > > > > > > > org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.begin(LogicalConnectionManagedImpl.java:238) > > > at > > > > > > > > > org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.begin(JdbcResourceLocalTransactionCoordinatorImpl.java:214) > > > at > > > > > > > > > org.hibernate.engine.transaction.internal.TransactionImpl.begin(TransactionImpl.java:52) > > > at > > > > > > org.hibernate.internal.SessionImpl.beginTransaction(SessionImpl.java:1512) > > > at > > > > org.hibernate.jpa.internal.TransactionImpl.begin(TransactionImpl.java:45) > > > ... 22 common frames omitted > > > Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Could not > > > read resultset: unexpected end of stream, read 0 bytes from 4 > > > Query is : set autocommit=0 > > > at > > > > > > > > > org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:909) > > > at > > > > > > > > > org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:604) > > > at > > > > > > > > > org.mariadb.jdbc.MariaDbStatement.executeInternal(MariaDbStatement.java:261) > > > ... 31 common frames omitted > > > Caused by: java.io.EOFException: unexpected end of stream, read 0 bytes > > > from 4 > > > at > > > > > > > > > org.mariadb.jdbc.internal.packet.read.ReadPacketFetcher.getReusableBuffer(ReadPacketFetcher.java:168) > > > at > > > > > > > > > org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:900) > > > ... 33 common frames omitted > > > 23:29:05.500 [qtp6422064-20] INFO o.a.f.c.l.c.ServiceExceptionFilter - > > > Responding with a service error ServiceError{code=409, > > message='Application > > > identity-v1 already exists!'} > > > 23:29:21.282 [Thread-15] INFO > > > o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext - Closing > > > > > > > > > org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@11438d26 > > > : > > > startup date [Tue Jun 12 23:02:02 UTC 2018]; parent: > > > > > > > > > org.springframework.context.annotation.AnnotationConfigApplicationContext@7dc5e7b4 > > > 23:29:23.717 [Thread-15] INFO o.s.c.s.DefaultLifecycleProcessor - > > Stopping > > > beans in phase 2147483647 > > > 23:29:24.267 [Thread-15] INFO o.s.c.s.DefaultLifecycleProcessor - > > Stopping > > > beans in phase 0 > > > 23:29:24.557 [Thread-15] INFO o.s.b.a.e.jmx.EndpointMBeanExporter - > > > Unregistering JMX-exposed beans on shutdown > > > 23:29:24.579 [Thread-15] INFO o.s.b.a.e.jmx.EndpointMBeanExporter - > > > Unregistering JMX-exposed beans > > > > > > The only thing I've left: start the databases (aka Cassandra) on a > > separate > > > machine (e. g. on Digital Ocean) and see how this is working then. > > > > > > ... FYI. > > > > > > Cheers, > > > > > > Aleks > > > > > > On Tue, Jun 12, 2018 at 11:16 PM Aleksandar Vidakovic < > > > [email protected]> wrote: > > > > > > > @Rajan: it's not running, because the demo server dies... See logs > from > > > > previous message... > > > > > > > > We just have to figure that one out and we should be good to go. > > > > > > > > > > > > On Tue, Jun 12, 2018, 9:11 PM Rajan Maurya <[email protected] > > > > > > wrote: > > > > > > > >> Thanks, Alex, > > > >> > > > >> I can see web app is running but I can't log in with the credentials > > > that > > > >> you shared earlier > > > >> > > > >> Tenant : playground > > > >> Username : operator > > > >> Password : init1@l > > > >> > > > >> Thanks > > > >> > > > >> On Wed, Jun 13, 2018 at 12:32 AM Aleksandar Vidakovic < > > > >> [email protected]> wrote: > > > >> > > > >> > Hi all, > > > >> > > > > >> > ... sweeeet! I got a considerable step further. > > > >> > > > > >> > @Markus: thanks for the conversation... gave me an idea how to fix > > > (most > > > >> > of) it! > > > >> > > > > >> > The problem was - as suspected - the memory (or lack of). The VM > > > >> instance > > > >> > at Apache has net 30G available. 16G are needed for Cassandra and > in > > > my > > > >> > previous tests I tried to run the demo server with up to 16G. The > > > >> problem > > > >> > is that the demo server needs considerably more memory than 16G. > > > >> According > > > >> > to my observation it's around 23-24G. Even if you limit the demo > > > server > > > >> > memory allocation with "-Xmx" to 12G (for example) it would just > > > >> continue > > > >> > to consume more memory. > > > >> > > > > >> > My solution: add a swap file. Not ideal in terms of performance, > but > > > it > > > >> > gets the whole thing going... at least mostly (please read on). > > > >> > > > > >> > Current status: > > > >> > > > > >> > - Cassandra (and all other Docker containers) are still > running; > > > >> > Cassandra died in my previous attempts at the latest when the > > > Teller > > > >> > application started, but with the swap file it's still running > > > >> > - all Spring Boot apps are starting now > > > >> > - I get once in a while some spurious timeouts (I think when > > either > > > >> the > > > >> > Eureka server is contacted or the config server) > > > >> > - there are also sometimes exceptions concerning MySQL > > connections, > > > >> but > > > >> > doesn't seem to matter overall > > > >> > - I could issue an authentication request with Postman to > > > >> > http://fineract-vm.apache.org:2021/identity/v1; just to test > if > > > >> > anything > > > >> > is responding > > > >> > - PROBLEM: but the demo server (the Spring Boot apps) > eventually > > > dies > > > >> > after a couple of minutes; I don't think that the memory is the > > > >> problem > > > >> > at > > > >> > this point; I think it's more of a timeout problem concerning > the > > > >> > communication with Eureka and/or the config server (maybe > because > > > the > > > >> > system is a bit slow) > > > >> > > > > >> > ... and here the memory footprint ("free -h") when everything is > > > >> running, > > > >> > just FYI: > > > >> > > > > >> > total used free shared > buff/cache > > > >> > available > > > >> > Mem: 31G 31G 226M 8.4M > 169M > > > >> > 47M > > > >> > Swap: 15G 6.8G 9.2G > > > >> > > > > >> > ... after the Payroll application is started (approx.) I see these > > > >> > exceptions appearing in the logs (a whole series of them): > > > >> > > > > >> > 18:19:36.465 [AsyncResolver-bootstrap-executor-0] INFO > > > >> > c.n.d.s.r.aws.ConfigClusterResolver - Resolving eureka endpoints > via > > > >> > configuration > > > >> > Resolving artifact > > > >> > org.apache.fineract.cn.group:service-boot:jar:0.1.0-BUILD-SNAPSHOT > > > >> > Resolving metadata > > > >> > > > > >> > > > > >> > > > > > > org.apache.fineract.cn.group:service-boot:0.1.0-BUILD-SNAPSHOT/maven-metadata.xml > > > >> > from /root/.m2/repository (enhanced) > > > >> > Resolved metadata > > > >> > > > > >> > > > > >> > > > > > > org.apache.fineract.cn.group:service-boot:0.1.0-BUILD-SNAPSHOT/maven-metadata.xml > > > >> > from /root/.m2/repository (enhanced) > > > >> > Resolved artifact > > > >> > org.apache.fineract.cn.group:service-boot:jar:0.1.0-BUILD-SNAPSHOT > > > from > > > >> > /root/.m2/repository (enhanced) > > > >> > 18:19:44.689 [DiscoveryClient-0] ERROR > > > >> c.n.discovery.TimedSupervisorTask - > > > >> > task supervisor timed out > > > >> > java.util.concurrent.TimeoutException: null > > > >> > at java.util.concurrent.FutureTask.get(FutureTask.java:205) > > > >> > at > > > >> > > > > >> > > > > > > com.netflix.discovery.TimedSupervisorTask.run(TimedSupervisorTask.java:64) > > > >> > at > > > >> > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > > >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > > >> > at > > > >> > > > > >> > > > > >> > > > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > > > >> > at > > > >> > > > > >> > > > > >> > > > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > > > >> > at > > > >> > > > > >> > > > > >> > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > > >> > at > > > >> > > > > >> > > > > >> > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > > >> > at java.lang.Thread.run(Thread.java:748) > > > >> > > > > >> > ... but it still seems to have no problem starting the next app > > > (Group) > > > >> and > > > >> > things look good for a while... > > > >> > > > > >> > ... until suddenly it seems that app instances are beginning to > > > >> shutdown... > > > >> > and I'm not sure why this happens: > > > >> > > > > >> > 18:27:19.603 [AsyncResolver-bootstrap-executor-0] INFO > > > >> > c.n.d.s.r.aws.ConfigClusterResolver - Resolving eureka endpoints > via > > > >> > configuration > > > >> > 18:27:39.974 [qtp1631119258-22] INFO > > > >> c.datastax.driver.core.ClockFactory - > > > >> > Using native clock to generate timestamps. > > > >> > 18:27:39.975 [qtp1631119258-22] WARN > > > >> c.datastax.driver.core.CodecRegistry > > > >> > - Ignoring codec LocalDateTimeCodec [timestamp <-> > > > >> java.time.LocalDateTime] > > > >> > because it collides with previously registered codec > > > LocalDateTimeCodec > > > >> > [timestamp <-> java.time.LocalDateTime] > > > >> > 18:27:41.309 [qtp1631119258-22] INFO > > > c.d.d.c.p.DCAwareRoundRobinPolicy > > > >> - > > > >> > Using data-center name 'datacenter1' for DCAwareRoundRobinPolicy > (if > > > >> this > > > >> > is incorrect, please provide the correct datacenter name with > > > >> > DCAwareRoundRobinPolicy constructor) > > > >> > 18:27:41.319 [qtp1631119258-22] INFO > > > com.datastax.driver.core.Cluster - > > > >> > New Cassandra host /127.0.0.1:9042 added > > > >> > 18:27:41.661 [AsyncResolver-bootstrap-executor-0] INFO > > > >> > c.n.d.s.r.aws.ConfigClusterResolver - Resolving eureka endpoints > via > > > >> > configuration > > > >> > 18:27:51.110 [Thread-15] INFO > > > >> > o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext - Closing > > > >> > > > > >> > > > > >> > > > > > > org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@11438d26 > > > >> > : > > > >> > startup date [Tue Jun 12 17:55:35 UTC 2018]; parent: > > > >> > > > > >> > > > > >> > > > > > > org.springframework.context.annotation.AnnotationConfigApplicationContext@7dc5e7b4 > > > >> > 18:27:51.341 [AsyncResolver-bootstrap-executor-0] INFO > > > >> > c.n.d.s.r.aws.ConfigClusterResolver - Resolving eureka endpoints > via > > > >> > configuration > > > >> > 18:27:53.373 [Thread-15] INFO o.s.c.s.DefaultLifecycleProcessor - > > > >> Stopping > > > >> > beans in phase 2147483647 > > > >> > 18:27:54.331 [Thread-15] INFO o.s.c.s.DefaultLifecycleProcessor - > > > >> Stopping > > > >> > beans in phase 0 > > > >> > 18:27:54.736 [Thread-15] INFO > o.s.b.a.e.jmx.EndpointMBeanExporter - > > > >> > Unregistering JMX-exposed beans on shutdown > > > >> > 18:27:54.740 [Thread-15] INFO > o.s.b.a.e.jmx.EndpointMBeanExporter - > > > >> > Unregistering JMX-exposed beans > > > >> > 18:27:54.798 [Thread-15] INFO o.s.j.e.a.AnnotationMBeanExporter - > > > >> > Unregistering JMX-exposed beans on shutdown > > > >> > 18:27:54.800 [Thread-15] INFO o.s.j.e.a.AnnotationMBeanExporter - > > > >> > Unregistering JMX-exposed beans > > > >> > 18:27:54.937 [Thread-15] INFO > > > >> > o.s.o.j.LocalContainerEntityManagerFactoryBean - Closing JPA > > > >> > EntityManagerFactory for persistence unit 'metaPU' > > > >> > 18:27:58.376 [async-processor-1] INFO > > > >> o.f.c.internal.util.VersionPrinter - > > > >> > Flyway 3.2.1 by Boxfuse > > > >> > 18:27:58.677 [async-processor-1] INFO > > > >> o.f.c.i.dbsupport.DbSupportFactory - > > > >> > Database: jdbc:mysql://localhost:3306/seshat (MySQL 10.3) > > > >> > 18:28:00.620 [async-processor-1] INFO > > > >> o.f.core.internal.command.DbValidate > > > >> > - Validated 2 migrations (execution time 00:01.169s) > > > >> > 18:28:00.901 [Thread-15] INFO o.e.jetty.server.AbstractConnector > - > > > >> Stopped > > > >> > ServerConnector@e162a35{HTTP/1.1,[http/1.1]}{0.0.0.0:2020} > > > >> > 18:28:01.268 [Thread-15] INFO > o.e.j.server.handler.ContextHandler - > > > >> > Stopped o.s.b.c.e.j.JettyEmbeddedWebAppContext@10850d17 > > > >> > > > > >> > > > > >> > > > > > > {/provisioner/v1,file:///tmp/jetty-docbase.5486166363042759626.2020/,UNAVAILABLE} > > > >> > > > > >> > ... the list is actually a lot longer... in the rest of the log > you > > > can > > > >> see > > > >> > that one by one all the apps are shutting down. > > > >> > > > > >> > The last (error) entries I see are these: > > > >> > > > > >> > E18:29:31.278 [DefaultMessageListenerContainer-3] ERROR > > > >> > o.s.j.l.DefaultMessageListenerContainer - Could not refresh JMS > > > >> Connection > > > >> > for destination 'cheques-v1' - retrying using > > > >> FixedBackOff{interval=5000, > > > >> > currentAttempts=0, maxAttempts=unlimited}. Cause: Error while > > > >> attempting to > > > >> > retrieve a connection from the pool; nested exception is > > > >> > javax.jms.JMSException: Could not connect to broker URL: > > > >> > tcp://localhost:61616. Reason: java.net.ConnectException: > Connection > > > >> > refused (Connection refused) > > > >> > > > > >> > ... I guess an embedded ActiveMQ was also shutdown in the process > of > > > the > > > >> > previous errors. > > > >> > > > > >> > So far from here... any input on the above highly appreciated. > > > Overall I > > > >> > think we are close. > > > >> > > > > >> > Cheers, > > > >> > > > > >> > Aleks > > > >> > > > > >> > > > > >> > On Tue, Jun 12, 2018 at 6:16 PM Markus Geiss <[email protected]> > > wrote: > > > >> > > > > >> > > One thing we maybe can do, given we are part of the Apache > Family, > > > is > > > >> to > > > >> > > ask the Cassandra community for some suggestions. > > > >> > > > > > >> > > Cheers > > > >> > > > > > >> > > Markus > > > >> > > > > > >> > > On Tue, Jun 12, 2018 at 6:04 PM Markus Geiss <[email protected]> > > > wrote: > > > >> > > > > > >> > > > Hey all, > > > >> > > > > > > >> > > > we are not running Cassandra in a container, we have dedicated > > VMs > > > >> to > > > >> > run > > > >> > > > our cluster. > > > >> > > > > > > >> > > > Given this we are not the best persons to help here, sorry. (; > > > >> > > > > > > >> > > > Cheers > > > >> > > > > > > >> > > > Markus > > > >> > > > > > > >> > > > On Tue, Jun 12, 2018 at 10:35 AM Ed Cable <[email protected]> > > > >> wrote: > > > >> > > > > > > >> > > >> Adding Patric directly to this thread too so he can give his > > > input > > > >> > > >> regarding to the challenges with Cassandra. > > > >> > > >> > > > >> > > >> Ed > > > >> > > >> On Tue, Jun 12, 2018 at 12:53 AM Aleksandar Vidakovic < > > > >> > > >> [email protected]> wrote: > > > >> > > >> > > > >> > > >>> Hi Victor, > > > >> > > >>> > > > >> > > >>> .... this is the relevant part of the docker-compose.yml > file: > > > >> > > >>> > > > >> > > >>> cassandra0: > > > >> > > >>> image: cassandra:3.11.1 > > > >> > > >>> container_name: cassandra0 > > > >> > > >>> ports: > > > >> > > >>> - 9042:9042 > > > >> > > >>> - 9160:9160 > > > >> > > >>> - 7199:7199 > > > >> > > >>> - 8778:8778 > > > >> > > >>> volumes: > > > >> > > >>> - ./cassandra:/etc/cassandra > > > >> > > >>> environment: > > > >> > > >>> - CASSANDRA_START_RPC=true > > > >> > > >>> - CASSANDRA_SEEDS=cassandra0 > > > >> > > >>> - CASSANDRA_CLUSTER_NAME=fineract_cluster > > > >> > > >>> ulimits: > > > >> > > >>> memlock: -1 > > > >> > > >>> nproc: 32768 > > > >> > > >>> nofile: 100000 > > > >> > > >>> > > > >> > > >>> ... and just to be complete... here's the Docker service > > > >> > configuration > > > >> > > >>> ("/lib/systemd/system/docker.service"): > > > >> > > >>> > > > >> > > >>> [Unit] > > > >> > > >>> Description=Docker Application Container Engine > > > >> > > >>> Documentation=https://docs.docker.com > > > >> > > >>> After=network-online.target docker.socket firewalld.service > > > >> > > >>> Wants=network-online.target > > > >> > > >>> Requires=docker.socket > > > >> > > >>> > > > >> > > >>> [Service] > > > >> > > >>> Type=notify > > > >> > > >>> # the default is not to use systemd for cgroups because the > > > >> delegate > > > >> > > >>> issues > > > >> > > >>> still > > > >> > > >>> # exists and systemd currently does not support the cgroup > > > feature > > > >> > set > > > >> > > >>> required > > > >> > > >>> # for containers run by docker > > > >> > > >>> ExecStart=/usr/bin/dockerd -H fd:// > > > >> > > >>> ExecReload=/bin/kill -s HUP $MAINPID > > > >> > > >>> LimitNOFILE=1048576 > > > >> > > >>> # Having non-zero Limit*s causes performance problems due to > > > >> > accounting > > > >> > > >>> overhead > > > >> > > >>> # in the kernel. We recommend using cgroups to do > > > container-local > > > >> > > >>> accounting. > > > >> > > >>> LimitNPROC=infinity > > > >> > > >>> LimitCORE=infinity > > > >> > > >>> LimitMEMLOCK=infinity > > > >> > > >>> # Uncomment TasksMax if your systemd version supports it. > > > >> > > >>> # Only systemd 226 and above support this version. > > > >> > > >>> TasksMax=infinity > > > >> > > >>> TimeoutStartSec=0 > > > >> > > >>> # set delegate yes so that systemd does not reset the > cgroups > > of > > > >> > docker > > > >> > > >>> containers > > > >> > > >>> Delegate=yes > > > >> > > >>> # kill only the docker process, not all processes in the > > cgroup > > > >> > > >>> KillMode=process > > > >> > > >>> # restart the docker process if it exits prematurely > > > >> > > >>> Restart=on-failure > > > >> > > >>> StartLimitBurst=3 > > > >> > > >>> StartLimitInterval=60s > > > >> > > >>> > > > >> > > >>> [Install] > > > >> > > >>> WantedBy=multi-user.target > > > >> > > >>> > > > >> > > >>> ... the one tweak I did there is to set LimitMEMLOCK to > > > >> infinity... > > > >> > do > > > >> > > >>> you > > > >> > > >>> think the problems are file handle related? Should I set > > > >> LimitNOFILE > > > >> > > also > > > >> > > >>> to infinity? > > > >> > > >>> > > > >> > > >>> Appreciate the help. > > > >> > > >>> > > > >> > > >>> Cheers, > > > >> > > >>> > > > >> > > >>> Aleks > > > >> > > >>> > > > >> > > >>> On Tue, Jun 12, 2018 at 9:09 AM Victor Romero < > > > >> > > >>> [email protected]> > > > >> > > >>> wrote: > > > >> > > >>> > > > >> > > >>> > Hi Aleks, > > > >> > > >>> > > > > >> > > >>> > Can you share the ulimits flags that the cassandra's > > container > > > >> is > > > >> > > using > > > >> > > >>> > while it is running? > > > >> > > >>> > > > > >> > > >>> > The values are being set in the composer file? Or in the > > > >> upstart or > > > >> > > >>> > systemd docker's deamon config files? > > > >> > > >>> > > > > >> > > >>> > > > > >> > > >>> > > > > >> > > >>> > Enviado desde TypeApp > > > >> > > >>> > > > > >> > > >>> > En jun. 11, 2018 10:02 AM, en 10:02 AM, Aleksandar > > Vidakovic < > > > >> > > >>> > [email protected]> escribió: > > > >> > > >>> > >... and I should have attached the Cassandra log dump... > so > > > >> here > > > >> > it > > > >> > > >>> > >is... > > > >> > > >>> > > > > > >> > > >>> > >On Mon, Jun 11, 2018 at 4:49 PM Aleksandar Vidakovic < > > > >> > > >>> > >[email protected]> wrote: > > > >> > > >>> > > > > > >> > > >>> > >> Hi all, > > > >> > > >>> > >> > > > >> > > >>> > >> ... as you might have noticed I did a couple of > restarts > > > >> > today... > > > >> > > >>> the > > > >> > > >>> > >> problem I am facing now: I can't get Cassandra to run > in > > a > > > >> > stable > > > >> > > >>> > >way. > > > >> > > >>> > >> > > > >> > > >>> > >> Things look quite OK for a while when running the demo > > > >> server, > > > >> > but > > > >> > > >>> > >then > > > >> > > >>> > >> suddenly Cassandra dies (sometimes it starts > misbehaving > > > with > > > >> > the > > > >> > > >>> > >deposit > > > >> > > >>> > >> microservice startup, sometimes with portfolio or > > teller). > > > I > > > >> > tried > > > >> > > >>> to > > > >> > > >>> > >> increase the memory (4G, 8G and 16G) and set some > Docker > > > >> limits > > > >> > to > > > >> > > >>> > >> "infinity" (especially LimitMEMLOCK). > > > >> > > >>> > >> > > > >> > > >>> > >> I've attached Cassandra's log dump... maybe someone can > > > help > > > >> out > > > >> > > >>> > >here? Is > > > >> > > >>> > >> it even possible to run Fineract CN on 32GB of memory? > > > >> > > >>> > >> > > > >> > > >>> > >> Other than that the setup would be ready to go... we > just > > > >> need > > > >> > to > > > >> > > >>> get > > > >> > > >>> > >the > > > >> > > >>> > >> database running more reliably. > > > >> > > >>> > >> > > > >> > > >>> > >> Cheers, > > > >> > > >>> > >> > > > >> > > >>> > >> Aleks > > > >> > > >>> > >> > > > >> > > >>> > >> On Mon, Jun 11, 2018 at 2:38 PM Aleksandar Vidakovic < > > > >> > > >>> > >> [email protected]> wrote: > > > >> > > >>> > >> > > > >> > > >>> > >>> @Rajan: I had to restart it again (needed to add some > > > >> > additional > > > >> > > >>> > >reverse > > > >> > > >>> > >>> proxy configuration for the web UI to work)... and > there > > > >> were > > > >> > > more > > > >> > > >>> > >>> exceptions that I hope will be fixed now. > > > >> > > >>> > >>> > > > >> > > >>> > >>> Just FYI > > > >> > > >>> > >>> > > > >> > > >>> > >>> On Mon, Jun 11, 2018 at 1:47 PM Aleksandar Vidakovic < > > > >> > > >>> > >>> [email protected]> wrote: > > > >> > > >>> > >>> > > > >> > > >>> > >>>> No prob.... Let me know how it goes... Can get back > to > > > >> testing > > > >> > > >>> only > > > >> > > >>> > >>>> later tonight. > > > >> > > >>> > >>>> > > > >> > > >>> > >>>> Cheers > > > >> > > >>> > >>>> > > > >> > > >>> > >>>> > > > >> > > >>> > >>>> On Mon, Jun 11, 2018, 1:45 PM Rajan Maurya > > > >> > > >>> > ><[email protected]> > > > >> > > >>> > >>>> wrote: > > > >> > > >>> > >>>> > > > >> > > >>> > >>>>> Sorry missed the 30 min, I will test after 30 min. > > > >> > > >>> > >>>>> > > > >> > > >>> > >>>>> Big thanks for this 🙂 > > > >> > > >>> > >>>>> > > > >> > > >>> > >>>>> > > > >> > > >>> > >>>>> > > > >> > > >>> > >>>>> > > > >> > > >>> > >>>>> On Mon, Jun 11, 2018 at 5:12 PM Aleksandar > Vidakovic < > > > >> > > >>> > >>>>> [email protected]> wrote: > > > >> > > >>> > >>>>> > > > >> > > >>> > >>>>> > @Rajan: can't see the image you posted. > > > >> > > >>> > >>>>> > > > > >> > > >>> > >>>>> > ... and as I said: the services are still > > starting... > > > >> and > > > >> > the > > > >> > > >>> > >best > > > >> > > >>> > >>>>> estimate > > > >> > > >>> > >>>>> > I have right now (as already mentioned): 30min > > > >> > > >>> > >>>>> > > > > >> > > >>> > >>>>> > On Mon, Jun 11, 2018 at 1:40 PM Rajan Maurya < > > > >> > > >>> > >>>>> [email protected]> > > > >> > > >>> > >>>>> > wrote: > > > >> > > >>> > >>>>> > > > > >> > > >>> > >>>>> > > [image: image.png] > > > >> > > >>> > >>>>> > > > > > >> > > >>> > >>>>> > > > > > >> > > >>> > >>>>> > > > > > >> > > >>> > >>>>> > > > > > >> > > >>> > >>>>> > > > > > >> > > >>> > >>>>> > > On Mon, Jun 11, 2018 at 5:08 PM Rajan Maurya < > > > >> > > >>> > >>>>> [email protected]> > > > >> > > >>> > >>>>> > > wrote: > > > >> > > >>> > >>>>> > > > > > >> > > >>> > >>>>> > >> [image: image.png] > > > >> > > >>> > >>>>> > >> I am getting this. > > > >> > > >>> > >>>>> > >> > > > >> > > >>> > >>>>> > >> > > > >> > > >>> > >>>>> > >> > > > >> > > >>> > >>>>> > >> > > > >> > > >>> > >>>>> > >> > > > >> > > >>> > >>>>> > >> On Mon, Jun 11, 2018 at 5:05 PM Aleksandar > > > Vidakovic > > > >> < > > > >> > > >>> > >>>>> > >> [email protected]> wrote: > > > >> > > >>> > >>>>> > >> > > > >> > > >>> > >>>>> > >>> Hi all, > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> so... the demo server is (almost) ready... it > > took > > > >> me a > > > >> > > >>> > >moment > > > >> > > >>> > >>>>> and a > > > >> > > >>> > >>>>> > >>> couple > > > >> > > >>> > >>>>> > >>> of restarts to figure out some boot > failures... > > > the > > > >> > > >>> services > > > >> > > >>> > >are > > > >> > > >>> > >>>>> quite > > > >> > > >>> > >>>>> > >>> resource hungry and default settings won't do > > it. > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> The services are still starting and this will > > > take a > > > >> > > while > > > >> > > >>> > >(my > > > >> > > >>> > >>>>> best > > > >> > > >>> > >>>>> > guess > > > >> > > >>> > >>>>> > >>> right now is around 30min or so). > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> You can access the web UI at: > > > >> > > >>> > >http://fineract-vm.apache.org/login > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> Credentials: > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> Tenant : playground > > > >> > > >>> > >>>>> > >>> Username : operator > > > >> > > >>> > >>>>> > >>> Password : init1@l > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> Note (to self): the safest way to compile this > > app > > > >> is > > > >> > > with > > > >> > > >>> > >NodeJS > > > >> > > >>> > >>>>> > 8.11.1 > > > >> > > >>> > >>>>> > >>> (I tried with 10.3.0 before, won't work). > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> ... and the webservices at: > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> Identity Service: > > > >> > > >>> > >http://fineract-vm.apache.org:2021/identity/v1 > > > >> > > >>> > >>>>> > >>> Office Service: > > > >> > > >>> http://fineract-vm.apache.org:2023/office/v1 > > > >> > > >>> > >>>>> > >>> Customer Service: > > > >> > > >>> > >http://fineract-vm.apache.org:2024/customer/v1 > > > >> > > >>> > >>>>> > >>> Accounting Service: > > > >> > > >>> > >>>>> http://fineract-vm.apache.org:2025/accounting/v1 > > > >> > > >>> > >>>>> > >>> Portfolio Service: > > > >> > > >>> > >>>>> http://fineract-vm.apache.org:2026/portfolio/v1 > > > >> > > >>> > >>>>> > >>> Deposit Service: > > > >> > > >>> > >http://fineract-vm.apache.org:2027/deposit/v1 > > > >> > > >>> > >>>>> > >>> Teller Service: > > > >> > > >>> http://fineract-vm.apache.org:2028/teller/v1 > > > >> > > >>> > >>>>> > >>> Reporting Service: > > > >> > > >>> > >>>>> http://fineract-vm.apache.org:2029/reporting/v1 > > > >> > > >>> > >>>>> > >>> Cheque Service: > > > >> > > >>> > >http://fineract-vm.apache.org:2030/cheques/v1 > > > >> > > >>> > >>>>> > >>> Payroll Service: > > > >> > > >>> > >http://fineract-vm.apache.org:2031/payroll/v1 > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> Note: restarting the services takes quite a > > > >> while... if > > > >> > > you > > > >> > > >>> > >>>>> encounter > > > >> > > >>> > >>>>> > >>> connection problems retry a couple of minutes > > > later > > > >> to > > > >> > > >>> > >ensure I > > > >> > > >>> > >>>>> am not > > > >> > > >>> > >>>>> > >>> currently working on something. > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> For the moment I'll restart the backend > services > > > >> once > > > >> > per > > > >> > > >>> > >day or > > > >> > > >>> > >>>>> so to > > > >> > > >>> > >>>>> > >>> reset the data; I guess we have to figure out > > how > > > we > > > >> > want > > > >> > > >>> to > > > >> > > >>> > >>>>> handle > > > >> > > >>> > >>>>> > this > > > >> > > >>> > >>>>> > >>> (also concerning passwords etc.). > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> If you have any suggestions where to put this > > demo > > > >> > server > > > >> > > >>> > >>>>> configuration > > > >> > > >>> > >>>>> > >>> then let me know here. > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> Let me know if you encounter any problems (I > > have > > > >> not > > > >> > > >>> > >extensively > > > >> > > >>> > >>>>> > tested > > > >> > > >>> > >>>>> > >>> it > > > >> > > >>> > >>>>> > >>> yet). > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> Cheers, > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> Aleks > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> On Mon, Jun 11, 2018 at 12:00 PM Aleksandar > > > >> Vidakovic < > > > >> > > >>> > >>>>> > >>> [email protected]> wrote: > > > >> > > >>> > >>>>> > >>> > > > >> > > >>> > >>>>> > >>> > Progress! All the modules seem to be > starting > > > fine > > > >> > > now... > > > >> > > >>> > >just > > > >> > > >>> > >>>>> have > > > >> > > >>> > >>>>> > to > > > >> > > >>> > >>>>> > >>> > work on a couple of improvements on the > config > > > >> files. > > > >> > > >>> > >>>>> > >>> > > > > >> > > >>> > >>>>> > >>> > Currently having another stab to get the > > webapp > > > >> > > running. > > > >> > > >>> > >>>>> > >>> > > > > >> > > >>> > >>>>> > >>> > BTW: it should be fairly easy to adapt the > > > >> simplistic > > > >> > > >>> > >Docker > > > >> > > >>> > >>>>> Compose > > > >> > > >>> > >>>>> > >>> file > > > >> > > >>> > >>>>> > >>> > for a CI pipeline. > > > >> > > >>> > >>>>> > >>> > > > > >> > > >>> > >>>>> > >>> > Will keep you posted. > > > >> > > >>> > >>>>> > >>> > > > > >> > > >>> > >>>>> > >>> > Cheers, > > > >> > > >>> > >>>>> > >>> > > > > >> > > >>> > >>>>> > >>> > Aleks > > > >> > > >>> > >>>>> > >>> > > > > >> > > >>> > >>>>> > >>> > On Mon, Jun 11, 2018 at 9:41 AM Aleksandar > > > >> Vidakovic > > > >> > < > > > >> > > >>> > >>>>> > >>> > [email protected]> wrote: > > > >> > > >>> > >>>>> > >>> > > > > >> > > >>> > >>>>> > >>> >> I am still working on it... Will ping you > > here > > > >> when > > > >> > > it's > > > >> > > >>> > >ready > > > >> > > >>> > >>>>> (hope > > > >> > > >>> > >>>>> > >>> >> soon). > > > >> > > >>> > >>>>> > >>> >> > > > >> > > >>> > >>>>> > >>> >> On Mon, Jun 11, 2018, 9:40 AM Rajan Maurya > < > > > >> > > >>> > >>>>> > [email protected]> > > > >> > > >>> > >>>>> > >>> >> wrote: > > > >> > > >>> > >>>>> > >>> >> > > > >> > > >>> > >>>>> > >>> >>> Hi Alex, > > > >> > > >>> > >>>>> > >>> >>> > > > >> > > >>> > >>>>> > >>> >>> Is the demo server up, If yes then what's > > the > > > >> base > > > >> > > url > > > >> > > >>> > >so I > > > >> > > >>> > >>>>> can > > > >> > > >>> > >>>>> > >>> access. > > > >> > > >>> > >>>>> > >>> >>> > > > >> > > >>> > >>>>> > >>> >>> > > > >> > > >>> > >>>>> > >>> >>> > > > >> > > >>> > >>>>> > >>> >>> > > > >> > > >>> > >>>>> > >>> >>> On Fri, Jun 8, 2018 at 2:03 PM Rajan > Maurya > > < > > > >> > > >>> > >>>>> > >>> [email protected]> > > > >> > > >>> > >>>>> > >>> >>> wrote: > > > >> > > >>> > >>>>> > >>> >>> > > > >> > > >>> > >>>>> > >>> >>> > Hi Alex, > > > >> > > >>> > >>>>> > >>> >>> > > > > >> > > >>> > >>>>> > >>> >>> > This is really great news, I will wait > > until > > > >> > then. > > > >> > > >>> > >>>>> > >>> >>> > > > > >> > > >>> > >>>>> > >>> >>> > Thanks > > > >> > > >>> > >>>>> > >>> >>> > Rajan Maurya > > > >> > > >>> > >>>>> > >>> >>> > > > > >> > > >>> > >>>>> > >>> >>> > > > > >> > > >>> > >>>>> > >>> >>> > On Fri, Jun 8, 2018 at 11:36 AM > Aleksandar > > > >> > > Vidakovic > > > >> > > >>> < > > > >> > > >>> > >>>>> > >>> >>> > [email protected]> wrote: > > > >> > > >>> > >>>>> > >>> >>> > > > > >> > > >>> > >>>>> > >>> >>> >> Hi Rajan, > > > >> > > >>> > >>>>> > >>> >>> >> > > > >> > > >>> > >>>>> > >>> >>> >> My plan is to get this up and running > > until > > > >> > > Monday. > > > >> > > >>> > >>>>> > >>> >>> >> > > > >> > > >>> > >>>>> > >>> >>> >> Cheers, > > > >> > > >>> > >>>>> > >>> >>> >> > > > >> > > >>> > >>>>> > >>> >>> >> Aleks > > > >> > > >>> > >>>>> > >>> >>> >> > > > >> > > >>> > >>>>> > >>> >>> >> On Fri, Jun 8, 2018, 7:23 AM Rajan > > Maurya < > > > >> > > >>> > >>>>> > >>> [email protected]> > > > >> > > >>> > >>>>> > >>> >>> >> wrote: > > > >> > > >>> > >>>>> > >>> >>> >> > > > >> > > >>> > >>>>> > >>> >>> >> > Hi Markus, Mark, Myrle, Alex, Ed > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > >>> > >>>>> > >>> >>> >> > I am mentoring fineract-cn-mobile > GSoC > > > >> > project. > > > >> > > I > > > >> > > >>> > >am > > > >> > > >>> > >>>>> waiting > > > >> > > >>> > >>>>> > for > > > >> > > >>> > >>>>> > >>> >>> this > > > >> > > >>> > >>>>> > >>> >>> >> demo > > > >> > > >>> > >>>>> > >>> >>> >> > server for a long time to see what's > > most > > > >> > > >>> important > > > >> > > >>> > >to > > > >> > > >>> > >>>>> > implement > > > >> > > >>> > >>>>> > >>> >>> in-app > > > >> > > >>> > >>>>> > >>> >>> >> > first. > > > >> > > >>> > >>>>> > >>> >>> >> > My GSoC students almost finished the > > task > > > >> that > > > >> > > was > > > >> > > >>> > >>>>> necessary. > > > >> > > >>> > >>>>> > We > > > >> > > >>> > >>>>> > >>> >>> have > > > >> > > >>> > >>>>> > >>> >>> >> kinda > > > >> > > >>> > >>>>> > >>> >>> >> > of stuck and We really need this demo > > > >> server > > > >> > and > > > >> > > >>> > >web app > > > >> > > >>> > >>>>> to > > > >> > > >>> > >>>>> > >>> assign > > > >> > > >>> > >>>>> > >>> >>> them > > > >> > > >>> > >>>>> > >>> >>> >> > tasks. > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > >>> > >>>>> > >>> >>> >> > It would be nice we can get the demo > > > >> server up > > > >> > > >>> > >ASAP. > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > >>> > >>>>> > >>> >>> >> > Thanks > > > >> > > >>> > >>>>> > >>> >>> >> > Rajan Maurya > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > >>> > >>>>> > >>> >>> >> > On Fri, Jun 1, 2018 at 6:41 PM Mark > van > > > >> Veen < > > > >> > > >>> > >>>>> > >>> >>> [email protected]> > > > >> > > >>> > >>>>> > >>> >>> >> > wrote: > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > >>> > >>>>> > >>> >>> >> > > Hi Aleks, > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > >> > > >>> > >>>>> > >>> >>> >> > > Re your second issue: > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > >> > > >>> > >>>>> > >>> >>> >> > > It looks like that node-sass has > not > > > been > > > >> > > >>> > >installed > > > >> > > >>> > >>>>> > correctly. > > > >> > > >>> > >>>>> > >>> >>> Can you > > > >> > > >>> > >>>>> > >>> >>> >> > try > > > >> > > >>> > >>>>> > >>> >>> >> > > to remove the node_modules folder > and > > > run > > > >> > npm > > > >> > > >>> > >install > > > >> > > >>> > >>>>> again? > > > >> > > >>> > >>>>> > >>> This > > > >> > > >>> > >>>>> > >>> >>> >> should > > > >> > > >>> > >>>>> > >>> >>> >> > > run without errors. > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > >> > > >>> > >>>>> > >>> >>> >> > > Mark > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > >> > > >>> > >>>>> > >>> >>> >> > > On Thu, May 31, 2018 at 9:05 PM > > Courage > > > >> > Angeh > > > >> > > < > > > >> > > >>> > >>>>> > >>> >>> [email protected] > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > >>> > >>>>> > >>> >>> >> > > wrote: > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > Well, i can it with containerized > > > >> MariaDB > > > >> > > and > > > >> > > >>> > >it > > > >> > > >>> > >>>>> work till > > > >> > > >>> > >>>>> > >>> my > > > >> > > >>> > >>>>> > >>> >>> >> computer > > > >> > > >>> > >>>>> > >>> >>> >> > > ran > > > >> > > >>> > >>>>> > >>> >>> >> > > > out of space (in the middle of > > > >> > provisioning) > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > On Thu, May 31, 2018, 6:29 PM > > Victor > > > >> > Romero > > > >> > > < > > > >> > > >>> > >>>>> > >>> >>> >> > > [email protected] > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > wrote: > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > Hi Aleks, > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > Could you please share the > > composer > > > >> > file? > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > Courage, > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > Have you tested Mifos while > using > > > >> > MariaDb? > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > I have faced issues while > running > > > >> Mifos > > > >> > > with > > > >> > > >>> > >>>>> MariaDb. > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > Regards, > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > Victor > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > Enviado desde TypeApp > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > En may. 31, 2018 6:10 AM, en > 6:10 > > > AM, > > > >> > > >>> Courage > > > >> > > >>> > >>>>> Angeh < > > > >> > > >>> > >>>>> > >>> >>> >> > > > > [email protected]> > > escribió: > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >You are welcome :-) > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >No, I am not. > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >On Thu, May 31, 2018 at 11:57 > > AM, > > > >> > > >>> Aleksandar > > > >> > > >>> > >>>>> Vidakovic > > > >> > > >>> > >>>>> > < > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >[email protected]> > > wrote: > > > >> > > >>> > >>>>> > >>> >>> >> > > > > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> Hi Courage, > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> ... ok... I see one > difference > > > >> that I > > > >> > > >>> > >have: I > > > >> > > >>> > >>>>> used > > > >> > > >>> > >>>>> > >>> MySQL > > > >> > > >>> > >>>>> > >>> >>> (as > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >described on > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> the top of the Wiki page)... > > > I'll > > > >> try > > > >> > > >>> > >MariaDB > > > >> > > >>> > >>>>> and > > > >> > > >>> > >>>>> > >>> should > > > >> > > >>> > >>>>> > >>> >>> get > > > >> > > >>> > >>>>> > >>> >>> >> the > > > >> > > >>> > >>>>> > >>> >>> >> > > same > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> results as you. > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> Thanks for providing your > > > command > > > >> > > line, I > > > >> > > >>> > >used > > > >> > > >>> > >>>>> the > > > >> > > >>> > >>>>> > >>> same. > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> Are you working on the same > > > >> server? > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> Cheers, > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> Aleks > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> On Thu, May 31, 2018 at > 12:32 > > PM > > > >> > > Courage > > > >> > > >>> > >Angeh > > > >> > > >>> > >>>>> > >>> >>> >> > > > > ><[email protected]> > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> wrote: > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > Hi Aleksandar > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > I containerized MariaDB > and > > > >> > > Cassandra, > > > >> > > >>> > >and I > > > >> > > >>> > >>>>> passed > > > >> > > >>> > >>>>> > >>> it to > > > >> > > >>> > >>>>> > >>> >>> >> demo > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >sever, it > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > works fine. > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > Here is the commad i use: > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > java > > > >> -Ddemoserver.provision=false > > > >> > > >>> > >>>>> > >>> >>> >> -Ddemoserver.persistent=true > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > > >> > > >>> > >-Dcustom.cassandra.contactPoints=0.0.0.0:9142 > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > > >> -Dcassandra.cluster.user=cassandra > > > >> > > >>> > >>>>> > >>> >>> >> > > -Dcassandra.cluster.pwd=password > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > > -Dcustom.mariadb.host=localhost > > > >> > > >>> > >>>>> > >>> >>> -Dcustom.mariadb.user=root > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > > -Dcustom.mariadb.password=mysql > > > >> > -jar > > > >> > > >>> > >>>>> > >>> >>> demo-server-0.1.0-BUILD- > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> SNAPSHOT.jar > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > you might have to re-check > > the > > > >> > values > > > >> > > >>> > >you > > > >> > > >>> > >>>>> passed > > > >> > > >>> > >>>>> > for > > > >> > > >>> > >>>>> > >>> >>> >> cassandra > > > >> > > >>> > >>>>> > >>> >>> >> > and > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> mariadb. > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > Thanks, > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > Courage > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > On Thu, May 31, 2018 at > > 12:02 > > > >> AM, > > > >> > > >>> > >Aleksandar > > > >> > > >>> > >>>>> > >>> Vidakovic < > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > [email protected] > > > > > >> wrote: > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > Hi all, > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > ... I made progress on > the > > > >> demo > > > >> > > >>> server > > > >> > > >>> > >>>>> > >>> installation: > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > > > > >> > > >>> > >>>>> > >>> >>> >> > > > > >> > > - I've followed the > > >
