Hi Isaac, Thanks for this effort.
I have tried testing with all the services. but, I get the error below. *complete log here: *https://pastebin.com/013aTJ8E 1. 06:21:47.050 [Thread-13] INFO o.e.j.server.handler.ContextHandler - Stopped o.s.b.c.e.j.JettyEmbeddedWebAppContext@3e1fd62b {/identity/v1,file:///tmp/jetty-docbase.1273617126957352135.2021/,UNAVAILABLE} 2. E06:21:47.790 [DefaultMessageListenerContainer-1] ERROR o.s.j.l.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'portfolio-v1' - retrying using FixedBackOff{interval=5000, currentAttempts=0, maxAttempts=unlimited}. Cause: Error while attempting to retrieve a connection from the pool; nested exception is javax.jms.JMSException: Could not connect to broker URL: tcp://localhost:61616. Reason: java.net.ConnectException: Connection refused (Connection refused) I usually get this same error even with the MariaDB setup. I know its related to the Message broker but I don't know how to resolve it or how to avoid it. Can anyone share their experience on this issue? Also, I dropped the playground and seshat keyspaces in Cassandra as well as in Postgres. I changed the max-connection to 120 in PostgresSQL and also restarted the system. I am convinced the problem is not with PostgreSQL or Cassandra, plus, it worked a few hours ago when I started it in lite mode. PS: In light of this, how soon will this be merged into the develop branches. I would love to completely switch to PostgreSQL. Toggling between MariaDB and PostgreSQL is not easy :) *At your service,* *Graham.* On Thu, 9 May 2019 at 20:13, Isaac Kamga <[email protected]> wrote: > Hello Dev, > > Thanks very much for testing Graham > > I've raised pull requests for the Migration to PostgreSQL. Demo-server > runs to completion successfully in both cases and databases and tables are > rightly being created. > I had to make some adjustments to default-setup's ledgers.csv file because > of an error that came from the CSVParser. > > *Testing demo-server lite for minimal set of services* > I have fixed the issue with connecting to the right databases. You can use > the postgres-setup-lite.sh > <https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh> > script to test the minimal set of libraries and services. After the run, > you should be able to ascertain that databases are created and tables are > created where they have to be. > > +Juhan Aasaru <[email protected]> please provide feedback how this test > goes. > > *Testing demo-server for all services* > Given that the connection issues have been resolved, it's time to do more > testing; > @Juhan, @Graham and anyone else, in order to test the migration in it's > entirety using postgres-setup.sh script > <https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup.sh>, > ensure to; > > 1. Install PostgreSQL > 2. Change password for user postgres to postgres > 3. Change max_connections to 120 in your postgresql.conf file ( usually in > /etc/postgresql/... ) and restart the Postgresql service.Many connections > to databases happening since all services are started. > 4. Ensure Cassandra has no keyspaces seshat and playground > 5. cd into a completely new directory from your usual Fineract CN > development pwd > 5. In integration-tests/fineract-cn-demo-server/build/libs/ and Run > demo-server jar using *java -Ddemoserver.provision=true > -Ddemoserver.persistent=true > -Dcustom.cassandra.contactPoints=127.0.0.1:9042 > <http://127.0.0.1:9042/> -Dcassandra.cluster.user=cassandra > -Dcassandra.cluster.pwd=password -Dcustom.postgresql.host=localhost > -Dcustom.postgresql.user=postgres -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar* > > Kindly provide feedback on how the run goes in the both cases. > > At Your Service, > Isaac Kamga. > > On Fri, May 3, 2019 at 7:21 PM Ebenezer Graham <[email protected]> > wrote: > >> Hello Isaac / Juhan >> >> I missed a number of relevant steps and this led to the initial failure. >> >> The test has been successful and I will list the conditions which led to >> the successful run below. >> >> 1. I created a new directory and cloned the repositories using the script >> you provided ( >> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh >> ) >> >> 2. I installed PostgreSQL server and changed the postgres user's password >> to "postgres" >> >> 3. I used the command below to start the demo-server. >> >> java -Ddemoserver.provision=true -Ddemoserver.persistent=true >> -Ddemoserver.lite=true -Dcustom.cassandra.contactPoints=127.0.0.1:9042 >> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password >> -Dcustom.postgresql.host=localhost -Dcustom.postgresql.user=postgres -jar >> demo-server-0.1.0-BUILD-SNAPSHOT.jar >> >> @Juhan, please try this approach and let's know the outcome. >> >> You should get an output similar to this. >> >> 18:11:28.717 [Thread-16] INFO o.e.jetty.server.AbstractConnector - >> Stopped ServerConnector@59a94d0f{HTTP/1.1,[http/1.1]}{0.0.0.0:2020} >> 18:11:28.720 [Thread-16] INFO o.e.j.server.handler.ContextHandler - >> Stopped o.s.b.c.e.j.JettyEmbeddedWebAppContext@1213ffbc >> {/provisioner/v1,file:///tmp/jetty-docbase.8613597725021372791.2020/,UNAVAILABLE} >> identity-v1 address:http://localhost:2021/identity/v1 >> office-v1 address:http://localhost:2023/office/v1 >> customer-v1 address:http://localhost:2024/customer/v1 >> >> At your service >> >> >> On Fri, 3 May 2019 at 21:09, Ebenezer Graham <[email protected]> >> wrote: >> >>> Hello Isaac/Juhan, >>> >>> I have tried testing the code in lite mode for 3 times and I got the error >>> below. >>> https://pastebin.com/TaJqSNjQ >>> >>> Before the failure, I got this info. >>> >>> 16:56:03.519 [qtp1262607708-22] INFO o.a.f.c.l.c.ServiceExceptionFilter >>> - Responding with a service error ServiceError{code=409, message='The given >>> identity instance didnt recognize the system token as valid. Perhaps the >>> system keys for the provisioner or for the identity manager are >>> misconfigured?'} >>> 16:57:03.620 [qtp1262607708-20] INFO o.a.f.c.l.c.ServiceExceptionFilter >>> - Responding with a service error ServiceError{code=409, message='The given >>> identity instance didnt recognize the system token as valid. Perhaps the >>> system keys for the provisioner or for the identity manager are >>> misconfigured?'} >>> >>> I will keep testing to get more insights into the error. >>> >>> *At your service,* >>> >>> *Ebenezer Graham* >>> >>> *BSc (Hons) Computing* >>> >>> >>> [image: EmailSignature.png] >>> >>> African Leadership University, >>> >>> Power Mill Road, Pamplemousses, >>> >>> Mauritius. >>> GitHub <https://github.com/ebenezergraham> | LinkedIn >>> <https://www.linkedin.com/in/ebenezer-graham/> | ebenezergraham.me >>> *Skype*: >>> ebenezer.graham <http://ebenezergraham.me/> >>> >>> *This email and any files transmitted with it are confidential and >>> intended solely for the use of the individual or entity to whom they are >>> addressed. If you have received this email in error please notify the >>> sender and then delete this email. If you are not the intended recipient >>> you are notified that disclosing, copying, distributing or taking any >>> action in reliance on the contents of this information is strictly >>> prohibited.* >>> >>> >>> On Thu, 2 May 2019 at 06:38, Isaac Kamga <[email protected]> wrote: >>> >>>> Hello Juhan, >>>> >>>> I've updated the PRs with commits to fix the errors you were >>>> experiencing. >>>> >>>> Could you kindly wipe Cassandra and PostgreSQL clean, test once more >>>> and provide feedback on how it goes ? >>>> >>>> Cheers, >>>> Isaac Kamga. >>>> >>>> On Tue, Apr 9, 2019 at 4:36 PM Isaac Kamga <[email protected]> >>>> wrote: >>>> >>>>> Hey Juhan, >>>>> >>>>> Thanks for testing. >>>>> >>>>> Wipe clean your Cassandra keyspaces first. >>>>> >>>>> I suspect that what's interferring is the keyspaces from your last run >>>>> when using MariaDB. >>>>> >>>>> Please provide feedback how it goes. >>>>> >>>>> Cheers, >>>>> Isaac Kamga. >>>>> >>>>> On Sun, Mar 31, 2019 at 10:00 AM Juhan Aasaru <[email protected]> >>>>> wrote: >>>>> >>>>>> I Isaac, >>>>>> >>>>>> I pulled the changes and rebuilt the projects. >>>>>> >>>>>> Now when I run (postgres and cassandra working locally): >>>>>> >>>>>> java -Ddemoserver.persistent=true -Ddemoserver.provision=true >>>>>> -Dcustom.cassandra.contactPoints=127.0.0.1:9042 >>>>>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password >>>>>> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar >>>>>> >>>>>> I get an error here: >>>>>> >>>>>> 11:18:42.066 [main] INFO o.s.b.c.e.j.JettyEmbeddedServletContainer - >>>>>> Jetty >>>>>> started on port(s) 2024 (http/1.1) >>>>>> 11:18:42.067 [main] INFO >>>>>> o.s.c.n.e.EurekaDiscoveryClientConfiguration - >>>>>> Updating port to 2024 >>>>>> 11:18:42.072 [main] INFO o.a.f.c.customer.CustomerApplication - >>>>>> Started >>>>>> CustomerApplication in 16.423 seconds (JVM running for 17.342) >>>>>> 11:18:57.435 [qtp1479696465-21] INFO >>>>>> o.a.f.c.l.c.ServiceExceptionFilter - >>>>>> Responding with a service error ServiceError{code=409, >>>>>> message='Application >>>>>> identity-v1 already exists!'} >>>>>> 11:18:57.464 [Thread-16] INFO >>>>>> o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext - Closing >>>>>> >>>>>> org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@4d50efb8 >>>>>> : >>>>>> startup date [Sun Mar 31 11:12:52 EEST 2019]; parent: >>>>>> >>>>>> org.springframework.context.annotation.AnnotationConfigApplicationContext@75a1cd57 >>>>>> >>>>>> I understand I have to use -Ddemoserver.provision=true flag to >>>>>> provision >>>>>> the initial database so I won't be able to take that one out. >>>>>> >>>>>> Juhan >>>>>> >>>>>> >>>>>> Kontakt Isaac Kamga (<[email protected]>) kirjutas kuupäeval R, >>>>>> 29. >>>>>> märts 2019 kell 04:17: >>>>>> >>>>>> > Hey Juhan, >>>>>> > >>>>>> > I have updated the Pull request to fineract-cn-provisioner with a >>>>>> commit >>>>>> > [1] which fixes the errors you're experiencing. >>>>>> > >>>>>> > I'll advise that you test using option -Ddemoserver.persistent=true >>>>>> and >>>>>> > -Ddemoserver.lite=true and we're testing just the minimal set of >>>>>> services >>>>>> > now. In fact,I tested and the tail of my logs showed thus >>>>>> > *identity-v1 address:http://localhost:2021/identity/v1 >>>>>> > < >>>>>> > >>>>>> https://www.google.com/url?q=http://localhost:2021/identity/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNEjCID96mw3LF1ZIPmjEIQkHh_LcQ >>>>>> > >* >>>>>> > >>>>>> > *office-v1 address:http://localhost:2023/office/v1 >>>>>> > < >>>>>> > >>>>>> https://www.google.com/url?q=http://localhost:2023/office/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNGjyk1BHroLddOTd0FhJUJG4zdVag >>>>>> > >customer-v1 >>>>>> > address:http://localhost:2024/customer/v1 >>>>>> > < >>>>>> > >>>>>> https://www.google.com/url?q=http://localhost:2024/customer/v1&sa=D&source=hangouts&ust=1553910362599000&usg=AFQjCNGfcoJZIxmcsX34YeEXRUzS0oC2hA >>>>>> > >* >>>>>> > Albeit, this error [2] turned up related to the Notifications >>>>>> service. Just >>>>>> > ignore it for now as it will be handled down the line during >>>>>> extensive >>>>>> > integration testing. >>>>>> > >>>>>> > We can keep testing using -Ddemoserver.provision=true for later too >>>>>> when >>>>>> > we've migrated and tested the individual services. >>>>>> > >>>>>> > Can you kindly update your provisioner repository and test once >>>>>> more and >>>>>> > let's keep fixing ? >>>>>> > >>>>>> > [1] >>>>>> > >>>>>> > >>>>>> https://github.com/Izakey/fineract-cn-provisioner/commit/b83331b723f175003ccf069764414e4ab371662c >>>>>> > >>>>>> > [2] https://pastebin.com/DBzNyHir >>>>>> > >>>>>> > Cheers, >>>>>> > Isaac Kamga. >>>>>> > >>>>>> > On Wed, Mar 27, 2019 at 10:23 PM Isaac Kamga <[email protected] >>>>>> > >>>>>> > wrote: >>>>>> > >>>>>> > > Hello Juhan, >>>>>> > > >>>>>> > > Thank you for taking the time to do some testing. >>>>>> > > >>>>>> > > On Wed, Mar 27, 2019 at 10:56 AM Juhan Aasaru <[email protected]> >>>>>> wrote: >>>>>> > > >>>>>> > >> Hi Isaac, >>>>>> > >> >>>>>> > >> great initiative towards pushing to PostgreSQL. >>>>>> > >> >>>>>> > >> I looked at your instructions and tried to run your changes >>>>>> locally. >>>>>> > >> The sh script came in handy to quickly clone and build >>>>>> everything that >>>>>> > was >>>>>> > >> needed. >>>>>> > >> But then I got confused about provisioning. >>>>>> > >> >>>>>> > >> Question #1: How exactly I should run demo-server? I tried the >>>>>> following >>>>>> > >> options: >>>>>> > >> >>>>>> > >> 1. I didn't have postgre and cassandra running locally. >>>>>> > >> java -jar -Ddemoserver.lite=true >>>>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar >>>>>> > >> >>>>>> > >> resulted >>>>>> > >> >>>>>> > >> org.springframework.beans.factory.BeanCreationException: Error >>>>>> creating >>>>>> > >> bean with name 'provisionerInitializer': >>>>>> > >> Invocation of init method failed; nested exception is >>>>>> > >> java.lang.IllegalStateException: Could not initialize service! >>>>>> > >> Caused by: org.postgresql.util.PSQLException: FATAL: database >>>>>> "seshat" >>>>>> > >> does >>>>>> > >> not exist >>>>>> > >> >>>>>> > > >>>>>> > > I haven't done testing using this option yet. >>>>>> > > >>>>>> > > >>>>>> > >> >>>>>> > >> 2. I started posgresql (and created a database seshat) and >>>>>> cassandra and >>>>>> > >> ran: >>>>>> > >> java -Ddemoserver.provision=true -Ddemoserver.persistent=true >>>>>> > >> -Dcustom.cassandra.contactPoints=127.0.0.1:9042 \ >>>>>> > >> -Dcassandra.cluster.user=cassandra >>>>>> -Dcassandra.cluster.pwd=password >>>>>> > >> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar >>>>>> > >> >>>>>> > >> Initially applications started up one after anther >>>>>> > >> (ProvisionerApplication, >>>>>> > >> RhythmApplication, OfficeApplication) >>>>>> > >> but CustomerApplication threw an exception: >>>>>> > >> >>>>>> > >> 11:50:39.377 [qtp1424482154-21] WARN >>>>>> o.e.jetty.servlet.ServletHandler - >>>>>> > >> org.springframework.web.util.NestedServletException: Request >>>>>> processing >>>>>> > >> failed; nested exception is java.lang.IllegalStateException: >>>>>> Could not >>>>>> > >> insert org.apache.fineract.cn.provisioner.tenant info! >>>>>> > >> Caused by: org.postgresql.util.PSQLException: ERROR: relation >>>>>> > >> "seshat.tenants" does not exist >>>>>> > >> >>>>>> > > >>>>>> > > Try running the same command above without the provision=true >>>>>> option and >>>>>> > > share what you get. >>>>>> > > >>>>>> > > I had to create database seshat in postgresql and the can't find >>>>>> database >>>>>> > > error went away. Down the road, also encountered the same error >>>>>> > > <https://pastebin.com/qCT1T5KM> "Could not insert .... >>>>>> > provisioner.tenant >>>>>> > > info !" >>>>>> > > I'm currently working on resolving this. >>>>>> > > >>>>>> > > >>>>>> > >> >>>>>> > >> Question #2 - do I still need to have Mysql running? I see that >>>>>> you >>>>>> > >> haven't >>>>>> > >> made code changes to identity >>>>>> > >> which as I understand still uses Mysql. >>>>>> > >> >>>>>> > > >>>>>> > > I don't think you need MySQL running to test the migrations >>>>>> currently >>>>>> > > underway and there was no need for code changes in the identity >>>>>> service. >>>>>> > > However, it could be helpful to see the structure of the >>>>>> databases in >>>>>> > MySQL >>>>>> > > so that it helps with debugging and testing. >>>>>> > > >>>>>> > > Whenever you're available, tell me so we schedule a call so I >>>>>> look into >>>>>> > > what your experiencing too. >>>>>> > > >>>>>> > > Cheers, >>>>>> > > Isaac Kamga. >>>>>> > > >>>>>> > > >>>>>> > >> >>>>>> > >> Kind regards >>>>>> > >> Juhan >>>>>> > >> >>>>>> > >> >>>>>> > >> Kontakt Isaac Kamga (<[email protected]>) kirjutas >>>>>> kuupäeval L, 23. >>>>>> > >> märts 2019 kell 04:31: >>>>>> > >> >>>>>> > >> > Hello fineracters, >>>>>> > >> > >>>>>> > >> > Trust that you had a great week. >>>>>> > >> > >>>>>> > >> > I just finished migrating libraries PostgreSQL[1] and Test[2] >>>>>> from >>>>>> > >> MariaDB >>>>>> > >> > to PostgreSQL, mostly adapting SQL statements & database >>>>>> connections >>>>>> > to >>>>>> > >> > work with PostgreSQL and reverting back to Hibernate ORM until >>>>>> > database >>>>>> > >> > migrations are completed. >>>>>> > >> > >>>>>> > >> > I have also migrated the "minimal" services viz provisioner[3], >>>>>> > >> rhythm[4], >>>>>> > >> > office[5], template[6] and customer[7] from MariaDB to >>>>>> PostgreSQL. The >>>>>> > >> > Spring datasource credentials had to be incorporated into >>>>>> > >> application.yaml >>>>>> > >> > files to make things work too. >>>>>> > >> > >>>>>> > >> > Given I updated service-starter [8] and demo-server [9], >>>>>> developers >>>>>> > can >>>>>> > >> > test the migrated services when running the >>>>>> *-Ddemoserver.lite=true* >>>>>> > >> option >>>>>> > >> > which builds the minimal set of services. >>>>>> > >> > >>>>>> > >> > You'll observe that some of the pull requests are heading to >>>>>> develop >>>>>> > >> > branches ( It appears comparing changes and creating PRs >>>>>> towards the >>>>>> > >> same >>>>>> > >> > branch name from origin to upstream expires after a few >>>>>> minutes on >>>>>> > >> > Github... weird ) >>>>>> > >> > >>>>>> > >> > Here is a script [10] to enable anyone clone and do an >>>>>> > integration-test >>>>>> > >> of >>>>>> > >> > the new changes using demo-server. >>>>>> > >> > >>>>>> > >> > Kindly test these changes and provide feedback. >>>>>> > >> > >>>>>> > >> > At Your Service, >>>>>> > >> > Isaac Kamga. >>>>>> > >> > >>>>>> > >> > [1] https://github.com/apache/fineract-cn-postgresql/pull/1 >>>>>> > >> > [2] https://github.com/apache/fineract-cn-test/pull/7 >>>>>> > >> > [3] https://github.com/apache/fineract-cn-provisioner/pull/6 >>>>>> > >> > [4] https://github.com/apache/fineract-cn-rhythm/pull/8 >>>>>> > >> > [5] https://github.com/apache/fineract-cn-office/pull/9 >>>>>> > >> > [6] https://github.com/apache/fineract-cn-template/pull/6 >>>>>> > >> > [7] https://github.com/apache/fineract-cn-customer/pull/10 >>>>>> > >> > [8] >>>>>> https://github.com/apache/fineract-cn-service-starter/pull/8 >>>>>> > >> > [9] https://github.com/apache/fineract-cn-demo-server/pull/27 >>>>>> > >> > [10] >>>>>> > >> > >>>>>> > >> > >>>>>> > >> >>>>>> > >>>>>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh >>>>>> > >> > >>>>>> > >> >>>>>> > > >>>>>> > >>>>>> >>>>>
