Hello Graham, Thanks for the feedback.
Given the test was successful in lite mode, did you also check that the tables in databases seshat and playground in PostgreSQL were rightly created ? As for the error you encountered, I think it's not from the PostgreSQL setup since you've encountered it before in the MariaDB one. Kindly test with all services once more and share the entire error log using pastebin. At Your Service, Isaac Kamga. On Fri, May 10, 2019 at 4:06 AM Ebenezer Graham <[email protected]> wrote: > Hi Isaac, > > Thanks for this effort. > > I have tried testing with all the services. but, I get the error below. > *complete > log here: *https://pastebin.com/013aTJ8E > > > 1. 06:21:47.050 [Thread-13] INFO o.e.j.server.handler.ContextHandler > - Stopped o.s.b.c.e.j.JettyEmbeddedWebAppContext@3e1fd62b > > {/identity/v1,file:///tmp/jetty-docbase.1273617126957352135.2021/,UNAVAILABLE} > 2. E06:21:47.790 [DefaultMessageListenerContainer-1] ERROR > o.s.j.l.DefaultMessageListenerContainer - Could not refresh JMS Connection > for destination 'portfolio-v1' - retrying using FixedBackOff{interval=5000, > currentAttempts=0, maxAttempts=unlimited}. Cause: Error while attempting to > retrieve a connection from the pool; nested exception is > javax.jms.JMSException: Could not connect to broker URL: > tcp://localhost:61616. Reason: java.net.ConnectException: Connection > refused (Connection refused) > > > > I usually get this same error even with the MariaDB setup. I know its > related to the Message broker but I don't know how to resolve it or how to > avoid it. Can anyone share their experience on this issue? > > Also, I dropped the playground and seshat keyspaces in Cassandra as well > as in Postgres. > I changed the max-connection to 120 in PostgresSQL and also restarted the > system. > > I am convinced the problem is not with PostgreSQL or Cassandra, plus, it > worked a few hours ago when I started it in lite mode. > > PS: In light of this, how soon will this be merged into the develop > branches. I would love to completely switch to PostgreSQL. Toggling > between MariaDB and PostgreSQL is not easy :) > > *At your service,* > *Graham.* > > On Thu, 9 May 2019 at 20:13, Isaac Kamga <[email protected]> wrote: > >> Hello Dev, >> >> Thanks very much for testing Graham >> >> I've raised pull requests for the Migration to PostgreSQL. Demo-server >> runs to completion successfully in both cases and databases and tables are >> rightly being created. >> I had to make some adjustments to default-setup's ledgers.csv file >> because of an error that came from the CSVParser. >> >> *Testing demo-server lite for minimal set of services* >> I have fixed the issue with connecting to the right databases. You can >> use the postgres-setup-lite.sh >> <https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh> >> script to test the minimal set of libraries and services. After the run, >> you should be able to ascertain that databases are created and tables are >> created where they have to be. >> >> +Juhan Aasaru <[email protected]> please provide feedback how this test >> goes. >> >> *Testing demo-server for all services* >> Given that the connection issues have been resolved, it's time to do more >> testing; >> @Juhan, @Graham and anyone else, in order to test the migration in it's >> entirety using postgres-setup.sh script >> <https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup.sh>, >> ensure to; >> >> 1. Install PostgreSQL >> 2. Change password for user postgres to postgres >> 3. Change max_connections to 120 in your postgresql.conf file ( usually >> in /etc/postgresql/... ) and restart the Postgresql service.Many >> connections to databases happening since all services are started. >> 4. Ensure Cassandra has no keyspaces seshat and playground >> 5. cd into a completely new directory from your usual Fineract CN >> development pwd >> 5. In integration-tests/fineract-cn-demo-server/build/libs/ and Run >> demo-server jar using *java -Ddemoserver.provision=true >> -Ddemoserver.persistent=true >> -Dcustom.cassandra.contactPoints=127.0.0.1:9042 >> <http://127.0.0.1:9042/> -Dcassandra.cluster.user=cassandra >> -Dcassandra.cluster.pwd=password -Dcustom.postgresql.host=localhost >> -Dcustom.postgresql.user=postgres -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar* >> >> Kindly provide feedback on how the run goes in the both cases. >> >> At Your Service, >> Isaac Kamga. >> >> On Fri, May 3, 2019 at 7:21 PM Ebenezer Graham <[email protected]> >> wrote: >> >>> Hello Isaac / Juhan >>> >>> I missed a number of relevant steps and this led to the initial failure. >>> >>> The test has been successful and I will list the conditions which led to >>> the successful run below. >>> >>> 1. I created a new directory and cloned the repositories using the >>> script you provided ( >>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh >>> ) >>> >>> 2. I installed PostgreSQL server and changed the postgres user's >>> password to "postgres" >>> >>> 3. I used the command below to start the demo-server. >>> >>> java -Ddemoserver.provision=true -Ddemoserver.persistent=true >>> -Ddemoserver.lite=true -Dcustom.cassandra.contactPoints=127.0.0.1:9042 >>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password >>> -Dcustom.postgresql.host=localhost -Dcustom.postgresql.user=postgres -jar >>> demo-server-0.1.0-BUILD-SNAPSHOT.jar >>> >>> @Juhan, please try this approach and let's know the outcome. >>> >>> You should get an output similar to this. >>> >>> 18:11:28.717 [Thread-16] INFO o.e.jetty.server.AbstractConnector - >>> Stopped ServerConnector@59a94d0f{HTTP/1.1,[http/1.1]}{0.0.0.0:2020} >>> 18:11:28.720 [Thread-16] INFO o.e.j.server.handler.ContextHandler - >>> Stopped o.s.b.c.e.j.JettyEmbeddedWebAppContext@1213ffbc >>> {/provisioner/v1,file:///tmp/jetty-docbase.8613597725021372791.2020/,UNAVAILABLE} >>> identity-v1 address:http://localhost:2021/identity/v1 >>> office-v1 address:http://localhost:2023/office/v1 >>> customer-v1 address:http://localhost:2024/customer/v1 >>> >>> At your service >>> >>> >>> On Fri, 3 May 2019 at 21:09, Ebenezer Graham <[email protected]> >>> wrote: >>> >>>> Hello Isaac/Juhan, >>>> >>>> I have tried testing the code in lite mode for 3 times and I got the error >>>> below. >>>> https://pastebin.com/TaJqSNjQ >>>> >>>> Before the failure, I got this info. >>>> >>>> 16:56:03.519 [qtp1262607708-22] INFO >>>> o.a.f.c.l.c.ServiceExceptionFilter - Responding with a service error >>>> ServiceError{code=409, message='The given identity instance didnt recognize >>>> the system token as valid. Perhaps the system keys for the provisioner or >>>> for the identity manager are misconfigured?'} >>>> 16:57:03.620 [qtp1262607708-20] INFO >>>> o.a.f.c.l.c.ServiceExceptionFilter - Responding with a service error >>>> ServiceError{code=409, message='The given identity instance didnt recognize >>>> the system token as valid. Perhaps the system keys for the provisioner or >>>> for the identity manager are misconfigured?'} >>>> >>>> I will keep testing to get more insights into the error. >>>> >>>> *At your service,* >>>> >>>> *Ebenezer Graham* >>>> >>>> *BSc (Hons) Computing* >>>> >>>> >>>> [image: EmailSignature.png] >>>> >>>> African Leadership University, >>>> >>>> Power Mill Road, Pamplemousses, >>>> >>>> Mauritius. >>>> GitHub <https://github.com/ebenezergraham> | LinkedIn >>>> <https://www.linkedin.com/in/ebenezer-graham/> | ebenezergraham.me >>>> *Skype*: >>>> ebenezer.graham <http://ebenezergraham.me/> >>>> >>>> *This email and any files transmitted with it are confidential and >>>> intended solely for the use of the individual or entity to whom they are >>>> addressed. If you have received this email in error please notify the >>>> sender and then delete this email. If you are not the intended recipient >>>> you are notified that disclosing, copying, distributing or taking any >>>> action in reliance on the contents of this information is strictly >>>> prohibited.* >>>> >>>> >>>> On Thu, 2 May 2019 at 06:38, Isaac Kamga <[email protected]> wrote: >>>> >>>>> Hello Juhan, >>>>> >>>>> I've updated the PRs with commits to fix the errors you were >>>>> experiencing. >>>>> >>>>> Could you kindly wipe Cassandra and PostgreSQL clean, test once more >>>>> and provide feedback on how it goes ? >>>>> >>>>> Cheers, >>>>> Isaac Kamga. >>>>> >>>>> On Tue, Apr 9, 2019 at 4:36 PM Isaac Kamga <[email protected]> >>>>> wrote: >>>>> >>>>>> Hey Juhan, >>>>>> >>>>>> Thanks for testing. >>>>>> >>>>>> Wipe clean your Cassandra keyspaces first. >>>>>> >>>>>> I suspect that what's interferring is the keyspaces from your last >>>>>> run when using MariaDB. >>>>>> >>>>>> Please provide feedback how it goes. >>>>>> >>>>>> Cheers, >>>>>> Isaac Kamga. >>>>>> >>>>>> On Sun, Mar 31, 2019 at 10:00 AM Juhan Aasaru <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> I Isaac, >>>>>>> >>>>>>> I pulled the changes and rebuilt the projects. >>>>>>> >>>>>>> Now when I run (postgres and cassandra working locally): >>>>>>> >>>>>>> java -Ddemoserver.persistent=true -Ddemoserver.provision=true >>>>>>> -Dcustom.cassandra.contactPoints=127.0.0.1:9042 >>>>>>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password >>>>>>> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar >>>>>>> >>>>>>> I get an error here: >>>>>>> >>>>>>> 11:18:42.066 [main] INFO o.s.b.c.e.j.JettyEmbeddedServletContainer >>>>>>> - Jetty >>>>>>> started on port(s) 2024 (http/1.1) >>>>>>> 11:18:42.067 [main] INFO >>>>>>> o.s.c.n.e.EurekaDiscoveryClientConfiguration - >>>>>>> Updating port to 2024 >>>>>>> 11:18:42.072 [main] INFO o.a.f.c.customer.CustomerApplication - >>>>>>> Started >>>>>>> CustomerApplication in 16.423 seconds (JVM running for 17.342) >>>>>>> 11:18:57.435 [qtp1479696465-21] INFO >>>>>>> o.a.f.c.l.c.ServiceExceptionFilter - >>>>>>> Responding with a service error ServiceError{code=409, >>>>>>> message='Application >>>>>>> identity-v1 already exists!'} >>>>>>> 11:18:57.464 [Thread-16] INFO >>>>>>> o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext - Closing >>>>>>> >>>>>>> org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@4d50efb8 >>>>>>> : >>>>>>> startup date [Sun Mar 31 11:12:52 EEST 2019]; parent: >>>>>>> >>>>>>> org.springframework.context.annotation.AnnotationConfigApplicationContext@75a1cd57 >>>>>>> >>>>>>> I understand I have to use -Ddemoserver.provision=true flag to >>>>>>> provision >>>>>>> the initial database so I won't be able to take that one out. >>>>>>> >>>>>>> Juhan >>>>>>> >>>>>>> >>>>>>> Kontakt Isaac Kamga (<[email protected]>) kirjutas kuupäeval R, >>>>>>> 29. >>>>>>> märts 2019 kell 04:17: >>>>>>> >>>>>>> > Hey Juhan, >>>>>>> > >>>>>>> > I have updated the Pull request to fineract-cn-provisioner with a >>>>>>> commit >>>>>>> > [1] which fixes the errors you're experiencing. >>>>>>> > >>>>>>> > I'll advise that you test using option >>>>>>> -Ddemoserver.persistent=true and >>>>>>> > -Ddemoserver.lite=true and we're testing just the minimal set of >>>>>>> services >>>>>>> > now. In fact,I tested and the tail of my logs showed thus >>>>>>> > *identity-v1 address:http://localhost:2021/identity/v1 >>>>>>> > < >>>>>>> > >>>>>>> https://www.google.com/url?q=http://localhost:2021/identity/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNEjCID96mw3LF1ZIPmjEIQkHh_LcQ >>>>>>> > >* >>>>>>> > >>>>>>> > *office-v1 address:http://localhost:2023/office/v1 >>>>>>> > < >>>>>>> > >>>>>>> https://www.google.com/url?q=http://localhost:2023/office/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNGjyk1BHroLddOTd0FhJUJG4zdVag >>>>>>> > >customer-v1 >>>>>>> > address:http://localhost:2024/customer/v1 >>>>>>> > < >>>>>>> > >>>>>>> https://www.google.com/url?q=http://localhost:2024/customer/v1&sa=D&source=hangouts&ust=1553910362599000&usg=AFQjCNGfcoJZIxmcsX34YeEXRUzS0oC2hA >>>>>>> > >* >>>>>>> > Albeit, this error [2] turned up related to the Notifications >>>>>>> service. Just >>>>>>> > ignore it for now as it will be handled down the line during >>>>>>> extensive >>>>>>> > integration testing. >>>>>>> > >>>>>>> > We can keep testing using -Ddemoserver.provision=true for later >>>>>>> too when >>>>>>> > we've migrated and tested the individual services. >>>>>>> > >>>>>>> > Can you kindly update your provisioner repository and test once >>>>>>> more and >>>>>>> > let's keep fixing ? >>>>>>> > >>>>>>> > [1] >>>>>>> > >>>>>>> > >>>>>>> https://github.com/Izakey/fineract-cn-provisioner/commit/b83331b723f175003ccf069764414e4ab371662c >>>>>>> > >>>>>>> > [2] https://pastebin.com/DBzNyHir >>>>>>> > >>>>>>> > Cheers, >>>>>>> > Isaac Kamga. >>>>>>> > >>>>>>> > On Wed, Mar 27, 2019 at 10:23 PM Isaac Kamga < >>>>>>> [email protected]> >>>>>>> > wrote: >>>>>>> > >>>>>>> > > Hello Juhan, >>>>>>> > > >>>>>>> > > Thank you for taking the time to do some testing. >>>>>>> > > >>>>>>> > > On Wed, Mar 27, 2019 at 10:56 AM Juhan Aasaru <[email protected]> >>>>>>> wrote: >>>>>>> > > >>>>>>> > >> Hi Isaac, >>>>>>> > >> >>>>>>> > >> great initiative towards pushing to PostgreSQL. >>>>>>> > >> >>>>>>> > >> I looked at your instructions and tried to run your changes >>>>>>> locally. >>>>>>> > >> The sh script came in handy to quickly clone and build >>>>>>> everything that >>>>>>> > was >>>>>>> > >> needed. >>>>>>> > >> But then I got confused about provisioning. >>>>>>> > >> >>>>>>> > >> Question #1: How exactly I should run demo-server? I tried the >>>>>>> following >>>>>>> > >> options: >>>>>>> > >> >>>>>>> > >> 1. I didn't have postgre and cassandra running locally. >>>>>>> > >> java -jar -Ddemoserver.lite=true >>>>>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar >>>>>>> > >> >>>>>>> > >> resulted >>>>>>> > >> >>>>>>> > >> org.springframework.beans.factory.BeanCreationException: Error >>>>>>> creating >>>>>>> > >> bean with name 'provisionerInitializer': >>>>>>> > >> Invocation of init method failed; nested exception is >>>>>>> > >> java.lang.IllegalStateException: Could not initialize service! >>>>>>> > >> Caused by: org.postgresql.util.PSQLException: FATAL: database >>>>>>> "seshat" >>>>>>> > >> does >>>>>>> > >> not exist >>>>>>> > >> >>>>>>> > > >>>>>>> > > I haven't done testing using this option yet. >>>>>>> > > >>>>>>> > > >>>>>>> > >> >>>>>>> > >> 2. I started posgresql (and created a database seshat) and >>>>>>> cassandra and >>>>>>> > >> ran: >>>>>>> > >> java -Ddemoserver.provision=true -Ddemoserver.persistent=true >>>>>>> > >> -Dcustom.cassandra.contactPoints=127.0.0.1:9042 \ >>>>>>> > >> -Dcassandra.cluster.user=cassandra >>>>>>> -Dcassandra.cluster.pwd=password >>>>>>> > >> -Ddemoserver.lite=true -jar >>>>>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar >>>>>>> > >> >>>>>>> > >> Initially applications started up one after anther >>>>>>> > >> (ProvisionerApplication, >>>>>>> > >> RhythmApplication, OfficeApplication) >>>>>>> > >> but CustomerApplication threw an exception: >>>>>>> > >> >>>>>>> > >> 11:50:39.377 [qtp1424482154-21] WARN >>>>>>> o.e.jetty.servlet.ServletHandler - >>>>>>> > >> org.springframework.web.util.NestedServletException: Request >>>>>>> processing >>>>>>> > >> failed; nested exception is java.lang.IllegalStateException: >>>>>>> Could not >>>>>>> > >> insert org.apache.fineract.cn.provisioner.tenant info! >>>>>>> > >> Caused by: org.postgresql.util.PSQLException: ERROR: relation >>>>>>> > >> "seshat.tenants" does not exist >>>>>>> > >> >>>>>>> > > >>>>>>> > > Try running the same command above without the provision=true >>>>>>> option and >>>>>>> > > share what you get. >>>>>>> > > >>>>>>> > > I had to create database seshat in postgresql and the can't find >>>>>>> database >>>>>>> > > error went away. Down the road, also encountered the same error >>>>>>> > > <https://pastebin.com/qCT1T5KM> "Could not insert .... >>>>>>> > provisioner.tenant >>>>>>> > > info !" >>>>>>> > > I'm currently working on resolving this. >>>>>>> > > >>>>>>> > > >>>>>>> > >> >>>>>>> > >> Question #2 - do I still need to have Mysql running? I see that >>>>>>> you >>>>>>> > >> haven't >>>>>>> > >> made code changes to identity >>>>>>> > >> which as I understand still uses Mysql. >>>>>>> > >> >>>>>>> > > >>>>>>> > > I don't think you need MySQL running to test the migrations >>>>>>> currently >>>>>>> > > underway and there was no need for code changes in the identity >>>>>>> service. >>>>>>> > > However, it could be helpful to see the structure of the >>>>>>> databases in >>>>>>> > MySQL >>>>>>> > > so that it helps with debugging and testing. >>>>>>> > > >>>>>>> > > Whenever you're available, tell me so we schedule a call so I >>>>>>> look into >>>>>>> > > what your experiencing too. >>>>>>> > > >>>>>>> > > Cheers, >>>>>>> > > Isaac Kamga. >>>>>>> > > >>>>>>> > > >>>>>>> > >> >>>>>>> > >> Kind regards >>>>>>> > >> Juhan >>>>>>> > >> >>>>>>> > >> >>>>>>> > >> Kontakt Isaac Kamga (<[email protected]>) kirjutas >>>>>>> kuupäeval L, 23. >>>>>>> > >> märts 2019 kell 04:31: >>>>>>> > >> >>>>>>> > >> > Hello fineracters, >>>>>>> > >> > >>>>>>> > >> > Trust that you had a great week. >>>>>>> > >> > >>>>>>> > >> > I just finished migrating libraries PostgreSQL[1] and Test[2] >>>>>>> from >>>>>>> > >> MariaDB >>>>>>> > >> > to PostgreSQL, mostly adapting SQL statements & database >>>>>>> connections >>>>>>> > to >>>>>>> > >> > work with PostgreSQL and reverting back to Hibernate ORM until >>>>>>> > database >>>>>>> > >> > migrations are completed. >>>>>>> > >> > >>>>>>> > >> > I have also migrated the "minimal" services viz >>>>>>> provisioner[3], >>>>>>> > >> rhythm[4], >>>>>>> > >> > office[5], template[6] and customer[7] from MariaDB to >>>>>>> PostgreSQL. The >>>>>>> > >> > Spring datasource credentials had to be incorporated into >>>>>>> > >> application.yaml >>>>>>> > >> > files to make things work too. >>>>>>> > >> > >>>>>>> > >> > Given I updated service-starter [8] and demo-server [9], >>>>>>> developers >>>>>>> > can >>>>>>> > >> > test the migrated services when running the >>>>>>> *-Ddemoserver.lite=true* >>>>>>> > >> option >>>>>>> > >> > which builds the minimal set of services. >>>>>>> > >> > >>>>>>> > >> > You'll observe that some of the pull requests are heading to >>>>>>> develop >>>>>>> > >> > branches ( It appears comparing changes and creating PRs >>>>>>> towards the >>>>>>> > >> same >>>>>>> > >> > branch name from origin to upstream expires after a few >>>>>>> minutes on >>>>>>> > >> > Github... weird ) >>>>>>> > >> > >>>>>>> > >> > Here is a script [10] to enable anyone clone and do an >>>>>>> > integration-test >>>>>>> > >> of >>>>>>> > >> > the new changes using demo-server. >>>>>>> > >> > >>>>>>> > >> > Kindly test these changes and provide feedback. >>>>>>> > >> > >>>>>>> > >> > At Your Service, >>>>>>> > >> > Isaac Kamga. >>>>>>> > >> > >>>>>>> > >> > [1] https://github.com/apache/fineract-cn-postgresql/pull/1 >>>>>>> > >> > [2] https://github.com/apache/fineract-cn-test/pull/7 >>>>>>> > >> > [3] https://github.com/apache/fineract-cn-provisioner/pull/6 >>>>>>> > >> > [4] https://github.com/apache/fineract-cn-rhythm/pull/8 >>>>>>> > >> > [5] https://github.com/apache/fineract-cn-office/pull/9 >>>>>>> > >> > [6] https://github.com/apache/fineract-cn-template/pull/6 >>>>>>> > >> > [7] https://github.com/apache/fineract-cn-customer/pull/10 >>>>>>> > >> > [8] >>>>>>> https://github.com/apache/fineract-cn-service-starter/pull/8 >>>>>>> > >> > [9] https://github.com/apache/fineract-cn-demo-server/pull/27 >>>>>>> > >> > [10] >>>>>>> > >> > >>>>>>> > >> > >>>>>>> > >> >>>>>>> > >>>>>>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh >>>>>>> > >> > >>>>>>> > >> >>>>>>> > > >>>>>>> > >>>>>>> >>>>>>
