Hi Harikrishna, Thank you for your prompt reply. Below are the comments/answers to your questions.
> Hi Andrei, > If the purpose of the second management server is about migration please > ignore > the previous reply. > You have the right pointer to the procedure and I hope you have followed it. > Please try to provide the following information. > 1. Is the old management server also in the 4.16.1 version? Yes, the old management server is running the same ACS version, 4.16.1. Both old and new management servers are running on Ubuntu servers. > 1. Which database.properties file you have changed to point to the new > database > ? I did not reach this phase of the migration plan as I should have checked if the new management server is working. Both the new and old management servers are connecting to the same database. > 1. Can you check the database table "configuration", what is the value for > the > configuration with the name "host", is it your new MS host address ? The value of the host in configuration is the IP address of the old management server: host The ip address of management server. This can also accept comma separated addresses. Advanced 192.168.169.13 > 1. Also, check the "mshost" table in the database if it is pointing to the > new > management server. the table mshost has both the old and the new management server entries (id 4 is the old ms and id5 is the new one). Currently the new management server is stopped, hence I guess the state is Down: | 4 | 115129173025114 | 1658099918669 | ais-cloudhost13.csprdc.arhont.com | 98405826-0861-11ea-a1da-80000003fe80 | Up | 4.16.1.0 | 127.0.0.1 | 9090 | 2022-07-27 09:55:20 | NULL | 0 | | 5 | 165004275141402 | 1658141860852 | ais-compute1.cloud.arhont.com | 0d1522a5-5d08-46af-b59c-b577aa22e9bb | Down | 4.16.1.0 | 192.168.169.21 | 9090 | 2022-07-18 11:18:51 | NULL | 1 | > Regards, > Harikrishna > From: Andrei Mikhailovsky <and...@arhont.com> > Sent: Monday, July 25, 2022 7:46 PM > To: users <users@cloudstack.apache.org> > Cc: Harikrishna Patnala <harikrishna.patn...@shapeblue.com> > Subject: Re: Unable to login to GUI onto second management server > Hi Harikrishna, > Having read the links that you've sent I am not sure that my issues are > related. > Perhaps I should have explained my current set up / intensions a bit more. My > main reasons for adding the multiple management servers is not to provide the > HA / load balancing, but rather to migrate the current management server from > old hardware to the new one. I was referring to the post sent by Andrija Panic > ( [ https://www.mail-archive.com/users@cloudstack.apache.org/msg32889.html | > https://www.mail-archive.com/users@cloudstack.apache.org/msg32889.html ] ) > where Andrija has suggested that one should install the second management > server, connect it to the database, move the database to a new server and > change the database properties to point the new management server to the new > db. > In my tests, I have installed the second management server without any > proxy/load balancing and I tried to connect and authenticate directly to the > IP > address of the second management server. I've tried it with the primary > management server switched on and off, but I still have the same issues. If I > am connecting directly to the new management server IP, I don't see how having > nginx proxy settings changes would fix my issue. Also, I have not seen > anything > in the documentation that explicitly requires having a proxy if you install > the > second management server. > Why do you think my issue relates to CORS? > Andrei > ----- Original Message ----- > > From: "Harikrishna Patnala" <harikrishna.patn...@shapeblue.com> > > To: "users" <users@cloudstack.apache.org> > > Sent: Wednesday, 20 July, 2022 05:10:13 > > Subject: Re: Unable to login to GUI onto second management server > > Hi Andrei, > > This looks to me like a CORS issue. > > Have you set up any load balancer for these management servers. There is a > > section >> [ >> http://docs.cloudstack.apache.org/en/4.16.1.0/adminguide/reliability.html#management-server-load-balancing > > | > http://docs.cloudstack.apache.org/en/4.16.1.0/adminguide/reliability.html#management-server-load-balancing > ] > > which you need to configure so that you will not face issues with HA and > > agents > > later on. > > You may need to consider setting cookies like below. > > If you are using nginx, try with "proxy_cookie_path / "/; Secure; > > SameSite=None;";" and a similar thing should work haproxy too. > > I got this reference from a previous discussion on a PR > > [ > > https://github.com/apache/cloudstack-primate/pull/898#issuecomment-760227366 > > | > https://github.com/apache/cloudstack-primate/pull/898#issuecomment-760227366 > ] , > > please refer to it if it helps solve your problem. > > Regards, > > Harikrishna > > ________________________________ > > From: Andrei Mikhailovsky <and...@arhont.com.INVALID> > > Sent: Tuesday, July 19, 2022 4:06 PM > > To: users <users@cloudstack.apache.org> > > Subject: Re: Unable to login to GUI onto second management server > > Bump please > > ----- Original Message ----- > >> From: "Andrei Mikhailovsky" <and...@arhont.com.INVALID> > >> To: "users" <users@cloudstack.apache.org> > >> Sent: Monday, 18 July, 2022 11:45:05 > >> Subject: Unable to login to GUI onto second management server > >> Hello, > >> I've recently installed a second management server ACS 4.16.1 following the > >> installation instructions in section Additional Management Servers from the > >> official documentation ( [ >>> [ >>> http://docs.cloudstack.apache.org/en/4.16.1.0/installguide/management-server/index.html > >> | > http://docs.cloudstack.apache.org/en/4.16.1.0/installguide/management-server/index.html > ] >>> [ >>> http://docs.cloudstack.apache.org/en/4.16.1.0/installguide/management-server/index.html > >> | > http://docs.cloudstack.apache.org/en/4.16.1.0/installguide/management-server/index.html > ] > >> ] ). I've installed the Ubuntu package on the second server of the same > >> version > >> as the primary management server. Configured the database with > >> cloudstack-setup-databases command followed by running > >> cloudstack-setup-management as per the documentation. There were no errors > >> in > >> the process and the cloudstack-management.service seems to have started > >> just > >> fine. The second ACS management service connected to the same database as > >> the > >> primary one and the login web GUI loaded just fine. The management server > >> logs > >> seems to show no apparent errors in the startup. The only exceptions I was > >> getting in the logs were from the host agents showing status Disconnected. > >> So, I have tried to login (using domain and ROOT login accounts) to the > >> web gui > >> of the second management server and the page just hangs after I enter the > >> credentials and press the Login button. I've tried several different > >> browsers > >> at no avail. Supplying the incorrect login credentials produce the error > >> though. The management server logs do not show any errors during the login > >> process. In fact, it seems that all commands produce " is allowed to > >> perform > >> API calls: 0.0.0.0/0,::/0 " message in the logs. There are no exceptions > >> that I > >> can see either: > >> -------------- > >> 2022-07-18 01:17:33,743 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-285:ctx-0cf08734) > >> (logid:94b277ba) ===START=== 192.168.169.251 -- POST > >> 2022-07-18 01:17:33,750 DEBUG [c.c.u.AccountManagerImpl] > >> (qtp681094281-285:ctx-0cf08734) (logid:94b277ba) Attempting to log in user: > >> andrei in domain 1 > >> 2022-07-18 01:17:33,752 DEBUG [o.a.c.s.a.PBKDF2UserAuthenticator] > >> (qtp681094281-285:ctx-0cf08734) (logid:94b277ba) Retrieving user: andrei > >> 2022-07-18 01:17:33,969 DEBUG [c.c.u.AccountManagerImpl] > >> (qtp681094281-285:ctx-0cf08734) (logid:94b277ba) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:33,969 DEBUG [c.c.u.AccountManagerImpl] > >> (qtp681094281-285:ctx-0cf08734) (logid:94b277ba) User: andrei in domain 1 > >> has > >> successfully logged in > >> 2022-07-18 01:17:34,011 INFO [c.c.a.ApiServer] > >> (qtp681094281-285:ctx-0cf08734) > >> (logid:94b277ba) Current user logged in under Etc/UTC timezone > >> 2022-07-18 01:17:34,011 INFO [c.c.a.ApiServer] > >> (qtp681094281-285:ctx-0cf08734) > >> (logid:94b277ba) Timezone offset from UTC is: 0.0 > >> 2022-07-18 01:17:34,015 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-285:ctx-0cf08734) > >> (logid:94b277ba) ===END=== 192.168.169.251 -- POST > >> 2022-07-18 01:17:34,123 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-280:ctx-fafe166c) > >> (logid:41d7b4d5) ===START=== 192.168.169.251 -- GET > >> listall=true&command=listZones&response=json > >> 2022-07-18 01:17:34,133 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-280:ctx-fafe166c > >> ctx-2269cc31) (logid:41d7b4d5) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,133 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-28:ctx-0906d03f) > >> (logid:56b10f23) ===START=== 192.168.169.251 -- GET > >> command=listApis&response=json > >> 2022-07-18 01:17:34,137 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-280:ctx-fafe166c > >> ctx-2269cc31) (logid:41d7b4d5) ===END=== 192.168.169.251 -- GET > >> listall=true&command=listZones&response=json > >> 2022-07-18 01:17:34,144 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-28:ctx-0906d03f > >> ctx-5a2a7dde) (logid:56b10f23) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,153 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-318:ctx-fc79b118) > >> (logid:8a349f6d) ===START=== 192.168.169.251 -- GET > >> command=cloudianIsEnabled&response=json > >> 2022-07-18 01:17:34,163 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-318:ctx-fc79b118 > >> ctx-40fd8f3a) (logid:8a349f6d) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,168 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-318:ctx-fc79b118 > >> ctx-40fd8f3a) (logid:8a349f6d) ===END=== 192.168.169.251 -- GET > >> command=cloudianIsEnabled&response=json > >> 2022-07-18 01:17:34,176 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-34:ctx-20a51695) > >> (logid:2436a576) ===START=== 192.168.12022-07-18 01:17:34,123 DEBUG > >> [c.c.a.ApiServlet] (qtp681094281-280:ctx-fafe166c) (logid:41d7b4d5) > >> ===START=== > >> 192.168.169.251 -- GET listall=true&command=listZones&response=json > >> 2022-07-18 01:17:34,133 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-280:ctx-fafe166c > >> ctx-2269cc31) (logid:41d7b4d5) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,133 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-28:ctx-0906d03f) > >> (logid:56b10f23) ===START=== 192.168.169.251 -- GET > >> command=listApis&response=json > >> 2022-07-18 01:17:34,137 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-280:ctx-fafe166c > >> ctx-2269cc31) (logid:41d7b4d5) ===END=== 192.168.169.251 -- GET > >> listall=true&command=listZones&response=json > >> 2022-07-18 01:17:34,144 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-28:ctx-0906d03f > >> ctx-5a2a7dde) (logid:56b10f23) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,153 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-318:ctx-fc79b118) > >> (logid:8a349f6d) ===START=== 192.168.169.251 -- GET > >> command=cloudianIsEnabled&response=json > >> 2022-07-18 01:17:34,163 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-318:ctx-fc79b118 > >> ctx-40fd8f3a) (logid:8a349f6d) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,168 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-318:ctx-fc79b118 > >> ctx-40fd8f3a) (logid:8a349f6d) ===END=== 192.168.169.251 -- GET > >> command=cloudianIsEnabled&response=json > >> 2022-07-18 01:17:34,176 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-34:ctx-20a51695) > >> (logid:2436a576) ===START=== 192.168.12022-07-18 01:17:34,123 DEBUG > >> [c.c.a.ApiServlet] (qtp681094281-280:ctx-fafe166c) (logid:41d7b4d5) > >> ===START=== > >> 192.168.169.251 -- GET listall=true&command=listZones&response=json > >> 2022-07-18 01:17:34,133 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-280:ctx-fafe166c > >> ctx-2269cc31) (logid:41d7b4d5) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,133 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-28:ctx-0906d03f) > >> (logid:56b10f23) ===START=== 192.168.169.251 -- GET > >> command=listApis&response=json > >> 2022-07-18 01:17:34,137 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-280:ctx-fafe166c > >> ctx-2269cc31) (logid:41d7b4d5) ===END=== 192.168.169.251 -- GET > >> listall=true&command=listZones&response=json > >> 2022-07-18 01:17:34,144 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-28:ctx-0906d03f > >> ctx-5a2a7dde) (logid:56b10f23) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,153 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-318:ctx-fc79b118) > >> (logid:8a349f6d) ===START=== 192.168.169.251 -- GET > >> command=cloudianIsEnabled&response=json > >> 2022-07-18 01:17:34,163 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-318:ctx-fc79b118 > >> ctx-40fd8f3a) (logid:8a349f6d) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,168 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-318:ctx-fc79b118 > >> ctx-40fd8f3a) (logid:8a349f6d) ===END=== 192.168.169.251 -- GET > >> command=cloudianIsEnabled&response=json > >> 2022-07-18 01:17:34,176 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-34:ctx-20a51695) > >> (logid:2436a576) ===START=== 192.168.169.251 -- GET > >> command=listLdapConfigurations&response=json > >> 2022-07-18 01:17:34,185 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-34:ctx-20a51695 > >> ctx-73e9ab8d) (logid:2436a576) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,188 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-34:ctx-20a51695 > >> ctx-73e9ab8d) (logid:2436a576) ===END=== 192.168.169.251 -- GET > >> command=listLdapConfigurations&response=json > >> 2022-07-18 01:17:34,196 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-343:ctx-43a80d6a) > >> (logid:8d0a86c5) ===START=== 192.168.169.251 -- GET > >> command=listCapabilities&response=json > >> 2022-07-18 01:17:34,208 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-343:ctx-43a80d6a > >> ctx-dc6fb55f) (logid:8d0a86c5) ===END=== 192.168.169.251 -- GET > >> command=listCapabilities&response=json > >> 2022-07-18 01:17:34,218 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-339:ctx-7d400edb) > >> (logid:a57fa769) ===START=== 192.168.169.251 -- GET > >> username=andrei&command=listUsers&response=json > >> 2022-07-18 01:17:34,227 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-339:ctx-7d400edb > >> ctx-2b12ac89) (logid:a57fa769) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,230 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-339:ctx-7d400edb > >> ctx-2b12ac89) (logid:a57fa769) ===END=== 192.168.169.251 -- GET > >> username=andrei&command=listUsers&response=json > >> -------------- > >> I can successfully login to the primary management server. I've done some > >> further investigation from the client browser side to see what requests are > >> being exchanged between the browser and the management server. It seems > >> that > >> the second management server gives me a bunch of 401 errors during the > >> login > >> session. There are some http 200 responses, but mainly 401For example: > >> Client Request: > >> POST /client/api/ HTTP/1.1 > >> Server Response: > >> HTTP/1.1 200 OK > >> {"loginresponse":{"username":"andrei","userid":"ee8bbe57-acce-47fa-8d9b-9e831dcf87a2","domainid":"334d7527-65f1-11e3-9bd1-d8d38559b2d0","timeout":1800,"account":"admin_group","firstname":"Andrei","lastname":"Mikhailovsky","type":"1","timezone":"Etc/UTC","timezoneoffset":"0.0","registered":"false","sessionkey":"XXXX"}} > >> ----- > >> Client Request: > >> GET /client/api/?listall=true&command=listZones&response=json HTTP/1.1 > >> Server Response: > >> HTTP/1.1 401 Unauthorized > >> {"listzonesresponse":{"uuidList":[],"errorcode":401,"cserrorcode":9999,"errortext":"The > >> given command 'listZones' either does not exist, is not available for > >> user."}} > >> ----- > >> Client Request: > >> GET /client/api/?command=listApis&response=json HTTP/1.1 > >> Server Response: > >> HTTP/1.1 200 OK > >> {"listapisresponse":{"count":96,"api":[{"name":"listResourceIcon","description":"Lists > >> the resource icon for the specified > >> resource(s)","since":"4.16.0.0","isasync":false,"related":"","params":[{"name":"resourcetype","description":"type > >> of the resource","type":"string","length":255,"required":true}, > >> (Followed by about 200K other data in the above request) > >> ----- > >> Client Requests: > >> GET /client/api/?username=andrei&command=listUsers&response=json HTTP/1.1 > >> GET /client/api/?command=listLdapConfigurations&response=json HTTP/1.1 > >> GET /client/api/?command=listCapabilities&response=json HTTP/1.1 > >> Server Response (for the above 3 requests): > >> HTTP/1.1 401 Unauthorized > >> {"listusersresponse":{"uuidList":[],"errorcode":401,"cserrorcode":9999,"errortext":"The > >> given command 'listUsers' either does not exist, is not available for > >> user."}} > >> ---------------- > >> Does anyone know what could be causing the login issues on the second > >> management > >> server? How do I solve the issue? > > > Many thanks
