Hi Harikrishna, I have added the new management server IP address into the host configuration from the gui. It now shows:
host The ip address of management server. This can also accept comma separated addresses. Advanced 192.168.169.13,192.168.169.21 After that I've started the new management server and unfortunately, I still have the same issue. I have also noticed that after starting the new management server, the table mshost has been updated to reflect the server status as Up.: | 4 | 115129173025114 | 1658099918669 | ais-cloudhost13.csprdc.arhont.com | 98405826-0861-11ea-a1da-80000003fe80 | Up | 4.16.1.0 | 127.0.0.1 | 9090 | 2022-07-27 13:10:05 | NULL | 0 | | 5 | 165004275141402 | 1658927302926 | ais-compute1.cloud.arhont.com | 0d1522a5-5d08-46af-b59c-b577aa22e9bb | Up | 4.16.1.0 | 192.168.169.21 | 9090 | 2022-07-27 13:08:32 | NULL | 0 | Anything else I should try? Thanks Andrei > From: "Harikrishna Patnala" <harikrishna.patn...@shapeblue.com> > To: "Andrei Mikhailovsky" <and...@arhont.com>, "users" > <users@cloudstack.apache.org> > Sent: Wednesday, 27 July, 2022 07:21:24 > Subject: Re: Unable to login to GUI onto second management server > Hi Andrei, > If the purpose of the second management server is about migration please > ignore > the previous reply. > You have the right pointer to the procedure and I hope you have followed it. > Please try to provide the following information. > 1. Is the old management server also in the 4.16.1 version? > 2. Which database.properties file you have changed to point to the new > database > ? > 3. Can you check the database table "configuration", what is the value for > the > configuration with the name "host", is it your new MS host address ? > 4. Also, check the "mshost" table in the database if it is pointing to the > new > management server. > Regards, > Harikrishna > From: Andrei Mikhailovsky <and...@arhont.com> > Sent: Monday, July 25, 2022 7:46 PM > To: users <users@cloudstack.apache.org> > Cc: Harikrishna Patnala <harikrishna.patn...@shapeblue.com> > Subject: Re: Unable to login to GUI onto second management server > Hi Harikrishna, > Having read the links that you've sent I am not sure that my issues are > related. > Perhaps I should have explained my current set up / intensions a bit more. My > main reasons for adding the multiple management servers is not to provide the > HA / load balancing, but rather to migrate the current management server from > old hardware to the new one. I was referring to the post sent by Andrija Panic > ( [ https://www.mail-archive.com/users@cloudstack.apache.org/msg32889.html | > https://www.mail-archive.com/users@cloudstack.apache.org/msg32889.html ] ) > where Andrija has suggested that one should install the second management > server, connect it to the database, move the database to a new server and > change the database properties to point the new management server to the new > db. > In my tests, I have installed the second management server without any > proxy/load balancing and I tried to connect and authenticate directly to the > IP > address of the second management server. I've tried it with the primary > management server switched on and off, but I still have the same issues. If I > am connecting directly to the new management server IP, I don't see how having > nginx proxy settings changes would fix my issue. Also, I have not seen > anything > in the documentation that explicitly requires having a proxy if you install > the > second management server. > Why do you think my issue relates to CORS? > Andrei > ----- Original Message ----- > > From: "Harikrishna Patnala" <harikrishna.patn...@shapeblue.com> > > To: "users" <users@cloudstack.apache.org> > > Sent: Wednesday, 20 July, 2022 05:10:13 > > Subject: Re: Unable to login to GUI onto second management server > > Hi Andrei, > > This looks to me like a CORS issue. > > Have you set up any load balancer for these management servers. There is a > > section >> [ >> http://docs.cloudstack.apache.org/en/4.16.1.0/adminguide/reliability.html#management-server-load-balancing > > | > http://docs.cloudstack.apache.org/en/4.16.1.0/adminguide/reliability.html#management-server-load-balancing > ] > > which you need to configure so that you will not face issues with HA and > > agents > > later on. > > You may need to consider setting cookies like below. > > If you are using nginx, try with "proxy_cookie_path / "/; Secure; > > SameSite=None;";" and a similar thing should work haproxy too. > > I got this reference from a previous discussion on a PR > > [ > > https://github.com/apache/cloudstack-primate/pull/898#issuecomment-760227366 > > | > https://github.com/apache/cloudstack-primate/pull/898#issuecomment-760227366 > ] , > > please refer to it if it helps solve your problem. > > Regards, > > Harikrishna > > ________________________________ > > From: Andrei Mikhailovsky <and...@arhont.com.INVALID> > > Sent: Tuesday, July 19, 2022 4:06 PM > > To: users <users@cloudstack.apache.org> > > Subject: Re: Unable to login to GUI onto second management server > > Bump please > > ----- Original Message ----- > >> From: "Andrei Mikhailovsky" <and...@arhont.com.INVALID> > >> To: "users" <users@cloudstack.apache.org> > >> Sent: Monday, 18 July, 2022 11:45:05 > >> Subject: Unable to login to GUI onto second management server > >> Hello, > >> I've recently installed a second management server ACS 4.16.1 following the > >> installation instructions in section Additional Management Servers from the > >> official documentation ( [ >>> [ >>> http://docs.cloudstack.apache.org/en/4.16.1.0/installguide/management-server/index.html > >> | > http://docs.cloudstack.apache.org/en/4.16.1.0/installguide/management-server/index.html > ] >>> [ >>> http://docs.cloudstack.apache.org/en/4.16.1.0/installguide/management-server/index.html > >> | > http://docs.cloudstack.apache.org/en/4.16.1.0/installguide/management-server/index.html > ] > >> ] ). I've installed the Ubuntu package on the second server of the same > >> version > >> as the primary management server. Configured the database with > >> cloudstack-setup-databases command followed by running > >> cloudstack-setup-management as per the documentation. There were no errors > >> in > >> the process and the cloudstack-management.service seems to have started > >> just > >> fine. The second ACS management service connected to the same database as > >> the > >> primary one and the login web GUI loaded just fine. The management server > >> logs > >> seems to show no apparent errors in the startup. The only exceptions I was > >> getting in the logs were from the host agents showing status Disconnected. > >> So, I have tried to login (using domain and ROOT login accounts) to the > >> web gui > >> of the second management server and the page just hangs after I enter the > >> credentials and press the Login button. I've tried several different > >> browsers > >> at no avail. Supplying the incorrect login credentials produce the error > >> though. The management server logs do not show any errors during the login > >> process. In fact, it seems that all commands produce " is allowed to > >> perform > >> API calls: 0.0.0.0/0,::/0 " message in the logs. There are no exceptions > >> that I > >> can see either: > >> -------------- > >> 2022-07-18 01:17:33,743 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-285:ctx-0cf08734) > >> (logid:94b277ba) ===START=== 192.168.169.251 -- POST > >> 2022-07-18 01:17:33,750 DEBUG [c.c.u.AccountManagerImpl] > >> (qtp681094281-285:ctx-0cf08734) (logid:94b277ba) Attempting to log in user: > >> andrei in domain 1 > >> 2022-07-18 01:17:33,752 DEBUG [o.a.c.s.a.PBKDF2UserAuthenticator] > >> (qtp681094281-285:ctx-0cf08734) (logid:94b277ba) Retrieving user: andrei > >> 2022-07-18 01:17:33,969 DEBUG [c.c.u.AccountManagerImpl] > >> (qtp681094281-285:ctx-0cf08734) (logid:94b277ba) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:33,969 DEBUG [c.c.u.AccountManagerImpl] > >> (qtp681094281-285:ctx-0cf08734) (logid:94b277ba) User: andrei in domain 1 > >> has > >> successfully logged in > >> 2022-07-18 01:17:34,011 INFO [c.c.a.ApiServer] > >> (qtp681094281-285:ctx-0cf08734) > >> (logid:94b277ba) Current user logged in under Etc/UTC timezone > >> 2022-07-18 01:17:34,011 INFO [c.c.a.ApiServer] > >> (qtp681094281-285:ctx-0cf08734) > >> (logid:94b277ba) Timezone offset from UTC is: 0.0 > >> 2022-07-18 01:17:34,015 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-285:ctx-0cf08734) > >> (logid:94b277ba) ===END=== 192.168.169.251 -- POST > >> 2022-07-18 01:17:34,123 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-280:ctx-fafe166c) > >> (logid:41d7b4d5) ===START=== 192.168.169.251 -- GET > >> listall=true&command=listZones&response=json > >> 2022-07-18 01:17:34,133 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-280:ctx-fafe166c > >> ctx-2269cc31) (logid:41d7b4d5) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,133 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-28:ctx-0906d03f) > >> (logid:56b10f23) ===START=== 192.168.169.251 -- GET > >> command=listApis&response=json > >> 2022-07-18 01:17:34,137 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-280:ctx-fafe166c > >> ctx-2269cc31) (logid:41d7b4d5) ===END=== 192.168.169.251 -- GET > >> listall=true&command=listZones&response=json > >> 2022-07-18 01:17:34,144 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-28:ctx-0906d03f > >> ctx-5a2a7dde) (logid:56b10f23) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,153 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-318:ctx-fc79b118) > >> (logid:8a349f6d) ===START=== 192.168.169.251 -- GET > >> command=cloudianIsEnabled&response=json > >> 2022-07-18 01:17:34,163 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-318:ctx-fc79b118 > >> ctx-40fd8f3a) (logid:8a349f6d) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,168 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-318:ctx-fc79b118 > >> ctx-40fd8f3a) (logid:8a349f6d) ===END=== 192.168.169.251 -- GET > >> command=cloudianIsEnabled&response=json > >> 2022-07-18 01:17:34,176 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-34:ctx-20a51695) > >> (logid:2436a576) ===START=== 192.168.12022-07-18 01:17:34,123 DEBUG > >> [c.c.a.ApiServlet] (qtp681094281-280:ctx-fafe166c) (logid:41d7b4d5) > >> ===START=== > >> 192.168.169.251 -- GET listall=true&command=listZones&response=json > >> 2022-07-18 01:17:34,133 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-280:ctx-fafe166c > >> ctx-2269cc31) (logid:41d7b4d5) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,133 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-28:ctx-0906d03f) > >> (logid:56b10f23) ===START=== 192.168.169.251 -- GET > >> command=listApis&response=json > >> 2022-07-18 01:17:34,137 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-280:ctx-fafe166c > >> ctx-2269cc31) (logid:41d7b4d5) ===END=== 192.168.169.251 -- GET > >> listall=true&command=listZones&response=json > >> 2022-07-18 01:17:34,144 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-28:ctx-0906d03f > >> ctx-5a2a7dde) (logid:56b10f23) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,153 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-318:ctx-fc79b118) > >> (logid:8a349f6d) ===START=== 192.168.169.251 -- GET > >> command=cloudianIsEnabled&response=json > >> 2022-07-18 01:17:34,163 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-318:ctx-fc79b118 > >> ctx-40fd8f3a) (logid:8a349f6d) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,168 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-318:ctx-fc79b118 > >> ctx-40fd8f3a) (logid:8a349f6d) ===END=== 192.168.169.251 -- GET > >> command=cloudianIsEnabled&response=json > >> 2022-07-18 01:17:34,176 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-34:ctx-20a51695) > >> (logid:2436a576) ===START=== 192.168.12022-07-18 01:17:34,123 DEBUG > >> [c.c.a.ApiServlet] (qtp681094281-280:ctx-fafe166c) (logid:41d7b4d5) > >> ===START=== > >> 192.168.169.251 -- GET listall=true&command=listZones&response=json > >> 2022-07-18 01:17:34,133 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-280:ctx-fafe166c > >> ctx-2269cc31) (logid:41d7b4d5) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,133 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-28:ctx-0906d03f) > >> (logid:56b10f23) ===START=== 192.168.169.251 -- GET > >> command=listApis&response=json > >> 2022-07-18 01:17:34,137 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-280:ctx-fafe166c > >> ctx-2269cc31) (logid:41d7b4d5) ===END=== 192.168.169.251 -- GET > >> listall=true&command=listZones&response=json > >> 2022-07-18 01:17:34,144 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-28:ctx-0906d03f > >> ctx-5a2a7dde) (logid:56b10f23) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,153 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-318:ctx-fc79b118) > >> (logid:8a349f6d) ===START=== 192.168.169.251 -- GET > >> command=cloudianIsEnabled&response=json > >> 2022-07-18 01:17:34,163 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-318:ctx-fc79b118 > >> ctx-40fd8f3a) (logid:8a349f6d) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,168 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-318:ctx-fc79b118 > >> ctx-40fd8f3a) (logid:8a349f6d) ===END=== 192.168.169.251 -- GET > >> command=cloudianIsEnabled&response=json > >> 2022-07-18 01:17:34,176 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-34:ctx-20a51695) > >> (logid:2436a576) ===START=== 192.168.169.251 -- GET > >> command=listLdapConfigurations&response=json > >> 2022-07-18 01:17:34,185 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-34:ctx-20a51695 > >> ctx-73e9ab8d) (logid:2436a576) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,188 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-34:ctx-20a51695 > >> ctx-73e9ab8d) (logid:2436a576) ===END=== 192.168.169.251 -- GET > >> command=listLdapConfigurations&response=json > >> 2022-07-18 01:17:34,196 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-343:ctx-43a80d6a) > >> (logid:8d0a86c5) ===START=== 192.168.169.251 -- GET > >> command=listCapabilities&response=json > >> 2022-07-18 01:17:34,208 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-343:ctx-43a80d6a > >> ctx-dc6fb55f) (logid:8d0a86c5) ===END=== 192.168.169.251 -- GET > >> command=listCapabilities&response=json > >> 2022-07-18 01:17:34,218 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-339:ctx-7d400edb) > >> (logid:a57fa769) ===START=== 192.168.169.251 -- GET > >> username=andrei&command=listUsers&response=json > >> 2022-07-18 01:17:34,227 DEBUG [c.c.a.ApiServer] > >> (qtp681094281-339:ctx-7d400edb > >> ctx-2b12ac89) (logid:a57fa769) CIDRs from which account > >> 'Acct[06eedc2c-65f2-11e3-9bd1-d8d38559b2d0-admin_group] -- Account {"id": > >> 2, > >> "name": "admin_group", "uuid": "06eedc2c-65f2-11e3-9bd1-d8d38559b2d0"}' is > >> allowed to perform API calls: 0.0.0.0/0,::/0 > >> 2022-07-18 01:17:34,230 DEBUG [c.c.a.ApiServlet] > >> (qtp681094281-339:ctx-7d400edb > >> ctx-2b12ac89) (logid:a57fa769) ===END=== 192.168.169.251 -- GET > >> username=andrei&command=listUsers&response=json > >> -------------- > >> I can successfully login to the primary management server. I've done some > >> further investigation from the client browser side to see what requests are > >> being exchanged between the browser and the management server. It seems > >> that > >> the second management server gives me a bunch of 401 errors during the > >> login > >> session. There are some http 200 responses, but mainly 401For example: > >> Client Request: > >> POST /client/api/ HTTP/1.1 > >> Server Response: > >> HTTP/1.1 200 OK > >> {"loginresponse":{"username":"andrei","userid":"ee8bbe57-acce-47fa-8d9b-9e831dcf87a2","domainid":"334d7527-65f1-11e3-9bd1-d8d38559b2d0","timeout":1800,"account":"admin_group","firstname":"Andrei","lastname":"Mikhailovsky","type":"1","timezone":"Etc/UTC","timezoneoffset":"0.0","registered":"false","sessionkey":"XXXX"}} > >> ----- > >> Client Request: > >> GET /client/api/?listall=true&command=listZones&response=json HTTP/1.1 > >> Server Response: > >> HTTP/1.1 401 Unauthorized > >> {"listzonesresponse":{"uuidList":[],"errorcode":401,"cserrorcode":9999,"errortext":"The > >> given command 'listZones' either does not exist, is not available for > >> user."}} > >> ----- > >> Client Request: > >> GET /client/api/?command=listApis&response=json HTTP/1.1 > >> Server Response: > >> HTTP/1.1 200 OK > >> {"listapisresponse":{"count":96,"api":[{"name":"listResourceIcon","description":"Lists > >> the resource icon for the specified > >> resource(s)","since":"4.16.0.0","isasync":false,"related":"","params":[{"name":"resourcetype","description":"type > >> of the resource","type":"string","length":255,"required":true}, > >> (Followed by about 200K other data in the above request) > >> ----- > >> Client Requests: > >> GET /client/api/?username=andrei&command=listUsers&response=json HTTP/1.1 > >> GET /client/api/?command=listLdapConfigurations&response=json HTTP/1.1 > >> GET /client/api/?command=listCapabilities&response=json HTTP/1.1 > >> Server Response (for the above 3 requests): > >> HTTP/1.1 401 Unauthorized > >> {"listusersresponse":{"uuidList":[],"errorcode":401,"cserrorcode":9999,"errortext":"The > >> given command 'listUsers' either does not exist, is not available for > >> user."}} > >> ---------------- > >> Does anyone know what could be causing the login issues on the second > >> management > >> server? How do I solve the issue? > > > Many thanks
