I've deleted all the numbers and when login to ellis it tells me you have no numbers, create one. When trying to create I get the same error failed to update the server. And yes I have shared_config on all nodes
Thanks, Shay From: Eleanor Merry [mailto:[email protected]] Sent: Friday, October 2, 2015 6:07 PM To: Shay Naeh; [email protected] Subject: RE: Deploying Clearwater with Haet Hi, It looks like there is a corrupted number in the Ellis/Homestead databases. Can you please remove the numbers? You can do this by running: To delete all numbers on Ellis, run the following: sudo mysql USE ellis UPDATE numbers SET owner_id=NULL; To delete all numbers on Homestead, run the following: cqlsh USE homestead_provisioning; TRUNCATE public; TRUNCATE private; TRUNCATE service_profiles; TRUNCATE implicit_registration_sets; USE homestead_cache; TRUNCATE impi; TRUNCATE impu; TRUNCATE impi_mapping; To delete all numbers on Homer, run the following: cqlsh USE homer; TRUNCATE simservs; Can you also please confirm that the shared_configuration file is present on all your nodes? Ellie From: Shay Naeh [mailto:[email protected]] Sent: 02 October 2015 15:27 To: Robert Day; [email protected]<mailto:[email protected]> Cc: Eleanor Merry Subject: RE: Deploying Clearwater with Haet Hi Rob and Ellie, 1. Sudo service homer restart didn't help, but Sudo monit start homer_proccess worked just fine. 2. Regarding ellis I played with it and what helped bringing up the Login screen was after provisioning additional phone numbers (don't understand why it is related). 3. Now I encounter an issue after the login, I get the error 'failed to update the server (see detailed diagnostics in the developer console)". I think I saw an error like that that was related to the Cassandra DB tables but I suspect that some configuration regarding IPs or FQDNS is wrong. In ellis log files I find the following uncaught exception. Thanks, Shay 02-10-2015 14:15:22.080 UTC INFO web.py:1447: 200 GET /ping (0.0.0.0) 17.18ms 02-10-2015 14:16:02.117 UTC INFO web.py:1447: 200 GET /ping (0.0.0.0) 0.37ms 02-10-2015 14:16:42.169 UTC INFO web.py:1447: 200 GET /ping (0.0.0.0) 0.46ms 02-10-2015 14:17:22.213 UTC INFO web.py:1447: 200 GET /ping (0.0.0.0) 0.28ms 02-10-2015 14:17:40.600 UTC INFO web.py:1447: 200 GET /css/bootstrap-responsive.css (0.0.0.0) 2.12ms 02-10-2015 14:17:40.822 UTC INFO web.py:1447: 200 GET /css/style.css (0.0.0.0) 1.49ms 02-10-2015 14:17:40.954 UTC INFO web.py:1447: 200 GET /js/jquery.cookie.js (0.0.0.0) 0.66ms 02-10-2015 14:17:41.133 UTC INFO web.py:1447: 200 GET /js/app.js (0.0.0.0) 5.38ms 02-10-2015 14:17:43.362 UTC WARNING homestead.py:309: Passing SIP password in the clear over http 02-10-2015 14:18:03.366 UTC WARNING homestead.py:309: Passing SIP password in the clear over http 02-10-2015 14:18:23.371 UTC WARNING utils.py:78: Non-OK HTTP response. HTTP 599: Timeout 02-10-2015 14:18:23.371 UTC WARNING numbers.py:195: Failed to update all the backends 02-10-2015 14:18:23.371 UTC WARNING homestead.py:309: Passing SIP password in the clear over http 02-10-2015 14:18:23.372 UTC WARNING utils.py:78: Non-OK HTTP response. HTTP 599: Timeout 02-10-2015 14:18:23.373 UTC ERROR iostream.py:307: Uncaught exception, closing connection. Traceback (most recent call last): File "/usr/share/clearwater/ellis/env/local/lib/python2.7/site-packages/tornado-2.3-py2.7.egg/tornado/iostream.py", line 304, in wrapper callback(*args) File "/usr/share/clearwater/ellis/env/local/lib/python2.7/site-packages/tornado-2.3-py2.7.egg/tornado/httpserver.py", line 243, in _on_headers remote_ip = self.address[0] IndexError: string index out of range 02-10-2015 14:18:23.373 UTC ERROR ioloop.py:435: Exception in callback <tornado.stack_context._StackContextWrapper object at 0x7fc2624aedb8> Traceback (most recent call last): File "/usr/share/clearwater/ellis/env/local/lib/python2.7/site-packages/tornado-2.3-py2.7.egg/tornado/ioloop.py", line 421, in _run_callback callback() File "/usr/share/clearwater/ellis/env/local/lib/python2.7/site-packages/tornado-2.3-py2.7.egg/tornado/iostream.py", line 304, in wrapper callback(*args) File "/usr/share/clearwater/ellis/env/local/lib/python2.7/site-packages/tornado-2.3-py2.7.egg/tornado/httpserver.py", line 243, in _on_headers remote_ip = self.address[0] From: Robert Day [mailto:[email protected]] Sent: Thursday, October 1, 2015 10:41 PM To: Shay Naeh; [email protected]<mailto:[email protected]> Subject: RE: Deploying Clearwater with Haet Hi Shay, Good news that your Homestead node is now OK! For your Homer node, do the log files in /var/log/homer contain any errors that help explain why it isn't starting? What happens if you run "sudo service homer restart" manually - does that print any errors? On the Ellis node, the /usr/share/clearwater/infrastructure/scripts/create-ellis-nginx-config script should create the Nginx config to redirect it to Ellis. Running 'sudo service clearwater-infrastructure' should have run that script, but what happens if you run /usr/share/clearwater/infrastructure/scripts/create-ellis-nginx-config manually? Does it create a /etc/nginx/sites-available/ellis file? If not, can you send us the contents of /etc/clearwater/config, /etc/clearwater/shared_config and /etc/clearwater/local_config? Thanks, Rob -- Rob Day Software Engineer, Project Clearwater From: Clearwater [mailto:[email protected]] On Behalf Of Shay Naeh Sent: 01 October 2015 19:57 To: [email protected]<mailto:[email protected]> Subject: [Clearwater] FW: Deploying Clearwater with Haet Hi ellie, 1. I had to copy the shared_config from ellis to homestead and now it is ok 2. I had the same problem on homer but after copying the shared_config and restarting Clearwater-infrastructure it tells me when doing 'sudo monit status' thar homer_process failed 3. When I point with the browser to Ellis IP address I get the welcome message of Nginx but no login screen as I am used to get. What application should run on top of Nginx? Thanks, Shay From: Eleanor Merry [mailto:[email protected]] Sent: Thursday, October 1, 2015 7:59 PM To: Shay Naeh; [email protected]<mailto:[email protected]> Subject: RE: Deploying Clearwater with Haet Hi, Is your etcd cluster healthy? What's the result of running sudo clearwater-etcdctl cluster-health? If these look OK then can you also please try running 'sudo /usr/share/clearwater/clearwater-config-manager/scripts/upload_shared_config' on the Ellis node, then the apply_shared_config script again on Homestead. If this still doesn't work, you can copy over the shared_config file on Ellis to Homestead, run 'sudo service clearwater infrastructure' and this should unblock you. I'd be interested to dig further into why the apply_shared_config isn't working though. Ellie From: Shay Naeh [mailto:[email protected]] Sent: 01 October 2015 17:48 To: Eleanor Merry Subject: RE: Deploying Clearwater with Haet It doesn't exist on Homestead but exists on Ellis. I did clearwater-infrastructure restart but it didn't help. Can we conduct a short webex, do you want me to send you webex details? Thanks, Shay From: Eleanor Merry [mailto:[email protected]] Sent: Thursday, October 1, 2015 7:46 PM To: Shay Naeh; [email protected]<mailto:[email protected]> Subject: RE: Deploying Clearwater with Haet Does the /etc/clearwater/shared_config file exist on your Homestead node? Does it exist on the Ellis node? If the shared_config file does exist, can you run 'sudo service clearwater-infrastructure restart'? Ellie From: Shay Naeh [mailto:[email protected]] Sent: 01 October 2015 17:22 To: Eleanor Merry; [email protected]<mailto:[email protected]> Subject: RE: Deploying Clearwater with Haet Thanks Eleanor, I run the /usr/share/clearwater/clearwater-config-manager/scripts/apply_shared_config But still I get the following errors when running 'sudo monit status' Process 'homestead_process' status Does not exist monitoring status Monitored data collected Thu, 01 Oct 2015 16:13:59 Program 'poll_homestead' status Initializing monitoring status Initializing data collected Thu, 01 Oct 2015 16:10:49 Process 'homestead-prov_process' status Execution failed monitoring status Monitored data collected Thu, 01 Oct 2015 16:13:59 Shay From: Eleanor Merry [mailto:[email protected]] Sent: Thursday, October 1, 2015 12:54 PM To: Shay Naeh; [email protected]<mailto:[email protected]> Subject: RE: Deploying Clearwater with Haet Hi Shay, It doesn't look like /etc/clearwater/shared_config has successfully propagated to the Homestead node. In order to set up the diameter configuration file (used by Homestead) and the local_settings file (used by Homestead-prov) correctly, Homestead/Homestead-prov need information from the /etc/clearwater/shared_config file (e.g. the hs_hostname). If the shared_config file is missing; the install will succeed, but Homestead/Homestead-prov won't be functional until they've got their configuration files set up. Looking at the template, we install the clearwater-management package after we install the Homestead/Homestead-prov packages, but don't then do anything to apply the shared configuration to the node. I've raised an issue (https://github.com/Metaswitch/clearwater-heat/issues/15) to fix this; in the meantime can you please run "sudo /usr/share/clearwater/clearwater-config-manager/scripts/apply_shared_config" on all your nodes? Ellie From: Clearwater [mailto:[email protected]] On Behalf Of Shay Naeh Sent: 29 September 2015 16:10 To: [email protected]<mailto:[email protected]> Subject: [Clearwater] Deploying Clearwater with Haet Hello, I am trying deploying Clearwater on my OpenStack using the Heat templates as defined in here https://github.com/Metaswitch/clearwater-heat I am running into issues with Homestead where I get (see below): 1. Why LOCAL_IP=MUST BE CONFIGURED is not configured? 2. Regarding the Diameter stack I saw an open issue https://github.com/Metaswitch/homestead/issues/73 but when deleting the certificates and starting the Clearwater-infrastructure it fails again on the same problem. Thanks, Shay 4007 - Description: Fatal - Failed to initialize Diameter stack in function fd_core_parseconf with error 2. @@Cause: The Diameter interface could not be initialized or encountered an error while running. @@Effect: The application will exit and restart until the problem is fixed. @@Action: (1). Check the configuration for the Diameter destination hosts. (2). Check the connectivity to the Diameter host using Wireshark. Sep 29 06:39:55 homestead-0 monit: Traceback (most recent call last): Sep 29 06:39:55 homestead-0 monit: File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main Sep 29 06:39:55 homestead-0 monit: "__main__", fname, loader, pkg_name) Sep 29 06:39:55 homestead-0 monit: File "/usr/lib/python2.7/runpy.py", line 72, in _run_code Sep 29 06:39:55 homestead-0 monit: exec code in run_globals Sep 29 06:39:55 homestead-0 monit: File "/usr/share/clearwater/homestead/env/lib/python2.7/site-packages/crest-0.1-py2.7.egg/metaswitch/crest/main.py", line 50, in <module> Sep 29 06:39:55 homestead-0 monit: from metaswitch.crest import api Sep 29 06:39:55 homestead-0 monit: File "/usr/share/clearwater/homestead/env/local/lib/python2.7/site-packages/crest-0.1-py2.7.egg/metaswitch/crest/api/__init__.py", line 35, in <module> Sep 29 06:39:55 homestead-0 monit: from metaswitch.crest.api import base Sep 29 06:39:55 homestead-0 monit: File "/usr/share/clearwater/homestead/env/local/lib/python2.7/site-packages/crest-0.1-py2.7.egg/metaswitch/crest/api/base.py", line 48, in <module> Sep 29 06:39:55 homestead-0 monit: from metaswitch.crest import settings Sep 29 06:39:55 homestead-0 monit: File "/usr/share/clearwater/homestead/env/local/lib/python2.7/site-packages/crest-0.1-py2.7.egg/metaswitch/crest/settings.py", line 139, in <module> Sep 29 06:39:55 homestead-0 monit: execfile(_local_settings_file) Sep 29 06:39:55 homestead-0 monit: File "/usr/share/clearwater/homestead/env/local/lib/python2.7/site-packages/crest-0.1-py2.7.egg/metaswitch/crest/local_settings.py", line 55, in <module> Sep 29 06:39:55 homestead-0 monit: LOCAL_IP = MUST_BE_CONFIGURED Sep 29 06:39:55 homestead-0 monit: NameError: name 'MUST_BE_CONFIGURED' is not defined Sep 29 06:39:55 homestead-0 monit: ...fail! Sep 29 06:40:01 homestead-0 CRON[15030]: (root) CMD (/usr/lib/sysstat/sadc 1 1 /var/log/sysstat/clearwater-sa`date +%d` > /dev/null 2>&1) Sep 29 06:40:22 homestead-0 monit: CMD /etc/init.d/homestead-prov start Sep 29 06:40:26 homestead-0 monit: Traceback (most recent call last): Sep 29 06:40:26 homestead-0 monit: File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main Sep 29 06:40:26 homestead-0 monit: "__main__", fname, loader, pkg_name) Sep 29 06:40:26 homestead-0 monit: File "/usr/lib/python2.7/runpy.py", line 72, in _run_code Sep 29 06:40:26 homestead-0 monit: exec code in run_globals Sep 29 06:40:26 homestead-0 monit: File "/usr/share/clearwater/homestead/env/lib/python2.7/site-packages/crest-0.1-py2.7.egg/metaswitch/crest/main.py", line 50, in <module> Sep 29 06:40:26 homestead-0 monit: from metaswitch.crest import api Sep 29 06:40:26 homestead-0 monit: File "/usr/share/clearwater/homestead/env/local/lib/python2.7/site-packages/crest-0.1-py2.7.egg/metaswitch/crest/api/__init__.py", line 35, in <module> Sep 29 06:40:26 homestead-0 monit: from metaswitch.crest.api import base Sep 29 06:40:26 homestead-0 monit: File "/usr/share/clearwater/homestead/env/local/lib/python2.7/site-packages/crest-0.1-py2.7.egg/metaswitch/crest/api/base.py", line 48, in <module> Sep 29 06:40:26 homestead-0 monit: from metaswitch.crest import settings Sep 29 06:40:26 homestead-0 monit: File "/usr/share/clearwater/homestead/env/local/lib/python2.7/site-packages/crest-0.1-py2.7.egg/metaswitch/crest/settings.py", line 139, in <module> Sep 29 06:40:26 homestead-0 monit: execfile(_local_settings_file) Sep 29 06:40:26 homestead-0 monit: File "/usr/share/clearwater/homestead/env/local/lib/python2.7/site-packages/crest-0.1-py2.7.egg/metaswitch/crest/local_settings.py", line 55, in <module> Sep 29 06:40:26 homestead-0 monit: LOCAL_IP = MUST_BE_CONFIGURED Sep 29 06:40:26 homestead-0 monit: NameError: name 'MUST_BE_CONFIGURED' is not defined Sep 29 06:41:01 homestead-0 CRON[15050]: (root) CMD (/usr/lib/sysstat/sadc 1 1 /var/log/sysstat/clearwater-sa`date +%d` > /dev/null 2>&1) Sep 29 06:41:03 homestead-0 monit: CMD /etc/init.d/homestead-prov restart Sep 29 06:41:03 homestead-0 monit: * Restarting homestead-prov homestead-prov Sep 29 06:41:03 homestead-0 homestead[15074]: 4005 - Description: Homestead started. @@Cause: The Homestead application is starting. @@Effect: Normal. @@Action: None. Sep 29 06:41:03 homestead-0 homestead[15074]: 1001 - Description: Diameter stack is starting. @@Cause: Diameter stack is beginning initialization. @@Effect: Normal. @@Action: None. Sep 29 06:41:03 homestead-0 homestead[15074]: 1002 - Description: Diameter stack initialization completed. @@Cause: Diameter stack has completed initialization. @@Effect: Normal. @@Action: None. Sep 29 06:41:03 homestead-0 homestead[15074]: 4007 - Description: Fatal - Failed to initialize Diameter stack in function fd_core_parseconf with error 2. @@Cause: The Diameter interface could not be initialized or encountered an error while running. @@Effect: The application will exit and restart until the problem is fixed. @@Action: (1). Check the configuration for the Diameter destination hosts. (2). Check the connectivity to the Diameter host using Wireshark. Sep 29 06:41:06 homestead-0 monit: Traceback (most recent call last): Sep 29 06:41:06 homestead-0 monit: File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main Sep 29 06:41:06 homestead-0 monit: "__main__", fname, loader, pkg_name) Sep 29 06:41:06 homestead-0 monit: File "/usr/lib/python2.7/runpy.py", line 72, in _run_code Sep 29 06:41:06 homestead-0 monit: exec code in run_globals Sep 29 06:41:06 homestead-0 monit: File "/usr/share/clearwater/homestead/env/lib/python2.7/site-packages/crest-0.1-py2.7.egg/metaswitch/crest/main.py", line 50, in <module> Sep 29 06:41:06 homestead-0 monit: from metaswitch.crest import api Sep 29 06:41:06 homestead-0 monit: File "/usr/share/clearwater/homestead/env/local/lib/python2.7/site-packages/crest-0.1-py2.7.egg/metaswitch/crest/api/__init__.py", line 35, in <module> Sep 29 06:41:06 homestead-0 monit: from metaswitch.crest.api import base Sep 29 06:41:06 homestead-0 monit: File "/usr/share/clearwater/homestead/env/local/lib/python2.7/site-packages/crest-0.1-py2.7.egg/metaswitch/crest/api/base.py", line 48, in <module> Sep 29 06:41:06 homestead-0 monit: from metaswitch.crest import settings Sep 29 06:41:06 homestead-0 monit: File "/usr/share/clearwater/homestead/env/local/lib/python2.7/site-packages/crest-0.1-py2.7.egg/metaswitch/crest/settings.py", line 139, in <module>
_______________________________________________ Clearwater mailing list [email protected] http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org
