Hi Alioune, When I asked for the local_config and shared_config contained on your Bono node I was asking for the contents of the files /etc/clearwater/local_config and /etc/clearwater/shared_config. The hostnames contained in these files will be resolved via DNS lookups and will be used for communicating information between processes that may be on different nodes. This is what led me to suspect that “icscf.” may appear in one of your config files based on the Bono logs you sent me.
You’ve included the contents of the /usr/share/clearwater/infrastructure/migration-utils/migrate_local_config and /usr/share/clearwater/infrastructure/migration-utils/migrate_shared_config in your latest email. These are scripts which can be used to sort the contents of the old style of config file we used: /etc/clearwater/config into the local_config and shared_config files. These scripts themselves do not contain any of your config. Could you please send me the contents of /etc/clearwater/local_config and /etc/clearwater/shared_config. Could you please also in future respond to these emails by replying to the [email protected]<mailto:[email protected]> address. This way our messages are saved to the mailing list archive so that if future Project Clearwater users hit a similar problem they will have a record of our conversation which could help them resolve their issue. Thanks, Andrew From: Alioune BA [mailto:[email protected]] Sent: Tuesday, June 27, 2017 3:13 PM To: Andrew Edmonds <[email protected]> Subject: Re: [Project Clearwater] Fail to test Clearwater with clearwater-live-test Hi Andrew, Yo can find the logs in bono server. In fact there icscf is present in at least one config file. Regards [1] https://docs.google.com/document/d/11dSvgxQFzSa0Ay3P5S3EBirR0uynlqs846qDzfiJ5HI/edit?usp=sharing [2] https://docs.google.com/document/d/1hT4-WJUrBaZef86Ntq_QMQ_OyPJxoJ0k6x8jcCRvhOs/edit?usp=sharing [3] https://docs.google.com/document/d/1HWM3HXX_WiDEpFDd3O8lPEIB6JBNyu7nWlmMLvQfz0Y/edit?usp=sharing 2017-06-27 15:56 GMT+02:00 Andrew Edmonds <[email protected]<mailto:[email protected]>>: Hi Alioune, Thank you for the extra diagnostics you have provided. In an earlier email you commented that it looked like the clearwater-live-tests scripts try to provision a subscriber account in Ellis, this is correct and we can see from your logs that it is failing to do so. The logs you have provided below do indicate an issue: that we are trying to resolve “icscf.” to an IP address. In may be that you have incorrectly configured a hostname in either the shared_config or local_config on your Bono node, could you please provide me with these files? Thanks, Andrew From: Alioune BA [mailto:[email protected]<mailto:[email protected]>] Sent: Thursday, June 22, 2017 1:07 PM To: Andrew Edmonds <[email protected]<mailto:[email protected]>> Subject: Re: [Project Clearwater] Fail to test Clearwater with clearwater-live-test I forget to say also I have this log error in bono node: 22-06-2017 12:00:00.335 UTC Error sip_connection_pool.cpp:189: Failed to resolve icscf. to an IP address - Not found (PJ_ENOTFOUND) 22-06-2017 12:00:00.335 UTC Error sip_connection_pool.cpp:189: Failed to resolve icscf. to an IP address - Not found (PJ_ENOTFOUND) 2017-06-22 13:54 GMT+02:00 Alioune BA <[email protected]<mailto:[email protected]>>: Hi Andrew, Did you receive my previous email ? I sent to you the trace and I would like to know how to register new user with clearwater-live-test. It seems that this could be the root the errors . Best Regards 2017-06-21 15:24 GMT+02:00 Alioune BA <[email protected]<mailto:[email protected]>>: Hi Andrew, Thanks for your reply. I redeployed clearwater with your previous suggestions and now there is no logs in /var/log/ellis. I got nginx error [1], it seems the clearwater-live test wants to create an account when I run "rake test[example.com<http://example.com>] SIGNUP_CODE=secret PROXY=192.168.74.6 ELLIS=192.168.74.7" 2017/06/21 13:05:30 [error] 9601#0: *1010 open() "/usr/share/nginx/html/session" failed (2: No such file or directory), client: ::ffff:192.168.74.13, server: ping, request: "POST /session HTTP/1.1", host: "192.168.74.7" Trace tcpdump [2] 192.168.74.6 = bono 192.168.74.7 = ellis 192.168.74.13 = clearwater-live-test server Best Regards, [1] https://docs.google.com/document/d/1AWUEwmbR83O56VRQ-gekfFRu3HdO-c0e5XSdN3pOUQM/edit?usp=sharing [2] https://drive.google.com/file/d/0B4ORJR1_yZ79OThRZzJkLUtQODA/view?usp=sharing 2017-06-21 11:57 GMT+02:00 Andrew Edmonds <[email protected]<mailto:[email protected]>>: Hi Alioune Thank you for the latest diagnostics. Looking through both logs it does appear that both the nginx and ellis services are starting correctly. However in the ellis there is still some suggestion of network communication issues: 14-06-2017 13:29:22.105 UTC ERROR homestead.py:68: Failed to ping Homestead at http://hs.example.com:8889/ping. Have you configured your HOMESTEAD_URL? Can you take a traffic capture on the ellis node (i.e. by using tcpdump) to see what is happening with the network traffic? It would be useful if you could restart the ellis process as you are capturing the network traffic. Thanks, Andrew From: Alioune BA [mailto:[email protected]<mailto:[email protected]>] Sent: Wednesday, June 14, 2017 2:41 PM To: [email protected]<mailto:[email protected]> Cc: Andrew Edmonds <[email protected]<mailto:[email protected]>> Subject: Re: [Project Clearwater] Fail to test Clearwater with clearwater-live-test Andrew, Thanks for your reply. I observed that nginx and ellis services was running correctly in ellis node. You can find the logs of ellis [1] and nginx [2] after restarting services but I wonder is it required to register users in homestead before running the "rake test" command ? Best Regards, [1] https://docs.google.com/document/d/1M_7jwnTwYF2pXVZtBp5VGLgAKZC2_rWW7B9uoUijAOA/edit?usp=sharing [2] https://docs.google.com/document/d/1H953N_g93uyVTXa7ldeMw1jT6XcvjeVFVmGvhyiId54/edit?usp=sharing sudo monit summary Monit 5.18.1 uptime: 8d 23h 4m Service Name Status Type node-ellis-0.example.com<http://node-ellis-0.example.com> Running System ntp_process Running Process nginx_process Running Process mysql_process Running Process ellis_process Running Process clearwater_queue_manager_pro... Running Process etcd_process Running Process clearwater_diags_monitor_pro... Running Process clearwater_config_manager_pr... Running Process nginx_ping Status ok Program nginx_uptime Status ok Program monit_uptime Status ok Program poll_ellis Status ok Program poll_ellis_https Status ok Program clearwater_queue_manager_uptime Status ok Program etcd_uptime Status ok Program poll_etcd_cluster Status failed Program poll_etcd Status ok Program 2017-06-14 13:45 GMT+02:00 Andrew Edmonds <[email protected]<mailto:[email protected]>>: Hi Alioune, Thank you for your question and for the detailed logs you have provided. When HTTP requests are sent to Ellis in order to provision subscribers they pass through nginx. nginx is a generic HTTP web server that should forward these HTTP requests on to Ellis. However if we look at the error logs we can see: - Account creation failed with HTTP code 502, body <html> <head><title>502 Bad Gateway</title></head> <body bgcolor="white"> <center><h1>502 Bad Gateway</h1></center> <hr><center>nginx/1.4.6 (Ubuntu)</center> </body> </html> This shows that the error is coming from nginx and it means that nginx can’t communicate with Ellis. This could be caused by one of the following: • The Ellis process is not running • Some state has got stuck that means that nginx and Ellis can’t communicate To investigate this access your Ellis VM’s console and run “sudo monit summary”, this will show you what processes are running . To restart nginx and the Ellis process run the commands “sudo service nginx restart” and “sudo service ellis restart”. If this still does not work you can check the most recent log in /var/log/ellis and the /var/log/nginx/error.log. Thanks, Andrew From: Clearwater [mailto:[email protected]<mailto:[email protected]>] On Behalf Of Alioune BA Sent: Tuesday, June 13, 2017 3:59 PM To: [email protected]<mailto:[email protected]> Subject: [Project Clearwater] Fail to test Clearwater with clearwater-live-test Hi all, I'm trying to setup Clearwater IMS core with the release-85 of clearwater-heat. Clearwater node are running and now I would like to test the IMS core using clearwater-live-test with commands [1] and [2] But according to the logs in [3] all test fails with messages "502 Bad Gateway" and "Account creation failed with HTTP code 502" Please do you have any suggestion to solve that issue ? I follow [4] to setup clearwater-live-test 192.168.74.8 = bono 192.168.74.4 = ellis The server from which i run [1] and [2] can ping all nodes of Clearwater [1] rake test[example.com<http://example.com>] SIGNUP_CODE=secret PROXY=192.168.74.8 ELLIS=192.168.74.14 [2] rake test[example.com<http://example.com>] SIGNUP_CODE=secret ELLIS=192.168.74.14 [3] https://docs.google.com/document/d/1rhdojSbDP58DgClFaAjyF8Tsy5urzO_8dRe8TlAeVUQ/edit?usp=sharing [4] http://clearwater.readthedocs.io/en/stable/Running_the_live_tests.html _______________________________________________ Clearwater mailing list [email protected]<mailto:[email protected]> http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org
_______________________________________________ Clearwater mailing list [email protected] http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org
