Hi Ahmed, Thank you for your continued use of Clearwater, we appreciate the feedback and logs you have provided here. A 500 error typically means that something has gone wrong with a node's internal processing. I've confirmed that this is the case by having a look through the Sprout logs you have attached, the first point at which we see something that looks like an error is the point at which Sprout attempts to save the authentication challenge for the subscriber in its local memory store, memcached:
03-04-2017 12:16:18.984 UTC Debug memcachedstore.cpp:144: Attempting to add data for key impi\\[email protected] 03-04-2017 12:16:18.984 UTC Debug memcachedstore.cpp:154: Attempting memcached ADD command 03-04-2017 12:16:18.985 UTC Debug memcachedstore.cpp:244: ADD/CAS returned rc = 7 (UNKNOWN READ FAILURE) (140292624120432) UNKNOWN READ FAILURE, host: 192.168.0.212:11311 -> libmemcached/response.cc:782 03-04-2017 12:16:18.985 UTC Debug memcachedstore.cpp:1093: libmemcached returned 7 03-04-2017 12:16:18.985 UTC Debug memcachedstore.cpp:1110: Blacklisting target 03-04-2017 12:16:18.985 UTC Debug baseresolver.cpp:400: Add 192.168.0.212:11311 transport 6 to blacklist for 30 seconds, graylist for 0 seconds 03-04-2017 12:16:18.985 UTC Debug connection_pool.h:267: Release connection to IP: 192.168.0.212, port: 11311 to pool 03-04-2017 12:16:18.985 UTC Debug memcachedstore.cpp:1366: Failed to write data for impi\\[email protected] to store with error UNKNOWN READ FAILURE 03-04-2017 12:16:18.985 UTC Error impistore.cpp:664: Failed to write IMPI for private_id [email protected] 03-04-2017 12:16:18.985 UTC Debug authenticationsproutlet.cpp:675: Failed to store nonce in memcached This means that Sprout would have no means by which to authenticate the subscriber and has to send out a 500 Internal Server Error. To try and determine the cause behind this issue can you answer the following questions: - Is memcached running? There are a few tools you can use to determine this: running "sudo monit summary" on your Sprout node should give the status of the memcached process, likewise " ps aux | grep memcached" will show memcached's process id if it is running. - Is the IP address that Sprout is attempting to use to communicate with memcached correct? It is shown as 192.168.0.212 in the above logs, this should be the IP address of one of your Sprout nodes. You can verify whether it is by logging on to each of your Sprout nodes, running "ifconfig" and taking note of the inet addr property of each node. Thanks, Andrew _______________________________________________ Clearwater mailing list [email protected] http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org
