Patrick Bannister created CASSANDRA-14320:
---------------------------------------------
Summary: dtest tools/jmxutils.py JolokiaAgent raises TypeError
using json.loads on bytes
Key: CASSANDRA-14320
URL: https://issues.apache.org/jira/browse/CASSANDRA-14320
Project: Cassandra
Issue Type: Bug
Components: Testing
Reporter: Patrick Bannister
Fix For: 3.0.x, 3.11.x
JolokiaAgent in tools/jmxutils.py raises a TypeError when used, because its
_query function tries to use json.loads (which only accepts string input) on a
bytes object.
{code:java}
def _query(self, body, verbose=True):
request_data = json.dumps(body).encode("utf-8")
url = 'http://%s:8778/jolokia/' %
(self.node.network_interfaces['binary'][0],)
req = urllib.request.Request(url)
response = urllib.request.urlopen(req, data=request_data, timeout=10.0)
if response.code != 200:
raise Exception("Failed to query Jolokia agent; HTTP response code:
%d; response: %s" % (response.code, response.readlines()))
raw_response = response.readline() # response is
http.client.HTTPResponse, which subclasses RawIOBase, which returns bytes when
read
response = json.loads(raw_response) # this raises a TypeError now
if response['status'] != 200:
stacktrace = response.get('stacktrace')
if stacktrace and verbose:
print("Stacktrace from Jolokia error follows:")
for line in stacktrace.splitlines():
print(line)
raise Exception("Jolokia agent returned non-200 status: %s" %
(response,))
return response{code}
This can be seen clearly by running the deprecated repair tests
(repair_tests/deprecated_repair_test.py). They all fail right now because of
this TypeError.
This is a side effect of the migration to Python 3, which makes bytes objects
fundamentally different from strings. This will also happen anytime we try to
json.loads data returned from stdout or stderr piped from subprocess. I need to
take a closer look at offline_tools_test.py and
cqlsh_tests/cqlsh_copy_tests.py, because I suspect they're impacted as well.
We can fix this issue by decoding bytes objects to strings before calling
json.loads(). For example, in the above:
{code:java}
response = json.loads(raw_response.decode(encoding='utf-8')){code}
I have a fix for the JolokiaAgent problem - I'll submit a pull request to
cassandra-dtest once I have this issue number to reference.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
