[
https://issues.apache.org/jira/browse/CASSANDRA-16053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179209#comment-17179209
]
Adam Holmberg commented on CASSANDRA-16053:
-------------------------------------------
CASSANDRA-11749 introduced passing the cluster to clean up sockets in forked
child processes. That patch assumed that forking would be used everywhere but
Windows.
Instead of making the parameter passing logic more complex to account for
platform and runtime variations, I suggest we rely on the underlying driver fix
which was later introduced, and stop trying to pass the Cluster.
[PYTHON-577|https://datastax-oss.atlassian.net/browse/PYTHON-577]
[patch|https://github.com/apache/cassandra/compare/trunk...aholmberg:CASSANDRA-16053?expand=1]
I've run the repro script from CASSANDRA-11749 several hundred times. Stefania
indicated that it would previously reproduce within around a half dozen
iterations.
> cqlsh COPY functions broken in Python 3.8 on Mac
> ------------------------------------------------
>
> Key: CASSANDRA-16053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16053
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
> Reporter: Adam Holmberg
> Assignee: Adam Holmberg
> Priority: Normal
> Fix For: 4.0-beta2
>
>
> {noformat}
> aholmberg-rmbp16:cassandra adamholmberg$ pyenv shell
> 3.8.2
> aholmberg-rmbp16:cassandra adamholmberg$ bin/cqlsh --debug
> Using CQL driver: <module 'cassandra' from
> '/Users/adamholmberg/code/cassandra/bin/../lib/cassandra-driver-internal-only-3.23.0.post0-1a184b99.zip/cassandra-driver-3.23.0.post0-1a184b99/cassandra/__init__.py'>
> Using connect timeout: 5 seconds
> Using 'utf-8' encoding
> Using ssl: False
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 4.0-beta2-SNAPSHOT | CQL spec 3.4.5 | Native
> protocol v4]
> Use HELP for help.
> cqlsh> copy test.testcopyto to 'asdf';
> Detected 12 core(s)
> Using 11 child processes
> Starting copy of test.testcopyto with columns [a, b, c, d].
> Traceback (most recent call last):
> File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 937, in onecmd
> self.handle_statement(st, statementtext)
> File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 974, in
> handle_statement
> return custom_handler(parsed)
> File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 1563, in
> do_copy
> task.run()
> File
> "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line
> 669, in run
> self.start_processes()
> File
> "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line
> 471, in start_processes
> process.start()
> File
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/process.py",
> line 121, in start
> self._popen = self._Popen(self)
> File
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py",
> line 224, in _Popen
> return _default_context.get_context().Process._Popen(process_obj)
> File
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py",
> line 283, in _Popen
> return Popen(process_obj)
> File
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py",
> line 32, in __init__
> super().__init__(process_obj)
> File
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_fork.py",
> line 19, in __init__
> self._launch(process_obj)
> File
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py",
> line 47, in _launch
> reduction.dump(process_obj, fp)
> File
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/reduction.py",
> line 60, in dump
> ForkingPickler(file, protocol).dump(obj)
> TypeError: cannot pickle '_thread.lock' object
> cqlsh>
> {noformat}
> multiprocessing uses a different default start method on Mac, and pickling
> fails trying to serialize the Cluster object.
> https://github.com/python/cpython/blob/db098bc1f05bd0773943e59f83489f05f28dedf8/Lib/multiprocessing/context.py#L313-L318
> https://bugs.python.org/issue33725
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
