[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17037196#comment-17037196 ] Dinesh Joshi commented on CASSANDRA-10190: -- Thank you everyone for your help! This has been committed to [cassandra|https://github.com/apache/cassandra/commit/bf9a1d487b9ba469e8d740cf7d1cd419535a7e79] and [cassandra-dtest|https://github.com/apache/cassandra-dtest/commit/175a083a6f3b4d5d58f3702d31ce6920af519669]. The one dtest failure seems to be flakey. All other dtests and cqlsh tests are green. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh, pull-request-available > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, Screen Shot > 2020-02-14 at 10.40.05 AM.png, coverage_notes.txt > > Time Spent: 1.5h > Remaining Estimate: 0h > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036633#comment-17036633 ] Dinesh Joshi commented on CASSANDRA-10190: -- Thanks everyone, the final CI run is [here|https://circleci.com/workflow-run/2bcea764-6230-4b69-8e88-fb8e3d76c77c]. I will commit this shortly. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh, pull-request-available > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > Time Spent: 1.5h > Remaining Estimate: 0h > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032024#comment-17032024 ] David Capwell commented on CASSANDRA-10190: --- For the C* changes: build looks good, the cqlsh bash command looks good, so +1 with that. I didn't review the python changes since I don't know it well enough to comment. I ran dtests against this branch and they were fine, so this doesn't regress the existing stuff. the dtest PR looks unrelated filled with style changes and dead code; nothing stood out as needed for this and saw test cases for other JIRAs. +1 to cassandra changes I leave it to your judgement on dtest > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh, pull-request-available > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > Time Spent: 1.5h > Remaining Estimate: 0h > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031896#comment-17031896 ] Dinesh Joshi commented on CASSANDRA-10190: -- [~ptbannister] and [~dcapwell] I have addressed your review comments. I also removed the Python 2 specific code from cassandra-dtests as dtests themselves are expected to be Python 3 only. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh, pull-request-available > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031857#comment-17031857 ] Dinesh Joshi commented on CASSANDRA-10190: -- [~jaikiran], http://cassandra.apache.org/doc/latest/development/ide.html#building-from-source > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh, pull-request-available > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031332#comment-17031332 ] Jaikiran Pai commented on CASSANDRA-10190: -- Hello [~djoshi] > [~jaikiran] ofcourse. You can clone my branch and build C* and use cqlsh that > is packaged in there. Is there a build instructions page somewhere detailing what commands to use to build the `cqlsh` python package? I have looked in that repository and I could only find instructions to run testsuite on the project. I tried building (by guessing the build steps) it myself but didn't get much far. So just checking if I should follow certain instructions or just wait for this to be officially available as a pip installable version from Apache. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh, pull-request-available > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031152#comment-17031152 ] David Capwell commented on CASSANDRA-10190: --- tried to look at the build changes and see config-2.2.yml isn't modified. config.yml* are auto generated from that file so changes need to be there. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh, pull-request-available > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > Time Spent: 20m > Remaining Estimate: 0h > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031093#comment-17031093 ] Dinesh Joshi commented on CASSANDRA-10190: -- Hi [~dcapwell], thanks for the feedback. I have created a PR for [cassandra|https://github.com/apache/cassandra/pull/438] and [cassandra-dtest|https://github.com/apache/cassandra-dtest/pull/57]. I already have defined build/test pipelines for testing with Python 2 & 3 with [Java 8|https://circleci.com/workflow-run/5dded0dc-59d6-47be-9c3b-e0a2b4074b51] and [11|https://circleci.com/workflow-run/5dded0dc-59d6-47be-9c3b-e0a2b4074b51]. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh, pull-request-available > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > Time Spent: 20m > Remaining Estimate: 0h > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030953#comment-17030953 ] David Capwell commented on CASSANDRA-10190: --- https://github.com/apache/cassandra/compare/trunk...dineshjoshi:10190-trunk-review#diff-6469b081699ab92c53e0513a499ca5ebR58. could simplify to return 0 or 1, then the branch could be {code} if is_supported_version "$version"; then {code} https://github.com/apache/cassandra/compare/trunk...dineshjoshi:10190-trunk-review#diff-6469b081699ab92c53e0513a499ca5ebR75 Should propagate the return code https://github.com/apache/cassandra/compare/trunk...dineshjoshi:10190-trunk-review#diff-6469b081699ab92c53e0513a499ca5ebR83 When disabling can you leave a comment so future maintainers know why? script LGTM, will start testing after lunch. I plan to do the following test 1) don't define CQLSH_PYTHON - run python dtests (circle ci is already doing this) 2) define CQLSH_PYTHON pointing to 2.7 - run python dtests 3) define CQLSH_PYTHON pointing to 3.6 - run Python dtests > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh, pull-request-available > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > Time Spent: 20m > Remaining Estimate: 0h > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030316#comment-17030316 ] David Capwell commented on CASSANDRA-10190: --- Starting review now; looking at the bash script since I know that better https://github.com/apache/cassandra/compare/trunk...dineshjoshi:10190-trunk-review#diff-6469b081699ab92c53e0513a499ca5ebL18 {code} $ shellcheck cqlsh In cqlsh line 53: version=$(which $interpreter > /dev/null 2>&1 && $interpreter -c "import os; print('{}.{}'.format(os.sys.version_info.major, os.sys.version_info.minor))") ^---^ SC2230: which is non-standard. Use builtin 'command -v' instead. ^--^ SC2086: Double quote to prevent globbing and word splitting. Did you mean: version=$(which "$interpreter" > /dev/null 2>&1 && $interpreter -c "import os; print('{}.{}'.format(os.sys.version_info.major, os.sys.version_info.minor))") In cqlsh line 60: if [ "$version" = "3.6" -o "$version" = "2.7" ]; then ^-- SC2166: Prefer [ p ] || [ q ] as [ p -o q ] is not well defined. In cqlsh line 70: version=$(get_python_version $interpreter) ^--^ SC2086: Double quote to prevent globbing and word splitting. Did you mean: version=$(get_python_version "$interpreter") In cqlsh line 72: if [ "$(is_supported_version $version)" = "supported" ]; then ^--^ SC2086: Double quote to prevent globbing and word splitting. Did you mean: if [ "$(is_supported_version "$version")" = "supported" ]; then In cqlsh line 73: exec $interpreter "`$interpreter -c "import os; print(os.path.dirname(os.path.realpath('$0')))"`/cqlsh.py" $params ^--^ SC2086: Double quote to prevent globbing and word splitting. ^-- SC2006: Use $(...) notation instead of legacy backticked `...`. ^-^ SC2086: Double quote to prevent globbing and word splitting. Did you mean: exec "$interpreter" "$($interpreter -c "import os; print(os.path.dirname(os.path.realpath('$0')))")/cqlsh.py" "$params" For more information: https://www.shellcheck.net/wiki/SC2166 -- Prefer [ p ] || [ q ] as [ p -o q... https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ... https://www.shellcheck.net/wiki/SC2230 -- which is non-standard. Use builti... {code} > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030193#comment-17030193 ] David Capwell commented on CASSANDRA-10190: --- was asked to help out with review. I don't know python well so won't be able to help too much there, so ill focus more on automation within the context of CI and help get this tested. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17027226#comment-17027226 ] Dinesh Joshi commented on CASSANDRA-10190: -- [~jaikiran] ofcourse. You can clone my branch and build C* and use cqlsh that is packaged in there. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17027192#comment-17027192 ] Jaikiran Pai commented on CASSANDRA-10190: -- Hello Dinesh, Would it be possible to do some beta release of the cqlsh python package which adds support for Python 3, so that some of us can try it out in our projects and report back any issues? > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026453#comment-17026453 ] Dinesh Joshi commented on CASSANDRA-10190: -- I have pushed some updates to my [branch|https://github.com/dineshjoshi/cassandra/tree/10190-trunk-review] and gotten a mostly clean [dtest run|https://circleci.com/workflow-run/ae00bd4f-1423-43bf-8f49-0bfd8a04d165]. The failure is unrelated to the changes and the tests passes locally. I think we need to add pipelines for cqlsh tests for Python 2 and Python 3 now that we will support both versions. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024803#comment-17024803 ] Dinesh Joshi commented on CASSANDRA-10190: -- I have pushed an update to my [branch|https://github.com/dineshjoshi/cassandra/tree/10190-trunk-review] which fixes {{test_complex_schema}} and {{test_complex_data_types}}. Turned out to be a minor issue. I also ran into an issue with running cqlsh on the command line. The code was trying to import {{six}} before it was included in the {{PYTHONPATH}}. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022679#comment-17022679 ] Dinesh Joshi commented on CASSANDRA-10190: -- I have rebased and squashed, [~ptbannister]'s original branch into my repo [here|https://github.com/dineshjoshi/cassandra/tree/10190-trunk-review] and run [circleci|https://circleci.com/workflow-run/53b5f86f-2a97-4ce8-8269-9fab522f9406] jobs. There are about 5 tests that are failing right now. The following tests pass locally but fail on CircleCI (let's debug these later) – - test_writing_with_token_boundaries - test_datetime_values - test_remote_query The following tests fail locally and on CircleCI so I think these are Genuine failures (let's focus on them right now) – - test_complex_schema - test_complex_data_types > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022618#comment-17022618 ] Dinesh Joshi commented on CASSANDRA-10190: -- Thanks [~andrew.tolbert]. I still see a [few failures|https://circleci.com/workflow-run/34346de3-a7fd-47de-8c15-ce17c0c105ad] but they're looking better than before. I will try to debug them. In the meanwhile if you have any fixes in the pipeline, please post them here. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16946590#comment-16946590 ] Andy Tolbert commented on CASSANDRA-10190: -- Added [^0001-Fix-issues-from-version-specific-logic-commit.patch] which fixes issues from my previous commit and should cause the 2 new failing tests to now pass. There's still a little more testing I'd like to do though, from the [{{cqlshlib6-rebase-20190322}}|https://github.com/ptbannister/cassandra-dtest/commits/cqlshlib6-rebase-20190322] branch, other than the virtual tables related failures, there's one test that fails because a unicode type instead of str type is given: {noformat} TestCqlsh.test_materialized_view ... > assert "Materialized view 'users_by_state' not found" in err E assert "Materialized view 'users_by_state' not found" in ":2:Materialized view u'users_by_state' not found\n" {noformat} > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16946544#comment-16946544 ] Andy Tolbert commented on CASSANDRA-10190: -- I see the problem, in {{make_params}}, calling {{ensure_text(query)}} will fail if using prepared statements, since you aren't operating on a string, but rather a prepared statement, this causes an unhandled exception. Instead we should only call {{ensure_text}} if not using a prepared statement. In addition, I discovered a couple other issues with my commit which I'll provide fixes for shortly. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16946196#comment-16946196 ] Andy Tolbert commented on CASSANDRA-10190: -- Hi [~djoshi],interesting! The 2 new failures seem to be related to copy code, so it's probably a regression from my commit. Nothing sticks out as obvious to me as well at the moment, I'll take a look tonight and will get to the bottom of it. Will also see if I can wrap my head around the {{test_describe}} test. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16946188#comment-16946188 ] Dinesh Joshi commented on CASSANDRA-10190: -- [~andrew.tolbert], looks like there are a few [test failures|https://circleci.com/gh/dineshjoshi/cassandra/1229#tests/containers/31]. I could not spot anything obvious. Do you mind looking into them? > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16946136#comment-16946136 ] Dinesh Joshi commented on CASSANDRA-10190: -- Hi [~andrew.tolbert], thanks for the review comments. I have pulled your changes to my branch [here|https://github.com/dineshjoshi/cassandra/tree/10190-rebase-20190609]. The circleci tests are running [here|https://circleci.com/workflow-run/4806c5fa-537d-4964-894a-bc4b05958679]. If everything looks good, I'll do a final review, squash and merge so folks can start testing out Python 3 compatibility. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945587#comment-16945587 ] Andy Tolbert commented on CASSANDRA-10190: -- {quote} I think it's worth updating the driver to a newer version {quote} I just read the comments earlier in the thread about the driver, mainly this one: {quote} Finally, extracting the checked in driver is something we'll look into. But that is a longpole and strictly not needed for Python 3 compatibility. {quote} Agree with that, don't think that should hold things up, it can be done out of band. Noticed that I had already opened up a ticket of updating the python driver a while back for a different reason ([CASSANDRA-14872]), and there's probably other 4.0 specific reasons it needs to be updated, so would be good to do that separately. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945585#comment-16945585 ] Andy Tolbert commented on CASSANDRA-10190: -- Hi [~ptbannister], this is much needed and great work! I took a look at your branch today and it looks really good to me. I have some minor suggestions, but I think things are pretty good as is! I made patches of my suggestions below ([^0001-Update-six-to-1.12.0.patch], [^0002-Simplify-version-specific-logic-by-using-six.moves-a.patch]). Note that other than the import change in {{winpty.py}}, these changes don't really fix anything, so I would not mind if you chose not to accept them. Recommended changes: {{six}} * I realized that a very old version of six (1.7.3) is being provided via lib as the cassandra-driver being pulled in claims to require six 1.9.0. We should consider upgrading that, especially since there are some newer features (covered below) that may be useful. * There are some additional features of six that could be used which would simplify/remove python version checking logic. For example, there's a lot of code that calls encode/decode if PY2 is used, {{six.ensure_text}} could be used instead. {{bin/cqlsh}} * L85-95: Commented out code probably not needed anymore? {{bin/cqlsh.py}} * L50-56: Could make use of [six.moves|https://six.readthedocs.io/#module-six.moves] to impor in a version-agnostic way, i.e. from {{six.moves import configparser}}, {{from six.moves import cStringIO as StringIO}}) * L867: Instead of {{_input}}, could use {{six.moves.input}}. {{pylib/copyutil.py}} * L48-54: use {{six.moves}} (see comment ^ {{cqlsh.py}}) * where {{range}} is used in favor of {{xrange}}, it would be good to use {{six.moves.range}} instead, since when using python2 {{xrange}} will be used so the existing behavior for python2 will be maintained. {{pylib/cqlshlib/formatting.py}} * {{ensure_text}} can be utilized in a places to simplify conversion logic. {{pylib/cqlshlib/saferscanner.py}} * Unless i'm mistaken {{Py35SaferScanner}} and {{Py36SaferScanner}} are the same, {{Py35SaferScanner}} not used, so can be removed. {{pylib/cqlshlib/sslhandling.py}} * L21-24, use {{six.moves}} {{pylib/cqlshlib/test/winpty.py}} & {{pylib/cqlshlib/util.py}} * import queue won't work with python2, & might as well use cStringIO The tests all passed for me except the one you mentioned with python3 (TestCqlshOutput#test_user_types). I’m pretty certain this is fixed in newer python driver versions. The driver bundled with C* is several minor versions old and is an internal build with some changes for transient replicas (I’m guessing has to do with the way you express replication when creating a keyspace with transient replicas). I think it's worth updating the driver to a newer version, and maybe that can be done in a separate patch unless you think it should be done here? > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944663#comment-16944663 ] Dinesh Joshi commented on CASSANDRA-10190: -- Hi [~ptbannister], the latest set of changes seem to help with the test failures. There is just 1 test failure that seems to be relevant. [https://circleci.com/gh/dineshjoshi/cassandra/1223#tests/containers/96] and [https://circleci.com/gh/dineshjoshi/cassandra/1222#tests/containers/4] {{The failure is in the describe command (test_describe - cqlsh_tests.test_cqlsh.TestCqlsh}}{{cqlsh_tests/test_cqlsh.py):}} {noformat} E AssertionError: assert ['CREATE KEYS...d, col)', ...] == ['CREATE KEYSP...d, col)', ...] E At index 21 diff: 'CREATE INDEX test_val_idx ON test.test (val);' != 'CREATE INDEX test_col_idx ON test.test (col);' E Use -v to get the full diff {noformat} > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939184#comment-16939184 ] null commented on CASSANDRA-10190: -- Thanks for working on this, I can't believe cqlsh is limited to Python 2.7, seems crazy when 1st January 2020 isn't too far away. How far away are you from a working cqlsh on Python 3.x? thanks again for your great work on this :) > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928785#comment-16928785 ] Patrick Bannister commented on CASSANDRA-10190: --- I'm experimenting with changing the cqlshlib unit tests in {{test_cqlsh_completion}}. These are the slowest part of the cqlshlib unit test suite. They currently work by running cqlsh in a subprocess to connect to a ccm cluster, sending partial commands to the cqlsh, and reading the completions suggested by the cqlsh. This is slow, because we have to use a timeout to wait for the cqlsh to accept our input and output its recommended completions. In general, these don't really look like unit tests, they look more like dtests. One approach would be to remove this from the unit tests and make it part of the dtests. But I'd prefer to convert them to something more like a unit test. We can make them run faster by using cqlshlib internals directly, instead of running cqlsh in a subprocess. We can call {{cql_complete}} on each partial command and assert that the completions are what we expect. Once I've rewritten the {{test_cqlsh_completion.py}} tests like this, it'll be up on the branch for consideration. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916262#comment-16916262 ] Patrick Bannister commented on CASSANDRA-10190: --- I added new unit tests for the {{get_ranges}} function of {{cqlshlib.copyutil.ExportTask}}. The tests exposed a bug in {{get_ranges}}, so I fixed that too. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16895682#comment-16895682 ] Patrick Bannister commented on CASSANDRA-10190: --- [~djoshi3] the behavior of the {{make_range}} function was different under Python 2.7 and Python 3.6. For a quick demonstration of the reason, try running this in a Python 2 interpreter, and then again in a Python 3 interpreter: {code:python} assert 0 > None {code} Under Python 2 this is fine, but under Python 3, attempting to compare {{NoneType}} to anything else will throw a {{TypeError}}. This problem was exposed by failures in the dtest {{test_writing_with_token_boundaries}} (from cqlsh_tests/test_cqlsh_copy.py). When the test ran the COPY command with the WITH BEGINTOKEN and WITH ENDTOKEN options, cqlsh crashed because of the {{TypeError}}. I like the idea of adding some unit tests around this, I'll do it next time I get a solid hour to work on this. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894599#comment-16894599 ] Dinesh Joshi commented on CASSANDRA-10190: -- Hi [~ptbannister], thanks for the changes and sorry about the slow responses on my end. I had a few questions and comments regarding [this|https://github.com/ptbannister/cassandra/commit/db10edd766f9d568f7df79f6d8cd16bc496dd6c3] change. I see you're trying to prevent a {{NoneType}} comparison. What test specifically broke due to this change? Could you add a unit test to ensure that this function works correctly with {{NoneType}} arguments? I want to also point out that the behavior of this code is same under Python 2.7 and 3.6. Since {{cassandra.metadata.Token}} implements {{__lt__()}} and {{__gt__()}} methods, you will get an error if you try comparing it with {{NoneType}} so I'm a bit puzzled why you needed this change. I agree that this piece of code is error prone in the face of {{NoneType}} arguments and we should fix it. I'm just trying to understand did we actually break something during migration? Other than this question, the only feedback I have so far is to remove commented out code from {{bin/cqlsh}}. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883359#comment-16883359 ] Patrick Bannister commented on CASSANDRA-10190: --- Rebased my dtest branch on the latest dtest master, no regressions in the cqlsh tests. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883278#comment-16883278 ] Patrick Bannister commented on CASSANDRA-10190: --- I rebased my cassandra branch on the latest trunk, addressed some failures in the dtests, and introduced a new --python option to the `cqlsh` launcher script so that a user may specify the path to a Python interpreter that they would like to use to run cqlsh. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882929#comment-16882929 ] Patrick Bannister commented on CASSANDRA-10190: --- [~spo...@gmail.com], yes, those are the current branches. For this ticket, the immediate goals are: * Port cqlshlib to run on Python 3.6 and Python 2.7 * Get the cqlshlib unit tests to parity between Python 2.7 and Python 3.6, except for documented cases with external problems, such as cases where Python 3 breaks the driver The dtest branch we're using doesn't enable the remaining dtests, it only gets the currently enabled dtests working. I would like to merge these changes to get cqlshlib ported first, before taking on the remaining dtests. It's more urgent to port cqlshlib. It would also help manage the scope of the problem to take care of cqlshlib before dealing with the remaining dtests. As for breaking out the cqlshlib tests, my recommendation is to continue delivering them as part of cqlshlib itself, because it feels wrong to break the tests away from the repo. But we can address this question separately, on CASSANDRA-14990. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871740#comment-16871740 ] Stefan Podkowinski commented on CASSANDRA-10190: I was just trying to catch up, so please correct me if I'm wrong. So we're adding 1) Python 3 support for cqlshlib and tests 2) fix tests in cqlshlib/test but don't actually run them on CI (will be done in CASSANDRA-14990) 3) fix and enable remaining cqlsh dtest leftovers from CASSANDRA-14298 with dependency to cqlshlib. Current branches are https://github.com/ptbannister/cassandra/tree/10190-rebase-20190609 https://github.com/ptbannister/cassandra-dtest/commits/cqlshlib6-rebase-20190322 > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866010#comment-16866010 ] Patrick Bannister commented on CASSANDRA-10190: --- [~djoshi3], I studied the two failures in the dtest cqlsh_tests. I was able to address one. For the other, I suggest we do a fresh run and see if it recurs. Details below for posterity. The failure in test_writing_with_token_boundaries was a Python 3 specific bug in the token range calculation function in cqlshlib copyutil.py, I pushed a fix to the branch to address it. The failure in test_reading_max_insert_errors was deeper than cqlshlib - it was a warning message from one of the Cassandra nodes about garbage collection of an unreleased ByteBuf in the Netty epollEventLoopGroup. The test where this happened intentionally causes some errors to occur with the COPY FROM command, so this might be an indication that we're not cleaning up properly under certain error conditions exercised by this test. This might be worth investigating, but I'd say it's not a cqlshlib issue. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863768#comment-16863768 ] Dinesh Joshi commented on CASSANDRA-10190: -- Hi [~ptbannister], thank you for making the changes. The latest set of changes look good. I ran the dtests. There are some cqlsh related failures in the vnodes case. Could you please take a look? Additionally, as part of this change, I think it would be desirable to run cqlsh tests with Python 2.7 as well as Python 3. I think it would be trivial to do it if we can override the Python interpreter's path by providing it as a parameter. WDYT? > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859468#comment-16859468 ] Patrick Bannister commented on CASSANDRA-10190: --- [~djoshi3], thanks for the feedback. I've incorporated all of your recommendations on the branch. I meant to completely delete FrozenType after testing with it commented out, but I forgot to follow through. The reason I wanted to remove the FrozenType class from cqlsh.py is because it was commented "Needed until the bundled python driver adds FrozenType.", and I noticed that [the Python driver includes FrozenType since version 2.5.0|[https://github.com/datastax/python-driver/blob/master/CHANGELOG.rst#250].] It looks like the bundled driver is at least version 3.7.0 (CASSANDRA-12736), so I think we should be able to remove FrozenType completely. I've made this change on my branch too. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856488#comment-16856488 ] Dinesh Joshi commented on CASSANDRA-10190: -- Hi [~ptbannister], I have gone over your patch and here are some minor comments - {{cqlsh.py}} * Why is {{FrozenType}} commented out? * You've replaced occurrences of %r with "'{}'". However, we should use "{!r}" instead which is equivalent to %r. Please remove the single quotes surrounding the object names. * Remove {{six}} import from L42. This is a premature import and breaks Python 2. {{six}} is imported on L150 once we have discovered the Python libraries path for cqlsh. * Python version check could be a bit more readable like this {{sys.version_info.major != 3 and (sys.version_info.major == 2 and sys.version_info.minor != 7)}} instead of using indexes. {{copyutil.py}} * L2514 remove commented out code. {{cqlsh}} * Nit: L51: python -> Python > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845259#comment-16845259 ] Dinesh Joshi commented on CASSANDRA-10190: -- [~spo...@gmail.com], if you look at the {{README}}, the ultimate intent is to have this integrated in the test pipeline. However, we ran into some issues. The current state allows for manually verifying cqlsh with Python 2 & 3. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845256#comment-16845256 ] Stefan Podkowinski commented on CASSANDRA-10190: I'm a bit busy at the moment, but will try to have a look. It's interesting to see that you've added docker support for running tests with bare py2/py3 installations. Maybe we can integrate that with circleci. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16841844#comment-16841844 ] Patrick Bannister commented on CASSANDRA-10190: --- [~spo...@gmail.com], this has been progressing, and we'd like your review and feedback. I have a branch from cassandra trunk that will make cqlsh, cqlshlib, and the accompanying Python unit tests work for Python 2.7 and 3.6: https://github.com/ptbannister/cassandra/tree/10190-rebase-20190329 If the cqlsh and cqlshlib changes are good, then we'd circle back to update the dtests too, but in a separate PR. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16833198#comment-16833198 ] Patrick Bannister commented on CASSANDRA-10190: --- In the cqlshlib tests, in test_cqlsh_output.py, test_user_types_output is failing on Python 3. This is because of an unhandled exception in the cassandra Python driver. This is happening because of the Python 3 behavior of throwing an exception when a str type object is compared to a NoneType object (which was not a problem in Python 2). The test data includes a user defined type that is populated with some null fields, and when the Cassandra driver tries to output this data, it does some comparison, and this causes an exception, which it does not handle. We will need to accept this as a known failure until we can address the behavior in the driver. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826742#comment-16826742 ] Dinesh Joshi commented on CASSANDRA-10190: -- [~ptbannister] I found a few issues and fixed them. Here's a branch with the changes and rebased on the current trunk - https://github.com/dineshjoshi/cassandra/tree/10190-review > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826668#comment-16826668 ] Dinesh Joshi commented on CASSANDRA-10190: -- [~ptbannister] I went over the tests in https://github.com/ptbannister/cassandra/tree/10190-rebase-20190329 and looks like we have reached parity with Python 2.7. Could you please create a PR for review? > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805523#comment-16805523 ] Dinesh Joshi commented on CASSANDRA-10190: -- Hi [~riley.mcdowell], Patrick & I had a chat and we will be first bringing the Python 3 implementation in line with the Python 2.7 which means if there are tests that are currently broken, we won't fix them as long as the failures are the same. Once we have a better insight into what the failures are about, we will go in and fix whatever is possible without touching the driver. Finally, extracting the checked in driver is something we'll look into this. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805519#comment-16805519 ] Riley McDowell commented on CASSANDRA-10190: [~ptbannister] The contributions to the upstream driver might take a while. Would you consider # Moving forward with the Python3 cqlsh support as-is to push for a release, and # Also opening two additional, smaller tickets to resolve those two failing tests? > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805188#comment-16805188 ] Dinesh Joshi commented on CASSANDRA-10190: -- [~ptbannister] could you please post your branch? Like we discussed, the driver that cqlsh uses is checked in [here|https://github.com/apache/cassandra/blob/trunk/lib/cassandra-driver-internal-only-3.12.0.post0-5838e2fd.zip]. We probably need to change that. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805180#comment-16805180 ] Patrick Bannister commented on CASSANDRA-10190: --- Update on the tests: I have everything passing, except for some problems that appear rooted in the driver: * cassandra-dtest cqlsh_tests/test_cqlsh_copy.py TestCqlshCopy test_unusual_dates: the driver is choking when it tries to read dates after the year 1. * cqlshlib tests pylib/cqlshlib/test/test_cqlsh_output.py TestCqlshOutput test_user_types_output: the driver is choking when it tries to read a user defined type for a frozen set with nulls. Python does not support ordering a collection with null values (NoneType). I don't believe either of these problems can be solved with cqlshlib alone. Probably we would need to contribute to the driver to resolve them. My feeling is that both of these problems are blockers. Any input from our watchers? > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805177#comment-16805177 ] Dinesh Joshi commented on CASSANDRA-10190: -- [~sangrampatil] this is complex piece and I am not sure if we will back port it the older releases. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805175#comment-16805175 ] Patrick Bannister commented on CASSANDRA-10190: --- It's difficult to provide an ETA. This is a big ticket, and most of us are working on a volunteer basis. The best I can say is that I've heard there's a lot of interest in this ticket, and that should provide some motivation to get it done. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805044#comment-16805044 ] Sangram Patil commented on CASSANDRA-10190: --- [~ptbannister] Thanks for the updates. Could you please share the ETA for cqlsh for python3. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804977#comment-16804977 ] Patrick Bannister commented on CASSANDRA-10190: --- [~djoshi3] is working on moving the cqlshlib tests to the dtests in CASSANDRA-14990. On his suggestion, I'll build on his work. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804971#comment-16804971 ] Patrick Bannister commented on CASSANDRA-10190: --- All of the cqlsh_tests dtests are passing again, time to look at the cqlshlib tests. Now we need to revisit the cqlshlib tests. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799483#comment-16799483 ] Patrick Bannister commented on CASSANDRA-10190: --- CASSANDRA-14298 has been merged, and I've resumed work on this ticket. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778839#comment-16778839 ] Sangram Patil commented on CASSANDRA-10190: --- Thank you [~ptbannister], I'll take a look at it. Could you please keep posted about any updates on python3 support of cqlsh. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778833#comment-16778833 ] Patrick Bannister commented on CASSANDRA-10190: --- I think I'll need to look at this more closely to understand what happened, but sharing my initial intuition: I suspect this is a change in the internals of the Python re module. The saferscanner class plays with fire by extending the Python re.Scanner class. Scanner is supposed to be internal, not a public API offering, and dealing with changes in re.Scanner between Python 2.7 and Python 3 was one of the first challenges I tackled in porting cqlsh to Python 3. That's just my gut feeling looking at the stacktrace. I can look more in depth after I do more work with this branch when CASSANDRA-14298 is merged. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1694#comment-1694 ] Sangram Patil commented on CASSANDRA-10190: --- [~ptbannister] Thank you very much for the details with necessary information. I have tried on both the branches cqlshlib3 and cqlshlib6 as well. In my environment Ubuntu 18.04 LTS, I have python3.6.7. When I try to import cqlsh it throws the error. Copied below from my terminal sangramp@cassandra-m:~$ python3 Python 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path.insert(0,'/home/sangramp/cassandra/bin') >>> import cqlsh Traceback (most recent call last) : File "", line 1, in File "/home/sangramp/cassandra/bin/cqlsh.py", line 168, in from cqlshlib import cql3handling, cqlhandling, pylexotron, sslhandling, cqlshhandling File "/home/sangramp/cassandra/bin/../pylib/cqlshlib/cql3handling.py", line 19, in from cqlshlib.cqlhandling import CqlParsingRuleSet, Hint File "/home/sangramp/cassandra/bin/../pylib/cqlshlib/cqlhandling.py", line 23, in from cqlshlib import pylexotron, util File "/home/sangramp/cassandra/bin/../pylib/cqlshlib/pylexotron.py", line 342, in class ParsingRuleSet: File "/home/sangramp/cassandra/bin/../pylib/cqlshlib/pylexotron.py", line 354, in ParsingRuleSet ], re.I | re.S | re.U) File "/home/sangramp/cassandra/bin/../pylib/cqlshlib/saferscanner.py", line 41, in __init__ self.scanner = re.sre_compile.compile(p) File "/usr/lib/python3.6/sre_compile.py", line 566, in compile code = _code(p, flags) File "/usr/lib/python3.6/sre_compile.py", line 551, in _code _compile(code, p.data, flags) File "/usr/lib/python3.6/sre_compile.py", line 187, in _compile _compile(code, av, flags) File "/usr/lib/python3.6/sre_compile.py", line 141, in _compile group, add_flags, del_flags, p = av ValueError: not enough values to unpack (expected 4, got 2) However, in my another environment, where I have python 3.5.2, it does not throw this exception at least. Python 3.6.7 is in my production environment, where I need cqlsh to be supported. Could you please have a quick look at the error and suggest me the possible reason for the error or guide me to fix the error. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777291#comment-16777291 ] Patrick Bannister commented on CASSANDRA-10190: --- Please check out my branches for this on Github: * [https://github.com/ptbannister/cassandra.git] * [https://github.com/ptbannister/cassandra-dtest.git] * [https://github.com/ptbannister/ccm.git] On one line of work I was working in straight Python 3, no Python 2.7 compatibility. For that one, check out the cassandra and cassandra-dtest branches called "cqlshlib3". But I built on that to make it compatible with both Python 3 and Python 2.7. I was working on that branch later, so I have a bit more confidence in it. For that one, check out the branches called "cqlshlib6." This also requires a small change to ccm, which is available in the cqlshlib6 branch of my ccm fork. Caveat that I haven't looked at this since October, so there may be some problems running it now unless you use a Cassandra Python driver from around that time. I've been planning to pick this back up and finish it after CASSANDRA-14298 gets merged. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776778#comment-16776778 ] Sangram Patil commented on CASSANDRA-10190: --- [~ptbannister] Any updates on this? Also can you please share the link for patch that got cqlsh working on 3 and 2.7 > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774015#comment-16774015 ] aman raparia commented on CASSANDRA-10190: -- Patrick can you please post the patch link for that and also is the new code for cqlsh for python 3 is updated on github repo > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774002#comment-16774002 ] Patrick Bannister commented on CASSANDRA-10190: --- We've resumed CASSANDRA-14298 in the past week, hopefully we'll get that done soon, and that would clear the way for this ticket. I had a patch that got cqlsh working on 3 and 2.7 in October. The saferscanner problems are solvable. Patrick Bannister > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773785#comment-16773785 ] aman raparia commented on CASSANDRA-10190: -- has porting been completed of cqlsh for Python 3.7?? as there no updated on the issue since August 2018. I am also trying to make the cqlsh compatible with Python 3.7. However stuck to resolve the issue in the saferscanner file of cqlshlib. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592916#comment-16592916 ] Patrick Bannister commented on CASSANDRA-10190: --- Quick update on this ticket. The porting work is mostly done - I had basically all the tests passing under Python 2.7 and Python 3.5 on CentOS and Ubuntu environments. Making the dtests Python 2/3 cross compatible was trivial, mostly I just had to add explicit inheritance from the object class to a few of the basic classes like dtest.py. There will be a small patch to ccm related to Unicode encoding when calling cqlsh. I've learned a bit about our test infrastructure, so I have a better idea of what will be involved for the testing environment subtask (CASSANDRA-14491). I'll continue work on this ticket after we complete CASSANDRA-14298 - that task is building momentum again so I think it will be soon. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504070#comment-16504070 ] Patrick Bannister commented on CASSANDRA-10190: --- In discussion on the cassandra-dev list, a consensus formed around maintaining support for Python 2. I'm working on making the cqlsh port to Python 3 cross-compatible with Python 2.7. The goal is a single cross-compatible implementation. I'm making progress and I suspect the port will be the easy part. Getting testing infrax right to support both versions will probably be more complex. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499263#comment-16499263 ] Patrick Bannister commented on CASSANDRA-10190: --- I added several subtasks to this ticket for testing of the areas that were not covered according to my coverage report. (Refuting the coverage report by identifying an existing test for the supposedly uncovered features would be superior to writing a new test.) > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498851#comment-16498851 ] Patrick Bannister commented on CASSANDRA-10190: --- [~jolynch], thanks! Help figuring out the testing infrastructure would be much appreciated, I don't have any experience with tox yet. Right now I'm testing on AWS EC2 instances, I haven't made the jump to CI systems yet. I'll add some subtasks to this ticket to address our testing gaps - authentication, Windows, displaying CJK languages, describe statements, and use of commas for thousands separators and decimal separators are the concrete items I think are urgent. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498303#comment-16498303 ] Joseph Lynch commented on CASSANDRA-10190: -- [~ptbannister] testing against both python2 and python3 is reasonably easy using tox and it sounds like supporting both is going to be the easiest path forwards until distros drop py27. If you want help testing or reviewing let me know, happy to take a look or contribute test cases, just let me know where I can help. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497274#comment-16497274 ] Patrick Bannister commented on CASSANDRA-10190: --- I ran the cqlshlib unittests (cassandra/pylib/cqlshlib/test/test_*.py) and the dtest cqlsh_tests under coverage.py. There are a few blind spots in the tests. Notes attached in coverage_notes.txt. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490904#comment-16490904 ] Patrick Bannister commented on CASSANDRA-10190: --- Discussion of porting, testing, and fixing cqlsh_tests, including tests that depend on cqlshlib. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490899#comment-16490899 ] Patrick Bannister commented on CASSANDRA-10190: --- Python 2.7 end of life, including security updates, is planned for 1 January 2020. We should port cqlshlib to Python 3, or make it Python 2.7/3.x cross compatible. I'm close to finishing a port to Python 3 that passes almost all of the cqlshlib unittests and the dtest cqlsh_tests on Ubuntu 16.04 LTS and CentOS 7.5 - I'm just working out some environmental issues that manifest on CentOS. Regardless of whether we go to straight Python 3 or a 2/3 cross compatible implementation, we have to get things working in Python 3. ([http://python3porting.com/strategies.html] has a good discussion on this point.) If we go with a cross compatible implementation, we would introduce complexity and carry a long term technical debt. For that reason, I personally prefer a straight Python 3 implementation. However, it may be easier for many users to transition to a cross compatible implementation, so we should at least consider it. Thorough testing would be important. After I get all the tests running, I'm planning to use [coverage.py|https://coverage.readthedocs.io/en/coverage-4.5.1] to identify areas of cqlshlib that aren't well tested by our current tests, so we'll know what features need more scrutiny. We will probably need users to help with testing if we hope to catch all the bugs. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14713581#comment-14713581 ] Andrew Pennebaker commented on CASSANDRA-10190: --- I'm not speaking of the driver, I'm speaking of the cqlsh command line program. Its try/catch syntax is incompatible with Python 3, forcing cqlsh users to carefully manage multiple Python versions. -- Cheers, Andrew Python 3 support for cqlsh -- Key: CASSANDRA-10190 URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 Project: Cassandra Issue Type: Improvement Reporter: Andrew Pennebaker Labels: cqlsh Users who operate in a Python 3 environment may have trouble launching cqlsh. Could we please update cqlsh's syntax to run in Python 3? As a workaround, users can setup pyenv, and cd to a directory with a .python-version containing 2.7. But it would be nice if cqlsh supported modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14713566#comment-14713566 ] Tyler Hobbs commented on CASSANDRA-10190: - The python driver already supports Python 3 through the [six|http://pythonhosted.org/six/#] library (which is now bundled in lib/), so this would only require updating {{cqlsh}} and {{pylib/cqlshlib/*.py}}. Python 3 support for cqlsh -- Key: CASSANDRA-10190 URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 Project: Cassandra Issue Type: Improvement Reporter: Andrew Pennebaker Labels: cqlsh Users who operate in a Python 3 environment may have trouble launching cqlsh. Could we please update cqlsh's syntax to run in Python 3? As a workaround, users can setup pyenv, and cd to a directory with a .python-version containing 2.7. But it would be nice if cqlsh supported modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14713625#comment-14713625 ] Tyler Hobbs commented on CASSANDRA-10190: - Right, I was also referring to cqlsh. The python driver is used by cqlsh, so Python 3 compatibility there is a prerequisite (that is fortunately already taken care of). Python 3 support for cqlsh -- Key: CASSANDRA-10190 URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 Project: Cassandra Issue Type: Improvement Reporter: Andrew Pennebaker Labels: cqlsh Users who operate in a Python 3 environment may have trouble launching cqlsh. Could we please update cqlsh's syntax to run in Python 3? As a workaround, users can setup pyenv, and cd to a directory with a .python-version containing 2.7. But it would be nice if cqlsh supported modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v6.3.4#6332)