[jira] [Commented] (CASSANDRA-10799) 2 cqlshlib tests still failing with cythonized driver installation
[
https://issues.apache.org/jira/browse/CASSANDRA-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052550#comment-15052550
]
Stefania commented on CASSANDRA-10799:
--
CI is OK except for a pep8 compliance problem, which I fixed without repeating
CI.
This is ready to commit.
> 2 cqlshlib tests still failing with cythonized driver installation
> --
>
> Key: CASSANDRA-10799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10799
> Project: Cassandra
> Issue Type: Test
> Components: Testing
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.2.x, 3.x
>
>
> We still have 2 cqlshlib tests failing on Jenkins:
> http://cassci.datastax.com/job/cassandra-3.0_cqlshlib/lastCompletedBuild/testReport/
> Locally, these tests only fail with a cythonized driver installation. If the
> driver is not cythonized (installed with {{--no_extensions}}) then the tests
> are fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10799) 2 cqlshlib tests still failing with cythonized driver installation
[
https://issues.apache.org/jira/browse/CASSANDRA-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052343#comment-15052343
]
Stefania commented on CASSANDRA-10799:
--
Thank [~pauloricardomg], I've renamed _import_export.py_ to _copyutil.py_
(singular to match _util.py_) and restarted CI. I'll move the ticket to ready
to commit once CI is OK.
||2.1||2.2||3.0||trunk||
|[patch|https://github.com/stef1927/cassandra/commits/10799-2.1]|[patch|https://github.com/stef1927/cassandra/commits/10799-2.2]|[patch|https://github.com/stef1927/cassandra/commits/10799-3.0]|[patch|https://github.com/stef1927/cassandra/commits/10799]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.1-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.2-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-3.0-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-dtest/]|
> 2 cqlshlib tests still failing with cythonized driver installation
> --
>
> Key: CASSANDRA-10799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10799
> Project: Cassandra
> Issue Type: Test
> Components: Testing
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.2.x, 3.x
>
>
> We still have 2 cqlshlib tests failing on Jenkins:
> http://cassci.datastax.com/job/cassandra-3.0_cqlshlib/lastCompletedBuild/testReport/
> Locally, these tests only fail with a cythonized driver installation. If the
> driver is not cythonized (installed with {{--no_extensions}}) then the tests
> are fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10799) 2 cqlshlib tests still failing with cythonized driver installation
[
https://issues.apache.org/jira/browse/CASSANDRA-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049371#comment-15049371
]
Paulo Motta commented on CASSANDRA-10799:
-
Feel free to mark as ready to commit when you're ready.
> 2 cqlshlib tests still failing with cythonized driver installation
> --
>
> Key: CASSANDRA-10799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10799
> Project: Cassandra
> Issue Type: Test
> Components: Testing
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.2.x, 3.x
>
>
> We still have 2 cqlshlib tests failing on Jenkins:
> http://cassci.datastax.com/job/cassandra-3.0_cqlshlib/lastCompletedBuild/testReport/
> Locally, these tests only fail with a cythonized driver installation. If the
> driver is not cythonized (installed with {{--no_extensions}}) then the tests
> are fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10799) 2 cqlshlib tests still failing with cythonized driver installation
[
https://issues.apache.org/jira/browse/CASSANDRA-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049369#comment-15049369
]
Paulo Motta commented on CASSANDRA-10799:
-
Sorry for the delay on this. LGTM, tested locally with the driver uninstalled
and worked. I'd only change the {{import_export}} name to {{copylib}} or
{{copyutils}}, but that's a minor personal thing, so feel free to disregard. :)
> 2 cqlshlib tests still failing with cythonized driver installation
> --
>
> Key: CASSANDRA-10799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10799
> Project: Cassandra
> Issue Type: Test
> Components: Testing
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.2.x, 3.x
>
>
> We still have 2 cqlshlib tests failing on Jenkins:
> http://cassci.datastax.com/job/cassandra-3.0_cqlshlib/lastCompletedBuild/testReport/
> Locally, these tests only fail with a cythonized driver installation. If the
> driver is not cythonized (installed with {{--no_extensions}}) then the tests
> are fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10799) 2 cqlshlib tests still failing with cythonized driver installation
[
https://issues.apache.org/jira/browse/CASSANDRA-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041536#comment-15041536
]
Stefania commented on CASSANDRA-10799:
--
I've adapted the cqlshlib tests to always use the embedded driver even when
there is a driver installed. Two things to note:
* Because the location of the lib folder containing the driver zip file is
calculated as relative to {{__file__}}, the soft link created was causing
problems and so I got rid of it. The reason for this soft link was exclusively
for importing cqlsh and since the renaming to cqlsh.py in 2.2 this is no longer
necessary. However in 2.1 we cannot get rid of a soft link (or copy) unless we
rename cqlsh to cqlsh.py and therefore I did not fix 2.1.
* If running the tests from _pylib/cqlshlib_, like we don on Jenkins,
_cqlshlib_ is added to {{sys.path}} and this caused an error when importing
{{cassandra.Cluster}} due to a conflict between the python copy module and
_cqlshlib/copy.py_. This is a serious problem and I fixed it in 2.1 as well.
The only reason we don't notice when running cqlsh is that we import
{{cassandra.Cluster}} after adding cqlshlib to {{sys.path}}.
Patches and CI:
||2.1||2.2||3.0||
|[patch|https://github.com/stef1927/cassandra/commits/10799-2.1]|[patch|https://github.com/stef1927/cassandra/commits/10799-2.2]|[patch|https://github.com/stef1927/cassandra/commits/10799-3.0]|
|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.1-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.2-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-3.0-testall/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.1-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.2-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-3.0-dtest/]|
2.2 applied cleanly to 3.0 and so I did not create the patches for 3.1 and
trunk but I can do so after review is completed.
> 2 cqlshlib tests still failing with cythonized driver installation
> --
>
> Key: CASSANDRA-10799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10799
> Project: Cassandra
> Issue Type: Test
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.2.x, 3.x
>
>
> We still have 2 cqlshlib tests failing on Jenkins:
> http://cassci.datastax.com/job/cassandra-3.0_cqlshlib/lastCompletedBuild/testReport/
> Locally, these tests only fail with a cythonized driver installation. If the
> driver is not cythonized (installed with {{--no_extensions}}) then the tests
> are fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10799) 2 cqlshlib tests still failing with cythonized driver installation
[
https://issues.apache.org/jira/browse/CASSANDRA-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15039774#comment-15039774
]
Stefania commented on CASSANDRA-10799:
--
Thanks for confirming.
I agree with changing the cqlshlib tests to always use the bundled driver.
> 2 cqlshlib tests still failing with cythonized driver installation
> --
>
> Key: CASSANDRA-10799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10799
> Project: Cassandra
> Issue Type: Test
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.2.x, 3.x
>
>
> We still have 2 cqlshlib tests failing on Jenkins:
> http://cassci.datastax.com/job/cassandra-3.0_cqlshlib/lastCompletedBuild/testReport/
> Locally, these tests only fail with a cythonized driver installation. If the
> driver is not cythonized (installed with {{--no_extensions}}) then the tests
> are fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10799) 2 cqlshlib tests still failing with cythonized driver installation
[
https://issues.apache.org/jira/browse/CASSANDRA-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15036026#comment-15036026
]
Adam Holmberg commented on CASSANDRA-10799:
---
bq. the cqlsh driver patch for converting blobs into byte arrays is failing
with cythonized driver installations
That's exactly right. When cython extensions are present we use a type out of
cassandra.deserializers instead of the one patched by cqlsh. Unfortunately the
way its written there's no way to force it back to the pure Python version.
That wouldn't be a difficult change, but I would advocate for testing using the
bundled driver if possible. That makes the tests the right thing and avoids
surprises with installed versions.
> 2 cqlshlib tests still failing with cythonized driver installation
> --
>
> Key: CASSANDRA-10799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10799
> Project: Cassandra
> Issue Type: Test
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.2.x, 3.x
>
>
> We still have 2 cqlshlib tests failing on Jenkins:
> http://cassci.datastax.com/job/cassandra-3.0_cqlshlib/lastCompletedBuild/testReport/
> Locally, these tests only fail with a cythonized driver installation. If the
> driver is not cythonized (installed with {{--no_extensions}}) then the tests
> are fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10799) 2 cqlshlib tests still failing with cythonized driver installation
[
https://issues.apache.org/jira/browse/CASSANDRA-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15035270#comment-15035270
]
Stefania commented on CASSANDRA-10799:
--
Both failing tests are caused by problems decoding blobs:
{code}
test_cqlsh: DEBUG: read "\x1b[0;1;31mFailed to format value
'\\x00\\x01\\x02\\x03\\x04\\x05\\xff\\xfe\\xfd' : 'ascii' codec can't decode
byte 0xff in position 6: ordinal not in range(128)\x1b[0m\r\n\x1b[0;1;31mFailed
to format value '\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff' : 'ascii' codec
can't decode byte 0xff in position 0: ordinal not in
range(128)\x1b[0m\r\n\x1b[0;1;31m1 more decoding errors suppressed.\x1b[0m\r\n"
from subproc
{code}
>From past experience, I suspect that the cqlsh driver patch for converting
>blobs into byte arrays is failing with cythonized driver installations, cc
>[~aholmber] to confirm this.
If I am correct, other than running these tests with the driver not cythonized,
is there anything else that can be done?
In fact, these tests should probably use the embedded driver (which is not
cythonized) rather than the installed driver.
> 2 cqlshlib tests still failing with cythonized driver installation
> --
>
> Key: CASSANDRA-10799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10799
> Project: Cassandra
> Issue Type: Test
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.2.x, 3.0.x
>
>
> We still have 2 cqlshlib tests failing on Jenkins:
> http://cassci.datastax.com/job/cassandra-3.0_cqlshlib/lastCompletedBuild/testReport/
> Locally, these tests only fail with a cythonized driver installation. If the
> driver is not cythonized (installed with {{--no_extensions}}) then the tests
> are fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
