[jira] [Commented] (CASSANDRA-10700) 2.1 sstableloader will fail if there are collections in the schema tables
[ https://issues.apache.org/jira/browse/CASSANDRA-10700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15058360#comment-15058360 ] T Jake Luciani commented on CASSANDRA-10700: Removed duplicate test case and that import, will commit once next test run is done > 2.1 sstableloader will fail if there are collections in the schema tables > - > > Key: CASSANDRA-10700 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10700 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Tyler Hobbs >Assignee: T Jake Luciani > Fix For: 2.1.x > > > In {{BulkLoader.ExternalClient}}, we use the Thrift {{execute_cql3_query()}} > method to read the system schema tables. Because it's a Thrift connection, > we use the v2 protocol format for serializing data. However, when we later > read the results with {{CFMetadata.fromThriftCqlRow()}}, we use the v3 > protocol format to deserialize the results. If there are any collections in > the results, such as entries in {{dropped_columns}}, the following error will > occur: > {noformat} > Caused by: java.lang.IllegalArgumentException: null > at java.nio.Buffer.limit(Buffer.java:275) ~[na:1.8.0_45-internal] > at > org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:543) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.CollectionSerializer.readValue(CollectionSerializer.java:124) > ~[cassandra-all-2.1.1 > 1.jar:2.1.11] > at > org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:101) > ~[cassandra-all- > 2.1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:30) > ~[cassandra-all-2 > .1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.CollectionSerializer.deserialize(CollectionSerializer.java:50) > ~[cassandra-all-2.1. > 11.jar:2.1.11] > at > org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:68) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.cql3.UntypedResultSet$Row.getMap(UntypedResultSet.java:287) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.config.CFMetaData.fromSchemaNoTriggers(CFMetaData.java:1833) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.config.CFMetaData.fromThriftCqlRow(CFMetaData.java:1126) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:330) > ~[cassandra-all-2.1.11.jar:na] > ... 7 common frames omitted > {noformat} > I believe this only affects 2.1 due to the re-working of > BulkLoader/sstableloader in 2.2 and 3.0, but I haven't confirmed that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10700) 2.1 sstableloader will fail if there are collections in the schema tables
[ https://issues.apache.org/jira/browse/CASSANDRA-10700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15058322#comment-15058322 ] Aleksey Yeschenko commented on CASSANDRA-10700: --- {{AbstractType}} import does seem to be redundant, but +1 to the patch, so long as cassci is happy. > 2.1 sstableloader will fail if there are collections in the schema tables > - > > Key: CASSANDRA-10700 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10700 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Tyler Hobbs >Assignee: T Jake Luciani > Fix For: 2.1.x > > > In {{BulkLoader.ExternalClient}}, we use the Thrift {{execute_cql3_query()}} > method to read the system schema tables. Because it's a Thrift connection, > we use the v2 protocol format for serializing data. However, when we later > read the results with {{CFMetadata.fromThriftCqlRow()}}, we use the v3 > protocol format to deserialize the results. If there are any collections in > the results, such as entries in {{dropped_columns}}, the following error will > occur: > {noformat} > Caused by: java.lang.IllegalArgumentException: null > at java.nio.Buffer.limit(Buffer.java:275) ~[na:1.8.0_45-internal] > at > org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:543) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.CollectionSerializer.readValue(CollectionSerializer.java:124) > ~[cassandra-all-2.1.1 > 1.jar:2.1.11] > at > org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:101) > ~[cassandra-all- > 2.1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:30) > ~[cassandra-all-2 > .1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.CollectionSerializer.deserialize(CollectionSerializer.java:50) > ~[cassandra-all-2.1. > 11.jar:2.1.11] > at > org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:68) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.cql3.UntypedResultSet$Row.getMap(UntypedResultSet.java:287) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.config.CFMetaData.fromSchemaNoTriggers(CFMetaData.java:1833) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.config.CFMetaData.fromThriftCqlRow(CFMetaData.java:1126) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:330) > ~[cassandra-all-2.1.11.jar:na] > ... 7 common frames omitted > {noformat} > I believe this only affects 2.1 due to the re-working of > BulkLoader/sstableloader in 2.2 and 3.0, but I haven't confirmed that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10700) 2.1 sstableloader will fail if there are collections in the schema tables
[ https://issues.apache.org/jira/browse/CASSANDRA-10700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056662#comment-15056662 ] T Jake Luciani commented on CASSANDRA-10700: [fix | https://github.com/tjake/cassandra/tree/externalclient] [testall | http://cassci.datastax.com/view/Dev/view/tjake/job/tjake-externalclient-testall/] [dtest|http://cassci.datastax.com/job/tjake-externalclient-dtest/] > 2.1 sstableloader will fail if there are collections in the schema tables > - > > Key: CASSANDRA-10700 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10700 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Tyler Hobbs >Assignee: T Jake Luciani > Fix For: 2.1.x > > > In {{BulkLoader.ExternalClient}}, we use the Thrift {{execute_cql3_query()}} > method to read the system schema tables. Because it's a Thrift connection, > we use the v2 protocol format for serializing data. However, when we later > read the results with {{CFMetadata.fromThriftCqlRow()}}, we use the v3 > protocol format to deserialize the results. If there are any collections in > the results, such as entries in {{dropped_columns}}, the following error will > occur: > {noformat} > Caused by: java.lang.IllegalArgumentException: null > at java.nio.Buffer.limit(Buffer.java:275) ~[na:1.8.0_45-internal] > at > org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:543) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.CollectionSerializer.readValue(CollectionSerializer.java:124) > ~[cassandra-all-2.1.1 > 1.jar:2.1.11] > at > org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:101) > ~[cassandra-all- > 2.1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:30) > ~[cassandra-all-2 > .1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.CollectionSerializer.deserialize(CollectionSerializer.java:50) > ~[cassandra-all-2.1. > 11.jar:2.1.11] > at > org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:68) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.cql3.UntypedResultSet$Row.getMap(UntypedResultSet.java:287) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.config.CFMetaData.fromSchemaNoTriggers(CFMetaData.java:1833) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.config.CFMetaData.fromThriftCqlRow(CFMetaData.java:1126) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:330) > ~[cassandra-all-2.1.11.jar:na] > ... 7 common frames omitted > {noformat} > I believe this only affects 2.1 due to the re-working of > BulkLoader/sstableloader in 2.2 and 3.0, but I haven't confirmed that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10700) 2.1 sstableloader will fail if there are collections in the schema tables
[ https://issues.apache.org/jira/browse/CASSANDRA-10700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025192#comment-15025192 ] cheng ren commented on CASSANDRA-10700: --- is 2.1 sstableloader supposed to work for loading sstables to other cassandra versions like 2.0 ? if yes, then this fix might break. "select is_dense from system.schema_columnfamilies" will fail since is_dense is not a column in 2.0. > 2.1 sstableloader will fail if there are collections in the schema tables > - > > Key: CASSANDRA-10700 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10700 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Tyler Hobbs > Fix For: 2.1.x > > > In {{BulkLoader.ExternalClient}}, we use the Thrift {{execute_cql3_query()}} > method to read the system schema tables. Because it's a Thrift connection, > we use the v2 protocol format for serializing data. However, when we later > read the results with {{CFMetadata.fromThriftCqlRow()}}, we use the v3 > protocol format to deserialize the results. If there are any collections in > the results, such as entries in {{dropped_columns}}, the following error will > occur: > {noformat} > Caused by: java.lang.IllegalArgumentException: null > at java.nio.Buffer.limit(Buffer.java:275) ~[na:1.8.0_45-internal] > at > org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:543) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.CollectionSerializer.readValue(CollectionSerializer.java:124) > ~[cassandra-all-2.1.1 > 1.jar:2.1.11] > at > org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:101) > ~[cassandra-all- > 2.1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:30) > ~[cassandra-all-2 > .1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.CollectionSerializer.deserialize(CollectionSerializer.java:50) > ~[cassandra-all-2.1. > 11.jar:2.1.11] > at > org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:68) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.cql3.UntypedResultSet$Row.getMap(UntypedResultSet.java:287) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.config.CFMetaData.fromSchemaNoTriggers(CFMetaData.java:1833) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.config.CFMetaData.fromThriftCqlRow(CFMetaData.java:1126) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:330) > ~[cassandra-all-2.1.11.jar:na] > ... 7 common frames omitted > {noformat} > I believe this only affects 2.1 due to the re-working of > BulkLoader/sstableloader in 2.2 and 3.0, but I haven't confirmed that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10700) 2.1 sstableloader will fail if there are collections in the schema tables
[ https://issues.apache.org/jira/browse/CASSANDRA-10700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15022166#comment-15022166 ] Aleksey Yeschenko commented on CASSANDRA-10700: --- The easiest fix would be replace {{SELECT * FROM system.schema_columnfamilies}} query with a fully-specified column list, and not read {{dropped_columns}} at all. That column is *not* required for {{BulkLoader}} purposes. > 2.1 sstableloader will fail if there are collections in the schema tables > - > > Key: CASSANDRA-10700 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10700 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Tyler Hobbs >Priority: Minor > Fix For: 2.1.x > > > In {{BulkLoader.ExternalClient}}, we use the Thrift {{execute_cql3_query()}} > method to read the system schema tables. Because it's a Thrift connection, > we use the v2 protocol format for serializing data. However, when we later > read the results with {{CFMetadata.fromThriftCqlRow()}}, we use the v3 > protocol format to deserialize the results. If there are any collections in > the results, such as entries in {{dropped_columns}}, the following error will > occur: > {noformat} > Caused by: java.lang.IllegalArgumentException: null > at java.nio.Buffer.limit(Buffer.java:275) ~[na:1.8.0_45-internal] > at > org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:543) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.CollectionSerializer.readValue(CollectionSerializer.java:124) > ~[cassandra-all-2.1.1 > 1.jar:2.1.11] > at > org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:101) > ~[cassandra-all- > 2.1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:30) > ~[cassandra-all-2 > .1.11.jar:2.1.11] > at > org.apache.cassandra.serializers.CollectionSerializer.deserialize(CollectionSerializer.java:50) > ~[cassandra-all-2.1. > 11.jar:2.1.11] > at > org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:68) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.cql3.UntypedResultSet$Row.getMap(UntypedResultSet.java:287) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.config.CFMetaData.fromSchemaNoTriggers(CFMetaData.java:1833) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.config.CFMetaData.fromThriftCqlRow(CFMetaData.java:1126) > ~[cassandra-all-2.1.11.jar:2.1.11] > at > org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:330) > ~[cassandra-all-2.1.11.jar:na] > ... 7 common frames omitted > {noformat} > I believe this only affects 2.1 due to the re-working of > BulkLoader/sstableloader in 2.2 and 3.0, but I haven't confirmed that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)