[jira] [Commented] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093955#comment-14093955
]
Mike Adamson commented on CASSANDRA-7726:
-
So having tested this there are a couple of changes needed to make it work
properly. I've attached a new version of the patch that works. Can this be
committed? or does it need a new jira?
Give CRR a default input_cql Statement
--
Key: CASSANDRA-7726
URL: https://issues.apache.org/jira/browse/CASSANDRA-7726
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Reporter: Russell Alexander Spitzer
Assignee: Mike Adamson
Fix For: 2.0.10, 2.1.0
Attachments: 7726.txt
Inorder to ease migration from CqlPagingRecordReader to CqlRecordReader, it
would be helpful if CRR input_cql defaulted to a select statement that would
mirror the behavior of CPRR.
For example for a give table with partition key `((x,y,z),c1,c2)`
It would automatically generate
{code}
input_cql = SELECT * FROM ks.tab WHERE token(x,y,z) ? AND token (x,y,z) =
?
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093974#comment-14093974
]
Mike Adamson commented on CASSANDRA-7726:
-
I have attached a new patch to fix the problems found during testing 7726-2.txt
Give CRR a default input_cql Statement
--
Key: CASSANDRA-7726
URL: https://issues.apache.org/jira/browse/CASSANDRA-7726
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Reporter: Russell Alexander Spitzer
Assignee: Mike Adamson
Fix For: 2.0.10, 2.1.0
Attachments: 7726-2.txt, 7726.txt
Inorder to ease migration from CqlPagingRecordReader to CqlRecordReader, it
would be helpful if CRR input_cql defaulted to a select statement that would
mirror the behavior of CPRR.
For example for a give table with partition key `((x,y,z),c1,c2)`
It would automatically generate
{code}
input_cql = SELECT * FROM ks.tab WHERE token(x,y,z) ? AND token (x,y,z) =
?
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14094114#comment-14094114
]
Brandon Williams commented on CASSANDRA-7726:
-
bq. Reopening as we found further issues during testing
Can we add a test for the original issue and the problems discovered?
Give CRR a default input_cql Statement
--
Key: CASSANDRA-7726
URL: https://issues.apache.org/jira/browse/CASSANDRA-7726
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Reporter: Russell Alexander Spitzer
Assignee: Mike Adamson
Fix For: 2.0.10, 2.1.0
Attachments: 7726-2.txt, 7726.txt
Inorder to ease migration from CqlPagingRecordReader to CqlRecordReader, it
would be helpful if CRR input_cql defaulted to a select statement that would
mirror the behavior of CPRR.
For example for a give table with partition key `((x,y,z),c1,c2)`
It would automatically generate
{code}
input_cql = SELECT * FROM ks.tab WHERE token(x,y,z) ? AND token (x,y,z) =
?
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14094344#comment-14094344
]
Mike Adamson commented on CASSANDRA-7726:
-
Attached a new version of 7726-2.txt with pig test for CqlRecordReader
Give CRR a default input_cql Statement
--
Key: CASSANDRA-7726
URL: https://issues.apache.org/jira/browse/CASSANDRA-7726
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Reporter: Russell Alexander Spitzer
Assignee: Mike Adamson
Fix For: 2.0.10, 2.1.0
Attachments: 7726-2.txt, 7726.txt
Inorder to ease migration from CqlPagingRecordReader to CqlRecordReader, it
would be helpful if CRR input_cql defaulted to a select statement that would
mirror the behavior of CPRR.
For example for a give table with partition key `((x,y,z),c1,c2)`
It would automatically generate
{code}
input_cql = SELECT * FROM ks.tab WHERE token(x,y,z) ? AND token (x,y,z) =
?
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14094927#comment-14094927
]
Russell Alexander Spitzer commented on CASSANDRA-7726:
--
I'm getting some errors with the following tables
{code}
CREATE TABLE mixedCase (
KeY text,
Field1 int,
field2 int,
FielD3 int,
field4 int,
field5 int,
PRIMARY KEY ((KeY, FielD3), field5));
CREATE TABLE mixedCaseCompact (
KeY text,
Field1 int,
field2 int,
FielD3 int,
field4 int,
field5 int,
PRIMARY KEY ((KeY, FielD3), field5))
WITH COMPACT STORAGE;
{code}
{code}
mixedCase = LOAD 'cql://cnspig/mixedCase' USING CqlNativeStorage();
mixedCaseCompact = LOAD 'cql://cnspig/mixedCaseCompact' USING
CqlNativeStorage();
dump mixedCase;
dump mixedCaseCompact;
{code}
* mixedCase errors out when dumping with the following *
{code}
Caused by: java.io.IOException: Task process exit with nonzero status of 65.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
at org.apache.pig.PigServer.openIterator(PigServer.java:856)
at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:683)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:490)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2997:
Unable to recreate exception from backed error: java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 65.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:217)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:149)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:383)
at org.apache.pig.PigServer.launchPlan(PigServer.java:1279)
at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1264)
at org.apache.pig.PigServer.storeEx(PigServer.java:961)
at org.apache.pig.PigServer.store(PigServer.java:928)
at org.apache.pig.PigServer.openIterator(PigServer.java:841)
... 12 more
{code}
* mixedCaseCompact fails when attempting to load the table *
{code}
ERROR 2245:
line 11, column 19 Cannot get schema from loadFunc
org.apache.cassandra.hadoop.pig.CqlNativeStorage
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2245:
line 11, column 19 Cannot get schema from loadFunc
org.apache.cassandra.hadoop.pig.CqlNativeStorage
at
org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:155)
at
org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110)
at
org.apache.pig.newplan.logical.visitor.LineageFindRelVisitor.visit(LineageFindRelVisitor.java:100)
at
org.apache.pig.newplan.logical.relational.LOLoad.accept(LOLoad.java:219)
at
org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
at
org.apache.pig.newplan.logical.visitor.CastLineageSetter.init(CastLineageSetter.java:57)
at org.apache.pig.PigServer$Graph.compile(PigServer.java:1644)
at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1575)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1547)
at org.apache.pig.PigServer.registerQuery(PigServer.java:549)
at
org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:971)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:490)
[jira] [Commented] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14094930#comment-14094930
]
Russell Alexander Spitzer commented on CASSANDRA-7726:
--
I'll write some test cases for C* tomorrow
Give CRR a default input_cql Statement
--
Key: CASSANDRA-7726
URL: https://issues.apache.org/jira/browse/CASSANDRA-7726
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Reporter: Russell Alexander Spitzer
Assignee: Mike Adamson
Fix For: 2.0.10, 2.1.0
Attachments: 7726-2.txt, 7726-3.txt, 7726.txt
Inorder to ease migration from CqlPagingRecordReader to CqlRecordReader, it
would be helpful if CRR input_cql defaulted to a select statement that would
mirror the behavior of CPRR.
For example for a give table with partition key `((x,y,z),c1,c2)`
It would automatically generate
{code}
input_cql = SELECT * FROM ks.tab WHERE token(x,y,z) ? AND token (x,y,z) =
?
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14094968#comment-14094968
]
Jeremiah Jordan commented on CASSANDRA-7726:
[~brandon.williams] looks like you missed the new file CqlRecordReaderTest.java
when committing
Give CRR a default input_cql Statement
--
Key: CASSANDRA-7726
URL: https://issues.apache.org/jira/browse/CASSANDRA-7726
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Reporter: Russell Alexander Spitzer
Assignee: Mike Adamson
Fix For: 2.0.10, 2.1.0
Attachments: 7726-2.txt, 7726-3.txt, 7726.txt
Inorder to ease migration from CqlPagingRecordReader to CqlRecordReader, it
would be helpful if CRR input_cql defaulted to a select statement that would
mirror the behavior of CPRR.
For example for a give table with partition key `((x,y,z),c1,c2)`
It would automatically generate
{code}
input_cql = SELECT * FROM ks.tab WHERE token(x,y,z) ? AND token (x,y,z) =
?
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092628#comment-14092628
]
Mike Adamson commented on CASSANDRA-7726:
-
So why are we letting the user provide a select statement at all? Why not go
back to what the CPRR did and accept rows and additional where clause as
parameters not a cql statement.
Give CRR a default input_cql Statement
--
Key: CASSANDRA-7726
URL: https://issues.apache.org/jira/browse/CASSANDRA-7726
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Reporter: Russell Alexander Spitzer
Assignee: Mike Adamson
Fix For: 2.0.10, 2.1.0
Attachments: 7726.txt
Inorder to ease migration from CqlPagingRecordReader to CqlRecordReader, it
would be helpful if CRR input_cql defaulted to a select statement that would
mirror the behavior of CPRR.
For example for a give table with partition key `((x,y,z),c1,c2)`
It would automatically generate
{code}
input_cql = SELECT * FROM ks.tab WHERE token(x,y,z) ? AND token (x,y,z) =
?
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092724#comment-14092724
]
Jeremiah Jordan commented on CASSANDRA-7726:
SGTM. CRR has been out for a while. So we should probably leave the ability to
provide the whole statement and let you either provide the whole thing (and you
are responsible for the token stuff) or use it like CPRR.
Give CRR a default input_cql Statement
--
Key: CASSANDRA-7726
URL: https://issues.apache.org/jira/browse/CASSANDRA-7726
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Reporter: Russell Alexander Spitzer
Assignee: Mike Adamson
Fix For: 2.0.10, 2.1.0
Attachments: 7726.txt
Inorder to ease migration from CqlPagingRecordReader to CqlRecordReader, it
would be helpful if CRR input_cql defaulted to a select statement that would
mirror the behavior of CPRR.
For example for a give table with partition key `((x,y,z),c1,c2)`
It would automatically generate
{code}
input_cql = SELECT * FROM ks.tab WHERE token(x,y,z) ? AND token (x,y,z) =
?
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092794#comment-14092794
]
Mike Adamson commented on CASSANDRA-7726:
-
So does this negate CASSANDRA-7725 then? As in, do we still need to validate a
customer supplied query?
Give CRR a default input_cql Statement
--
Key: CASSANDRA-7726
URL: https://issues.apache.org/jira/browse/CASSANDRA-7726
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Reporter: Russell Alexander Spitzer
Assignee: Mike Adamson
Fix For: 2.0.10, 2.1.0
Attachments: 7726.txt
Inorder to ease migration from CqlPagingRecordReader to CqlRecordReader, it
would be helpful if CRR input_cql defaulted to a select statement that would
mirror the behavior of CPRR.
For example for a give table with partition key `((x,y,z),c1,c2)`
It would automatically generate
{code}
input_cql = SELECT * FROM ks.tab WHERE token(x,y,z) ? AND token (x,y,z) =
?
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092822#comment-14092822
]
Jeremiah Jordan commented on CASSANDRA-7726:
If we provide the safe way, then we probably don't need to. We can close it as
a dup of this.
Give CRR a default input_cql Statement
--
Key: CASSANDRA-7726
URL: https://issues.apache.org/jira/browse/CASSANDRA-7726
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Reporter: Russell Alexander Spitzer
Assignee: Mike Adamson
Fix For: 2.0.10, 2.1.0
Attachments: 7726.txt
Inorder to ease migration from CqlPagingRecordReader to CqlRecordReader, it
would be helpful if CRR input_cql defaulted to a select statement that would
mirror the behavior of CPRR.
For example for a give table with partition key `((x,y,z),c1,c2)`
It would automatically generate
{code}
input_cql = SELECT * FROM ks.tab WHERE token(x,y,z) ? AND token (x,y,z) =
?
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093031#comment-14093031
]
Mike Adamson commented on CASSANDRA-7726:
-
I have included changed to CqlNativeStorage in this patch to use the new options
Give CRR a default input_cql Statement
--
Key: CASSANDRA-7726
URL: https://issues.apache.org/jira/browse/CASSANDRA-7726
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Reporter: Russell Alexander Spitzer
Assignee: Mike Adamson
Fix For: 2.0.10, 2.1.0
Attachments: 7726.txt
Inorder to ease migration from CqlPagingRecordReader to CqlRecordReader, it
would be helpful if CRR input_cql defaulted to a select statement that would
mirror the behavior of CPRR.
For example for a give table with partition key `((x,y,z),c1,c2)`
It would automatically generate
{code}
input_cql = SELECT * FROM ks.tab WHERE token(x,y,z) ? AND token (x,y,z) =
?
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093341#comment-14093341
]
Jeremiah Jordan commented on CASSANDRA-7726:
One nit on this. We should probably complain if they specify both inputcql and
where/columns
Give CRR a default input_cql Statement
--
Key: CASSANDRA-7726
URL: https://issues.apache.org/jira/browse/CASSANDRA-7726
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Reporter: Russell Alexander Spitzer
Assignee: Mike Adamson
Fix For: 2.0.10, 2.1.0
Attachments: 7726.txt
Inorder to ease migration from CqlPagingRecordReader to CqlRecordReader, it
would be helpful if CRR input_cql defaulted to a select statement that would
mirror the behavior of CPRR.
For example for a give table with partition key `((x,y,z),c1,c2)`
It would automatically generate
{code}
input_cql = SELECT * FROM ks.tab WHERE token(x,y,z) ? AND token (x,y,z) =
?
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091218#comment-14091218
]
Jeremiah Jordan commented on CASSANDRA-7726:
We should also append the token stuff to the users query if they don't put it
there. As without the token stuff the query is not going to do what the user
expects.
Give CRR a default input_cql Statement
--
Key: CASSANDRA-7726
URL: https://issues.apache.org/jira/browse/CASSANDRA-7726
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Reporter: Russell Alexander Spitzer
Assignee: Mike Adamson
Attachments: 7726.txt
Inorder to ease migration from CqlPagingRecordReader to CqlRecordReader, it
would be helpful if CRR input_cql defaulted to a select statement that would
mirror the behavior of CPRR.
For example for a give table with partition key `((x,y,z),c1,c2)`
It would automatically generate
{code}
input_cql = SELECT * FROM ks.tab WHERE token(x,y,z) ? AND token (x,y,z) =
?
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
