Hey Timothy, Can you file a JIRA for this and attach a patch? Or is there one already?
I'll get it committed forthwith. J On Tue, Jun 11, 2013 at 11:38 AM, Jacques Nadeau <[email protected]>wrote: > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/11587/ > > Ship it! > > Ship It! > > > - Jacques > > On June 11th, 2013, 6:18 a.m., Timothy Chen wrote: > Review request for drill and Jacques Nadeau. > By Timothy Chen. > > *Updated June 11, 2013, 6:18 a.m.* > Description > > Added the JSONRecordReader based on the previous ScanJson work. > Does not support nested fields, maps or lists yet. > Currently it detects to move on to the next batch when any of the field > batch cannot hold another item for the current item being written. This also > assumes the default batch size can always hold at least one item from any > field (which only is a problem for variable length vectors). > > Diffs > > - > sandbox/prototype/common/src/main/java/org/apache/drill/common/logical/data/Scan.java > (c8d396b) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/DeadBuf.java > (dafb68c) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/AbstractFixedValueVector.java > (b32f067) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/BaseValueVector.java > (b001add) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/Fixed4.java > (650029d) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableBit.java > (PRE-CREATION) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableFixed4.java > (cc18538) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableValueVector.java > (692ab87) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableVarLen4.java > (PRE-CREATION) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/TypeHelper.java > (8e89c41) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VarLen1.java > (d87029d) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VarLen2.java > (ebd440a) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VarLen4.java > (b3cd712) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VariableVector.java > (4247f14) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/InboundRpcMessage.java > (be1ff6b) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/DiffSchema.java > (016e097) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/Field.java > (e19c099) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/IdGenerator.java > (728e8e1) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/ListSchema.java > (efdc8fd) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/NamedField.java > (aa0d6aa) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/OrderedField.java > (67fd2fa) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/SchemaRecorder.java > (54a4e0e) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/JacksonHelper.java > (0643710) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/PhysicalOperator.java > (e450ee9) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/PhysicalOperatorIterator.java > (bf4053e) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/ScanJson.java > (a1c30e9) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/transform/ProtobufSchemaTransformer.java > (a81a9d9) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/transform/SchemaTransformer.java > (54d851f) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/server/DrillbitContext.java > (d5aaab2) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/BatchExceededException.java > (PRE-CREATION) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/JSONRecordReader.java > (PRE-CREATION) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordReader.java > (67c84ed) > - > sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/VectorHolder.java > (PRE-CREATION) > - > sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/JSONRecordReaderTest.java > (PRE-CREATION) > - sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_1.json > (PRE-CREATION) > - sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_2.json > (PRE-CREATION) > - sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_3.json > (PRE-CREATION) > - sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_4.json > (PRE-CREATION) > - sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_5.json > (PRE-CREATION) > - sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_6.json > (PRE-CREATION) > - sandbox/prototype/exec/ref/src/test/resources/donuts.json (2d98b59) > - sandbox/prototype/exec/ref/src/test/resources/simple_plan.json > (0f38476) > - sandbox/prototype/pom.xml (25f156d) > > View Diff <https://reviews.apache.org/r/11587/diff/> >
