----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32867/ -----------------------------------------------------------
(Updated April 13, 2015, midnight) Review request for drill, Aman Sinha and Hanifi Gunes. Changes ------- Updated patch to completely remove ok(), inform() and resetState() from ComplexWriter. This was causing problems as we were still getting failures in some situations despite the auto reallocating vectors. A few regression tests are failing and am working through those but wanted to get Hanifi to do initial review. Repository: drill-git Description ------- Add Support for large in conditions through the use of the Values operator. Update JSON reader to support reading Extended JSON. Update JSON writer to support writing extended JSON data. Update JSON reader to automatically unwrap a file that includes a single top-level array. Update Options manager to use getOption(<Type>Validator) to directly retrieve typed value. Remove JSON rewinding Diffs (updated) ----- common/src/main/java/org/apache/drill/common/JSONOptions.java 945cd92 common/src/main/java/org/apache/drill/common/logical/data/Constant.java 460803d common/src/main/java/org/apache/drill/common/logical/data/Values.java PRE-CREATION common/src/main/java/org/apache/drill/common/logical/data/visitors/AbstractLogicalVisitor.java 92e370f common/src/main/java/org/apache/drill/common/logical/data/visitors/LogicalVisitor.java 3a426bf common/src/main/java/org/apache/drill/common/types/Types.java c41727d contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java 15ef197 exec/java-exec/src/main/codegen/templates/BaseWriter.java 2ce4c3c exec/java-exec/src/main/codegen/templates/ComplexReaders.java cd69d0d exec/java-exec/src/main/codegen/templates/ComplexWriters.java 576fd83 exec/java-exec/src/main/codegen/templates/JsonOutputRecordWriter.java 1d0dc9d exec/java-exec/src/main/codegen/templates/ListWriters.java ff4c3d8 exec/java-exec/src/main/codegen/templates/MapWriters.java 4dab990 exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java bd93206 exec/java-exec/src/main/java/org/apache/drill/exec/expr/EvaluationVisitor.java 386ab79 exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/DateUtility.java a031bee exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/conv/JsonConvertFrom.java c828cf4 exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/conv/JsonConvertTo.java ccfae14 exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractPhysicalVisitor.java 1633912 exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/PhysicalVisitor.java fde27af exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/Values.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenTemplate.java c5d3d93 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/values/ValuesBatchCreator.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillOptiq.java 5dd6ed5 exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java 92a804e exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillValuesRel.java a3551e7 exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/ScanFieldDeterminer.java 59c65f9 exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/ValuesPrel.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/ValuesPrule.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/server/options/BaseOptionManager.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/server/options/FallbackOptionManager.java 45d393c exec/java-exec/src/main/java/org/apache/drill/exec/server/options/OptionManager.java 4ffe9a3 exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java 1a8559e exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java 6cf1ce5 exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader.java cc7cb83 exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JsonProcessor.java ce6017b exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JsonRecordWriter.java 76c4ace exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/RewindableUtf8Reader.java b9075de exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/reader/BaseJsonProcessor.java 509798a exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/reader/CountingJsonReader.java 1ef71e7 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet2/DrillParquetReader.java fa78213 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet2/DrillParquetRecordMaterializer.java 720e8be exec/java-exec/src/main/java/org/apache/drill/exec/vector/RepeatedVector.java 2c2ff54 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/WriteState.java 43dba65 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/BasicJsonOutput.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/DateOutputFormat.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/ExtendedJsonOutput.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/ExtendedType.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/ExtendedTypeName.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonOutput.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java 9738ff8 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonWriter.java de52b73 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/VectorOutput.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/WorkingBuffer.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/AbstractBaseWriter.java 5d85d0a exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/ComplexWriterImpl.java a110dcc exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/RepeatedListReaderImpl.java 3d0a861 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/RepeatedMapReaderImpl.java aa98818 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/SingleMapReaderImpl.java ec6009e exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/VectorContainerWriter.java c3c9354 exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/filter/TestLargeInClause.java PRE-CREATION exec/java-exec/src/test/java/org/apache/drill/exec/store/json/TestJsonRecordReader.java c4bfcce exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestExtendedTypes.java PRE-CREATION exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestRepeated.java 098c7de exec/java-exec/src/test/resources/store/json/listdoc.json PRE-CREATION exec/java-exec/src/test/resources/vector/complex/extended.json PRE-CREATION Diff: https://reviews.apache.org/r/32867/diff/ Testing ------- Unit, Regression, SF100 Thanks, Jacques Nadeau
