----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30785/ -----------------------------------------------------------
Review request for drill, Aman Sinha and Steven Phillips. Repository: drill-git Description ------- New text reader based on univocity. Adds better handling of quoted strings, escaping, comments. Also adds support for skipping first record. Diffs ----- common/src/main/java/org/apache/drill/common/config/DrillConfig.java f075cc9 contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseRecordReader.java da38707 contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java 69c50ea contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java 4b73600 exec/java-exec/pom.xml f5d9962 exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 52a3868 exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java d39040e exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java aa5b702 exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java 8e9d395 exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java f48b06d exec/java-exec/src/main/java/org/apache/drill/exec/client/PrintingResultsListener.java 1ed3cb3 exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/FunctionImplementationRegistry.java 25dcbbc exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorStats.java 0e9da0e exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/OperatorCreatorRegistry.java 82a9a63 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java 6b7294d exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectorTemplate.java a6294d8 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java 52b892e exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java f20627d exec/java-exec/src/main/java/org/apache/drill/exec/store/AbstractRecordReader.java 554b3e7 exec/java-exec/src/main/java/org/apache/drill/exec/store/LocalSyncableFileSystem.java 37b9c9d exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordReader.java 09495f5 exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/DrillFileSystem.java f5730a1 exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/easy/EasyFormatPlugin.java 431b362 exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/easy/EasyGroupScan.java 9902443 exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader.java 557c0f0 exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/TextFormatPlugin.java bf46395 exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/CompliantTextRecordReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/RepeatedVarCharOutput.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/StreamFinishedPseudoException.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/TextInput.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/TextOutput.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/TextParsingContext.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/TextParsingSettings.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/TextReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/mock/MockRecordReader.java 43e6416 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetScanBatchCreator.java ad1bf32 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java 5b9212c exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet2/DrillParquetReader.java 8778ef8 exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoRecordReader.java b7ffbf0 exec/java-exec/src/main/java/org/apache/drill/exec/store/schedule/BlockMapBuilder.java 1aac14c exec/java-exec/src/main/java/org/apache/drill/exec/store/schedule/CompleteFileWork.java 04838bd exec/java-exec/src/main/java/org/apache/drill/exec/store/text/DrillTextRecordReader.java 7c1f888 exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetRecordReaderTest.java 219e66f exec/java-exec/src/test/java/org/apache/drill/exec/store/text/TestNewTextReader.java PRE-CREATION exec/java-exec/src/test/java/org/apache/drill/exec/store/text/TestTextColumn.java e5a2c94 exec/java-exec/src/test/resources/bootstrap-storage-plugins.json e9772cf exec/java-exec/src/test/resources/store/text/data/letters.txt 14b9cb6 exec/java-exec/src/test/resources/textinput/input1.csv PRE-CREATION exec/java-exec/src/test/resources/textinput/input2.csv PRE-CREATION Diff: https://reviews.apache.org/r/30785/diff/ Testing ------- Unit test, TPCH100 Parquet and Text, expanded functional suite. Running perf regression now. Thanks, Jacques Nadeau
