Chao Sun has uploaded a new change for review. http://gerrit.cloudera.org:8080/2988
Change subject: Add complex type support for client side ...................................................................... Add complex type support for client side This adds complex type support for the client side (Java). All the implementations here conform to the Apache Arrow's standard, with the exception of: - struct type is different in that it doesn't have a "is_null" array associated with. This is due to how struct is handled in Impala, and will be changed once that is fixed. - no support for union type. All the work are included under package com.cloudera.recordservice.core.data. It is also accompanied with some some tests in TestArray class. Some of these use the functional.complextypestbl on the Impala side. The next step is to integrate this with input formats and frameworks. Change-Id: Ia2c14c85eb9eeb06ae30d7feebf89170136235df --- M api/RecordService.thrift M cpp/samples/record-service-client.cc M cpp/samples/rs-grep.cc M cpp/tests/client-test.cc M cpp/tests/test-common.h M java/avro/src/main/java/com/cloudera/recordservice/avro/GenericRecords.java M java/avro/src/main/java/com/cloudera/recordservice/avro/KeyValueRecords.java M java/avro/src/main/java/com/cloudera/recordservice/avro/RecordUtil.java M java/core/src/main/java/com/cloudera/recordservice/core/Records.java M java/core/src/main/java/com/cloudera/recordservice/core/Schema.java A java/core/src/main/java/com/cloudera/recordservice/core/data/AbstractArray.java A java/core/src/main/java/com/cloudera/recordservice/core/data/Array.java R java/core/src/main/java/com/cloudera/recordservice/core/data/ByteArray.java A java/core/src/main/java/com/cloudera/recordservice/core/data/CharArray.java R java/core/src/main/java/com/cloudera/recordservice/core/data/Decimal.java A java/core/src/main/java/com/cloudera/recordservice/core/data/ListArray.java A java/core/src/main/java/com/cloudera/recordservice/core/data/ListArrayBase.java A java/core/src/main/java/com/cloudera/recordservice/core/data/Map.java A java/core/src/main/java/com/cloudera/recordservice/core/data/MapArray.java A java/core/src/main/java/com/cloudera/recordservice/core/data/PrimitiveArray.java A java/core/src/main/java/com/cloudera/recordservice/core/data/Record.java A java/core/src/main/java/com/cloudera/recordservice/core/data/StructArray.java R java/core/src/main/java/com/cloudera/recordservice/core/data/TimestampNanos.java M java/core/src/test/java/com/cloudera/recordservice/core/SampleClient.java M java/core/src/test/java/com/cloudera/recordservice/core/TestBase.java M java/core/src/test/java/com/cloudera/recordservice/core/TestBasicClient.java M java/core/src/test/java/com/cloudera/recordservice/core/TestFaultTolerance.java M java/core/src/test/java/com/cloudera/recordservice/core/TestKerberosConnection.java M java/core/src/test/java/com/cloudera/recordservice/core/TestS3.java M java/core/src/test/java/com/cloudera/recordservice/core/TestSchema.java A java/core/src/test/java/com/cloudera/recordservice/core/data/TestComplexTypes.java R java/core/src/test/java/com/cloudera/recordservice/core/data/TestUtil.java M java/examples/src/main/java/com/cloudera/recordservice/examples/RSCat.java M java/examples/src/main/java/com/cloudera/recordservice/examples/SumQueryBenchmark.java M java/examples/src/main/java/com/cloudera/recordservice/examples/terasort/RecordServiceTeraInputFormat.java M java/hcatalog-pig-adapter/src/main/java/com/cloudera/recordservice/pig/PigHCatUtil.java M java/mr/src/main/java/com/cloudera/recordservice/mapred/TextInputFormat.java M java/mr/src/main/java/com/cloudera/recordservice/mapreduce/RecordServiceInputFormat.java M java/mr/src/main/java/com/cloudera/recordservice/mapreduce/TextInputFormat.java M java/mr/src/main/java/com/cloudera/recordservice/mr/DecimalWritable.java M java/mr/src/main/java/com/cloudera/recordservice/mr/RecordServiceRecord.java M java/mr/src/main/java/com/cloudera/recordservice/mr/TimestampNanosWritable.java M java/spark/src/main/scala/com/cloudera/recordservice/spark/RecordServiceRDD.scala M java/spark/src/main/scala/com/cloudera/recordservice/spark/RecordServiceRecordRDD.scala M java/spark/src/main/scala/com/cloudera/recordservice/spark/RecordServiceRelation.scala M java/spark/src/main/scala/com/cloudera/recordservice/spark/SchemaRecordServiceRDD.scala M java/spark/src/test/scala/com/cloudera/recordservice/spark/BasicClientSuite.scala M tests/benchmarks.py M tests/create-tbls.sql M tests/load-test-data.sh 50 files changed, 2,274 insertions(+), 490 deletions(-) git pull ssh://gerrit.cloudera.org:29418/RecordServiceClient refs/changes/88/2988/1 -- To view, visit http://gerrit.cloudera.org:8080/2988 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia2c14c85eb9eeb06ae30d7feebf89170136235df Gerrit-PatchSet: 1 Gerrit-Project: RecordServiceClient Gerrit-Branch: master_dev Gerrit-Owner: Chao Sun <[email protected]>
