[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2792 ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228386311
--- Diff: store/CSDK/src/CarbonReader.h ---
@@ -35,6 +40,13 @@ class CarbonReader {
*/
jobject carbonReaderObject;
+public:
+
+/**
+ * jni env
+ */
+JNIEnv *jniEnv;
+
/**
* create a CarbonReaderBuilder object for building carbonReader,
* CarbonReaderBuilder object can configure different parameter
--- End diff --
ok, done
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228192785
--- Diff: store/CSDK/src/CarbonReader.cpp ---
@@ -79,21 +78,19 @@ jobject CarbonReader::build() {
jmethodID buildID = jniEnv->GetMethodID(carbonReaderBuilderClass,
"build",
"()Lorg/apache/carbondata/sdk/file/CarbonReader;");
carbonReaderObject =
jniEnv->CallObjectMethod(carbonReaderBuilderObject, buildID);
+jclass carbonReader = jniEnv->GetObjectClass(carbonReaderObject);
+hasNextID = jniEnv->GetMethodID(carbonReader, "hasNext", "()Z");
+readNextRowID = jniEnv->GetMethodID(carbonReader, "readNextRow",
+"()Ljava/lang/Object;");
return carbonReaderObject;
}
jboolean CarbonReader::hasNext() {
-jclass carbonReader = jniEnv->GetObjectClass(carbonReaderObject);
-jmethodID hasNextID = jniEnv->GetMethodID(carbonReader, "hasNext",
"()Z");
-unsigned char hasNext = jniEnv->CallBooleanMethod(carbonReaderObject,
hasNextID);
-return hasNext;
+return jniEnv->CallBooleanMethod(carbonReaderObject, hasNextID);
}
-jobjectArray CarbonReader::readNextRow() {
-jclass carbonReader = jniEnv->GetObjectClass(carbonReaderObject);
-jmethodID readNextRow2ID = jniEnv->GetMethodID(carbonReader,
"readNextStringRow", "()[Ljava/lang/Object;");
-jobjectArray row = (jobjectArray)
jniEnv->CallObjectMethod(carbonReaderObject, readNextRow2ID);
-return row;
+jobject CarbonReader::readNextRow() {
+return (jobject) jniEnv->CallObjectMethod(carbonReaderObject,
readNextRowID);
}
jboolean CarbonReader::close() {
--- End diff --
ok, done
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228192284
--- Diff: store/CSDK/src/CarbonReader.cpp ---
@@ -79,21 +78,19 @@ jobject CarbonReader::build() {
jmethodID buildID = jniEnv->GetMethodID(carbonReaderBuilderClass,
"build",
"()Lorg/apache/carbondata/sdk/file/CarbonReader;");
carbonReaderObject =
jniEnv->CallObjectMethod(carbonReaderBuilderObject, buildID);
+jclass carbonReader = jniEnv->GetObjectClass(carbonReaderObject);
+hasNextID = jniEnv->GetMethodID(carbonReader, "hasNext", "()Z");
--- End diff --
ok, done
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228189848
--- Diff: store/CSDK/src/CarbonReader.cpp ---
@@ -79,21 +78,19 @@ jobject CarbonReader::build() {
jmethodID buildID = jniEnv->GetMethodID(carbonReaderBuilderClass,
"build",
"()Lorg/apache/carbondata/sdk/file/CarbonReader;");
carbonReaderObject =
jniEnv->CallObjectMethod(carbonReaderBuilderObject, buildID);
+jclass carbonReader = jniEnv->GetObjectClass(carbonReaderObject);
+hasNextID = jniEnv->GetMethodID(carbonReader, "hasNext", "()Z");
--- End diff --
hasNextId is for method of class, only need one times.
If move to haCarbonReader::hasNext(), it will increase read carbon file
time
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228188421
--- Diff: store/CSDK/src/CarbonReader.h ---
@@ -35,6 +40,13 @@ class CarbonReader {
*/
jobject carbonReaderObject;
+public:
+
+/**
+ * jni env
+ */
+JNIEnv *jniEnv;
+
/**
* create a CarbonReaderBuilder object for building carbonReader,
* CarbonReaderBuilder object can configure different parameter
--- End diff --
ok, build
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228188056
--- Diff: store/CSDK/src/CarbonReader.cpp ---
@@ -19,15 +19,14 @@
#include
jobject CarbonReader::builder(JNIEnv *env, char *path, char *tableName) {
-
jniEnv = env;
jclass carbonReaderClass =
env->FindClass("org/apache/carbondata/sdk/file/CarbonReader");
jmethodID carbonReaderBuilderID =
env->GetStaticMethodID(carbonReaderClass, "builder",
"(Ljava/lang/String;Ljava/lang/String;)Lorg/apache/carbondata/sdk/file/CarbonReaderBuilder;");
-jstring jpath = env->NewStringUTF(path);
+jstring jPath = env->NewStringUTF(path);
jstring jtableName = env->NewStringUTF(tableName);
jvalue args[2];
-args[0].l = jpath;
+args[0].l = jPath;
args[1].l = jtableName;
carbonReaderBuilderObject =
env->CallStaticObjectMethodA(carbonReaderClass, carbonReaderBuilderID, args);
return carbonReaderBuilderObject;
--- End diff --
ok
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228186934
--- Diff: store/CSDK/CarbonReader.cpp ---
@@ -19,15 +19,14 @@
#include
jobject CarbonReader::builder(JNIEnv *env, char *path, char *tableName) {
-
jniEnv = env;
jclass carbonReaderClass =
env->FindClass("org/apache/carbondata/sdk/file/CarbonReader");
jmethodID carbonReaderBuilderID =
env->GetStaticMethodID(carbonReaderClass, "builder",
--- End diff --
I throw exception in build and handle it in main test case.
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228187015
--- Diff: store/CSDK/CarbonReader.cpp ---
@@ -38,9 +37,9 @@ jobject CarbonReader::builder(JNIEnv *env, char *path) {
jclass carbonReaderClass =
env->FindClass("org/apache/carbondata/sdk/file/CarbonReader");
--- End diff --
added
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228130454
--- Diff: store/CSDK/src/CarbonReader.cpp ---
@@ -79,21 +78,19 @@ jobject CarbonReader::build() {
jmethodID buildID = jniEnv->GetMethodID(carbonReaderBuilderClass,
"build",
"()Lorg/apache/carbondata/sdk/file/CarbonReader;");
carbonReaderObject =
jniEnv->CallObjectMethod(carbonReaderBuilderObject, buildID);
+jclass carbonReader = jniEnv->GetObjectClass(carbonReaderObject);
+hasNextID = jniEnv->GetMethodID(carbonReader, "hasNext", "()Z");
+readNextRowID = jniEnv->GetMethodID(carbonReader, "readNextRow",
+"()Ljava/lang/Object;");
return carbonReaderObject;
}
jboolean CarbonReader::hasNext() {
-jclass carbonReader = jniEnv->GetObjectClass(carbonReaderObject);
-jmethodID hasNextID = jniEnv->GetMethodID(carbonReader, "hasNext",
"()Z");
-unsigned char hasNext = jniEnv->CallBooleanMethod(carbonReaderObject,
hasNextID);
-return hasNext;
+return jniEnv->CallBooleanMethod(carbonReaderObject, hasNextID);
}
-jobjectArray CarbonReader::readNextRow() {
-jclass carbonReader = jniEnv->GetObjectClass(carbonReaderObject);
-jmethodID readNextRow2ID = jniEnv->GetMethodID(carbonReader,
"readNextStringRow", "()[Ljava/lang/Object;");
-jobjectArray row = (jobjectArray)
jniEnv->CallObjectMethod(carbonReaderObject, readNextRow2ID);
-return row;
+jobject CarbonReader::readNextRow() {
+return (jobject) jniEnv->CallObjectMethod(carbonReaderObject,
readNextRowID);
}
jboolean CarbonReader::close() {
--- End diff --
call CallObjectMethod() instead of boolean as return of close is always
void(), update header also
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228128521
--- Diff: store/CSDK/src/CarbonReader.cpp ---
@@ -79,21 +78,19 @@ jobject CarbonReader::build() {
jmethodID buildID = jniEnv->GetMethodID(carbonReaderBuilderClass,
"build",
"()Lorg/apache/carbondata/sdk/file/CarbonReader;");
carbonReaderObject =
jniEnv->CallObjectMethod(carbonReaderBuilderObject, buildID);
+jclass carbonReader = jniEnv->GetObjectClass(carbonReaderObject);
+hasNextID = jniEnv->GetMethodID(carbonReader, "hasNext", "()Z");
--- End diff --
please move this hasNextID to CarbonReader::hasNext().
if not null then only assign (only one time assignment)
Also same comment for readNextRowID.
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228126086
--- Diff: store/CSDK/src/CarbonReader.h ---
@@ -35,6 +40,13 @@ class CarbonReader {
*/
jobject carbonReaderObject;
+public:
+
+/**
+ * jni env
+ */
+JNIEnv *jniEnv;
+
/**
* create a CarbonReaderBuilder object for building carbonReader,
* CarbonReaderBuilder object can configure different parameter
--- End diff --
builder, projection, withHaaddopConf -- No need to have return value. They
are not used. we can return void
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228124647
--- Diff: store/CSDK/src/CarbonReader.cpp ---
@@ -19,15 +19,14 @@
#include
jobject CarbonReader::builder(JNIEnv *env, char *path, char *tableName) {
-
jniEnv = env;
jclass carbonReaderClass =
env->FindClass("org/apache/carbondata/sdk/file/CarbonReader");
jmethodID carbonReaderBuilderID =
env->GetStaticMethodID(carbonReaderClass, "builder",
"(Ljava/lang/String;Ljava/lang/String;)Lorg/apache/carbondata/sdk/file/CarbonReaderBuilder;");
-jstring jpath = env->NewStringUTF(path);
+jstring jPath = env->NewStringUTF(path);
jstring jtableName = env->NewStringUTF(tableName);
jvalue args[2];
-args[0].l = jpath;
+args[0].l = jPath;
args[1].l = jtableName;
carbonReaderBuilderObject =
env->CallStaticObjectMethodA(carbonReaderClass, carbonReaderBuilderID, args);
return carbonReaderBuilderObject;
--- End diff --
No need to return object because we are storing this variable. make it void.
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228074939
--- Diff: store/CSDK/CarbonReader.cpp ---
@@ -19,15 +19,14 @@
#include
jobject CarbonReader::builder(JNIEnv *env, char *path, char *tableName) {
-
jniEnv = env;
jclass carbonReaderClass =
env->FindClass("org/apache/carbondata/sdk/file/CarbonReader");
jmethodID carbonReaderBuilderID =
env->GetStaticMethodID(carbonReaderClass, "builder",
--- End diff --
No please handle at once. Because these are same feature. suggest we fix it
and make it clean. we should not merge PR which has issues as if some user
uses it, he will get jvm crash
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228073382
--- Diff: docs/csdk-guide.md ---
@@ -30,106 +30,13 @@ code and without CarbonSession.
In the carbon jars package, there exist a carbondata-sdk.jar,
including SDK reader for CSDK.
## Quick example
-```
-// 1. init JVM
-JavaVM *jvm;
-JNIEnv *initJVM() {
-JNIEnv *env;
-JavaVMInitArgs vm_args;
-int parNum = 3;
-int res;
-JavaVMOption options[parNum];
-
-options[0].optionString = "-Djava.compiler=NONE";
-options[1].optionString =
"-Djava.class.path=../../sdk/target/carbondata-sdk.jar";
-options[2].optionString = "-verbose:jni";
-vm_args.version = JNI_VERSION_1_8;
-vm_args.nOptions = parNum;
-vm_args.options = options;
-vm_args.ignoreUnrecognized = JNI_FALSE;
-
-res = JNI_CreateJavaVM(, (void **) , _args);
-if (res < 0) {
-fprintf(stderr, "\nCan't create Java VM\n");
-exit(1);
-}
-
-return env;
-}
-
-// 2. create carbon reader and read data
-// 2.1 read data from local disk
-/**
- * test read data from local disk, without projection
- *
- * @param env jni env
- * @return
- */
-bool readFromLocalWithoutProjection(JNIEnv *env) {
-
-CarbonReader carbonReaderClass;
-carbonReaderClass.builder(env, "../resources/carbondata", "test");
-carbonReaderClass.build();
-
-while (carbonReaderClass.hasNext()) {
-jobjectArray row = carbonReaderClass.readNextRow();
-jsize length = env->GetArrayLength(row);
-int j = 0;
-for (j = 0; j < length; j++) {
-jobject element = env->GetObjectArrayElement(row, j);
-char *str = (char *) env->GetStringUTFChars((jstring) element,
JNI_FALSE);
-printf("%s\t", str);
-}
-printf("\n");
-}
-carbonReaderClass.close();
-}
-
-// 2.2 read data from S3
-
-/**
- * read data from S3
- * parameter is ak sk endpoint
- *
- * @param env jni env
- * @param argv argument vector
- * @return
- */
-bool readFromS3(JNIEnv *env, char *argv[]) {
-CarbonReader reader;
-
-char *args[3];
-// "your access key"
-args[0] = argv[1];
-// "your secret key"
-args[1] = argv[2];
-// "your endPoint"
-args[2] = argv[3];
-
-reader.builder(env, "s3a://sdk/WriterOutput", "test");
-reader.withHadoopConf(3, args);
-reader.build();
-printf("\nRead data from S3:\n");
-while (reader.hasNext()) {
-jobjectArray row = reader.readNextRow();
-jsize length = env->GetArrayLength(row);
-
-int j = 0;
-for (j = 0; j < length; j++) {
-jobject element = env->GetObjectArrayElement(row, j);
-char *str = (char *) env->GetStringUTFChars((jstring) element,
JNI_FALSE);
-printf("%s\t", str);
-}
-printf("\n");
-}
-
-reader.close();
-}
-
-// 3. destory JVM
-(jvm)->DestroyJavaVM();
-```
-Find example code at main.cpp of CSDK module
+
+Please find example code at main.cpp of CSDK module
--- End diff --
ok, done
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r228072138 --- Diff: store/CSDK/main.cpp --- @@ -21,6 +21,7 @@ #include --- End diff -- OKï¼done ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228067882
--- Diff:
store/sdk/src/test/java/org/apache/carbondata/sdk/file/CarbonReaderTest.java ---
@@ -1522,4 +1522,208 @@ public boolean accept(File dir, String name) {
e.printStackTrace();
}
}
+
+ @Test
+ public void testReadNextRowWithRowUtil() {
--- End diff --
If has cast type method, we can remove
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228067726
--- Diff: store/CSDK/CarbonReader.cpp ---
@@ -19,15 +19,14 @@
#include
jobject CarbonReader::builder(JNIEnv *env, char *path, char *tableName) {
-
jniEnv = env;
jclass carbonReaderClass =
env->FindClass("org/apache/carbondata/sdk/file/CarbonReader");
jmethodID carbonReaderBuilderID =
env->GetStaticMethodID(carbonReaderClass, "builder",
--- End diff --
I planed to handle exception in the future before. Can we raise a new PR
for exception?
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228062513
--- Diff: store/CSDK/CarbonRow.cpp ---
@@ -0,0 +1,129 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include "CarbonRow.h"
+
+CarbonRow::CarbonRow(JNIEnv *env) {
+this->rowUtilClass =
env->FindClass("org/apache/carbondata/sdk/file/RowUtil");
+this->jniEnv = env;
+getShortId = jniEnv->GetStaticMethodID(rowUtilClass, "getShort",
+"([Ljava/lang/Object;I)S");
+getIntId = jniEnv->GetStaticMethodID(rowUtilClass, "getInt",
+"([Ljava/lang/Object;I)I");
+getLongId = jniEnv->GetStaticMethodID(rowUtilClass, "getLong",
+"([Ljava/lang/Object;I)J");
+getDoubleId = jniEnv->GetStaticMethodID(rowUtilClass, "getDouble",
+"([Ljava/lang/Object;I)D");
+getFloatId = jniEnv->GetStaticMethodID(rowUtilClass, "getFloat",
+"([Ljava/lang/Object;I)F");
+getBooleanId = jniEnv->GetStaticMethodID(rowUtilClass, "getBoolean",
+"([Ljava/lang/Object;I)Z");
+getStringId = jniEnv->GetStaticMethodID(rowUtilClass, "getString",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getDecimalId = jniEnv->GetStaticMethodID(rowUtilClass, "getDecimal",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getVarcharId = jniEnv->GetStaticMethodID(rowUtilClass, "getVarchar",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getArrayId = jniEnv->GetStaticMethodID(rowUtilClass, "getArray",
+"([Ljava/lang/Object;I)[Ljava/lang/Object;");
+}
+
+void CarbonRow::setCarbonRow(jobject data) {
+this->carbonRow = data;
+}
+
+short CarbonRow::getShort(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticShortMethodA(rowUtilClass, getShortId, args);
--- End diff --
can't cast like this, I tried before
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228061918
--- Diff: store/CSDK/CarbonRow.cpp ---
@@ -0,0 +1,129 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include "CarbonRow.h"
+
+CarbonRow::CarbonRow(JNIEnv *env) {
+this->rowUtilClass =
env->FindClass("org/apache/carbondata/sdk/file/RowUtil");
+this->jniEnv = env;
+getShortId = jniEnv->GetStaticMethodID(rowUtilClass, "getShort",
+"([Ljava/lang/Object;I)S");
+getIntId = jniEnv->GetStaticMethodID(rowUtilClass, "getInt",
+"([Ljava/lang/Object;I)I");
+getLongId = jniEnv->GetStaticMethodID(rowUtilClass, "getLong",
+"([Ljava/lang/Object;I)J");
+getDoubleId = jniEnv->GetStaticMethodID(rowUtilClass, "getDouble",
+"([Ljava/lang/Object;I)D");
+getFloatId = jniEnv->GetStaticMethodID(rowUtilClass, "getFloat",
+"([Ljava/lang/Object;I)F");
+getBooleanId = jniEnv->GetStaticMethodID(rowUtilClass, "getBoolean",
+"([Ljava/lang/Object;I)Z");
+getStringId = jniEnv->GetStaticMethodID(rowUtilClass, "getString",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getDecimalId = jniEnv->GetStaticMethodID(rowUtilClass, "getDecimal",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getVarcharId = jniEnv->GetStaticMethodID(rowUtilClass, "getVarchar",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getArrayId = jniEnv->GetStaticMethodID(rowUtilClass, "getArray",
+"([Ljava/lang/Object;I)[Ljava/lang/Object;");
+}
+
+void CarbonRow::setCarbonRow(jobject data) {
+this->carbonRow = data;
+}
+
+short CarbonRow::getShort(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticShortMethodA(rowUtilClass, getShortId, args);
+}
+
+int CarbonRow::getInt(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticIntMethodA(rowUtilClass, getIntId, args);
+}
+
+long CarbonRow::getLong(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticLongMethodA(rowUtilClass, getLongId, args);
+}
+
+double CarbonRow::getDouble(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticDoubleMethodA(rowUtilClass, getDoubleId,
args);
+}
+
+
+float CarbonRow::getFloat(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticFloatMethodA(rowUtilClass, getFloatId, args);
+}
+
+jboolean CarbonRow::getBoolean(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticBooleanMethodA(rowUtilClass, getBooleanId,
args);
--- End diff --
can't, I tried before.
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228056225
--- Diff: store/CSDK/CarbonRow.cpp ---
@@ -0,0 +1,129 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include "CarbonRow.h"
+
+CarbonRow::CarbonRow(JNIEnv *env) {
+this->rowUtilClass =
env->FindClass("org/apache/carbondata/sdk/file/RowUtil");
+this->jniEnv = env;
+getShortId = jniEnv->GetStaticMethodID(rowUtilClass, "getShort",
+"([Ljava/lang/Object;I)S");
+getIntId = jniEnv->GetStaticMethodID(rowUtilClass, "getInt",
+"([Ljava/lang/Object;I)I");
+getLongId = jniEnv->GetStaticMethodID(rowUtilClass, "getLong",
+"([Ljava/lang/Object;I)J");
+getDoubleId = jniEnv->GetStaticMethodID(rowUtilClass, "getDouble",
+"([Ljava/lang/Object;I)D");
+getFloatId = jniEnv->GetStaticMethodID(rowUtilClass, "getFloat",
+"([Ljava/lang/Object;I)F");
+getBooleanId = jniEnv->GetStaticMethodID(rowUtilClass, "getBoolean",
+"([Ljava/lang/Object;I)Z");
+getStringId = jniEnv->GetStaticMethodID(rowUtilClass, "getString",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getDecimalId = jniEnv->GetStaticMethodID(rowUtilClass, "getDecimal",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getVarcharId = jniEnv->GetStaticMethodID(rowUtilClass, "getVarchar",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getArrayId = jniEnv->GetStaticMethodID(rowUtilClass, "getArray",
+"([Ljava/lang/Object;I)[Ljava/lang/Object;");
+}
+
+void CarbonRow::setCarbonRow(jobject data) {
+this->carbonRow = data;
+}
+
+short CarbonRow::getShort(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticShortMethodA(rowUtilClass, getShortId, args);
+}
+
+int CarbonRow::getInt(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticIntMethodA(rowUtilClass, getIntId, args);
+}
+
+long CarbonRow::getLong(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticLongMethodA(rowUtilClass, getLongId, args);
+}
+
+double CarbonRow::getDouble(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticDoubleMethodA(rowUtilClass, getDoubleId,
args);
+}
+
+
+float CarbonRow::getFloat(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticFloatMethodA(rowUtilClass, getFloatId, args);
+}
+
+jboolean CarbonRow::getBoolean(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticBooleanMethodA(rowUtilClass, getBooleanId,
args);
--- End diff --
(jboolean)carobnRow[ordinal]
just we can type cast like above ?
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r228055673 --- Diff: store/CSDK/main.cpp --- @@ -21,6 +21,7 @@ #include --- End diff -- we just have to create a folder and move this file to that folder. other files we keep in source folder. update cmakelist with proper path. This gives good idea for new developer to identify which is source and which is test code. ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228056749
--- Diff: docs/csdk-guide.md ---
@@ -30,106 +30,13 @@ code and without CarbonSession.
In the carbon jars package, there exist a carbondata-sdk.jar,
including SDK reader for CSDK.
## Quick example
-```
-// 1. init JVM
-JavaVM *jvm;
-JNIEnv *initJVM() {
-JNIEnv *env;
-JavaVMInitArgs vm_args;
-int parNum = 3;
-int res;
-JavaVMOption options[parNum];
-
-options[0].optionString = "-Djava.compiler=NONE";
-options[1].optionString =
"-Djava.class.path=../../sdk/target/carbondata-sdk.jar";
-options[2].optionString = "-verbose:jni";
-vm_args.version = JNI_VERSION_1_8;
-vm_args.nOptions = parNum;
-vm_args.options = options;
-vm_args.ignoreUnrecognized = JNI_FALSE;
-
-res = JNI_CreateJavaVM(, (void **) , _args);
-if (res < 0) {
-fprintf(stderr, "\nCan't create Java VM\n");
-exit(1);
-}
-
-return env;
-}
-
-// 2. create carbon reader and read data
-// 2.1 read data from local disk
-/**
- * test read data from local disk, without projection
- *
- * @param env jni env
- * @return
- */
-bool readFromLocalWithoutProjection(JNIEnv *env) {
-
-CarbonReader carbonReaderClass;
-carbonReaderClass.builder(env, "../resources/carbondata", "test");
-carbonReaderClass.build();
-
-while (carbonReaderClass.hasNext()) {
-jobjectArray row = carbonReaderClass.readNextRow();
-jsize length = env->GetArrayLength(row);
-int j = 0;
-for (j = 0; j < length; j++) {
-jobject element = env->GetObjectArrayElement(row, j);
-char *str = (char *) env->GetStringUTFChars((jstring) element,
JNI_FALSE);
-printf("%s\t", str);
-}
-printf("\n");
-}
-carbonReaderClass.close();
-}
-
-// 2.2 read data from S3
-
-/**
- * read data from S3
- * parameter is ak sk endpoint
- *
- * @param env jni env
- * @param argv argument vector
- * @return
- */
-bool readFromS3(JNIEnv *env, char *argv[]) {
-CarbonReader reader;
-
-char *args[3];
-// "your access key"
-args[0] = argv[1];
-// "your secret key"
-args[1] = argv[2];
-// "your endPoint"
-args[2] = argv[3];
-
-reader.builder(env, "s3a://sdk/WriterOutput", "test");
-reader.withHadoopConf(3, args);
-reader.build();
-printf("\nRead data from S3:\n");
-while (reader.hasNext()) {
-jobjectArray row = reader.readNextRow();
-jsize length = env->GetArrayLength(row);
-
-int j = 0;
-for (j = 0; j < length; j++) {
-jobject element = env->GetObjectArrayElement(row, j);
-char *str = (char *) env->GetStringUTFChars((jstring) element,
JNI_FALSE);
-printf("%s\t", str);
-}
-printf("\n");
-}
-
-reader.close();
-}
-
-// 3. destory JVM
-(jvm)->DestroyJavaVM();
-```
-Find example code at main.cpp of CSDK module
+
+Please find example code at main.cpp of CSDK module
--- End diff --
can we add a link here for main.cpp ? so that just click will take to
main.cpp
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228055937
--- Diff: store/CSDK/CarbonRow.cpp ---
@@ -0,0 +1,129 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include "CarbonRow.h"
+
+CarbonRow::CarbonRow(JNIEnv *env) {
+this->rowUtilClass =
env->FindClass("org/apache/carbondata/sdk/file/RowUtil");
+this->jniEnv = env;
+getShortId = jniEnv->GetStaticMethodID(rowUtilClass, "getShort",
+"([Ljava/lang/Object;I)S");
+getIntId = jniEnv->GetStaticMethodID(rowUtilClass, "getInt",
+"([Ljava/lang/Object;I)I");
+getLongId = jniEnv->GetStaticMethodID(rowUtilClass, "getLong",
+"([Ljava/lang/Object;I)J");
+getDoubleId = jniEnv->GetStaticMethodID(rowUtilClass, "getDouble",
+"([Ljava/lang/Object;I)D");
+getFloatId = jniEnv->GetStaticMethodID(rowUtilClass, "getFloat",
+"([Ljava/lang/Object;I)F");
+getBooleanId = jniEnv->GetStaticMethodID(rowUtilClass, "getBoolean",
+"([Ljava/lang/Object;I)Z");
+getStringId = jniEnv->GetStaticMethodID(rowUtilClass, "getString",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getDecimalId = jniEnv->GetStaticMethodID(rowUtilClass, "getDecimal",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getVarcharId = jniEnv->GetStaticMethodID(rowUtilClass, "getVarchar",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getArrayId = jniEnv->GetStaticMethodID(rowUtilClass, "getArray",
+"([Ljava/lang/Object;I)[Ljava/lang/Object;");
+}
+
+void CarbonRow::setCarbonRow(jobject data) {
+this->carbonRow = data;
+}
+
+short CarbonRow::getShort(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticShortMethodA(rowUtilClass, getShortId, args);
--- End diff --
(short)carbonRow[ordinal]
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228054034
--- Diff: store/CSDK/CarbonReader.cpp ---
@@ -38,9 +37,9 @@ jobject CarbonReader::builder(JNIEnv *env, char *path) {
jclass carbonReaderClass =
env->FindClass("org/apache/carbondata/sdk/file/CarbonReader");
--- End diff --
same as Above, for all the class in carbonReader.cpp handle the negative
scenarios by adding validation. Else we get jvm crash.
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228053433
--- Diff: store/CSDK/CarbonReader.cpp ---
@@ -19,15 +19,14 @@
#include
jobject CarbonReader::builder(JNIEnv *env, char *path, char *tableName) {
-
jniEnv = env;
jclass carbonReaderClass =
env->FindClass("org/apache/carbondata/sdk/file/CarbonReader");
jmethodID carbonReaderBuilderID =
env->GetStaticMethodID(carbonReaderClass, "builder",
--- End diff --
If class path is wrong or class is not found in given path. This will be
NPE and jvm will crash.
Need to validate this and throw exception or return null with log to the
user.
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228047898
--- Diff: docs/csdk-guide.md ---
@@ -68,20 +68,42 @@ JNIEnv *initJVM() {
bool readFromLocalWithoutProjection(JNIEnv *env) {
CarbonReader carbonReaderClass;
-carbonReaderClass.builder(env, "../resources/carbondata", "test");
+carbonReaderClass.builder(env, "../resources/carbondata");
--- End diff --
SDK guide also has this problem, do we need remove it too?
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228046987
--- Diff: store/CSDK/CarbonRow.cpp ---
@@ -0,0 +1,129 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include "CarbonRow.h"
+
+CarbonRow::CarbonRow(JNIEnv *env) {
+this->rowUtilClass =
env->FindClass("org/apache/carbondata/sdk/file/RowUtil");
+this->jniEnv = env;
+getShortId = jniEnv->GetStaticMethodID(rowUtilClass, "getShort",
+"([Ljava/lang/Object;I)S");
+getIntId = jniEnv->GetStaticMethodID(rowUtilClass, "getInt",
+"([Ljava/lang/Object;I)I");
+getLongId = jniEnv->GetStaticMethodID(rowUtilClass, "getLong",
+"([Ljava/lang/Object;I)J");
+getDoubleId = jniEnv->GetStaticMethodID(rowUtilClass, "getDouble",
+"([Ljava/lang/Object;I)D");
+getFloatId = jniEnv->GetStaticMethodID(rowUtilClass, "getFloat",
+"([Ljava/lang/Object;I)F");
+getBooleanId = jniEnv->GetStaticMethodID(rowUtilClass, "getBoolean",
+"([Ljava/lang/Object;I)Z");
+getStringId = jniEnv->GetStaticMethodID(rowUtilClass, "getString",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getDecimalId = jniEnv->GetStaticMethodID(rowUtilClass, "getDecimal",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getVarcharId = jniEnv->GetStaticMethodID(rowUtilClass, "getVarchar",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getArrayId = jniEnv->GetStaticMethodID(rowUtilClass, "getArray",
+"([Ljava/lang/Object;I)[Ljava/lang/Object;");
+}
+
+void CarbonRow::setCarbonRow(jobject data) {
+this->carbonRow = data;
+}
+
+short CarbonRow::getShort(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticShortMethodA(rowUtilClass, getShortId, args);
--- End diff --
How to cast type?
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r228046803 --- Diff: store/CSDK/main.cpp --- @@ -21,6 +21,7 @@ #include --- End diff -- SCDK will add test framework(such as google test) in the future. ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r228046466 --- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/RowUtil.java --- @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + *http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.carbondata.sdk.file; --- End diff -- How to cast type? ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228044190
--- Diff: store/CSDK/CarbonRow.cpp ---
@@ -0,0 +1,129 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include "CarbonRow.h"
+
+CarbonRow::CarbonRow(JNIEnv *env) {
+this->rowUtilClass =
env->FindClass("org/apache/carbondata/sdk/file/RowUtil");
+this->jniEnv = env;
+getShortId = jniEnv->GetStaticMethodID(rowUtilClass, "getShort",
+"([Ljava/lang/Object;I)S");
+getIntId = jniEnv->GetStaticMethodID(rowUtilClass, "getInt",
+"([Ljava/lang/Object;I)I");
+getLongId = jniEnv->GetStaticMethodID(rowUtilClass, "getLong",
+"([Ljava/lang/Object;I)J");
+getDoubleId = jniEnv->GetStaticMethodID(rowUtilClass, "getDouble",
+"([Ljava/lang/Object;I)D");
+getFloatId = jniEnv->GetStaticMethodID(rowUtilClass, "getFloat",
+"([Ljava/lang/Object;I)F");
+getBooleanId = jniEnv->GetStaticMethodID(rowUtilClass, "getBoolean",
+"([Ljava/lang/Object;I)Z");
+getStringId = jniEnv->GetStaticMethodID(rowUtilClass, "getString",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getDecimalId = jniEnv->GetStaticMethodID(rowUtilClass, "getDecimal",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getVarcharId = jniEnv->GetStaticMethodID(rowUtilClass, "getVarchar",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+getArrayId = jniEnv->GetStaticMethodID(rowUtilClass, "getArray",
+"([Ljava/lang/Object;I)[Ljava/lang/Object;");
+}
+
+void CarbonRow::setCarbonRow(jobject data) {
+this->carbonRow = data;
+}
+
+short CarbonRow::getShort(int ordinal) {
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticShortMethodA(rowUtilClass, getShortId, args);
--- End diff --
Here rowUtils is just converting java object to Java data type. This is
just a typecast operation. No need of rowUtil Class. Can directly typecast here.
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r228041857 --- Diff: store/CSDK/main.cpp --- @@ -21,6 +21,7 @@ #include --- End diff -- Can we move this to test ? because this is not a product code. ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228040809
--- Diff:
store/sdk/src/test/java/org/apache/carbondata/sdk/file/CarbonReaderTest.java ---
@@ -1522,4 +1522,208 @@ public boolean accept(File dir, String name) {
e.printStackTrace();
}
}
+
+ @Test
+ public void testReadNextRowWithRowUtil() {
--- End diff --
As class not required. Test cases are not required for it
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r228040609 --- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/RowUtil.java --- @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + *http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.carbondata.sdk.file; --- End diff -- I think we don't need this class at all. This is just a typecast. If user already knows data type, he can typecast instead of calling a method that does typecast. please remove this call ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228039418
--- Diff: docs/csdk-guide.md ---
@@ -68,20 +68,42 @@ JNIEnv *initJVM() {
bool readFromLocalWithoutProjection(JNIEnv *env) {
CarbonReader carbonReaderClass;
-carbonReaderClass.builder(env, "../resources/carbondata", "test");
+carbonReaderClass.builder(env, "../resources/carbondata");
--- End diff --
Already main.cpp has these same examples. Please give a link to that file
and remove from here.
Because if any future change happens, we need not have to change at two
places and keep duplicate code samples.
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228010100
--- Diff:
store/sdk/src/test/java/org/apache/carbondata/sdk/file/CarbonReaderTest.java ---
@@ -1522,4 +1522,204 @@ public boolean accept(File dir, String name) {
e.printStackTrace();
}
}
+
+ @Test
+ public void testReadNextRowWithRowUtil() {
+String path = "./carbondata";
+try {
+ FileUtils.deleteDirectory(new File(path));
+
+ Field[] fields = new Field[12];
+ fields[0] = new Field("stringField", DataTypes.STRING);
+ fields[1] = new Field("shortField", DataTypes.SHORT);
+ fields[2] = new Field("intField", DataTypes.INT);
+ fields[3] = new Field("longField", DataTypes.LONG);
+ fields[4] = new Field("doubleField", DataTypes.DOUBLE);
+ fields[5] = new Field("boolField", DataTypes.BOOLEAN);
+ fields[6] = new Field("dateField", DataTypes.DATE);
+ fields[7] = new Field("timeField", DataTypes.TIMESTAMP);
+ fields[8] = new Field("decimalField", DataTypes.createDecimalType(8,
2));
+ fields[9] = new Field("varcharField", DataTypes.VARCHAR);
+ fields[10] = new Field("arrayField",
DataTypes.createArrayType(DataTypes.STRING));
+ fields[11] = new Field("floatField", DataTypes.FLOAT);
+ Map map = new HashMap<>();
+ map.put("complex_delimiter_level_1", "#");
+ CarbonWriter writer = CarbonWriter.builder()
+ .outputPath(path)
+ .withLoadOptions(map)
+ .withCsvInput(new Schema(fields)).build();
+
+ for (int i = 0; i < 10; i++) {
+String[] row2 = new String[]{
+"robot" + (i % 10),
+String.valueOf(i % 1),
+String.valueOf(i),
+String.valueOf(Long.MAX_VALUE - i),
+String.valueOf((double) i / 2),
+String.valueOf(true),
+"2019-03-02",
+"2019-02-12 03:03:34",
+"12.345",
+"varchar",
+"Hello#World#From#Carbon",
+"1.23"
+};
+writer.write(row2);
+ }
+ writer.close();
+
+ File[] dataFiles = new File(path).listFiles(new FilenameFilter() {
+@Override
+public boolean accept(File dir, String name) {
+ if (name == null) {
+return false;
+ }
+ return name.endsWith("carbonindex");
+}
+ });
+ if (dataFiles == null || dataFiles.length < 1) {
+throw new RuntimeException("Carbon index file not exists.");
+ }
+ Schema schema = CarbonSchemaReader
+ .readSchemaInIndexFile(dataFiles[0].getAbsolutePath())
+ .asOriginOrder();
+ // Transform the schema
+ int count = 0;
+ for (int i = 0; i < schema.getFields().length; i++) {
+if (!((schema.getFields())[i].getFieldName().contains("."))) {
+ count++;
+}
+ }
+ String[] strings = new String[count];
+ int index = 0;
+ for (int i = 0; i < schema.getFields().length; i++) {
+if (!((schema.getFields())[i].getFieldName().contains("."))) {
+ strings[index] = (schema.getFields())[i].getFieldName();
+ index++;
+}
+ }
+ // Read data
+ CarbonReader reader = CarbonReader
+ .builder(path, "_temp")
+ .projection(strings)
+ .build();
+
+ int i = 0;
+ while (reader.hasNext()) {
+Object[] data = (Object[]) reader.readNextRow();
+
+assert (RowUtil.getString(data, 0).equals("robot" + i));
+assertEquals(RowUtil.getShort(data, 1), i);
+assertEquals(RowUtil.getInt(data, 2), i);
+assertEquals(RowUtil.getLong(data, 3), Long.MAX_VALUE - i);
+assertEquals(RowUtil.getDouble(data, 4), ((double) i) / 2);
+assert (RowUtil.getBoolean(data, 5));
+assertEquals(RowUtil.getInt(data, 6), 17957);
+assert (RowUtil.getDecimal(data, 8).equals("12.35"));
+assert (RowUtil.getVarchar(data, 9).equals("varchar"));
+
+Object[] arr = RowUtil.getArray(data, 10);
+assert (arr[0].equals("Hello"));
+assert (arr[1].equals("World"));
+assert (arr[2].equals("From"));
+assert (arr[3].equals("Carbon"));
+
+assertEquals(RowUtil.getFloat(data, 11), (float) 1.23);
+i++;
+ }
+ reader.close();
+} catch (Throwable e) {
+ e.printStackTrace();
+} finally {
+ try {
+FileUtils.deleteDirectory(new File(path));
+ } catch
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228009608
--- Diff: docs/csdk-guide.md ---
@@ -68,20 +68,42 @@ JNIEnv *initJVM() {
bool readFromLocalWithoutProjection(JNIEnv *env) {
CarbonReader carbonReaderClass;
-carbonReaderClass.builder(env, "../resources/carbondata", "test");
+carbonReaderClass.builder(env, "../resources/carbondata");
carbonReaderClass.build();
+printf("\nRead data from local without projection:\n");
+
+CarbonRow carbonRow(env);
while (carbonReaderClass.hasNext()) {
-jobjectArray row = carbonReaderClass.readNextRow();
-jsize length = env->GetArrayLength(row);
+jobject row = carbonReaderClass.readNextCarbonRow();
--- End diff --
ok, done
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228009633
--- Diff: docs/csdk-guide.md ---
@@ -106,20 +128,41 @@ bool readFromS3(JNIEnv *env, char *argv[]) {
// "your endPoint"
args[2] = argv[3];
-reader.builder(env, "s3a://sdk/WriterOutput", "test");
-reader.withHadoopConf(3, args);
+reader.builder(env, "s3a://sdk/WriterOutput/carbondata/", "test");
+reader.withHadoopConf("fs.s3a.access.key", argv[1]);
+reader.withHadoopConf("fs.s3a.secret.key", argv[2]);
+reader.withHadoopConf("fs.s3a.endpoint", argv[3]);
reader.build();
printf("\nRead data from S3:\n");
+CarbonRow carbonRow(env);
while (reader.hasNext()) {
-jobjectArray row = reader.readNextRow();
-jsize length = env->GetArrayLength(row);
-
+jobject row = reader.readNextCarbonRow();
--- End diff --
ok, done
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r228009906
--- Diff:
store/sdk/src/test/java/org/apache/carbondata/sdk/file/CarbonReaderTest.java ---
@@ -1522,4 +1522,204 @@ public boolean accept(File dir, String name) {
e.printStackTrace();
}
}
+
+ @Test
+ public void testReadNextRowWithRowUtil() {
+String path = "./carbondata";
+try {
+ FileUtils.deleteDirectory(new File(path));
+
+ Field[] fields = new Field[12];
+ fields[0] = new Field("stringField", DataTypes.STRING);
+ fields[1] = new Field("shortField", DataTypes.SHORT);
+ fields[2] = new Field("intField", DataTypes.INT);
+ fields[3] = new Field("longField", DataTypes.LONG);
+ fields[4] = new Field("doubleField", DataTypes.DOUBLE);
+ fields[5] = new Field("boolField", DataTypes.BOOLEAN);
+ fields[6] = new Field("dateField", DataTypes.DATE);
+ fields[7] = new Field("timeField", DataTypes.TIMESTAMP);
+ fields[8] = new Field("decimalField", DataTypes.createDecimalType(8,
2));
+ fields[9] = new Field("varcharField", DataTypes.VARCHAR);
+ fields[10] = new Field("arrayField",
DataTypes.createArrayType(DataTypes.STRING));
+ fields[11] = new Field("floatField", DataTypes.FLOAT);
+ Map map = new HashMap<>();
+ map.put("complex_delimiter_level_1", "#");
+ CarbonWriter writer = CarbonWriter.builder()
+ .outputPath(path)
+ .withLoadOptions(map)
+ .withCsvInput(new Schema(fields)).build();
+
+ for (int i = 0; i < 10; i++) {
+String[] row2 = new String[]{
+"robot" + (i % 10),
+String.valueOf(i % 1),
+String.valueOf(i),
+String.valueOf(Long.MAX_VALUE - i),
+String.valueOf((double) i / 2),
+String.valueOf(true),
+"2019-03-02",
+"2019-02-12 03:03:34",
+"12.345",
+"varchar",
+"Hello#World#From#Carbon",
+"1.23"
+};
+writer.write(row2);
+ }
+ writer.close();
+
+ File[] dataFiles = new File(path).listFiles(new FilenameFilter() {
+@Override
+public boolean accept(File dir, String name) {
+ if (name == null) {
+return false;
+ }
+ return name.endsWith("carbonindex");
+}
+ });
+ if (dataFiles == null || dataFiles.length < 1) {
+throw new RuntimeException("Carbon index file not exists.");
+ }
+ Schema schema = CarbonSchemaReader
+ .readSchemaInIndexFile(dataFiles[0].getAbsolutePath())
+ .asOriginOrder();
+ // Transform the schema
+ int count = 0;
+ for (int i = 0; i < schema.getFields().length; i++) {
+if (!((schema.getFields())[i].getFieldName().contains("."))) {
+ count++;
+}
+ }
+ String[] strings = new String[count];
+ int index = 0;
+ for (int i = 0; i < schema.getFields().length; i++) {
+if (!((schema.getFields())[i].getFieldName().contains("."))) {
+ strings[index] = (schema.getFields())[i].getFieldName();
+ index++;
+}
+ }
+ // Read data
+ CarbonReader reader = CarbonReader
+ .builder(path, "_temp")
+ .projection(strings)
+ .build();
+
+ int i = 0;
+ while (reader.hasNext()) {
+Object[] data = (Object[]) reader.readNextRow();
+
+assert (RowUtil.getString(data, 0).equals("robot" + i));
+assertEquals(RowUtil.getShort(data, 1), i);
+assertEquals(RowUtil.getInt(data, 2), i);
+assertEquals(RowUtil.getLong(data, 3), Long.MAX_VALUE - i);
+assertEquals(RowUtil.getDouble(data, 4), ((double) i) / 2);
+assert (RowUtil.getBoolean(data, 5));
+assertEquals(RowUtil.getInt(data, 6), 17957);
+assert (RowUtil.getDecimal(data, 8).equals("12.35"));
+assert (RowUtil.getVarchar(data, 9).equals("varchar"));
+
+Object[] arr = RowUtil.getArray(data, 10);
+assert (arr[0].equals("Hello"));
+assert (arr[1].equals("World"));
+assert (arr[2].equals("From"));
+assert (arr[3].equals("Carbon"));
+
+assertEquals(RowUtil.getFloat(data, 11), (float) 1.23);
+i++;
+ }
+ reader.close();
+} catch (Throwable e) {
+ e.printStackTrace();
--- End diff --
ok, done
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user KanakaKumar commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r227839509
--- Diff:
store/sdk/src/test/java/org/apache/carbondata/sdk/file/CarbonReaderTest.java ---
@@ -1522,4 +1522,204 @@ public boolean accept(File dir, String name) {
e.printStackTrace();
}
}
+
+ @Test
+ public void testReadNextRowWithRowUtil() {
+String path = "./carbondata";
+try {
+ FileUtils.deleteDirectory(new File(path));
+
+ Field[] fields = new Field[12];
+ fields[0] = new Field("stringField", DataTypes.STRING);
+ fields[1] = new Field("shortField", DataTypes.SHORT);
+ fields[2] = new Field("intField", DataTypes.INT);
+ fields[3] = new Field("longField", DataTypes.LONG);
+ fields[4] = new Field("doubleField", DataTypes.DOUBLE);
+ fields[5] = new Field("boolField", DataTypes.BOOLEAN);
+ fields[6] = new Field("dateField", DataTypes.DATE);
+ fields[7] = new Field("timeField", DataTypes.TIMESTAMP);
+ fields[8] = new Field("decimalField", DataTypes.createDecimalType(8,
2));
+ fields[9] = new Field("varcharField", DataTypes.VARCHAR);
+ fields[10] = new Field("arrayField",
DataTypes.createArrayType(DataTypes.STRING));
+ fields[11] = new Field("floatField", DataTypes.FLOAT);
+ Map map = new HashMap<>();
+ map.put("complex_delimiter_level_1", "#");
+ CarbonWriter writer = CarbonWriter.builder()
+ .outputPath(path)
+ .withLoadOptions(map)
+ .withCsvInput(new Schema(fields)).build();
+
+ for (int i = 0; i < 10; i++) {
+String[] row2 = new String[]{
+"robot" + (i % 10),
+String.valueOf(i % 1),
+String.valueOf(i),
+String.valueOf(Long.MAX_VALUE - i),
+String.valueOf((double) i / 2),
+String.valueOf(true),
+"2019-03-02",
+"2019-02-12 03:03:34",
+"12.345",
+"varchar",
+"Hello#World#From#Carbon",
+"1.23"
+};
+writer.write(row2);
+ }
+ writer.close();
+
+ File[] dataFiles = new File(path).listFiles(new FilenameFilter() {
+@Override
+public boolean accept(File dir, String name) {
+ if (name == null) {
+return false;
+ }
+ return name.endsWith("carbonindex");
+}
+ });
+ if (dataFiles == null || dataFiles.length < 1) {
+throw new RuntimeException("Carbon index file not exists.");
+ }
+ Schema schema = CarbonSchemaReader
+ .readSchemaInIndexFile(dataFiles[0].getAbsolutePath())
+ .asOriginOrder();
+ // Transform the schema
+ int count = 0;
+ for (int i = 0; i < schema.getFields().length; i++) {
+if (!((schema.getFields())[i].getFieldName().contains("."))) {
+ count++;
+}
+ }
+ String[] strings = new String[count];
+ int index = 0;
+ for (int i = 0; i < schema.getFields().length; i++) {
+if (!((schema.getFields())[i].getFieldName().contains("."))) {
+ strings[index] = (schema.getFields())[i].getFieldName();
+ index++;
+}
+ }
+ // Read data
+ CarbonReader reader = CarbonReader
+ .builder(path, "_temp")
+ .projection(strings)
+ .build();
+
+ int i = 0;
+ while (reader.hasNext()) {
+Object[] data = (Object[]) reader.readNextRow();
+
+assert (RowUtil.getString(data, 0).equals("robot" + i));
+assertEquals(RowUtil.getShort(data, 1), i);
+assertEquals(RowUtil.getInt(data, 2), i);
+assertEquals(RowUtil.getLong(data, 3), Long.MAX_VALUE - i);
+assertEquals(RowUtil.getDouble(data, 4), ((double) i) / 2);
+assert (RowUtil.getBoolean(data, 5));
+assertEquals(RowUtil.getInt(data, 6), 17957);
+assert (RowUtil.getDecimal(data, 8).equals("12.35"));
+assert (RowUtil.getVarchar(data, 9).equals("varchar"));
+
+Object[] arr = RowUtil.getArray(data, 10);
+assert (arr[0].equals("Hello"));
+assert (arr[1].equals("World"));
+assert (arr[2].equals("From"));
+assert (arr[3].equals("Carbon"));
+
+assertEquals(RowUtil.getFloat(data, 11), (float) 1.23);
+i++;
+ }
+ reader.close();
+} catch (Throwable e) {
+ e.printStackTrace();
+} finally {
+ try {
+FileUtils.deleteDirectory(new File(path));
+ } catch
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user KanakaKumar commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r227839131
--- Diff:
store/sdk/src/test/java/org/apache/carbondata/sdk/file/CarbonReaderTest.java ---
@@ -1522,4 +1522,204 @@ public boolean accept(File dir, String name) {
e.printStackTrace();
}
}
+
+ @Test
+ public void testReadNextRowWithRowUtil() {
+String path = "./carbondata";
+try {
+ FileUtils.deleteDirectory(new File(path));
+
+ Field[] fields = new Field[12];
+ fields[0] = new Field("stringField", DataTypes.STRING);
+ fields[1] = new Field("shortField", DataTypes.SHORT);
+ fields[2] = new Field("intField", DataTypes.INT);
+ fields[3] = new Field("longField", DataTypes.LONG);
+ fields[4] = new Field("doubleField", DataTypes.DOUBLE);
+ fields[5] = new Field("boolField", DataTypes.BOOLEAN);
+ fields[6] = new Field("dateField", DataTypes.DATE);
+ fields[7] = new Field("timeField", DataTypes.TIMESTAMP);
+ fields[8] = new Field("decimalField", DataTypes.createDecimalType(8,
2));
+ fields[9] = new Field("varcharField", DataTypes.VARCHAR);
+ fields[10] = new Field("arrayField",
DataTypes.createArrayType(DataTypes.STRING));
+ fields[11] = new Field("floatField", DataTypes.FLOAT);
+ Map map = new HashMap<>();
+ map.put("complex_delimiter_level_1", "#");
+ CarbonWriter writer = CarbonWriter.builder()
+ .outputPath(path)
+ .withLoadOptions(map)
+ .withCsvInput(new Schema(fields)).build();
+
+ for (int i = 0; i < 10; i++) {
+String[] row2 = new String[]{
+"robot" + (i % 10),
+String.valueOf(i % 1),
+String.valueOf(i),
+String.valueOf(Long.MAX_VALUE - i),
+String.valueOf((double) i / 2),
+String.valueOf(true),
+"2019-03-02",
+"2019-02-12 03:03:34",
+"12.345",
+"varchar",
+"Hello#World#From#Carbon",
+"1.23"
+};
+writer.write(row2);
+ }
+ writer.close();
+
+ File[] dataFiles = new File(path).listFiles(new FilenameFilter() {
+@Override
+public boolean accept(File dir, String name) {
+ if (name == null) {
+return false;
+ }
+ return name.endsWith("carbonindex");
+}
+ });
+ if (dataFiles == null || dataFiles.length < 1) {
+throw new RuntimeException("Carbon index file not exists.");
+ }
+ Schema schema = CarbonSchemaReader
+ .readSchemaInIndexFile(dataFiles[0].getAbsolutePath())
+ .asOriginOrder();
+ // Transform the schema
+ int count = 0;
+ for (int i = 0; i < schema.getFields().length; i++) {
+if (!((schema.getFields())[i].getFieldName().contains("."))) {
+ count++;
+}
+ }
+ String[] strings = new String[count];
+ int index = 0;
+ for (int i = 0; i < schema.getFields().length; i++) {
+if (!((schema.getFields())[i].getFieldName().contains("."))) {
+ strings[index] = (schema.getFields())[i].getFieldName();
+ index++;
+}
+ }
+ // Read data
+ CarbonReader reader = CarbonReader
+ .builder(path, "_temp")
+ .projection(strings)
+ .build();
+
+ int i = 0;
+ while (reader.hasNext()) {
+Object[] data = (Object[]) reader.readNextRow();
+
+assert (RowUtil.getString(data, 0).equals("robot" + i));
+assertEquals(RowUtil.getShort(data, 1), i);
+assertEquals(RowUtil.getInt(data, 2), i);
+assertEquals(RowUtil.getLong(data, 3), Long.MAX_VALUE - i);
+assertEquals(RowUtil.getDouble(data, 4), ((double) i) / 2);
+assert (RowUtil.getBoolean(data, 5));
+assertEquals(RowUtil.getInt(data, 6), 17957);
+assert (RowUtil.getDecimal(data, 8).equals("12.35"));
+assert (RowUtil.getVarchar(data, 9).equals("varchar"));
+
+Object[] arr = RowUtil.getArray(data, 10);
+assert (arr[0].equals("Hello"));
+assert (arr[1].equals("World"));
+assert (arr[2].equals("From"));
+assert (arr[3].equals("Carbon"));
+
+assertEquals(RowUtil.getFloat(data, 11), (float) 1.23);
+i++;
+ }
+ reader.close();
+} catch (Throwable e) {
+ e.printStackTrace();
--- End diff --
I think we should make test fail for any exception. Should not ignore
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user KanakaKumar commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r227836644
--- Diff: docs/csdk-guide.md ---
@@ -106,20 +128,41 @@ bool readFromS3(JNIEnv *env, char *argv[]) {
// "your endPoint"
args[2] = argv[3];
-reader.builder(env, "s3a://sdk/WriterOutput", "test");
-reader.withHadoopConf(3, args);
+reader.builder(env, "s3a://sdk/WriterOutput/carbondata/", "test");
+reader.withHadoopConf("fs.s3a.access.key", argv[1]);
+reader.withHadoopConf("fs.s3a.secret.key", argv[2]);
+reader.withHadoopConf("fs.s3a.endpoint", argv[3]);
reader.build();
printf("\nRead data from S3:\n");
+CarbonRow carbonRow(env);
while (reader.hasNext()) {
-jobjectArray row = reader.readNextRow();
-jsize length = env->GetArrayLength(row);
-
+jobject row = reader.readNextCarbonRow();
--- End diff --
readNextCarbonRow is removed from this PR
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user KanakaKumar commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r227836440
--- Diff: docs/csdk-guide.md ---
@@ -68,20 +68,42 @@ JNIEnv *initJVM() {
bool readFromLocalWithoutProjection(JNIEnv *env) {
CarbonReader carbonReaderClass;
-carbonReaderClass.builder(env, "../resources/carbondata", "test");
+carbonReaderClass.builder(env, "../resources/carbondata");
carbonReaderClass.build();
+printf("\nRead data from local without projection:\n");
+
+CarbonRow carbonRow(env);
while (carbonReaderClass.hasNext()) {
-jobjectArray row = carbonReaderClass.readNextRow();
-jsize length = env->GetArrayLength(row);
+jobject row = carbonReaderClass.readNextCarbonRow();
--- End diff --
Need to correct readNextCarbonRow is no more present
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r227807282
--- Diff:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonReader.java ---
@@ -93,32 +92,10 @@ public T readNextRow() throws IOException,
InterruptedException {
}
/**
- * Read and return next string row object
- * limitation: only single dimension Array is supported
- * TODO: support didfferent data type
+ * Read and return next carbon row object
*/
- public Object[] readNextStringRow() throws IOException,
InterruptedException {
-validateReader();
-T t = currentReader.getCurrentValue();
-Object[] objects = (Object[]) t;
-String[] strings = new String[objects.length];
-for (int i = 0; i < objects.length; i++) {
- if (objects[i] instanceof Object[]) {
-Object[] arrayString = (Object[]) objects[i];
-StringBuffer stringBuffer = new StringBuffer();
-stringBuffer.append(String.valueOf(arrayString[0]));
-if (arrayString.length > 1) {
- for (int j = 1; j < arrayString.length; j++) {
-stringBuffer.append(CarbonCommonConstants.ARRAY_SEPARATOR)
-.append(String.valueOf(arrayString[j]));
- }
-}
-strings[i] = stringBuffer.toString();
- } else {
-strings[i] = String.valueOf(objects[i]);
- }
-}
-return strings;
+ public Object[] readNextCarbonRow() throws IOException,
InterruptedException {
+return (Object[]) readNextRow();
--- End diff --
ok, done
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r227798659
--- Diff: store/CSDK/CarbonRow.cpp ---
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include "CarbonRow.h"
+
+CarbonRow::CarbonRow(JNIEnv *env) {
+this->rowUtilClass =
env->FindClass("org/apache/carbondata/sdk/file/RowUtil");
+this->jniEnv = env;
+}
+
+void CarbonRow::setCarbonRow(jobject data) {
+this->carbonRow = data;
+}
+
+short CarbonRow::getShort(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass, "getShort",
+"([Ljava/lang/Object;I)S");
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticShortMethodA(rowUtilClass, buildID, args);
+}
+
+int CarbonRow::getInt(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass, "getInt",
+"([Ljava/lang/Object;I)I");
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticIntMethodA(rowUtilClass, buildID, args);
+}
+
+long CarbonRow::getLong(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass, "getLong",
+"([Ljava/lang/Object;I)J");
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticLongMethodA(rowUtilClass, buildID, args);
+}
+
+double CarbonRow::getDouble(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass,
"getDouble",
+"([Ljava/lang/Object;I)D");
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticDoubleMethodA(rowUtilClass, buildID, args);
+}
+
+
+float CarbonRow::getFloat(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass, "getFloat",
+"([Ljava/lang/Object;I)F");
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticFloatMethodA(rowUtilClass, buildID, args);
+}
+
+jboolean CarbonRow::getBoolean(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass,
"getBoolean",
+"([Ljava/lang/Object;I)Z");
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticBooleanMethodA(rowUtilClass, buildID, args);
+}
+
+char *CarbonRow::getString(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass,
"getString",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+jobject data = jniEnv->CallStaticObjectMethodA(rowUtilClass, buildID,
args);
+
+char *str = (char *) jniEnv->GetStringUTFChars((jstring) data,
JNI_FALSE);
+return str;
+}
+
+char *CarbonRow::getDecimal(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass,
"getDecimal",
--- End diff --
ok, done
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user KanakaKumar commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r227741794
--- Diff: store/CSDK/CarbonRow.cpp ---
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include "CarbonRow.h"
+
+CarbonRow::CarbonRow(JNIEnv *env) {
+this->rowUtilClass =
env->FindClass("org/apache/carbondata/sdk/file/RowUtil");
+this->jniEnv = env;
+}
+
+void CarbonRow::setCarbonRow(jobject data) {
+this->carbonRow = data;
+}
+
+short CarbonRow::getShort(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass, "getShort",
+"([Ljava/lang/Object;I)S");
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticShortMethodA(rowUtilClass, buildID, args);
+}
+
+int CarbonRow::getInt(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass, "getInt",
+"([Ljava/lang/Object;I)I");
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticIntMethodA(rowUtilClass, buildID, args);
+}
+
+long CarbonRow::getLong(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass, "getLong",
+"([Ljava/lang/Object;I)J");
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticLongMethodA(rowUtilClass, buildID, args);
+}
+
+double CarbonRow::getDouble(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass,
"getDouble",
+"([Ljava/lang/Object;I)D");
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticDoubleMethodA(rowUtilClass, buildID, args);
+}
+
+
+float CarbonRow::getFloat(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass, "getFloat",
+"([Ljava/lang/Object;I)F");
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticFloatMethodA(rowUtilClass, buildID, args);
+}
+
+jboolean CarbonRow::getBoolean(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass,
"getBoolean",
+"([Ljava/lang/Object;I)Z");
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+return jniEnv->CallStaticBooleanMethodA(rowUtilClass, buildID, args);
+}
+
+char *CarbonRow::getString(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass,
"getString",
+"([Ljava/lang/Object;I)Ljava/lang/String;");
+jvalue args[2];
+args[0].l = carbonRow;
+args[1].i = ordinal;
+jobject data = jniEnv->CallStaticObjectMethodA(rowUtilClass, buildID,
args);
+
+char *str = (char *) jniEnv->GetStringUTFChars((jstring) data,
JNI_FALSE);
+return str;
+}
+
+char *CarbonRow::getDecimal(int ordinal) {
+jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass,
"getDecimal",
--- End diff --
jmethodID buildID = jniEnv->GetStaticMethodID(rowUtilClass, "getDecimal",
"([Ljava/lang/Object;I)Ljava/lang/String;");
jvalue args[2];
Accessing the static method and initializing the array is done for every
row reading. Create once and reuse may improve performance. Can you please
try?
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r226830397
--- Diff:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/RowUtil.java ---
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.sdk.file;
+
+import java.io.Serializable;
+import java.math.BigDecimal;
+
+/**
+ * This row class is used to transfer the row data from one step to other
step
+ */
+public class RowUtil implements Serializable {
+
+ public static String getString(Object[] data, int ordinal) {
+return (String) data[ordinal];
+ }
+
+ /**
+ * get short type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return short data type data
+ */
+ public static short getShort(Object[] data, int ordinal) {
+return (short) data[ordinal];
+ }
+
+ /**
+ * get int data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return int data type data
+ */
+ public static int getInt(Object[] data, int ordinal) {
+return (Integer) data[ordinal];
+ }
+
+ /**
+ * get long data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return long data type data
+ */
+ public static long getLong(Object[] data, int ordinal) {
+return (long) data[ordinal];
+ }
+
+ /**
+ * get array data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return array data type data
+ */
+ public static Object[] getArray(Object[] data, int ordinal) {
+return (Object[]) data[ordinal];
+ }
+
+ /**
+ * get double data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return double data type data
+ */
+ public static double getDouble(Object[] data, int ordinal) {
+return (double) data[ordinal];
+ }
+
+ /**
+ * get boolean data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return boolean data type data
+ */
+ public static boolean getBoolean(Object[] data, int ordinal) {
+return (boolean) data[ordinal];
+ }
+
+ /**
+ * get byte data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return byte data type data
+ */
+ public static Byte getByte(Object[] data, int ordinal) {
+return (Byte) data[ordinal];
+ }
+
+ /**
+ * get float data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return float data type data
+ */
+ public static float getFloat(Object[] data, int ordinal) {
+return (float) data[ordinal];
+ }
+
+ /**
+ * get varchar data type data by ordinal
+ * This is for CSDK
+ * JNI don't support varchar, so carbon convert decimal to string
--- End diff --
sorry, I changed, please check.
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r226821049
--- Diff:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/RowUtil.java ---
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.sdk.file;
+
+import java.io.Serializable;
+import java.math.BigDecimal;
+
+/**
+ * This row class is used to transfer the row data from one step to other
step
+ */
+public class RowUtil implements Serializable {
+
+ public static String getString(Object[] data, int ordinal) {
+return (String) data[ordinal];
+ }
+
+ /**
+ * get short type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return short data type data
+ */
+ public static short getShort(Object[] data, int ordinal) {
+return (short) data[ordinal];
+ }
+
+ /**
+ * get int data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return int data type data
+ */
+ public static int getInt(Object[] data, int ordinal) {
+return (Integer) data[ordinal];
+ }
+
+ /**
+ * get long data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return long data type data
+ */
+ public static long getLong(Object[] data, int ordinal) {
+return (long) data[ordinal];
+ }
+
+ /**
+ * get array data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return array data type data
+ */
+ public static Object[] getArray(Object[] data, int ordinal) {
+return (Object[]) data[ordinal];
+ }
+
+ /**
+ * get double data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return double data type data
+ */
+ public static double getDouble(Object[] data, int ordinal) {
+return (double) data[ordinal];
+ }
+
+ /**
+ * get boolean data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return boolean data type data
+ */
+ public static boolean getBoolean(Object[] data, int ordinal) {
+return (boolean) data[ordinal];
+ }
+
+ /**
+ * get byte data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return byte data type data
+ */
+ public static Byte getByte(Object[] data, int ordinal) {
+return (Byte) data[ordinal];
+ }
+
+ /**
+ * get float data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return float data type data
+ */
+ public static float getFloat(Object[] data, int ordinal) {
+return (float) data[ordinal];
+ }
+
+ /**
+ * get varchar data type data by ordinal
+ * This is for CSDK
+ * JNI don't support varchar, so carbon convert decimal to string
--- End diff --
but this method is for varchar not decimal...
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r226562638 --- Diff: docs/documentation.md --- @@ -31,7 +31,7 @@ Apache CarbonData is a new big data file format for faster interactive query usi **CarbonData SQL Language Reference:** CarbonData extends the Spark SQL language and adds several [DDL](./ddl-of-carbondata.md) and [DML](./dml-of-carbondata.md) statements to support operations on it.Refer to the [Reference Manual](./language-manual.md) to understand the supported features and functions. -**Programming Guides:** You can read our guides about [APIs supported](./sdk-guide.md) to learn how to integrate CarbonData with your applications. +**Programming Guides:** You can read our guides about [Java APIs supported](./sdk-guide.md) or [C++ APIs supported](./CSDK-guide.md) to learn how to integrate CarbonData with your applications. --- End diff -- ok, done ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r226562555
--- Diff: docs/CSDK-guide.md ---
@@ -68,20 +68,42 @@ JNIEnv *initJVM() {
bool readFromLocalWithoutProjection(JNIEnv *env) {
CarbonReader carbonReaderClass;
-carbonReaderClass.builder(env, "../resources/carbondata", "test");
+carbonReaderClass.builder(env, "../resources/carbondata");
carbonReaderClass.build();
+printf("\nRead data from local without projection:\n");
--- End diff --
ok, done
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r226562463 --- Diff: README.md --- @@ -61,6 +61,7 @@ CarbonData is built using Apache Maven, to [build CarbonData](https://github.com * [CarbonData Pre-aggregate DataMap](https://github.com/apache/carbondata/blob/master/docs/preaggregate-datamap-guide.md) * [CarbonData Timeseries DataMap](https://github.com/apache/carbondata/blob/master/docs/timeseries-datamap-guide.md) * [SDK Guide](https://github.com/apache/carbondata/blob/master/docs/sdk-guide.md) +* [CSDK Guide](https://github.com/apache/carbondata/blob/master/docs/CSDK-guide.md) --- End diff -- ok ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r226560678 --- Diff: docs/documentation.md --- @@ -31,7 +31,7 @@ Apache CarbonData is a new big data file format for faster interactive query usi **CarbonData SQL Language Reference:** CarbonData extends the Spark SQL language and adds several [DDL](./ddl-of-carbondata.md) and [DML](./dml-of-carbondata.md) statements to support operations on it.Refer to the [Reference Manual](./language-manual.md) to understand the supported features and functions. -**Programming Guides:** You can read our guides about [APIs supported](./sdk-guide.md) to learn how to integrate CarbonData with your applications. +**Programming Guides:** You can read our guides about [Java APIs supported](./sdk-guide.md) or [C++ APIs supported](./CSDK-guide.md) to learn how to integrate CarbonData with your applications. --- End diff -- How to optimize? ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r226560544
--- Diff:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/RowUtil.java ---
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.sdk.file;
+
+import java.io.Serializable;
+import java.math.BigDecimal;
+
+/**
+ * This row class is used to transfer the row data from one step to other
step
+ */
+public class RowUtil implements Serializable {
+
+ public static String getString(Object[] data, int ordinal) {
+return (String) data[ordinal];
+ }
+
+ /**
+ * get short type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return short data type data
+ */
+ public static short getShort(Object[] data, int ordinal) {
+return (short) data[ordinal];
+ }
+
+ /**
+ * get int data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return int data type data
+ */
+ public static int getInt(Object[] data, int ordinal) {
+return (Integer) data[ordinal];
+ }
+
+ /**
+ * get long data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return long data type data
+ */
+ public static long getLong(Object[] data, int ordinal) {
+return (long) data[ordinal];
+ }
+
+ /**
+ * get array data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return array data type data
+ */
+ public static Object[] getArray(Object[] data, int ordinal) {
+return (Object[]) data[ordinal];
+ }
+
+ /**
+ * get double data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return double data type data
+ */
+ public static double getDouble(Object[] data, int ordinal) {
+return (double) data[ordinal];
+ }
+
+ /**
+ * get boolean data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return boolean data type data
+ */
+ public static boolean getBoolean(Object[] data, int ordinal) {
+return (boolean) data[ordinal];
+ }
+
+ /**
+ * get byte data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return byte data type data
+ */
+ public static Byte getByte(Object[] data, int ordinal) {
+return (Byte) data[ordinal];
+ }
+
+ /**
+ * get float data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return float data type data
+ */
+ public static float getFloat(Object[] data, int ordinal) {
+return (float) data[ordinal];
+ }
+
+ /**
+ * get varchar data type data by ordinal
+ * This is for CSDK
+ * JNI don't support varchar, so carbon convert decimal to string
--- End diff --
yes
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xuchuanyin commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r226553168 --- Diff: README.md --- @@ -61,6 +61,7 @@ CarbonData is built using Apache Maven, to [build CarbonData](https://github.com * [CarbonData Pre-aggregate DataMap](https://github.com/apache/carbondata/blob/master/docs/preaggregate-datamap-guide.md) * [CarbonData Timeseries DataMap](https://github.com/apache/carbondata/blob/master/docs/timeseries-datamap-guide.md) * [SDK Guide](https://github.com/apache/carbondata/blob/master/docs/sdk-guide.md) +* [CSDK Guide](https://github.com/apache/carbondata/blob/master/docs/CSDK-guide.md) --- End diff -- please keep the file name lowercase ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r22619
--- Diff:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/RowUtil.java ---
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.sdk.file;
+
+import java.io.Serializable;
+import java.math.BigDecimal;
+
+/**
+ * This row class is used to transfer the row data from one step to other
step
+ */
+public class RowUtil implements Serializable {
+
+ public static String getString(Object[] data, int ordinal) {
+return (String) data[ordinal];
+ }
+
+ /**
+ * get short type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return short data type data
+ */
+ public static short getShort(Object[] data, int ordinal) {
+return (short) data[ordinal];
+ }
+
+ /**
+ * get int data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return int data type data
+ */
+ public static int getInt(Object[] data, int ordinal) {
+return (Integer) data[ordinal];
+ }
+
+ /**
+ * get long data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return long data type data
+ */
+ public static long getLong(Object[] data, int ordinal) {
+return (long) data[ordinal];
+ }
+
+ /**
+ * get array data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return array data type data
+ */
+ public static Object[] getArray(Object[] data, int ordinal) {
+return (Object[]) data[ordinal];
+ }
+
+ /**
+ * get double data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return double data type data
+ */
+ public static double getDouble(Object[] data, int ordinal) {
+return (double) data[ordinal];
+ }
+
+ /**
+ * get boolean data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return boolean data type data
+ */
+ public static boolean getBoolean(Object[] data, int ordinal) {
+return (boolean) data[ordinal];
+ }
+
+ /**
+ * get byte data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return byte data type data
+ */
+ public static Byte getByte(Object[] data, int ordinal) {
+return (Byte) data[ordinal];
+ }
+
+ /**
+ * get float data type data by ordinal
+ *
+ * @param data carbon row data
+ * @param ordinal the data index of Row
+ * @return float data type data
+ */
+ public static float getFloat(Object[] data, int ordinal) {
+return (float) data[ordinal];
+ }
+
+ /**
+ * get varchar data type data by ordinal
+ * This is for CSDK
+ * JNI don't support varchar, so carbon convert decimal to string
--- End diff --
`convert decimal to string`?
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r226553321
--- Diff: docs/CSDK-guide.md ---
@@ -68,20 +68,42 @@ JNIEnv *initJVM() {
bool readFromLocalWithoutProjection(JNIEnv *env) {
CarbonReader carbonReaderClass;
-carbonReaderClass.builder(env, "../resources/carbondata", "test");
+carbonReaderClass.builder(env, "../resources/carbondata");
carbonReaderClass.build();
+printf("\nRead data from local without projection:\n");
--- End diff --
extra blanks found in `local without`
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r226555131
--- Diff: docs/CSDK-guide.md ---
@@ -68,20 +68,42 @@ JNIEnv *initJVM() {
bool readFromLocalWithoutProjection(JNIEnv *env) {
CarbonReader carbonReaderClass;
-carbonReaderClass.builder(env, "../resources/carbondata", "test");
+carbonReaderClass.builder(env, "../resources/carbondata");
carbonReaderClass.build();
+printf("\nRead data from local without projection:\n");
+
+CarbonRow carbonRow(env);
--- End diff --
oh, I find this in *.cpp.
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r226554587
--- Diff: docs/CSDK-guide.md ---
@@ -68,20 +68,42 @@ JNIEnv *initJVM() {
bool readFromLocalWithoutProjection(JNIEnv *env) {
CarbonReader carbonReaderClass;
-carbonReaderClass.builder(env, "../resources/carbondata", "test");
+carbonReaderClass.builder(env, "../resources/carbondata");
carbonReaderClass.build();
+printf("\nRead data from local without projection:\n");
+
+CarbonRow carbonRow(env);
--- End diff --
What is this? I cannot recognize this grammar.
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xuchuanyin commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r226554812 --- Diff: docs/documentation.md --- @@ -31,7 +31,7 @@ Apache CarbonData is a new big data file format for faster interactive query usi **CarbonData SQL Language Reference:** CarbonData extends the Spark SQL language and adds several [DDL](./ddl-of-carbondata.md) and [DML](./dml-of-carbondata.md) statements to support operations on it.Refer to the [Reference Manual](./language-manual.md) to understand the supported features and functions. -**Programming Guides:** You can read our guides about [APIs supported](./sdk-guide.md) to learn how to integrate CarbonData with your applications. +**Programming Guides:** You can read our guides about [Java APIs supported](./sdk-guide.md) or [C++ APIs supported](./CSDK-guide.md) to learn how to integrate CarbonData with your applications. --- End diff -- remember to optimize the filename of the doc ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r223267582 --- Diff: README.md --- @@ -52,6 +52,7 @@ CarbonData is built using Apache Maven, to [build CarbonData](https://github.com * [Configuring Carbondata](https://github.com/apache/carbondata/blob/master/docs/configuration-parameters.md) * [Streaming Ingestion](https://github.com/apache/carbondata/blob/master/docs/streaming-guide.md) * [SDK Guide](https://github.com/apache/carbondata/blob/master/docs/sdk-guide.md) +* [CSDK Guide](https://github.com/apache/carbondata/blob/master/docs/CSDK-guide.md) --- End diff -- ok, done. I only find documentation.md. CSDK don't support write, so other place no need to update. ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user kunal642 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r223260021 --- Diff: README.md --- @@ -52,6 +52,7 @@ CarbonData is built using Apache Maven, to [build CarbonData](https://github.com * [Configuring Carbondata](https://github.com/apache/carbondata/blob/master/docs/configuration-parameters.md) * [Streaming Ingestion](https://github.com/apache/carbondata/blob/master/docs/streaming-guide.md) * [SDK Guide](https://github.com/apache/carbondata/blob/master/docs/sdk-guide.md) +* [CSDK Guide](https://github.com/apache/carbondata/blob/master/docs/CSDK-guide.md) --- End diff -- Other places also have to be update with this link like documentation.md. can you refer sdk-guide.md and update other places as well. ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r223244987
--- Diff:
hadoop/src/main/java/org/apache/carbondata/hadoop/readsupport/impl/DictionaryDecodeReadSupport.java
---
@@ -81,7 +82,24 @@
data[i] = dictionaries[i].getDictionaryValueForKey((int) data[i]);
}
}
-return (T)data;
+return (T) data;
+ }
+
+ /**
+ * get carbonRow, including data and datatpes
+ *
+ * @param data row data
+ * @return CarbonRow Object
+ */
+ public T readCarbonRow(Object[] data) {
--- End diff --
I add Row class and optimized readCarbonRow method
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r223243327 --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/row/CarbonRow.java --- @@ -18,8 +18,11 @@ package org.apache.carbondata.core.datastore.row; import java.io.Serializable; +import java.math.BigDecimal; --- End diff -- ok, I added ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r223240030
--- Diff: store/CSDK/CarbonReader.cpp ---
@@ -89,10 +89,18 @@ jboolean CarbonReader::hasNext() {
return hasNext;
}
+jobject CarbonReader::readNextCarbonRow() {
+jclass carbonReader = jniEnv->GetObjectClass(carbonReaderObject);
+jmethodID readNextCarbonRowID = jniEnv->GetMethodID(carbonReader,
"readNextCarbonRow",
+"()Lorg/apache/carbondata/core/datastore/row/CarbonRow;");
+jobject carbonRow = (jobject)
jniEnv->CallObjectMethod(carbonReaderObject, readNextCarbonRowID);
+return carbonRow;
+}
+
jobjectArray CarbonReader::readNextRow() {
jclass carbonReader = jniEnv->GetObjectClass(carbonReaderObject);
-jmethodID readNextRow2ID = jniEnv->GetMethodID(carbonReader,
"readNextStringRow", "()[Ljava/lang/Object;");
-jobjectArray row = (jobjectArray)
jniEnv->CallObjectMethod(carbonReaderObject, readNextRow2ID);
+jmethodID readNextStringRowID = jniEnv->GetMethodID(carbonReader,
"readNextStringRow", "()[Ljava/lang/Object;");
--- End diff --
I removed readNextStringRow in java code and readNextRow in c code
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r223238726
--- Diff:
hadoop/src/main/java/org/apache/carbondata/hadoop/CarbonRecordReader.java ---
@@ -116,6 +117,25 @@ public void initialize(InputSplit inputSplit,
TaskAttemptContext context)
return readSupport.readRow(carbonIterator.next());
}
+ /**
+ * get CarbonRow data, including data and datatypes
+ *
+ * @return carbonRow object or data array or T
+ * @throws IOException
+ * @throws InterruptedException
+ */
+ public T getCarbonRow() throws IOException, InterruptedException {
--- End diff --
ok, done
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r223238343
--- Diff:
core/src/main/java/org/apache/carbondata/core/datastore/row/CarbonRow.java ---
@@ -57,6 +74,154 @@ public String getString(int ordinal) {
return (String) data[ordinal];
}
+ /**
+ * get short type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public short getShort(int ordinal) {
+return (short) data[ordinal];
+ }
+
+ /**
+ * get int data type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public int getInt(int ordinal) {
+return (Integer) data[ordinal];
+ }
+
+ /**
+ * get long data type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public long getLong(int ordinal) {
+return (long) data[ordinal];
+ }
+
+ /**
+ * get array data type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public Object[] getArray(int ordinal) {
+return (Object[]) data[ordinal];
+ }
+
+ /**
+ * get double data type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public double getDouble(int ordinal) {
+return (double) data[ordinal];
+ }
+
+ /**
+ * get boolean data type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public boolean getBoolean(int ordinal) {
+return (boolean) data[ordinal];
+ }
+
+ /**
+ * get byte data type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public Byte getByte(int ordinal) {
+return (Byte) data[ordinal];
+ }
+
+ /**
+ * get float data type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public float getFloat(int ordinal) {
+return (float) data[ordinal];
+ }
+
+ /**
+ * get varchar data type data by ordinal
+ * This is for CSDK
+ * JNI don't support varchar, so carbon convert decimal to string
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public String getVarchar(int ordinal) {
+return (String) data[ordinal];
+ }
+
+ /**
+ * get decimal data type data by ordinal
+ * This is for CSDK
+ * JNI don't support Decimal, so carbon convert decimal to string
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public String getDecimal(int ordinal) {
+return ((BigDecimal) data[ordinal]).toString();
+ }
+
+ /**
+ * get data type by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public DataType getDataType(int ordinal) {
+return dataTypes[ordinal];
+ }
+
+ /**
+ * get data type name by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public String getDataTypeName(int ordinal) {
+return dataTypes[ordinal].getName();
+ }
+
+ /**
+ * get element type name by ordinal
+ * child schema data type name
+ * for example: return STRING if it's Array in java
+ *
+ * @param ordinal the data index of carbonRow
+ * @return element type name
+ */
+ public String getElementTypeName(int ordinal) {
--- End diff --
ok, done
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user KanakaKumar commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r222691292
--- Diff: store/CSDK/CarbonReader.cpp ---
@@ -89,10 +89,18 @@ jboolean CarbonReader::hasNext() {
return hasNext;
}
+jobject CarbonReader::readNextCarbonRow() {
+jclass carbonReader = jniEnv->GetObjectClass(carbonReaderObject);
+jmethodID readNextCarbonRowID = jniEnv->GetMethodID(carbonReader,
"readNextCarbonRow",
+"()Lorg/apache/carbondata/core/datastore/row/CarbonRow;");
+jobject carbonRow = (jobject)
jniEnv->CallObjectMethod(carbonReaderObject, readNextCarbonRowID);
+return carbonRow;
+}
+
jobjectArray CarbonReader::readNextRow() {
jclass carbonReader = jniEnv->GetObjectClass(carbonReaderObject);
-jmethodID readNextRow2ID = jniEnv->GetMethodID(carbonReader,
"readNextStringRow", "()[Ljava/lang/Object;");
-jobjectArray row = (jobjectArray)
jniEnv->CallObjectMethod(carbonReaderObject, readNextRow2ID);
+jmethodID readNextStringRowID = jniEnv->GetMethodID(carbonReader,
"readNextStringRow", "()[Ljava/lang/Object;");
--- End diff --
We can remove "readNextStringRow" and add a utility method in JNI to
achieve the same.
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user KanakaKumar commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2792#discussion_r222691023 --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/row/CarbonRow.java --- @@ -18,8 +18,11 @@ package org.apache.carbondata.core.datastore.row; import java.io.Serializable; +import java.math.BigDecimal; --- End diff -- CarbonRow has different fields like data, rawData, rangeID etc. It seems not intended for end user API. I think we can add a simple Row class for SDK scope. ---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user KanakaKumar commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r222690123
--- Diff:
hadoop/src/main/java/org/apache/carbondata/hadoop/readsupport/impl/DictionaryDecodeReadSupport.java
---
@@ -81,7 +82,24 @@
data[i] = dictionaries[i].getDictionaryValueForKey((int) data[i]);
}
}
-return (T)data;
+return (T) data;
+ }
+
+ /**
+ * get carbonRow, including data and datatpes
+ *
+ * @param data row data
+ * @return CarbonRow Object
+ */
+ public T readCarbonRow(Object[] data) {
--- End diff --
Instead of changing the DictionaryDecodeReadSupport & other classes
hierarchy, I suggest to use a new Row class as utility and just provide
required methods to avoid impact on base code.
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user KanakaKumar commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r222689040
--- Diff:
hadoop/src/main/java/org/apache/carbondata/hadoop/CarbonRecordReader.java ---
@@ -116,6 +117,25 @@ public void initialize(InputSplit inputSplit,
TaskAttemptContext context)
return readSupport.readRow(carbonIterator.next());
}
+ /**
+ * get CarbonRow data, including data and datatypes
+ *
+ * @return carbonRow object or data array or T
+ * @throws IOException
+ * @throws InterruptedException
+ */
+ public T getCarbonRow() throws IOException, InterruptedException {
--- End diff --
I think instead of confusing T, we can define the return type as CarbonRow
itself
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
Github user KanakaKumar commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2792#discussion_r222686566
--- Diff:
core/src/main/java/org/apache/carbondata/core/datastore/row/CarbonRow.java ---
@@ -57,6 +74,154 @@ public String getString(int ordinal) {
return (String) data[ordinal];
}
+ /**
+ * get short type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public short getShort(int ordinal) {
+return (short) data[ordinal];
+ }
+
+ /**
+ * get int data type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public int getInt(int ordinal) {
+return (Integer) data[ordinal];
+ }
+
+ /**
+ * get long data type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public long getLong(int ordinal) {
+return (long) data[ordinal];
+ }
+
+ /**
+ * get array data type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public Object[] getArray(int ordinal) {
+return (Object[]) data[ordinal];
+ }
+
+ /**
+ * get double data type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public double getDouble(int ordinal) {
+return (double) data[ordinal];
+ }
+
+ /**
+ * get boolean data type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public boolean getBoolean(int ordinal) {
+return (boolean) data[ordinal];
+ }
+
+ /**
+ * get byte data type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public Byte getByte(int ordinal) {
+return (Byte) data[ordinal];
+ }
+
+ /**
+ * get float data type data by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public float getFloat(int ordinal) {
+return (float) data[ordinal];
+ }
+
+ /**
+ * get varchar data type data by ordinal
+ * This is for CSDK
+ * JNI don't support varchar, so carbon convert decimal to string
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public String getVarchar(int ordinal) {
+return (String) data[ordinal];
+ }
+
+ /**
+ * get decimal data type data by ordinal
+ * This is for CSDK
+ * JNI don't support Decimal, so carbon convert decimal to string
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public String getDecimal(int ordinal) {
+return ((BigDecimal) data[ordinal]).toString();
+ }
+
+ /**
+ * get data type by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public DataType getDataType(int ordinal) {
+return dataTypes[ordinal];
+ }
+
+ /**
+ * get data type name by ordinal
+ *
+ * @param ordinal the data index of carbonRow
+ * @return
+ */
+ public String getDataTypeName(int ordinal) {
+return dataTypes[ordinal].getName();
+ }
+
+ /**
+ * get element type name by ordinal
+ * child schema data type name
+ * for example: return STRING if it's Array in java
+ *
+ * @param ordinal the data index of carbonRow
+ * @return element type name
+ */
+ public String getElementTypeName(int ordinal) {
--- End diff --
If this method can work only for Array, we can rename it to
getArrayElementTypeName and throw exception if its not array type. return null
cause integration errors for unsupported ata types
---
[GitHub] carbondata pull request #2792: [CARBONDATA-2981] Support read primitive data...
GitHub user xubo245 opened a pull request: https://github.com/apache/carbondata/pull/2792 [CARBONDATA-2981] Support read primitive data type in CSDK [CARBONDATA-2981] Support read primitive data type in CSDK 1.support readNextCarbonRow 2.support read different primitive data type in c code from java side: int double short long string 3.support some data type and convert: date timestamp varchar decimal array 3.1 return int when read date 3.2 return long when read timestamp 3.3 return string when read varchar 3.4 return string when read decimal 3.5 support array This PR is based on PR2738, and will remove related commit after PR2738 merged. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? add new interface - [ ] Any backward compatibility impacted? No - [ ] Document update required? Yes - [ ] Testing done update test case in c code - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. jira 2951 You can merge this pull request into a Git repository by running: $ git pull https://github.com/xubo245/carbondata CARBONDATA-2981_primitiveDataType Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2792.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2792 commit 5f93bfc999dc7309671d59b1e73e4085d2684d58 Author: xubo245 Date: 2018-09-20T10:35:34Z [CARBONDATA-2952] Provide CarbonReader C++ interface for SDK 1.init carbonreader,config data path and tablename 2.config ak sk endpoing for S3 3.configure projection 4.build carbon reader 5.hasNext 6.readNextRow 7.close optimize commit cd181b91c33d32e66a3f0026f1e3167a148b37e7 Author: xubo245 Date: 2018-09-29T09:06:03Z [CARBONDATA-2981] Support read primitive data type in CSDK 1.support readNextCarbonRow 2.support read different primitive data type in c code from java side: int double short long string 3.support some data type and convert: date timestamp varchar decimal array su ---
