[GitHub] [carbondata] zzcclp commented on a change in pull request #3553: [CARBONDATA-3648] Support Alter Table Compaction Level Threshold
zzcclp commented on a change in pull request #3553: [CARBONDATA-3648] Support
Alter Table Compaction Level Threshold
URL: https://github.com/apache/carbondata/pull/3553#discussion_r362294232
##
File path:
core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java
##
@@ -81,7 +81,7 @@ public static String getFolderContainingFile(String
carbonFilePath) {
int lastIndex = carbonFilePath.lastIndexOf('/');
// below code for handling windows environment
if (-1 == lastIndex) {
- lastIndex =
carbonFilePath.lastIndexOf(CarbonCommonConstants.FILE_SEPARATOR);
+ lastIndex =
carbonFilePath.lastIndexOf(CarbonCommonConstants.WINDOWS_FILE_SEPARATOR);
Review comment:
in the linux env, is lastIndex right if use 'WINDOWS_FILE_SEPARATOR'?
it better set the right separator according to system env, for example: use
java.io.File.separator
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert array('')/array() into Struct column …
CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert
array('')/array() into Struct column …
URL: https://github.com/apache/carbondata/pull/3549#issuecomment-569972329
Build Success with Spark 2.3.4, Please check CI
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1404/
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert array('')/array() into Struct column …
CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert
array('')/array() into Struct column …
URL: https://github.com/apache/carbondata/pull/3549#issuecomment-569970289
Build Failed with Spark 2.2.1, Please check CI
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1392/
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3550: [WIP] Remove global dictionary in query
CarbonDataQA1 commented on issue #3550: [WIP] Remove global dictionary in query URL: https://github.com/apache/carbondata/pull/3550#issuecomment-569965113 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1393/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3550: [WIP] Remove global dictionary in query
CarbonDataQA1 commented on issue #3550: [WIP] Remove global dictionary in query URL: https://github.com/apache/carbondata/pull/3550#issuecomment-569964967 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1405/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3550: [WIP] Remove global dictionary in query
CarbonDataQA1 commented on issue #3550: [WIP] Remove global dictionary in query URL: https://github.com/apache/carbondata/pull/3550#issuecomment-569964761 Build Failed with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1383/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert array('')/array() into Struct column …
CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert
array('')/array() into Struct column …
URL: https://github.com/apache/carbondata/pull/3549#issuecomment-569963957
Build Success with Spark 2.1.0, Please check CI
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1382/
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] marchpure commented on a change in pull request #3553: [CARBONDATA-3648] Support Alter Table Compaction Level Threshold
marchpure commented on a change in pull request #3553: [CARBONDATA-3648]
Support Alter Table Compaction Level Threshold
URL: https://github.com/apache/carbondata/pull/3553#discussion_r362250557
##
File path:
core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java
##
@@ -81,7 +81,7 @@ public static String getFolderContainingFile(String
carbonFilePath) {
int lastIndex = carbonFilePath.lastIndexOf('/');
// below code for handling windows environment
if (-1 == lastIndex) {
- lastIndex =
carbonFilePath.lastIndexOf(CarbonCommonConstants.FILE_SEPARATOR);
+ lastIndex =
carbonFilePath.lastIndexOf(CarbonCommonConstants.WINDOWS_FILE_SEPARATOR);
Review comment:
carbonFilePath is in type of '/root/schema', while it is in type of
'D:\schema'. The separator is different, dividing into FILE_SEPARATOR ('/') and
WINDOWS_FILE_SEPARATOR ('\').
In the Windows Env. The variant 'lastIndex' is always -1,
carbonFilePath.substring(0, lastIndex) will throw SubStringIndexOutOfBound
Exception.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] marchpure commented on a change in pull request #3553: [CARBONDATA-3648] Support Alter Table Compaction Level Threshold
marchpure commented on a change in pull request #3553: [CARBONDATA-3648]
Support Alter Table Compaction Level Threshold
URL: https://github.com/apache/carbondata/pull/3553#discussion_r362250557
##
File path:
core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java
##
@@ -81,7 +81,7 @@ public static String getFolderContainingFile(String
carbonFilePath) {
int lastIndex = carbonFilePath.lastIndexOf('/');
// below code for handling windows environment
if (-1 == lastIndex) {
- lastIndex =
carbonFilePath.lastIndexOf(CarbonCommonConstants.FILE_SEPARATOR);
+ lastIndex =
carbonFilePath.lastIndexOf(CarbonCommonConstants.WINDOWS_FILE_SEPARATOR);
Review comment:
carbonFilePath is in type of '/root/schema', while it is in type of
'D:\schema'. The separator is different, dividing into FILE_SEPARATOR ('/') and
WINDOWS_FILE_SEPARATOR ('\').
In the Windows Env. The variant 'lastIndex' is always -1,
carbonFilePath.substring(0, lastIndex) will be throw SubStringIndexOutOfBound
Exception.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] zzcclp commented on a change in pull request #3553: [CARBONDATA-3648] Support Alter Table Compaction Level Threshold
zzcclp commented on a change in pull request #3553: [CARBONDATA-3648] Support
Alter Table Compaction Level Threshold
URL: https://github.com/apache/carbondata/pull/3553#discussion_r362247975
##
File path:
core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java
##
@@ -81,7 +81,7 @@ public static String getFolderContainingFile(String
carbonFilePath) {
int lastIndex = carbonFilePath.lastIndexOf('/');
// below code for handling windows environment
if (-1 == lastIndex) {
- lastIndex =
carbonFilePath.lastIndexOf(CarbonCommonConstants.FILE_SEPARATOR);
+ lastIndex =
carbonFilePath.lastIndexOf(CarbonCommonConstants.WINDOWS_FILE_SEPARATOR);
Review comment:
why it need to use windows file separator?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3553: [CARBONDATA-3648] Support Alter Table Compaction Level Threshold
CarbonDataQA1 commented on issue #3553: [CARBONDATA-3648] Support Alter Table Compaction Level Threshold URL: https://github.com/apache/carbondata/pull/3553#issuecomment-569957033 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1402/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3518: [DOC] add performance-tuning with codegen parameters support
CarbonDataQA1 commented on issue #3518: [DOC] add performance-tuning with codegen parameters support URL: https://github.com/apache/carbondata/pull/3518#issuecomment-569952924 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1403/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3518: [DOC] add performance-tuning with codegen parameters support
CarbonDataQA1 commented on issue #3518: [DOC] add performance-tuning with codegen parameters support URL: https://github.com/apache/carbondata/pull/3518#issuecomment-569951199 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1391/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3553: [CARBONDATA-3648] Support Alter Table Compaction Level Threshold
CarbonDataQA1 commented on issue #3553: [CARBONDATA-3648] Support Alter Table Compaction Level Threshold URL: https://github.com/apache/carbondata/pull/3553#issuecomment-569944040 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1390/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3518: [DOC] add performance-tuning with codegen parameters support
CarbonDataQA1 commented on issue #3518: [DOC] add performance-tuning with codegen parameters support URL: https://github.com/apache/carbondata/pull/3518#issuecomment-569944042 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1381/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] MarvinLitt commented on a change in pull request #3518: [DOC] add performance-tuning with codegen parameters support
MarvinLitt commented on a change in pull request #3518: [DOC] add performance-tuning with codegen parameters support URL: https://github.com/apache/carbondata/pull/3518#discussion_r362229446 ## File path: docs/performance-tuning.md ## @@ -173,6 +173,8 @@ | carbon.sort.temp.compressor | spark/carbonlib/carbon.properties | Data loading | Specify the name of compressor to compress the intermediate sort temporary files during sort procedure in data loading. | The optional values are 'SNAPPY','GZIP','BZIP2','LZ4','ZSTD', and empty. Specially, empty means that Carbondata will not compress the sort temp files. This parameter will be useful if you encounter disk bottleneck. | | carbon.load.skewedDataOptimization.enabled | spark/carbonlib/carbon.properties | Data loading | Whether to enable size based block allocation strategy for data loading. | When loading, carbondata will use file size based block allocation strategy for task distribution. It will make sure that all the executors process the same size of data -- It's useful if the size of your input data files varies widely, say 1MB to 1GB. | | carbon.load.min.size.enabled | spark/carbonlib/carbon.properties | Data loading | Whether to enable node minumun input data size allocation strategy for data loading.| When loading, carbondata will use node minumun input data size allocation strategy for task distribution. It will make sure the nodes load the minimum amount of data -- It's useful if the size of your input data files very small, say 1MB to 256MB,Avoid generating a large number of small files. | +| spark.sql.codegen.wholeStage | spark/conf/spark-defaults.conf | Querying | improves the execution performance of a query by collapsing a query tree into a single optimized function that eliminates virtual function calls and leverages CPU registers for intermediate data. | The whole stage CodeGen mechanism introduced by spark SQL in version 2. X causes. This configuration is recommended to be off at spark 2.1 and on at spark 2.3. Because under spark2.1 user can only use spark.sql.codegen.wholeStage to control whether to use codegen, but can not config the size of the method. In fact, this parameter should be configured to be the same as the local JDK. Under spark2.3 support spark.sql.codegen.hugeMethodLimit use can use that to config the method size. | Review comment: i has move spark sql codegen optimization guid line to a new md file named query-with-spark-sql-performacne-tuning. please check. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loade
CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 10, giving Null pointer exception URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569941202 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1401/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3553: [CARBONDATA-3648] Support Alter Table Compaction Level Threshold
CarbonDataQA1 commented on issue #3553: [CARBONDATA-3648] Support Alter Table Compaction Level Threshold URL: https://github.com/apache/carbondata/pull/3553#issuecomment-569937203 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1380/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loade
CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 10, giving Null pointer exception URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569936628 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1389/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] marchpure opened a new pull request #3553: [CARBONDATA-3648] Support Alter Table Compaction Level Threshold
marchpure opened a new pull request #3553: [CARBONDATA-3648] Support Alter Table Compaction Level Threshold URL: https://github.com/apache/carbondata/pull/3553 ### Why is this PR needed? (1) The Alter Table sould support Compaction Level Threshold. (2) The upper limit of Compaction Level Threshold is 100, which is too small to meet the scenario with massive small files (3) There is a bug to limit alter table in windows env. ### What changes were proposed in this PR? (1) AlterTableUtil support alter Compaction_Level_Threshold (2) CarbonProperties increases the upper limit of Compaction_Level_Threshold to 1 from 100. (3) Fix the bug which limit alter table in windows env ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (CARBONDATA-3648) Support Alter Table Compaction Level Threshold
Xingjun Hao created CARBONDATA-3648: --- Summary: Support Alter Table Compaction Level Threshold Key: CARBONDATA-3648 URL: https://issues.apache.org/jira/browse/CARBONDATA-3648 Project: CarbonData Issue Type: Improvement Reporter: Xingjun Hao The Alter Table sould support Compaction Level Threshold. Also, the upper limit is 100, which is too small to meet the scenario with massive small files -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loade
CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 10, giving Null pointer exception URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569927454 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1379/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loade
CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 10, giving Null pointer exception URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569924691 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1384/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loa
vikramahuja1001 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 10, giving Null pointer exception URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569923726 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loade
CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 10, giving Null pointer exception URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569920825 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1400/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3546: [CARBONDATA-3642] Add column idx in error msg when string length exceed 32000
CarbonDataQA1 commented on issue #3546: [CARBONDATA-3642] Add column idx in error msg when string length exceed 32000 URL: https://github.com/apache/carbondata/pull/3546#issuecomment-569919488 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1398/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loade
CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 10, giving Null pointer exception URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569918773 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1388/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on a change in pull request #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with In
vikramahuja1001 commented on a change in pull request #3537: [CARBONDATA-3646]
[CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter
query failure with Index server when loaded with global_sort_partition =
10, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#discussion_r362203886
##
File path:
integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
##
@@ -45,6 +47,8 @@ import
org.apache.carbondata.processing.loading.events.LoadEvents.{LoadMetadataE
import org.apache.carbondata.spark.rdd.SparkReadSupport
import org.apache.carbondata.spark.readsupport.SparkRowReadSupportImpl
+
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on a change in pull request #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with In
vikramahuja1001 commented on a change in pull request #3537: [CARBONDATA-3646]
[CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter
query failure with Index server when loaded with global_sort_partition =
10, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#discussion_r362203892
##
File path:
integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
##
@@ -125,6 +130,21 @@ class CarbonEnv {
LOGGER.info("Initialize CarbonEnv completed...")
}
+
+ def createIndexServerTmp(sparkSession: SparkSession): Unit = {
+val conf = sparkSession.sparkContext.hadoopConfiguration
+val fs = FileSystem.get(conf)
+if (fs.getUri.toString.startsWith("hdfs://")) {
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] kunal642 commented on a change in pull request #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index ser
kunal642 commented on a change in pull request #3537: [CARBONDATA-3646]
[CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter
query failure with Index server when loaded with global_sort_partition =
10, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#discussion_r362202583
##
File path:
integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
##
@@ -125,6 +130,21 @@ class CarbonEnv {
LOGGER.info("Initialize CarbonEnv completed...")
}
+
+ def createIndexServerTmp(sparkSession: SparkSession): Unit = {
+val conf = sparkSession.sparkContext.hadoopConfiguration
+val fs = FileSystem.get(conf)
+if (fs.getUri.toString.startsWith("hdfs://")) {
Review comment:
Use CarbonUtil.checkAndAppendFileSystemURIScheme to add URI to the path
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] kunal642 commented on a change in pull request #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index ser
kunal642 commented on a change in pull request #3537: [CARBONDATA-3646]
[CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter
query failure with Index server when loaded with global_sort_partition =
10, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#discussion_r362201899
##
File path:
integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
##
@@ -45,6 +47,8 @@ import
org.apache.carbondata.processing.loading.events.LoadEvents.{LoadMetadataE
import org.apache.carbondata.spark.rdd.SparkReadSupport
import org.apache.carbondata.spark.readsupport.SparkRowReadSupportImpl
+
Review comment:
revert this change
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3552: [CARBONDATA-3644] Support Configuration of Complex Delimiters in Carbon…
CarbonDataQA1 commented on issue #3552: [CARBONDATA-3644] Support Configuration of Complex Delimiters in Carbon… URL: https://github.com/apache/carbondata/pull/3552#issuecomment-569913742 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1397/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3546: [CARBONDATA-3642] Add column idx in error msg when string length exceed 32000
CarbonDataQA1 commented on issue #3546: [CARBONDATA-3642] Add column idx in error msg when string length exceed 32000 URL: https://github.com/apache/carbondata/pull/3546#issuecomment-569913567 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1387/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loade
CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 10, giving Null pointer exception URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569912856 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1378/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3552: [CARBONDATA-3644] Support Configuration of Complex Delimiters in Carbon…
CarbonDataQA1 commented on issue #3552: [CARBONDATA-3644] Support Configuration of Complex Delimiters in Carbon… URL: https://github.com/apache/carbondata/pull/3552#issuecomment-569912037 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1386/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (CARBONDATA-3647) On a 1 million rows table loaded with "global_sort_partitions"="100000", a filter query will get a failure Job on IndexServer
Vikram Ahuja created CARBONDATA-3647: Summary: On a 1 million rows table loaded with "global_sort_partitions"="10", a filter query will get a failure Job on IndexServer Key: CARBONDATA-3647 URL: https://issues.apache.org/jira/browse/CARBONDATA-3647 Project: CarbonData Issue Type: Bug Components: spark-integration Affects Versions: 2.0.0 Reporter: Vikram Ahuja Fix For: 2.0.0 Filter query failure with Index server when loaded with global_sort_partition = 10 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CARBONDATA-3646) Select query * fails when using index server
Vikram Ahuja created CARBONDATA-3646: Summary: Select query * fails when using index server Key: CARBONDATA-3646 URL: https://issues.apache.org/jira/browse/CARBONDATA-3646 Project: CarbonData Issue Type: Bug Components: spark-integration Affects Versions: 2.0.0 Reporter: Vikram Ahuja Fix For: 2.0.0 Select * query does not complete in the Index server and goes to fallback mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert array('')/array() into Struct column …
CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert
array('')/array() into Struct column …
URL: https://github.com/apache/carbondata/pull/3549#issuecomment-569910170
Build Failed with Spark 2.3.4, Please check CI
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1396/
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert array('')/array() into Struct column …
CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert
array('')/array() into Struct column …
URL: https://github.com/apache/carbondata/pull/3549#issuecomment-569910147
Build Failed with Spark 2.2.1, Please check CI
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1385/
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert array('')/array() into Struct column …
CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert
array('')/array() into Struct column …
URL: https://github.com/apache/carbondata/pull/3549#issuecomment-569909911
Build Success with Spark 2.1.0, Please check CI
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1375/
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on issue #3537: Metacache issues
vikramahuja1001 commented on issue #3537: Metacache issues URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569908950 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3546: [CARBONDATA-3642] Add column idx in error msg when string length exceed 32000
CarbonDataQA1 commented on issue #3546: [CARBONDATA-3642] Add column idx in error msg when string length exceed 32000 URL: https://github.com/apache/carbondata/pull/3546#issuecomment-569907489 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1377/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3551: [CARBONDATA-3645] BadRecords are inserted as NULL when column is of complex data type and BAD_RECORDS_ACTION is IGNORE
CarbonDataQA1 commented on issue #3551: [CARBONDATA-3645] BadRecords are inserted as NULL when column is of complex data type and BAD_RECORDS_ACTION is IGNORE URL: https://github.com/apache/carbondata/pull/3551#issuecomment-569906343 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1393/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3552: [CARBONDATA-3644] Support Configuration of Complex Delimiters in Carbon…
CarbonDataQA1 commented on issue #3552: [CARBONDATA-3644] Support Configuration of Complex Delimiters in Carbon… URL: https://github.com/apache/carbondata/pull/3552#issuecomment-569904813 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1376/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3551: [CARBONDATA-3645] BadRecords are inserted as NULL when column is of complex data type and BAD_RECORDS_ACTION is IGNORE
CarbonDataQA1 commented on issue #3551: [CARBONDATA-3645] BadRecords are inserted as NULL when column is of complex data type and BAD_RECORDS_ACTION is IGNORE URL: https://github.com/apache/carbondata/pull/3551#issuecomment-569903705 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1383/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3537: Metacache issues
CarbonDataQA1 commented on issue #3537: Metacache issues URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569900901 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1395/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (CARBONDATA-3631) StringIndexOutOfBoundsException When Inserting Select From a Parquet Table with Empty array/map
[
https://issues.apache.org/jira/browse/CARBONDATA-3631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17006022#comment-17006022
]
Hong Shen edited comment on CARBONDATA-3631 at 12/31/19 9:52 AM:
-
Sorry for late, when I prepare to raise a pr, I found it was fix by another
commit . https://github.com/apache/carbondata/pull/3545
was (Author: shenhong):
Sorry for late, when I prepare to raise a pr, I found it was fix by another
commit .
> StringIndexOutOfBoundsException When Inserting Select From a Parquet Table
> with Empty array/map
> ---
>
> Key: CARBONDATA-3631
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3631
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.6.1, 2.0.0
>Reporter: Xingjun Hao
>Priority: Minor
> Fix For: 2.0.0
>
>
> sql("insert into datatype_array_parquet values(array())")
> sql("insert into datatype_array_carbondata select f from
> datatype_array_parquet")
>
> {code:java}
> java.lang.StringIndexOutOfBoundsException: String index out of range: -1
> at java.lang.AbstractStringBuilder.substring(AbstractStringBuilder.java:935)
> at java.lang.StringBuilder.substring(StringBuilder.java:76)
> at scala.collection.mutable.StringBuilder.substring(StringBuilder.scala:166)
> at
> org.apache.carbondata.streaming.parser.FieldConverter$.objectToString(FieldConverter.scala:77)
> at
> org.apache.carbondata.spark.util.CarbonScalaUtil$.getString(CarbonScalaUtil.scala:71)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [carbondata] marchpure opened a new pull request #3552: [CARBONDATA-3644] Support Configuration of Complex Delimiters in Carbon…
marchpure opened a new pull request #3552: [CARBONDATA-3644] Support Configuration of Complex Delimiters in Carbon… URL: https://github.com/apache/carbondata/pull/3552 … Properties ### Why is this PR needed? Modification reason: In the insert carbontable select from a parquet table processing, if the binary column has the content '\001', like 'col1\001col2', the content before '\001' will be truncated as '\001' is the Complex Delimiter. The problem is that Complex Delimiter can't be configured in the insert flow, which needs to improve. ### What changes were proposed in this PR? Modification content: We add the configuration of complex delimiters in the CarbonProperties, which will be loaded in the LoadOption. ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (CARBONDATA-3631) StringIndexOutOfBoundsException When Inserting Select From a Parquet Table with Empty array/map
[
https://issues.apache.org/jira/browse/CARBONDATA-3631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17006022#comment-17006022
]
Hong Shen commented on CARBONDATA-3631:
---
Sorry for late, when I prepare to raise a pr, I found it was fix by another
commit .
> StringIndexOutOfBoundsException When Inserting Select From a Parquet Table
> with Empty array/map
> ---
>
> Key: CARBONDATA-3631
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3631
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.6.1, 2.0.0
>Reporter: Xingjun Hao
>Priority: Minor
> Fix For: 2.0.0
>
>
> sql("insert into datatype_array_parquet values(array())")
> sql("insert into datatype_array_carbondata select f from
> datatype_array_parquet")
>
> {code:java}
> java.lang.StringIndexOutOfBoundsException: String index out of range: -1
> at java.lang.AbstractStringBuilder.substring(AbstractStringBuilder.java:935)
> at java.lang.StringBuilder.substring(StringBuilder.java:76)
> at scala.collection.mutable.StringBuilder.substring(StringBuilder.scala:166)
> at
> org.apache.carbondata.streaming.parser.FieldConverter$.objectToString(FieldConverter.scala:77)
> at
> org.apache.carbondata.spark.util.CarbonScalaUtil$.getString(CarbonScalaUtil.scala:71)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3537: Metacache issues
CarbonDataQA1 commented on issue #3537: Metacache issues URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569898197 Build Failed with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1374/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on issue #3537: Metacache issues
vikramahuja1001 commented on issue #3537: Metacache issues URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569896692 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3551: [CARBONDATA-3645] BadRecords are inserted as NULL when column is of complex data type and BAD_RECORDS_ACTION is IGNORE
CarbonDataQA1 commented on issue #3551: [CARBONDATA-3645] BadRecords are inserted as NULL when column is of complex data type and BAD_RECORDS_ACTION is IGNORE URL: https://github.com/apache/carbondata/pull/3551#issuecomment-569896368 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1373/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] Indhumathi27 opened a new pull request #3551: [CARBONDATA-3645] BadRecords are inserted as NULL when column is of complex data type and BAD_RECORDS_ACTION is IGNORE
Indhumathi27 opened a new pull request #3551: [CARBONDATA-3645] BadRecords are inserted as NULL when column is of complex data type and BAD_RECORDS_ACTION is IGNORE URL: https://github.com/apache/carbondata/pull/3551 ### Why is this PR needed? In case if `BAD_RECORDS_ACTION` is `IGNORE`, Carbon should skip those bad records while loading. But in case of Complex data type, badrecords are inserted as NULL. Here `badRecordAdded` flag is not set, from second BadRecord found,since message is already added to `columnMessageMap` for first found BadRecord. ### What changes were proposed in this PR? Set `badRecordAdded` flag, if badRecord is found. ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (CARBONDATA-3645) BadRecords are inserted as NULL when column is of complex data type and BAD_RECORDS_ACTION is IGNORE
[ https://issues.apache.org/jira/browse/CARBONDATA-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated CARBONDATA-3645: - Summary: BadRecords are inserted as NULL when column is of complex data type and BAD_RECORDS_ACTION is IGNORE (was: Bad record data is inserted as NULL if column datatype is of complex type and BAD_RECORDS_ACTION is IGNORE) > BadRecords are inserted as NULL when column is of complex data type and > BAD_RECORDS_ACTION is IGNORE > > > Key: CARBONDATA-3645 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3645 > Project: CarbonData > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CARBONDATA-3645) Bad record data is inserted as NULL if column datatype is of complex type and BAD_RECORDS_ACTION is IGNORE
Indhumathi Muthumurugesh created CARBONDATA-3645: Summary: Bad record data is inserted as NULL if column datatype is of complex type and BAD_RECORDS_ACTION is IGNORE Key: CARBONDATA-3645 URL: https://issues.apache.org/jira/browse/CARBONDATA-3645 Project: CarbonData Issue Type: Bug Reporter: Indhumathi Muthumurugesh -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert array('')/array() into Struct column …
CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert
array('')/array() into Struct column …
URL: https://github.com/apache/carbondata/pull/3549#issuecomment-569889786
Build Success with Spark 2.3.4, Please check CI
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1391/
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (CARBONDATA-3644) Support Configuration of Complex Delimiters in Carbon Properties
Xingjun Hao created CARBONDATA-3644: --- Summary: Support Configuration of Complex Delimiters in Carbon Properties Key: CARBONDATA-3644 URL: https://issues.apache.org/jira/browse/CARBONDATA-3644 Project: CarbonData Issue Type: Improvement Reporter: Xingjun Hao In the insert carbontable select from a parquet table processing, if the binary column has the content '\001', like 'col1\001col2', the content before '\001' will be truncated as '\001' is the Complex Delimiter. The problem is that Complex Delimiter can't be configured in the insert flow, which needs to improve. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3546: [CARBONDATA-3642] Add column idx in error msg when string length exceed 32000
CarbonDataQA1 commented on issue #3546: [CARBONDATA-3642] Add column idx in error msg when string length exceed 32000 URL: https://github.com/apache/carbondata/pull/3546#issuecomment-569889012 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1382/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3550: [WIP] Remove global dictionary in query
CarbonDataQA1 commented on issue #3550: [WIP] Remove global dictionary in query URL: https://github.com/apache/carbondata/pull/3550#issuecomment-569888087 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1380/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3550: [WIP] Remove global dictionary in query
CarbonDataQA1 commented on issue #3550: [WIP] Remove global dictionary in query URL: https://github.com/apache/carbondata/pull/3550#issuecomment-569887678 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1390/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3546: [CARBONDATA-3642] Add column idx in error msg when string length exceed 32000
CarbonDataQA1 commented on issue #3546: [CARBONDATA-3642] Add column idx in error msg when string length exceed 32000 URL: https://github.com/apache/carbondata/pull/3546#issuecomment-569887598 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1392/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert array('')/array() into Struct column …
CarbonDataQA1 commented on issue #3549: [Carbondata-3643] Insert
array('')/array() into Struct column …
URL: https://github.com/apache/carbondata/pull/3549#issuecomment-569886182
Build Failed with Spark 2.2.1, Please check CI
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1381/
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
