[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3817: [CARBONDATA-3845] Bucket table creation fails with exception for empt…
ajantha-bhat commented on a change in pull request #3817:
URL: https://github.com/apache/carbondata/pull/3817#discussion_r453465582
##
File path:
integration/spark/src/main/scala/org/apache/spark/sql/parser/CarbonSpark2SqlParser.scala
##
@@ -766,13 +766,13 @@ class CarbonSpark2SqlParser extends CarbonDDLSqlParser {
throw new MalformedCarbonCommandException("Invalid table properties")
}
if (options.isBucketingEnabled) {
- if (options.bucketNumber.toString.contains("-") ||
- options.bucketNumber.toString.contains("+") || options.bucketNumber
== 0) {
+ if (options.bucketNumber.isEmpty ||
+ options.bucketNumber.get.toString.contains("-") ||
options.bucketNumber.get == 0) {
Review comment:
Just keep `options.bucketNumber.get <= 0`
it can avoid two condition check and redundant string conversion
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3772: [CARBONDATA-3832]Added block and blocket pruning for the polygon expression processing
CarbonDataQA1 commented on pull request #3772: URL: https://github.com/apache/carbondata/pull/3772#issuecomment-657388621 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1623/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3772: [CARBONDATA-3832]Added block and blocket pruning for the polygon expression processing
CarbonDataQA1 commented on pull request #3772: URL: https://github.com/apache/carbondata/pull/3772#issuecomment-657388354 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3364/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3836: [CARBONDATA-3897] Fixed external table with location creation and deleting issues
ajantha-bhat commented on a change in pull request #3836:
URL: https://github.com/apache/carbondata/pull/3836#discussion_r453458380
##
File path:
core/src/main/java/org/apache/carbondata/core/datastore/impl/FileFactory.java
##
@@ -620,8 +620,9 @@ public static boolean checkIfPrefixExists(String path) {
}
final String lowerPath = path.toLowerCase(Locale.getDefault());
-return lowerPath.contains("://") ||
lowerPath.startsWith(CarbonCommonConstants.HDFSURL_PREFIX)
-|| lowerPath.startsWith(CarbonCommonConstants.VIEWFSURL_PREFIX) ||
lowerPath
+return lowerPath.contains(":/") || lowerPath.contains("://") || lowerPath
Review comment:
why `lowerPath.contains(":/")` is added ? It is for which file system ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on pull request #3822: [CARBONDATA-3887] Fixed insert failure for global sort null data
akashrn5 commented on pull request #3822: URL: https://github.com/apache/carbondata/pull/3822#issuecomment-657383823 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] marchpure commented on a change in pull request #3839: [CARBONDATA-3898] Support Option 'carbon.enable.querywithmv'
marchpure commented on a change in pull request #3839:
URL: https://github.com/apache/carbondata/pull/3839#discussion_r453433678
##
File path:
core/src/main/java/org/apache/carbondata/core/util/CarbonProperties.java
##
@@ -1762,6 +1762,20 @@ public boolean isDistributedPruningEnabled(String
dbName, String tableName) {
return isServerEnabledByUser;
}
+ /**
+ * Check whether the MV is enabled by the user or not.
+ */
+ public boolean isMVEnabled() {
+String mvEnabled = CarbonProperties.getInstance().getProperty(
+CarbonCommonConstants.CARBON_ENABLE_QUERYWITHMV);
+if (mvEnabled == null) {
+ return
Boolean.parseBoolean(CarbonCommonConstants.CARBON_ENABLE_QUERYWITHMV_DEFAULT);
+} else {
+ // return false only if false is set. any other case return true
+ return !mvEnabled.equalsIgnoreCase("false");
Review comment:
modified
##
File path:
integration/spark/src/test/scala/org/apache/carbondata/view/MVTest.scala
##
@@ -63,6 +63,30 @@ class MVTest extends QueryTest with BeforeAndAfterAll {
sql("drop table source")
}
+ test("test disable mv with carbonproperties and sessionparam") {
+assert(CarbonProperties.getInstance().isMVEnabled)
+sql("drop materialized view if exists mv1")
+sql("drop table if exists source")
+sql("create table source as select * from fact_table")
+sql("create materialized view mv1 as select empname, deptname, avg(salary)
from source group by empname, deptname")
+
+
CarbonProperties.getInstance().addProperty(CarbonCommonConstants.CARBON_ENABLE_QUERYWITHMV,"false")
+var df = sql("select empname, avg(salary) from source group by empname")
+assert(!isTableAppearedInPlan(df.queryExecution.optimizedPlan, "mv1"))
+
+df = sql("select empname, avg(salary) from source group by empname")
+
CarbonProperties.getInstance().removeProperty(CarbonCommonConstants.CARBON_ENABLE_QUERYWITHMV)
+assert(isTableAppearedInPlan(df.queryExecution.optimizedPlan, "mv1"))
+
+sql("set carbon.mv.enable = false")
Review comment:
modified
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] marchpure commented on a change in pull request #3839: [CARBONDATA-3898] Support Option 'carbon.enable.querywithmv'
marchpure commented on a change in pull request #3839:
URL: https://github.com/apache/carbondata/pull/3839#discussion_r453433760
##
File path:
integration/spark/src/main/scala/org/apache/spark/sql/optimizer/MVRewriteRule.scala
##
@@ -98,14 +98,8 @@ class MVRewriteRule(session: SparkSession) extends
Rule[LogicalPlan] {
if (!canApply) {
return logicalPlan
}
-val sessionInformation = ThreadLocalSessionInfo.getCarbonSessionInfo
-if (sessionInformation != null && sessionInformation.getThreadParams !=
null) {
- val disableViewRewrite = sessionInformation.getThreadParams.getProperty(
-CarbonCommonConstants.DISABLE_SQL_REWRITE)
- if (disableViewRewrite != null &&
-disableViewRewrite.equalsIgnoreCase("true")) {
-return logicalPlan
- }
Review comment:
checked, some testcases are added
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] brijoobopanna commented on pull request #3772: [CARBONDATA-3832]Added block and blocket pruning for the polygon expression processing
brijoobopanna commented on pull request #3772: URL: https://github.com/apache/carbondata/pull/3772#issuecomment-657349617 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] kevinjmh commented on a change in pull request #3839: [CARBONDATA-3898] Support Option 'carbon.enable.querywithmv'
kevinjmh commented on a change in pull request #3839:
URL: https://github.com/apache/carbondata/pull/3839#discussion_r453394545
##
File path:
integration/spark/src/main/scala/org/apache/spark/sql/optimizer/MVRewriteRule.scala
##
@@ -98,14 +98,8 @@ class MVRewriteRule(session: SparkSession) extends
Rule[LogicalPlan] {
if (!canApply) {
return logicalPlan
}
-val sessionInformation = ThreadLocalSessionInfo.getCarbonSessionInfo
-if (sessionInformation != null && sessionInformation.getThreadParams !=
null) {
- val disableViewRewrite = sessionInformation.getThreadParams.getProperty(
-CarbonCommonConstants.DISABLE_SQL_REWRITE)
- if (disableViewRewrite != null &&
-disableViewRewrite.equalsIgnoreCase("true")) {
-return logicalPlan
- }
Review comment:
please check this changes again, for the case multiple sessions with
different setting
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] kevinjmh commented on a change in pull request #3839: [CARBONDATA-3898] Support Option 'carbon.enable.querywithmv'
kevinjmh commented on a change in pull request #3839:
URL: https://github.com/apache/carbondata/pull/3839#discussion_r453393117
##
File path:
integration/spark/src/test/scala/org/apache/carbondata/view/MVTest.scala
##
@@ -63,6 +63,30 @@ class MVTest extends QueryTest with BeforeAndAfterAll {
sql("drop table source")
}
+ test("test disable mv with carbonproperties and sessionparam") {
+assert(CarbonProperties.getInstance().isMVEnabled)
+sql("drop materialized view if exists mv1")
+sql("drop table if exists source")
+sql("create table source as select * from fact_table")
+sql("create materialized view mv1 as select empname, deptname, avg(salary)
from source group by empname, deptname")
+
+
CarbonProperties.getInstance().addProperty(CarbonCommonConstants.CARBON_ENABLE_QUERYWITHMV,"false")
+var df = sql("select empname, avg(salary) from source group by empname")
+assert(!isTableAppearedInPlan(df.queryExecution.optimizedPlan, "mv1"))
+
+df = sql("select empname, avg(salary) from source group by empname")
+
CarbonProperties.getInstance().removeProperty(CarbonCommonConstants.CARBON_ENABLE_QUERYWITHMV)
+assert(isTableAppearedInPlan(df.queryExecution.optimizedPlan, "mv1"))
+
+sql("set carbon.mv.enable = false")
Review comment:
not consist with the property name added. And I think this one is better
than `carbon.enable.querywithmv`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] kevinjmh commented on a change in pull request #3839: [CARBONDATA-3898] Support Option 'carbon.enable.querywithmv'
kevinjmh commented on a change in pull request #3839:
URL: https://github.com/apache/carbondata/pull/3839#discussion_r453392507
##
File path:
core/src/main/java/org/apache/carbondata/core/util/CarbonProperties.java
##
@@ -1762,6 +1762,20 @@ public boolean isDistributedPruningEnabled(String
dbName, String tableName) {
return isServerEnabledByUser;
}
+ /**
+ * Check whether the MV is enabled by the user or not.
+ */
+ public boolean isMVEnabled() {
+String mvEnabled = CarbonProperties.getInstance().getProperty(
+CarbonCommonConstants.CARBON_ENABLE_QUERYWITHMV);
+if (mvEnabled == null) {
+ return
Boolean.parseBoolean(CarbonCommonConstants.CARBON_ENABLE_QUERYWITHMV_DEFAULT);
+} else {
+ // return false only if false is set. any other case return true
+ return !mvEnabled.equalsIgnoreCase("false");
Review comment:
```suggestion
return mvEnabled.equalsIgnoreCase("true");
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3836: [CARBONDATA-3897] Fixed external table with location creation and deleting issues
CarbonDataQA1 commented on pull request #3836: URL: https://github.com/apache/carbondata/pull/3836#issuecomment-657295113 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1622/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3836: [CARBONDATA-3897] Fixed external table with location creation and deleting issues
CarbonDataQA1 commented on pull request #3836: URL: https://github.com/apache/carbondata/pull/3836#issuecomment-657294855 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3363/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3836: [CARBONDATA-3897] Fixed external table with location creation and deleting issues
CarbonDataQA1 commented on pull request #3836: URL: https://github.com/apache/carbondata/pull/3836#issuecomment-657278659 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1621/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3839: [CARBONDATA-3898] Support Option 'carbon.enable.querywithmv'
CarbonDataQA1 commented on pull request #3839: URL: https://github.com/apache/carbondata/pull/3839#issuecomment-657271385 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3361/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3839: [CARBONDATA-3898] Support Option 'carbon.enable.querywithmv'
CarbonDataQA1 commented on pull request #3839: URL: https://github.com/apache/carbondata/pull/3839#issuecomment-657271121 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1620/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3836: [CARBONDATA-3897] Fixed external table with location creation and deleting issues
CarbonDataQA1 commented on pull request #3836: URL: https://github.com/apache/carbondata/pull/3836#issuecomment-657265035 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1619/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3836: [CARBONDATA-3897] Fixed external table with location creation and deleting issues
CarbonDataQA1 commented on pull request #3836: URL: https://github.com/apache/carbondata/pull/3836#issuecomment-657264650 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3360/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3838: [WIP]Fix load failure in cluster when csv present in local file system in case of global sort
CarbonDataQA1 commented on pull request #3838: URL: https://github.com/apache/carbondata/pull/3838#issuecomment-657258575 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1618/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3838: [WIP]Fix load failure in cluster when csv present in local file system in case of global sort
CarbonDataQA1 commented on pull request #3838: URL: https://github.com/apache/carbondata/pull/3838#issuecomment-657258474 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3359/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3837: [wip]remove compressor name from tupleID
CarbonDataQA1 commented on pull request #3837: URL: https://github.com/apache/carbondata/pull/3837#issuecomment-657256916 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3358/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3837: [wip]remove compressor name from tupleID
CarbonDataQA1 commented on pull request #3837: URL: https://github.com/apache/carbondata/pull/3837#issuecomment-657256684 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1617/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] marchpure opened a new pull request #3839: [CARBONDATA-3898] Support Option 'carbon.enable.querywithmv'
marchpure opened a new pull request #3839: URL: https://github.com/apache/carbondata/pull/3839 ### Why is this PR needed? When MV enabled, SQL rewrite takes a lot of time, a new option 'carbon.enable.querywithmv' shall be supported, which can turn off SQL Rewrite when the configured value is false ### What changes were proposed in this PR? Add option 'carbon.enable.querywithmv', then logicplan won't be changed if configured value is false ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (CARBONDATA-3898) Support Option 'carbon.enable.querywithmv'
Xingjun Hao created CARBONDATA-3898: --- Summary: Support Option 'carbon.enable.querywithmv' Key: CARBONDATA-3898 URL: https://issues.apache.org/jira/browse/CARBONDATA-3898 Project: CarbonData Issue Type: New Feature Reporter: Xingjun Hao When MV enabled, SQL rewrite takes a lot of time, a new option 'carbon.enable.querywithmv' shall be supported, which can turn off SQL Rewrite when the configured value is false -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] kunal642 commented on pull request #3836: [CARBONDATA-3897] Fixed external table with location creation and deleting issues
kunal642 commented on pull request #3836: URL: https://github.com/apache/carbondata/pull/3836#issuecomment-657250639 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #3833: [CARBONDATA-3894] [IUD]decrease the size of tableupdatestaus file by remove the invalid segments not exist in tablestatus
asfgit closed pull request #3833: URL: https://github.com/apache/carbondata/pull/3833 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #3832: [CARBONDATA-3893] [IUD] Fix getting block name in compacted segment with dot for horizontal compaction delta files
asfgit closed pull request #3832: URL: https://github.com/apache/carbondata/pull/3832 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] jackylk commented on pull request #3832: [CARBONDATA-3893] [IUD] Fix getting block name in compacted segment with dot for horizontal compaction delta files
jackylk commented on pull request #3832: URL: https://github.com/apache/carbondata/pull/3832#issuecomment-657249430 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3835: [CARBONDATA-3852] [CARBONDATA-3851] Fix Data update issue with SCD/CDC for partition table when directlyWriteData2Hdfs is enabled
CarbonDataQA1 commented on pull request #3835: URL: https://github.com/apache/carbondata/pull/3835#issuecomment-657242838 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1616/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 opened a new pull request #3838: [WIP]Fix load failure in cluster when csv present in local file system in case of global sort
akashrn5 opened a new pull request #3838: URL: https://github.com/apache/carbondata/pull/3838 ### Why is this PR needed? ### What changes were proposed in this PR? ### Does this PR introduce any user interface change? - No - Yes. (please explain the change and update document) ### Is any new testcase added? - No - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3835: [CARBONDATA-3852] [CARBONDATA-3851] Fix Data update issue with SCD/CDC for partition table when directlyWriteData2Hdfs is enabled
CarbonDataQA1 commented on pull request #3835: URL: https://github.com/apache/carbondata/pull/3835#issuecomment-657241951 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3357/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 opened a new pull request #3837: [wip]remove compressor name from tupleID
akashrn5 opened a new pull request #3837: URL: https://github.com/apache/carbondata/pull/3837 ### Why is this PR needed? ### What changes were proposed in this PR? ### Does this PR introduce any user interface change? - No - Yes. (please explain the change and update document) ### Is any new testcase added? - No - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3835: [CARBONDATA-3852] [CARBONDATA-3851] Fix Data update issue with SCD/CDC for partition table when directlyWriteData2Hdfs is enabled
CarbonDataQA1 commented on pull request #3835: URL: https://github.com/apache/carbondata/pull/3835#issuecomment-657220952 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3356/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3835: [CARBONDATA-3852] [CARBONDATA-3851] Fix Data update issue with SCD/CDC for partition table when directlyWriteData2Hdfs is enabled
CarbonDataQA1 commented on pull request #3835: URL: https://github.com/apache/carbondata/pull/3835#issuecomment-657220725 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1615/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3836: [CARBONDATA-3897] Fixed external table with location creation and deleting issues
CarbonDataQA1 commented on pull request #3836: URL: https://github.com/apache/carbondata/pull/3836#issuecomment-657218542 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1614/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3836: [CARBONDATA-3897] Fixed external table with location creation and deleting issues
CarbonDataQA1 commented on pull request #3836: URL: https://github.com/apache/carbondata/pull/3836#issuecomment-657218442 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3355/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (CARBONDATA-3888) Should move .flattened-pom.xml to target folder
[ https://issues.apache.org/jira/browse/CARBONDATA-3888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manhua Jiang resolved CARBONDATA-3888. -- Fix Version/s: 2.1.0 Resolution: Fixed > Should move .flattened-pom.xml to target folder > --- > > Key: CARBONDATA-3888 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3888 > Project: CarbonData > Issue Type: Improvement >Reporter: David Cai >Priority: Minor > Fix For: 2.1.0 > > Time Spent: 50m > Remaining Estimate: 0h > > after .flattened-pom.xml is generated in the project folder, it will impact > the project import of Intellij idea -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] dependabot[bot] commented on pull request #3447: Bump dep.jackson.version from 2.6.5 to 2.10.1 in /store/sdk
dependabot[bot] commented on pull request #3447: URL: https://github.com/apache/carbondata/pull/3447#issuecomment-657216713 Dependabot tried to update this pull request, but something went wrong. We're looking into it, but in the meantime you can retry the update by commenting `@dependabot rebase`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] dependabot[bot] commented on pull request #3456: Bump solr.version from 6.3.0 to 8.3.0 in /datamap/lucene
dependabot[bot] commented on pull request #3456: URL: https://github.com/apache/carbondata/pull/3456#issuecomment-657216710 Dependabot tried to update this pull request, but something went wrong. We're looking into it, but in the meantime you can retry the update by commenting `@dependabot rebase`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #3824: [CARBONDATA-3888] Move .flattened-pom.xml to target folder
asfgit closed pull request #3824: URL: https://github.com/apache/carbondata/pull/3824 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] kunal642 commented on a change in pull request #3822: [CARBONDATA-3887] Fixed insert failure for global sort null data
kunal642 commented on a change in pull request #3822:
URL: https://github.com/apache/carbondata/pull/3822#discussion_r453299220
##
File path:
integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
##
@@ -921,52 +921,55 @@ object CommonUtil {
var i = 0
val fieldTypesLen = fields.length
while (i < fieldTypesLen) {
- if (!row.isNullAt(i)) {
-fields(i).dataType match {
- case StringType =>
-data(i) =
DataTypeUtil.getBytesDataDataTypeForNoDictionaryColumn(row.getString(i),
+ fields(i).dataType match {
Review comment:
It is already handled for other types..Test cases is added
##
File path:
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/dataload/TestGlobalSortDataLoad.scala
##
@@ -468,6 +470,16 @@ class TestGlobalSortDataLoad extends QueryTest with
BeforeAndAfterEach with Befo
sql("SELECT * FROM carbon_localsort_difftypes ORDER BY shortField"))
}
+ test("test global sort with null values") {
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] kunal642 opened a new pull request #3836: [CARBONDATA-3897] Fixed external table with location creation and deleting issues
kunal642 opened a new pull request #3836: URL: https://github.com/apache/carbondata/pull/3836 ### Why is this PR needed? 1. Creating external table without Fs schema fails on HDFS 2. Desc formatted on an external table throws StackOverflowException ### What changes were proposed in this PR? 1. Added default FS Schema to the table path 2. Added a check for non-transactional table in refreshIndexInfo method. ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (CARBONDATA-3897) desc formatted on an external table throws StackOverFlow Exception
Kunal Kapoor created CARBONDATA-3897: Summary: desc formatted on an external table throws StackOverFlow Exception Key: CARBONDATA-3897 URL: https://issues.apache.org/jira/browse/CARBONDATA-3897 Project: CarbonData Issue Type: Bug Affects Versions: 2.0.0, 2.0.1 Reporter: Kunal Kapoor Assignee: Kunal Kapoor Fix For: 2.1.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
