[GitHub] [carbondata] CarbonDataQA1 commented on issue #3565: [CARBONDATA-3662]: Changes to show metacache command
CarbonDataQA1 commented on issue #3565: [CARBONDATA-3662]: Changes to show metacache command URL: https://github.com/apache/carbondata/pull/3565#issuecomment-575026364 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1655/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: Fix query failure with Index Server
CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: Fix query failure with Index Server URL: https://github.com/apache/carbondata/pull/3537#issuecomment-575026446 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1656/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: Fix query failure with Index Server
vikramahuja1001 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: Fix query failure with Index Server URL: https://github.com/apache/carbondata/pull/3537#issuecomment-575024427 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on issue #3565: [CARBONDATA-3662]: Changes to show metacache command
vikramahuja1001 commented on issue #3565: [CARBONDATA-3662]: Changes to show metacache command URL: https://github.com/apache/carbondata/pull/3565#issuecomment-575024665 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3579: [CARBONDATA-3664]Add SchemaEvolutionEntry after alter set sort columns
CarbonDataQA1 commented on issue #3579: [CARBONDATA-3664]Add SchemaEvolutionEntry after alter set sort columns URL: https://github.com/apache/carbondata/pull/3579#issuecomment-575017250 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1654/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (CARBONDATA-3592) Query on bloom datamap fails in case of multiple data files in one segment
[ https://issues.apache.org/jira/browse/CARBONDATA-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akash R Nilugal resolved CARBONDATA-3592. - Fix Version/s: 2.0.0 Resolution: Fixed > Query on bloom datamap fails in case of multiple data files in one segment > -- > > Key: CARBONDATA-3592 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3592 > Project: CarbonData > Issue Type: Bug >Reporter: Shivam Goyal >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 6h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] asfgit closed pull request #3474: [CARBONDATA-3592] Fix query on bloom in case of multiple data files in one segment
asfgit closed pull request #3474: [CARBONDATA-3592] Fix query on bloom in case of multiple data files in one segment URL: https://github.com/apache/carbondata/pull/3474 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3565: [CARBONDATA-3662]: Changes to show metacache command
CarbonDataQA1 commented on issue #3565: [CARBONDATA-3662]: Changes to show metacache command URL: https://github.com/apache/carbondata/pull/3565#issuecomment-575005149 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1653/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on a change in pull request #3565: [CARBONDATA-3662]: Changes to show metacache command
vikramahuja1001 commented on a change in pull request #3565: [CARBONDATA-3662]:
Changes to show metacache command
URL: https://github.com/apache/carbondata/pull/3565#discussion_r367249750
##
File path:
integration/spark-carbon-common-test/src/test/scala/org/apache/carbondata/sql/commands/TestCarbonShowCacheCommand.scala
##
@@ -175,13 +175,13 @@ class TestCarbonShowCacheCommand extends QueryTest with
BeforeAndAfterAll {
// Empty database
sql("use cache_empty_db").collect()
val result1 = sql("show metacache").collect()
-assertResult(2)(result1.length)
-assertResult(Row("cache_empty_db", "ALL", "0 B", "0 B", "0 B",
"DRIVER"))(result1(1))
+sql("show metacache").show()
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on issue #3565: [CARBONDATA-3662]: Changes to show metacache command
vikramahuja1001 commented on issue #3565: [CARBONDATA-3662]: Changes to show metacache command URL: https://github.com/apache/carbondata/pull/3565#issuecomment-575003820 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on a change in pull request #3565: [CARBONDATA-3662]: Changes to show metacache command
vikramahuja1001 commented on a change in pull request #3565: [CARBONDATA-3662]:
Changes to show metacache command
URL: https://github.com/apache/carbondata/pull/3565#discussion_r367249802
##
File path:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/cache/CarbonShowCacheCommand.scala
##
@@ -169,41 +224,32 @@ case class CarbonShowCacheCommand(tableIdentifier:
Option[TableIdentifier],
}
val (driverIndexSize, driverDatamapSize, allDictSize) =
getAllDriverCacheSize(tablePaths
.toList)
- if (driverRows.nonEmpty) {
+ if (driverIndexSize + driverDatamapSize + allDictSize != 0 &&
driverRows.size!=0) {
(Seq(
- Row("ALL", "ALL", driverIndexSize, driverDatamapSize, allDictSize,
"DRIVER"),
- Row(currentDatabase,
-"ALL",
-driverdbIndexSize,
-driverdbDatamapSize,
-driverdbDictSize,
-"DRIVER")
+ Row("TOTAL", driverIndexSize, driverDatamapSize, allDictSize,
"DRIVER")
) ++ driverRows).collect {
- case row if row.getLong(2) != 0L || row.getLong(3) != 0L ||
row.getLong(4) != 0L =>
-Row(row(0), row(1), bytesToDisplaySize(row.getLong(2)),
- bytesToDisplaySize(row.getLong(3)),
bytesToDisplaySize(row.getLong(4)), "DRIVER")
+ case row if row.getLong(1) != 0L || row.getLong(2) != 0L ||
row.getLong(3) != 0L =>
+Row(row(0), bytesToDisplaySize(row.getLong(1)),
+ bytesToDisplaySize(row.getLong(2)),
bytesToDisplaySize(row.getLong(3)), "DRIVER")
}
} else {
-makeEmptyCacheRows(currentDatabase)
+Seq()
}
} else {
- makeEmptyCacheRows(currentDatabase)
+ Seq()
}
-// val (serverIndexSize, serverDataMapSize) =
getAllIndexServerCacheSize
-val indexDisplayRows = if (indexServerRows.nonEmpty) {
+val indexDisplayRows = if (indexAllIndexSize + indexAllDatamapSize != 0 &&
+ indexServerRows.size != 0) {
(Seq(
-Row("ALL", "ALL", indexAllIndexSize, indexAllDatamapSize,
indexAllDictSize, "INDEX SERVER"),
-Row(currentDatabase,
- "ALL",
- indexdbIndexSize,
- indexdbDatamapSize,
- driverdbDictSize,
- "INDEX SERVER")
+Row("TOTAL", indexAllIndexSize, indexAllDatamapSize, indexAllDictSize,
"INDEX SERVER")
) ++ indexServerRows).collect {
-case row if row.getLong(2) != 0L || row.getLong(3) != 0L ||
row.getLong(4) != 0L =>
- Row(row.get(0), row.get(1), bytesToDisplaySize(row.getLong(2)),
-bytesToDisplaySize(row.getLong(3)),
bytesToDisplaySize(row.getLong(4)), "INDEX SERVER")
+case row if row.getLong(1) != 0L || row.getLong(2) != 0L ||
row.getLong(3) != 0L =>
+ Row(row.get(0),
+bytesToDisplaySize(row.getLong(1)),
+bytesToDisplaySize(row.getLong(2)),
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on a change in pull request #3565: [CARBONDATA-3662]: Changes to show metacache command
vikramahuja1001 commented on a change in pull request #3565: [CARBONDATA-3662]:
Changes to show metacache command
URL: https://github.com/apache/carbondata/pull/3565#discussion_r367249817
##
File path:
integration/spark2/src/main/scala/org/apache/spark/sql/parser/CarbonSpark2SqlParser.scala
##
@@ -560,9 +560,9 @@ class CarbonSpark2SqlParser extends CarbonDDLSqlParser {
}
protected lazy val showCache: Parser[LogicalPlan] =
-SHOW ~> METACACHE ~> opt(ontable) <~ opt(";") ^^ {
- case table =>
-CarbonShowCacheCommand(table)
+(SHOW ~> opt(EXECUTOR) <~ METACACHE) ~ opt(ontable) <~ opt(";") ^^ {
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on a change in pull request #3565: [CARBONDATA-3662]: Changes to show metacache command
vikramahuja1001 commented on a change in pull request #3565: [CARBONDATA-3662]:
Changes to show metacache command
URL: https://github.com/apache/carbondata/pull/3565#discussion_r367249765
##
File path:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/cache/CarbonShowCacheCommand.scala
##
@@ -110,13 +132,41 @@ case class CarbonShowCacheCommand(tableIdentifier:
Option[TableIdentifier],
}
}
+ def getAllExecutorCache(sparkSession: SparkSession): Seq[Row] = {
+val isDistributedPruningEnabled = CarbonProperties.getInstance()
+ .isDistributedPruningEnabled("", "")
+if (!isDistributedPruningEnabled) {
+ // Block here. this feature is only with index server enabled
+ throw new UnsupportedOperationException(
+"Show Executor Metacache is only avalable with Index Server Enabled")
+ Seq()
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: Fix query failure with Index Server
CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: Fix query failure with Index Server URL: https://github.com/apache/carbondata/pull/3537#issuecomment-575001422 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1652/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: Fix query failure with Index Server
vikramahuja1001 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: Fix query failure with Index Server URL: https://github.com/apache/carbondata/pull/3537#issuecomment-574999691 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: Fix query failure with Index Server
vikramahuja1001 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: Fix query failure with Index Server URL: https://github.com/apache/carbondata/pull/3537#issuecomment-574999652 @kunal642 , done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3466: [CARBONDATA-3586] [CARBONDATA-3587] [CARBONDATA-3595]:Adding valid segments into segments to be refreshed map before inserting segments to
CarbonDataQA1 commented on issue #3466: [CARBONDATA-3586] [CARBONDATA-3587] [CARBONDATA-3595]:Adding valid segments into segments to be refreshed map before inserting segments to index server URL: https://github.com/apache/carbondata/pull/3466#issuecomment-574998909 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1651/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] akashrn5 commented on issue #3474: [CARBONDATA-3592] Fix query on bloom in case of multiple data files in one segment
akashrn5 commented on issue #3474: [CARBONDATA-3592] Fix query on bloom in case of multiple data files in one segment URL: https://github.com/apache/carbondata/pull/3474#issuecomment-574997761 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] vikramahuja1001 commented on issue #3466: [CARBONDATA-3586] [CARBONDATA-3587] [CARBONDATA-3595]:Adding valid segments into segments to be refreshed map before inserting segments
vikramahuja1001 commented on issue #3466: [CARBONDATA-3586] [CARBONDATA-3587] [CARBONDATA-3595]:Adding valid segments into segments to be refreshed map before inserting segments to index server URL: https://github.com/apache/carbondata/pull/3466#issuecomment-574997286 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] kunal642 commented on a change in pull request #3565: [CARBONDATA-3662]: Changes to show metacache command
kunal642 commented on a change in pull request #3565: [CARBONDATA-3662]:
Changes to show metacache command
URL: https://github.com/apache/carbondata/pull/3565#discussion_r367232302
##
File path:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/cache/CarbonShowCacheCommand.scala
##
@@ -87,7 +108,8 @@ case class CarbonShowCacheCommand(tableIdentifier:
Option[TableIdentifier],
val indexRawResults = if
(CarbonProperties.getInstance().isDistributedPruningEnabled
(tableIdentifier.get.database.getOrElse(sparkSession.catalog.currentDatabase),
tableIdentifier.get.table)) {
-getTableCacheFromIndexServer(carbonTable,
numberOfIndexFiles)(sparkSession)
+getTableCacheFromIndexServer(carbonTable,
+ numberOfIndexFiles)(showExecutorCache)(sparkSession)
Review comment:
Revert this change
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] kunal642 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: Fix query failure with Index Server
kunal642 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: Fix query failure with Index Server URL: https://github.com/apache/carbondata/pull/3537#issuecomment-574988863 @vikramahuja1001 Please rebase This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] asfgit closed pull request #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
asfgit closed pull request #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation URL: https://github.com/apache/carbondata/pull/3566 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] kunal642 commented on issue #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
kunal642 commented on issue #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation URL: https://github.com/apache/carbondata/pull/3566#issuecomment-574987460 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3561: [HOTFIX] Fix INSERT STAGE footer read error
CarbonDataQA1 commented on issue #3561: [HOTFIX] Fix INSERT STAGE footer read error URL: https://github.com/apache/carbondata/pull/3561#issuecomment-574987391 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1650/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] kunal642 commented on issue #3561: [HOTFIX] Fix INSERT STAGE footer read error
kunal642 commented on issue #3561: [HOTFIX] Fix INSERT STAGE footer read error URL: https://github.com/apache/carbondata/pull/3561#issuecomment-574986035 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] kunal642 commented on a change in pull request #3565: [CARBONDATA-3662]: Changes to show metacache command
kunal642 commented on a change in pull request #3565: [CARBONDATA-3662]:
Changes to show metacache command
URL: https://github.com/apache/carbondata/pull/3565#discussion_r367233023
##
File path:
integration/spark2/src/main/scala/org/apache/spark/sql/parser/CarbonSpark2SqlParser.scala
##
@@ -560,9 +560,9 @@ class CarbonSpark2SqlParser extends CarbonDDLSqlParser {
}
protected lazy val showCache: Parser[LogicalPlan] =
-SHOW ~> METACACHE ~> opt(ontable) <~ opt(";") ^^ {
- case table =>
-CarbonShowCacheCommand(table)
+(SHOW ~> opt(EXECUTOR) <~ METACACHE) ~ opt(ontable) <~ opt(";") ^^ {
Review comment:
add this DDL change in documentation
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] kunal642 commented on a change in pull request #3565: [CARBONDATA-3662]: Changes to show metacache command
kunal642 commented on a change in pull request #3565: [CARBONDATA-3662]:
Changes to show metacache command
URL: https://github.com/apache/carbondata/pull/3565#discussion_r367232863
##
File path:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/cache/CarbonShowCacheCommand.scala
##
@@ -169,41 +224,32 @@ case class CarbonShowCacheCommand(tableIdentifier:
Option[TableIdentifier],
}
val (driverIndexSize, driverDatamapSize, allDictSize) =
getAllDriverCacheSize(tablePaths
.toList)
- if (driverRows.nonEmpty) {
+ if (driverIndexSize + driverDatamapSize + allDictSize != 0 &&
driverRows.size!=0) {
(Seq(
- Row("ALL", "ALL", driverIndexSize, driverDatamapSize, allDictSize,
"DRIVER"),
- Row(currentDatabase,
-"ALL",
-driverdbIndexSize,
-driverdbDatamapSize,
-driverdbDictSize,
-"DRIVER")
+ Row("TOTAL", driverIndexSize, driverDatamapSize, allDictSize,
"DRIVER")
) ++ driverRows).collect {
- case row if row.getLong(2) != 0L || row.getLong(3) != 0L ||
row.getLong(4) != 0L =>
-Row(row(0), row(1), bytesToDisplaySize(row.getLong(2)),
- bytesToDisplaySize(row.getLong(3)),
bytesToDisplaySize(row.getLong(4)), "DRIVER")
+ case row if row.getLong(1) != 0L || row.getLong(2) != 0L ||
row.getLong(3) != 0L =>
+Row(row(0), bytesToDisplaySize(row.getLong(1)),
+ bytesToDisplaySize(row.getLong(2)),
bytesToDisplaySize(row.getLong(3)), "DRIVER")
}
} else {
-makeEmptyCacheRows(currentDatabase)
+Seq()
}
} else {
- makeEmptyCacheRows(currentDatabase)
+ Seq()
}
-// val (serverIndexSize, serverDataMapSize) =
getAllIndexServerCacheSize
-val indexDisplayRows = if (indexServerRows.nonEmpty) {
+val indexDisplayRows = if (indexAllIndexSize + indexAllDatamapSize != 0 &&
+ indexServerRows.size != 0) {
(Seq(
-Row("ALL", "ALL", indexAllIndexSize, indexAllDatamapSize,
indexAllDictSize, "INDEX SERVER"),
-Row(currentDatabase,
- "ALL",
- indexdbIndexSize,
- indexdbDatamapSize,
- driverdbDictSize,
- "INDEX SERVER")
+Row("TOTAL", indexAllIndexSize, indexAllDatamapSize, indexAllDictSize,
"INDEX SERVER")
) ++ indexServerRows).collect {
-case row if row.getLong(2) != 0L || row.getLong(3) != 0L ||
row.getLong(4) != 0L =>
- Row(row.get(0), row.get(1), bytesToDisplaySize(row.getLong(2)),
-bytesToDisplaySize(row.getLong(3)),
bytesToDisplaySize(row.getLong(4)), "INDEX SERVER")
+case row if row.getLong(1) != 0L || row.getLong(2) != 0L ||
row.getLong(3) != 0L =>
+ Row(row.get(0),
+bytesToDisplaySize(row.getLong(1)),
+bytesToDisplaySize(row.getLong(2)),
Review comment:
revert this
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] kunal642 commented on a change in pull request #3565: [CARBONDATA-3662]: Changes to show metacache command
kunal642 commented on a change in pull request #3565: [CARBONDATA-3662]:
Changes to show metacache command
URL: https://github.com/apache/carbondata/pull/3565#discussion_r367232430
##
File path:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/cache/CarbonShowCacheCommand.scala
##
@@ -110,13 +132,41 @@ case class CarbonShowCacheCommand(tableIdentifier:
Option[TableIdentifier],
}
}
+ def getAllExecutorCache(sparkSession: SparkSession): Seq[Row] = {
+val isDistributedPruningEnabled = CarbonProperties.getInstance()
+ .isDistributedPruningEnabled("", "")
+if (!isDistributedPruningEnabled) {
+ // Block here. this feature is only with index server enabled
+ throw new UnsupportedOperationException(
+"Show Executor Metacache is only avalable with Index Server Enabled")
+ Seq()
Review comment:
no need to return Seq() if you are already throwing exception
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] kunal642 commented on a change in pull request #3565: [CARBONDATA-3662]: Changes to show metacache command
kunal642 commented on a change in pull request #3565: [CARBONDATA-3662]:
Changes to show metacache command
URL: https://github.com/apache/carbondata/pull/3565#discussion_r367232302
##
File path:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/cache/CarbonShowCacheCommand.scala
##
@@ -87,7 +108,8 @@ case class CarbonShowCacheCommand(tableIdentifier:
Option[TableIdentifier],
val indexRawResults = if
(CarbonProperties.getInstance().isDistributedPruningEnabled
(tableIdentifier.get.database.getOrElse(sparkSession.catalog.currentDatabase),
tableIdentifier.get.table)) {
-getTableCacheFromIndexServer(carbonTable,
numberOfIndexFiles)(sparkSession)
+getTableCacheFromIndexServer(carbonTable,
+ numberOfIndexFiles)(showExecutorCache)(sparkSession)
Review comment:
Revert this change
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] kunal642 commented on a change in pull request #3565: [CARBONDATA-3662]: Changes to show metacache command
kunal642 commented on a change in pull request #3565: [CARBONDATA-3662]:
Changes to show metacache command
URL: https://github.com/apache/carbondata/pull/3565#discussion_r367227011
##
File path:
integration/spark-carbon-common-test/src/test/scala/org/apache/carbondata/sql/commands/TestCarbonShowCacheCommand.scala
##
@@ -175,13 +175,13 @@ class TestCarbonShowCacheCommand extends QueryTest with
BeforeAndAfterAll {
// Empty database
sql("use cache_empty_db").collect()
val result1 = sql("show metacache").collect()
-assertResult(2)(result1.length)
-assertResult(Row("cache_empty_db", "ALL", "0 B", "0 B", "0 B",
"DRIVER"))(result1(1))
+sql("show metacache").show()
Review comment:
Remove this
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3538: [WIP] Separate Insert and load to later optimize insert.
CarbonDataQA1 commented on issue #3538: [WIP] Separate Insert and load to later optimize insert. URL: https://github.com/apache/carbondata/pull/3538#issuecomment-574957397 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1649/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3479: [CARBONDATA-3271] Integrating deep learning framework TensorFlow
CarbonDataQA1 commented on issue #3479: [CARBONDATA-3271] Integrating deep learning framework TensorFlow URL: https://github.com/apache/carbondata/pull/3479#issuecomment-574956508 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1648/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3580: [WIP] Support TimeBased Cache expiration using Guava Cache
CarbonDataQA1 commented on issue #3580: [WIP] Support TimeBased Cache expiration using Guava Cache URL: https://github.com/apache/carbondata/pull/3580#issuecomment-574939662 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1647/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3580: [WIP] Support TimeBased Cache expiration using Guava Cache
CarbonDataQA1 commented on issue #3580: [WIP] Support TimeBased Cache expiration using Guava Cache URL: https://github.com/apache/carbondata/pull/3580#issuecomment-574696831 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1646/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] Indhumathi27 opened a new pull request #3580: [WIP] Support TimeBased Cache expiration using Guava Cache
Indhumathi27 opened a new pull request #3580: [WIP] Support TimeBased Cache expiration using Guava Cache URL: https://github.com/apache/carbondata/pull/3580 ### Why is this PR needed? ### What changes were proposed in this PR? ### Does this PR introduce any user interface change? - No - Yes. (please explain the change and update document) ### Is any new testcase added? - No - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3538: [WIP] Separate Insert and load to later optimize insert.
CarbonDataQA1 commented on issue #3538: [WIP] Separate Insert and load to later optimize insert. URL: https://github.com/apache/carbondata/pull/3538#issuecomment-574600563 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1645/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3541: [CARBONDATA-3636]Timeseries query is not hitting datamap if granularity in query is given case insensitive
CarbonDataQA1 commented on issue #3541: [CARBONDATA-3636]Timeseries query is not hitting datamap if granularity in query is given case insensitive URL: https://github.com/apache/carbondata/pull/3541#issuecomment-574579295 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1644/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3474: [CARBONDATA-3592] Fix query on bloom in case of multiple data files in one segment
CarbonDataQA1 commented on issue #3474: [CARBONDATA-3592] Fix query on bloom in case of multiple data files in one segment URL: https://github.com/apache/carbondata/pull/3474#issuecomment-574576568 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1643/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3576: [CARBONDATA-3514] Support spark 2.4 integration
CarbonDataQA1 commented on issue #3576: [CARBONDATA-3514] Support spark 2.4 integration URL: https://github.com/apache/carbondata/pull/3576#issuecomment-574566036 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1642/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] jackylk commented on issue #3479: [CARBONDATA-3271] Integrating deep learning framework TensorFlow
jackylk commented on issue #3479: [CARBONDATA-3271] Integrating deep learning framework TensorFlow URL: https://github.com/apache/carbondata/pull/3479#issuecomment-574565741 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] IceMimosa commented on issue #3569: [CARBONDATA-3657]Support alter hive table add columns with complex types
IceMimosa commented on issue #3569: [CARBONDATA-3657]Support alter hive table add columns with complex types URL: https://github.com/apache/carbondata/pull/3569#issuecomment-574555233 @jackylk Done, thanks for reviewing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (CARBONDATA-3657) [FOLLOW-UP] Support alter hive table add columns with complex types
[ https://issues.apache.org/jira/browse/CARBONDATA-3657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ChenKai updated CARBONDATA-3657: Description: FOLLOW-UP CARBONDATA-3628 Alter hive table is not fully supported in carbon, the unsupported are as follows: * Map * Array * Struct * Decimal with precision and scale * Column with comments was:FOLLOW-UP CARBONDATA-3628 > [FOLLOW-UP] Support alter hive table add columns with complex types > --- > > Key: CARBONDATA-3657 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3657 > Project: CarbonData > Issue Type: Bug > Components: spark-integration >Affects Versions: 1.6.1 >Reporter: ChenKai >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > FOLLOW-UP CARBONDATA-3628 > Alter hive table is not fully supported in carbon, the unsupported are as > follows: > * Map > * Array > * Struct > * Decimal with precision and scale > * Column with comments -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] Indhumathi27 commented on issue #3541: [CARBONDATA-3636]Timeseries query is not hitting datamap if granularity in query is given case insensitive
Indhumathi27 commented on issue #3541: [CARBONDATA-3636]Timeseries query is not hitting datamap if granularity in query is given case insensitive URL: https://github.com/apache/carbondata/pull/3541#issuecomment-574549797 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] jackylk commented on a change in pull request #3578: [CARBONDATA-3663] Support loading stage files in batches
jackylk commented on a change in pull request #3578: [CARBONDATA-3663] Support loading stage files in batches URL: https://github.com/apache/carbondata/pull/3578#discussion_r366744149 ## File path: docs/dml-of-carbondata.md ## @@ -316,12 +316,14 @@ CarbonData DML statements are documented here,which includes: You can use this command to insert them into the table, so that making them visible for query. ``` - INSERT INTO STAGE + INSERT INTO STAGE OPTIONS(property_name=property_value, ...) Review comment: please describe all supported options in this document This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] akashrn5 commented on issue #3474: [CARBONDATA-3592] Fix query on bloom in case of multiple data files in one segment
akashrn5 commented on issue #3474: [CARBONDATA-3592] Fix query on bloom in case of multiple data files in one segment URL: https://github.com/apache/carbondata/pull/3474#issuecomment-574546730 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] akashrn5 commented on issue #3474: [CARBONDATA-3592] Fix query on bloom in case of multiple data files in one segment
akashrn5 commented on issue #3474: [CARBONDATA-3592] Fix query on bloom in case of multiple data files in one segment URL: https://github.com/apache/carbondata/pull/3474#issuecomment-574546664 @kunal642 please add the fix done in IndexDataMapRebuildRDD also in the description. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] asfgit closed pull request #3526: [CARBONDATA-3627] C++ SDK support write data withSchemaFile
asfgit closed pull request #3526: [CARBONDATA-3627] C++ SDK support write data withSchemaFile URL: https://github.com/apache/carbondata/pull/3526 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] jackylk commented on issue #3526: [CARBONDATA-3627] C++ SDK support write data withSchemaFile
jackylk commented on issue #3526: [CARBONDATA-3627] C++ SDK support write data withSchemaFile URL: https://github.com/apache/carbondata/pull/3526#issuecomment-574542665 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] jackylk commented on issue #3569: [CARBONDATA-3657]Support alter hive table add columns with complex types
jackylk commented on issue #3569: [CARBONDATA-3657]Support alter hive table add columns with complex types URL: https://github.com/apache/carbondata/pull/3569#issuecomment-574542174 @IceMimosa This PR is ok, can you add a brief description to this PR This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
