[GitHub] [carbondata] nihal0107 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
nihal0107 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773075985 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
CarbonDataQA2 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773076642 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3665/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] areyouokfreejoe commented on a change in pull request #4086: [CARBONDATA-4115] Successful load and insert will return segment ID
areyouokfreejoe commented on a change in pull request #4086:
URL: https://github.com/apache/carbondata/pull/4086#discussion_r569985309
##
File path:
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonInsertIntoCommand.scala
##
@@ -276,7 +280,15 @@ case class CarbonInsertIntoCommand(databaseNameOp:
Option[String],
}
throw ex
}
-Seq.empty
+if(loadResultForReturn!=null && loadResultForReturn.getLoadName!=null) {
+ Seq(Row(loadResultForReturn.getLoadName))
+} else {
+ rowsForReturn
Review comment:
It's not number of rows.
It's segment id from partition table case.
Ok, Comment has been added.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] areyouokfreejoe commented on a change in pull request #4086: [CARBONDATA-4115] Successful load and insert will return segment ID
areyouokfreejoe commented on a change in pull request #4086:
URL: https://github.com/apache/carbondata/pull/4086#discussion_r569985571
##
File path:
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonInsertIntoCommand.scala
##
@@ -276,7 +280,15 @@ case class CarbonInsertIntoCommand(databaseNameOp:
Option[String],
}
throw ex
}
-Seq.empty
+if(loadResultForReturn!=null && loadResultForReturn.getLoadName!=null) {
Review comment:
Ok, I have it fixed and re push.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] areyouokfreejoe commented on a change in pull request #4086: [CARBONDATA-4115] Successful load and insert will return segment ID
areyouokfreejoe commented on a change in pull request #4086:
URL: https://github.com/apache/carbondata/pull/4086#discussion_r569988736
##
File path:
integration/spark/src/test/scala/org/apache/spark/util/CarbonCommandSuite.scala
##
@@ -100,6 +138,56 @@ class CarbonCommandSuite extends QueryTest with
BeforeAndAfterAll {
private lazy val location = CarbonProperties.getStorePath()
+ test("Return segment ID after load and insert") {
+val tableName = "test_table"
+val inputTableName = "csv_table"
+val inputPath = s"$resourcesPath/data_alltypes.csv"
+dropTable(tableName)
+dropTable(inputTableName)
+createAndLoadInputTable(inputTableName, inputPath)
+createTestTable(tableName)
+checkAnswer(sql(
+ s"""
+ | INSERT INTO TABLE $tableName
+ | SELECT shortField, intField, bigintField, doubleField, stringField,
+ | from_unixtime(unix_timestamp(timestampField,'/M/dd'))
timestampField, decimalField,
+ | cast(to_date(from_unixtime(unix_timestamp(dateField,'/M/dd')))
as date), charField
+ | FROM $inputTableName
+ """.stripMargin), Seq(Row("0")))
+checkAnswer(sql(
+ s"LOAD DATA LOCAL INPATH '$inputPath'" +
+ s" INTO TABLE $tableName" +
+ " OPTIONS('FILEHEADER'=" +
+ "'shortField,intField,bigintField,doubleField,stringField," +
+ "timestampField,decimalField,dateField,charField')"), Seq(Row("1")))
Review comment:
In JDBC case, it will return like this
++
|Segment ID|
++
| 0|
++
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] areyouokfreejoe commented on a change in pull request #4086: [CARBONDATA-4115] Successful load and insert will return segment ID
areyouokfreejoe commented on a change in pull request #4086:
URL: https://github.com/apache/carbondata/pull/4086#discussion_r569990919
##
File path:
integration/spark/src/test/scala/org/apache/spark/util/CarbonCommandSuite.scala
##
@@ -100,6 +138,56 @@ class CarbonCommandSuite extends QueryTest with
BeforeAndAfterAll {
private lazy val location = CarbonProperties.getStorePath()
+ test("Return segment ID after load and insert") {
+val tableName = "test_table"
+val inputTableName = "csv_table"
+val inputPath = s"$resourcesPath/data_alltypes.csv"
+dropTable(tableName)
+dropTable(inputTableName)
+createAndLoadInputTable(inputTableName, inputPath)
+createTestTable(tableName)
+checkAnswer(sql(
+ s"""
+ | INSERT INTO TABLE $tableName
+ | SELECT shortField, intField, bigintField, doubleField, stringField,
+ | from_unixtime(unix_timestamp(timestampField,'/M/dd'))
timestampField, decimalField,
+ | cast(to_date(from_unixtime(unix_timestamp(dateField,'/M/dd')))
as date), charField
+ | FROM $inputTableName
+ """.stripMargin), Seq(Row("0")))
+checkAnswer(sql(
+ s"LOAD DATA LOCAL INPATH '$inputPath'" +
+ s" INTO TABLE $tableName" +
+ " OPTIONS('FILEHEADER'=" +
+ "'shortField,intField,bigintField,doubleField,stringField," +
+ "timestampField,decimalField,dateField,charField')"), Seq(Row("1")))
Review comment:
And in spark sql, it will return as just a num string. The retrun is
simple and I think in this case the user can easily catch it and use it as a
input of next step. I don't think the too many word is needed here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] areyouokfreejoe commented on a change in pull request #4086: [CARBONDATA-4115] Successful load and insert will return segment ID
areyouokfreejoe commented on a change in pull request #4086:
URL: https://github.com/apache/carbondata/pull/4086#discussion_r569988736
##
File path:
integration/spark/src/test/scala/org/apache/spark/util/CarbonCommandSuite.scala
##
@@ -100,6 +138,56 @@ class CarbonCommandSuite extends QueryTest with
BeforeAndAfterAll {
private lazy val location = CarbonProperties.getStorePath()
+ test("Return segment ID after load and insert") {
+val tableName = "test_table"
+val inputTableName = "csv_table"
+val inputPath = s"$resourcesPath/data_alltypes.csv"
+dropTable(tableName)
+dropTable(inputTableName)
+createAndLoadInputTable(inputTableName, inputPath)
+createTestTable(tableName)
+checkAnswer(sql(
+ s"""
+ | INSERT INTO TABLE $tableName
+ | SELECT shortField, intField, bigintField, doubleField, stringField,
+ | from_unixtime(unix_timestamp(timestampField,'/M/dd'))
timestampField, decimalField,
+ | cast(to_date(from_unixtime(unix_timestamp(dateField,'/M/dd')))
as date), charField
+ | FROM $inputTableName
+ """.stripMargin), Seq(Row("0")))
+checkAnswer(sql(
+ s"LOAD DATA LOCAL INPATH '$inputPath'" +
+ s" INTO TABLE $tableName" +
+ " OPTIONS('FILEHEADER'=" +
+ "'shortField,intField,bigintField,doubleField,stringField," +
+ "timestampField,decimalField,dateField,charField')"), Seq(Row("1")))
Review comment:
In JDBC case, it will return like this
++
|Segment ID|
++
| 0 |
++
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] areyouokfreejoe commented on a change in pull request #4086: [CARBONDATA-4115] Successful load and insert will return segment ID
areyouokfreejoe commented on a change in pull request #4086:
URL: https://github.com/apache/carbondata/pull/4086#discussion_r569991855
##
File path:
integration/spark/src/test/scala/org/apache/spark/util/CarbonCommandSuite.scala
##
@@ -82,6 +83,43 @@ class CarbonCommandSuite extends QueryTest with
BeforeAndAfterAll {
""".stripMargin)
}
+ protected def createTestTable(tableName: String): Unit = {
+sql(
+ s"""
Review comment:
Can you help point out which test case I can add validation? Because
there are too many.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a change in pull request #4076: [CARBONDATA-4107] Added related MV tables Map to fact table and added lock while touchMDTFile
akashrn5 commented on a change in pull request #4076: URL: https://github.com/apache/carbondata/pull/4076#discussion_r569996310 ## File path: docs/mv-guide.md ## @@ -1,3 +1,4 @@ + Review comment: remove this empty line change This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a change in pull request #4076: [CARBONDATA-4107] Added related MV tables Map to fact table and added lock while touchMDTFile
akashrn5 commented on a change in pull request #4076: URL: https://github.com/apache/carbondata/pull/4076#discussion_r569996744 ## File path: docs/mv-guide.md ## @@ -241,6 +242,11 @@ The current information includes: | Refresh Mode | FULL / INCREMENTAL refresh to MV | | Refresh Trigger Mode | ON_COMMIT / ON_MANUAL refresh to MV provided by user | | Properties | Table properties of the materialized view | + +**NOTE**: For materialized views created +before [CARBONDATA-4107](https://issues.apache.org/jira/browse/CARBONDATA-4107) issue fix, run +refresh mv command to add mv name to fact table property and to enable it. If refresh command Review comment: ```suggestion refresh mv command to add mv name to fact table's table properties and to enable it. If refresh command ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4080: [CARBONDATA-4111] Filter query having invalid results after add segment to table having SI with Indexserver
CarbonDataQA2 commented on pull request #4080: URL: https://github.com/apache/carbondata/pull/4080#issuecomment-773101117 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5426/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4080: [CARBONDATA-4111] Filter query having invalid results after add segment to table having SI with Indexserver
CarbonDataQA2 commented on pull request #4080: URL: https://github.com/apache/carbondata/pull/4080#issuecomment-773101426 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3666/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
CarbonDataQA2 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773123973 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3667/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
nihal0107 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773125323 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4086: [CARBONDATA-4115] Successful load and insert will return segment ID
CarbonDataQA2 commented on pull request #4086: URL: https://github.com/apache/carbondata/pull/4086#issuecomment-773127593 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5428/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4086: [CARBONDATA-4115] Successful load and insert will return segment ID
CarbonDataQA2 commented on pull request #4086: URL: https://github.com/apache/carbondata/pull/4086#issuecomment-773140387 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3668/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #4080: [CARBONDATA-4111] Filter query having invalid results after add segment to table having SI with Indexserver
Indhumathi27 commented on pull request #4080: URL: https://github.com/apache/carbondata/pull/4080#issuecomment-773142182 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] kunal642 commented on pull request #4070: [CARBONDATA-4082] Fix alter table add segment query on adding a segment having delete delta files.
kunal642 commented on pull request #4070: URL: https://github.com/apache/carbondata/pull/4070#issuecomment-773154078 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4076: [CARBONDATA-4107] Added related MV tables Map to fact table and added lock while touchMDTFile
CarbonDataQA2 commented on pull request #4076: URL: https://github.com/apache/carbondata/pull/4076#issuecomment-773157919 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5429/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4076: [CARBONDATA-4107] Added related MV tables Map to fact table and added lock while touchMDTFile
CarbonDataQA2 commented on pull request #4076: URL: https://github.com/apache/carbondata/pull/4076#issuecomment-773158309 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3669/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on pull request #4076: [CARBONDATA-4107] Added related MV tables Map to fact table and added lock while touchMDTFile
akashrn5 commented on pull request #4076: URL: https://github.com/apache/carbondata/pull/4076#issuecomment-773160807 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #4070: [CARBONDATA-4082] Fix alter table add segment query on adding a segment having delete delta files.
asfgit closed pull request #4070: URL: https://github.com/apache/carbondata/pull/4070 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #4076: [CARBONDATA-4107] Added related MV tables Map to fact table and added lock while touchMDTFile
asfgit closed pull request #4076: URL: https://github.com/apache/carbondata/pull/4076 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on pull request #4080: [CARBONDATA-4111] Filter query having invalid results after add segment to table having SI with Indexserver
akashrn5 commented on pull request #4080: URL: https://github.com/apache/carbondata/pull/4080#issuecomment-773182792 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
CarbonDataQA2 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773183668 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5430/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #4080: [CARBONDATA-4111] Filter query having invalid results after add segment to table having SI with Indexserver
asfgit closed pull request #4080: URL: https://github.com/apache/carbondata/pull/4080 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
CarbonDataQA2 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773185285 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3670/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
nihal0107 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773185819 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4087: [WIP] SI compatability issue fix
CarbonDataQA2 commented on pull request #4087: URL: https://github.com/apache/carbondata/pull/4087#issuecomment-773224131 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3671/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4087: [WIP] SI compatability issue fix
CarbonDataQA2 commented on pull request #4087: URL: https://github.com/apache/carbondata/pull/4087#issuecomment-773224317 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5431/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
CarbonDataQA2 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773242373 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3672/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
CarbonDataQA2 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773243978 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5432/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
nihal0107 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773245019 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4087: [WIP] SI compatability issue fix
CarbonDataQA2 commented on pull request #4087: URL: https://github.com/apache/carbondata/pull/4087#issuecomment-773286185 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5433/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4087: [WIP] SI compatability issue fix
CarbonDataQA2 commented on pull request #4087: URL: https://github.com/apache/carbondata/pull/4087#issuecomment-773286315 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3673/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
CarbonDataQA2 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773295921 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3674/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
CarbonDataQA2 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773306381 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5434/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
Indhumathi27 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773350323 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
asfgit closed pull request #4071: URL: https://github.com/apache/carbondata/pull/4071 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #1380: [CARBONDATA-1485] Timestamp no dictionary bug
CarbonDataQA2 commented on pull request #1380: URL: https://github.com/apache/carbondata/pull/1380#issuecomment-773358902 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5435/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #1380: [CARBONDATA-1485] Timestamp no dictionary bug
CarbonDataQA2 commented on pull request #1380: URL: https://github.com/apache/carbondata/pull/1380#issuecomment-773358902 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5435/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a change in pull request #4076: [CARBONDATA-4107] Added related MV tables Map to fact table and added lock while touchMDTFile
akashrn5 commented on a change in pull request #4076: URL: https://github.com/apache/carbondata/pull/4076#discussion_r569996310 ## File path: docs/mv-guide.md ## @@ -1,3 +1,4 @@ + Review comment: remove this empty line change ## File path: docs/mv-guide.md ## @@ -241,6 +242,11 @@ The current information includes: | Refresh Mode | FULL / INCREMENTAL refresh to MV | | Refresh Trigger Mode | ON_COMMIT / ON_MANUAL refresh to MV provided by user | | Properties | Table properties of the materialized view | + +**NOTE**: For materialized views created +before [CARBONDATA-4107](https://issues.apache.org/jira/browse/CARBONDATA-4107) issue fix, run +refresh mv command to add mv name to fact table property and to enable it. If refresh command Review comment: ```suggestion refresh mv command to add mv name to fact table's table properties and to enable it. If refresh command ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] kunal642 commented on pull request #4070: [CARBONDATA-4082] Fix alter table add segment query on adding a segment having delete delta files.
kunal642 commented on pull request #4070: URL: https://github.com/apache/carbondata/pull/4070#issuecomment-773154078 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4086: [CARBONDATA-4115] Successful load and insert will return segment ID
CarbonDataQA2 commented on pull request #4086: URL: https://github.com/apache/carbondata/pull/4086#issuecomment-773127593 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
CarbonDataQA2 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773073732 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4087: [WIP] SI compatability issue fix
CarbonDataQA2 commented on pull request #4087: URL: https://github.com/apache/carbondata/pull/4087#issuecomment-773224131 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] areyouokfreejoe commented on a change in pull request #4086: [CARBONDATA-4115] Successful load and insert will return segment ID
areyouokfreejoe commented on a change in pull request #4086:
URL: https://github.com/apache/carbondata/pull/4086#discussion_r569985309
##
File path:
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonInsertIntoCommand.scala
##
@@ -276,7 +280,15 @@ case class CarbonInsertIntoCommand(databaseNameOp:
Option[String],
}
throw ex
}
-Seq.empty
+if(loadResultForReturn!=null && loadResultForReturn.getLoadName!=null) {
+ Seq(Row(loadResultForReturn.getLoadName))
+} else {
+ rowsForReturn
Review comment:
It's not number of rows.
It's segment id from partition table case.
Ok, Comment has been added.
##
File path:
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonInsertIntoCommand.scala
##
@@ -276,7 +280,15 @@ case class CarbonInsertIntoCommand(databaseNameOp:
Option[String],
}
throw ex
}
-Seq.empty
+if(loadResultForReturn!=null && loadResultForReturn.getLoadName!=null) {
Review comment:
Ok, I have it fixed and re push.
##
File path:
integration/spark/src/test/scala/org/apache/spark/util/CarbonCommandSuite.scala
##
@@ -100,6 +138,56 @@ class CarbonCommandSuite extends QueryTest with
BeforeAndAfterAll {
private lazy val location = CarbonProperties.getStorePath()
+ test("Return segment ID after load and insert") {
+val tableName = "test_table"
+val inputTableName = "csv_table"
+val inputPath = s"$resourcesPath/data_alltypes.csv"
+dropTable(tableName)
+dropTable(inputTableName)
+createAndLoadInputTable(inputTableName, inputPath)
+createTestTable(tableName)
+checkAnswer(sql(
+ s"""
+ | INSERT INTO TABLE $tableName
+ | SELECT shortField, intField, bigintField, doubleField, stringField,
+ | from_unixtime(unix_timestamp(timestampField,'/M/dd'))
timestampField, decimalField,
+ | cast(to_date(from_unixtime(unix_timestamp(dateField,'/M/dd')))
as date), charField
+ | FROM $inputTableName
+ """.stripMargin), Seq(Row("0")))
+checkAnswer(sql(
+ s"LOAD DATA LOCAL INPATH '$inputPath'" +
+ s" INTO TABLE $tableName" +
+ " OPTIONS('FILEHEADER'=" +
+ "'shortField,intField,bigintField,doubleField,stringField," +
+ "timestampField,decimalField,dateField,charField')"), Seq(Row("1")))
Review comment:
In JDBC case, it will return like this
++
|Segment ID|
++
| 0|
++
##
File path:
integration/spark/src/test/scala/org/apache/spark/util/CarbonCommandSuite.scala
##
@@ -100,6 +138,56 @@ class CarbonCommandSuite extends QueryTest with
BeforeAndAfterAll {
private lazy val location = CarbonProperties.getStorePath()
+ test("Return segment ID after load and insert") {
+val tableName = "test_table"
+val inputTableName = "csv_table"
+val inputPath = s"$resourcesPath/data_alltypes.csv"
+dropTable(tableName)
+dropTable(inputTableName)
+createAndLoadInputTable(inputTableName, inputPath)
+createTestTable(tableName)
+checkAnswer(sql(
+ s"""
+ | INSERT INTO TABLE $tableName
+ | SELECT shortField, intField, bigintField, doubleField, stringField,
+ | from_unixtime(unix_timestamp(timestampField,'/M/dd'))
timestampField, decimalField,
+ | cast(to_date(from_unixtime(unix_timestamp(dateField,'/M/dd')))
as date), charField
+ | FROM $inputTableName
+ """.stripMargin), Seq(Row("0")))
+checkAnswer(sql(
+ s"LOAD DATA LOCAL INPATH '$inputPath'" +
+ s" INTO TABLE $tableName" +
+ " OPTIONS('FILEHEADER'=" +
+ "'shortField,intField,bigintField,doubleField,stringField," +
+ "timestampField,decimalField,dateField,charField')"), Seq(Row("1")))
Review comment:
And in spark sql, it will return as just a num string. The retrun is
simple and I think in this case the user can easily catch it and use it as a
input of next step. I don't think the too many word is needed here.
##
File path:
integration/spark/src/test/scala/org/apache/spark/util/CarbonCommandSuite.scala
##
@@ -100,6 +138,56 @@ class CarbonCommandSuite extends QueryTest with
BeforeAndAfterAll {
private lazy val location = CarbonProperties.getStorePath()
+ test("Return segment ID after load and insert") {
+val tableName = "test_table"
+val inputTableName = "csv_table"
+val inputPath = s"$resourcesPath/data_alltypes.csv"
+dropTable(tableName)
+dropTable(inputTableName)
+createAndLoadInputTable(inputTableName, inputPath)
+createTestTable(tableName)
+checkAnswer(sql(
+ s"""
+ | INSERT INTO TABLE $tableName
+ | SELECT shortField, intField, bigintField, doubleField, stringField,
+ | from_unixtime(unix_timestamp(timestampField,'/M/dd'))
timestampF
[GitHub] [carbondata] asfgit closed pull request #4080: [CARBONDATA-4111] Filter query having invalid results after add segment to table having SI with Indexserver
asfgit closed pull request #4080: URL: https://github.com/apache/carbondata/pull/4080 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #4070: [CARBONDATA-4082] Fix alter table add segment query on adding a segment having delete delta files.
asfgit closed pull request #4070: URL: https://github.com/apache/carbondata/pull/4070 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #4080: [CARBONDATA-4111] Filter query having invalid results after add segment to table having SI with Indexserver
Indhumathi27 commented on pull request #4080: URL: https://github.com/apache/carbondata/pull/4080#issuecomment-773142182 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4080: [CARBONDATA-4111] Filter query having invalid results after add segment to table having SI with Indexserver
CarbonDataQA2 commented on pull request #4080: URL: https://github.com/apache/carbondata/pull/4080#issuecomment-773101117 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4076: [CARBONDATA-4107] Added related MV tables Map to fact table and added lock while touchMDTFile
CarbonDataQA2 commented on pull request #4076: URL: https://github.com/apache/carbondata/pull/4076#issuecomment-773157919 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on pull request #4080: [CARBONDATA-4111] Filter query having invalid results after add segment to table having SI with Indexserver
akashrn5 commented on pull request #4080: URL: https://github.com/apache/carbondata/pull/4080#issuecomment-773182792 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
Indhumathi27 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773350323 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on pull request #4076: [CARBONDATA-4107] Added related MV tables Map to fact table and added lock while touchMDTFile
akashrn5 commented on pull request #4076: URL: https://github.com/apache/carbondata/pull/4076#issuecomment-773160807 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
nihal0107 commented on pull request #4071: URL: https://github.com/apache/carbondata/pull/4071#issuecomment-773075985 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #4071: [CARBONDATA-4102] Added UT and FT to improve coverage of SI module.
asfgit closed pull request #4071: URL: https://github.com/apache/carbondata/pull/4071 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #4076: [CARBONDATA-4107] Added related MV tables Map to fact table and added lock while touchMDTFile
asfgit closed pull request #4076: URL: https://github.com/apache/carbondata/pull/4076 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Karan980 opened a new pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.
Karan980 opened a new pull request #4088: URL: https://github.com/apache/carbondata/pull/4088 ### Why is this PR needed? Prepriming is not working in Index Server. Server.getRemoteUser returns null value in async call of prepriming which results in NPE and crashes the indexServer application. ### What changes were proposed in this PR? Computed the Server.getRemoteUser value before making the async prepriming call and then used the same value during async call. ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.
CarbonDataQA2 commented on pull request #4088: URL: https://github.com/apache/carbondata/pull/4088#issuecomment-774024419 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5436/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.
CarbonDataQA2 commented on pull request #4088: URL: https://github.com/apache/carbondata/pull/4088#issuecomment-774026587 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3675/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ShreelekhyaG opened a new pull request #4089: [CARBONDATA-4117] cg index query with Index server fails with NPE
ShreelekhyaG opened a new pull request #4089: URL: https://github.com/apache/carbondata/pull/4089 ### Why is this PR needed? 1. Test cg index query with Index server fails with NPE. 2. While initializing the index model, a parsing error is thrown when trying to uncompress with snappy. ### What changes were proposed in this PR? Have set segment and schema details to `BlockletIndexInputSplit `object. While writing minmax object, write byte size instead of position. ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4089: [CARBONDATA-4117] cg index query with Index server fails with NPE
CarbonDataQA2 commented on pull request #4089: URL: https://github.com/apache/carbondata/pull/4089#issuecomment-774148335 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5437/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4089: [CARBONDATA-4117] cg index query with Index server fails with NPE
CarbonDataQA2 commented on pull request #4089: URL: https://github.com/apache/carbondata/pull/4089#issuecomment-774152452 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3676/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 opened a new pull request #4090: [WIP] Use CarbonFile API instead of java File API while writing Flink data
Indhumathi27 opened a new pull request #4090: URL: https://github.com/apache/carbondata/pull/4090 ### Why is this PR needed? ### What changes were proposed in this PR? ### Does this PR introduce any user interface change? - No - Yes. (please explain the change and update document) ### Is any new testcase added? - No - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4090: [WIP] Use CarbonFile API instead of java File API while writing Flink data
CarbonDataQA2 commented on pull request #4090: URL: https://github.com/apache/carbondata/pull/4090#issuecomment-774671125 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5438/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4090: [WIP] Use CarbonFile API instead of java File API while writing Flink data
CarbonDataQA2 commented on pull request #4090: URL: https://github.com/apache/carbondata/pull/4090#issuecomment-774671972 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3677/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4090: [WIP] Use CarbonFile API instead of java File API while writing Flink data
CarbonDataQA2 commented on pull request #4090: URL: https://github.com/apache/carbondata/pull/4090#issuecomment-774700250 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5439/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4090: [WIP] Use CarbonFile API instead of java File API while writing Flink data
CarbonDataQA2 commented on pull request #4090: URL: https://github.com/apache/carbondata/pull/4090#issuecomment-774701297 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3678/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4090: [CARBONDATA-4122] Support HDFS Carbon writer for Flink Carbon Streaming
CarbonDataQA2 commented on pull request #4090: URL: https://github.com/apache/carbondata/pull/4090#issuecomment-774971494 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5440/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4090: [CARBONDATA-4122] Support HDFS Carbon writer for Flink Carbon Streaming
CarbonDataQA2 commented on pull request #4090: URL: https://github.com/apache/carbondata/pull/4090#issuecomment-774975863 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3679/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.
CarbonDataQA2 commented on pull request #4088: URL: https://github.com/apache/carbondata/pull/4088#issuecomment-775114494 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5441/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.
CarbonDataQA2 commented on pull request #4088: URL: https://github.com/apache/carbondata/pull/4088#issuecomment-775119897 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3680/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 opened a new pull request #4091: [WIP] Fix Refresh MV which does not exist error message
Indhumathi27 opened a new pull request #4091: URL: https://github.com/apache/carbondata/pull/4091 ### Why is this PR needed? ### What changes were proposed in this PR? ### Does this PR introduce any user interface change? - No - Yes. (please explain the change and update document) ### Is any new testcase added? - No - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #4078: [CARBONDATA-4075] Using withEvents instead of fireEvent
Indhumathi27 commented on pull request #4078: URL: https://github.com/apache/carbondata/pull/4078#issuecomment-775715984 @QiangCai please rebase This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4091: [WIP] Fix Refresh MV which does not exist error message
CarbonDataQA2 commented on pull request #4091: URL: https://github.com/apache/carbondata/pull/4091#issuecomment-775723987 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5442/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4091: [WIP] Fix Refresh MV which does not exist error message
CarbonDataQA2 commented on pull request #4091: URL: https://github.com/apache/carbondata/pull/4091#issuecomment-775728978 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3681/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4089: [CARBONDATA-4117][[CARBONDATA-4123] cg index and bloom index query issue with Index server
CarbonDataQA2 commented on pull request #4089: URL: https://github.com/apache/carbondata/pull/4089#issuecomment-775826315 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5443/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4089: [CARBONDATA-4117][[CARBONDATA-4123] cg index and bloom index query issue with Index server
CarbonDataQA2 commented on pull request #4089: URL: https://github.com/apache/carbondata/pull/4089#issuecomment-775826770 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3682/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] vikramahuja1001 opened a new pull request #4092: [WIP] Clean stale files in success segments
vikramahuja1001 opened a new pull request #4092: URL: https://github.com/apache/carbondata/pull/4092 ### Why is this PR needed? ### What changes were proposed in this PR? ### Does this PR introduce any user interface change? - No - Yes. (please explain the change and update document) ### Is any new testcase added? - No - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4092: [WIP] Clean stale files in success segments
CarbonDataQA2 commented on pull request #4092: URL: https://github.com/apache/carbondata/pull/4092#issuecomment-775877278 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3683/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4092: [WIP] Clean stale files in success segments
CarbonDataQA2 commented on pull request #4092: URL: https://github.com/apache/carbondata/pull/4092#issuecomment-775877799 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5444/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4092: [WIP] Clean stale files in success segments
CarbonDataQA2 commented on pull request #4092: URL: https://github.com/apache/carbondata/pull/4092#issuecomment-775969824 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5445/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4092: [WIP] Clean stale files in success segments
CarbonDataQA2 commented on pull request #4092: URL: https://github.com/apache/carbondata/pull/4092#issuecomment-775972308 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3684/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #4090: [CARBONDATA-4122] Support HDFS Carbon writer for Flink Carbon Streaming
ajantha-bhat commented on a change in pull request #4090: URL: https://github.com/apache/carbondata/pull/4090#discussion_r572953288 ## File path: docs/flink-integration-guide.md ## @@ -78,7 +78,7 @@ limitations under the License. val carbonProperties = new Properties // Set the carbon properties here, such as date format, store location, etc. -// Create carbon bulk writer factory. Two writer types are supported: 'Local' and 'S3'. +// Create carbon bulk writer factory. Three writer types are supported: 'Local', Hdfs' and 'S3'. Review comment: Everywhere if we use file factory API and support HDFS conf input, only one writer is enough right ? do we need 3 writers ? Because in carbon table or SDK we don't create multiple type of writers to handle this kind of scenario. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #4090: [CARBONDATA-4122] Support HDFS Carbon writer for Flink Carbon Streaming
Indhumathi27 commented on a change in pull request #4090: URL: https://github.com/apache/carbondata/pull/4090#discussion_r572968625 ## File path: docs/flink-integration-guide.md ## @@ -78,7 +78,7 @@ limitations under the License. val carbonProperties = new Properties // Set the carbon properties here, such as date format, store location, etc. -// Create carbon bulk writer factory. Two writer types are supported: 'Local' and 'S3'. +// Create carbon bulk writer factory. Three writer types are supported: 'Local', Hdfs' and 'S3'. Review comment: yes. i also thought about the same. But since, already they have implemented writers for LOCAL and S3 type, i have implemented for HDFS. But i can see, there are some differences only for S3 writer, some extra configurations are needed and they are not creating directory while writing stage directories in S3. you can check CarbonS3Writer.commit. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Scan best Fix Secondary Index table on Index Server and prune main table
CarbonDataQA2 commented on pull request #4081: URL: https://github.com/apache/carbondata/pull/4081#issuecomment-776017835 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5448/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
CarbonDataQA2 commented on pull request #4081: URL: https://github.com/apache/carbondata/pull/4081#issuecomment-776020699 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3687/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
CarbonDataQA2 commented on pull request #4081: URL: https://github.com/apache/carbondata/pull/4081#issuecomment-776133890 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5449/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
CarbonDataQA2 commented on pull request #4081: URL: https://github.com/apache/carbondata/pull/4081#issuecomment-776134616 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3688/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #4090: [CARBONDATA-4122] Use CarbonFile API instead of java File API for Flink CarbonLocalWriter
Indhumathi27 commented on a change in pull request #4090: URL: https://github.com/apache/carbondata/pull/4090#discussion_r573488319 ## File path: docs/flink-integration-guide.md ## @@ -78,7 +78,7 @@ limitations under the License. val carbonProperties = new Properties // Set the carbon properties here, such as date format, store location, etc. -// Create carbon bulk writer factory. Two writer types are supported: 'Local' and 'S3'. +// Create carbon bulk writer factory. Three writer types are supported: 'Local', Hdfs' and 'S3'. Review comment: Chaned code to use CarbonLocalWriter itself to handle Local and Hdfs FileSystems. Please review This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4090: [CARBONDATA-4122] Use CarbonFile API instead of java File API for Flink CarbonLocalWriter
CarbonDataQA2 commented on pull request #4090: URL: https://github.com/apache/carbondata/pull/4090#issuecomment-776532399 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5450/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4090: [CARBONDATA-4122] Use CarbonFile API instead of java File API for Flink CarbonLocalWriter
CarbonDataQA2 commented on pull request #4090: URL: https://github.com/apache/carbondata/pull/4090#issuecomment-776532783 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3689/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
CarbonDataQA2 commented on pull request #4081: URL: https://github.com/apache/carbondata/pull/4081#issuecomment-776634120 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3690/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
CarbonDataQA2 commented on pull request #4081: URL: https://github.com/apache/carbondata/pull/4081#issuecomment-776642321 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5451/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4087: [CARBONDATA-4125] SI compatability issue fix
CarbonDataQA2 commented on pull request #4087: URL: https://github.com/apache/carbondata/pull/4087#issuecomment-776646101 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5452/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4087: [CARBONDATA-4125] SI compatability issue fix
CarbonDataQA2 commented on pull request #4087: URL: https://github.com/apache/carbondata/pull/4087#issuecomment-776648606 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3691/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #4090: [CARBONDATA-4122] Use CarbonFile API instead of java File API for Flink CarbonLocalWriter
ajantha-bhat commented on pull request #4090: URL: https://github.com/apache/carbondata/pull/4090#issuecomment-776719317 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #4090: [CARBONDATA-4122] Use CarbonFile API instead of java File API for Flink CarbonLocalWriter
asfgit closed pull request #4090: URL: https://github.com/apache/carbondata/pull/4090 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Karan980 opened a new pull request #4093: [CARBONDATA-4126] Concurrent compaction failed with load on table.
Karan980 opened a new pull request #4093: URL: https://github.com/apache/carbondata/pull/4093 ### Why is this PR needed? Concurrent compaction was failing when run in parallel with load. During load we acquire SegmentLock for a particular segment, and when this same lock we try to acquire during compaction, we were not able to acquire this lock and compaction fails. ### What changes were proposed in this PR? Skipped compaction for segments for which we are not able to acquire the SegmentLock instead of throwing the exception. ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
