[jira] [Resolved] (CARBONDATA-3990) Fix DropCache log error when indexmap is null
[ https://issues.apache.org/jira/browse/CARBONDATA-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akash R Nilugal resolved CARBONDATA-3990. - Fix Version/s: 2.1.0 Resolution: Fixed > Fix DropCache log error when indexmap is null > -- > > Key: CARBONDATA-3990 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3990 > Project: CarbonData > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Priority: Minor > Fix For: 2.1.0 > > Time Spent: 2h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] asfgit closed pull request #3928: [CARBONDATA-3990] Fix DropCache log error when indexmap is null on SI table
asfgit closed pull request #3928: URL: https://github.com/apache/carbondata/pull/3928 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on pull request #3928: [CARBONDATA-3990] Fix DropCache log error when indexmap is null on SI table
akashrn5 commented on pull request #3928: URL: https://github.com/apache/carbondata/pull/3928#issuecomment-694650725 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Pickupolddriver commented on pull request #3935: [CARBONDATA-3993] Remove deletePartialLoadData in data loading process
Pickupolddriver commented on pull request #3935: URL: https://github.com/apache/carbondata/pull/3935#issuecomment-694591124 > From this pr we can see some delete actions are removed, as a result there would be some unused files left, any idea about how to handle the left files? The stale data will remain there until the user calls clean files action manually. Since the data will be check by the table status file, the stale data will not do harm to data management. Since data deleted by accident is a critical problem, but the stale data exist is acceptable. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3936: [WIP][CARBONDATA-3914] Fixed exception on reading data from carbon-hive empty table.
CarbonDataQA1 commented on pull request #3936: URL: https://github.com/apache/carbondata/pull/3936#issuecomment-694442422 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2380/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3936: [WIP][CARBONDATA-3914] Fixed exception on reading data from carbon-hive empty table.
CarbonDataQA1 commented on pull request #3936: URL: https://github.com/apache/carbondata/pull/3936#issuecomment-694441987 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4121/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3451: Bump libthrift from 0.9.3 to 0.12.0 in /format
CarbonDataQA1 commented on pull request #3451: URL: https://github.com/apache/carbondata/pull/3451#issuecomment-694438127 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2379/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] nihal0107 opened a new pull request #3936: [WIP][CARBONDATA-3914] Fixed exception on reading data from carbon-hive empty table.
nihal0107 opened a new pull request #3936: URL: https://github.com/apache/carbondata/pull/3936 ### Why is this PR needed? Reading data from empty carbontable through hive beeline was giving 'Unable read Carbon Schema' exception, when no data is present in the carbon table. ### What changes were proposed in this PR? Return the empty data if no data is present. ### Does this PR introduce any user interface change? - No - Yes. (please explain the change and update document) ### Is any new testcase added? - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (CARBONDATA-3891) Loading Data to the partitioned table will update all segments updateDeltaEndTimestamp
[ https://issues.apache.org/jira/browse/CARBONDATA-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajantha Bhat resolved CARBONDATA-3891. -- Fix Version/s: 2.1.0 Resolution: Fixed > Loading Data to the partitioned table will update all segments > updateDeltaEndTimestamp > -- > > Key: CARBONDATA-3891 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3891 > Project: CarbonData > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: ChenKai >Priority: Major > Fix For: 2.1.0 > > Time Spent: 4.5h > Remaining Estimate: 0h > > Loading Data to the partitioned table will update all segments > updateDeltaEndTimestamp，that will cause the driver to clear all segments > cache when doing the query. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] asfgit closed pull request #3848: [CARBONDATA-3891] Fix loading data will update all segments updateDeltaEndTimestamp
asfgit closed pull request #3848: URL: https://github.com/apache/carbondata/pull/3848 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3848: [CARBONDATA-3891] Fix loading data will update all segments updateDeltaEndTimestamp
CarbonDataQA1 commented on pull request #3848: URL: https://github.com/apache/carbondata/pull/3848#issuecomment-694363397 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4120/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3928: [CARBONDATA-3990] Fix DropCache log error when indexmap is null on SI table
CarbonDataQA1 commented on pull request #3928: URL: https://github.com/apache/carbondata/pull/3928#issuecomment-694362854 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2377/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3848: [CARBONDATA-3891] Fix loading data will update all segments updateDeltaEndTimestamp
CarbonDataQA1 commented on pull request #3848: URL: https://github.com/apache/carbondata/pull/3848#issuecomment-694361985 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2378/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3928: [CARBONDATA-3990] Fix DropCache log error when indexmap is null on SI table
CarbonDataQA1 commented on pull request #3928: URL: https://github.com/apache/carbondata/pull/3928#issuecomment-694361493 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4119/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] kumarvishal09 commented on pull request #3932: [CARBONDATA-3994] Skip Order by for map task if it is sort column and use limit pushdown for array_contains filter
kumarvishal09 commented on pull request #3932: URL: https://github.com/apache/carbondata/pull/3932#issuecomment-694359603 @ajantha-bhat This change will give incorrect result if table has multiple sort columns and order by is on any other sort column other on first position, as apart from first sort column rest other will be partially sorted). I think this is very specific scenario handling. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] IceMimosa commented on pull request #3848: [CARBONDATA-3891] Fix loading data will update all segments updateDeltaEndTimestamp
IceMimosa commented on pull request #3848: URL: https://github.com/apache/carbondata/pull/3848#issuecomment-694288895 @ajantha-bhat Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3930: [CARBONDATA-3991]Fix the set modified time function on S3 and Alluxio…
CarbonDataQA1 commented on pull request #3930: URL: https://github.com/apache/carbondata/pull/3930#issuecomment-694272499 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2376/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3930: [CARBONDATA-3991]Fix the set modified time function on S3 and Alluxio…
CarbonDataQA1 commented on pull request #3930: URL: https://github.com/apache/carbondata/pull/3930#issuecomment-694267620 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4118/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (CARBONDATA-3981) Presto filter check on binary datatype
[
https://issues.apache.org/jira/browse/CARBONDATA-3981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ajantha Bhat resolved CARBONDATA-3981.
--
Fix Version/s: 2.1.0
Resolution: Fixed
> Presto filter check on binary datatype
> --
>
> Key: CARBONDATA-3981
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3981
> Project: CarbonData
> Issue Type: Bug
> Components: presto-integration
>Reporter: Akshay
>Priority: Major
> Fix For: 2.1.0
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> Due to the absence of binary datatype check, there was a problem during
> object serialisation in presto filter queries.
> "select * from table where bin = cast('abc' as varbinary)" threw - error
> during serialisation.
> So have added required check in prestoFIlterUtil.java
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CARBONDATA-3942) Fix type cast when loading data into partitioned table
[ https://issues.apache.org/jira/browse/CARBONDATA-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajantha Bhat resolved CARBONDATA-3942. -- Fix Version/s: 2.1.0 Resolution: Fixed > Fix type cast when loading data into partitioned table > -- > > Key: CARBONDATA-3942 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3942 > Project: CarbonData > Issue Type: Bug > Components: spark-integration >Affects Versions: 2.1.0 >Reporter: ChenKai >Priority: Major > Fix For: 2.1.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Loading Int type data to carbondata double type, the value will be broken > like this: > +---++++ > |cnt |name|time| > +---++++ > |4.9E-323|a |2020| > |1.0E-322|b |2020| > +---++++ > original cnt is: 10, 20 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] ajantha-bhat commented on pull request #3848: [CARBONDATA-3891] Fix loading data will update all segments updateDeltaEndTimestamp
ajantha-bhat commented on pull request #3848: URL: https://github.com/apache/carbondata/pull/3848#issuecomment-694238864 @IceMimosa : After merging #3884 , there is a conflict. please rebase the PR. I will merge that This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #3884: [CARBONDATA-3942] Fix type cast when loading data into partitioned table
asfgit closed pull request #3884: URL: https://github.com/apache/carbondata/pull/3884 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #3920: [CARBONDATA-3981] Presto filter check on binary, byte and float datatypes
asfgit closed pull request #3920: URL: https://github.com/apache/carbondata/pull/3920 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3928: [CARBONDATA-3990] Fix DropCache log error when indexmap is null on SI table
CarbonDataQA1 commented on pull request #3928: URL: https://github.com/apache/carbondata/pull/3928#issuecomment-694183813 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2375/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3928: [CARBONDATA-3990] Fix DropCache log error when indexmap is null on SI table
CarbonDataQA1 commented on pull request #3928: URL: https://github.com/apache/carbondata/pull/3928#issuecomment-694182742 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4117/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #3928: [CARBONDATA-3990] Fix DropCache log error when indexmap is null on SI table
Indhumathi27 commented on pull request #3928: URL: https://github.com/apache/carbondata/pull/3928#issuecomment-694129422 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3848: [CARBONDATA-3891] Fix loading data will update all segments updateDeltaEndTimestamp
CarbonDataQA1 commented on pull request #3848: URL: https://github.com/apache/carbondata/pull/3848#issuecomment-694128346 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2374/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3848: [CARBONDATA-3891] Fix loading data will update all segments updateDeltaEndTimestamp
CarbonDataQA1 commented on pull request #3848: URL: https://github.com/apache/carbondata/pull/3848#issuecomment-694126460 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4116/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #3848: [CARBONDATA-3891] Fix loading data will update all segments updateDeltaEndTimestamp
ajantha-bhat commented on pull request #3848: URL: https://github.com/apache/carbondata/pull/3848#issuecomment-694122754 LGTM. I will merge today once the build passes. Thank you for the contribution. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3920: [CARBONDATA-3981] Presto filter check on binary, byte and float datatypes
CarbonDataQA1 commented on pull request #3920: URL: https://github.com/apache/carbondata/pull/3920#issuecomment-694122070 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4114/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #3920: [CARBONDATA-3981] Presto filter check on binary, byte and float datatypes
ajantha-bhat commented on pull request #3920: URL: https://github.com/apache/carbondata/pull/3920#issuecomment-694121769 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3920: [CARBONDATA-3981] Presto filter check on binary, byte and float datatypes
CarbonDataQA1 commented on pull request #3920: URL: https://github.com/apache/carbondata/pull/3920#issuecomment-694120316 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2372/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3884: [CARBONDATA-3942] Fix type cast when loading data into partitioned table
CarbonDataQA1 commented on pull request #3884: URL: https://github.com/apache/carbondata/pull/3884#issuecomment-694095481 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4113/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3884: [CARBONDATA-3942] Fix type cast when loading data into partitioned table
CarbonDataQA1 commented on pull request #3884: URL: https://github.com/apache/carbondata/pull/3884#issuecomment-694092465 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2371/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Zhangshunyu commented on pull request #3935: [CARBONDATA-3993] Remove deletePartialLoadData in data loading process
Zhangshunyu commented on pull request #3935: URL: https://github.com/apache/carbondata/pull/3935#issuecomment-694079287 From this pr we can see some delete actions are removed, as a result there would be some unused files left, any idea about how to handle the left files? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] IceMimosa commented on a change in pull request #3848: [CARBONDATA-3891] Fix loading data will update all segments updateDeltaEndTimestamp
IceMimosa commented on a change in pull request #3848:
URL: https://github.com/apache/carbondata/pull/3848#discussion_r490043882
##
File path:
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/segment/ShowSegmentTestCase.scala
##
@@ -224,6 +224,37 @@ class ShowSegmentTestCase extends QueryTest with
BeforeAndAfterAll {
sql("drop table if exists a")
}
+ test("test loading data into partitioned table with segment's
updateDeltaEndTimestamp not change") {
Review comment:
Thanks for reviewing, done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] akkio-97 commented on a change in pull request #3920: [CARBONDATA-3981] Presto filter check on binary, byte and float datatypes
akkio-97 commented on a change in pull request #3920:
URL: https://github.com/apache/carbondata/pull/3920#discussion_r490034868
##
File path:
integration/presto/src/main/prestosql/org/apache/carbondata/presto/PrestoFilterUtil.java
##
@@ -78,16 +78,22 @@ private static DataType
spi2CarbondataTypeMapper(HiveColumnHandle columnHandle)
HiveType colType = columnHandle.getHiveType();
if (colType.equals(HiveType.HIVE_BOOLEAN)) {
return DataTypes.BOOLEAN;
+} else if (colType.equals(HiveType.HIVE_BINARY)) {
+ return DataTypes.BINARY;
} else if (colType.equals(HiveType.HIVE_SHORT)) {
return DataTypes.SHORT;
} else if (colType.equals(HiveType.HIVE_INT)) {
return DataTypes.INT;
+} else if (colType.equals(HiveType.HIVE_FLOAT)) {
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #3884: [CARBONDATA-3942] Fix type cast when loading data into partitioned table
ajantha-bhat commented on pull request #3884: URL: https://github.com/apache/carbondata/pull/3884#issuecomment-693985853 LGTM. will merge once the retest passes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #3884: [CARBONDATA-3942] Fix type cast when loading data into partitioned table
ajantha-bhat commented on pull request #3884: URL: https://github.com/apache/carbondata/pull/3884#issuecomment-693984990 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
