[GitHub] [carbondata] VenuReddy2103 commented on a change in pull request #3953: [CARBONDATA-4008]Fixed IN filter on date column is returning 0 results when 'carbon.push.rowfilters.for.vector' is true
VenuReddy2103 commented on a change in pull request #3953:
URL: https://github.com/apache/carbondata/pull/3953#discussion_r494050860
##
File path:
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/filterexpr/TestInFilter.scala
##
@@ -165,8 +168,27 @@ class TestInFilter extends QueryTest with
BeforeAndAfterAll{
Seq(Row(4, 1.00, 2.00, 3.00)))
}
- override def afterAll(): Unit = {
+ test("test infilter with date, timestamp columns") {
+sql("create table test_table(i int, dt date, ts timestamp) stored as
carbondata")
+sql("insert into test_table select 1, '2020-03-30', '2020-03-30 10:00:00'")
Review comment:
Have already added drop table in afterEach()
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] kunal642 commented on a change in pull request #3953: [CARBONDATA-4008]Fixed IN filter on date column is returning 0 results when 'carbon.push.rowfilters.for.vector' is true
kunal642 commented on a change in pull request #3953:
URL: https://github.com/apache/carbondata/pull/3953#discussion_r494047162
##
File path:
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/filterexpr/TestInFilter.scala
##
@@ -165,8 +168,27 @@ class TestInFilter extends QueryTest with
BeforeAndAfterAll{
Seq(Row(4, 1.00, 2.00, 3.00)))
}
- override def afterAll(): Unit = {
+ test("test infilter with date, timestamp columns") {
+sql("create table test_table(i int, dt date, ts timestamp) stored as
carbondata")
+sql("insert into test_table select 1, '2020-03-30', '2020-03-30 10:00:00'")
Review comment:
drop table before create to avoid failure if previous test fails
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] kunal642 commented on a change in pull request #3953: [CARBONDATA-4008]Fixed IN filter on date column is returning 0 results when 'carbon.push.rowfilters.for.vector' is true
kunal642 commented on a change in pull request #3953: URL: https://github.com/apache/carbondata/pull/3953#discussion_r494046607 ## File path: core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelFilterExecutorImpl.java ## @@ -138,6 +145,8 @@ public RowLevelFilterExecutorImpl(List dimColEvalua this.exp = exp; this.tableIdentifier = tableIdentifier; this.complexDimensionInfoMap = complexDimensionInfoMap; +this.dateDictionaryGenerator = + DirectDictionaryKeyGeneratorFactory.getDirectDictionaryGenerator(DataTypes.DATE); Review comment: this issue is not happening with timestamp column? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] vikramahuja1001 commented on pull request #3952: [CARBONDATA-4006] Fix for currentUser as NULL in getcount method during index server fallback mode
vikramahuja1001 commented on pull request #3952: URL: https://github.com/apache/carbondata/pull/3952#issuecomment-698117428 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] VenuReddy2103 commented on pull request #3953: [CARBONDATA-4008]Fixed IN filter on date column is returning 0 results when 'carbon.push.rowfilters.for.vector' is true
VenuReddy2103 commented on pull request #3953: URL: https://github.com/apache/carbondata/pull/3953#issuecomment-698100642 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] marchpure closed pull request #3942: [WIP] test CI
marchpure closed pull request #3942: URL: https://github.com/apache/carbondata/pull/3942 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3953: [CARBONDATA-4008]Fixed IN filter on date column is returning 0 results when 'carbon.push.rowfilters.for.vector' is true
CarbonDataQA1 commented on pull request #3953: URL: https://github.com/apache/carbondata/pull/3953#issuecomment-698045079 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4199/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3953: [CARBONDATA-4008]Fixed IN filter on date column is returning 0 results when 'carbon.push.rowfilters.for.vector' is true
CarbonDataQA1 commented on pull request #3953: URL: https://github.com/apache/carbondata/pull/3953#issuecomment-698041869 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2456/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] VenuReddy2103 opened a new pull request #3953: [CARBONDATA-4008]Fixed IN filter on date column is returning 0 results when 'carbon.push.rowfilters.for.vector' is true
VenuReddy2103 opened a new pull request #3953: URL: https://github.com/apache/carbondata/pull/3953 ### Why is this PR needed? IN filter on date column is returning 0 results when 'carbon.push.rowfilters.for.vector' is set to true. `RowLevelFilterExecutorImpl.applyFilter()` calls `createRow()` and applies filter at row level with `expression.evaluate(row)` invocation. `expression` in this case is `InExpression`. But `createRow()` missed to fill the date column value. Thus expression always evaluates to false and returns 0 rows. ### What changes were proposed in this PR? 1. Filled the date column value in `createRow()`. 2. Have removed unused imports introduced in previous PR.(SqlAstBuilderHelper.scala) ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (CARBONDATA-4008) IN filter on date column is returning 0 results when 'carbon.push.rowfilters.for.vector' is true
[
https://issues.apache.org/jira/browse/CARBONDATA-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venugopal Reddy K updated CARBONDATA-4008:
--
Summary: IN filter on date column is returning 0 results when
'carbon.push.rowfilters.for.vector' is true (was: IN filter with date column
in condition is returning 0 results when 'carbon.push.rowfilters.for.vector' is
true)
> IN filter on date column is returning 0 results when
> 'carbon.push.rowfilters.for.vector' is true
>
>
> Key: CARBONDATA-4008
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4008
> Project: CarbonData
> Issue Type: Bug
> Components: core
>Affects Versions: 2.0.0
>Reporter: Venugopal Reddy K
>Priority: Major
> Fix For: 2.1.0
>
>
> *Issue:*
> IN filter with date column in condition is returning 0 results when
> 'carbon.push.rowfilters.for.vector' is set to true.
>
> *Steps to reproduce:*
> sql("set carbon.push.rowfilters.for.vector=true")
> sql("create table test_table(i int, dt date, ts timestamp) stored as
> carbondata")
> sql("insert into test_table select 1, '2020-03-30', '2020-03-30 10:00:00'")
> sql("insert into test_table select 2, '2020-07-04', '2020-07-04 14:12:15'")
> sql("insert into test_table select 3, '2020-09-23', '2020-09-23 12:30:45'")
> sql("select * from test_table where dt IN ('2020-03-30',
> '2020-09-23')").show()
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CARBONDATA-4008) IN filter with date column in condition is returning 0 results when 'carbon.push.rowfilters.for.vector' is true
Venugopal Reddy K created CARBONDATA-4008:
-
Summary: IN filter with date column in condition is returning 0
results when 'carbon.push.rowfilters.for.vector' is true
Key: CARBONDATA-4008
URL: https://issues.apache.org/jira/browse/CARBONDATA-4008
Project: CarbonData
Issue Type: Bug
Components: core
Affects Versions: 2.0.0
Reporter: Venugopal Reddy K
Fix For: 2.1.0
*Issue:*
IN filter with date column in condition is returning 0 results when
'carbon.push.rowfilters.for.vector' is set to true.
*Steps to reproduce:*
sql("set carbon.push.rowfilters.for.vector=true")
sql("create table test_table(i int, dt date, ts timestamp) stored as
carbondata")
sql("insert into test_table select 1, '2020-03-30', '2020-03-30 10:00:00'")
sql("insert into test_table select 2, '2020-07-04', '2020-07-04 14:12:15'")
sql("insert into test_table select 3, '2020-09-23', '2020-09-23 12:30:45'")
sql("select * from test_table where dt IN ('2020-03-30', '2020-09-23')").show()
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3916: [CARBONDATA-3935]Support partition table transactional write in presto
CarbonDataQA1 commented on pull request #3916: URL: https://github.com/apache/carbondata/pull/3916#issuecomment-697924469 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4197/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3916: [CARBONDATA-3935]Support partition table transactional write in presto
CarbonDataQA1 commented on pull request #3916: URL: https://github.com/apache/carbondata/pull/3916#issuecomment-697922443 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2454/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3952: [CARBONDATA-4006] Fix for currentUser as NULL in getcount method during index server fallback mode
CarbonDataQA1 commented on pull request #3952: URL: https://github.com/apache/carbondata/pull/3952#issuecomment-697267999 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2453/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3952: [CARBONDATA-4006] Fix for currentUser as NULL in getcount method during index server fallback mode
CarbonDataQA1 commented on pull request #3952: URL: https://github.com/apache/carbondata/pull/3952#issuecomment-697260962 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4196/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (CARBONDATA-4005) SI with cache level blocklet issue
[ https://issues.apache.org/jira/browse/CARBONDATA-4005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akash R Nilugal resolved CARBONDATA-4005. - Fix Version/s: 2.1.0 Resolution: Fixed > SI with cache level blocklet issue > -- > > Key: CARBONDATA-4005 > URL: https://issues.apache.org/jira/browse/CARBONDATA-4005 > Project: CarbonData > Issue Type: Bug >Reporter: SHREELEKHYA GAMPA >Priority: Minor > Fix For: 2.1.0 > > Time Spent: 40m > Remaining Estimate: 0h > > Select query on SI column returns blank resultset after changing the cache > level to blocklet > PR: https://github.com/apache/carbondata/pull/3951 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] asfgit closed pull request #3951: [CARBONDATA-4005] SI with cache level blocklet issue
asfgit closed pull request #3951: URL: https://github.com/apache/carbondata/pull/3951 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on pull request #3951: [CARBONDATA-4005] SI with cache level blocklet issue
akashrn5 commented on pull request #3951: URL: https://github.com/apache/carbondata/pull/3951#issuecomment-697254002 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3914: [CARBONDATA-3979] Added Hive local dictionary support example
CarbonDataQA1 commented on pull request #3914: URL: https://github.com/apache/carbondata/pull/3914#issuecomment-697239698 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2450/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3914: [CARBONDATA-3979] Added Hive local dictionary support example
CarbonDataQA1 commented on pull request #3914: URL: https://github.com/apache/carbondata/pull/3914#issuecomment-697235855 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4193/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (CARBONDATA-4002) Altering the value of sort columns and unsetting the longStringColumns results in deletion of columns from table schema.
[
https://issues.apache.org/jira/browse/CARBONDATA-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akash R Nilugal resolved CARBONDATA-4002.
-
Fix Version/s: (was: 2.0.0)
2.1.0
Resolution: Fixed
> Altering the value of sort columns and unsetting the longStringColumns
> results in deletion of columns from table schema.
> -
>
> Key: CARBONDATA-4002
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4002
> Project: CarbonData
> Issue Type: Bug
> Components: core
>Reporter: Karan
>Priority: Major
> Fix For: 2.1.0
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> When we change the value of sortColumns by alter table query and then run
> unset for longStringColumn. it removes some columns from table schema.
> CREATE TABLE if not exists $longStringTable(id INT, name STRING, description
> STRING, address STRING, note STRING) STORED AS
> carbondataTBLPROPERTIES('sort_columns'='id,name');
> alter table long_string_table set
> tblproperties('sort_columns'='ID','sort_scope'='no_sort');
> alter table long_string_table unset tblproperties('long_string_columns');
> these queries will remove Name column from the schema because initially it
> was a sortColumn and after that value of sortColumns is changed.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [carbondata] asfgit closed pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.
asfgit closed pull request #3946: URL: https://github.com/apache/carbondata/pull/3946 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.
akashrn5 commented on pull request #3946: URL: https://github.com/apache/carbondata/pull/3946#issuecomment-697227686 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #3951: [CARBONDATA-4005] SI with cache level blocklet issue
Indhumathi27 commented on pull request #3951: URL: https://github.com/apache/carbondata/pull/3951#issuecomment-697221985 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.
CarbonDataQA1 commented on pull request #3946: URL: https://github.com/apache/carbondata/pull/3946#issuecomment-697219206 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4192/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.
CarbonDataQA1 commented on pull request #3946: URL: https://github.com/apache/carbondata/pull/3946#issuecomment-697212708 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2449/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (CARBONDATA-4007) ArrayIndexOutofBoundsException when IUD operations performed using SDK
Chetan Bhat created CARBONDATA-4007: --- Summary: ArrayIndexOutofBoundsException when IUD operations performed using SDK Key: CARBONDATA-4007 URL: https://issues.apache.org/jira/browse/CARBONDATA-4007 Project: CarbonData Issue Type: Bug Components: data-load Affects Versions: 2.1.0 Environment: Spark 2.4.5 jars used for compilation of SDK Reporter: Chetan Bhat Issue - ArrayIndexOutofBoundsException when IUD operations performed using SDK. Exception - java.lang.ArrayIndexOutOfBoundsException: 1 at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat$1.close(CarbonTableOutputFormat.java:579) at org.apache.carbondata.sdk.file.CarbonIUD.delete(CarbonIUD.java:110) at org.apache.carbondata.sdk.file.CarbonIUD.deleteExecution(CarbonIUD.java:238) at org.apache.carbondata.sdk.file.CarbonIUD.closeDelete(CarbonIUD.java:123) at org.apache.carbondata.sdk.file.CarbonIUD.commit(CarbonIUD.java:221) at com.apache.spark.SdkIUD_Test.testDelete(SdkIUD_Test.java:130) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33) at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230) at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3950: [CARBONDATA-3889] Enable scalastyle check for all scala test code
CarbonDataQA1 commented on pull request #3950: URL: https://github.com/apache/carbondata/pull/3950#issuecomment-697204200 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2452/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] vikramahuja1001 commented on pull request #3952: [CARBONDATA-4006] Fix for currentUser as NULL in getcount method during index server fallback mode
vikramahuja1001 commented on pull request #3952: URL: https://github.com/apache/carbondata/pull/3952#issuecomment-697200499 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (CARBONDATA-4006) Get count method of Index server gives currentUser as NULL in fallback mode, can later lead to Null pointer exception
Vikram Ahuja created CARBONDATA-4006: Summary: Get count method of Index server gives currentUser as NULL in fallback mode, can later lead to Null pointer exception Key: CARBONDATA-4006 URL: https://issues.apache.org/jira/browse/CARBONDATA-4006 Project: CarbonData Issue Type: Improvement Reporter: Vikram Ahuja -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] vikramahuja1001 commented on pull request #3788: [CARBONDATA-3844]Fix scan the relevant database instead of scanning all
vikramahuja1001 commented on pull request #3788: URL: https://github.com/apache/carbondata/pull/3788#issuecomment-697198143 @VenuReddy2103 , maybe such a test case can be added in the code This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3950: [CARBONDATA-3889] Enable scalastyle check for all scala test code
CarbonDataQA1 commented on pull request #3950: URL: https://github.com/apache/carbondata/pull/3950#issuecomment-697196771 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4195/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3952: Fix for getcount method during index server fallback mode
CarbonDataQA1 commented on pull request #3952: URL: https://github.com/apache/carbondata/pull/3952#issuecomment-697196407 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4194/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] vikramahuja1001 opened a new pull request #3952: Fix for getcount method during index server fallback mode
vikramahuja1001 opened a new pull request #3952: URL: https://github.com/apache/carbondata/pull/3952 ### Why is this PR needed? In case of index server fallback, Server.getRemoteUser.getShortUserName always returns NULL. Can give null pointer exception in any generic events. ### What changes were proposed in this PR? Do not launch any generic events in case of index server fallback mode. The behaviour is similar as getSplits and invalidateSegmentCache API's. ### Does this PR introduce any user interface change? - No - Yes. (please explain the change and update document) ### Is any new testcase added? - No - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a change in pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.
akashrn5 commented on a change in pull request #3946:
URL: https://github.com/apache/carbondata/pull/3946#discussion_r493241706
##
File path:
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/longstring/VarcharDataTypesBasicTestCase.scala
##
@@ -434,6 +437,43 @@ class VarcharDataTypesBasicTestCase extends QueryTest with
BeforeAndAfterEach wi
sql("DROP TABLE IF EXISTS varchar_complex_table")
}
+
+ test("check new schema after modifying schema through alter table queries
when long_string_column is not present") {
+sql(
+ s"""
+ | CREATE TABLE if not exists $longStringTable(
+ | id INT, name STRING, description STRING, address STRING, note STRING
+ | ) STORED AS carbondata
+ | TBLPROPERTIES('sort_columns'='id,name')
+ |""".
+stripMargin)
+sql(s"alter table long_string_table set
tblproperties('sort_columns'='ID','sort_scope'='no_sort')")
+sql(s"alter table long_string_table unset
tblproperties('long_string_columns')")
+val carbonTable =
CarbonMetadata.getInstance().getCarbonTable(CarbonCommonConstants.DATABASE_DEFAULT_NAME,
+ "long_string_table")
+val columns =
carbonTable.getTableInfo.getFactTable.getListOfColumns.asScala.toList
+ .filter(column => column.getColumnName.equalsIgnoreCase("name"))
+assert(columns != null && columns.size > 0 &&
columns.head.getColumnName.equals("name"))
+ }
+
+ test("check new schema after modifying schema through alter table queries
when long_string_column is present") {
+sql(
+ s"""
+ | CREATE TABLE if not exists $longStringTable(
+ | id INT, name STRING, description STRING, address STRING, note STRING
+ | ) STORED AS carbondata
+ | TBLPROPERTIES('sort_columns'='id,name')
+ |""".
+stripMargin)
+sql(s"alter table long_string_table set
tblproperties('long_string_columns'='address')")
+sql(s"alter table long_string_table set
tblproperties('sort_columns'='ID','sort_scope'='no_sort')")
Review comment:
add a test where after setting long string column , add that column in
sort columns also and check
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] kunal642 commented on pull request #3914: [CARBONDATA-3979] Added Hive local dictionary support example
kunal642 commented on pull request #3914: URL: https://github.com/apache/carbondata/pull/3914#issuecomment-697175526 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a change in pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.
akashrn5 commented on a change in pull request #3946:
URL: https://github.com/apache/carbondata/pull/3946#discussion_r493241706
##
File path:
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/longstring/VarcharDataTypesBasicTestCase.scala
##
@@ -434,6 +437,43 @@ class VarcharDataTypesBasicTestCase extends QueryTest with
BeforeAndAfterEach wi
sql("DROP TABLE IF EXISTS varchar_complex_table")
}
+
+ test("check new schema after modifying schema through alter table queries
when long_string_column is not present") {
+sql(
+ s"""
+ | CREATE TABLE if not exists $longStringTable(
+ | id INT, name STRING, description STRING, address STRING, note STRING
+ | ) STORED AS carbondata
+ | TBLPROPERTIES('sort_columns'='id,name')
+ |""".
+stripMargin)
+sql(s"alter table long_string_table set
tblproperties('sort_columns'='ID','sort_scope'='no_sort')")
+sql(s"alter table long_string_table unset
tblproperties('long_string_columns')")
+val carbonTable =
CarbonMetadata.getInstance().getCarbonTable(CarbonCommonConstants.DATABASE_DEFAULT_NAME,
+ "long_string_table")
+val columns =
carbonTable.getTableInfo.getFactTable.getListOfColumns.asScala.toList
+ .filter(column => column.getColumnName.equalsIgnoreCase("name"))
+assert(columns != null && columns.size > 0 &&
columns.head.getColumnName.equals("name"))
+ }
+
+ test("check new schema after modifying schema through alter table queries
when long_string_column is present") {
+sql(
+ s"""
+ | CREATE TABLE if not exists $longStringTable(
+ | id INT, name STRING, description STRING, address STRING, note STRING
+ | ) STORED AS carbondata
+ | TBLPROPERTIES('sort_columns'='id,name')
+ |""".
+stripMargin)
+sql(s"alter table long_string_table set
tblproperties('long_string_columns'='address')")
+sql(s"alter table long_string_table set
tblproperties('sort_columns'='ID','sort_scope'='no_sort')")
Review comment:
add a test where after setting long string column , add that column in
sort columns also and check
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] Karan980 commented on a change in pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.
Karan980 commented on a change in pull request #3946:
URL: https://github.com/apache/carbondata/pull/3946#discussion_r493224739
##
File path:
integration/spark/src/main/scala/org/apache/spark/util/AlterTableUtil.scala
##
@@ -442,7 +442,11 @@ object AlterTableUtil {
// update schema for long string columns
updateSchemaForLongStringColumns(thriftTable, longStringColumns.get)
} else if (propKeys.exists(_.equalsIgnoreCase("long_string_columns") &&
!set)) {
-updateSchemaForLongStringColumns(thriftTable, "")
+val varcharColumns =
carbonTable.getTableInfo.getFactTable.getListOfColumns.asScala.toList
Review comment:
Done
##
File path:
integration/spark/src/main/scala/org/apache/spark/util/AlterTableUtil.scala
##
@@ -442,7 +442,11 @@ object AlterTableUtil {
// update schema for long string columns
updateSchemaForLongStringColumns(thriftTable, longStringColumns.get)
} else if (propKeys.exists(_.equalsIgnoreCase("long_string_columns") &&
!set)) {
-updateSchemaForLongStringColumns(thriftTable, "")
+val varcharColumns =
carbonTable.getTableInfo.getFactTable.getListOfColumns.asScala.toList
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] Zhangshunyu edited a comment on pull request #3947: [CARBONDATA-4003] Improve IUD Concurrency
Zhangshunyu edited a comment on pull request #3947: URL: https://github.com/apache/carbondata/pull/3947#issuecomment-697074579 please handle the failed test cases This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Zhangshunyu edited a comment on pull request #3947: [CARBONDATA-4003] Improve IUD Concurrency
Zhangshunyu edited a comment on pull request #3947: URL: https://github.com/apache/carbondata/pull/3947#issuecomment-697078961 Consider a scenario that the user want to insert data and then update the table, the dml commands are sent one by one in time series, but the 2nd(update) cmd is sent by another driver and start to execute while the 1st(insert) is still running, therefore the concurrency scenario described in this PR appears. If we allow update table who has segment with status 'INSERT_INPROGRESS', the 2nd cmd(update) will execute successfully and the data inserted by the 1st cmd would not updated by the 2nd cmd. In this case, the result may not be what the user expected. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
