[jira] [Closed] (CARBONDATA-3811) In Flat folder enabled table, it is returning no records while querying.

2020-05-20 Thread Prasanna Ravichandran (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanna Ravichandran closed CARBONDATA-3811.
-

Fixed.

> In Flat folder enabled table, it is returning no records while querying.
> 
>
> Key: CARBONDATA-3811
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3811
> Project: CarbonData
>  Issue Type: Bug
> Environment: opensource ANT cluster
>Reporter: Prasanna Ravichandran
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: Flat_folder_returning_zero.png
>
>
> Flat folder table is retuning no records for select queries.
>  
> Test queries:
> drop table if exists uniqdata1;
> CREATE TABLE uniqdata1 (cust_id int,cust_name String,active_emui_version 
> string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2 
> bigint,decimal_column1 decimal(30,10), decimal_column2 
> decimal(36,36),double_column1 double, double_column2 double,integer_column1 
> int) stored as carbondata TBLPROPERTIES('flat_folder'='true');
> load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into 
> table uniqdata1 
> options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
> select count(*) from uniqdata1;--0;
> select * from uniqdata1 limit 10;--0;



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [carbondata] ajantha-bhat opened a new pull request #3770: [CARBONDATA-3829] Support pagination in SDK reader

2020-05-20 Thread GitBox


ajantha-bhat opened a new pull request #3770:
URL: https://github.com/apache/carbondata/pull/3770


### Why is this PR needed?
Please refer the design document attached in the JIRA.
Carbondata SDK now currently doesn't support pagination. 

### What changes were proposed in this PR?
   a) Support pagination from java SDK with LRU cache support.
   b) Support pagination in python SDK by calling JAVA SDK.
   
### Does this PR introduce any user interface change?
- No [Added new interfaces]

### Is any new testcase added?
- Yes
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ajantha-bhat commented on pull request #3770: [CARBONDATA-3829] Support pagination in SDK reader

2020-05-20 Thread GitBox


ajantha-bhat commented on pull request #3770:
URL: https://github.com/apache/carbondata/pull/3770#issuecomment-631874964


   @jackylk , @xubo245 : please check



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (CARBONDATA-3829) Support pagination in SDK reader

2020-05-20 Thread Ajantha Bhat (Jira)
Ajantha Bhat created CARBONDATA-3829:


 Summary: Support pagination in SDK reader
 Key: CARBONDATA-3829
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3829
 Project: CarbonData
  Issue Type: New Feature
Reporter: Ajantha Bhat
Assignee: Ajantha Bhat
 Attachments: CarbonData SDK support pagination.pdf

Please find the solution document attached. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [carbondata] dependabot[bot] commented on pull request #3447: Bump dep.jackson.version from 2.6.5 to 2.10.1 in /store/sdk

2020-05-20 Thread GitBox


dependabot[bot] commented on pull request #3447:
URL: https://github.com/apache/carbondata/pull/3447#issuecomment-631553778


   Dependabot tried to update this pull request, but something went wrong. 
We're looking into it, but in the meantime you can retry the update by 
commenting `@dependabot rebase`.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] dependabot[bot] commented on pull request #3456: Bump solr.version from 6.3.0 to 8.3.0 in /datamap/lucene

2020-05-20 Thread GitBox


dependabot[bot] commented on pull request #3456:
URL: https://github.com/apache/carbondata/pull/3456#issuecomment-631553839


   Dependabot tried to update this pull request, but something went wrong. 
We're looking into it, but in the meantime you can retry the update by 
commenting `@dependabot rebase`.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Resolved] (CARBONDATA-3767) Add spark binary version to modules which are dependent on spark

2020-05-20 Thread Kunal Kapoor (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Kapoor resolved CARBONDATA-3767.
--
Resolution: Fixed

> Add spark binary version to modules which are dependent on spark
> 
>
> Key: CARBONDATA-3767
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3767
> Project: CarbonData
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Kunal Kapoor
>Assignee: Kunal Kapoor
>Priority: Minor
> Fix For: 2.0.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3617) loadDataUsingGlobalSort should based on SortColumns Instead Of Whole CarbonRow

2020-05-20 Thread Kunal Kapoor (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Kapoor updated CARBONDATA-3617:
-
Fix Version/s: (was: 2.0.0)
   (was: 1.6.1)
   2.1.0

> loadDataUsingGlobalSort should based on SortColumns Instead Of Whole CarbonRow
> --
>
> Key: CARBONDATA-3617
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3617
> Project: CarbonData
>  Issue Type: Improvement
>  Components: data-load
>Affects Versions: 1.6.1, 2.0.0
>Reporter: Xingjun Hao
>Priority: Minor
> Fix For: 2.1.0
>
>  Time Spent: 7h 50m
>  Remaining Estimate: 0h
>
> During loading Data usesing globalsort, the sortby processing is based the 
> whole carbon row, the overhead of gc is huge when there are many columns. 
> Theoretically, the sortby processing can works well just based on the sort 
> columns, which will brings less time overhead and gc overhead.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3670) Support compress offheap columnpage directly, avoding a copy of data from offhead to heap when compressed.

2020-05-20 Thread Kunal Kapoor (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Kapoor updated CARBONDATA-3670:
-
Fix Version/s: (was: 2.0.0)
   2.1.0

> Support compress offheap columnpage directly, avoding a copy of data from 
> offhead to heap when compressed.
> --
>
> Key: CARBONDATA-3670
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3670
> Project: CarbonData
>  Issue Type: Wish
>  Components: core
>Affects Versions: 2.0.0
>Reporter: Xingjun Hao
>Priority: Minor
> Fix For: 2.1.0
>
>  Time Spent: 4h 10m
>  Remaining Estimate: 0h
>
> When writing data, the columnpages are stored on the offheap,  the pages will 
> be compressed to save storage cost. Now, in the compression processing, the 
> data will be copied from the offheap to the heap before compressed, which 
> leads to heavier GC overhead compared with compress offhead directly.
> To sum up, we support compress offheap columnpage directly, avoding a copy of 
> data from offhead to heap when compressed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3643) Insert array('')/array() into Struct column will result in array(null), which is inconsist with Parquet

2020-05-20 Thread Kunal Kapoor (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Kapoor updated CARBONDATA-3643:
-
Fix Version/s: (was: 2.0.0)
   2.1.0

> Insert array('')/array() into Struct column will result in 
> array(null), which is inconsist with Parquet
> --
>
> Key: CARBONDATA-3643
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3643
> Project: CarbonData
>  Issue Type: Bug
>Affects Versions: 1.6.1, 2.0.0
>Reporter: Xingjun Hao
>Priority: Minor
> Fix For: 2.1.0
>
>
>  
> {code:java}
> //
> sql("create table datatype_struct_parquet(price struct>) 
> stored as parquet") 
> sql("insert into table datatype_struct_parquet values(named_struct('b', 
> array('')))") 
> sql("create table datatype_struct_carbondata(price struct>) 
> stored as carbondata") 
> sql("insert into datatype_struct_carbondata select * from 
> datatype_struct_parquet")
> checkAnswer( sql("SELECT * FROM datatype_struct_carbondata"), sql("SELECT * 
> FROM datatype_struct_parquet"))
> !== Correct Answer - 1 == == Spark Answer - 1 == 
> ![[WrappedArray()]] [[WrappedArray(null)]]
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3615) Show metacache shows the index server index-dictionary files when data loaded after index server disabled using set command

2020-05-20 Thread Kunal Kapoor (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Kapoor updated CARBONDATA-3615:
-
Fix Version/s: (was: 2.0.0)
   2.1.0

> Show metacache shows the index server index-dictionary files when data loaded 
> after index server disabled using set command
> ---
>
> Key: CARBONDATA-3615
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3615
> Project: CarbonData
>  Issue Type: Bug
>  Components: core
>Affects Versions: 2.0.0
>Reporter: Vikram Ahuja
>Priority: Minor
> Fix For: 2.1.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Show metacache shows the index server index-dictionary files when data loaded 
> after index server disabled using set command
> +-+-+-+-+--+
> |    Field    |  Size   |         Comment         | Cache Location  |
> +-+-+-+-+--+
> | Index       | 0 B     | 0/2 index files cached  | DRIVER          |
> | Dictionary  | 0 B     |                         | DRIVER          |
> *| Index       | 1.5 KB  | 2/2 index files cached  | INDEX SERVER    |*
> *| Dictionary  | 0 B     |                         | INDEX SERVER    |*
> *+-+-+-+*-+--+



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3746) Support column chunk cache creation and basic read/write

2020-05-20 Thread Kunal Kapoor (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Kapoor updated CARBONDATA-3746:
-
Fix Version/s: (was: 2.0.0)
   2.1.0

> Support column chunk cache creation and basic read/write
> 
>
> Key: CARBONDATA-3746
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3746
> Project: CarbonData
>  Issue Type: Sub-task
>Reporter: Jacky Li
>Assignee: Jacky Li
>Priority: Major
> Fix For: 2.1.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3608) Drop 'STORED BY' syntax in create table

2020-05-20 Thread Kunal Kapoor (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Kapoor updated CARBONDATA-3608:
-
Fix Version/s: (was: 2.0.0)
   2.1.0

> Drop 'STORED BY' syntax in create table
> ---
>
> Key: CARBONDATA-3608
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3608
> Project: CarbonData
>  Issue Type: Sub-task
>Reporter: Jacky Li
>Priority: Major
> Fix For: 2.1.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (CARBONDATA-3705) Support create and load MV for spark datasource table

2020-05-20 Thread Kunal Kapoor (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Kapoor resolved CARBONDATA-3705.
--
Resolution: Fixed

> Support create and load MV for spark datasource table
> -
>
> Key: CARBONDATA-3705
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3705
> Project: CarbonData
>  Issue Type: Sub-task
>Reporter: Jacky Li
>Priority: Major
> Fix For: 2.0.0
>
>  Time Spent: 4.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3603) Feature Change in CarbonData 2.0

2020-05-20 Thread Kunal Kapoor (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Kapoor updated CARBONDATA-3603:
-
Fix Version/s: (was: 2.0.0)
   2.1.0

> Feature Change in CarbonData 2.0
> 
>
> Key: CARBONDATA-3603
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3603
> Project: CarbonData
>  Issue Type: Improvement
>Reporter: Jacky Li
>Priority: Major
> Fix For: 2.1.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3559) Support adding carbon file into CarbonData table

2020-05-20 Thread Kunal Kapoor (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Kapoor updated CARBONDATA-3559:
-
Fix Version/s: (was: 2.0.0)
   2.1.0

> Support adding carbon file into CarbonData table
> 
>
> Key: CARBONDATA-3559
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3559
> Project: CarbonData
>  Issue Type: Improvement
>Reporter: Jacky Li
>Assignee: Jacky Li
>Priority: Major
> Fix For: 2.1.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Since adding parquet/orc files into CarbonData table are supported now, 
> adding carbon files should be supported as well



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3370) fix missing version of maven-duplicate-finder-plugin

2020-05-20 Thread Kunal Kapoor (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Kapoor updated CARBONDATA-3370:
-
Fix Version/s: (was: 2.0.0)
   2.1.0

> fix missing version of maven-duplicate-finder-plugin
> 
>
> Key: CARBONDATA-3370
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3370
> Project: CarbonData
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 1.5.3
>Reporter: lamber-ken
>Priority: Critical
> Fix For: 2.1.0
>
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> fix missing version of maven-duplicate-finder-plugin in pom file



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3766: [WIP][Perf] Support RuntimeFilter for inner/leftsemi equi-join

2020-05-20 Thread GitBox


CarbonDataQA1 commented on pull request #3766:
URL: https://github.com/apache/carbondata/pull/3766#issuecomment-631371683


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1328/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3766: [WIP][Perf] Support RuntimeFilter for inner/leftsemi equi-join

2020-05-20 Thread GitBox


CarbonDataQA1 commented on pull request #3766:
URL: https://github.com/apache/carbondata/pull/3766#issuecomment-631371077


   Build Failed  with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3048/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] kevinjmh closed pull request #2917: [WIP]Show load/insert/update/delete row number

2020-05-20 Thread GitBox


kevinjmh closed pull request #2917:
URL: https://github.com/apache/carbondata/pull/2917


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] kevinjmh closed pull request #3023: [CARBONDATA-3197][BloomDataMap] Include bloomindex merging in loading/compaction/datamap rebuild transaction

2020-05-20 Thread GitBox


kevinjmh closed pull request #3023:
URL: https://github.com/apache/carbondata/pull/3023


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org