[GitHub] [carbondata] kevinjmh closed pull request #2917: [WIP]Show load/insert/update/delete row number
kevinjmh closed pull request #2917: URL: https://github.com/apache/carbondata/pull/2917 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] kevinjmh closed pull request #3023: [CARBONDATA-3197][BloomDataMap] Include bloomindex merging in loading/compaction/datamap rebuild transaction
kevinjmh closed pull request #3023: URL: https://github.com/apache/carbondata/pull/3023 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3766: [WIP][Perf] Support RuntimeFilter for inner/leftsemi equi-join
CarbonDataQA1 commented on pull request #3766: URL: https://github.com/apache/carbondata/pull/3766#issuecomment-631371077 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3048/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3766: [WIP][Perf] Support RuntimeFilter for inner/leftsemi equi-join
CarbonDataQA1 commented on pull request #3766: URL: https://github.com/apache/carbondata/pull/3766#issuecomment-631371683 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1328/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (CARBONDATA-3370) fix missing version of maven-duplicate-finder-plugin
[ https://issues.apache.org/jira/browse/CARBONDATA-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor updated CARBONDATA-3370: - Fix Version/s: (was: 2.0.0) 2.1.0 > fix missing version of maven-duplicate-finder-plugin > > > Key: CARBONDATA-3370 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3370 > Project: CarbonData > Issue Type: Improvement > Components: build >Affects Versions: 1.5.3 >Reporter: lamber-ken >Priority: Critical > Fix For: 2.1.0 > > Time Spent: 2h 50m > Remaining Estimate: 0h > > fix missing version of maven-duplicate-finder-plugin in pom file -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3559) Support adding carbon file into CarbonData table
[ https://issues.apache.org/jira/browse/CARBONDATA-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor updated CARBONDATA-3559: - Fix Version/s: (was: 2.0.0) 2.1.0 > Support adding carbon file into CarbonData table > > > Key: CARBONDATA-3559 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3559 > Project: CarbonData > Issue Type: Improvement >Reporter: Jacky Li >Assignee: Jacky Li >Priority: Major > Fix For: 2.1.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Since adding parquet/orc files into CarbonData table are supported now, > adding carbon files should be supported as well -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3603) Feature Change in CarbonData 2.0
[ https://issues.apache.org/jira/browse/CARBONDATA-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor updated CARBONDATA-3603: - Fix Version/s: (was: 2.0.0) 2.1.0 > Feature Change in CarbonData 2.0 > > > Key: CARBONDATA-3603 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3603 > Project: CarbonData > Issue Type: Improvement >Reporter: Jacky Li >Priority: Major > Fix For: 2.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3608) Drop 'STORED BY' syntax in create table
[ https://issues.apache.org/jira/browse/CARBONDATA-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor updated CARBONDATA-3608: - Fix Version/s: (was: 2.0.0) 2.1.0 > Drop 'STORED BY' syntax in create table > --- > > Key: CARBONDATA-3608 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3608 > Project: CarbonData > Issue Type: Sub-task >Reporter: Jacky Li >Priority: Major > Fix For: 2.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CARBONDATA-3705) Support create and load MV for spark datasource table
[ https://issues.apache.org/jira/browse/CARBONDATA-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor resolved CARBONDATA-3705. -- Resolution: Fixed > Support create and load MV for spark datasource table > - > > Key: CARBONDATA-3705 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3705 > Project: CarbonData > Issue Type: Sub-task >Reporter: Jacky Li >Priority: Major > Fix For: 2.0.0 > > Time Spent: 4.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3746) Support column chunk cache creation and basic read/write
[ https://issues.apache.org/jira/browse/CARBONDATA-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor updated CARBONDATA-3746: - Fix Version/s: (was: 2.0.0) 2.1.0 > Support column chunk cache creation and basic read/write > > > Key: CARBONDATA-3746 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3746 > Project: CarbonData > Issue Type: Sub-task >Reporter: Jacky Li >Assignee: Jacky Li >Priority: Major > Fix For: 2.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3615) Show metacache shows the index server index-dictionary files when data loaded after index server disabled using set command
[ https://issues.apache.org/jira/browse/CARBONDATA-3615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor updated CARBONDATA-3615: - Fix Version/s: (was: 2.0.0) 2.1.0 > Show metacache shows the index server index-dictionary files when data loaded > after index server disabled using set command > --- > > Key: CARBONDATA-3615 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3615 > Project: CarbonData > Issue Type: Bug > Components: core >Affects Versions: 2.0.0 >Reporter: Vikram Ahuja >Priority: Minor > Fix For: 2.1.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Show metacache shows the index server index-dictionary files when data loaded > after index server disabled using set command > +-+-+-+-+--+ > | Field | Size | Comment | Cache Location | > +-+-+-+-+--+ > | Index | 0 B | 0/2 index files cached | DRIVER | > | Dictionary | 0 B | | DRIVER | > *| Index | 1.5 KB | 2/2 index files cached | INDEX SERVER |* > *| Dictionary | 0 B | | INDEX SERVER |* > *+-+-+-+*-+--+ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3643) Insert array('')/array() into Struct column will result in array(null), which is inconsist with Parquet
[
https://issues.apache.org/jira/browse/CARBONDATA-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Kapoor updated CARBONDATA-3643:
-
Fix Version/s: (was: 2.0.0)
2.1.0
> Insert array('')/array() into Struct column will result in
> array(null), which is inconsist with Parquet
> --
>
> Key: CARBONDATA-3643
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3643
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.6.1, 2.0.0
>Reporter: Xingjun Hao
>Priority: Minor
> Fix For: 2.1.0
>
>
>
> {code:java}
> //
> sql("create table datatype_struct_parquet(price struct>)
> stored as parquet")
> sql("insert into table datatype_struct_parquet values(named_struct('b',
> array('')))")
> sql("create table datatype_struct_carbondata(price struct>)
> stored as carbondata")
> sql("insert into datatype_struct_carbondata select * from
> datatype_struct_parquet")
> checkAnswer( sql("SELECT * FROM datatype_struct_carbondata"), sql("SELECT *
> FROM datatype_struct_parquet"))
> !== Correct Answer - 1 == == Spark Answer - 1 ==
> ![[WrappedArray()]] [[WrappedArray(null)]]
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3617) loadDataUsingGlobalSort should based on SortColumns Instead Of Whole CarbonRow
[ https://issues.apache.org/jira/browse/CARBONDATA-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor updated CARBONDATA-3617: - Fix Version/s: (was: 2.0.0) (was: 1.6.1) 2.1.0 > loadDataUsingGlobalSort should based on SortColumns Instead Of Whole CarbonRow > -- > > Key: CARBONDATA-3617 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3617 > Project: CarbonData > Issue Type: Improvement > Components: data-load >Affects Versions: 1.6.1, 2.0.0 >Reporter: Xingjun Hao >Priority: Minor > Fix For: 2.1.0 > > Time Spent: 7h 50m > Remaining Estimate: 0h > > During loading Data usesing globalsort, the sortby processing is based the > whole carbon row, the overhead of gc is huge when there are many columns. > Theoretically, the sortby processing can works well just based on the sort > columns, which will brings less time overhead and gc overhead. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3670) Support compress offheap columnpage directly, avoding a copy of data from offhead to heap when compressed.
[ https://issues.apache.org/jira/browse/CARBONDATA-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor updated CARBONDATA-3670: - Fix Version/s: (was: 2.0.0) 2.1.0 > Support compress offheap columnpage directly, avoding a copy of data from > offhead to heap when compressed. > -- > > Key: CARBONDATA-3670 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3670 > Project: CarbonData > Issue Type: Wish > Components: core >Affects Versions: 2.0.0 >Reporter: Xingjun Hao >Priority: Minor > Fix For: 2.1.0 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > When writing data, the columnpages are stored on the offheap, the pages will > be compressed to save storage cost. Now, in the compression processing, the > data will be copied from the offheap to the heap before compressed, which > leads to heavier GC overhead compared with compress offhead directly. > To sum up, we support compress offheap columnpage directly, avoding a copy of > data from offhead to heap when compressed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CARBONDATA-3767) Add spark binary version to modules which are dependent on spark
[ https://issues.apache.org/jira/browse/CARBONDATA-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor resolved CARBONDATA-3767. -- Resolution: Fixed > Add spark binary version to modules which are dependent on spark > > > Key: CARBONDATA-3767 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3767 > Project: CarbonData > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Kunal Kapoor >Assignee: Kunal Kapoor >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] dependabot[bot] commented on pull request #3447: Bump dep.jackson.version from 2.6.5 to 2.10.1 in /store/sdk
dependabot[bot] commented on pull request #3447: URL: https://github.com/apache/carbondata/pull/3447#issuecomment-631553778 Dependabot tried to update this pull request, but something went wrong. We're looking into it, but in the meantime you can retry the update by commenting `@dependabot rebase`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] dependabot[bot] commented on pull request #3456: Bump solr.version from 6.3.0 to 8.3.0 in /datamap/lucene
dependabot[bot] commented on pull request #3456: URL: https://github.com/apache/carbondata/pull/3456#issuecomment-631553839 Dependabot tried to update this pull request, but something went wrong. We're looking into it, but in the meantime you can retry the update by commenting `@dependabot rebase`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (CARBONDATA-3829) Support pagination in SDK reader
Ajantha Bhat created CARBONDATA-3829: Summary: Support pagination in SDK reader Key: CARBONDATA-3829 URL: https://issues.apache.org/jira/browse/CARBONDATA-3829 Project: CarbonData Issue Type: New Feature Reporter: Ajantha Bhat Assignee: Ajantha Bhat Attachments: CarbonData SDK support pagination.pdf Please find the solution document attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] ajantha-bhat opened a new pull request #3770: [CARBONDATA-3829] Support pagination in SDK reader
ajantha-bhat opened a new pull request #3770: URL: https://github.com/apache/carbondata/pull/3770 ### Why is this PR needed? Please refer the design document attached in the JIRA. Carbondata SDK now currently doesn't support pagination. ### What changes were proposed in this PR? a) Support pagination from java SDK with LRU cache support. b) Support pagination in python SDK by calling JAVA SDK. ### Does this PR introduce any user interface change? - No [Added new interfaces] ### Is any new testcase added? - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #3770: [CARBONDATA-3829] Support pagination in SDK reader
ajantha-bhat commented on pull request #3770: URL: https://github.com/apache/carbondata/pull/3770#issuecomment-631874964 @jackylk , @xubo245 : please check This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (CARBONDATA-3811) In Flat folder enabled table, it is returning no records while querying.
[
https://issues.apache.org/jira/browse/CARBONDATA-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanna Ravichandran closed CARBONDATA-3811.
-
Fixed.
> In Flat folder enabled table, it is returning no records while querying.
>
>
> Key: CARBONDATA-3811
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3811
> Project: CarbonData
> Issue Type: Bug
> Environment: opensource ANT cluster
>Reporter: Prasanna Ravichandran
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: Flat_folder_returning_zero.png
>
>
> Flat folder table is retuning no records for select queries.
>
> Test queries:
> drop table if exists uniqdata1;
> CREATE TABLE uniqdata1 (cust_id int,cust_name String,active_emui_version
> string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
> bigint,decimal_column1 decimal(30,10), decimal_column2
> decimal(36,36),double_column1 double, double_column2 double,integer_column1
> int) stored as carbondata TBLPROPERTIES('flat_folder'='true');
> load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into
> table uniqdata1
> options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
> select count(*) from uniqdata1;--0;
> select * from uniqdata1 limit 10;--0;
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3770: [CARBONDATA-3829] Support pagination in SDK reader
CarbonDataQA1 commented on pull request #3770: URL: https://github.com/apache/carbondata/pull/3770#issuecomment-631920232 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1329/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
