[jira] [Commented] (CARBONDATA-2282) presto carbon not support query partition table

2018-04-06 Thread anubhav tarar (JIRA) 

[ 
https://issues.apache.org/jira/browse/CARBONDATA-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16428463#comment-16428463
 ] 

anubhav tarar commented on CARBONDATA-2282:
---

[~chenliang613]  

In this PR we are handling the scenario of querying data to standard hive 
partitioned table in carbondata.

although you are getting the correct data from partition table but in current 
master branch we were iterating over all the partitions to get the data not the 
one which is required,which will be slow you can check value of property 
mapreduce.input.carboninputformat.partitions.to.prune in carboninputformat 
class after executing query on a particular partition it is null so when 
getPartitionsToPrune method is called it is returning null value instead of 
required partition specs 

 

but now in this pr we have identified the required partitions, so the query 
will be performed only on the required partition.
Firstly we are identifying the partition columns through domain constraints in 
presto and after that we are adding the Partition Spec to the configuration 
object to query partition

> presto carbon not support query partition table
> ---
>
> Key: CARBONDATA-2282
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2282
> Project: CarbonData
>  Issue Type: Bug
>  Components: core, presto-integration
>Affects Versions: 1.3.0
>Reporter: zhangwei
>Assignee: anubhav tarar
>Priority: Major
> Fix For: 1.3.0
>
> Attachments: partitonToPrune.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CARBONDATA-2282) presto carbon not support query partition table

2018-04-06 Thread Liang Chen (JIRA) 

[ 
https://issues.apache.org/jira/browse/CARBONDATA-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16428065#comment-16428065
 ] 

Liang Chen commented on CARBONDATA-2282:


Just i tried:

1.create partition table as below in sparkshell

scala> carbon.sql("create table partition_table(id Int,vin String, phonenumber 
Long, country String, area String, salary Int) PARTITIONED BY (logdate Date) 
STORED BY 'carbondata' TBLPROPERTIES('SORT_COLUMNS'='id,vin')")

2.Can query successfully in presto :

presto:default> select * from partition_table;
 id | vin | phonenumber | country | area | salary | logdate
+--+-+-+--++
 6 | A42258434831 | 125371346 | China | Asia | 10005 | 2016-12-13
 7 | A42158475831 | 125371347 | UK | OutSpace | 10006 | 2016-12-13
 8 | A41158494830 | 225371348 | China | Asia | 10007 | 2016-12-13
 12 | A42151477823 | 425371312 | China | Asia | 10011 | 2016-12-13
 14 | A42258434835 | 525371314 | China | Asia | 10013 | 2016-12-13
 20 | A42151477824 | 225371320 | China | Asia | 10019 | 2016-12-13
 22 | A42258434837 | 25371322 | China | Asia | 10021 | 2016-12-13
 24 | A41158494839 | 625371324 | China | Asia | 10023 | 2016-12-13
 15 | A42158475836 | 625371315 | UK | OutSpace | 10014 | 2014-05-15
 16 | A41158494838 | 525371316 | China | Asia | 10015 | 2014-05-15
 18 | A42158473832 | 325371318 | China | Asia | 10017 | 2014-05-15
 19 | A42152474834 | 225371319 | US | America | 10018 | 2014-05-15
 21 | A42158474137 | 325371321 | Japan | Asia | 10020 | 2014-05-15
 23 | A42158475838 | 425371323 | UK | OutSpace | 10022 | 2014-05-15
 25 | A41158494840 | 626381324 | Good | OutSpace | 10024 | 2014-05-15
 26 | A41158494843 | 625378824 | NotGood | OutSpace | 10025 | 2014-05-15
 1 | A42158424831 | 125371341 | China | Asia | 1 | 2016-02-12
 2 | A42158473831 | 125371342 | China | Asia | 10001 | 2016-02-12
 3 | A42152474832 | 125371343 | US | America | 10002 | 2016-02-12
 4 | A42151477823 | 125371344 | China | OutSpace | 10003 | 2016-02-12
 5 | A42158474135 | 125371345 | Japan | OutSpace | 10004 | 2016-02-12
 9 | A42158424831 | 225371349 | China | OutSpace | 10008 | 2016-02-12
 10 | A42158473830 | 225371310 | China | Asia | 10009 | 2016-02-12
 11 | A42152474830 | 325371311 | US | America | 10010 | 2016-02-12
 13 | A42158474133 | 325371313 | Japan | Asia | 10012 | 2016-02-12
 17 | A42158424833 | 425371317 | China | Asia | 10016 | 2016-02-12
(26 rows)

> presto carbon not support query partition table
> ---
>
> Key: CARBONDATA-2282
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2282
> Project: CarbonData
>  Issue Type: Bug
>  Components: core, presto-integration
>Affects Versions: 1.3.0
>Reporter: zhangwei
>Assignee: anubhav tarar
>Priority: Major
> Fix For: 1.3.0
>
> Attachments: partitonToPrune.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CARBONDATA-2282) presto carbon not support query partition table

2018-03-26 Thread zhangwei (JIRA) 

[ 
https://issues.apache.org/jira/browse/CARBONDATA-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16413528#comment-16413528
 ] 

zhangwei commented on CARBONDATA-2282:
--

presto doesn't support query partition table ,so I add express only contain 
partition information to configuration and then use expression to evaluate the 
partition from each blocklet,  the blocklet which match the express will be 
recognized as the one need to be loaded

> presto carbon not support query partition table
> ---
>
> Key: CARBONDATA-2282
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2282
> Project: CarbonData
>  Issue Type: Bug
>  Components: core, presto-integration
>Affects Versions: 1.3.0
>Reporter: zhangwei
>Priority: Major
> Fix For: 1.3.0
>
> Attachments: partitonToPrune.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)