[jira] [Commented] (CARBONDATA-222) Query issue for all dimensions are no dictionary columns

2016-09-08 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/CARBONDATA-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15473082#comment-15473082
 ] 

ASF GitHub Bot commented on CARBONDATA-222:
---

Github user foryou2030 commented on a diff in the pull request:

https://github.com/apache/incubator-carbondata/pull/137#discussion_r77957259
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/carbon/datastore/impl/btree/BTreeDataRefNodeFinder.java
 ---
@@ -240,9 +240,16 @@ private int compareIndexes(IndexKey first, IndexKey 
second) {
   firstNoDictionaryKeyBuffer.getShort(nonDictionaryKeyOffset + 
SHORT_SIZE_IN_BYTES);
   secondNodeDictionaryLength =
   secondNoDictionaryKeyBuffer.getShort(nonDictionaryKeyOffset 
+ SHORT_SIZE_IN_BYTES);
-  compareResult = ByteUtil.UnsafeComparer.INSTANCE
-  .compareTo(first.getNoDictionaryKeys(), actualOffset, 
firstNoDcitionaryLength,
-  second.getNoDictionaryKeys(), actualOffset, 
secondNodeDictionaryLength);
+  int minLength = Math.min(firstNoDcitionaryLength, 
secondNodeDictionaryLength);
--- End diff --

ok, handled. thanks for you @gvramana @kumarvishal09 


> Query issue for all dimensions are no dictionary columns
> 
>
> Key: CARBONDATA-222
> URL: https://issues.apache.org/jira/browse/CARBONDATA-222
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Gin-zhj
>Assignee: Gin-zhj
>Priority: Minor
>
> step 1:
> CREATE TABLE uniqdata_no (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION 
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 
> int) STORED BY 'org.apache.carbondata.format' 
> TBLPROPERTIES('DICTIONARY_EXCLUDE'='CUST_NAME,ACTIVE_EMUI_VERSION');
> step 2:
> LOAD DATA INPATH 'D:/download/3lakh_3.csv' into table uniqdata_no 
> OPTIONS('DELIMITER'=',' , 
> 'QUOTECHAR'='"','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> step 3:
> select * from uniqdata_no limit 5;
> the fact file is:
> ,,,0
> query failed, catch exception:
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
>   at 
> org.apache.carbondata.core.util.ByteUtil$UnsafeComparer.compareTo(ByteUtil.java:197)
>   at 
> org.apache.carbondata.core.carbon.datastore.impl.btree.BTreeDataRefNodeFinder.compareIndexes(BTreeDataRefNodeFinder.java:243)
>   at 
> org.apache.carbondata.core.carbon.datastore.impl.btree.BTreeDataRefNodeFinder.findFirstLeafNode(BTreeDataRefNodeFinder.java:121)
>   at 
> org.apache.carbondata.core.carbon.datastore.impl.btree.BTreeDataRefNodeFinder.findFirstDataBlock(BTreeDataRefNodeFinder.java:80)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getDataBlocksOfIndex(CarbonInputFormat.java:546)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getDataBlocksOfSegment(CarbonInputFormat.java:473)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getSplits(CarbonInputFormat.java:342)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getSplitsNonFilter(CarbonInputFormat.java:304)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getSplits(CarbonInputFormat.java:277)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CARBONDATA-222) Query issue for all dimensions are no dictionary columns

2016-09-07 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/CARBONDATA-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470840#comment-15470840
 ] 

ASF GitHub Bot commented on CARBONDATA-222:
---

GitHub user foryou2030 opened a pull request:

https://github.com/apache/incubator-carbondata/pull/137

[CARBONDATA-222]  Handled query issue for all dimensions are no dictionary 
columns

# Why raise this pr?
query failed when all dimensions are no-dictionary column, and the data 
contains one row that all no-dictionary column value are empty
# How to resolve?
when find First Data Block, we need to compare indexes, in this scenario,we 
should use other compareTo method.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/foryou2030/incubator-carbondata no_dict_null

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/137.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #137






> Query issue for all dimensions are no dictionary columns
> 
>
> Key: CARBONDATA-222
> URL: https://issues.apache.org/jira/browse/CARBONDATA-222
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Gin-zhj
>Assignee: Gin-zhj
>Priority: Minor
>
> step 1:
> CREATE TABLE uniqdata_no (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION 
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 
> int) STORED BY 'org.apache.carbondata.format' 
> TBLPROPERTIES('DICTIONARY_EXCLUDE'='CUST_NAME,ACTIVE_EMUI_VERSION');
> step 2:
> LOAD DATA INPATH 'D:/download/3lakh_3.csv' into table uniqdata_no 
> OPTIONS('DELIMITER'=',' , 
> 'QUOTECHAR'='"','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> step 3:
> select * from uniqdata_no limit 5;
> the fact file is:
> ,,,0
> query failed, catch exception:
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
>   at 
> org.apache.carbondata.core.util.ByteUtil$UnsafeComparer.compareTo(ByteUtil.java:197)
>   at 
> org.apache.carbondata.core.carbon.datastore.impl.btree.BTreeDataRefNodeFinder.compareIndexes(BTreeDataRefNodeFinder.java:243)
>   at 
> org.apache.carbondata.core.carbon.datastore.impl.btree.BTreeDataRefNodeFinder.findFirstLeafNode(BTreeDataRefNodeFinder.java:121)
>   at 
> org.apache.carbondata.core.carbon.datastore.impl.btree.BTreeDataRefNodeFinder.findFirstDataBlock(BTreeDataRefNodeFinder.java:80)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getDataBlocksOfIndex(CarbonInputFormat.java:546)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getDataBlocksOfSegment(CarbonInputFormat.java:473)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getSplits(CarbonInputFormat.java:342)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getSplitsNonFilter(CarbonInputFormat.java:304)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getSplits(CarbonInputFormat.java:277)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)