Re: [DISCUSSION] Remove BTree related code

[David CaiQiang]

+0 for 1. delete 11 files

Better to add Start/End keys to DataMapRow also.
In my opinion, the union of Min/Max values and Start/End keys can work
better.



-
Best Regards
David Cai
--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [DISCUSSION] Support Standard Spark's FileFormat interface in Carbondata

[xm_zzc]

+1, Good feature.



--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [DISCUSSION] Support Standard Spark's FileFormat interface in Carbondata

[manishgupta88]

+1

Regards
Manish Gupta



--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [DISCUSSION] Support Standard Spark's FileFormat interface in Carbondata

[Liang Chen]

HI

+1, agree to support standard spark file format interface in carbondata, it
will be significantly helpful for broadening apache carbondata's ecosystem.

Regards
Liang


ravipesala wrote
> Hi,
> 
> Current Carbondata has deep integration with Spark to provide
> optimizations
> in performance and also supports features like compaction, IUD, data maps
> and metadata management etc. This type of integration forces user to use
> CarbonSession instance to use carbon even for read and write operations.
> 
> So I am proposing standard spark's FileFormat implementation in carbon for
> simple integration with Spark. Please check the jira for the design
> document.
> https://issues.apache.org/jira/browse/CARBONDATA-2872
> 
> -- 
> Thanks & Regards,
> Ravindra





--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[DISCUSSION] Remove BTree related code

[manhua]

Hi All, 
 Since I read latest code of carbon and found that BTree related code is
only used by a test class called`BTreeBlockFinderTest`. So I try delete
those codes and test shows it works fine. But I wonder whether to delete
those code now or anyone thinks it can be used for something else ?


---changes I did:

1. delete 11 files 
AbstractBTreeBuilder.java
AbstractBTreeLeafNode.java
BlockBTreeBuilder.java
BlockBTreeLeafNode.java
BTreeDataRefNodeFinder.java
BTreeNode.java
BTreeNonLeafNode.java
BtreeBuilder.java
BTreeBuilderInfo.java
DataRefNodeFinder.java
BTreeBlockFinderTest.java

2. remove following lines of codes and only keep codes in if block, because
all `dataRefNode` are instance of BlockletDataRefNode which created in
IndexWrapper

https://github.com/apache/carbondata/blob/6029b28006d54db74682e9348a92bebdf518d07d/core/src/main/java/org/apache/carbondata/core/scan/result/iterator/AbstractDetailQueryResultIterator.java#L131-L155





--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[DISCUSSION] Implement file-level Min/Max index for streaming segment

[David CaiQiang]

Hi All,
Currently, the filter queries on the streaming table always scan all
streaming files, even though there are no data in streaming files that meet
the filter conditions.
So I try to support file-level min/max index on streaming segment. It
helps to reduce the task number and improve the performance of filter scan
in some cases.
Please check the document in JIRA:  
https://issues.apache.org/jira/browse/CARBONDATA-2853
  
Any question, suggestion?



-
Best Regards
David Cai
--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/