GitHub user sujith71955 opened a pull request:
https://github.com/apache/carbondata/pull/2365
[CARBONDATA-2532][Integration] Carbon to support spark 2.3 version
```
## What changes were proposed in this pull request?
In this PR inorder to hide the compatibility issues of columnar vector
API's from the existing common classes, i introduced an interface of the proxy
vector readers, this
proxy vector readers will take care the compatibility issues with respect
to spark different versions.
Column vector and Columnar Batch interface compatibility issues has been
addressed in this PR, The changes were related to below modifications done in
spark interface.
Highlights:
a) This is a refactoring of ColumnVector hierarchy and related classes.
b) make ColumnVector read-only
c) introduce WritableColumnVector with write interface
d) remove ReadOnlyColumnVector
e) Fixed spark-carbon integration API compatibility issues - By Sandi
f) Corrected the testcases based on spark 2.3.0 behaviour change
g) Excluded following dependency from pom.xml files
<groupId>net.jpountz</groupId><artifactId>lz4</artifactId> as spark 2.3.0
changed
it to org.lz4, so removed from the test class path of
spark2,spark-common-test,spark2-examples
## How was this patch tested?
Manual testing, and existing test-case execution
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sujith71955/incubator-carbondata
master_spark_carbon_upgradation
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2365.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2365
----
commit 7b0f6a6c68170501368446111c386469d76fa414
Author: sujith71955 <sujithchacko.2010@...>
Date: 2018-05-24T05:51:50Z
[CARBONDATA-2532][Integration] Carbon to support spark 2.3 version
## What changes were proposed in this pull request?
Column vector and Columnar Batch interface compatibility issues has been
addressed in this PR, The changes were related to below modifications done in
spark interface
a) This is a refactoring of ColumnVector hierarchy and related classes.
b) make ColumnVector read-only
c) introduce WritableColumnVector with write interface
d) remove ReadOnlyColumnVector
In this PR inorder to hide the compatibility issues of columnar vector
API's from the existing common classes, i introduced an interface of the proxy
vector readers, this
proxy vector readers will take care the compatibility issues with respect
to spark different versions.
## How was this patch tested?
Manual testing, and existing test-case execution
[CARBONDATA-2532][Integration] Carbon to support spark 2.3 version
## What changes were proposed in this pull request?
Column vector and Columnar Batch interface compatibility issues has been
addressed in this PR, The changes were related to below modifications done in
spark interface
a) This is a refactoring of ColumnVector hierarchy and related classes.
b) make ColumnVector read-only
c) introduce WritableColumnVector with write interface
d) remove ReadOnlyColumnVector
In this PR inorder to hide the compatibility issues of columnar vector
API's from the existing common classes, i introduced an interface of the proxy
vector readers, this
proxy vector readers will take care the compatibility issues with respect
to spark different versions.
## How was this patch tested?
Manual testing, and existing test-case execution
commit 5b0005a9f241b5406fcf621c9f8e107dda83a1b8
Author: sandeep-katta <sandeep.katta2007@...>
Date: 2018-05-25T10:17:48Z
WIP changed to completed.Only 1 testcase related to projection is failing.
Carbon support for 2.3.1
----
---
