GitHub user sujith71955 opened a pull request:
https://github.com/apache/carbondata/pull/2642
[CARBONDATA-2532][Integration] Carbon to support spark 2.3.1 version
In this PR inorder to hide the compatibility issues of columnar vector
API's from the existing common classes, i introduced an interface of the proxy
vector readers, this
proxy vector readers will take care the compatibility issues with respect
to spark different versions.
Column vector and Columnar Batch interface compatibility issues has been
addressed in this PR, The changes were related to below modifications done in
spark interface.
Highlights:
a) This is a refactoring of ColumnVector hierarchy and related classes. By
Sujith
b) make ColumnVector read-only. By Sujith
c) introduce WritableColumnVector with write interface. By Sujith
d) remove ReadOnlyColumnVector. By Sujith
e) Fixed spark-carbon integration API compatibility issues - By sandeep
katta
f) Corrected the testcases based on spark 2.3.0 behaviour change - By
sandeep katta
g) Excluded following dependency from pom.xml files net.jpountzlz4 as spark
2.3.0 changed
it to org.lz4, so removed from the test class path of
spark2,spark-common-test,spark2-examples
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sujith71955/incubator-carbondata
mas_mig_spark2.3_carbon_latest
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2642.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2642
----
commit 7359151b612d3403e53c4759c853e1ab681fae7f
Author: sujith71955 <sujithchacko.2010@...>
Date: 2018-05-24T05:51:50Z
[CARBONDATA-2532][Integration] Carbon to support spark 2.3 version,
ColumnVector Interface
Column vector and Columnar Batch interface compatibility issues has been
addressed in this PR, The changes were related to below modifications
done in spark interface
a) This is a refactoring of ColumnVector hierarchy and related classes.
b) make ColumnVector read-only
c) introduce WritableColumnVector with write interface
d) remove ReadOnlyColumnVector
In this PR inorder to hide the compatibility issues of columnar vector
API's from the existing common classes, i introduced an interface of the
proxy vector readers, this
proxy vector readers will take care the compatibility issues with
respect to spark different versions.
commit 5934d975b53276b2490c6c178ae5b71f539dac60
Author: sandeep-katta <sandeep.katta2007@...>
Date: 2018-07-06T04:31:29Z
[CARBONDATA-2532][Integration] Carbon to support spark 2.3 version,
compatability issues
All compatability issues when supporting 2.3 addressed
Supported pom profile -P"spark-2.3"
----
---
