Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2869#discussion_r230273015
--- Diff:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonReaderBuilder.java
---
@@ -158,14 +173,31 @@ public CarbonReaderBuilder withHadoopConf(String key,
String value) {
}
try {
- final List<InputSplit> splits =
- format.getSplits(new JobContextImpl(job.getConfiguration(), new
JobID()));
-
+ List<InputSplit> splits;
+ if (filterExpression == null) {
+ splits = format.getAllFileSplits(job);
+ } else {
+ splits = format.getSplits(new
JobContextImpl(job.getConfiguration(), new JobID()));
+ }
List<RecordReader<Void, T>> readers = new ArrayList<>(splits.size());
for (InputSplit split : splits) {
TaskAttemptContextImpl attempt =
new TaskAttemptContextImpl(job.getConfiguration(), new
TaskAttemptID());
- RecordReader reader = format.createRecordReader(split, attempt);
+ RecordReader reader;
+ QueryModel queryModel = format.createQueryModel(split, attempt);
+ boolean hasComplex = false;
+ for (ProjectionDimension projectionDimension :
queryModel.getProjectionDimensions()) {
+ if (projectionDimension.getDimension().isComplex()) {
--- End diff --
Vectorised Reader is not supported for complex types. When carbonsession
support complex then we can enable for SDK alse.
---
