Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2850#discussion_r230315785
--- Diff:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonReader.java ---
@@ -114,6 +115,57 @@ public static CarbonReaderBuilder builder(String
tablePath) {
return builder(tablePath, tableName);
}
+ /**
+ * Breaks the list of CarbonRecordReader in CarbonReader into multiple
+ * CarbonReader objects, each iterating through some 'carbondata' files
+ * and return that list of CarbonReader objects
+ *
+ * If the no. of files is greater than maxSplits, then break the
+ * CarbonReader into maxSplits splits, with each split iterating
+ * through >= 1 file.
+ *
+ * If the no. of files is less than maxSplits, then return list of
+ * CarbonReader with size as the no. of files, with each CarbonReader
+ * iterating through exactly one file
+ *
+ * @param maxSplits: Int
+ * @return list of {@link CarbonReader} objects
+ */
+ public List<CarbonReader> split(int maxSplits) throws IOException {
+ validateReader();
+ if (maxSplits < 1) {
+ throw new RuntimeException(
+ this.getClass().getSimpleName() + ".split: maxSplits must be
positive");
+ }
+
+ List<CarbonReader> carbonReaders = new ArrayList<>();
+
+ if (maxSplits < this.readers.size()) {
--- End diff --
@ravipesala Let us add test cases in a separate PR. would it be okay?
---
