Github user sounakr commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2131#discussion_r178594069
--- Diff:
core/src/main/java/org/apache/carbondata/core/datamap/TableDataMap.java ---
@@ -79,26 +80,27 @@
*
* @param segments
* @param filterExp
+ * @param readCommitted
* @return
*/
public List<ExtendedBlocklet> prune(List<Segment> segments,
FilterResolverIntf filterExp,
- List<PartitionSpec> partitions) throws IOException {
+ List<PartitionSpec> partitions, ReadCommitted readCommitted) throws
IOException {
--- End diff --
ReadCommited is implemented in order to set the read scope. Currently for
Unmanaged Table the Read Scope is LatestFileRead which is going to take a
snapshot of the list of Index files during query phase starting i.e. in
CarbonInputSplit. In later phase of query life cycle even though new files get
placed, only files read during initial snapshot is only returned.
Similarly TableStatusReadCommitter is going to take a snapshot of Table
Status file and will be used by Managed Table. One time snapshot of
LoadMetadataDetails will be taken in CarbonTableInputFormat and later the Table
Status snapshot will be only referred to return the list of files. So in
parallel in case Table Status gets updated then new updated segment and file
list wont be reflected. This way we defined the read commit scope.
---
