+1 1. Add the impacted areas in design document. 2. If any executor goes down then update the index cache to executor mapping in driver accordingly. 3. Even though the cache would be divided based on index files, the minimum unit of cache need to be fixed. Example: 1 segment cache should belong to 1 executor only. 4. One possible suggestion: Instead of reading the splits in Carbon driver, let each executor in index server write to a file and pass the List[FilePaths] to the driver and let each Carbon Executors to read from that path. This is the case when the number of splits exceed the threshold.
Regards Manish Gupta -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
