Hi Kunal, This feature looks great from the design.
Still, I need some more clarifications on the below points. (1) How segment deletion will be handled? Whether the next query takes care of clearing this segment and update the driver map or the delete operation will update? (2) Is there any impact on the CLEAN FILES command? (3) Is there any impact on the COLUMN_META_CACHE property? This is a session property and can be changed through ALTER command. If this property is altered, accordingly the current cache implementation will invalidate the datamap cache, in required cases. (4) Executor shut down/failures will be handled by the spark cluster manager? In between query execution, if some executor fails, then the tasks will be re-launched in any other available executors? -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
