shaofengshi commented on a change in pull request #502: KYLIN-3839 Strorage
clean up after refreshing and deleting segment
URL: https://github.com/apache/kylin/pull/502#discussion_r266297943
##########
File path:
server-base/src/main/java/org/apache/kylin/rest/service/CubeService.java
##########
@@ -550,7 +558,30 @@ public CubeInstance deleteSegment(CubeInstance cube,
String segmentName) throws
logger.warn(String.format(Locale.ROOT,
msg.getDELETE_SEGMENT_CAUSE_GAPS(), cube.getName(), segmentName));
}
- return
CubeManager.getInstance(getConfig()).updateCubeDropSegments(cube, toDelete);
+ CubeInstance cubeInstance =
CubeManager.getInstance(getConfig()).updateCubeDropSegments(cube, toDelete);
+
+ cleanSegmentStorage(Collections.singletonList(toDelete));
+
+ return cubeInstance;
+ }
+
+ // clean segment data in hbase and hdfs
+ private void cleanSegmentStorage(List<CubeSegment> toRemoveSegs) throws
IOException {
+ if (KylinConfig.getInstanceFromEnv().cleanStorageAfterDelOperation()) {
+ if (toRemoveSegs != null) {
+ List<String> toDropHTables =
Lists.newArrayListWithCapacity(toRemoveSegs.size());
+ List<String> toDelHDFSPaths =
Lists.newArrayListWithCapacity(toRemoveSegs.size());
+ for (CubeSegment seg : toRemoveSegs) {
+ toDropHTables.add(seg.getStorageLocationIdentifier());
+
toDelHDFSPaths.add(JobBuilderSupport.getJobWorkingDir(seg.getConfig().getHdfsWorkingDirectory(),
seg.getLastBuildJobID()));
+ }
+
+ if (!toRemoveSegs.isEmpty()) {
Review comment:
Please combine this line with the previous " if (toRemoveSegs != null)"
check.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
