[
https://issues.apache.org/jira/browse/KYLIN-4434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang updated KYLIN-4434:
--------------------------------
Description:
When I submit a build job, such as building the data between
2012-01-01～2012-02-01 of a cube, if it is not submitted successfully for
NoClassDefFoundError, the corresponding segment information will be saved in
the cube storage. The status of this segment is always new and cannot be
rebuilt or refresh.
!image-2020-03-27-11-10-16-554.png|width=562,height=270!
In fact, the following code will delete segment information if an exception
occurs when job be submitted. However, for NoClassDefFoundError, it cannot
catch.
{code:java}CubeSegment newSeg = null;
try {
if (buildType == CubeBuildTypeEnum.BUILD)
{ ISource source = SourceManager.getSource(cube); SourcePartition src = new
SourcePartition(tsRange, segRange, sourcePartitionOffsetStart,
sourcePartitionOffsetEnd); src = source.enrichSourcePartitionBeforeBuild(cube,
src); newSeg = getCubeManager().appendSegment(cube, src); job =
EngineFactory.createBatchCubingJob(newSeg, submitter, priorityOffset); }
else if (buildType == CubeBuildTypeEnum.MERGE)
{ newSeg = getCubeManager().mergeSegments(cube, tsRange, segRange, force); job
= EngineFactory.createBatchMergeJob(newSeg, submitter); }
else if (buildType == CubeBuildTypeEnum.REFRESH)
{ newSeg = getCubeManager().refreshSegment(cube, tsRange, segRange); job =
EngineFactory.createBatchCubingJob(newSeg, submitter, priorityOffset); }
else
{ throw new BadRequestException(String.format(Locale.ROOT,
msg.getINVALID_BUILD_TYPE(), buildType)); }
getExecutableManager().addJob(job);
} catch (Exception e) {
if (newSeg != null) {
logger.error("Job submission might failed for NEW segment {}, will clean the
NEW segment from cube",
newSeg.getName());
try
{ // Remove this segment getCubeManager().updateCubeDropSegments(cube, newSeg);
}
catch (Exception ee)
{ // swallow the exception logger.error("Clean New segment failed, ignoring
it", e); }
}
throw e;
}{code}
was:
When I submit a build job, such as building the data between
2012-01-01～2012-02-01 of a cube, it is not submitted successfully for
NoClassDefFoundError, but the corresponding segment information is saved in the
cube storage. The status of this segment is always new and cannot be rebuilt or
refresh.
!image-2020-03-27-11-10-16-554.png|width=562,height=270!
In fact, the following code will delete segment information if an exception
occurs when job be submitted. However, for NoClassDefFoundError, it cannot
catch.
{code:java}CubeSegment newSeg = null;
try {
if (buildType == CubeBuildTypeEnum.BUILD)
{ ISource source = SourceManager.getSource(cube); SourcePartition src = new
SourcePartition(tsRange, segRange, sourcePartitionOffsetStart,
sourcePartitionOffsetEnd); src = source.enrichSourcePartitionBeforeBuild(cube,
src); newSeg = getCubeManager().appendSegment(cube, src); job =
EngineFactory.createBatchCubingJob(newSeg, submitter, priorityOffset); }
else if (buildType == CubeBuildTypeEnum.MERGE)
{ newSeg = getCubeManager().mergeSegments(cube, tsRange, segRange, force); job
= EngineFactory.createBatchMergeJob(newSeg, submitter); }
else if (buildType == CubeBuildTypeEnum.REFRESH)
{ newSeg = getCubeManager().refreshSegment(cube, tsRange, segRange); job =
EngineFactory.createBatchCubingJob(newSeg, submitter, priorityOffset); }
else
{ throw new BadRequestException(String.format(Locale.ROOT,
msg.getINVALID_BUILD_TYPE(), buildType)); }
getExecutableManager().addJob(job);
} catch (Exception e) {
if (newSeg != null) {
logger.error("Job submission might failed for NEW segment {}, will clean the
NEW segment from cube",
newSeg.getName());
try
{ // Remove this segment getCubeManager().updateCubeDropSegments(cube, newSeg);
}
catch (Exception ee)
{ // swallow the exception logger.error("Clean New segment failed, ignoring
it", e); }
}
throw e;
}{code}
> The segment build job was not submitted successfully, but the storage of the
> cube has saved this segment.
> ---------------------------------------------------------------------------------------------------------
>
> Key: KYLIN-4434
> URL: https://issues.apache.org/jira/browse/KYLIN-4434
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
> Affects Versions: v3.0.1
> Reporter: Yaqian Zhang
> Assignee: Yaqian Zhang
> Priority: Minor
> Fix For: v3.1.0
>
> Attachments: image-2020-03-27-11-10-16-554.png
>
>
> When I submit a build job, such as building the data between
> 2012-01-01～2012-02-01 of a cube, if it is not submitted successfully for
> NoClassDefFoundError, the corresponding segment information will be saved in
> the cube storage. The status of this segment is always new and cannot be
> rebuilt or refresh.
> !image-2020-03-27-11-10-16-554.png|width=562,height=270!
> In fact, the following code will delete segment information if an exception
> occurs when job be submitted. However, for NoClassDefFoundError, it cannot
> catch.
> {code:java}CubeSegment newSeg = null;
> try {
> if (buildType == CubeBuildTypeEnum.BUILD)
> { ISource source = SourceManager.getSource(cube); SourcePartition src = new
> SourcePartition(tsRange, segRange, sourcePartitionOffsetStart,
> sourcePartitionOffsetEnd); src =
> source.enrichSourcePartitionBeforeBuild(cube, src); newSeg =
> getCubeManager().appendSegment(cube, src); job =
> EngineFactory.createBatchCubingJob(newSeg, submitter, priorityOffset); }
> else if (buildType == CubeBuildTypeEnum.MERGE)
> { newSeg = getCubeManager().mergeSegments(cube, tsRange, segRange, force);
> job = EngineFactory.createBatchMergeJob(newSeg, submitter); }
> else if (buildType == CubeBuildTypeEnum.REFRESH)
> { newSeg = getCubeManager().refreshSegment(cube, tsRange, segRange); job =
> EngineFactory.createBatchCubingJob(newSeg, submitter, priorityOffset); }
> else
> { throw new BadRequestException(String.format(Locale.ROOT,
> msg.getINVALID_BUILD_TYPE(), buildType)); }
> getExecutableManager().addJob(job);
> } catch (Exception e) {
> if (newSeg != null) {
> logger.error("Job submission might failed for NEW segment {}, will clean the
> NEW segment from cube",
> newSeg.getName());
> try
> { // Remove this segment getCubeManager().updateCubeDropSegments(cube,
> newSeg); }
> catch (Exception ee)
> { // swallow the exception logger.error("Clean New segment failed, ignoring
> it", e); }
> }
> throw e;
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
