[
https://issues.apache.org/jira/browse/HUDI-1584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
张超明 updated HUDI-1584:
----------------------
Description:
The maker file path now start with "*./hoodie/*", which should start with the
target base path set by user.
{code:java}
public static final String METAFOLDER_NAME = ".hoodie";
public static final String TEMPFOLDER_NAME = METAFOLDER_NAME + File.separator +
".temp";
public static final String AUXILIARYFOLDER_NAME = METAFOLDER_NAME +
File.separator + ".aux";
...
@Override
public void prepareSnapshotPreBarrier(long checkpointId) throws Exception {
super.prepareSnapshotPreBarrier(checkpointId);
String instantMarkerFileName = String.format("%d%s%d%s%d",
indexOfThisSubtask, DELIMITER, checkpointId, DELIMITER, recordCounter.get());
Path path = new Path(new Path(HoodieTableMetaClient.AUXILIARYFOLDER_NAME,
INSTANT_MARKER_FOLDER_NAME), instantMarkerFileName);
// create marker file
fs.create(path, true);
LOG.info("Subtask [{}] at checkpoint [{}] created marker file [{}]",
indexOfThisSubtask, checkpointId, instantMarkerFileName);
if (isMain) {
// check whether the last instant is completed, will try specific times
until an exception is thrown
if (!StringUtils.isNullOrEmpty(latestInstant)) {
doCheck();
// last instant completed, set it empty
latestInstant = "";
}
boolean receivedDataInCurrentCP = checkReceivedData(checkpointId);
// no data no new instant
if (receivedDataInCurrentCP) {
latestInstant = startNewInstant(checkpointId);
}
}
}
{code}
was:The maker file path now start with "*./hoodie/*", which should start with
the target base path set by user.
> The maker file path error
> -------------------------
>
> Key: HUDI-1584
> URL: https://issues.apache.org/jira/browse/HUDI-1584
> Project: Apache Hudi
> Issue Type: Bug
> Components: Flink Integration
> Reporter: 张超明
> Priority: Major
> Labels: pull-request-available
>
> The maker file path now start with "*./hoodie/*", which should start with the
> target base path set by user.
>
> {code:java}
> public static final String METAFOLDER_NAME = ".hoodie";
> public static final String TEMPFOLDER_NAME = METAFOLDER_NAME + File.separator
> + ".temp";
> public static final String AUXILIARYFOLDER_NAME = METAFOLDER_NAME +
> File.separator + ".aux";
> ...
> @Override
> public void prepareSnapshotPreBarrier(long checkpointId) throws Exception {
> super.prepareSnapshotPreBarrier(checkpointId);
> String instantMarkerFileName = String.format("%d%s%d%s%d",
> indexOfThisSubtask, DELIMITER, checkpointId, DELIMITER, recordCounter.get());
> Path path = new Path(new Path(HoodieTableMetaClient.AUXILIARYFOLDER_NAME,
> INSTANT_MARKER_FOLDER_NAME), instantMarkerFileName);
> // create marker file
> fs.create(path, true);
> LOG.info("Subtask [{}] at checkpoint [{}] created marker file [{}]",
> indexOfThisSubtask, checkpointId, instantMarkerFileName);
> if (isMain) {
> // check whether the last instant is completed, will try specific times
> until an exception is thrown
> if (!StringUtils.isNullOrEmpty(latestInstant)) {
> doCheck();
> // last instant completed, set it empty
> latestInstant = "";
> }
> boolean receivedDataInCurrentCP = checkReceivedData(checkpointId);
> // no data no new instant
> if (receivedDataInCurrentCP) {
> latestInstant = startNewInstant(checkpointId);
> }
> }
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
