This is an automated email from the ASF dual-hosted git repository.
voonhous pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new 989badc3952e fix(common): normalize hudi table base path in
implicit-key lock providers (#18814)
989badc3952e is described below
commit 989badc3952e05052754dc1f028800c660646aee
Author: Davis-Zhang-Onehouse
<[email protected]>
AuthorDate: Mon Aug 17 10:21:10 2026 -0700
fix(common): normalize hudi table base path in implicit-key lock providers
(#18814)
* fix(common): normalize hudi table base path in implicit-key lock providers
The implicit-key lock providers -
DynamoDBBasedImplicitPartitionKeyLockProvider
and ZookeeperBasedImplicitBasePathLockProvider - hash the hudi table base
path
to derive a DynamoDB partition key / Zookeeper znode for the lock. The hash
function (XXH64) is avalanche: any byte-level difference in the input
produces
a completely different output. Today the only normalization applied before
hashing is s3aToS3 (s3a:// -> s3://). Trailing slashes, repeated slashes,
and
surrounding whitespace pass through unchanged.
When two writers for the same hudi table disagree on those benign formatting
details - e.g. one engine supplies "s3://bucket/table" while another
supplies
"s3://bucket/table/" - they end up acquiring different lock rows / znodes
and
lose mutual exclusion, even though they are targeting the same table. Two
writers that should serialize can then write concurrently and corrupt the
hudi timeline.
Fix: introduce FSUtils.normalizeBasePathForLocking() as the single source of
truth for canonicalization before hashing:
1. Reject null / empty / whitespace-only basePath.
2. trim() surrounding whitespace.
3. Apply existing s3aToS3 (case-insensitive s3a:// -> s3://).
4. Strip all trailing '/' then add exactly one.
Both implicit-key lock providers now route through it. The Dynamo provider
gains a small public static derivePartitionKey(String) so the formula is
testable without a DynamoDB client.
Inner consecutive slashes (s3://b//x vs s3://b/x) are intentionally NOT
collapsed - they can resolve to a legitimate S3 key.
Tests:
- TestFSUtils#testNormalizeBasePathForLocking exercises the normalization
rules directly: trailing slash, multi-slash, whitespace, s3a/s3 schemes,
inner-slash preservation, null/empty rejection.
- TestDynamoDBBasedImplicitPartitionKeyLockProvider verifies that all
trailing-slash, multi-slash, whitespace, and s3a variants of the same
base path produce the same DynamoDB partition key.
- TestZookeeperBasedImplicitBasePathLockProvider verifies the same
invariant for the Zookeeper lock base path.
Compatibility note: locks taken under the previous (no-trailing-slash or
whitespace-sensitive) form effectively orphan at deploy time, since the new
code looks at a different lock row / znode for the same logical table.
Deploys should be coordinated across all writers that share a hudi table,
or accept a brief writer quiesce while rolling out.
* Address review: reject scheme-only paths, SLF4J logging, IT fix
- normalizeBasePathForLocking: reject scheme-only inputs (s3://, s3a:///)
and all-slash inputs that strip to nothing meaningful to hash
- DynamoDBBasedImplicitPartitionKeyLockProvider#derivePartitionKey:
switch String.format to SLF4J parameterized logging (consistent with
the sibling ZK provider); javadoc note explaining the static helper
accepts raw input (super-constructor ordering precludes using the
instance field)
- ZookeeperBasedImplicitBasePathLockProvider#getLockBasePath: same
javadoc note for symmetry
- ITTestDynamoDBBasedLockProvider#testAcquireLock: compare against the
normalized (trailing-slash) form of the hash input — the previous
assertion would have failed after the canonicalization change
- TestFSUtils: cover s3:///, s3a:///, all-slash, and a special-char path
* Address review: rename field to normalizedHudiTableBasePath
The private field stores the post-normalization basePath; rename it so
the name reflects that storage shape, matching what the constructor
actually assigns.
* Address review: narrow scheme-only reject so S3 keys ending in ':' pass
The previous heuristic rejected any input whose stripped form ended with
':', which inadvertently rejected legitimate S3 keys like
"s3://bucket/foo:" or "s3://bucket/foo:bar:" — S3 object keys are
allowed to contain ':'. That was a silent behavior change vs the
pre-PR s3aToS3-only code.
Narrow the rejection: only treat ":"-terminating inputs as scheme-only
when no '/' character remains in the stripped form. Scheme-only inputs
("s3://", "s3a:///") collapse to "<scheme>:" with no '/' and are still
rejected; legitimate paths retain the '/' characters from the "://"
separator and pass through.
Tests cover both the new accept-path (final segment ending in ':') and
the still-rejected scheme-only cases.
* Address review: canonicalize base path without trailing slash
Per @yihua's review: previous releases hashed the basePath after only
s3aToS3 (no trailing slash), and most callers supply a basePath without
a trailing slash. Forcing exactly one trailing slash therefore changed
the derived lock key for the common case vs the prior release.
Strip all trailing slashes instead of appending one. This keeps the
canonical form (and the resulting DynamoDB partition key / ZK znode)
identical to the prior release for the common no-trailing-slash callers,
while still collapsing trailing-slash / multi-slash / whitespace / s3a
variants of the same table to a single lock key.
Updated TestFSUtils#testNormalizeBasePathForLocking and the
ITTestDynamoDBBasedLockProvider assertion to the no-trailing-slash form.
* review(18814): fix normalizer idempotence, pin lock keys, cover contention
- normalizeBasePathForLocking now strips trailing '/' and whitespace in a
single
pass. trim-then-strip left a trailing space on "s3://b/t /", which hashed
to a
different lock than "s3://b/t" and broke the idempotence both providers
document.
- Pin the derived DynamoDB partition key and ZK znode to golden literals,
and assert
the no-trailing-slash form still equals hash(s3aToS3(basePath)), so the
upgrade-stability claim is enforced somewhere that actually runs. The
only prior
golden assertion lives in an IT that has been @Disabled since HUDI-7475.
- Add a two-writer contention test on the existing TestingServer harness:
base paths
differing only by a trailing slash must contend for one znode. Fails on
master.
- Route InProcessLockProvider through the same helper. Degenerate paths
such as
HoodieTimeGeneratorConfig.defaultConfig("") fall back to the raw string.
- Collapse the duplicated provider tests into parameterized ones and drop
the
s3a-scheme cases, which passed with or without the fix.
- Document the rollout for trailing-slash tables and what is deliberately
left
un-normalized. Revert the no-op @Disabled IT hunk. Plain ASCII.
* review(18814): drop the InProcessLockProvider change from this PR
It is a distinct bug in a distinct provider (in-process map, no rollout
implications), so it belongs in its own PR rather than riding along with the
implicit-key lock provider fix.
It also does not apply here. Master's package reorg (#19193, 3cc8fd128b40)
moved
the real class to org.apache.hudi.core.transaction.lock and left a
@Deprecated
@CompatAlias shim at the old path, so this edit both conflicted with master
and
would have landed on the shim. The follow-up needs to be authored against
core/transaction/lock/InProcessLockProvider.java:79, which still keys
LOCK_INSTANCE_PER_BASEPATH on the raw base path.
* review(18814): fix checkstyle breakage and close the residual idempotence
hole
Round-2 review of the previous commit turned up two build blockers and one
real
correctness gap.
- Two new test methods were named `aBasePath...`, which violates checkstyle
MethodName (^[a-z][a-z0-9][a-zA-Z0-9_]*$) and, since checkstyle is bound
to the
compile phase with failOnViolation and includeTestSourceDirectory, failed
the
build for hudi-aws and hudi-client-common. Renamed.
- The strip loop used Character.isWhitespace while the preceding trim() uses
c <= ' '. Those are different sets: U+0000-U+0008 and U+000E-U+001B are
stripped
by trim but not by isWhitespace, so a path ending in one of those
followed by a
slash still normalized to a value that itself normalized further, i.e. it
was
not idempotent. The loop now uses c <= ' ' to match trim exactly. Fuzzing
all
inputs up to length 4 over an alphabet of slash, every whitespace class,
control
chars, colon and letters: 44266 accepted, 0 non-idempotent, 0 that reject
their
own output. Added control-char cases to the idempotence test.
- Documented that scheme roots are rejected for every scheme, not just s3,
and
that this is a behaviour change rather than tightened validation: those
inputs
previously hashed to a working key and now fail at provider construction.
Added
file:/, file://, file:///, hdfs:/ and gs:/// to the reject test.
- Documented the trailing-whitespace trade-off: a base path differing only
by
trailing whitespace now shares a lock. Over-serializing is the safe
direction.
- Restored the import order in
DynamoDBBasedImplicitPartitionKeyLockProvider to
match master, which fixed it in #18886 and raised ImportOrder to
severity=error.
Without this the file fails checkstyle once rebased.
- Nested the contention test's two providers so writerA's curator client is
closed
if writerB's constructor throws.
Verified with checkstyle 9.3 against both this branch's config and master's
stricter one: 0 violations in all 7 changed files.
* review(18814): warn when a lock key actually moves, and correct two
javadocs
Round-3 review items.
- Nothing signalled when a writer's lock key had actually changed. Both
providers
logged raw + normalized at INFO on every construction, so the writers
that must
not do a rolling upgrade looked identical in the logs to the ones that
are fine.
Both now emit a WARN, naming the old and new key, exactly when the
canonical
form differs from the pre-change s3aToS3 form. Verified it stays silent
for
s3a:// inputs, whose key did not move.
- The javadoc justified leaving inner consecutive slashes and scheme
spelling
alone on the grounds that collapsing them "could map unrelated tables
onto one
lock". That is false: HoodieTableMetaClient:196 wraps the base path in a
StoragePath, which collapses both, so s3://b/x//t and s3://b/x/t are the
same
table and still derive different lock keys. Reworded as the known
residual gap
it is, with the reason it is out of scope here. URL encoding stays a
genuine
deliberate exclusion.
- The ZK provider's javadoc pointed at a caller that does not exist: the
normalizedHudiTableBasePath field is only ever read by
generateLogSuffixString
and is never passed back into getLockBasePath. Dropped the parenthetical.
- Marked derivePartitionKey @VisibleForTesting; its only out-of-class
caller is
the same-package test.
Checkstyle 9.3 against both this branch's config and master's: 0 violations.
* review(18814): drop the @VisibleForTesting annotation to keep the merge
clean
The annotation's import was the only line making this file's hudi import
group
differ from master's, which #18886 had already rewritten. Both sides
editing the
same block made the merge conflict even though the results were compatible.
The
annotation was cosmetic (the only out-of-class caller is a same-package
test), so
dropping it is cheaper than asking a committer to hand-resolve an import
block.
---------
Co-authored-by: voon <[email protected]>
---
...amoDBBasedImplicitPartitionKeyLockProvider.java | 55 ++++++++---
...amoDBBasedImplicitPartitionKeyLockProvider.java | 90 ++++++++++++++++++
...ZookeeperBasedImplicitBasePathLockProvider.java | 46 ++++++++--
.../TestZookeeperBasedLockProvider.java | 53 +++++++++++
...ZookeeperBasedImplicitBasePathLockProvider.java | 90 ++++++++++++++++++
.../java/org/apache/hudi/common/fs/FSUtils.java | 80 ++++++++++++++++
.../org/apache/hudi/common/fs/TestFSUtils.java | 101 ++++++++++++++++++++-
7 files changed, 493 insertions(+), 22 deletions(-)
diff --git
a/hudi-aws/src/main/java/org/apache/hudi/aws/transaction/lock/DynamoDBBasedImplicitPartitionKeyLockProvider.java
b/hudi-aws/src/main/java/org/apache/hudi/aws/transaction/lock/DynamoDBBasedImplicitPartitionKeyLockProvider.java
index 1ada2c42949a..11fd1ba7ec58 100644
---
a/hudi-aws/src/main/java/org/apache/hudi/aws/transaction/lock/DynamoDBBasedImplicitPartitionKeyLockProvider.java
+++
b/hudi-aws/src/main/java/org/apache/hudi/aws/transaction/lock/DynamoDBBasedImplicitPartitionKeyLockProvider.java
@@ -30,6 +30,7 @@ import
software.amazon.awssdk.services.dynamodb.DynamoDbClient;
import javax.annotation.concurrent.NotThreadSafe;
+import static org.apache.hudi.common.fs.FSUtils.normalizeBasePathForLocking;
import static org.apache.hudi.common.fs.FSUtils.s3aToS3;
/**
@@ -41,8 +42,8 @@ import static org.apache.hudi.common.fs.FSUtils.s3aToS3;
public class DynamoDBBasedImplicitPartitionKeyLockProvider extends
DynamoDBBasedLockProviderBase {
protected static final Logger LOG =
LoggerFactory.getLogger(DynamoDBBasedImplicitPartitionKeyLockProvider.class);
- private final String hudiTableBasePath;
-
+ private final String normalizedHudiTableBasePath;
+
public DynamoDBBasedImplicitPartitionKeyLockProvider(final LockConfiguration
lockConfiguration, final StorageConfiguration<?> conf) {
this(lockConfiguration, conf, null);
}
@@ -50,24 +51,56 @@ public class DynamoDBBasedImplicitPartitionKeyLockProvider
extends DynamoDBBased
public DynamoDBBasedImplicitPartitionKeyLockProvider(
final LockConfiguration lockConfiguration, final StorageConfiguration<?>
conf, DynamoDbClient dynamoDB) {
super(lockConfiguration, conf, dynamoDB);
- hudiTableBasePath =
s3aToS3(lockConfiguration.getConfig().getString(HoodieCommonConfig.BASE_PATH.key()));
+ normalizedHudiTableBasePath = normalizeBasePathForLocking(
+
lockConfiguration.getConfig().getString(HoodieCommonConfig.BASE_PATH.key()));
+ }
+
+ /**
+ * Compute the DynamoDB partition key for a given Hudi table base path.
Exposed as a static
+ * helper so that the formula is testable without standing up a DynamoDB
client.
+ *
+ * <p>Accepts a raw basePath - normalization is applied here. {@code
normalizeBasePathForLocking}
+ * is idempotent, so passing an already-normalized path is safe. Note that
the instance field
+ * {@code normalizedHudiTableBasePath} cannot be used here: the parent
constructor invokes this
+ * through {@code getDynamoDBPartitionKey} before the subclass has a chance
to assign the field.
+ * That ordering is also why the path is normalized twice per construction -
once here and once
+ * for the field; it is a pure function over a short string, run once per
provider.
+ *
+ * <p>ROLLOUT: for a table whose configured {@code hoodie.base.path} ends in
'/' or carries
+ * surrounding whitespace, this returns a different DynamoDB partition key
than releases before
+ * HUDI's normalization fix. Such a table must have all of its writers
upgraded together - a
+ * rolling upgrade would leave old and new writers on two different lock
rows for the same
+ * table, losing mutual exclusion. Base paths without a trailing slash are
unaffected.
+ */
+ public static String derivePartitionKey(String hudiTableBasePath) {
+ String normalized = normalizeBasePathForLocking(hudiTableBasePath);
+ String partitionKey = HashID.generateXXHashAsString(normalized,
HashID.Size.BITS_64);
+ LOG.info("The DynamoDB partition key of the lock provider for the base
path {} (normalized: {}) is {}",
+ hudiTableBasePath, normalized, partitionKey);
+ // Releases before this change hashed s3aToS3(basePath) directly. When the
canonical form
+ // differs, this writer has moved to a new lock row and cannot exclude a
writer still running
+ // the old code, so say so loudly rather than leaving it to be inferred
from the INFO line.
+ String legacyForm = s3aToS3(hudiTableBasePath);
+ if (!legacyForm.equals(normalized)) {
+ LOG.warn("DynamoDB partition key for base path {} moved from {} to {}.
Every writer of this "
+ + "table must be upgraded together; a writer still on the
previous release locks on "
+ + "the old partition key and will NOT be excluded by this one.",
+ hudiTableBasePath,
+ HashID.generateXXHashAsString(legacyForm, HashID.Size.BITS_64),
+ partitionKey);
+ }
+ return partitionKey;
}
@Override
public String getDynamoDBPartitionKey(LockConfiguration lockConfiguration) {
- // Ensure consistent format for S3 URI.
- String hudiTableBasePathNormalized =
s3aToS3(lockConfiguration.getConfig().getString(
- HoodieCommonConfig.BASE_PATH.key()));
- String partitionKey =
HashID.generateXXHashAsString(hudiTableBasePathNormalized, HashID.Size.BITS_64);
- LOG.info(String.format("The DynamoDB partition key of the lock provider
for the base path %s is %s",
- hudiTableBasePathNormalized, partitionKey));
- return partitionKey;
+ return
derivePartitionKey(lockConfiguration.getConfig().getString(HoodieCommonConfig.BASE_PATH.key()));
}
@Override
protected String generateLogSuffixString() {
return StringUtils.join("DynamoDb table = ", tableName,
", partition key = ", dynamoDBPartitionKey,
- ", hudi table base path = ", hudiTableBasePath);
+ ", hudi table base path = ", normalizedHudiTableBasePath);
}
}
diff --git
a/hudi-aws/src/test/java/org/apache/hudi/aws/transaction/lock/TestDynamoDBBasedImplicitPartitionKeyLockProvider.java
b/hudi-aws/src/test/java/org/apache/hudi/aws/transaction/lock/TestDynamoDBBasedImplicitPartitionKeyLockProvider.java
new file mode 100644
index 000000000000..7030b69c9b8a
--- /dev/null
+++
b/hudi-aws/src/test/java/org/apache/hudi/aws/transaction/lock/TestDynamoDBBasedImplicitPartitionKeyLockProvider.java
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.aws.transaction.lock;
+
+import org.apache.hudi.common.util.hash.HashID;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+/**
+ * Exercises {@link
DynamoDBBasedImplicitPartitionKeyLockProvider#derivePartitionKey} as a pure
+ * function - no DynamoDB client required.
+ *
+ * <p>Two writers on the same Hudi table must derive the same DynamoDB
partition key, or they take
+ * independent locks and lose mutual exclusion. This class pins both halves of
that contract: the
+ * exact key produced for the canonical base path (so the formula cannot move
silently), and the
+ * set of benign formatting variants that must fold onto it.
+ */
+class TestDynamoDBBasedImplicitPartitionKeyLockProvider {
+
+ private static final String CANONICAL_BASE_PATH =
"s3://my-bucket/my_lake/my_table";
+
+ /**
+ * Golden value. Deliberately a literal rather than a recomputation: an
equality-only test
+ * ({@code key(a).equals(key(b))}) still passes if the whole derivation
changes, which is
+ * exactly how a lock-key scheme change ships undetected.
+ */
+ private static final String CANONICAL_PARTITION_KEY = "C0E15D0CE1AD11CC";
+
+ @Test
+ void derivesThePinnedPartitionKeyForTheCanonicalBasePath() {
+ assertEquals(CANONICAL_PARTITION_KEY,
+
DynamoDBBasedImplicitPartitionKeyLockProvider.derivePartitionKey(CANONICAL_BASE_PATH));
+ }
+
+ @Test
+ void basePathWithoutTrailingSlashKeepsThePreNormalizationKey() {
+ // Releases before the normalization fix hashed s3aToS3(basePath)
directly. For the common
+ // no-trailing-slash form the canonicalized input is byte-identical, so
the partition key must
+ // not move - otherwise every deployed lock row is orphaned on upgrade.
+ assertEquals(HashID.generateXXHashAsString(CANONICAL_BASE_PATH,
HashID.Size.BITS_64),
+
DynamoDBBasedImplicitPartitionKeyLockProvider.derivePartitionKey(CANONICAL_BASE_PATH));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {
+ "s3://my-bucket/my_lake/my_table/",
+ "s3://my-bucket/my_lake/my_table//",
+ "s3://my-bucket/my_lake/my_table///",
+ " s3://my-bucket/my_lake/my_table ",
+ "\ts3://my-bucket/my_lake/my_table/\n",
+ // Whitespace in front of the trailing slash: the strip must consume
both, otherwise a
+ // trailing space survives and this hashes to a different row than the
canonical form.
+ "s3://my-bucket/my_lake/my_table /",
+ "s3a://my-bucket/my_lake/my_table",
+ "s3a://my-bucket/my_lake/my_table/",
+ "S3A://my-bucket/my_lake/my_table//",
+ })
+ void benignFormattingVariantsFoldOntoTheCanonicalPartitionKey(String
basePath) {
+ assertEquals(CANONICAL_PARTITION_KEY,
+
DynamoDBBasedImplicitPartitionKeyLockProvider.derivePartitionKey(basePath));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"", " ", "/", "///", "s3://", "s3a:///"})
+ void unlockableBasePathsAreRejected(String basePath) {
+ assertThrows(IllegalArgumentException.class,
+ () ->
DynamoDBBasedImplicitPartitionKeyLockProvider.derivePartitionKey(basePath));
+ }
+}
diff --git
a/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/ZookeeperBasedImplicitBasePathLockProvider.java
b/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/ZookeeperBasedImplicitBasePathLockProvider.java
index e3e26e58c598..587ece951769 100644
---
a/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/ZookeeperBasedImplicitBasePathLockProvider.java
+++
b/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/ZookeeperBasedImplicitBasePathLockProvider.java
@@ -22,7 +22,6 @@ import org.apache.hudi.common.config.HoodieCommonConfig;
import org.apache.hudi.common.config.LockConfiguration;
import org.apache.hudi.common.lock.LockProvider;
import org.apache.hudi.common.util.StringUtils;
-import org.apache.hudi.common.util.ValidationUtils;
import org.apache.hudi.common.util.hash.HashID;
import org.apache.hudi.storage.StorageConfiguration;
@@ -30,6 +29,7 @@ import lombok.extern.slf4j.Slf4j;
import javax.annotation.concurrent.NotThreadSafe;
+import static org.apache.hudi.common.fs.FSUtils.normalizeBasePathForLocking;
import static org.apache.hudi.common.fs.FSUtils.s3aToS3;
/**
@@ -44,25 +44,51 @@ import static org.apache.hudi.common.fs.FSUtils.s3aToS3;
public class ZookeeperBasedImplicitBasePathLockProvider extends
BaseZookeeperBasedLockProvider {
public static final String LOCK_KEY = "lock_key";
- private final String hudiTableBasePath;
+ private final String normalizedHudiTableBasePath;
+ /**
+ * Compute the Zookeeper lock base path for a given Hudi table base path.
+ *
+ * <p>Accepts a raw basePath - normalization is applied here. {@code
normalizeBasePathForLocking}
+ * is idempotent, so an already-normalized value can be passed through
without harm.
+ *
+ * <p>ROLLOUT: for a table whose configured {@code hoodie.base.path} ends in
'/' or carries
+ * surrounding whitespace, this returns a different znode than releases
before HUDI's
+ * normalization fix. Such a table must have all of its writers upgraded
together - a rolling
+ * upgrade would leave old and new writers holding two different znodes for
the same table,
+ * losing mutual exclusion. Base paths without a trailing slash are
unaffected.
+ */
public static String getLockBasePath(String hudiTableBasePath) {
- // Ensure consistent format for S3 URI.
- String lockBasePath = "/tmp/" +
HashID.generateXXHashAsString(s3aToS3(hudiTableBasePath), HashID.Size.BITS_64);
- log.info("The Zookeeper lock key for the base path {} is {}",
hudiTableBasePath, lockBasePath);
+ String normalized = normalizeBasePathForLocking(hudiTableBasePath);
+ String lockBasePath = "/tmp/" + HashID.generateXXHashAsString(normalized,
HashID.Size.BITS_64);
+ log.info("The Zookeeper lock key for the base path {} (normalized: {}) is
{}",
+ hudiTableBasePath, normalized, lockBasePath);
+ // Releases before this change hashed s3aToS3(basePath) directly. When the
canonical form
+ // differs, this writer has moved to a new znode and cannot exclude a
writer still running
+ // the old code, so say so loudly rather than leaving it to be inferred
from the INFO line.
+ String legacyForm = s3aToS3(hudiTableBasePath);
+ if (!legacyForm.equals(normalized)) {
+ log.warn("Zookeeper lock key for base path {} moved from {} to {}. Every
writer of this "
+ + "table must be upgraded together; a writer still on the
previous release locks "
+ + "on the old znode and will NOT be excluded by this one.",
+ hudiTableBasePath,
+ "/tmp/" + HashID.generateXXHashAsString(legacyForm,
HashID.Size.BITS_64),
+ lockBasePath);
+ }
return lockBasePath;
}
public ZookeeperBasedImplicitBasePathLockProvider(final LockConfiguration
lockConfiguration, final StorageConfiguration<?> conf) {
super(lockConfiguration, conf);
- hudiTableBasePath =
s3aToS3(lockConfiguration.getConfig().getString(HoodieCommonConfig.BASE_PATH.key()));
+ normalizedHudiTableBasePath = normalizeBasePathForLocking(
+
lockConfiguration.getConfig().getString(HoodieCommonConfig.BASE_PATH.key()));
}
@Override
protected String getZkBasePath(LockConfiguration lockConfiguration) {
- String hudiTableBasePath =
lockConfiguration.getConfig().getString(HoodieCommonConfig.BASE_PATH.key());
- ValidationUtils.checkArgument(hudiTableBasePath != null);
- return getLockBasePath(hudiTableBasePath);
+ // No explicit null check: TypedProperties#getString already throws
IllegalArgumentException
+ // for a missing key, and getLockBasePath rejects null/blank/unlockable
paths.
+ return
getLockBasePath(lockConfiguration.getConfig().getString(HoodieCommonConfig.BASE_PATH.key()));
}
@Override
@@ -73,6 +99,6 @@ public class ZookeeperBasedImplicitBasePathLockProvider
extends BaseZookeeperBas
@Override
protected String generateLogSuffixString() {
return StringUtils.join("ZkBasePath = ", zkBasePath,
- ", lock key = ", lockKey, ", hudi table base path = ",
hudiTableBasePath);
+ ", lock key = ", lockKey, ", hudi table base path = ",
normalizedHudiTableBasePath);
}
}
diff --git
a/hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/TestZookeeperBasedLockProvider.java
b/hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/TestZookeeperBasedLockProvider.java
index fa1a8329b65f..8bd4a79e272d 100644
---
a/hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/TestZookeeperBasedLockProvider.java
+++
b/hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/TestZookeeperBasedLockProvider.java
@@ -165,6 +165,59 @@ public class TestZookeeperBasedLockProvider {
Assertions.assertEquals(IllegalArgumentException.class,
ex.getCause().getCause().getClass());
}
+ /**
+ * Build an implicit-provider lock config for {@code hudiTableBasePath},
independent of the
+ * shared fixtures above (those are mutated progressively in {@link
#setup()}).
+ */
+ private static LockConfiguration implicitLockConfig(String
hudiTableBasePath) {
+ Properties props = new Properties();
+ props.setProperty(ZK_CONNECT_URL_PROP_KEY, server.getConnectString());
+ props.setProperty(LOCK_ACQUIRE_RETRY_WAIT_TIME_IN_MILLIS_PROP_KEY, "1000");
+ props.setProperty(LOCK_ACQUIRE_RETRY_MAX_WAIT_TIME_IN_MILLIS_PROP_KEY,
"3000");
+ props.setProperty(LOCK_ACQUIRE_CLIENT_NUM_RETRIES_PROP_KEY, "3");
+ props.setProperty(LOCK_ACQUIRE_NUM_RETRIES_PROP_KEY, "3");
+ props.setProperty(ZK_SESSION_TIMEOUT_MS_PROP_KEY, "10000");
+ props.setProperty(ZK_CONNECTION_TIMEOUT_MS_PROP_KEY, "10000");
+ props.setProperty(LOCK_ACQUIRE_WAIT_TIMEOUT_MS_PROP_KEY, "1000");
+ props.setProperty(HoodieCommonConfig.BASE_PATH.key(), hudiTableBasePath);
+ props.setProperty(HoodieTableConfig.HOODIE_TABLE_NAME_KEY,
"ma_po_tofu_is_awesome");
+ return new LockConfiguration(props);
+ }
+
+ /**
+ * Two writers on the SAME table whose base paths differ only by a trailing
slash must contend
+ * for one znode. Before the base path was canonicalized they hashed to two
different znodes and
+ * both acquired, silently losing mutual exclusion and letting concurrent
writers corrupt the
+ * timeline. This is the end-to-end guard for that; the pure-function
coverage lives in
+ * {@code TestZookeeperBasedImplicitBasePathLockProvider}.
+ */
+ @Test
+ void testTrailingSlashBasePathContendsForTheSameLock() {
+ String tableBasePath =
"s3://my-bucket-8b2a4b30/1718662238400/be715573/my_lake/contended_table";
+ // Each constructor starts its own curator client, so they are nested
rather than created
+ // side by side: if writerB's constructor throws, writerA still gets
closed.
+ ZookeeperBasedImplicitBasePathLockProvider writerA =
+ new
ZookeeperBasedImplicitBasePathLockProvider(implicitLockConfig(tableBasePath),
null);
+ try {
+ ZookeeperBasedImplicitBasePathLockProvider writerB =
+ new
ZookeeperBasedImplicitBasePathLockProvider(implicitLockConfig(tableBasePath +
"/"), null);
+ try {
+ Assertions.assertTrue(writerA.tryLock(1000, TimeUnit.MILLISECONDS));
+ // BaseZookeeperBasedLockProvider#tryLock throws rather than returning
false when the
+ // mutex cannot be acquired within the timeout.
+ Assertions.assertThrows(HoodieLockException.class,
+ () -> writerB.tryLock(1000, TimeUnit.MILLISECONDS),
+ "Writer B derived a different znode for the same table and lost
mutual exclusion");
+ } finally {
+ // close() releases the lock if held and then shuts the curator client
down, so it
+ // covers unlock() as well and never throws.
+ writerB.close();
+ }
+ } finally {
+ writerA.close();
+ }
+ }
+
@Test
public void testUnLock() {
ZookeeperBasedLockProvider zookeeperBasedLockProvider = new
ZookeeperBasedLockProvider(zkConfWithZkBasePathAndLockKeyLock, null);
diff --git
a/hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/lock/TestZookeeperBasedImplicitBasePathLockProvider.java
b/hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/lock/TestZookeeperBasedImplicitBasePathLockProvider.java
new file mode 100644
index 000000000000..af6bcd3783c5
--- /dev/null
+++
b/hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/lock/TestZookeeperBasedImplicitBasePathLockProvider.java
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.client.transaction.lock;
+
+import org.apache.hudi.common.util.hash.HashID;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+/**
+ * Exercises {@link
ZookeeperBasedImplicitBasePathLockProvider#getLockBasePath} as a pure function
+ * - no Zookeeper server required. See
+ * {@code
TestZookeeperBasedLockProvider#testTrailingSlashBasePathContendsForTheSameLock}
for the
+ * end-to-end proof that two providers deriving the same znode actually
exclude each other.
+ *
+ * <p>The lock base path is a znode that deployed clusters hold, so this pins
the exact string
+ * rather than only asserting that variants agree with each other.
+ */
+class TestZookeeperBasedImplicitBasePathLockProvider {
+
+ private static final String CANONICAL_BASE_PATH =
"s3://my-bucket/my_lake/my_table";
+
+ /**
+ * Golden value. Deliberately a literal rather than a recomputation: an
equality-only test
+ * ({@code path(a).equals(path(b))}) still passes if the whole derivation
changes, which is
+ * exactly how a lock-key scheme change ships undetected.
+ */
+ private static final String CANONICAL_LOCK_BASE_PATH =
"/tmp/C0E15D0CE1AD11CC";
+
+ @Test
+ void derivesThePinnedLockBasePathForTheCanonicalBasePath() {
+ assertEquals(CANONICAL_LOCK_BASE_PATH,
+
ZookeeperBasedImplicitBasePathLockProvider.getLockBasePath(CANONICAL_BASE_PATH));
+ }
+
+ @Test
+ void basePathWithoutTrailingSlashKeepsThePreNormalizationZnode() {
+ // Releases before the normalization fix hashed s3aToS3(basePath)
directly. For the common
+ // no-trailing-slash form the canonicalized input is byte-identical, so
the znode must not
+ // move - otherwise every in-flight lock is orphaned on upgrade.
+ assertEquals("/tmp/" + HashID.generateXXHashAsString(CANONICAL_BASE_PATH,
HashID.Size.BITS_64),
+
ZookeeperBasedImplicitBasePathLockProvider.getLockBasePath(CANONICAL_BASE_PATH));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {
+ "s3://my-bucket/my_lake/my_table/",
+ "s3://my-bucket/my_lake/my_table//",
+ "s3://my-bucket/my_lake/my_table///",
+ " s3://my-bucket/my_lake/my_table ",
+ "\ts3://my-bucket/my_lake/my_table/\n",
+ // Whitespace in front of the trailing slash: the strip must consume
both, otherwise a
+ // trailing space survives and this hashes to a different znode than the
canonical form.
+ "s3://my-bucket/my_lake/my_table /",
+ "s3a://my-bucket/my_lake/my_table",
+ "s3a://my-bucket/my_lake/my_table/",
+ "S3A://my-bucket/my_lake/my_table//",
+ })
+ void benignFormattingVariantsFoldOntoTheCanonicalLockBasePath(String
basePath) {
+ assertEquals(CANONICAL_LOCK_BASE_PATH,
+ ZookeeperBasedImplicitBasePathLockProvider.getLockBasePath(basePath));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"", " ", "/", "///", "s3://", "s3a:///"})
+ void unlockableBasePathsAreRejected(String basePath) {
+ assertThrows(IllegalArgumentException.class,
+ () ->
ZookeeperBasedImplicitBasePathLockProvider.getLockBasePath(basePath));
+ }
+}
diff --git a/hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java
b/hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java
index 448438aee017..de9b61988a25 100644
--- a/hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java
+++ b/hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java
@@ -789,6 +789,86 @@ public class FSUtils {
return s3aUrl.replaceFirst("(?i)^s3a://", "s3://");
}
+ /**
+ * Canonicalize a Hudi table base path for use as the input to implicit
lock-key derivation.
+ *
+ * <p>Implicit lock providers (DynamoDB and Zookeeper variants) hash this
string to choose the
+ * lock row / znode for a table. Two callers writing to the same table must
produce the same
+ * hash, so any benign formatting drift in the basePath has to be eliminated
before hashing.
+ * This method normalizes s3a:// to s3://, then strips every trailing '/'
and whitespace
+ * character in a single pass.
+ *
+ * <p>The single pass is what makes the result idempotent. Trimming first
and stripping
+ * slashes afterwards leaves a trailing space behind on an input like {@code
"s3://b/t /"},
+ * which would then hash differently from {@code "s3://b/t"} - the very
drift this method
+ * exists to remove.
+ *
+ * <p>No trailing slash is appended: most callers already supply a basePath
without one, so
+ * the canonical form matches what previous releases hashed (which applied
only s3aToS3) for
+ * those callers, keeping the derived lock key stable across the upgrade.
Callers that did
+ * supply a trailing slash (or surrounding whitespace) DO move to a new lock
key, so all
+ * writers of such a table must be upgraded together rather than one at a
time.
+ *
+ * <p>NOT normalized here, and for two different reasons:
+ * <ul>
+ * <li>Inner consecutive slashes ({@code "s3://b//x"} vs {@code
"s3://b/x"}) and scheme
+ * spelling ({@code "file:///x"} vs {@code "file:/x"}) are a KNOWN
RESIDUAL GAP, not a
+ * safety choice. {@code HoodieTableMetaClient} wraps the base path in
a
+ * {@code StoragePath}, which collapses both, so those spellings
genuinely address the
+ * same table while still deriving different lock keys. Left alone
only to keep this
+ * change's rollout surface to the drift actually seen in the field;
closing it moves
+ * more keys and wants its own change.</li>
+ * <li>URL encoding is deliberately left alone - Hudi does not re-encode
paths internally,
+ * and an encoded and decoded spelling are not interchangeable to the
storage layer.</li>
+ * </ul>
+ *
+ * <p>Scheme-root inputs for any scheme (e.g. {@code "s3://"}, {@code
"s3a:///"},
+ * {@code "file:///"}, {@code "hdfs:/"}) and all-slash inputs (e.g. {@code
"/"},
+ * {@code "///"}) are rejected - stripping the trailing slashes from those
leaves nothing
+ * meaningful to lock against. Note this is a behaviour change: those inputs
previously
+ * hashed to a working lock key, and now fail the writer at provider
construction with
+ * {@link IllegalArgumentException}. Paths whose final key segment
legitimately ends with
+ * {@code ':'} (e.g. {@code "s3://bucket/foo:/"}) are preserved - S3 object
keys are allowed
+ * to contain {@code ':'}.
+ *
+ * <p>One consequence of stripping trailing whitespace: a base path that
differs from another
+ * only by trailing whitespace shares its lock. Such a key is legal in S3
but not something
+ * Hudi can address, since {@code StoragePath} keeps it distinct while the
rest of the config
+ * plumbing does not. Over-serializing two such tables is the safe direction
to err in.
+ */
+ public static String normalizeBasePathForLocking(String basePath) {
+ if (basePath == null) {
+ throw new IllegalArgumentException("Hudi table base path cannot be
null");
+ }
+ String trimmed = basePath.trim();
+ if (trimmed.isEmpty()) {
+ throw new IllegalArgumentException("Hudi table base path cannot be
empty");
+ }
+ String schemeNormalized = s3aToS3(trimmed);
+ // Strip trailing slashes and whitespace together, not in two separate
passes - see the
+ // idempotence note above. The whitespace test is deliberately `<= ' '`
rather than
+ // Character.isWhitespace: it has to match String#trim above exactly.
isWhitespace is a
+ // different set (it excludes U+0000-U+0008 and U+000E-U+001B, which trim
does strip), and
+ // any disagreement between the two reopens the non-idempotence this loop
exists to close.
+ int end = schemeNormalized.length();
+ while (end > 0
+ && (schemeNormalized.charAt(end - 1) == '/'
+ || schemeNormalized.charAt(end - 1) <= ' ')) {
+ end--;
+ }
+ // Reject "///"-style inputs (nothing left after stripping) and
scheme-only inputs
+ // like "s3://" / "s3a:///" - those collapse to just "<scheme>:" with no
'/' character
+ // remaining. Real paths that end with ':' (e.g. "s3://bucket/foo:/") keep
the '/'
+ // characters from the scheme's "://" separator, so they pass this check.
+ if (end == 0
+ || (schemeNormalized.charAt(end - 1) == ':'
+ && schemeNormalized.lastIndexOf('/', end - 1) < 0)) {
+ throw new IllegalArgumentException(
+ "Hudi table base path is not a valid lockable path: '" + basePath +
"'");
+ }
+ return schemeNormalized.substring(0, end);
+ }
+
public static StoragePathInfo toStoragePathInfo(HoodieFileStatus fileStatus)
{
if (null == fileStatus) {
return null;
diff --git
a/hudi-hadoop-common/src/test/java/org/apache/hudi/common/fs/TestFSUtils.java
b/hudi-hadoop-common/src/test/java/org/apache/hudi/common/fs/TestFSUtils.java
index 45daacb9eeb2..97263c70b5b4 100644
---
a/hudi-hadoop-common/src/test/java/org/apache/hudi/common/fs/TestFSUtils.java
+++
b/hudi-hadoop-common/src/test/java/org/apache/hudi/common/fs/TestFSUtils.java
@@ -49,7 +49,9 @@ import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.CsvSource;
+import org.junit.jupiter.params.provider.MethodSource;
import org.junit.jupiter.params.provider.ValueSource;
import java.io.IOException;
@@ -790,11 +792,108 @@ public class TestFSUtils extends HoodieCommonTestHarness
{
"gs://my-bucket/path/to/s3a://object",
"gs://my-bucket s3a://my-object",
})
-
void testUriDoesNotChange(String uri) {
assertEquals(uri, FSUtils.s3aToS3(uri));
}
+ static Stream<Arguments> normalizeBasePathForLockingCases() {
+ return Stream.of(
+ // Canonical form strips all trailing slashes (none is appended).
+ Arguments.of("s3://my-bucket/path", "s3://my-bucket/path"),
+ Arguments.of("s3://my-bucket/path/", "s3://my-bucket/path"),
+ Arguments.of("s3://my-bucket/path///", "s3://my-bucket/path"),
+ // s3a:// is normalized to s3:// (delegates to s3aToS3),
case-insensitively.
+ Arguments.of("s3a://my-bucket/path", "s3://my-bucket/path"),
+ Arguments.of("S3A://my-bucket/path/", "s3://my-bucket/path"),
+ // Whitespace surrounding the path is stripped.
+ Arguments.of(" s3://my-bucket/path ", "s3://my-bucket/path"),
+ Arguments.of("\ts3a://my-bucket/path/\n", "s3://my-bucket/path"),
+ // Whitespace BETWEEN the path and its trailing slashes must be
stripped too. Trimming
+ // first and stripping slashes afterwards leaves "s3://my-bucket/path
" here, which
+ // hashes to a different lock than the canonical form and is not
idempotent.
+ Arguments.of("s3://my-bucket/path /", "s3://my-bucket/path"),
+ Arguments.of("s3://my-bucket/path // ", "s3://my-bucket/path"),
+ // Non-S3 schemes pass through (still get trailing-slash stripping).
+ Arguments.of("gs://my-bucket/path", "gs://my-bucket/path"),
+ Arguments.of("gs://my-bucket/path//", "gs://my-bucket/path"),
+ // Inner consecutive slashes are intentionally NOT touched (could be a
real S3 key).
+ Arguments.of("s3://my-bucket//inner/path",
"s3://my-bucket//inner/path"),
+ // S3 object keys are allowed to end with ':' - a final-segment colon
must NOT be
+ // mis-classified as the "scheme-only" case. The trailing ':' is part
of the key and
+ // is preserved after any trailing slashes are stripped.
+ Arguments.of("s3://my-bucket/foo:", "s3://my-bucket/foo:"),
+ Arguments.of("s3://my-bucket/foo:/", "s3://my-bucket/foo:"),
+ Arguments.of("s3://my-bucket/foo:bar:/", "s3://my-bucket/foo:bar:"),
+ Arguments.of("s3a://my-bucket/foo:///", "s3://my-bucket/foo:"));
+ }
+
+ /**
+ * Every benign formatting variant of one table's base path must
canonicalize to the same
+ * string, since implicit lock providers hash this to pick a lock row /
znode.
+ */
+ @ParameterizedTest
+ @MethodSource("normalizeBasePathForLockingCases")
+ void testNormalizeBasePathForLocking(String input, String expected) {
+ assertEquals(expected, FSUtils.normalizeBasePathForLocking(input));
+ }
+
+ @Test
+ void testNormalizeBasePathForLockingPreservesUnusualCharacters() {
+ // URL-unsafe and equals/colon/plus/hash/ampersand/space characters pass
through unchanged
+ // except for the trailing-slash and s3a-scheme rules. Hudi does not
re-encode paths
+ // internally so the lock key must be byte-stable across these characters.
+ assertEquals(
+
"s3://my-bucket/datalake/db=foo:bar/dt=2024-01-01T00:00:00+05:30/region=us
east/category=a&b=c/vehicle#1/file",
+ FSUtils.normalizeBasePathForLocking(
+
"s3a://my-bucket/datalake/db=foo:bar/dt=2024-01-01T00:00:00+05:30/region=us
east/category=a&b=c/vehicle#1/file"));
+ }
+
+ /**
+ * The canonical form must be a fixed point. Both implicit lock providers
document that this
+ * helper is idempotent and pass already-normalized values back through it.
+ */
+ @ParameterizedTest
+ @ValueSource(strings = {
+ "s3://my-bucket/path",
+ "s3://my-bucket/path/",
+ " s3://my-bucket/path ",
+ "s3://my-bucket/path /",
+ "s3://my-bucket/path /// ",
+ // Control characters that String#trim strips but Character#isWhitespace
does not. The
+ // strip loop has to use the same rule as trim, or these reopen the
non-idempotence.
+ "s3://my-bucket/path\u0001/",
+ "s3://my-bucket/path\u001b//",
+ "\u0002s3a://my-bucket/path\u0008/",
+ "s3a://my-bucket/path//",
+ "s3://my-bucket//inner/path/",
+ "s3://my-bucket/foo:/",
+ "gs://my-bucket/path//",
+ })
+ void testNormalizeBasePathForLockingIsIdempotent(String input) {
+ String once = FSUtils.normalizeBasePathForLocking(input);
+ assertEquals(once, FSUtils.normalizeBasePathForLocking(once));
+ }
+
+ @Test
+ void testNormalizeBasePathForLockingRejectsNull() {
+ assertThrows(IllegalArgumentException.class, () ->
FSUtils.normalizeBasePathForLocking(null));
+ }
+
+ /**
+ * Empty, all-slash and scheme-only inputs are rejected - stripping leaves
nothing meaningful
+ * to lock against, and hashing them would collapse unrelated tables onto
one lock.
+ */
+ @ParameterizedTest
+ @ValueSource(strings = {
+ "", " ", "/", "///", " / ",
+ // Scheme roots are rejected for every scheme, not just s3. These
previously hashed to a
+ // working lock key, so this is a behaviour change, not just tightened
validation.
+ "s3://", "s3:///", "s3a://", "s3a:///", "file:/", "file://", "file:///",
"hdfs:/", "gs:///",
+ })
+ void testNormalizeBasePathForLockingRejectsUnlockablePaths(String basePath) {
+ assertThrows(IllegalArgumentException.class, () ->
FSUtils.normalizeBasePathForLocking(basePath));
+ }
+
private StoragePath getHoodieTempDir() {
return new StoragePath(baseUri.toString(), ".hoodie/.temp");
}
