virajjasani commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-971417508
SnapshotDescription is used at client side to store TTL value. Now, the
default value is `-1` if not specified.
Now, while creating snapshot, `SnapshotDescription` is converted to
`SnapshotProtos.SnapshotDescription` and here is the logic for TTL conversion:
```
if (snapshotDesc.getTtl() != -1L &&
snapshotDesc.getTtl() <
TimeUnit.MILLISECONDS.toSeconds(Long.MAX_VALUE)) {
builder.setTtl(snapshotDesc.getTtl());
}
```
Hence, by default `-1` won't be set in `SnapshotProtos.SnapshotDescription`
and the default value of `SnapshotProtos.SnapshotDescription` will be used:
```
message SnapshotDescription {
required string name = 1;
optional string table = 2; // not needed for delete, but checked for in
taking snapshot
optional int64 creation_time = 3 [default = 0];
enum Type {
DISABLED = 0;
FLUSH = 1;
SKIPFLUSH = 2;
}
optional Type type = 4 [default = FLUSH];
optional int32 version = 5;
optional string owner = 6;
optional UsersAndPermissions users_and_permissions = 7;
optional int64 ttl = 8 [default = 0];
optional int64 max_file_size = 9 [default = 0];
}
```
As per this proto message, default value of TTL for
`SnapshotProtos.SnapshotDescription` would be `0`.
Hence, as per this logic, `0` is `NO_SNAPSHOT_TTL_SPECIFIED` and default TTL
will be used that we set with config `hbase.master.snapshot.ttl`.
```
long ttl = snapshot.getTtl();
// set default ttl(sec) if it is not set already or the value is out of
the range
if (ttl == SnapshotDescriptionUtils.NO_SNAPSHOT_TTL_SPECIFIED ||
ttl > TimeUnit.MILLISECONDS.toSeconds(Long.MAX_VALUE)) {
final long defaultSnapshotTtl =
conf.getLong(HConstants.DEFAULT_SNAPSHOT_TTL_CONFIG_KEY,
HConstants.DEFAULT_SNAPSHOT_TTL);
if (LOG.isDebugEnabled()) {
LOG.debug("Snapshot current TTL value: {} resetting it to default
value: {}", ttl,
defaultSnapshotTtl);
}
ttl = defaultSnapshotTtl;
}
SnapshotDescription.Builder builder = snapshot.toBuilder();
builder.setTtl(ttl);
snapshot = builder.build();
```
@joswiatek No specifying TTL while creating snapshot should result in using
default TTL as specified by `hbase.master.snapshot.ttl`. For the testing
purpose, value of `hbase.master.cleaner.snapshot.interval` can be reduced to
test this scenario so that `SnapshotCleanerChore` would be able to delete
expired snapshots.
Also, here is the SnapshotCleanerChore logic that would delete only
snapshots with value > 0 and value < Long.MAX:
```
long snapshotTtl = snapshotDescription.getTtl();
/*
* Backward compatibility after the patch deployment on HMaster
* Any snapshot with ttl 0 is to be considered as snapshot to keep
FOREVER
* Default ttl value specified by {@HConstants.DEFAULT_SNAPSHOT_TTL}
*/
if (snapshotCreatedTime > 0 && snapshotTtl > 0 &&
snapshotTtl <
TimeUnit.MILLISECONDS.toSeconds(Long.MAX_VALUE)) {
long currentTime = EnvironmentEdgeManager.currentTime();
if ((snapshotCreatedTime + TimeUnit.SECONDS.toMillis(snapshotTtl))
< currentTime) {
LOG.info("Event: {} Name: {}, CreatedTime: {}, TTL: {},
currentTime: {}",
DELETE_SNAPSHOT_EVENT, snapshotDescription.getName(),
snapshotCreatedTime,
snapshotTtl, currentTime);
deleteExpiredSnapshot(snapshotDescription);
}
}
```
I am confident that the current logic should work fine for the case where
client doesn't specify snapshot resulting into using TTL specified by config
`hbase.master.snapshot.ttl`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
