[
https://issues.apache.org/jira/browse/HDDS-403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607787#comment-16607787
]
Xiaoyu Yao edited comment on HDDS-403 at 9/7/18 11:46 PM:
----------------------------------------------------------
{quote}The issue is that while creating the entities we use UTC but when
retrieving entities we try to parse it into GMT and hence the garbage values.
{quote}
We persisted the timestamp in the format of milliseconds since
1970-01-01T00:00:00Z using Time.now(). This will be the same no matter UTC or
GMT.
When interpret the persisted point in time values in human readable format,
there will be subtle difference: At second level, they are the same. UTC is
more precise at sub-second level where it uses a dedicated integer to saves the
nano parts along with a long for the second part. (More details can be found in
JLS or documents of java.time.Instant class from JDK)
The issue is caused by the bug you fixed in HddsClientUtils where we interpret
the milliseconds as second when formatting.
was (Author: xyao):
{quote}The issue is that while creating the entities we use UTC but when
retrieving entities we try to parse it into GMT and hence the garbage values.
{quote}
We persisted the timestamp in the format of milliseconds since
1970-01-01T00:00:00Z using Time.now(). This will be the same no matter UTC or
GMT.
When interpret the persisted point in time values in human readable format,
there will be subtle difference: At second level, they are the same. UTC is
more precise at sub-second level where it uses a dedicated integer to saves the
nano parts along with a long for the second part. (More details can be found in
JLS or documents of java.time.Instant class from JDK)
> infoVolume, infoBucket, infoKey shows wrong "createdOn" / "modifiedOn"
> metadata for volume, bucket, key
> -------------------------------------------------------------------------------------------------------
>
> Key: HDDS-403
> URL: https://issues.apache.org/jira/browse/HDDS-403
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Manager
> Affects Versions: 0.2.1
> Reporter: Nilotpal Nandi
> Assignee: Dinesh Chitlangia
> Priority: Blocker
> Labels: newbie
> Fix For: 0.2.1
>
> Attachments: HDDS-403.001.patch, HDDS-403.002.patch,
> HDDS-403.003.patch
>
>
> Example for Key shown below. Similar observations can be made for Bucket and
> Volume.
> # ran putKey command for a file
> {noformat}
> [root@ctr-e138-1518143905142-459606-01-000003 bin]# ./ozone oz -putKey
> /test-vol1/test-bucket1/file1 -file /etc/passwd -v
> 2018-09-05 10:25:11,498 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> Volume Name : test-vol1
> Bucket Name : test-bucket1
> Key Name : file1
> File Hash : 8164cc3d5b05c44b73a6277661aa4645
> 2018-09-05 10:25:12,377 INFO conf.ConfUtils: raft.rpc.type = GRPC (default)
> 2018-09-05 10:25:12,390 INFO conf.ConfUtils: raft.grpc.message.size.max =
> 33554432 (custom)
> 2018-09-05 10:25:12,402 INFO conf.ConfUtils: raft.client.rpc.retryInterval =
> 300 ms (default)
> 2018-09-05 10:25:12,407 INFO conf.ConfUtils:
> raft.client.async.outstanding-requests.max = 100 (default)
> 2018-09-05 10:25:12,407 INFO conf.ConfUtils:
> raft.client.async.scheduler-threads = 3 (default)
> 2018-09-05 10:25:12,518 INFO conf.ConfUtils: raft.grpc.flow.control.window =
> 1MB (=1048576) (default)
> 2018-09-05 10:25:12,518 INFO conf.ConfUtils: raft.grpc.message.size.max =
> 33554432 (custom)
> 2018-09-05 10:25:12,866 INFO conf.ConfUtils: raft.client.rpc.request.timeout
> = 3000 ms (default)
> 2018-09-05 10:25:13,644 INFO conf.ConfUtils: raft.grpc.flow.control.window =
> 1MB (=1048576) (default)
> 2018-09-05 10:25:13,644 INFO conf.ConfUtils: raft.grpc.message.size.max =
> 33554432 (custom)
> 2018-09-05 10:25:13,645 INFO conf.ConfUtils: raft.client.rpc.request.timeout
> = 3000 ms (default)
> [root@ctr-e138-1518143905142-459606-01-000003 bin]# ./ozone oz -getKey
> /test-vol1/test-bucket1/file1 -file getkey3
> 2018-09-05 10:25:22,020 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> 2018-09-05 10:25:22,778 INFO conf.ConfUtils: raft.rpc.type = GRPC (default)
> 2018-09-05 10:25:22,790 INFO conf.ConfUtils: raft.grpc.message.size.max =
> 33554432 (custom)
> 2018-09-05 10:25:22,800 INFO conf.ConfUtils: raft.client.rpc.retryInterval =
> 300 ms (default)
> 2018-09-05 10:25:22,804 INFO conf.ConfUtils:
> raft.client.async.outstanding-requests.max = 100 (default)
> 2018-09-05 10:25:22,805 INFO conf.ConfUtils:
> raft.client.async.scheduler-threads = 3 (default)
> 2018-09-05 10:25:22,890 INFO conf.ConfUtils: raft.grpc.flow.control.window =
> 1MB (=1048576) (default)
> 2018-09-05 10:25:22,890 INFO conf.ConfUtils: raft.grpc.message.size.max =
> 33554432 (custom)
> 2018-09-05 10:25:23,250 INFO conf.ConfUtils: raft.client.rpc.request.timeout
> = 3000 ms (default)
> 2018-09-05 10:25:24,066 INFO conf.ConfUtils: raft.grpc.flow.control.window =
> 1MB (=1048576) (default)
> 2018-09-05 10:25:24,067 INFO conf.ConfUtils: raft.grpc.message.size.max =
> 33554432 (custom)
> 2018-09-05 10:25:24,067 INFO conf.ConfUtils: raft.client.rpc.request.timeout
> = 3000 ms (default){noformat}
> 2. Ran infoKey on that key
> {noformat}
> [root@ctr-e138-1518143905142-459606-01-000003 bin]# ./ozone oz -infoKey
> /test-vol1/test-bucket1/file1 -v
> 2018-09-05 10:54:42,053 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> Volume Name : test-vol1
> Bucket Name : test-bucket1
> Key Name : file1
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Sat, 14 Dec +114522267 00:51:17 GMT",
> "modifiedOn" : "Fri, 09 Jun +50648 04:30:12 GMT",
> "size" : 4659,
> "keyName" : "file1",
> "keyLocations" : [ {
> "containerID" : 16,
> "localID" : 1536143112267,
> "length" : 4659,
> "offset" : 0
> } ]
> }{noformat}
> "createdOn" and "modifiedOn" metadata are incorrect.
> Here is the current date:
> {noformat}
> [root@ctr-e138-1518143905142-459606-01-000003 bin]# date
> Wed Sep 5 10:54:52 UTC 2018{noformat}
> Also , the "md5hash" for the key is showing as null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
