[
https://issues.apache.org/jira/browse/HDDS-2467?focusedWorklogId=346469&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-346469
]
ASF GitHub Bot logged work on HDDS-2467:
----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Nov/19 05:35
Start Date: 20/Nov/19 05:35
Worklog Time Spent: 10m
Work Description: dineshchitlangia commented on pull request #152:
HDDS-2467. Allow running Freon validators with limited memory
URL: https://github.com/apache/hadoop-ozone/pull/152
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 346469)
Time Spent: 20m (was: 10m)
> Allow running Freon validators with limited memory
> --------------------------------------------------
>
> Key: HDDS-2467
> URL: https://issues.apache.org/jira/browse/HDDS-2467
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: freon
> Reporter: Attila Doroszlai
> Assignee: Attila Doroszlai
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.5.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Freon validators read each item to be validated completely into a {{byte[]}}
> buffer. This allows timing only the read (and buffer allocation), but not
> the subsequent digest calculation. However, it also means that memory
> required for running the validators is proportional to key size.
> I propose to add a command-line flag to allow calculating the digest while
> reading the input stream. This changes timing results a bit, since values
> will include the time required for digest calculation. On the other hand,
> Freon will be able to validate huge keys with limited memory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
