[
https://issues.apache.org/jira/browse/CRUNCH-678?focusedWorklogId=200971&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200971
]
ASF GitHub Bot logged work on CRUNCH-678:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Feb/19 23:28
Start Date: 19/Feb/19 23:28
Worklog Time Spent: 10m
Work Description: noslowerdna commented on pull request #18: CRUNCH-678:
Avoid unnecessary last modified time retrieval
URL: https://github.com/apache/crunch/pull/18
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 200971)
Time Spent: 10m
Remaining Estimate: 0h
> Avoid unnecessary retrieval of last modified time
> -------------------------------------------------
>
> Key: CRUNCH-678
> URL: https://issues.apache.org/jira/browse/CRUNCH-678
> Project: Crunch
> Issue Type: Improvement
> Components: Core
> Reporter: Andrew Olson
> Assignee: Josh Wills
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There is no assurance that the last modified time can be retrieved
> efficiently for all file systems. In particular, with object stores and large
> data sets it could be very slow. Since this information is actually not
> always needed, we should only retrieve it when necessary (i.e. when the write
> mode is checkpoint) for sources and targets.
> CRUNCH-658 expressed similar concerns for the getSize method. This would be a
> simpler and safer optimization to make.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
