[
https://issues.apache.org/jira/browse/BEAM-66?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía resolved BEAM-66.
------------------------------
Resolution: Fixed
Fix Version/s: Not applicable
Resolved via `.watchForNewFiles()`
> Unbounded source for directory/file scanning
> --------------------------------------------
>
> Key: BEAM-66
> URL: https://issues.apache.org/jira/browse/BEAM-66
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
> Reporter: Daniel Halperin
> Priority: Minor
> Fix For: Not applicable
>
>
> (Blocked on SplittableDoFn, https://issues.apache.org/jira/browse/BEAM-65.)
> This is a streaming glob watcher, for instance a directory tailer -- poll a
> directory read the data from new files as they show up. To get this right in
> a streaming pipeline requires careful management of timestamps and watermarks
> -- e.g., a filename or other metadata should reflect a timestamp earlier than
> any records in it, so that we are able to provide accurate lower bounds on
> output timestamp.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
