[
https://issues.apache.org/jira/browse/APEXMALHAR-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311605#comment-15311605
]
ASF GitHub Bot commented on APEXMALHAR-2103:
--------------------------------------------
Github user chaithu14 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/300#discussion_r65475558
--- Diff:
library/src/main/java/com/datatorrent/lib/io/fs/FileSplitterInput.java ---
@@ -375,11 +375,14 @@ public void run()
lastScannedInfo = null;
numDiscoveredPerIteration = 0;
for (String afile : files) {
- String filePath = new File(afile).getAbsolutePath();
+ Path filePath = new Path(afile);
LOG.debug("Scan started for input {}", filePath);
- Map<String, Long> lastModifiedTimesForInputDir;
- lastModifiedTimesForInputDir = referenceTimes.get(filePath);
- scan(new Path(afile), null, lastModifiedTimesForInputDir);
+ Map<String, Long> lastModifiedTimesForInputDir = null;
+ if (fs.exists(filePath)) {
+ FileStatus fileStatus = fs.getFileStatus(filePath);
+ lastModifiedTimesForInputDir =
referenceTimes.get(fileStatus.getPath().toUri().getPath());
--- End diff --
No. In createScannedFileInfo(), the directory/file path specified as
absolute path. But, In case of local file system, files might be relative path.
> scanner issues in FileSplitterInput class
> -----------------------------------------
>
> Key: APEXMALHAR-2103
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2103
> Project: Apache Apex Malhar
> Issue Type: Bug
> Reporter: Chaitanya
> Assignee: Chaitanya
>
> Issue: FileSplitter continuously emitting filemetadata even though there is
> a single file.
> Observation: For the same file, While updating and accessing the
> referenceTimes map in FIleSplitterInput and TimeBasedScanner, the Keys are
> different. Because of this, the oldestTimeModification is always null in
> TimeBasedScanner.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
