[
https://issues.apache.org/jira/browse/APEXMALHAR-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15341888#comment-15341888
]
ASF GitHub Bot commented on APEXMALHAR-2116:
--------------------------------------------
GitHub user yogidevendra opened a pull request:
https://github.com/apache/apex-malhar/pull/326
APEXMALHAR-2116 Added FS record reader operator, module, test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yogidevendra/apex-malhar
APEXMALHAR-2116-record-reader
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/apex-malhar/pull/326.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #326
----
commit aaeb7fe34dc69de71dae120ea01452149389854f
Author: yogidevendra <[email protected]>
Date: 2016-06-20T05:47:08Z
Added FS record reader operator, module, test
----
> File Record reader module
> -------------------------
>
> Key: APEXMALHAR-2116
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2116
> Project: Apache Apex Malhar
> Issue Type: New Feature
> Reporter: Yogi Devendra
> Assignee: Yogi Devendra
>
> This will be useful for the usecases which involves reading from files "line
> by line" in parallel and emit each line as seperate tuple.
> Proposal is to have new Module which would allow users to monitor
> directories, read files and emit data records(tuple). Records are based on
> record separator (e.g. newline) or fixed size (no of bytes).
> Plan is as follows:
> 1. New operator FileRecordReader which will extend BlockReader.
> 2. This operator will have configuration option to select mode for
> FIXED_LENGTH, SEPARATOR_BASED recors.
> 3. Using appropriate ReaderContext based on mode.
> 4. New module FileRecordReaderModule which wraps (FileSplitter (existing) +
> FileRecordReader operator)
> Reason for having different operator than BlockReader is because output port
> signature is different than BlockReader.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
