[ https://issues.apache.org/jira/browse/APEXMALHAR-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15341888#comment-15341888 ]
ASF GitHub Bot commented on APEXMALHAR-2116: -------------------------------------------- GitHub user yogidevendra opened a pull request: https://github.com/apache/apex-malhar/pull/326 APEXMALHAR-2116 Added FS record reader operator, module, test You can merge this pull request into a Git repository by running: $ git pull https://github.com/yogidevendra/apex-malhar APEXMALHAR-2116-record-reader Alternatively you can review and apply these changes as the patch at: https://github.com/apache/apex-malhar/pull/326.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #326 ---- commit aaeb7fe34dc69de71dae120ea01452149389854f Author: yogidevendra <yogideven...@apache.org> Date: 2016-06-20T05:47:08Z Added FS record reader operator, module, test ---- > File Record reader module > ------------------------- > > Key: APEXMALHAR-2116 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2116 > Project: Apache Apex Malhar > Issue Type: New Feature > Reporter: Yogi Devendra > Assignee: Yogi Devendra > > This will be useful for the usecases which involves reading from files "line > by line" in parallel and emit each line as seperate tuple. > Proposal is to have new Module which would allow users to monitor > directories, read files and emit data records(tuple). Records are based on > record separator (e.g. newline) or fixed size (no of bytes). > Plan is as follows: > 1. New operator FileRecordReader which will extend BlockReader. > 2. This operator will have configuration option to select mode for > FIXED_LENGTH, SEPARATOR_BASED recors. > 3. Using appropriate ReaderContext based on mode. > 4. New module FileRecordReaderModule which wraps (FileSplitter (existing) + > FileRecordReader operator) > Reason for having different operator than BlockReader is because output port > signature is different than BlockReader. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)