[
https://issues.apache.org/jira/browse/CRUNCH-199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13644246#comment-13644246
]
Josh Wills commented on CRUNCH-199:
-----------------------------------
Hey Micah-- the thing that moves the data from the temporary location that is
created during the crunch job run to the final location is the
MSCROutputHandler in o.a.c.impl.mr.plan. I think the source of the issue is at
the end of the configureMapReduce function in the TrevniKeyTarget, where you
reset the outputPath. I think that this line should go.
Also, I agree with your comment: the TrevniKeyOutputFormat isn't aware of
multiple schema outputs within the context of a single job, so you'll need to
use a FormatBundle directly so that different config schemas don't conflict
with each other. I'll post a patch to demonstrate.
> Add Source and Target Support for AvroTrevniKey files
> -----------------------------------------------------
>
> Key: CRUNCH-199
> URL: https://issues.apache.org/jira/browse/CRUNCH-199
> Project: Crunch
> Issue Type: Wish
> Components: Core, IO
> Reporter: Micah Whitacre
> Assignee: Josh Wills
> Priority: Minor
> Attachments: CRUNCH-199.patch
>
>
> Add Source, Target, and SourceTarget, support for AvroTrevniKeyInputFormat[1]
> and AvroTrevniKeyOutputFormat[2].
> [1] -
> http://avro.apache.org/docs/1.7.4/api/java/org/apache/trevni/avro/mapreduce/AvroTrevniKeyInputFormat.html
> [2] -
> http://avro.apache.org/docs/1.7.4/api/java/org/apache/trevni/avro/mapreduce/AvroTrevniKeyOutputFormat.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
