[
https://issues.apache.org/jira/browse/CRUNCH-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846062#comment-13846062
]
Micah Whitacre commented on CRUNCH-311:
---------------------------------------
Looking at the code..
{code}
Path basePath = new Path(getOutputPath(taskAttemptContext),
conf.get("mapreduce.output.basename", "part"));
return new AvroFilePerKeyRecordWriter<T>(basePath,
getUniqueFile(taskAttemptContext, "part", ".avro"), conf);
{code}
I essentially want to override the "part" in the second line with an injectable
value. In normal Hadoop configuration[1] this would be done through conf by
setting the "mapreduce.output.basename" but it looks like we are overriding
that value for the basePath. We could add a new config value for that as a
possibility.
[1] -
https://github.com/apache/hadoop-common/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputFormat.java#L287
> Expose injection of FileNamingScheme into AvroPathPerKeyTarget
> --------------------------------------------------------------
>
> Key: CRUNCH-311
> URL: https://issues.apache.org/jira/browse/CRUNCH-311
> Project: Crunch
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.9.0
> Reporter: Micah Whitacre
> Assignee: Micah Whitacre
> Fix For: 0.9.0
>
> Attachments: CRUNCH-311.patch
>
>
> Similar to AvroFileTarget, injection of custom FileNamingScheme's should be
> supported on the brand new AvroPathPerKeyTarget.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
