[
https://issues.apache.org/jira/browse/CASSANDRA-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14230101#comment-14230101
]
Michael Shuler commented on CASSANDRA-8367:
-------------------------------------------
[~zvo] do you have a patch that implements the changeover to a different config
key? Someone could look at that :)
> Clash between Cassandra and Crunch mapreduce config
> ---------------------------------------------------
>
> Key: CASSANDRA-8367
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8367
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Radovan Zvoncek
> Priority: Minor
>
> We would like to use Cassandra's (Cql)BulkOutputFormats to implement Resource
> IOs for Crunch. We want to do this to allow Crunch users write results of
> their jobs directly to Cassandra (thus skipping writing them to file system).
> In the process of doing this, we found out there is a clash in the mapreduce
> job config. The affected config key is 'mapreduce.output.basename'. Cassandra
> is using it [1] for something different than Crunch [2]. This is resulting in
> some obscure behavior I personally don't understand, but it causes the jobs
> to fail.
> We went ahead and re-implemented the output format classes to use different
> config key, but we'd very much like to stop using them.
> [1]
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hadoop/ConfigHelper.java#L54
> [2]
> https://github.com/apache/crunch/blob/3f13ee65c9debcf6bd7366607f58beae6c73ffe2/crunch-core/src/main/java/org/apache/crunch/io/CrunchOutputs.java#L99
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
