[
https://issues.apache.org/jira/browse/CRUNCH-481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tom White updated CRUNCH-481:
-----------------------------
Attachment: CRUNCH-481.patch
The test hang is happening in Kite (see
https://issues.cloudera.org/browse/CDK-756, you have to apply the patch there
and adjust the Crunch version in the POM).
Here's an updated patch which fixes the problem by setting the job ID. It's
slightly tricky due to Hadoop 1/2 differences. It passes all Crunch tests when
run under both Hadoop 1 and 2, and passes the test in CDK-756.
> Support independent output committers for multiple outputs
> ----------------------------------------------------------
>
> Key: CRUNCH-481
> URL: https://issues.apache.org/jira/browse/CRUNCH-481
> Project: Crunch
> Issue Type: Bug
> Components: Core
> Reporter: Aniket Kulkarni
> Assignee: Josh Wills
> Priority: Minor
> Fix For: 0.12.0
>
> Attachments: CRUNCH-481.patch, CRUNCH-481.patch, CRUNCH-481.patch,
> CRUNCH-481c.patch
>
>
> I faced this issue while trying to write to Kite and HDFS in the same
> pipeline. A similar issue was logged for Kite[1][2].
> I was attempting to write a PCollection to Kite and a different PTable to
> HDFS as a text file. The write to Kite succeeded, however the write to HDFS
> only produced a _SUCCESS file with no text file.
> Commenting out the write to Kite seems to solve the issue and I can see the
> text file being written.
> [1] - https://issues.cloudera.org/browse/CDK-756
> [2] -
> http://mail-archives.apache.org/mod_mbox/crunch-dev/201401.mbox/%3ccaf-wd4qcue0toh3qewpdnnom3u786pvjlgh7t6go_abctpl...@mail.gmail.com%3E
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
