Oh... I forgot the Crunch is only an abstract for MapReduce pipeline.
But anyone tried use it with S3 job output ? It's strange, seems the
job froze after write the _SUCESS output to S3. The last log appeared
in my job log file is like below:
2016-09-22 10:05:37,194 INFO
(Thread-5): Job status available at:
2016-09-22 10:12:13,692 INFO
(Thread-5): close closed:false
2016-09-22 1:09 GMT+08:00 Josh Wills <josh.wi...@gmail.com>:
> I don't follow- Hadoop handles compression transparently for most of the
> commonly used input formats and compression schemes; you shouldn't have to
> do anything.
> On Wed, Sep 21, 2016 at 12:53 AM wu lihu <routermanwul...@gmail.com> wrote:
>> Hi Everyone
>> I want to ask one question about process the logs files end with
>> compressed files ? Is there any example for that ?