Re: Can Hadoop process pictures and videos?

[Ted Dunning]

It's also easy to launch processes from Java.  If you have unsplittable
input files, Java can read the entire file and pass it to the child process.


On 4/22/08 11:49 PM, "Owen O'Malley" <[EMAIL PROTECTED]> wrote:

> 
> On Apr 22, 2008, at 11:44 PM, Roland Rabben wrote:
> 
>> Thanks
>> Do you have any pointers to how you would normally approach this? Is
>> Hadoop Streaming the solution I am looking for?
> 
> Probably not, given that Hadoop streaming doesn't work well with
> binary data. It is probably easiest to the use the C++ interface.
> Look at the examples in src/examples/pipes.
> 
> -- Owen

Re: Can Hadoop process pictures and videos?

[Owen O'Malley]

On Apr 22, 2008, at 11:44 PM, Roland Rabben wrote:


Thanks
Do you have any pointers to how you would normally approach this? Is
Hadoop Streaming the solution I am looking for?


Probably not, given that Hadoop streaming doesn't work well with  
binary data. It is probably easiest to the use the C++ interface.  
Look at the examples in src/examples/pipes.


-- Owen

RE: Can Hadoop process pictures and videos?

[Roland Rabben]

Thanks
Do you have any pointers to how you would normally approach this? Is
Hadoop Streaming the solution I am looking for?

Many thanks again.

Roland

> -Original Message-
> From: Ted Dunning [mailto:[EMAIL PROTECTED]
> Sent: 22. april 2008 22:54
> To: core-user@hadoop.apache.org
> Subject: Re: Can Hadoop process pictures and videos?
> 
> 
> Yes you can.
> 
> One issue is typically that linux based video codecs are not as
> numberous as
> windows based codecs so you may be a bit limited as to what kinds of
> video
> you can process.
> 
> Also, most video processing and transcoding is embarrassingly parallel
> at
> the file level with little need for map-reduce.  That may make hadoop
> less
> useful than it might otherwise be.  On the other hand, hadoop does
> expose
> URL's from which you can read file data so you might be just as happy
> using
> that.
> 
> 
> On 4/22/08 1:48 PM, "Roland Rabben" <[EMAIL PROTECTED]>
wrote:
> 
> > Hi
> >
> > Sorry for my ignorance, but I am trying to understand if I can use
> > Hadoop and Map/Reduce to process video files and images. Encoding
and
> > transcoding videos is an example of what I would like to do.
> >
> >
> >
> > Thank you for your patience.
> >
> >
> >
> > Regards
> >
> > Roland
> >

Re: Can Hadoop process pictures and videos?

[Ted Dunning]

Yes you can.

One issue is typically that linux based video codecs are not as numberous as
windows based codecs so you may be a bit limited as to what kinds of video
you can process.

Also, most video processing and transcoding is embarrassingly parallel at
the file level with little need for map-reduce.  That may make hadoop less
useful than it might otherwise be.  On the other hand, hadoop does expose
URL's from which you can read file data so you might be just as happy using
that.


On 4/22/08 1:48 PM, "Roland Rabben" <[EMAIL PROTECTED]> wrote:

> Hi
> 
> Sorry for my ignorance, but I am trying to understand if I can use
> Hadoop and Map/Reduce to process video files and images. Encoding and
> transcoding videos is an example of what I would like to do.
> 
>  
> 
> Thank you for your patience.
> 
>  
> 
> Regards
> 
> Roland 
> 

Can Hadoop process pictures and videos?

[Roland Rabben]

Hi

Sorry for my ignorance, but I am trying to understand if I can use
Hadoop and Map/Reduce to process video files and images. Encoding and
transcoding videos is an example of what I would like to do.

 

Thank you for your patience.

 

Regards

Roland