t; we've suggested that a user should read from hdfs themselves (eg.,
> to
> >> > read
> >> > > multiple files together in one partition) -- with*out* reusing the
> code
> >> > in
> >> > > HadoopRDD, though they would lose things like th
> > preferred locations you get from HadoopRDD. Does HadoopRDD need to
>> some
>> > > refactoring to make that easier to do? Or do we just need a good
>> > example?
>> > >
>> > > Imran
>> > >
>> > > (sorry for hijacking y
oopRDD need to
> some
> > > refactoring to make that easier to do? Or do we just need a good
> > example?
> > >
> > > Imran
> > >
> > > (sorry for hijacking your thread, Koert)
> > >
> > >
> > >
> > > On Mon, Mar
> >
> > (sorry for hijacking your thread, Koert)
> >
> >
> >
> > On Mon, Mar 23, 2015 at 3:52 PM, Koert Kuipers
> wrote:
> >
> > > see email below. reynold suggested i send it to dev instead of user
> > >
> > > -- Forwarded message --
>
gt;> >>> >> criteria
> >> >>> >> are:
> >> >>> >> (a) common operations
> >> >>> >> (b) error-prone / difficult to implement
> >> >>> >> (c) non-obvious, but important for p
>>> >> I think this case fits (a) & (c), so I think its still worthwhile.
>> >>> >> But its
>> >>> >> also worth asking whether or not its too difficult for a user to
>> >>> >> extend
>> >>> >&g
read from hdfs themselves (eg.,
> to
> >>> >> read
> >>> >> multiple files together in one partition) -- with*out* reusing the
> >>> >> code in
> >>> >> HadoopRDD, though they would lose things like the metric tracking &
&
gt;>> >> where
>>> >> we've suggested that a user should read from hdfs themselves (eg., to
>>> >> read
>>> >> multiple files together in one partition) -- with*out* reusing the
>>> >> code in
>>> >> HadoopRDD, th
adoopRDD need to
>> some
>> >> refactoring to make that easier to do? Or do we just need a good
>> example?
>> >>
>> >> Imran
>> >>
>> >> (sorry for hijacking your thread, Koert)
>> >>
>> >>
>> >&
gt;
> >> Imran
> >>
> >> (sorry for hijacking your thread, Koert)
> >>
> >>
> >>
> >> On Mon, Mar 23, 2015 at 3:52 PM, Koert Kuipers
> wrote:
> >>
> >> > see email below. reynold suggested i send it to dev instead of use
gt;> refactoring to make that easier to do? Or do we just need a good example?
>>
>> Imran
>>
>> (sorry for hijacking your thread, Koert)
>>
>>
>>
>> On Mon, Mar 23, 2015 at 3:52 PM, Koert Kuipers wrote:
>>
>> > see email below. r
t;
> Imran
>
> (sorry for hijacking your thread, Koert)
>
>
>
> On Mon, Mar 23, 2015 at 3:52 PM, Koert Kuipers wrote:
>
> > see email below. reynold suggested i send it to dev instead of user
> >
> > ------ Forwarded message ------
> > F
> see email below. reynold suggested i send it to dev instead of user
>
> -- Forwarded message --
> From: Koert Kuipers
> Date: Mon, Mar 23, 2015 at 4:36 PM
> Subject: hadoop input/output format advanced control
> To: "u...@spark.apache.org"
>
>
see email below. reynold suggested i send it to dev instead of user
-- Forwarded message --
From: Koert Kuipers
Date: Mon, Mar 23, 2015 at 4:36 PM
Subject: hadoop input/output format advanced control
To: "u...@spark.apache.org"
currently its pretty hard to control
14 matches
Mail list logo
