Hey Mark,
You guys correct me if I am wrong but I believe the putfile create
directories is an option that was put in because of trail and error.
Basically what happened is it use to do it by default (just like putHDFS),
but a user accidentally entered an expression which caused the system to
automatically write single files out to as many subdirectories as the Linux
OS would allow. A few iterations later after removing the option, the
consensus was that it had to be, at the very least, an option for
operations teams because of how flexible it allowed the system to be. Which
is why it is a mandatory option now, so that you are aware of exactly what
the processor is doing.
later!
On Thu, Nov 12, 2015 at 10:13 AM, Joe Witt <[email protected]> wrote:
> Right - PutHDFS definitely creates directories on the fly if necessary
> (based on the path attribute of the flowfile) and if it has
> permissions to do so.
>
> Thanks
> Joe
>
> On Thu, Nov 12, 2015 at 8:34 AM, Bryan Bende <[email protected]> wrote:
> > I think PutHDFS always creates them so it isn't an option through the
> > properties.
> >
> > -Bryan
> >
> > On Thu, Nov 12, 2015 at 8:19 AM, Mark Payne <[email protected]>
> wrote:
> >>
> >> Mark,
> >>
> >> My guess is that it was an oversight. I don't believe it was intentional
> >> to leave it out of PutHDFS.
> >>
> >> Thanks
> >> -Mark
> >>
> >>
> >> > On Nov 12, 2015, at 1:57 AM, Mark Petronic <[email protected]>
> >> > wrote:
> >> >
> >> > Just wondering about the history behind why one has the logic to
> >> > create them but the other does not?
> >>
> >
>
--
Corey Flowers
Vice President, Onyx Point, Inc
(410) 541-6699
[email protected]
-- This account not approved for unencrypted proprietary information --
