Re: Processor that Decompresses Files?

[Aldrin Piri]

Hi Keren,

NiFi comes with an UnpackContent [1] processor which should take care of
this for you.  Let us know if it comes up short in terms of your needs.

--aldrin

[1]
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.UnpackContent/

On Wed, Oct 5, 2016 at 9:44 AM, Tseytlin, Keren <
keren.tseyt...@capitalone.com> wrote:

> Hi all,
>
>
>
> I am wondering if it would be useful to create a processor whose purpose
> is to unzip large files. I currently have a lot of zip log files coming in,
> and we need to unpackage them in Nifi. I know there is an
> ExecuteStreamCommand processor where I can put in a command to do the
> unzipping, which effectively does the same thing. Is it best practice for
> me to use the ExecuteStreamCommand processor, or perhaps make something
> like a DecompressFile processor?
>
>
>
> Best,
>
> Keren
>
> --
>
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>

Processor that Decompresses Files?

[Tseytlin, Keren]

Hi all,

I am wondering if it would be useful to create a processor whose purpose is to 
unzip large files. I currently have a lot of zip log files coming in, and we 
need to unpackage them in Nifi. I know there is an ExecuteStreamCommand 
processor where I can put in a command to do the unzipping, which effectively 
does the same thing. Is it best practice for me to use the ExecuteStreamCommand 
processor, or perhaps make something like a DecompressFile processor?

Best,
Keren


The information contained in this e-mail is confidential and/or proprietary to 
Capital One and/or its affiliates and may only be used solely in performance of 
work or services for Capital One. The information transmitted herewith is 
intended only for use by the individual or entity to which it is addressed. If 
the reader of this message is not the intended recipient, you are hereby 
notified that any review, retransmission, dissemination, distribution, copying 
or other use of, or taking of any action in reliance upon this information is 
strictly prohibited. If you have received this communication in error, please 
contact the sender and delete the material from your computer.

Re: Processor that Decompresses Files?

[Mark Payne]

Hi Keren,

Just to clarify - what Joe mentioned here is the recommended approach of your 
data is GZIP'd. If you have data
that is actually ZIP'd instead, you can use UnpackContent.

Thanks
-Mark

> On Oct 5, 2016, at 9:47 AM, Joe Percivall  wrote:
> 
> Hello Keren,
> 
> The "decompress" mode is an option of the CompressContent[1] processor and 
> should solve your use-case.
> 
> [1] 
> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.CompressContent/index.html
> 
> Joe
> 
> - - - - - - 
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joeperciv...@yahoo.com
> 
> 
> 
> On Wednesday, October 5, 2016 9:45 AM, "Tseytlin, Keren" 
>  wrote:
> 
> 
> 
> 
> Hi all,
> 
> I am wondering if it would be useful to create a processor whose purpose is 
> to unzip large files. I currently have a lot of zip log files coming in, and 
> we need to unpackage them in Nifi. I know there is an ExecuteStreamCommand 
> processor where I can put in a command to do the unzipping, which effectively 
> does the same thing. Is it best practice for me to use the 
> ExecuteStreamCommand processor, or perhaps make something like a 
> DecompressFile processor?
> 
> Best,
> Keren
> 
> 
> The information contained in this e-mail is confidential and/or proprietary 
> to Capital One and/or its affiliates and may only be used solely in 
> performance of work or services for Capital One. The information transmitted 
> herewith is intended only for use by the individual or entity to which it is 
> addressed. If the reader of this message is not the intended recipient, you 
> are hereby notified that any review, retransmission, dissemination, 
> distribution, copying or other use of, or taking of any action in reliance 
> upon this information is strictly prohibited. If you have received this 
> communication in error, please contact the sender and delete the material 
> from your computer.

Re: Processor that Decompresses Files?

[Aldrin Piri]

Indeed, CompressContent would be the right one.  UnpackContent is tailored
toward managing FlowFiles.

On Wed, Oct 5, 2016 at 9:47 AM, Joe Percivall 
wrote:

> Hello Keren,
>
> The "decompress" mode is an option of the CompressContent[1] processor and
> should solve your use-case.
>
> [1] https://nifi.apache.org/docs/nifi-docs/components/org.
> apache.nifi.processors.standard.CompressContent/index.html
>
> Joe
>
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joeperciv...@yahoo.com
>
>
>
> On Wednesday, October 5, 2016 9:45 AM, "Tseytlin, Keren" <
> keren.tseyt...@capitalone.com> wrote:
>
>
>
>
> Hi all,
>
> I am wondering if it would be useful to create a processor whose purpose
> is to unzip large files. I currently have a lot of zip log files coming in,
> and we need to unpackage them in Nifi. I know there is an
> ExecuteStreamCommand processor where I can put in a command to do the
> unzipping, which effectively does the same thing. Is it best practice for
> me to use the ExecuteStreamCommand processor, or perhaps make something
> like a DecompressFile processor?
>
> Best,
> Keren
> 
>
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>

Re: Processor that Decompresses Files?

[Joe Percivall]

Hello Keren,

The "decompress" mode is an option of the CompressContent[1] processor and 
should solve your use-case.

[1] 
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.CompressContent/index.html
 
Joe

- - - - - - 
Joseph Percivall
linkedin.com/in/Percivall
e: joeperciv...@yahoo.com



On Wednesday, October 5, 2016 9:45 AM, "Tseytlin, Keren" 
 wrote:




Hi all,
 
I am wondering if it would be useful to create a processor whose purpose is to 
unzip large files. I currently have a lot of zip log files coming in, and we 
need to unpackage them in Nifi. I know there is an ExecuteStreamCommand 
processor where I can put in a command to do the unzipping, which effectively 
does the same thing. Is it best practice for me to use the ExecuteStreamCommand 
processor, or perhaps make something like a DecompressFile processor?
 
Best,
Keren


The information contained in this e-mail is confidential and/or proprietary to 
Capital One and/or its affiliates and may only be used solely in performance of 
work or services for Capital One. The information transmitted herewith is 
intended only for use by the individual or entity to which it is addressed. If 
the reader of this message is not the intended recipient, you are hereby 
notified that any review, retransmission, dissemination, distribution, copying 
or other use of, or taking of any action in reliance upon this information is 
strictly prohibited. If you have received this communication in error, please 
contact the sender and delete the material from your computer.