[DISCUSS} Closing in on a NiFi 1.4.0 release?

[Jeff]

All,

On the dev and users lists there have been some questions about when the
next release of NiFi would be.  I would like perform RM duties, and get
things started on identifying what the community would like to include in
the release of NiFi 1.4.0 that has not already been reviewed and committed
to master.

There are 11 unresolved JIRAs tagged with a fix version of 1.4.0.

https://issues.apache.org/jira/issues/?jql=project%20%3D%20NIFI%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%201.4.0%20ORDER%20BY%20priority%20DESC%2C%20key%20DESC

Are there any reasons to hold off on a 1.4.0 release?  Are there particular
JIRAs that the community considers necessary to have as part of the
release? Let's discuss!

Thanks,
Jeff

Re: Duplicated processors when using nifi processors dependency

[Christopher Currie]

Sorry if this duplicates, I sent from the wrong address the first time.

I’ve opened a PR on GitHub for this issue: https://github.com/apache/
nifi/pull/2140

Let me know if there’s any other follow up I need to do; I’d love to see
this get in for 1.4.0.

Christopher


On 2017-05-22 06:16, Bryan Bende  wrote:

Thanks James.>

For the abstract AWS processors, I would go with one of the following...>

1) Another module under nifi-aws-bundle like:>

nifi-aws-bunde>
   nifi-aws-service-api>
   nifi-aws-service-api-nar>
   nifi-aws-abstract-processors (or some similar name)>
   nifi-aws-processors>
   nifi-aws-nar>

2) A new module under nifi-nar-bundles/nifi-extension-utils like
nifi-aws-utils>

Either way, the desired result is to have a JAR that someone can>
depend on that only brings in the abstract processors.>

We have been putting generic abstract processors into>
nifi-nar-bundles/nifi-extension-utils /nifi-processor-utils, but since>
this case depends on the AWS SDK, then we would be introducing that>
dependency to all the NARs that depend on nifi-processor-utils, which>
is why I was suggesting a new module.>

Let me know if this doesn't make sense.>


On Mon, May 22, 2017 at 1:24 AM, James Wing  wrote:>

I created a JIRA for the nifi-aws-bundle structure,>
https://issues.apache.org/jira/browse/NIFI-3950.>


Is there a best practice for the abstract classes like Robert describes?>
Are they a better fit for the service API bundle or the processors bundle?>
Or does that become irrelevant once the NAR dependencies are aligned and>
they can be references as Java libraries?>


Thanks,>


James>


On Sun, May 21, 2017 at 10:28 AM, Bryan Bende  wrote:>


Currently the AWS NAR contains the processors, controller service API, and>
controller service implementation, all in one NAR.>


Typically the controller service API should be in its own NAR, so there>
should be something like:>


nifi-aws-bunde>
   nifi-aws-service-api>
   nifi-aws-service-api-nar>
   nifi-aws-processors>
   nifi-aws-nar>


The nifi-aws-service-api module would have the>
AWSCredentialsProviderService interface. The nifi-aws-nar would have a NAR>
dependency on nifi-aws-service-api-nar.>


Setting it up this way, you nifi-aws-utils project could have a provided>
dependency on the nifi-aws-service-api, and then your custom NAR could
have>
a NAR dependency on nifi-aws-service-api-nar.>


There is some more information here about setting up controller services:>


https://cwiki.apache.org/confluence/display/NIFI/Maven+>
Projects+for+Extensions#MavenProjectsforExtensions->
LinkingProcessorsandControllerServices 
confluence/display/NIFI/Maven+Projects+for+Extensions#>
MavenProjectsforExtensions-LinkingProcessorsandControllerServices>>


https://github.com/bbende/nifi-dependency-example <>
https://github.com/bbende/nifi-dependency-example>>



On May 20, 2017, at 10:06 AM, Robert  wrote:>


I tired to do move abstract aws processors from>
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors to newly created>
nifi-aws-utils.>


Problem that I face now is with AWSCredentialsProviderControllerService>

that>

imo should stay under nifi-aws-bundle, but it is required by>
AbstractAWSCredentialsProviderProcessor that should be moved to>
nifi-aws-utils as it is used by abstract aws processors.>


What is the preferred way to solve this?>
Should I created new 'nifi-aws-service-bundle' under nifi-nar-bundles and>
both nifi-aws-processors and nifi-aws-utils can depend on it?>






-->
View this message in context: http://apache-nifi-developer->

list.39713.n7.nabble.com/Duplicated-processors-when-using-nifi-processors->
dependency-tp15902p15937.html>

Sent from the Apache NiFi Developer List mailing list archive at>

Nabble.com.>

Re: Url encoding with S3 processors

[Joe Gresock]

Thanks James, I wasn't aware of the urlDecode() EL expression.

On Sun, Sep 10, 2017 at 5:31 PM, James Wing  wrote:

> Joe,
>
> The good news is that you can use expression language directly in
> FetchS3Object's Object Key property to decode your keys.  For example,
>
> ${filename:urlDecode()}
>
> However, I'm not sure that could be done without you specifying that the
> key should be decoded. As I understand it, the following two keys:
>
> file with space.txt
> file+with+space.txt
>
> are both valid, but distinctly separate keys. If we decoded by default, we
> could create confusion with unexpected key changes.
>
> Thanks,
>
> James
>
> On Thu, Sep 7, 2017 at 6:59 AM, Joe Gresock  wrote:
>
> > Has anyone run into any URL encoding issues with the AWS nifi processors
> in
> > version 1.3.0?  My flow has GetSQS and then FetchS3Object, along with the
> > AWS feature of automated SQS events upon S3 file upload.  Some of my S3
> > objects apparently have characters in their object keys that need to be
> URL
> > encoded (e.g., %28 for open paren and + for space), and they come across
> > encoded in the SQS message.  However, FetchS3Object apparently doesn't
> like
> > this encoded URL, and it complains that the key does not exist.
> >
> > I suppose I could add an UpdateAttributes processor to translate certain
> > urls in the keys, but this seems like a workaround.
> >
> >
> > --
> > I know what it is to be in need, and I know what it is to have plenty.  I
> > have learned the secret of being content in any and every situation,
> > whether well fed or hungry, whether living in plenty or in want.  I can
> do
> > all this through him who gives me strength.*-Philippians 4:12-13*
> >
>



-- 
I know what it is to be in need, and I know what it is to have plenty.  I
have learned the secret of being content in any and every situation,
whether well fed or hungry, whether living in plenty or in want.  I can do
all this through him who gives me strength.*-Philippians 4:12-13*

Re: Url encoding with S3 processors

[James Wing]

Joe,

The good news is that you can use expression language directly in
FetchS3Object's Object Key property to decode your keys.  For example,

${filename:urlDecode()}

However, I'm not sure that could be done without you specifying that the
key should be decoded. As I understand it, the following two keys:

file with space.txt
file+with+space.txt

are both valid, but distinctly separate keys. If we decoded by default, we
could create confusion with unexpected key changes.

Thanks,

James

On Thu, Sep 7, 2017 at 6:59 AM, Joe Gresock  wrote:

> Has anyone run into any URL encoding issues with the AWS nifi processors in
> version 1.3.0?  My flow has GetSQS and then FetchS3Object, along with the
> AWS feature of automated SQS events upon S3 file upload.  Some of my S3
> objects apparently have characters in their object keys that need to be URL
> encoded (e.g., %28 for open paren and + for space), and they come across
> encoded in the SQS message.  However, FetchS3Object apparently doesn't like
> this encoded URL, and it complains that the key does not exist.
>
> I suppose I could add an UpdateAttributes processor to translate certain
> urls in the keys, but this seems like a workaround.
>
>
> --
> I know what it is to be in need, and I know what it is to have plenty.  I
> have learned the secret of being content in any and every situation,
> whether well fed or hungry, whether living in plenty or in want.  I can do
> all this through him who gives me strength.*-Philippians 4:12-13*
>