[GitHub] nifi pull request: NIFI-1103: Add support for long polling in GetS...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/122


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1234 Correcting container functionality fo...

[apiri]

GitHub user apiri opened a pull request:

https://github.com/apache/nifi/pull/136

NIFI-1234 Correcting container functionality for single Avro records.

@joemeszaros Had a user open an issue concerning the processor's handling 
of events with only one record.  I agree that the functionality seems a bit odd 
and was wondering if there was a specific case that would be missed should the 
containerOption be the only item to dictate format.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apiri/incubator-nifi NIFI-1234

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/136.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #136


commit 4e07619fd48737b003361070a583330abd2e5d8b
Author: Aldrin Piri 
Date:   2015-12-01T22:38:46Z

NIFI-1234 Correcting container functionality for single Avro records.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: PutSQL : feature enhancement - PutPropertyToSQL

[Bob ZHao]

Hi Mark,

Thanks for the workaround. This ETL trick always works! Only not sure for
SQL injection.

Do you have any idea  about "configure the DBCP connection pool for hive" ?
It's painful and we can't wait until 1.0 release.

Thanks,
Bob


Mark Payne wrote
> You can also potentially use the ReplaceText processor to build a SQL
> statement
> using the Expression Language, something like:
> 
> INSERT INTO MyTable (col1, col2) VALUES ('${attr1}', '${attr2}')
> 
> -Mark
> 
> 
>> On Dec 1, 2015, at 5:10 PM, Bryan Bende 

> bbende@

>  wrote:
>> 
>> Bob,
>> 
>> I think this is slightly different than what you are suggesting, but in
>> the
>> next release there is a new AttributesToJson processor makes it very easy
>> to construct a new json document from values that were extracted using
>> EvaluateJsonPath.
>> 
>> Would this makes things a bit easier for sending json to
>> ConvertJsonToSql?
>> 
>> -Bryan
>> 
>> On Tue, Dec 1, 2015 at 4:14 PM, Bob ZHao 

> bob.zhao.work@

>  wrote:
>> 
>>> Hello,
>>> 
>>> I like the idea to support more on tabular data target. But it might
>>> take a
>>> long way to go.
>>> I have a quick enhancement suggestion.
>>> 
>>> *COnvertJSONToSQL* only support flat JSON structure instead of the
>>> complicated format -- this make sense. But since we can use
>>> *EvaluateJsonPath* to filter our the target fields, could we combine
>>> them
>>> together for a new one *PutPropertyToSQL*? This could be very quick
>>> implementation.
>>> 
>>> NIFI needn't to worry about the complex of Json,  user can use JsonPath
>>> to
>>> chop the fields and fragments for customization.
>>> Then put the result to target table. BTW, please use a trick to save the
>>> raw
>>> Json - by pointing to a column.
>>> 
>>> Thanks,
>>> Bob
>>> 
>>> 
>>> 
>>> 
>>> --
>>> View this message in context:
>>> http://apache-nifi-developer-list.39713.n7.nabble.com/PutSQL-feature-enhancement-PutPropertyToSQL-tp5536.html
>>> Sent from the Apache NiFi Developer List mailing list archive at
>>> Nabble.com.
>>>





--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/PutSQL-feature-enhancement-PutPropertyToSQL-tp5536p5541.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Need access to edit Apache NiFi confluence pages.

[Matthew Clarke]

Devs,

Please grant me access to edit/add entries in the Apache NiFi confluence
Wiki.

Thanks,
Matt

Re: Release wrangling: 1 week until our hopeful 0.4.0 release

[Joe Witt]

Team,

As you can see here [1] we are 'done' as far as 0.4.0 known tickets
go.  I am ready to create the RC and would love to do so.

Before I kick out Apache NiFi 0.4.0 RC1 I propose we spend a day or so
doing the same sort of items we would have and if it is quiet-ish then
we proceed with putting out an RC.  There are a tremendous amount of
tickets/items involved in this release [2] with a few that are rather
substantive changes/features such as [3] for example.

[1] 
https://issues.apache.org/jira/browse/NIFI-1237?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%200.4.0%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC

[2] 
https://issues.apache.org/jira/browse/NIFI-1122?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%200.4.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC

[3] https://issues.apache.org/jira/browse/NIFI-655

Thanks
Joe

On Fri, Nov 20, 2015 at 11:05 AM, Tony Kurc  wrote:
> FWIW I submitted at patch for lineendings that I don't believe will
> conflict with NIFI-655
> On Nov 20, 2015 8:57 AM, "Joe Witt"  wrote:
>
>> Getting closer!
>>
>>
>> https://issues.apache.org/jira/browse/NIFI-1122?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%200.4.0%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>>
>> Key stuff left:
>> - Wrap up NIFI-655 uname/pword stuff
>> - Make the kafka client config a little more flexible
>> - Make the user experience better for processors requiring input
>> - Fix up the tail file stuff which we all now appreciate as being
>> extraordinarily edge case challenged
>>
>> Then
>> - Tony fixes line endings (except for tests which require specific endings)
>> - I will do RM to kick out RC
>> - We all do focused testing
>> - We do the vote
>>
>> We're a little behind the ideal timeline but these are definitely
>> things worth waiting for and we're still finding bugs during testing.
>> Once the popcorn stops popping as much (so to speak) we can zero in on
>> a release.
>>
>> Thanks
>> Joe
>>
>> On Mon, Nov 16, 2015 at 11:47 AM, Joe Percivall
>>  wrote:
>> > Per NiFi-1165: a discussion is occurring on the ticket:
>> https://issues.apache.org/jira/browse/NIFI-1165
>> >
>> >
>> > Overall, most of the issues are identified and pending a fix from Mark,
>> Oleg and I. The issues were encountered on two different windows 8 machines
>> by me and on windows 2012 R2 by Mark. My configuration is maven 3.3.3 and
>> Java 1.8.0_45 (on the machine I have in front of me).
>> >
>> > Should have a patch resolving the issues in the next couple days.
>> >
>> > Joe
>> >
>> > - - - - - - Joseph Percivall
>> > linkedin.com/in/Percivall
>> > e: joeperciv...@yahoo.com
>> >
>> >
>> >
>> >
>> > On Monday, November 16, 2015 10:42 AM, Sean Busbey 
>> wrote:
>> >
>> >
>> >
>> > re: NIFI-1165 I also have a windows 7 laptop I can test on. (though it is
>> > low power)
>> >
>> >
>> > On Sun, Nov 15, 2015 at 10:48 AM, Aldrin Piri 
>> wrote:
>> >
>> >> I have another set of eyes for NIFI-748.  Will do so now.
>> >>
>> >> On Sun, Nov 15, 2015 at 10:39 AM, Tony Kurc  wrote:
>> >>
>> >> > For those not watching commits@nifi
>> >> > I need another set of eyes on the review for NIFI-748
>> >> >
>> >> > On Sun, Nov 15, 2015 at 8:33 AM, Joe Witt  wrote:
>> >> >
>> >> > > NIFI-1082 (this should move to next release unless a resolution is
>> >> > > imminent)
>> >> > >
>> >> > > NIFI-1108 (move to next release)
>> >> > >
>> >> > > NIFI-1139 (recommend moving to 0.5.0)
>> >> > >
>> >> > > NIFI-1164 (this should get fixed now - it makes builds unreliable)
>> >> > >
>> >> > > NIFI-1165 (should tackle now.  have a windows laptop i can build on)
>> >> > >
>> >> > > Thanks for pushing tony.
>> >> > >
>> >> > > On Sun, Nov 15, 2015 at 8:18 AM, Tony Kurc 
>> wrote:
>> >> > > > Update:
>> >> > > >
>> >> > > > Presumably fixed by NIFI-1086 (Joe Percivall). Reviewed, awaiting
>> >> > > revision
>> >> > > > NIFI-61
>> >> > > > NIFI-812
>> >> > > > NIFI-980
>> >> > > > NIFI-1009
>> >> > > > NIFI-1086
>> >> > > > NIFI-1133
>> >> > > >
>> >> > > > Multiple Auths (Matt Gilman) no patch yet, making progress
>> >> > > > NIFI-655
>> >> > > >
>> >> > > > Provenance Search Improvement (Oleg Zhurakousky) PR in, being
>> >> reviewed
>> >> > by
>> >> > > > Tony Kurc
>> >> > > > NIFI-748
>> >> > > >
>> >> > > > Create a Getting Started Guide (Mark Payne) Review complete, being
>> >> > merged
>> >> > > > in by Tony Kurc
>> >> > > > NIFI-973
>> >> > > >
>> >> > > > Line ending fix (Tony Kurc) Finger hovering over "go" button
>> >> > > > NIFI-1054
>> >> > > >
>> >> > > > ExecuteStreamCommand (Joe Percivall). Reviewed, awaiting revision
>> >> > > > NIFI-1081
>> >> > > >
>> >> > > > Provenance repository search (Mark Payne) original patch reverted,
>> >> 

[GitHub] nifi pull request: NIFI-1234 Correcting container functionality fo...

[apiri]

Github user apiri commented on the pull request:

https://github.com/apache/nifi/pull/136#issuecomment-161178953
  
@trkurc good catches on the comments. Overall would you feel like this is a 
bug? That's the impression I get and trying to think if this is adjusted is it 
breaking? I imagine it could be but have a hard time envisioning that behavior 
being desirable. Thoughts?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1234 Correcting container functionality fo...

[trkurc]

Github user trkurc commented on the pull request:

https://github.com/apache/nifi/pull/136#issuecomment-161179877
  
both the code and processor description strongly indicated this was 
intentional.

"If an incoming FlowFile contains a stream of multiple Avro records, the 
resultant FlowFile will contain a JSON Array containing all of the Avro 
records."


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1118 Update SplitText Processor - add supp...

[jskora]

GitHub user jskora opened a pull request:

https://github.com/apache/nifi/pull/135

NIFI-1118 Update SplitText Processor - add support for split size lim…

…its and header line markers.

1) Add "Maximum Fragment Size" property.  A new split file will be created 
if the next line to be added to the current split file exceeds this 
user-defined maximum file size.
2) Add "Header Line Marker Character" property.  Lines that begin with 
these user-defined character(s) will be considered header line(s) rather than a 
predetermined number of lines.  The existing property "Header Line Count" must 
be zero for this new property and behavior to be used.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jskora/nifi NIFI-1118

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/135.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #135


commit ddd394824e3f36429e5b43b755dd8d9df25f05ab
Author: Joe Skora 
Date:   2015-12-01T18:32:36Z

NIFI-1118 Update SplitText Processor - add support for split size limits 
and header line markers.
1) Add "Maximum Fragment Size" property.  A new split file will be created 
if the next line to be added to the current split file exceeds this 
user-defined maximum file size.
2) Add "Header Line Marker Character" property.  Lines that begin with 
these user-defined character(s) will be considered header line(s) rather than a 
predetermined number of lines.  The existing property "Header Line Count" must 
be zero for this new property and behavior to be used.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Need access to edit Apache NiFi confluence pages.

[David Wynne]

Please grant me access to the Confluence Apache-NiFi wiki page.

Re: Need access to edit Apache NiFi confluence pages.

[Bryan Bende]

David,

I just granted you access, let us know if it doesn't work.

Thanks,

Bryan


On Tue, Dec 1, 2015 at 2:30 PM, David Wynne  wrote:

> Please grant me access to the Confluence Apache-NiFi wiki page.
>