[jira] [Commented] (NIFI-4081) GrokReader - add the option to keep raw message in a dedicated field

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158900#comment-16158900
 ] 

ASF GitHub Bot commented on NIFI-4081:
--

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/1921
  
@pvillard31 all looks good from my end. +1 merged to master.


> GrokReader - add the option to keep raw message in a dedicated field
> 
>
> Key: NIFI-4081
> URL: https://issues.apache.org/jira/browse/NIFI-4081
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
> Fix For: 1.4.0
>
>
> In some use cases, it can be useful to keep the raw message in the record. I 
> propose to add a parameter to the GrokReader allowing the user to store in 
> the field {{rawMessage}} the raw content of the record (stack trace included 
> if any).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4081) GrokReader - add the option to keep raw message in a dedicated field

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158899#comment-16158899
 ] 

ASF subversion and git services commented on NIFI-4081:
---

Commit 65596044563bd8217a1fcfcd9bfafa02f2f13160 in nifi's branch 
refs/heads/master from [~pvillard]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=6559604 ]

NIFI-4081 - Added raw message option in GrokReader
This closes #1921.


> GrokReader - add the option to keep raw message in a dedicated field
> 
>
> Key: NIFI-4081
> URL: https://issues.apache.org/jira/browse/NIFI-4081
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> In some use cases, it can be useful to keep the raw message in the record. I 
> propose to add a parameter to the GrokReader allowing the user to store in 
> the field {{rawMessage}} the raw content of the record (stack trace included 
> if any).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4081) GrokReader - add the option to keep raw message in a dedicated field

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130935#comment-16130935
 ] 

ASF GitHub Bot commented on NIFI-4081:
--

Github user pvillard31 commented on the issue:

https://github.com/apache/nifi/pull/1921
  
@markap14 done and updated a bit to match the recent refactoring.


> GrokReader - add the option to keep raw message in a dedicated field
> 
>
> Key: NIFI-4081
> URL: https://issues.apache.org/jira/browse/NIFI-4081
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> In some use cases, it can be useful to keep the raw message in the record. I 
> propose to add a parameter to the GrokReader allowing the user to store in 
> the field {{rawMessage}} the raw content of the record (stack trace included 
> if any).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4081) GrokReader - add the option to keep raw message in a dedicated field

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130830#comment-16130830
 ] 

ASF GitHub Bot commented on NIFI-4081:
--

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/1921
  
@pvillard31 looks like this PR has some conflicts. Do you mind rebasing?


> GrokReader - add the option to keep raw message in a dedicated field
> 
>
> Key: NIFI-4081
> URL: https://issues.apache.org/jira/browse/NIFI-4081
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> In some use cases, it can be useful to keep the raw message in the record. I 
> propose to add a parameter to the GrokReader allowing the user to store in 
> the field {{rawMessage}} the raw content of the record (stack trace included 
> if any).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4081) GrokReader - add the option to keep raw message in a dedicated field

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091365#comment-16091365
 ] 

ASF GitHub Bot commented on NIFI-4081:
--

Github user pvillard31 commented on the issue:

https://github.com/apache/nifi/pull/1921
  
Hey @trixpan, consistency is always a good thing and thanks for spotting 
that. I changed the name to ``_raw``. Let me know if something else is needed.


> GrokReader - add the option to keep raw message in a dedicated field
> 
>
> Key: NIFI-4081
> URL: https://issues.apache.org/jira/browse/NIFI-4081
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> In some use cases, it can be useful to keep the raw message in the record. I 
> propose to add a parameter to the GrokReader allowing the user to store in 
> the field {{rawMessage}} the raw content of the record (stack trace included 
> if any).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4081) GrokReader - add the option to keep raw message in a dedicated field

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088802#comment-16088802
 ] 

ASF GitHub Bot commented on NIFI-4081:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1921
  
@pvillard31 I accept it is a bit of bit picking but is there any chance we 
call it _raw? IIRC this is the name we used on ParseCEF. Naming conventions 
make user experience smoother 


> GrokReader - add the option to keep raw message in a dedicated field
> 
>
> Key: NIFI-4081
> URL: https://issues.apache.org/jira/browse/NIFI-4081
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> In some use cases, it can be useful to keep the raw message in the record. I 
> propose to add a parameter to the GrokReader allowing the user to store in 
> the field {{rawMessage}} the raw content of the record (stack trace included 
> if any).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4081) GrokReader - add the option to keep raw message in a dedicated field

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063317#comment-16063317
 ] 

ASF GitHub Bot commented on NIFI-4081:
--

Github user pvillard31 commented on the issue:

https://github.com/apache/nifi/pull/1921
  
Thanks @bbende, I updated my PR with latest master code.


> GrokReader - add the option to keep raw message in a dedicated field
> 
>
> Key: NIFI-4081
> URL: https://issues.apache.org/jira/browse/NIFI-4081
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> In some use cases, it can be useful to keep the raw message in the record. I 
> propose to add a parameter to the GrokReader allowing the user to store in 
> the field {{rawMessage}} the raw content of the record (stack trace included 
> if any).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4081) GrokReader - add the option to keep raw message in a dedicated field

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061428#comment-16061428
 ] 

ASF GitHub Bot commented on NIFI-4081:
--

Github user bbende commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1921#discussion_r123831362
  
--- Diff: 
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/grok/GrokRecordReader.java
 ---
@@ -89,6 +93,7 @@ public Record nextRecord() throws IOException, 
MalformedRecordException {
 // the next call to nextRecord()) or we will attach it to the 
previously read record.
 String stackTrace = null;
 final StringBuilder toAppend = new StringBuilder();
+final StringBuilder raw = new StringBuilder(line);
--- End diff --

I think this needs to be updated slightly based on some changes in master, 
looks like the beginning of this method changed a little bit and the 'line' 
variable is no longer visible here 


> GrokReader - add the option to keep raw message in a dedicated field
> 
>
> Key: NIFI-4081
> URL: https://issues.apache.org/jira/browse/NIFI-4081
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> In some use cases, it can be useful to keep the raw message in the record. I 
> propose to add a parameter to the GrokReader allowing the user to store in 
> the field {{rawMessage}} the raw content of the record (stack trace included 
> if any).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4081) GrokReader - add the option to keep raw message in a dedicated field

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16051147#comment-16051147
 ] 

ASF GitHub Bot commented on NIFI-4081:
--

GitHub user pvillard31 opened a pull request:

https://github.com/apache/nifi/pull/1921

NIFI-4081 - Added raw message option in GrokReader

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [X] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [X] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [X] Is your initial contribution a single, squashed commit?

### For code changes:
- [X] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [X] Have you written or updated unit tests to verify your changes?
- [X] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pvillard31/nifi NIFI-4081

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1921.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1921


commit ad7e8ee657d30c80ad16de9c3550837d645b574b
Author: Pierre Villard 
Date:   2017-06-15T22:09:45Z

NIFI-4081 - Added raw message option in GrokReader




> GrokReader - add the option to keep raw message in a dedicated field
> 
>
> Key: NIFI-4081
> URL: https://issues.apache.org/jira/browse/NIFI-4081
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> In some use cases, it can be useful to keep the raw message in the record. I 
> propose to add a parameter to the GrokReader allowing the user to store in 
> the field {{rawMessage}} the raw content of the record (stack trace included 
> if any).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)