[
https://issues.apache.org/jira/browse/NIFI-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16371267#comment-16371267
]
ASF GitHub Bot commented on NIFI-4892:
--------------------------------------
Github user e-kolpakov commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2481#discussion_r169608651
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestValidateCsv.java
---
@@ -333,4 +333,33 @@ public void testMultipleRuns() {
runner.assertTransferCount(ValidateCsv.REL_VALID, 2);
runner.assertTransferCount(ValidateCsv.REL_INVALID, 0);
}
+
+ @Test
+ public void testEscapingLineByLine() {
+ final TestRunner runner = TestRunners.newTestRunner(new
ValidateCsv());
+ runner.setProperty(ValidateCsv.DELIMITER_CHARACTER, ",");
+ runner.setProperty(ValidateCsv.END_OF_LINE_CHARACTER, "\r\n");
+ runner.setProperty(ValidateCsv.QUOTE_CHARACTER, "\"");
+ runner.setProperty(ValidateCsv.HEADER, "true");
+ runner.setProperty(ValidateCsv.VALIDATION_STRATEGY,
ValidateCsv.VALIDATE_LINES_INDIVIDUALLY);
+
+ final String row =
"Header1,\"Header2,escaped\",Header3\r\nField1,\"Field2,escaped\",Field3";
+ runner.setProperty(ValidateCsv.SCHEMA,
"ParseInt(),ParseInt(),ParseInt()");
+
+ runner.enqueue(row);
+ runner.run(1);
+
+ runner.assertTransferCount(ValidateCsv.REL_VALID, 0);
+ runner.assertTransferCount(ValidateCsv.REL_INVALID, 1);
+
runner.getFlowFilesForRelationship(ValidateCsv.REL_INVALID).get(0).assertContentEquals(row);
+ runner.clearTransferState();
+
+ runner.setProperty(ValidateCsv.SCHEMA, "null,null,null");
--- End diff --
@pvillard31 Suggestion: it might make sense to split this into a dedicated
test, so that there are two independent scenarios:
1. Escaping is preserved in "valid" output
2. Escaping is preserved in "invalid" output.
This test as is covers both, but have a minor drawback that it would stop
at invalid output assertion (line 354) even if both valid and invalid outputs
are affected.
> ValidateCSV Processor - no doublequote escaping in "invalid" output
> -------------------------------------------------------------------
>
> Key: NIFI-4892
> URL: https://issues.apache.org/jira/browse/NIFI-4892
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.4.0, 1.5.0
> Reporter: Eugeny Kolpakov
> Assignee: Pierre Villard
> Priority: Major
> Attachments: ValidateCSV_LineByLine_EscapingBug.xml
>
>
> ValidateCSV processor fails to preserve field escaping for flow files routed
> to "invalid" relation when "Line by Line" validation strategy is used.
>
> Minimal sample flow attached.
>
> Observed in 1.4.0, but 1.5.0 might be affected as well - I'll update this
> ticket after upgrading to 1.5.0 (unless someone could do that earlier)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
