Lehel44 commented on a change in pull request #5437:
URL: https://github.com/apache/nifi/pull/5437#discussion_r724562953
##########
File path:
nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractHadoopProcessor.java
##########
@@ -674,39 +681,33 @@ protected Path getNormalizedPath(ProcessContext context,
PropertyDescriptor prop
}
protected Path getNormalizedPath(final String rawPath) {
- final Path path = new Path(rawPath);
- final URI uri = path.toUri();
-
- final URI fileSystemUri = getFileSystem().getUri();
-
- if (uri.getScheme() != null) {
- if (!uri.getScheme().equals(fileSystemUri.getScheme()) ||
!uri.getAuthority().equals(fileSystemUri.getAuthority())) {
- getLogger().warn("The filesystem component of the URI
configured ({}) does not match the filesystem URI from the Hadoop configuration
file ({}) " +
- "and will be ignored.", uri, fileSystemUri);
- }
-
- return new Path(uri.getPath());
- } else {
- return path;
- }
+ return getNormalizedPath(rawPath, Optional.empty());
}
protected Path getNormalizedPath(final ProcessContext context, final
PropertyDescriptor property, final FlowFile flowFile) {
final String propertyValue =
context.getProperty(property).evaluateAttributeExpressions(flowFile).getValue();
- final Path path = new Path(propertyValue);
- final URI uri = path.toUri();
+ return getNormalizedPath(propertyValue,
Optional.of(property.getDisplayName()));
+ }
+ private Path getNormalizedPath(final String rawPath, final
Optional<String> propertyName) {
+ final URI uri = new Path(rawPath).toUri();
final URI fileSystemUri = getFileSystem().getUri();
+ final String path;
if (uri.getScheme() != null) {
if (!uri.getScheme().equals(fileSystemUri.getScheme()) ||
!uri.getAuthority().equals(fileSystemUri.getAuthority())) {
- getLogger().warn("The filesystem component of the URI
configured in the '{}' property ({}) does not match the filesystem URI from the
Hadoop configuration file ({}) " +
- "and will be ignored.", property.getDisplayName(),
uri, fileSystemUri);
+ if (propertyName.isPresent()) {
Review comment:
Could you please add a test case which covers this part? It seems like
Path normalizes the paths automatically and the difference can't be seen
between the if and else branches related to that.
##########
File path:
nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractHadoopProcessor.java
##########
@@ -674,39 +681,33 @@ protected Path getNormalizedPath(ProcessContext context,
PropertyDescriptor prop
}
protected Path getNormalizedPath(final String rawPath) {
Review comment:
This is only called from FetchHDFS line 128 where the getPath method
uses a _FILENAME_ property:
`protected String getPath(final ProcessContext context, final FlowFile
flowFile) {
return
context.getProperty(FILENAME).evaluateAttributeExpressions(flowFile).getValue();
}`
I think if we call
`path = getNormalizedPath(context, FILENAME, flowFile);`
there, we can eliminate this method and also the optional propertyName
attribute from AbtractHadoopProcessor 692.
##########
File path:
nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractHadoopProcessor.java
##########
@@ -674,39 +681,33 @@ protected Path getNormalizedPath(ProcessContext context,
PropertyDescriptor prop
}
protected Path getNormalizedPath(final String rawPath) {
- final Path path = new Path(rawPath);
- final URI uri = path.toUri();
-
- final URI fileSystemUri = getFileSystem().getUri();
-
- if (uri.getScheme() != null) {
- if (!uri.getScheme().equals(fileSystemUri.getScheme()) ||
!uri.getAuthority().equals(fileSystemUri.getAuthority())) {
- getLogger().warn("The filesystem component of the URI
configured ({}) does not match the filesystem URI from the Hadoop configuration
file ({}) " +
- "and will be ignored.", uri, fileSystemUri);
- }
-
- return new Path(uri.getPath());
- } else {
- return path;
- }
+ return getNormalizedPath(rawPath, Optional.empty());
}
protected Path getNormalizedPath(final ProcessContext context, final
PropertyDescriptor property, final FlowFile flowFile) {
final String propertyValue =
context.getProperty(property).evaluateAttributeExpressions(flowFile).getValue();
- final Path path = new Path(propertyValue);
- final URI uri = path.toUri();
+ return getNormalizedPath(propertyValue,
Optional.of(property.getDisplayName()));
+ }
+ private Path getNormalizedPath(final String rawPath, final
Optional<String> propertyName) {
Review comment:
If you can successfully eliminate the _getNormalizedPath_ method on 683
(see my previous comment), I think this can be merged with the method right
above. This would reduce the number of overloaded methods from 4 to 2.
##########
File path:
nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractHadoopProcessor.java
##########
@@ -674,39 +681,33 @@ protected Path getNormalizedPath(ProcessContext context,
PropertyDescriptor prop
}
protected Path getNormalizedPath(final String rawPath) {
Review comment:
And there would be only one log message type with property.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
