turcsanyip commented on code in PR #5990:
URL: https://github.com/apache/nifi/pull/5990#discussion_r871098130
##########
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/AbstractAzureBlobProcessor_v12.java:
##########
@@ -83,45 +87,38 @@ public abstract class AbstractAzureBlobProcessor_v12
extends AbstractProcessor {
.description("Unsuccessful operations will be transferred to the
failure relationship.")
.build();
+ protected static final List<PropertyDescriptor> PROPERTIES =
Collections.unmodifiableList(Arrays.asList(
+ STORAGE_CREDENTIALS_SERVICE,
+ AzureStorageUtils.PROXY_CONFIGURATION_SERVICE
+ ));
Review Comment:
The abstract processor did not contain common properties intentionally. I
strongly recommend to use simple property lists in processor classes instead.
This way each processor can define its properties in the right order and
without cumbersome code (adding/deleting properties).
##########
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/AbstractAzureBlobProcessor_v12.java:
##########
@@ -83,45 +87,38 @@ public abstract class AbstractAzureBlobProcessor_v12
extends AbstractProcessor {
.description("Unsuccessful operations will be transferred to the
failure relationship.")
.build();
+ protected static final List<PropertyDescriptor> PROPERTIES =
Collections.unmodifiableList(Arrays.asList(
+ STORAGE_CREDENTIALS_SERVICE,
+ AzureStorageUtils.PROXY_CONFIGURATION_SERVICE
+ ));
+
private static final Set<Relationship> RELATIONSHIPS =
Collections.unmodifiableSet(new HashSet<>(Arrays.asList(
REL_SUCCESS,
REL_FAILURE
)));
private BlobServiceClient storageClient;
- @Override
- public Set<Relationship> getRelationships() {
- return RELATIONSHIPS;
- }
-
- @OnScheduled
- public void onScheduled(ProcessContext context) {
- storageClient = createStorageClient(context);
- }
-
- @OnStopped
- public void onStopped() {
- storageClient = null;
- }
-
- protected BlobServiceClient getStorageClient() {
- return storageClient;
- }
-
public static BlobServiceClient createStorageClient(PropertyContext
context) {
- AzureStorageCredentialsService_v12 credentialsService =
context.getProperty(STORAGE_CREDENTIALS_SERVICE).asControllerService(AzureStorageCredentialsService_v12.class);
- AzureStorageCredentialsDetails_v12 credentialsDetails =
credentialsService.getCredentialsDetails();
+ final AzureStorageCredentialsService_v12 credentialsService =
context.getProperty(STORAGE_CREDENTIALS_SERVICE).asControllerService(AzureStorageCredentialsService_v12.class);
+ final AzureStorageCredentialsDetails_v12 credentialsDetails =
credentialsService.getCredentialsDetails();
- BlobServiceClientBuilder clientBuilder = new
BlobServiceClientBuilder();
+ final BlobServiceClientBuilder clientBuilder = new
BlobServiceClientBuilder();
clientBuilder.endpoint(String.format("https://%s.%s";,
credentialsDetails.getAccountName(), credentialsDetails.getEndpointSuffix()));
+ final NettyAsyncHttpClientBuilder nettyClientBuilder = new
NettyAsyncHttpClientBuilder();
+ AzureStorageUtils.configureProxy(nettyClientBuilder, context);
+
+ final HttpClient nettyClient = nettyClientBuilder.build();
+ clientBuilder.httpClient(nettyClient);
+
configureCredential(clientBuilder, credentialsService,
credentialsDetails);
return clientBuilder.buildClient();
}
- private static void configureCredential(BlobServiceClientBuilder
clientBuilder, AzureStorageCredentialsService_v12 credentialsService,
AzureStorageCredentialsDetails_v12 credentialsDetails) {
+ private static void configureCredential(BlobServiceClientBuilder
clientBuilder, AzureStorageCredentialsService_v12 credentialsService,
+ AzureStorageCredentialsDetails_v12
credentialsDetails) {
Review Comment:
Max line length is 200. Please do not apply unnecessary reformatting.
##########
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/AbstractAzureBlobProcessor_v12.java:
##########
@@ -131,7 +128,7 @@ private static void
configureCredential(BlobServiceClientBuilder clientBuilder,
break;
case MANAGED_IDENTITY:
clientBuilder.credential(new ManagedIdentityCredentialBuilder()
-
.clientId(credentialsDetails.getManagedIdentityClientId())
+
.clientId(credentialsDetails.getManagedIdentityClientId())
Review Comment:
This is a good example for fixing formatting. Thanks!
##########
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/PutAzureDataLakeStorage.java:
##########
@@ -83,18 +83,36 @@ public class PutAzureDataLakeStorage extends
AbstractAzureDataLakeStorageProcess
.allowableValues(FAIL_RESOLUTION, REPLACE_RESOLUTION,
IGNORE_RESOLUTION)
.build();
- private List<PropertyDescriptor> properties;
+ private static final List<PropertyDescriptor> PROPERTIES =
Collections.unmodifiableList(Arrays.asList(
+ CONFLICT_RESOLUTION
+ ));
- @Override
- protected void init(final ProcessorInitializationContext context) {
- final List<PropertyDescriptor> props = new
ArrayList<>(super.getSupportedPropertyDescriptors());
- props.add(CONFLICT_RESOLUTION);
- properties = Collections.unmodifiableList(props);
+ public static long MAX_CHUNK_SIZE = 100 * 1024 * 1024; // current chunk
limit is 100 MiB on Azure
+
+ static void uploadContent(DataLakeFileClient fileClient, InputStream in,
long length) {
Review Comment:
Please do not reorder class members because it makes the git history hard to
follow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
