[
https://issues.apache.org/jira/browse/NIFI-14318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18038553#comment-18038553
]
ASF subversion and git services commented on NIFI-14318:
--------------------------------------------------------
Commit 39edaaedb2b48f22635b2841ac3212bc6dec9c63 in nifi's branch
refs/heads/main from pkelly-nifi
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=39edaaedb2 ]
NIFI-14318 Add support for uploading files larger than 200GB to Azure Blob
Storage
Signed-off-by: Pierre Villard <[email protected]>
This closes #10533.
> PutAzureBlobStorage_v12 cannot upload files larger than 200GB
> -------------------------------------------------------------
>
> Key: NIFI-14318
> URL: https://issues.apache.org/jira/browse/NIFI-14318
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 2.6.0
> Reporter: Paul Kelly
> Assignee: Paul Kelly
> Priority: Major
> Labels: azure, azureblob
> Time Spent: 10m
> Remaining Estimate: 0h
>
> PutAzureBlobStorage_v12 currently uses the Azure Blob Storage default block
> size of 4MB. Azure Blob Storage has a maximum number of blocks of 50,000 per
> blob. Therefore, the largest file that PutAzureBlobStorage_v12 can currently
> upload is 4MB * 50,000 = 200GB. The service supports blocks up to 4,000MiB
> for a total of 4,000MiB * 50,000 = 190.7TiB per blob, about 1,000x larger
> than NiFi can currently upload. The block size should be calculated
> dynamically – or at least be configurable – to allow uploading files larger
> than 200GB.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
