[jira] [Commented] (OAK-7702) [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for uploads
[ https://issues.apache.org/jira/browse/OAK-7702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826489#comment-16826489 ] Matt Ryan commented on OAK-7702: Some more basic info about the testing I did: * Blobs of different sizes were stored in an Azure blob storage container * Signed download URIs (both CDN and non-CDN) were created for each blob * Each signed URI was requested 50 times, and measurements for time to completion and throughput captured for each request * These tests were conducted from Azure and AWS virtual machines in eight different regions worldwide: ** US Northwest (where the blob storage was) ** US East ** Sao Paulo ** Ireland ** Paris ** Mumbai ** Sydney ** Tokyo > [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for > uploads > - > > Key: OAK-7702 > URL: https://issues.apache.org/jira/browse/OAK-7702 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob-cloud, blob-cloud-azure >Reporter: Alexander Klimetschek >Assignee: Matt Ryan >Priority: Major > > Azure Blob Store and S3 support CDNs in front of private containers/buckets, > which also work with presigned URLs ([S3 > docs|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-urls.html], > [cloudfront presigning > java|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/CFPrivateDistJavaDevelopment.html], > [Azure > docs|https://docs.microsoft.com/en-us/azure/cdn/cdn-sas-storage-support]). > The binary access feature should support these for download URLs via a > configuration on the DataStore. > Currently, the S3 datastore has a config for [transfer > acceleration|https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html] > that if enabled, will make both upload & download URLs use the acceleration > option. However, this feature only makes sense for uploads, especially if CDN > is an option for downloads. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7702) [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for uploads
[ https://issues.apache.org/jira/browse/OAK-7702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826481#comment-16826481 ] Alexander Klimetschek commented on OAK-7702: S3 Transfer acceleration doesn’t cache and is not a CDN. > [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for > uploads > - > > Key: OAK-7702 > URL: https://issues.apache.org/jira/browse/OAK-7702 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob-cloud, blob-cloud-azure >Reporter: Alexander Klimetschek >Assignee: Matt Ryan >Priority: Major > > Azure Blob Store and S3 support CDNs in front of private containers/buckets, > which also work with presigned URLs ([S3 > docs|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-urls.html], > [cloudfront presigning > java|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/CFPrivateDistJavaDevelopment.html], > [Azure > docs|https://docs.microsoft.com/en-us/azure/cdn/cdn-sas-storage-support]). > The binary access feature should support these for download URLs via a > configuration on the DataStore. > Currently, the S3 datastore has a config for [transfer > acceleration|https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html] > that if enabled, will make both upload & download URLs use the acceleration > option. However, this feature only makes sense for uploads, especially if CDN > is an option for downloads. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7702) [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for uploads
[ https://issues.apache.org/jira/browse/OAK-7702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826472#comment-16826472 ] Matt Ryan commented on OAK-7702: FWIF, S3 transfer acceleration offers similar benefits to CloudFront (AWS's CDN) but with a different technology. Transfer acceleration allows you to resolve your request at a local node and use AWS's higher speed network to fulfill the request from the origin. There is no caching involved with transfer acceleration, so subsequent requests still are resolved at a local node but fulfilled from origin every time. CloudFront would follow a similar pattern for the first request, but the response would be cached at the local node so would be resolved and fulfilled at the local node in subsequent requests. [~alexander.klimetschek] it would be helpful to have some more context around your comment pertaining to S3 transfer acceleration where you asserted that "this feature only makes sense for uploads". Is that because uploads by definition are non-cacheable so CloudFront offers no additional advantage over transfer acceleration when it comes to uploads? If so I'd argue it may still be worthwhile to add support for uploading to S3 via CloudFront URIs, because this would mean a customer was not required to turn on transfer acceleration in addition to CloudFront to gain the upload speed improvement. > [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for > uploads > - > > Key: OAK-7702 > URL: https://issues.apache.org/jira/browse/OAK-7702 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob-cloud, blob-cloud-azure >Reporter: Alexander Klimetschek >Assignee: Matt Ryan >Priority: Major > > Azure Blob Store and S3 support CDNs in front of private containers/buckets, > which also work with presigned URLs ([S3 > docs|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-urls.html], > [cloudfront presigning > java|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/CFPrivateDistJavaDevelopment.html], > [Azure > docs|https://docs.microsoft.com/en-us/azure/cdn/cdn-sas-storage-support]). > The binary access feature should support these for download URLs via a > configuration on the DataStore. > Currently, the S3 datastore has a config for [transfer > acceleration|https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html] > that if enabled, will make both upload & download URLs use the acceleration > option. However, this feature only makes sense for uploads, especially if CDN > is an option for downloads. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7702) [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for uploads
[ https://issues.apache.org/jira/browse/OAK-7702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826469#comment-16826469 ] Matt Ryan commented on OAK-7702: In a conversation with some others on the Oak team yesterday we went over the test results I had seen from Azure and how generally using the CDN URI - at least for downloads - offers equal if not better performance in most cases. I'm planning on moving forward with the implementation change in Oak to allow configuring a CDN domain, once this is better tested and verified to also work for S3. > [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for > uploads > - > > Key: OAK-7702 > URL: https://issues.apache.org/jira/browse/OAK-7702 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob-cloud, blob-cloud-azure >Reporter: Alexander Klimetschek >Assignee: Matt Ryan >Priority: Major > > Azure Blob Store and S3 support CDNs in front of private containers/buckets, > which also work with presigned URLs ([S3 > docs|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-urls.html], > [cloudfront presigning > java|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/CFPrivateDistJavaDevelopment.html], > [Azure > docs|https://docs.microsoft.com/en-us/azure/cdn/cdn-sas-storage-support]). > The binary access feature should support these for download URLs via a > configuration on the DataStore. > Currently, the S3 datastore has a config for [transfer > acceleration|https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html] > that if enabled, will make both upload & download URLs use the acceleration > option. However, this feature only makes sense for uploads, especially if CDN > is an option for downloads. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7702) [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for uploads
[
https://issues.apache.org/jira/browse/OAK-7702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826464#comment-16826464
]
Matt Ryan commented on OAK-7702:
I did a quick POC using {{AzureDataStore}} which was fairly straightforward.
All that is required for Azure is for the user to create an Azure CDN connected
to their blob storage (e.g. via the Azure portal), then to use the new Azure
CDN domain name in the signed URI instead of the standard domain name.
In my testing, generally using the CDN URI gives equal or better performance as
compared to a standard URI. This appears to be true even for the initial
request, before the object has actually been put into the local CDN node cache.
One exception to this is if the machine requesting the URI is an Azure VM
running in the same Azure region as the Azure Blob Storage, in which case
requesting the standard URI offers better performance.
It is also possible to perform uploads via a signed CDN URI. I haven't had the
opportunity to conduct testing to compare this to standard uploads but hope to
do this soon.
I haven't done any experimentation with S3 yet - but again hope to be able to
do this soon.
> [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for
> uploads
> -
>
> Key: OAK-7702
> URL: https://issues.apache.org/jira/browse/OAK-7702
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: blob-cloud, blob-cloud-azure
>Reporter: Alexander Klimetschek
>Assignee: Matt Ryan
>Priority: Major
>
> Azure Blob Store and S3 support CDNs in front of private containers/buckets,
> which also work with presigned URLs ([S3
> docs|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-urls.html],
> [cloudfront presigning
> java|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/CFPrivateDistJavaDevelopment.html],
> [Azure
> docs|https://docs.microsoft.com/en-us/azure/cdn/cdn-sas-storage-support]).
> The binary access feature should support these for download URLs via a
> configuration on the DataStore.
> Currently, the S3 datastore has a config for [transfer
> acceleration|https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html]
> that if enabled, will make both upload & download URLs use the acceleration
> option. However, this feature only makes sense for uploads, especially if CDN
> is an option for downloads.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
