[jira] [Commented] (OAK-8696) [Direct Binary Access] upload algorithm documentation should also be on the website
[ https://issues.apache.org/jira/browse/OAK-8696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16983924#comment-16983924 ] Alexander Klimetschek commented on OAK-8696: Link is fine if it stays up to date. However, to me as a developer using Oak, the javadoc is actually the place that is duplicated (different versions, urls have changed and broke). To most readers, only the published javadoc counts, not the source code. So I think from the perspective of the readers the documentation page is the best central place, also given that this is what a Google search for “oak direct binary” et al. will surface first. The algorithm description doesn’t really change much anymore and copy pasting should be minimal effort whenever that might happen again - simply copy the generated javadoc html into the markdown. > [Direct Binary Access] upload algorithm documentation should also be on the > website > --- > > Key: OAK-8696 > URL: https://issues.apache.org/jira/browse/OAK-8696 > Project: Jackrabbit Oak > Issue Type: Bug > Components: doc >Reporter: Alexander Klimetschek >Assignee: Matt Ryan >Priority: Major > > The upload algorithm description that is currently a bit hidden in the > javadocs of > [https://github.com/apache/jackrabbit-oak/blob/trunk/oak-jackrabbit-api/src/main/java/org/apache/jackrabbit/api/binary/BinaryUpload.java] > should also be included on the documentation site: > [https://jackrabbit.apache.org/oak/docs/features/direct-binary-access.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OAK-8695) [Direct Binary Access] upload algorithm documentation should make it clear that not all URIs have to be used
[ https://issues.apache.org/jira/browse/OAK-8695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16983922#comment-16983922 ] Alexander Klimetschek commented on OAK-8695: I was thinking this detail should be mentioned inside the detailed algorithm description (currently in the javadoc, see description) not in the high level steps on the documentation page. > [Direct Binary Access] upload algorithm documentation should make it clear > that not all URIs have to be used > > > Key: OAK-8695 > URL: https://issues.apache.org/jira/browse/OAK-8695 > Project: Jackrabbit Oak > Issue Type: Bug > Components: api, doc >Reporter: Alexander Klimetschek >Assignee: Matt Ryan >Priority: Major > > Regarding [BinaryUpload > javadoc|http://jackrabbit.apache.org/oak/docs/apidocs/org/apache/jackrabbit/api/binary/BinaryUpload.html] > ([code > here|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-jackrabbit-api/src/main/java/org/apache/jackrabbit/api/binary/BinaryUpload.java]): > In the "Steps" section for #3, it should explicitly state that not all the > URIs need to be used. It could be inferred based on what is there, but it's > not obvious. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (OAK-8696) [Direct Binary Access] upload algorithm documentation should also be on the website
[ https://issues.apache.org/jira/browse/OAK-8696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek reassigned OAK-8696: -- Assignee: Matt Ryan > [Direct Binary Access] upload algorithm documentation should also be on the > website > --- > > Key: OAK-8696 > URL: https://issues.apache.org/jira/browse/OAK-8696 > Project: Jackrabbit Oak > Issue Type: Bug > Components: doc >Reporter: Alexander Klimetschek >Assignee: Matt Ryan >Priority: Major > > The upload algorithm description that is currently a bit hidden in the > javadocs of > [https://github.com/apache/jackrabbit-oak/blob/trunk/oak-jackrabbit-api/src/main/java/org/apache/jackrabbit/api/binary/BinaryUpload.java] > should also be included on the documentation site: > [https://jackrabbit.apache.org/oak/docs/features/direct-binary-access.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (OAK-8695) [Direct Binary Access] upload algorithm documentation should make it clear that not all URIs have to be used
[ https://issues.apache.org/jira/browse/OAK-8695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek reassigned OAK-8695: -- Assignee: Matt Ryan (was: Matt Ryan) > [Direct Binary Access] upload algorithm documentation should make it clear > that not all URIs have to be used > > > Key: OAK-8695 > URL: https://issues.apache.org/jira/browse/OAK-8695 > Project: Jackrabbit Oak > Issue Type: Bug > Components: api, doc >Reporter: Alexander Klimetschek >Assignee: Matt Ryan >Priority: Major > > Regarding [BinaryUpload > javadoc|http://jackrabbit.apache.org/oak/docs/apidocs/org/apache/jackrabbit/api/binary/BinaryUpload.html] > ([code > here|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-jackrabbit-api/src/main/java/org/apache/jackrabbit/api/binary/BinaryUpload.java]): > In the "Steps" section for #3, it should explicitly state that not all the > URIs need to be used. It could be inferred based on what is there, but it's > not obvious. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (OAK-8695) [Direct Binary Access] upload algorithm documentation should make it clear that not all URIs have to be used
[ https://issues.apache.org/jira/browse/OAK-8695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek reassigned OAK-8695: -- Assignee: Matt Ryan > [Direct Binary Access] upload algorithm documentation should make it clear > that not all URIs have to be used > > > Key: OAK-8695 > URL: https://issues.apache.org/jira/browse/OAK-8695 > Project: Jackrabbit Oak > Issue Type: Bug > Components: api, doc >Reporter: Alexander Klimetschek >Assignee: Matt Ryan >Priority: Major > > Regarding [BinaryUpload > javadoc|http://jackrabbit.apache.org/oak/docs/apidocs/org/apache/jackrabbit/api/binary/BinaryUpload.html] > ([code > here|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-jackrabbit-api/src/main/java/org/apache/jackrabbit/api/binary/BinaryUpload.java]): > In the "Steps" section for #3, it should explicitly state that not all the > URIs need to be used. It could be inferred based on what is there, but it's > not obvious. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OAK-8695) [Direct Binary Access] upload algorithm documentation should make it clear that not all URIs have to be used
[ https://issues.apache.org/jira/browse/OAK-8695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-8695: --- Component/s: doc > [Direct Binary Access] upload algorithm documentation should make it clear > that not all URIs have to be used > > > Key: OAK-8695 > URL: https://issues.apache.org/jira/browse/OAK-8695 > Project: Jackrabbit Oak > Issue Type: Bug > Components: api, doc >Reporter: Alexander Klimetschek >Priority: Major > > Regarding [BinaryUpload > javadoc|http://jackrabbit.apache.org/oak/docs/apidocs/org/apache/jackrabbit/api/binary/BinaryUpload.html] > ([code > here|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-jackrabbit-api/src/main/java/org/apache/jackrabbit/api/binary/BinaryUpload.java]): > In the "Steps" section for #3, it should explicitly state that not all the > URIs need to be used. It could be inferred based on what is there, but it's > not obvious. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (OAK-8696) [Direct Binary Access] upload algorithm documentation should also be on the website
Alexander Klimetschek created OAK-8696: -- Summary: [Direct Binary Access] upload algorithm documentation should also be on the website Key: OAK-8696 URL: https://issues.apache.org/jira/browse/OAK-8696 Project: Jackrabbit Oak Issue Type: Bug Components: doc Reporter: Alexander Klimetschek The upload algorithm description that is currently a bit hidden in the javadocs of [https://github.com/apache/jackrabbit-oak/blob/trunk/oak-jackrabbit-api/src/main/java/org/apache/jackrabbit/api/binary/BinaryUpload.java] should also be included on the documentation site: [https://jackrabbit.apache.org/oak/docs/features/direct-binary-access.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (OAK-8695) [Direct Binary Access] upload algorithm documentation should make it clear that not all URIs have to be used
Alexander Klimetschek created OAK-8695: -- Summary: [Direct Binary Access] upload algorithm documentation should make it clear that not all URIs have to be used Key: OAK-8695 URL: https://issues.apache.org/jira/browse/OAK-8695 Project: Jackrabbit Oak Issue Type: Bug Components: api Reporter: Alexander Klimetschek Regarding [BinaryUpload javadoc|http://jackrabbit.apache.org/oak/docs/apidocs/org/apache/jackrabbit/api/binary/BinaryUpload.html] ([code here|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-jackrabbit-api/src/main/java/org/apache/jackrabbit/api/binary/BinaryUpload.java]): In the "Steps" section for #3, it should explicitly state that not all the URIs need to be used. It could be inferred based on what is there, but it's not obvious. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (OAK-8552) Minimize network calls required when creating a direct download URI
[ https://issues.apache.org/jira/browse/OAK-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917841#comment-16917841 ] Alexander Klimetschek edited comment on OAK-8552 at 8/28/19 2:54 PM: - [~amitjain] I don’t think [~ianeboston] was suggesting to use the jcr:lastModified JCR property for that. All of these could be internal fields not exposed on the JCR level. With direct binary access we also no longer support de-duplication (too costly), so that aspect can be ignored for binaries uploaded that way or if a corresponding configuration is set. But there is still copying/versioning of nodes which leads to multiple references to the same blob - does this lead to a last modified update of the blob? (Just curious because I wasn’t aware of that) Nonetheless, for the issue at hand I don’t think we need to change anything stored in the NodeStore. The size/length is already stored in the NS as part of the internal blob id (OAK-8314). was (Author: alexander.klimetschek): [~amitjain] I don’t think [~ianeboston] was suggesting to use the jcr:lastModified JCR property for that. All of these could be internal fields not exposed on the JCR level. With direct binary access we also no longer support de-duplication (too costly), so that aspect can be ignored for binaries uploaded that way or if a corresponding configuration is set. But there is still copying/versioning of nodes which leads to multiple references to the same blob - does this lead to a last modified update of the blob? (Just curious because I wasn’t aware of that) Nonetheless, for the issue at hand I don’t think we need to change anything stored in the NodeStore. The size/length is already stored in the NS as part of the internal blob id. > Minimize network calls required when creating a direct download URI > --- > > Key: OAK-8552 > URL: https://issues.apache.org/jira/browse/OAK-8552 > Project: Jackrabbit Oak > Issue Type: Sub-task > Components: blob-cloud, blob-cloud-azure >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-8552_ApiChange.patch > > > We need to isolate and try to optimize network calls required to create a > direct download URI. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (OAK-8552) Minimize network calls required when creating a direct download URI
[ https://issues.apache.org/jira/browse/OAK-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917841#comment-16917841 ] Alexander Klimetschek commented on OAK-8552: [~amitjain] I don’t think [~ianeboston] was suggesting to use the jcr:lastModified JCR property for that. All of these could be internal fields not exposed on the JCR level. With direct binary access we also no longer support de-duplication (too costly), so that aspect can be ignored for binaries uploaded that way or if a corresponding configuration is set. But there is still copying/versioning of nodes which leads to multiple references to the same blob - does this lead to a last modified update of the blob? (Just curious because I wasn’t aware of that) Nonetheless, for the issue at hand I don’t think we need to change anything stored in the NodeStore. The size/length is already stored in the NS as part of the internal blob id. > Minimize network calls required when creating a direct download URI > --- > > Key: OAK-8552 > URL: https://issues.apache.org/jira/browse/OAK-8552 > Project: Jackrabbit Oak > Issue Type: Sub-task > Components: blob-cloud, blob-cloud-azure >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-8552_ApiChange.patch > > > We need to isolate and try to optimize network calls required to create a > direct download URI. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (OAK-8580) Stack trace logging for all binary downloading of cloud blob stores
[
https://issues.apache.org/jira/browse/OAK-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917291#comment-16917291
]
Alexander Klimetschek commented on OAK-8580:
A few more thoughts based on practical usage of the above patch:
* it logs downloading of {{reference.key}} which could probably be filtered out
* since this is in the BlobStore impl, this will only log once, but then it's
in the DS cache, and thus the code is not called again. this seems like a
feature, as it reduces the log output a lot. and the goal of the log output is
to get the unique stack traces, not every single invocation.
* we cannot see the JCR path in the logs, which would be extremely helpful.
there is typically the GET /some/url in the log output as part of the thread
name, which helps to some degree, but not always.
it is not trivial because down in the blob store the knowledge of the binary
has been lost. but maybe there is a trick to add it, for example by updating
the thread name somewhere higher up the stack where we retrieve the binary:
{code:java}
Thread.currentThread().setName(Thread.currentThread().getName() + " (jcr path:
" + jcrPath + ")"); {code}
> Stack trace logging for all binary downloading of cloud blob stores
> ---
>
> Key: OAK-8580
> URL: https://issues.apache.org/jira/browse/OAK-8580
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: blob-cloud, blob-cloud-azure
>Affects Versions: 1.16.0, 1.8.16
>Reporter: Alexander Klimetschek
>Assignee: Matt Ryan
>Priority: Major
> Attachments: OAK-8580.patch
>
>
> We want to be able to trace application code that invokes
> Binary.getInputStream() when a cloud blob store is present.
> The Azure blob store has one place where it supports trace logging
> [here|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-cloud-azure/src/main/java/org/apache/jackrabbit/oak/blob/cloud/azure/blobstorage/AzureBlobStoreBackend.java#L1202-L1207],
> but this misses another place blobs are fetched in
> [read()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-cloud-azure/src/main/java/org/apache/jackrabbit/oak/blob/cloud/azure/blobstorage/AzureBlobStoreBackend.java#L218].
> Furthermore, it would be nice to have a separate logger for that in order to
> create a log file that _only_ includes these stack traces, and no other debug
> logs of the AzureBackend class.
> And same for the S3 BlobStore implementation.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OAK-8580) Stack trace logging for all binary downloading of cloud blob stores
[ https://issues.apache.org/jira/browse/OAK-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917271#comment-16917271 ] Alexander Klimetschek commented on OAK-8580: Here is a patch for the AzureBlobStoreBackend: [^OAK-8580.patch] This adds a new logger "oak.datastore.download.streams" to make it separate from the existing logger based on the class. The same name could be used in the S3 BlobStore implementation. Better name welcome. > Stack trace logging for all binary downloading of cloud blob stores > --- > > Key: OAK-8580 > URL: https://issues.apache.org/jira/browse/OAK-8580 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-cloud, blob-cloud-azure >Affects Versions: 1.16.0, 1.8.16 >Reporter: Alexander Klimetschek >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-8580.patch > > > We want to be able to trace application code that invokes > Binary.getInputStream() when a cloud blob store is present. > The Azure blob store has one place where it supports trace logging > [here|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-cloud-azure/src/main/java/org/apache/jackrabbit/oak/blob/cloud/azure/blobstorage/AzureBlobStoreBackend.java#L1202-L1207], > but this misses another place blobs are fetched in > [read()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-cloud-azure/src/main/java/org/apache/jackrabbit/oak/blob/cloud/azure/blobstorage/AzureBlobStoreBackend.java#L218]. > Furthermore, it would be nice to have a separate logger for that in order to > create a log file that _only_ includes these stack traces, and no other debug > logs of the AzureBackend class. > And same for the S3 BlobStore implementation. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (OAK-8580) Stack trace logging for all binary downloading of cloud blob stores
[ https://issues.apache.org/jira/browse/OAK-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-8580: --- Attachment: OAK-8580.patch > Stack trace logging for all binary downloading of cloud blob stores > --- > > Key: OAK-8580 > URL: https://issues.apache.org/jira/browse/OAK-8580 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-cloud, blob-cloud-azure >Affects Versions: 1.16.0, 1.8.16 >Reporter: Alexander Klimetschek >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-8580.patch > > > We want to be able to trace application code that invokes > Binary.getInputStream() when a cloud blob store is present. > The Azure blob store has one place where it supports trace logging > [here|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-cloud-azure/src/main/java/org/apache/jackrabbit/oak/blob/cloud/azure/blobstorage/AzureBlobStoreBackend.java#L1202-L1207], > but this misses another place blobs are fetched in > [read()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-cloud-azure/src/main/java/org/apache/jackrabbit/oak/blob/cloud/azure/blobstorage/AzureBlobStoreBackend.java#L218]. > Furthermore, it would be nice to have a separate logger for that in order to > create a log file that _only_ includes these stack traces, and no other debug > logs of the AzureBackend class. > And same for the S3 BlobStore implementation. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (OAK-8580) Stack trace logging for all binary downloading of cloud blob stores
Alexander Klimetschek created OAK-8580: -- Summary: Stack trace logging for all binary downloading of cloud blob stores Key: OAK-8580 URL: https://issues.apache.org/jira/browse/OAK-8580 Project: Jackrabbit Oak Issue Type: Bug Components: blob-cloud, blob-cloud-azure Affects Versions: 1.8.16, 1.16.0 Reporter: Alexander Klimetschek Assignee: Matt Ryan We want to be able to trace application code that invokes Binary.getInputStream() when a cloud blob store is present. The Azure blob store has one place where it supports trace logging [here|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-cloud-azure/src/main/java/org/apache/jackrabbit/oak/blob/cloud/azure/blobstorage/AzureBlobStoreBackend.java#L1202-L1207], but this misses another place blobs are fetched in [read()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-cloud-azure/src/main/java/org/apache/jackrabbit/oak/blob/cloud/azure/blobstorage/AzureBlobStoreBackend.java#L218]. Furthermore, it would be nice to have a separate logger for that in order to create a log file that _only_ includes these stack traces, and no other debug logs of the AzureBackend class. And same for the S3 BlobStore implementation. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (OAK-8552) Minimize network calls required when creating a direct download URI
[ https://issues.apache.org/jira/browse/OAK-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916427#comment-16916427 ] Alexander Klimetschek commented on OAK-8552: [~amitjain] +1 to Blob#isInlined(), that seems like the right solution in place of the getReference() == null. > Minimize network calls required when creating a direct download URI > --- > > Key: OAK-8552 > URL: https://issues.apache.org/jira/browse/OAK-8552 > Project: Jackrabbit Oak > Issue Type: Sub-task > Components: blob-cloud, blob-cloud-azure >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-8552_ApiChange.patch > > > We need to isolate and try to optimize network calls required to create a > direct download URI. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (OAK-8552) Minimize network calls required when creating a direct download URI
[ https://issues.apache.org/jira/browse/OAK-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916411#comment-16916411 ] Alexander Klimetschek edited comment on OAK-8552 at 8/27/19 6:22 AM: - To add some context: In our case that uncovered this issue, the problem really only exists because of a pretty special case: we uploaded binaries through JCR (InputStream) with async uploads in caching DS enabled. Then immediately after the session.save() we requested presigned GET URLs for these assets to pass them on to an external service. But the blobs were still uploading to the Azure blob store at that point and were not reliably available yet for the external service. That lead us first to make the presigned URL generation support existence check (OAK-7998, plus the implicit check in getReference()) and „polling“ by returning null for not-yet-in-blob-store cases. However, that is shifting the solution to the wrong end and increasing application complexity (polling loop). (Also note this edge case is a short term solution to be replaced at some point with proper direct binary access for upload) The source of the problem here is the async upload: we need to switch this to synchronous uploads = blocking session.save(), to avoid the issue in the first place. In all regular cases, binaries are uploaded through the new direct binary access, which by design guarantees the presence of the binary when the reference is added to the NodeStore. If the binary gets deleted from the blob store due to some malfunctioning, then it does not matter to the application if we return null or an URL that returns 404. The binary is not available either way. But the latter allows us to completely drop any existence checks upon presigned GET URL generation. Same for inlined: configuration must prevent this in the first place, then no special check is required at access time. was (Author: alexander.klimetschek): To add some context: In our case that uncovered this issue, the problem really only exists because in some special case, we uploaded binaries through JCR with async uploads in caching DS enabled, and then immediately after the session.save() requested presigned GET URLs to pass them on to an external service. That lead us first to make the presigned URL generation support existence check (OAK-7998, plus the implicit check in getReference()) and „polling“ by returning null for not-yet-in-blob-store cases. However, that is shifting the solution to the wrong end and increasing application complexity (polling loop). (Also note this edge case is a short term solution to be replaced at some point with proper direct binary access for upload) The source of the problem here is the async upload: we need to switch this to synchronous uploads = blocking session.save(), to avoid the issue in the first place. In all regular cases, binaries are uploaded through the new direct binary access, which by design guarantees the presence of the binary when the reference is added to the NodeStore. If the binary gets deleted from the blob store due to some malfunctioning, then it does not matter to the application if we return null or an URL that returns 404. The binary is not available either way. But the latter allows us to completely drop any existence checks upon presigned GET URL generation. Same for inlined: configuration must prevent this in the first place, then no special check is required at access time. > Minimize network calls required when creating a direct download URI > --- > > Key: OAK-8552 > URL: https://issues.apache.org/jira/browse/OAK-8552 > Project: Jackrabbit Oak > Issue Type: Sub-task > Components: blob-cloud, blob-cloud-azure >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-8552_ApiChange.patch > > > We need to isolate and try to optimize network calls required to create a > direct download URI. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (OAK-8552) Minimize network calls required when creating a direct download URI
[ https://issues.apache.org/jira/browse/OAK-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916411#comment-16916411 ] Alexander Klimetschek edited comment on OAK-8552 at 8/27/19 6:19 AM: - To add some context: In our case that uncovered this issue, the problem really only exists because in some special case, we uploaded binaries through JCR with async uploads in caching DS enabled, and then immediately after the session.save() requested presigned GET URLs to pass them on to an external service. That lead us first to make the presigned URL generation support existence check (OAK-7998, plus the implicit check in getReference()) and „polling“ by returning null for not-yet-in-blob-store cases. However, that is shifting the solution to the wrong end and increasing application complexity (polling loop). (Also note this edge case is a short term solution to be replaced at some point with proper direct binary access for upload) The source of the problem here is the async upload: we need to switch this to synchronous uploads = blocking session.save(), to avoid the issue in the first place. In all regular cases, binaries are uploaded through the new direct binary access, which by design guarantees the presence of the binary when the reference is added to the NodeStore. If the binary gets deleted from the blob store due to some malfunctioning, then it does not matter to the application if we return null or an URL that returns 404. The binary is not available either way. But the latter allows us to completely drop any existence checks upon presigned GET URL generation. Same for inlined: configuration must prevent this in the first place, then no special check is required at access time. was (Author: alexander.klimetschek): To add some context: In our case that uncovered this issue, the problem really only exists because in some special case, we uploaded binaries through JCR with async uploads in caching DS enabled, and then immediately after the session.save() requested presigned GET URLs to pass them on to an external service. That lead us first to make the presigned URL generation support existence check and „polling“ by returning null for not-yet-in-blob-store cases. However, that is shifting the solution to the wrong end and increasing application complexity (polling loop). (Also note this edge case is a short term solution to be replaced at some point with proper direct binary access for upload) The source of the problem here is the async upload: we need to switch this to synchronous uploads = blocking session.save(), to avoid the issue in the first place. In all regular cases, binaries are uploaded through the new direct binary access, which by design guarantees the presence of the binary when the reference is added to the NodeStore. If the binary gets deleted from the blob store due to some malfunctioning, then it does not matter to the application if we return null or an URL that returns 404. The binary is not available either way. But the latter allows us to completely drop any existence checks upon presigned GET URL generation. Same for inlined: configuration must prevent this in the first place, then no special check is required at access time. > Minimize network calls required when creating a direct download URI > --- > > Key: OAK-8552 > URL: https://issues.apache.org/jira/browse/OAK-8552 > Project: Jackrabbit Oak > Issue Type: Sub-task > Components: blob-cloud, blob-cloud-azure >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-8552_ApiChange.patch > > > We need to isolate and try to optimize network calls required to create a > direct download URI. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (OAK-8552) Minimize network calls required when creating a direct download URI
[ https://issues.apache.org/jira/browse/OAK-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916411#comment-16916411 ] Alexander Klimetschek edited comment on OAK-8552 at 8/27/19 6:16 AM: - To add some context: In our case that uncovered this issue, the problem really only exists because in some special case, we uploaded binaries through JCR with async uploads in caching DS enabled, and then immediately after the session.save() requested presigned GET URLs to pass them on to an external service. That lead us first to make the presigned URL generation support existence check and „polling“ by returning null for not-yet-in-blob-store cases. However, that is shifting the solution to the wrong end and increasing application complexity (polling loop). (Also note this edge case is a short term solution to be replaced at some point with proper direct binary access for upload) The source of the problem here is the async upload: we need to switch this to synchronous uploads = blocking session.save(), to avoid the issue in the first place. In all regular cases, binaries are uploaded through the new direct binary access, which by design guarantees the presence of the binary when the reference is added to the NodeStore. If the binary gets deleted from the blob store due to some malfunctioning, then it does not matter to the application if we return null or an URL that returns 404. The binary is not available either way. But the latter allows us to completely drop any existence checks upon presigned GET URL generation. Same for inlined: configuration must prevent this in the first place, then no special check is required at access time. was (Author: alexander.klimetschek): To add some context: In our case that uncovered this issue, the problem really only exists because in some special case, we uploaded binaries through JCR with async uploads in caching DS enabled, and then immediately after the session.save() requested presigned GET URLs to pass them on to an external service. That lead us first to make the presigned URL generation support existence check and „polling“ by returning null for not-yet-in-blob-store cases. However, that is shifting the solution to the wrong end and increasing application complexity (polling loop). (Also note this edge case is a short term solution to be replaced at some point with proper direct binary access for upload) The source of the problem here is the async upload: we need to switch this to synchronous uploads = blocking session.save(), to avoid the issue in the first place. In all regular cases, binaries are uploaded through the new direct binary access, which by design guarantees the presence of the binary when the reference is in the NodeStore. If the binary gets deleted from the blob store due to some malfunctioning, then it does not matter to the application if we return null or an URL that returns 404. But the latter allows us to completely drop any existence checks upon presigned GET URL generation. Same for inlined: configuration must prevent this in the first place, then no special check is required at access time. > Minimize network calls required when creating a direct download URI > --- > > Key: OAK-8552 > URL: https://issues.apache.org/jira/browse/OAK-8552 > Project: Jackrabbit Oak > Issue Type: Sub-task > Components: blob-cloud, blob-cloud-azure >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-8552_ApiChange.patch > > > We need to isolate and try to optimize network calls required to create a > direct download URI. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (OAK-8552) Minimize network calls required when creating a direct download URI
[ https://issues.apache.org/jira/browse/OAK-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916411#comment-16916411 ] Alexander Klimetschek commented on OAK-8552: To add some context: In our case that uncovered this issue, the problem really only exists because in some special case, we uploaded binaries through JCR with async uploads in caching DS enabled, and then immediately after the session.save() requested presigned GET URLs to pass them on to an external service. That lead us first to make the presigned URL generation support existence check and „polling“ by returning null for not-yet-in-blob-store cases. However, that is shifting the solution to the wrong end and increasing application complexity (polling loop). (Also note this edge case is a short term solution to be replaced at some point with proper direct binary access for upload) The source of the problem here is the async upload: we need to switch this to synchronous uploads = blocking session.save(), to avoid the issue in the first place. In all regular cases, binaries are uploaded through the new direct binary access, which by design guarantees the presence of the binary when the reference is in the NodeStore. If the binary gets deleted from the blob store due to some malfunctioning, then it does not matter to the application if we return null or an URL that returns 404. But the latter allows us to completely drop any existence checks upon presigned GET URL generation. Same for inlined: configuration must prevent this in the first place, then no special check is required at access time. > Minimize network calls required when creating a direct download URI > --- > > Key: OAK-8552 > URL: https://issues.apache.org/jira/browse/OAK-8552 > Project: Jackrabbit Oak > Issue Type: Sub-task > Components: blob-cloud, blob-cloud-azure >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-8552_ApiChange.patch > > > We need to isolate and try to optimize network calls required to create a > direct download URI. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (OAK-8520) [Direct Binary Access] Avoid overwriting existing binaries via direct binary upload
[
https://issues.apache.org/jira/browse/OAK-8520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901583#comment-16901583
]
Alexander Klimetschek commented on OAK-8520:
[~mattvryan] Awesome, thanks! Looks like it was even simpler than I thought.
> [Direct Binary Access] Avoid overwriting existing binaries via direct binary
> upload
> ---
>
> Key: OAK-8520
> URL: https://issues.apache.org/jira/browse/OAK-8520
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: blob-cloud, blob-cloud-azure, blob-plugins, doc
>Reporter: Matt Ryan
>Assignee: Matt Ryan
>Priority: Major
> Fix For: 1.18.0, 1.10.4
>
>
> Since direct binary upload generates a unique blob ID for each upload, it is
> generally impossible to overwrite any existing binary. However, if a client
> issues the {{completeBinaryUpload()}} call more than one time with the same
> upload token, it is possible to overwrite an existing binary.
> One use case where this can happen is if a client call to complete the upload
> times out. Lacking a successful return a client could assume that it needs
> to repeat the call to complete the upload. If the binary was already
> uploaded before, the subsequent call to complete the upload would have the
> effect of overwriting the binary with new content generated from any
> uncommitted uploaded blocks. In practice usually there are no uncommitted
> blocks so this generates a zero-length binary.
> There may be a use case for a zero-length binary so simply failing in such a
> case is not sufficient.
> One easy way to handle this would be to simply check for the existence of the
> binary before completing the upload. This would have the effect of making
> uploaded binaries un-modifiable by the client. In such a case the
> implementation could throw an exception indicating that the binary already
> exists and cannot be written again.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (OAK-8520) [Direct Binary Access] Avoid overwriting existing binaries via direct binary upload
[
https://issues.apache.org/jira/browse/OAK-8520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900629#comment-16900629
]
Alexander Klimetschek commented on OAK-8520:
IMO this should be classified as a bug and given a higher priority. The blob
store implementation has to ensure the immutability of blobs and not wipe them
out if applications (accidentally) call completeBinaryUpload() twice.
AFACS {{completeBinaryUpload()}} should simply be an idempotent operation,
returning the existing blob as {{Binary}} if called the 2nd (or 3rd, or 4th...)
time with the same upload token. Then clients can safely retry their request
that leads to the application code to call {{completeBinaryUpload() }}and try
writing the same JCR structure.
> [Direct Binary Access] Avoid overwriting existing binaries via direct binary
> upload
> ---
>
> Key: OAK-8520
> URL: https://issues.apache.org/jira/browse/OAK-8520
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: blob-cloud, blob-cloud-azure, blob-plugins, doc
>Reporter: Matt Ryan
>Assignee: Matt Ryan
>Priority: Major
>
> Since direct binary upload generates a unique blob ID for each upload, it is
> generally impossible to overwrite any existing binary. However, if a client
> issues the {{completeBinaryUpload()}} call more than one time with the same
> upload token, it is possible to overwrite an existing binary.
> One use case where this can happen is if a client call to complete the upload
> times out. Lacking a successful return a client could assume that it needs
> to repeat the call to complete the upload. If the binary was already
> uploaded before, the subsequent call to complete the upload would have the
> effect of overwriting the binary with new content generated from any
> uncommitted uploaded blocks. In practice usually there are no uncommitted
> blocks so this generates a zero-length binary.
> There may be a use case for a zero-length binary so simply failing in such a
> case is not sufficient.
> One easy way to handle this would be to simply check for the existence of the
> binary before completing the upload. This would have the effect of making
> uploaded binaries un-modifiable by the client. In such a case the
> implementation could throw an exception indicating that the binary already
> exists and cannot be written again.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (OAK-7998) [DirectBinaryAccess] Verify that binary exists in cloud before creating signed download URI
[ https://issues.apache.org/jira/browse/OAK-7998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894125#comment-16894125 ] Alexander Klimetschek edited comment on OAK-7998 at 7/26/19 8:54 PM: - [~mattvryan] How is it possible to get the presigned URL if the upload has not completed yet? There should be no reference in the NodeStore at all before the complete. And after a complete, if the blob/file is empty, that is IMO a perfectly valid situation from the blob store perspective (it doesn't know whether a client might want to store an empty file deliberately), for which a presigned GET URL should be generated. We designed it explicitly so that the binary in Oak is guaranteed to be immutable, and thus only visible after upload has completed. was (Author: alexander.klimetschek): [~mattvryan] How is it possible to get the presigned URL if the upload has not completed yet? There should be no reference in the NodeStore at all before the complete. And after a complete, if the blob/file is empty, that is IMO a perfectly valid situation from the blob store perspective (it doesn't know whether a client might want to store an empty file deliberately), for which a presigned GET URL should be generated. > [DirectBinaryAccess] Verify that binary exists in cloud before creating > signed download URI > --- > > Key: OAK-7998 > URL: https://issues.apache.org/jira/browse/OAK-7998 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob-cloud, blob-cloud-azure >Affects Versions: 1.10.0 >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Fix For: 1.16.0, 1.10.4 > > > The direct binary access download logic doesn't actually verify that the > requested blob is available in the cloud before creating the signed download > URI. It is possible that a user could request a download URI for a blob that > is "in the repo" but hasn't actually been uploaded yet. > We should verify this by uploading a new blob, preventing it being uploaded > to the cloud (retain in cache), and then request the download URI. We should > get a null back or get some other error or exception; if we get a URI it > would return an HTTP 404 if the blob is not actually uploaded yet (maybe this > would also be ok). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OAK-7998) [DirectBinaryAccess] Verify that binary exists in cloud before creating signed download URI
[ https://issues.apache.org/jira/browse/OAK-7998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894125#comment-16894125 ] Alexander Klimetschek commented on OAK-7998: [~mattvryan] How is it possible to get the presigned URL if the upload has not completed yet? There should be no reference in the NodeStore at all before the complete. And after a complete, if the blob/file is empty, that is IMO a perfectly valid situation from the blob store perspective (it doesn't know whether a client might want to store an empty file deliberately), for which a presigned GET URL should be generated. > [DirectBinaryAccess] Verify that binary exists in cloud before creating > signed download URI > --- > > Key: OAK-7998 > URL: https://issues.apache.org/jira/browse/OAK-7998 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob-cloud, blob-cloud-azure >Affects Versions: 1.10.0 >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Fix For: 1.16.0, 1.10.4 > > > The direct binary access download logic doesn't actually verify that the > requested blob is available in the cloud before creating the signed download > URI. It is possible that a user could request a download URI for a blob that > is "in the repo" but hasn't actually been uploaded yet. > We should verify this by uploading a new blob, preventing it being uploaded > to the cloud (retain in cache), and then request the download URI. We should > get a null back or get some other error or exception; if we get a URI it > would return an HTTP 404 if the blob is not actually uploaded yet (maybe this > would also be ok). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OAK-7702) [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for uploads
[ https://issues.apache.org/jira/browse/OAK-7702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826481#comment-16826481 ] Alexander Klimetschek commented on OAK-7702: S3 Transfer acceleration doesn’t cache and is not a CDN. > [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for > uploads > - > > Key: OAK-7702 > URL: https://issues.apache.org/jira/browse/OAK-7702 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob-cloud, blob-cloud-azure >Reporter: Alexander Klimetschek >Assignee: Matt Ryan >Priority: Major > > Azure Blob Store and S3 support CDNs in front of private containers/buckets, > which also work with presigned URLs ([S3 > docs|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-urls.html], > [cloudfront presigning > java|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/CFPrivateDistJavaDevelopment.html], > [Azure > docs|https://docs.microsoft.com/en-us/azure/cdn/cdn-sas-storage-support]). > The binary access feature should support these for download URLs via a > configuration on the DataStore. > Currently, the S3 datastore has a config for [transfer > acceleration|https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html] > that if enabled, will make both upload & download URLs use the acceleration > option. However, this feature only makes sense for uploads, especially if CDN > is an option for downloads. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-8013) [Direct Binary Access] DataRecordDownloadOptions creates invalid Content-Disposition headers
[
https://issues.apache.org/jira/browse/OAK-8013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762939#comment-16762939
]
Alexander Klimetschek commented on OAK-8013:
+1 for dropping {{filename*}} as intermediate fix.
While not the optimal, this increases the range of supported filenames a lot
(since spaces are common and non-ISO-8859-1 are not).
> [Direct Binary Access] DataRecordDownloadOptions creates invalid
> Content-Disposition headers
>
>
> Key: OAK-8013
> URL: https://issues.apache.org/jira/browse/OAK-8013
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: blob-plugins
>Reporter: Alexander Klimetschek
>Assignee: Matt Ryan
>Priority: Major
>
> DataRecordDownloadOptions always adds the extended parameter filename* to the
> header, [without any
> escaping|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordDownloadOptions.java#L130].
> Such extended parameters must not include spaces (and only a small predefined
> list of basic ascii chars), otherwise they have to be percent encoded. The
> RFC is https://tools.ietf.org/html/rfc5987 and note the definition of
> value-chars in the grammar.
> Because of this, if a filename includes a space or another character that
> must be percent encoded, this currently creates an invalid header that fails
> to be parsed by other clients.
> See also https://github.com/jshttp/content-disposition/issues/24
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-8013) [Direct Binary Access] DataRecordDownloadOptions creates invalid Content-Disposition headers
[ https://issues.apache.org/jira/browse/OAK-8013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-8013: --- Description: DataRecordDownloadOptions always adds the extended parameter filename* to the header, [without any escaping|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordDownloadOptions.java#L130]. Such extended parameters must not include spaces (and only a small predefined list of basic ascii chars), otherwise they have to be percent encoded. The RFC is https://tools.ietf.org/html/rfc5987 and note the definition of value-chars in the grammar. Because of this, if a filename includes a space or another character that must be percent encoded, this currently creates an invalid header that fails to be parsed by other clients. See also https://github.com/jshttp/content-disposition/issues/24 was: DataRecordDownloadOptions always adds the extended parameter filename* to the header, [without any escaping|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordDownloadOptions.java#L130]. Such extended parameters must not include spaces (and only a small predefined list of basic ascii chars), otherwise they have to be percent escaped. The RFC is https://tools.ietf.org/html/rfc5987 and note the definition of value-chars in the grammar. Because of this, if a filename includes a space, this currently creates an invalid header that fails to be parsed by other clients. See also https://github.com/jshttp/content-disposition/issues/24 > [Direct Binary Access] DataRecordDownloadOptions creates invalid > Content-Disposition headers > > > Key: OAK-8013 > URL: https://issues.apache.org/jira/browse/OAK-8013 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Reporter: Alexander Klimetschek >Priority: Major > > DataRecordDownloadOptions always adds the extended parameter filename* to the > header, [without any > escaping|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordDownloadOptions.java#L130]. > Such extended parameters must not include spaces (and only a small predefined > list of basic ascii chars), otherwise they have to be percent encoded. The > RFC is https://tools.ietf.org/html/rfc5987 and note the definition of > value-chars in the grammar. > Because of this, if a filename includes a space or another character that > must be percent encoded, this currently creates an invalid header that fails > to be parsed by other clients. > See also https://github.com/jshttp/content-disposition/issues/24 > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-8013) [Direct Binary Access] DataRecordDownloadOptions creates invalid Content-Disposition headers
[ https://issues.apache.org/jira/browse/OAK-8013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-8013: --- Description: DataRecordDownloadOptions always adds the extended parameter filename* to the header, [without any escaping|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordDownloadOptions.java#L130]. Such extended parameters must not include spaces (and only a small predefined list of basic ascii chars), otherwise they have to be percent escaped. The RFC is https://tools.ietf.org/html/rfc5987 and note the definition of value-chars in the grammar. Because of this, if a filename includes a space, this currently creates an invalid header that fails to be parsed by other clients. See also https://github.com/jshttp/content-disposition/issues/24 was: DataRecordDownloadOptions always adds the extended parameter filename* to the header, [without any escaping|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordDownloadOptions.java#L130]. Such extended parameters must not include spaces (and only a small predefined list of basic ascii chars), otherwise they have to be percent escaped. The RFC is https://tools.ietf.org/html/rfc5987 and note the definition of the value-chars in the Because of this, if a filename includes a space, this currently creates an invalid header that fails to be parsed by other clients. See also https://github.com/jshttp/content-disposition/issues/24 > [Direct Binary Access] DataRecordDownloadOptions creates invalid > Content-Disposition headers > > > Key: OAK-8013 > URL: https://issues.apache.org/jira/browse/OAK-8013 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Reporter: Alexander Klimetschek >Priority: Major > > DataRecordDownloadOptions always adds the extended parameter filename* to the > header, [without any > escaping|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordDownloadOptions.java#L130]. > Such extended parameters must not include spaces (and only a small predefined > list of basic ascii chars), otherwise they have to be percent escaped. The > RFC is https://tools.ietf.org/html/rfc5987 and note the definition of > value-chars in the grammar. > Because of this, if a filename includes a space, this currently creates an > invalid header that fails to be parsed by other clients. > See also https://github.com/jshttp/content-disposition/issues/24 > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OAK-8013) [Direct Binary Access] DataRecordDownloadOptions creates invalid Content-Disposition headers
Alexander Klimetschek created OAK-8013: -- Summary: [Direct Binary Access] DataRecordDownloadOptions creates invalid Content-Disposition headers Key: OAK-8013 URL: https://issues.apache.org/jira/browse/OAK-8013 Project: Jackrabbit Oak Issue Type: Bug Components: blob-plugins Reporter: Alexander Klimetschek DataRecordDownloadOptions always adds the extended parameter filename* to the header, [without any escaping|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordDownloadOptions.java#L130]. Such extended parameters must not include spaces (and only a small predefined list of basic ascii chars), otherwise they have to be percent escaped. The RFC is https://tools.ietf.org/html/rfc5987 and note the definition of the value-chars in the Because of this, if a filename includes a space, this currently creates an invalid header that fails to be parsed by other clients. See also https://github.com/jshttp/content-disposition/issues/24 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7702) [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for uploads
[ https://issues.apache.org/jira/browse/OAK-7702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-7702: --- Description: Azure Blob Store and S3 support CDNs in front of private containers/buckets, which also work with presigned URLs ([S3 docs|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-urls.html], [cloudfront presigning java|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/CFPrivateDistJavaDevelopment.html], [Azure docs|https://docs.microsoft.com/en-us/azure/cdn/cdn-sas-storage-support]). The binary access feature should support these for download URLs via a configuration on the DataStore. Currently, the S3 datastore has a config for [transfer acceleration|https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html] that if enabled, will make both upload & download URLs use the acceleration option. However, this feature only makes sense for uploads, especially if CDN is an option for downloads. was: Azure Blob Store and S3 support CDNs in front of private containers/buckets, which also work with presigned URLs ([S3 docs|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-urls.html], [Azure docs|https://docs.microsoft.com/en-us/azure/cdn/cdn-sas-storage-support]). The binary access feature should support these for download URLs via a configuration on the DataStore. Currently, the S3 datastore has a config for [transfer acceleration|https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html] that if enabled, will make both upload & download URLs use the acceleration option. However, this feature only makes sense for uploads, especially if CDN is an option for downloads. > [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for > uploads > - > > Key: OAK-7702 > URL: https://issues.apache.org/jira/browse/OAK-7702 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob-cloud, blob-cloud-azure >Reporter: Alexander Klimetschek >Priority: Major > > Azure Blob Store and S3 support CDNs in front of private containers/buckets, > which also work with presigned URLs ([S3 > docs|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-urls.html], > [cloudfront presigning > java|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/CFPrivateDistJavaDevelopment.html], > [Azure > docs|https://docs.microsoft.com/en-us/azure/cdn/cdn-sas-storage-support]). > The binary access feature should support these for download URLs via a > configuration on the DataStore. > Currently, the S3 datastore has a config for [transfer > acceleration|https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html] > that if enabled, will make both upload & download URLs use the acceleration > option. However, this feature only makes sense for uploads, especially if CDN > is an option for downloads. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek resolved OAK-7832. Resolution: Fixed Assignee: Alexander Klimetschek Fix Version/s: 1.8.9 1.10 Committed to trunk in 1844932. Backported to 1.8 in 1844933. > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, store-spi >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > Fix For: 1.10, 1.8.9 > > Attachments: OAK-7832.patch, oak-run-1.8.9-SNAPSHOT.jar > > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16654724#comment-16654724 ] Alexander Klimetschek commented on OAK-7832: Note that JsonSerializer continues to behave like before (throw on any exception), unless the new constructor with the new „checkException“ argument set to true is used, which only the export command from oak-run does. But please double check. > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, store-spi >Reporter: Alexander Klimetschek >Priority: Major > Attachments: OAK-7832.patch, oak-run-1.8.9-SNAPSHOT.jar > > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16654436#comment-16654436 ] Alexander Klimetschek commented on OAK-7832: Fixed the license header (my IDE should have a smart copyright profile detection :D). I could commit myself. This change was done against the 1.8 branch, but it patches without changes against trunk. Given many production users (for which oak-run is useful) are on 1.8, should I commit to both? > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, store-spi >Reporter: Alexander Klimetschek >Priority: Major > Attachments: OAK-7832.patch, oak-run-1.8.9-SNAPSHOT.jar > > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16654436#comment-16654436 ] Alexander Klimetschek edited comment on OAK-7832 at 10/18/18 12:27 AM: --- [~chetanm] Fixed the license header (my IDE should have a smart copyright profile detection :D). I could commit myself. This change was done against the 1.8 branch, but it patches without changes against trunk. Given many production users (for which oak-run is useful) are on 1.8, *should I commit to both*? was (Author: alexander.klimetschek): Fixed the license header (my IDE should have a smart copyright profile detection :D). I could commit myself. This change was done against the 1.8 branch, but it patches without changes against trunk. Given many production users (for which oak-run is useful) are on 1.8, should I commit to both? > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, store-spi >Reporter: Alexander Klimetschek >Priority: Major > Attachments: OAK-7832.patch, oak-run-1.8.9-SNAPSHOT.jar > > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-7832: --- Attachment: OAK-7832.patch > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, store-spi >Reporter: Alexander Klimetschek >Priority: Major > Attachments: OAK-7832.patch, oak-run-1.8.9-SNAPSHOT.jar > > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-7832: --- Attachment: (was: OAK-7832.patch) > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, store-spi >Reporter: Alexander Klimetschek >Priority: Major > Attachments: OAK-7832.patch, oak-run-1.8.9-SNAPSHOT.jar > > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[
https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650789#comment-16650789
]
Alexander Klimetschek edited comment on OAK-7832 at 10/15/18 9:04 PM:
--
[~chetanm] Could you maybe take a look at the patch?
Some notes:
* for the {{pn}} command I copied the toString code from NodeState et al into
the new NodeStateHelper.java inside oak-run
* for the {{export}} command I had to touch the JsonSerializer of
oak-store-spi which I see is also used in other cases (possibly at runtime, not
just oak-run utility), so I added a new flag "checkExceptions" which is off by
default to trigger the new behavior. By default the class will behave as before
and throw any exception (like SNFE).
* A related improvement for the {{export}} would be to create one json file
per page/asset/folder /jcr:content etc., especially if one is exporting larger
trees. A single nodestate.json is hard to "browse" as a human. That's probably
something for another issue I guess.
was (Author: alexander.klimetschek):
[~chetanm] Could you maybe take a look at the patch?
Some notes:
* for the {{pn}} command I copied the toString code from NodeState et al into
the new NodeStateHelper.java inside oak-run
* for the {{export}} command I had to touch the JsonSerializer of
oak-store-spi which I see is also used in other cases (possibly at runtime, not
just oak-run utility), so I added a new flag "checkExceptions" which is off by
default to trigger the new behavior. By default the class will behave as before
and throw any exception (like SNFE).
> oak-run console export should handle exceptions such as missing segments
>
>
> Key: OAK-7832
> URL: https://issues.apache.org/jira/browse/OAK-7832
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run, store-spi
>Reporter: Alexander Klimetschek
>Priority: Major
> Attachments: OAK-7832.patch, oak-run-1.8.9-SNAPSHOT.jar
>
>
> Problem: When trying to look at data using "pn" or running "export" oak-run
> console currently will choke and abort the traversal on exceptions such as
> SegmentNotFoundException.
> Expected: It would be nice if it would be best-effort, log the issue or embed
> in the exported json, and just continue. This also ensures all missing
> segments for a subtree are listed with one command (if it fails right now you
> just see the first missing segment).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[
https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650789#comment-16650789
]
Alexander Klimetschek commented on OAK-7832:
[~chetanm] Could you maybe take a look at the patch?
Some notes:
* for the {{pn}} command I copied the toString code from NodeState et al into
the new NodeStateHelper.java inside oak-run
* for the {{export}} command I had to touch the JsonSerializer of
oak-store-spi which I see is also used in other cases (possibly at runtime, not
just oak-run utility), so I added a new flag "checkExceptions" which is off by
default to trigger the new behavior. By default the class will behave as before
and throw any exception (like SNFE).
> oak-run console export should handle exceptions such as missing segments
>
>
> Key: OAK-7832
> URL: https://issues.apache.org/jira/browse/OAK-7832
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run, store-spi
>Reporter: Alexander Klimetschek
>Priority: Major
> Attachments: OAK-7832.patch, oak-run-1.8.9-SNAPSHOT.jar
>
>
> Problem: When trying to look at data using "pn" or running "export" oak-run
> console currently will choke and abort the traversal on exceptions such as
> SegmentNotFoundException.
> Expected: It would be nice if it would be best-effort, log the issue or embed
> in the exported json, and just continue. This also ensures all missing
> segments for a subtree are listed with one command (if it fails right now you
> just see the first missing segment).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek reassigned OAK-7832: -- Assignee: (was: Alexander Klimetschek) > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, store-spi >Reporter: Alexander Klimetschek >Priority: Major > Attachments: OAK-7832.patch, oak-run-1.8.9-SNAPSHOT.jar > > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[
https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649199#comment-16649199
]
Alexander Klimetschek commented on OAK-7832:
Attached is a [^OAK-7832.patch] against the oak 1.8 branch that. Also attached
a ready to use jar with the changes included: [^oak-run-1.8.9-SNAPSHOT.jar].
1. Changed the console {{export}} command to log exceptions and include them in
the json, but then continue and produce a valid json.
Log output:
{noformat}
/content/some/folder> export
01:02:01.589 [main] ERROR o.a.j.oak.json.JsonSerializer - Cannot read property
value /content/some/folder/mypage/jcr:content/cq:lastModifiedBy :
org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
dec74dbe-d24f-4f3d-a7ef-75374bca3572 not found
01:02:01.596 [main] ERROR o.a.j.oak.json.JsonSerializer - Cannot read node
/content/some/folder/mypage/jcr:content/content/paragraph/par/component0 :
org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
e8c16c25-cc3c-4874-ab0b-f515d6523c24 not found
{noformat}
nodestates.json output:
{noformat}
...
"cq:cssClass": "ERROR: Cannot read property value
/content/some/folder/mypage/jcr:content/content/paragraph/par/component/cq:cssClass
: org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
e8c16c25-cc3c-4874-ab0b-f515d6523c24 not found"
...
"component0": {
"_error": "ERROR: Cannot read node
/content/some/folder/mypage/jcr:content/content/paragraph/par/component0 :
org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
e8c16c25-cc3c-4874-ab0b-f515d6523c24 not found"
},
{noformat}
2. Also maked the {{pn}} command handle exceptions.
{noformat}
/content/some/folder> pn
{ jcr:primaryType = cq:Page, jcr:createdBy = silcottl, jcr:created =
2018-09-20T15:05:43.264Z, :childOrder = [jcr:content, one, two, three, four,
give], one = { ... }, two = { ... }, three = { ... }, jcr:content = { ... },
four = { ERROR on node:
org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
a9d9a87e-e452-452f-a3d3-d89228bef4d6 not found }, five = { ... } }
{noformat}
In both cases it catches on property value access and child node access, which
at least with SegmentTar are the places where you'd get a
SegmentNotFoundException if the segment storing that information is missing.
For handling the getChildNodeEntries() case in the JsonSerializer I had to to a
bit of a trick with a MemoryChildNodeEntry to get a nice json output with as
much information as possible, without rewriting more of the JsonSerializer.
> oak-run console export should handle exceptions such as missing segments
>
>
> Key: OAK-7832
> URL: https://issues.apache.org/jira/browse/OAK-7832
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run, store-spi
>Reporter: Alexander Klimetschek
>Assignee: Alexander Klimetschek
>Priority: Major
> Attachments: OAK-7832.patch, oak-run-1.8.9-SNAPSHOT.jar
>
>
> Problem: When trying to look at data using "pn" or running "export" oak-run
> console currently will choke and abort the traversal on exceptions such as
> SegmentNotFoundException.
> Expected: It would be nice if it would be best-effort, log the issue or embed
> in the exported json, and just continue. This also ensures all missing
> segments for a subtree are listed with one command (if it fails right now you
> just see the first missing segment).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-7832: --- Attachment: oak-run-1.8.9-SNAPSHOT.jar > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, store-spi >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > Attachments: OAK-7832.patch, oak-run-1.8.9-SNAPSHOT.jar > > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-7832: --- Attachment: OAK-7832.patch > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, store-spi >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > Attachments: OAK-7832.patch > > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-7832: --- Attachment: (was: OAK-7832.patch) > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, store-spi >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > Attachments: OAK-7832.patch > > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-7832: --- Attachment: (was: oak-run-1.8.9-SNAPSHOT.jar) > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, store-spi >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > Attachments: OAK-7832.patch > > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-7832: --- Attachment: oak-run-1.8.9-SNAPSHOT.jar > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, store-spi >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > Attachments: OAK-7832.patch, oak-run-1.8.9-SNAPSHOT.jar > > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-7832: --- Attachment: OAK-7832.patch > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, store-spi >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > Attachments: OAK-7832.patch > > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-7832: --- Component/s: store-spi > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, store-spi >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OAK-7832) oak-run console export should handle exceptions such as missing segments
[ https://issues.apache.org/jira/browse/OAK-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek reassigned OAK-7832: -- Assignee: Alexander Klimetschek > oak-run console export should handle exceptions such as missing segments > > > Key: OAK-7832 > URL: https://issues.apache.org/jira/browse/OAK-7832 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > > Problem: When trying to look at data using "pn" or running "export" oak-run > console currently will choke and abort the traversal on exceptions such as > SegmentNotFoundException. > Expected: It would be nice if it would be best-effort, log the issue or embed > in the exported json, and just continue. This also ensures all missing > segments for a subtree are listed with one command (if it fails right now you > just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OAK-7832) oak-run console export should handle exceptions such as missing segments
Alexander Klimetschek created OAK-7832: -- Summary: oak-run console export should handle exceptions such as missing segments Key: OAK-7832 URL: https://issues.apache.org/jira/browse/OAK-7832 Project: Jackrabbit Oak Issue Type: Improvement Components: run Reporter: Alexander Klimetschek Problem: When trying to look at data using "pn" or running "export" oak-run console currently will choke and abort the traversal on exceptions such as SegmentNotFoundException. Expected: It would be nice if it would be best-effort, log the issue or embed in the exported json, and just continue. This also ensures all missing segments for a subtree are listed with one command (if it fails right now you just see the first missing segment). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16588529#comment-16588529 ] Alexander Klimetschek edited comment on OAK-7569 at 8/22/18 8:17 AM: - [~mreutegg] The javadoc patch is ready in JCR-4355, it could be committed (JCR-4355-v3.patch) and a new jackrabbit-api released. was (Author: alexander.klimetschek): [~mreutegg] The javadoc patch is ready in JCR-4355, it could be committed and a new jackrabbit-api released. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16588529#comment-16588529 ] Alexander Klimetschek edited comment on OAK-7569 at 8/22/18 8:17 AM: - [~mreutegg] The javadoc patch is ready in JCR-4355, it could be committed (JCR-4355-v3.patch) and a new jackrabbit-api released. was (Author: alexander.klimetschek): [~mreutegg] The javadoc patch is ready in JCR-4355, it could be committed (JCR-4355-v3.patch) and a new jackrabbit-api released. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16588529#comment-16588529 ] Alexander Klimetschek commented on OAK-7569: [~mreutegg] The javadoc patch is ready in JCR-4355, it could be committed and a new jackrabbit-api released. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7693) [DirectBinaryAccess] Documentation
[ https://issues.apache.org/jira/browse/OAK-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576908#comment-16576908 ] Alexander Klimetschek commented on OAK-7693: [~amitjain] Common properties: yes, that would be good. But maybe [~mattvryan] had a good reason for them to be different. However, let's look at this later. We also have OAK-7702 to tackle where we might add/change config properties. [~mreutegg] Here is the final doc change in case you want to commit it with OAK-7569: https://github.com/alexkli/jackrabbit-oak/commit/d633b51c34bcc77865533cd769b52f9682f69895 (including binary files) Otherwise let me know when things are in trunk and I'll commit it. When should we publish to the documentation site? As soon as the feature is in trunk or only when it's in an official oak release? (The doc says it's since 1.10, so it should be clear) > [DirectBinaryAccess] Documentation > -- > > Key: OAK-7693 > URL: https://issues.apache.org/jira/browse/OAK-7693 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: doc >Reporter: Marcel Reutegger >Assignee: Alexander Klimetschek >Priority: Minor > Fix For: 1.10 > > > Add official Oak documentation for the direct binary access feature. > There's a [wiki > page|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access] that > already describes the feature, but official documentation should be created > in the oak-doc module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OAK-7701) Documentation should link to Jackrabbit API
[ https://issues.apache.org/jira/browse/OAK-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek resolved OAK-7701. Resolution: Fixed Change * committed [http://svn.apache.org/viewvc?view=revision=1837836] * and published to [https://jackrabbit.apache.org/oak/docs/index.html] > Documentation should link to Jackrabbit API > --- > > Key: OAK-7701 > URL: https://issues.apache.org/jira/browse/OAK-7701 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: doc >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > Attachments: OAK-7701.patch > > > The [Oak documentation|https://jackrabbit.apache.org/oak/docs/] currently > links to the "JCR API" and the (internal) "Oak API" in the navigation on the > left side, but it is missing the "Jackrabbit API" (JCR api extensions) that > are very important for developers using Oak. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7693) [DirectBinaryAccess] Documentation
[ https://issues.apache.org/jira/browse/OAK-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575289#comment-16575289 ] Alexander Klimetschek edited comment on OAK-7693 at 8/9/18 7:23 PM: I updated the docs: [preview|https://github.com/alexkli/jackrabbit-oak/blob/direct-binary-access-v2-with-partial-value-factory/oak-doc/src/site/markdown/features/direct-binary-access.md] * moved underneath "Blob Storage" in the navigation * adding a new high level diagram at the top * added important info on the transfer acceleration config * links to S3, Azure BS, the oak direct binary access wiki page * some editorial improvements The questions above are still open. BTW, it's probably easiest for me to commit this once approved since it involves the diagram images that aren't included in patch files. was (Author: alexander.klimetschek): I updated the docs: [preview|https://github.com/alexkli/jackrabbit-oak/blob/direct-binary-access-v2-with-partial-value-factory/oak-doc/src/site/markdown/features/direct-binary-access.md] * moved underneath "Blob Storage" in the navigation * adding a new high level diagram at the top * added important info on the transfer acceleration config * links to S3, Azure BS, the oak direct binary access wiki page * some editorial improvements The questions above are still open. > [DirectBinaryAccess] Documentation > -- > > Key: OAK-7693 > URL: https://issues.apache.org/jira/browse/OAK-7693 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: doc >Reporter: Marcel Reutegger >Assignee: Alexander Klimetschek >Priority: Minor > Fix For: 1.10 > > > Add official Oak documentation for the direct binary access feature. > There's a [wiki > page|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access] that > already describes the feature, but official documentation should be created > in the oak-doc module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7693) [DirectBinaryAccess] Documentation
[ https://issues.apache.org/jira/browse/OAK-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575289#comment-16575289 ] Alexander Klimetschek commented on OAK-7693: I updated the docs: [preview|https://github.com/alexkli/jackrabbit-oak/blob/direct-binary-access-v2-with-partial-value-factory/oak-doc/src/site/markdown/features/direct-binary-access.md] * moved underneath "Blob Storage" in the navigation * adding a new high level diagram at the top * added important info on the transfer acceleration config * links to S3, Azure BS, the oak direct binary access wiki page * some editorial improvements The questions above are still open. > [DirectBinaryAccess] Documentation > -- > > Key: OAK-7693 > URL: https://issues.apache.org/jira/browse/OAK-7693 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: doc >Reporter: Marcel Reutegger >Assignee: Alexander Klimetschek >Priority: Minor > Fix For: 1.10 > > > Add official Oak documentation for the direct binary access feature. > There's a [wiki > page|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access] that > already describes the feature, but official documentation should be created > in the oak-doc module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7693) [DirectBinaryAccess] Documentation
[ https://issues.apache.org/jira/browse/OAK-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575277#comment-16575277 ] Alexander Klimetschek edited comment on OAK-7693 at 8/9/18 7:10 PM: [~amitjain] Are there any documentations for the S3 and Azure DataStores (i.e. regarding their configuration etc.)? I would like to link to them in the doc and possibly update them with the new configuration options. The [BlobStore page|https://jackrabbit.apache.org/oak/docs/plugins/blobstore.html] doesn't explain them in detail nor mentions their configurations. The best I can find is the [Adobe AEM documentation for Oak|https://helpx.adobe.com/experience-manager/6-4/sites/deploying/using/data-store-config.html], but I don't think we should link to that from the Oak documentation. was (Author: alexander.klimetschek): [~amitjain] Are there any documentations for the S3 and Azure DataStores (i.e. regarding their configuration etc.)? I would like to link to them in the doc and possibly update them with the new configuration options. The best I can find is the [Adobe AEM documentation for Oak|https://helpx.adobe.com/experience-manager/6-4/sites/deploying/using/data-store-config.html], but I don't think we should link to that from the Oak documentation. > [DirectBinaryAccess] Documentation > -- > > Key: OAK-7693 > URL: https://issues.apache.org/jira/browse/OAK-7693 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: doc >Reporter: Marcel Reutegger >Assignee: Alexander Klimetschek >Priority: Minor > Fix For: 1.10 > > > Add official Oak documentation for the direct binary access feature. > There's a [wiki > page|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access] that > already describes the feature, but official documentation should be created > in the oak-doc module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7693) [DirectBinaryAccess] Documentation
[ https://issues.apache.org/jira/browse/OAK-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575277#comment-16575277 ] Alexander Klimetschek commented on OAK-7693: [~amitjain] Are there any documentations for the S3 and Azure DataStores (i.e. regarding their configuration etc.)? I would like to link to them in the doc and possibly update them with the new configuration options. The best I can find is the [Adobe AEM documentation for Oak|https://helpx.adobe.com/experience-manager/6-4/sites/deploying/using/data-store-config.html], but I don't think we should link to that from the Oak documentation. > [DirectBinaryAccess] Documentation > -- > > Key: OAK-7693 > URL: https://issues.apache.org/jira/browse/OAK-7693 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: doc >Reporter: Marcel Reutegger >Assignee: Alexander Klimetschek >Priority: Minor > Fix For: 1.10 > > > Add official Oak documentation for the direct binary access feature. > There's a [wiki > page|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access] that > already describes the feature, but official documentation should be created > in the oak-doc module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7693) [DirectBinaryAccess] Documentation
[
https://issues.apache.org/jira/browse/OAK-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575261#comment-16575261
]
Alexander Klimetschek edited comment on OAK-7693 at 8/9/18 6:42 PM:
1.
[~mreutegg] The doc currently includes this statement:
{quote}Please note that only Binary objects returned from Property.getBinary(),
Property.getValue().getBinary() or Property.getValues() ... getBinary() will
support a functional BinaryDownload.
{quote}
Over in OAK-7569 you mentioned
{quote}Direct binary access is now also working for authorizable properties and
values obtained through events.
{quote}
Should I update the above statement then? If yes, what is a good way to
describe the different cases, list all of them?
2.
Will Oak 1.10 be the "official" release this feature is expected to be in? The
doc states {{@since Oak 1.10}} and I wasn't sure if that can mention the
unstable releases, e.g. 1.9.x currently.
was (Author: alexander.klimetschek):
1.
[~mreutegg] The doc currently includes this statement:
bq. Please note that only Binary objects returned from Property.getBinary(),
Property.getValue().getBinary() or Property.getValues() ... getBinary() will
support a functional BinaryDownload.
Over in OAK-7569 you mentioned
bq. Direct binary access is now also working for authorizable properties and
values obtained through events.
Should I update the above statement then? If yes, what is a good way to
describe the different cases, list all of them?
2.
Will Oak 1.10 be the "official" release this feature is expected to be in? The
doc states {{@since Oak 1.10}} and I wasn't sure if that can mention the
unstable releases, e.g. 1.9.x currently.
> [DirectBinaryAccess] Documentation
> --
>
> Key: OAK-7693
> URL: https://issues.apache.org/jira/browse/OAK-7693
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: doc
>Reporter: Marcel Reutegger
>Assignee: Alexander Klimetschek
>Priority: Minor
> Fix For: 1.10
>
>
> Add official Oak documentation for the direct binary access feature.
> There's a [wiki
> page|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access] that
> already describes the feature, but official documentation should be created
> in the oak-doc module.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7693) [DirectBinaryAccess] Documentation
[
https://issues.apache.org/jira/browse/OAK-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575261#comment-16575261
]
Alexander Klimetschek commented on OAK-7693:
1.
[~mreutegg] The doc currently includes this statement:
bq. Please note that only Binary objects returned from Property.getBinary(),
Property.getValue().getBinary() or Property.getValues() ... getBinary() will
support a functional BinaryDownload.
Over in OAK-7569 you mentioned
bq. Direct binary access is now also working for authorizable properties and
values obtained through events.
Should I update the above statement then? If yes, what is a good way to
describe the different cases, list all of them?
2.
Will Oak 1.10 be the "official" release this feature is expected to be in? The
doc states {{@since Oak 1.10}} and I wasn't sure if that can mention the
unstable releases, e.g. 1.9.x currently.
> [DirectBinaryAccess] Documentation
> --
>
> Key: OAK-7693
> URL: https://issues.apache.org/jira/browse/OAK-7693
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: doc
>Reporter: Marcel Reutegger
>Assignee: Alexander Klimetschek
>Priority: Minor
> Fix For: 1.10
>
>
> Add official Oak documentation for the direct binary access feature.
> There's a [wiki
> page|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access] that
> already describes the feature, but official documentation should be created
> in the oak-doc module.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7692) [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
[
https://issues.apache.org/jira/browse/OAK-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575257#comment-16575257
]
Alexander Klimetschek commented on OAK-7692:
Fixed the license header and attached the updated patch: [^OAK-7692.patch]
> [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
> ---
>
> Key: OAK-7692
> URL: https://issues.apache.org/jira/browse/OAK-7692
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: blob-plugins
>Reporter: Alexander Klimetschek
>Assignee: Alexander Klimetschek
>Priority: Major
> Fix For: 1.10
>
> Attachments: OAK-7692.patch
>
>
> The upload token's hmac signature (after the #) is not base64 encoded. This
> might create problems for clients passing that string around if it can
> contain non-ascii characters.
> Example:
> {noformat}
> ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
> {noformat}
> Code is
> [here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148].
> Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7692) [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
[
https://issues.apache.org/jira/browse/OAK-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Klimetschek updated OAK-7692:
---
Attachment: OAK-7692.patch
> [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
> ---
>
> Key: OAK-7692
> URL: https://issues.apache.org/jira/browse/OAK-7692
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: blob-plugins
>Reporter: Alexander Klimetschek
>Assignee: Alexander Klimetschek
>Priority: Major
> Fix For: 1.10
>
> Attachments: OAK-7692.patch
>
>
> The upload token's hmac signature (after the #) is not base64 encoded. This
> might create problems for clients passing that string around if it can
> contain non-ascii characters.
> Example:
> {noformat}
> ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
> {noformat}
> Code is
> [here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148].
> Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (OAK-7692) [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
[
https://issues.apache.org/jira/browse/OAK-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574030#comment-16574030
]
Alexander Klimetschek edited comment on OAK-7692 at 8/9/18 6:31 AM:
[~mreutegg] Fix including unit test available
[here|https://github.com/alexkli/jackrabbit-oak/commit/cdc945d89eb74d0be1163bbfb70d8250a0625e83],
and as [patch
file|https://github.com/alexkli/jackrabbit-oak/commit/cdc945d89eb74d0be1163bbfb70d8250a0625e83.diff].
Note that I also made all the exception messages for an invalid token the same
("Invalid upload token") so that possibly attacking clients don't get too much
information.
was (Author: alexander.klimetschek):
[~mreutegg] Fix including unit test available
[here|https://github.com/mattvryan/jackrabbit-oak/pull/14], and as [patch
file|https://patch-diff.githubusercontent.com/raw/mattvryan/jackrabbit-oak/pull/14.diff].
Note that I also made all the exception messages for an invalid token the same
("Invalid upload token") so that possibly attacking clients don't get too much
information.
> [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
> ---
>
> Key: OAK-7692
> URL: https://issues.apache.org/jira/browse/OAK-7692
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: blob-plugins
>Reporter: Alexander Klimetschek
>Assignee: Alexander Klimetschek
>Priority: Major
> Fix For: 1.10
>
>
> The upload token's hmac signature (after the #) is not base64 encoded. This
> might create problems for clients passing that string around if it can
> contain non-ascii characters.
> Example:
> {noformat}
> ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
> {noformat}
> Code is
> [here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148].
> Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OAK-7695) [DirectBinaryAccess] Fix and improve javadocs of new API
[ https://issues.apache.org/jira/browse/OAK-7695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek resolved OAK-7695. Resolution: Duplicate Created JCR-4355 > [DirectBinaryAccess] Fix and improve javadocs of new API > > > Key: OAK-7695 > URL: https://issues.apache.org/jira/browse/OAK-7695 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: api >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > > Here are some changes to the javadocs for the new API: > [OAK-7569-api-javadoc-improvements.patch|https://issues.apache.org/jira/secure/attachment/12934364/12934364_OAK-7569-api-javadoc-improvements.patch] > * more concise descriptions > * correcting some inaccuracies (clients cannot choose whether to do single or > multipart upload, multipart might be strictly required depending on the size) > * most importantly the upload algorithm (standard partSize calculation was > wrong) > * focus on API users, separated notes to implementors > * for BinaryDownloadOptions added note from which jcr properties a client > would normally take these values from > * added security considerations -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7693) [DirectBinaryAccess] Documentation
[ https://issues.apache.org/jira/browse/OAK-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574342#comment-16574342 ] Alexander Klimetschek commented on OAK-7693: [~mreutegg] Documentation is available [here|https://github.com/alexkli/jackrabbit-oak/commit/0d27d31f606347c22a48d9856f3546e62f3dd244], and as [patch file|https://github.com/alexkli/jackrabbit-oak/commit/0d27d31f606347c22a48d9856f3546e62f3dd244.diff]. This adds a new "Direct Binary Access" page to the docs, see a [preview here|https://github.com/alexkli/jackrabbit-oak/blob/direct-binary-access-v2-with-partial-value-factory/oak-doc/src/site/markdown/features/direct-binary-access.md]. > [DirectBinaryAccess] Documentation > -- > > Key: OAK-7693 > URL: https://issues.apache.org/jira/browse/OAK-7693 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: doc >Reporter: Marcel Reutegger >Assignee: Alexander Klimetschek >Priority: Minor > Fix For: 1.10 > > > Add official Oak documentation for the direct binary access feature. > There's a [wiki > page|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access] that > already describes the feature, but official documentation should be created > in the oak-doc module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OAK-7693) [DirectBinaryAccess] Documentation
[ https://issues.apache.org/jira/browse/OAK-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek reassigned OAK-7693: -- Assignee: Alexander Klimetschek > [DirectBinaryAccess] Documentation > -- > > Key: OAK-7693 > URL: https://issues.apache.org/jira/browse/OAK-7693 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: doc >Reporter: Marcel Reutegger >Assignee: Alexander Klimetschek >Priority: Minor > Fix For: 1.10 > > > Add official Oak documentation for the direct binary access feature. > There's a [wiki > page|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access] that > already describes the feature, but official documentation should be created > in the oak-doc module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OAK-7702) [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for uploads
Alexander Klimetschek created OAK-7702: -- Summary: [Direct Binary Access] Add CDN support, use S3 Transfer Acceleration only for uploads Key: OAK-7702 URL: https://issues.apache.org/jira/browse/OAK-7702 Project: Jackrabbit Oak Issue Type: Improvement Components: blob-cloud, blob-cloud-azure Reporter: Alexander Klimetschek Azure Blob Store and S3 support CDNs in front of private containers/buckets, which also work with presigned URLs ([S3 docs|https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-urls.html], [Azure docs|https://docs.microsoft.com/en-us/azure/cdn/cdn-sas-storage-support]). The binary access feature should support these for download URLs via a configuration on the DataStore. Currently, the S3 datastore has a config for [transfer acceleration|https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html] that if enabled, will make both upload & download URLs use the acceleration option. However, this feature only makes sense for uploads, especially if CDN is an option for downloads. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7701) Documentation should link to Jackrabbit API
[ https://issues.apache.org/jira/browse/OAK-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574039#comment-16574039 ] Alexander Klimetschek commented on OAK-7701: This patch: [^OAK-7701.patch] adds a link to [https://jackrabbit.apache.org/jcr/jcr-api.html] with label "Jackrabbit API" underneath the "JCR API" link. > Documentation should link to Jackrabbit API > --- > > Key: OAK-7701 > URL: https://issues.apache.org/jira/browse/OAK-7701 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: doc >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > Attachments: OAK-7701.patch > > > The [Oak documentation|https://jackrabbit.apache.org/oak/docs/] currently > links to the "JCR API" and the (internal) "Oak API" in the navigation on the > left side, but it is missing the "Jackrabbit API" (JCR api extensions) that > are very important for developers using Oak. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7701) Documentation should link to Jackrabbit API
[ https://issues.apache.org/jira/browse/OAK-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-7701: --- Attachment: OAK-7701.patch > Documentation should link to Jackrabbit API > --- > > Key: OAK-7701 > URL: https://issues.apache.org/jira/browse/OAK-7701 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: doc >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > Attachments: OAK-7701.patch > > > The [Oak documentation|https://jackrabbit.apache.org/oak/docs/] currently > links to the "JCR API" and the (internal) "Oak API" in the navigation on the > left side, but it is missing the "Jackrabbit API" (JCR api extensions) that > are very important for developers using Oak. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OAK-7701) Documentation should link to Jackrabbit API
[ https://issues.apache.org/jira/browse/OAK-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek reassigned OAK-7701: -- Assignee: Alexander Klimetschek > Documentation should link to Jackrabbit API > --- > > Key: OAK-7701 > URL: https://issues.apache.org/jira/browse/OAK-7701 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: doc >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > Attachments: OAK-7701.patch > > > The [Oak documentation|https://jackrabbit.apache.org/oak/docs/] currently > links to the "JCR API" and the (internal) "Oak API" in the navigation on the > left side, but it is missing the "Jackrabbit API" (JCR api extensions) that > are very important for developers using Oak. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OAK-7701) Documentation should link to Jackrabbit API
Alexander Klimetschek created OAK-7701: -- Summary: Documentation should link to Jackrabbit API Key: OAK-7701 URL: https://issues.apache.org/jira/browse/OAK-7701 Project: Jackrabbit Oak Issue Type: Improvement Components: doc Reporter: Alexander Klimetschek The [Oak documentation|https://jackrabbit.apache.org/oak/docs/] currently links to the "JCR API" and the (internal) "Oak API" in the navigation on the left side, but it is missing the "Jackrabbit API" (JCR api extensions) that are very important for developers using Oak. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7692) [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
[
https://issues.apache.org/jira/browse/OAK-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574030#comment-16574030
]
Alexander Klimetschek edited comment on OAK-7692 at 8/8/18 11:33 PM:
-
[~mreutegg] Fix including unit test available
[here|https://github.com/mattvryan/jackrabbit-oak/pull/14], and as [patch
file|https://patch-diff.githubusercontent.com/raw/mattvryan/jackrabbit-oak/pull/14.diff].
Note that I also made all the exception messages for an invalid token the same
("Invalid upload token") so that possibly attacking clients don't get too much
information.
was (Author: alexander.klimetschek):
[~mreutegg] Fix including unit test available
[here|https://github.com/mattvryan/jackrabbit-oak/pull/14], and as [patch
file|https://patch-diff.githubusercontent.com/raw/mattvryan/jackrabbit-oak/pull/14.diff].
> [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
> ---
>
> Key: OAK-7692
> URL: https://issues.apache.org/jira/browse/OAK-7692
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: blob-plugins
>Reporter: Alexander Klimetschek
>Assignee: Alexander Klimetschek
>Priority: Major
>
> The upload token's hmac signature (after the #) is not base64 encoded. This
> might create problems for clients passing that string around if it can
> contain non-ascii characters.
> Example:
> {noformat}
> ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
> {noformat}
> Code is
> [here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148].
> Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OAK-7692) [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
[
https://issues.apache.org/jira/browse/OAK-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Klimetschek resolved OAK-7692.
Resolution: Fixed
> [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
> ---
>
> Key: OAK-7692
> URL: https://issues.apache.org/jira/browse/OAK-7692
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: blob-plugins
>Reporter: Alexander Klimetschek
>Assignee: Alexander Klimetschek
>Priority: Major
>
> The upload token's hmac signature (after the #) is not base64 encoded. This
> might create problems for clients passing that string around if it can
> contain non-ascii characters.
> Example:
> {noformat}
> ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
> {noformat}
> Code is
> [here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148].
> Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7692) [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
[
https://issues.apache.org/jira/browse/OAK-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574030#comment-16574030
]
Alexander Klimetschek commented on OAK-7692:
[~mreutegg] Fix including unit test available
[here|https://github.com/mattvryan/jackrabbit-oak/pull/14], and as [patch
file|https://patch-diff.githubusercontent.com/raw/mattvryan/jackrabbit-oak/pull/14.diff].
> [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
> ---
>
> Key: OAK-7692
> URL: https://issues.apache.org/jira/browse/OAK-7692
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: blob-plugins
>Reporter: Alexander Klimetschek
>Assignee: Alexander Klimetschek
>Priority: Major
>
> The upload token's hmac signature (after the #) is not base64 encoded. This
> might create problems for clients passing that string around if it can
> contain non-ascii characters.
> Example:
> {noformat}
> ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
> {noformat}
> Code is
> [here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148].
> Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (OAK-7692) [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
[
https://issues.apache.org/jira/browse/OAK-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Klimetschek reassigned OAK-7692:
--
Assignee: Alexander Klimetschek
> [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
> ---
>
> Key: OAK-7692
> URL: https://issues.apache.org/jira/browse/OAK-7692
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: blob-plugins
>Reporter: Alexander Klimetschek
>Assignee: Alexander Klimetschek
>Priority: Major
>
> The upload token's hmac signature (after the #) is not base64 encoded. This
> might create problems for clients passing that string around if it can
> contain non-ascii characters.
> Example:
> {noformat}
> ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
> {noformat}
> Code is
> [here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148].
> Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7695) [DirectBinaryAccess] Fix and improve javadocs of new API
[ https://issues.apache.org/jira/browse/OAK-7695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-7695: --- Description: Here are some changes to the javadocs for the new API: [OAK-7569-api-javadoc-improvements.patch|https://issues.apache.org/jira/secure/attachment/12934364/12934364_OAK-7569-api-javadoc-improvements.patch] * more concise descriptions * correcting some inaccuracies (clients cannot choose whether to do single or multipart upload, multipart might be strictly required depending on the size) * most importantly the upload algorithm (standard partSize calculation was wrong) * focus on API users, separated notes to implementors * for BinaryDownloadOptions added note from which jcr properties a client would normally take these values from * added security considerations was: Here are some changes to the javadocs for the new API: [^OAK-7569-api-javadoc-improvements.patch] * more concise descriptions * correcting some inaccuracies (clients cannot choose whether to do single or multipart upload, multipart might be strictly required depending on the size) * most importantly the upload algorithm (standard partSize calculation was wrong) * focus on API users, separated notes to implementors * for BinaryDownloadOptions added note from which jcr properties a client would normally take these values from * added security considerations > [DirectBinaryAccess] Fix and improve javadocs of new API > > > Key: OAK-7695 > URL: https://issues.apache.org/jira/browse/OAK-7695 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: api >Reporter: Alexander Klimetschek >Assignee: Alexander Klimetschek >Priority: Major > > Here are some changes to the javadocs for the new API: > [OAK-7569-api-javadoc-improvements.patch|https://issues.apache.org/jira/secure/attachment/12934364/12934364_OAK-7569-api-javadoc-improvements.patch] > * more concise descriptions > * correcting some inaccuracies (clients cannot choose whether to do single or > multipart upload, multipart might be strictly required depending on the size) > * most importantly the upload algorithm (standard partSize calculation was > wrong) > * focus on API users, separated notes to implementors > * for BinaryDownloadOptions added note from which jcr properties a client > would normally take these values from > * added security considerations -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OAK-7695) [DirectBinaryAccess] Fix and improve javadocs of new API
Alexander Klimetschek created OAK-7695: -- Summary: [DirectBinaryAccess] Fix and improve javadocs of new API Key: OAK-7695 URL: https://issues.apache.org/jira/browse/OAK-7695 Project: Jackrabbit Oak Issue Type: Technical task Components: api Reporter: Alexander Klimetschek Assignee: Alexander Klimetschek Here are some changes to the javadocs for the new API: [^OAK-7569-api-javadoc-improvements.patch] * more concise descriptions * correcting some inaccuracies (clients cannot choose whether to do single or multipart upload, multipart might be strictly required depending on the size) * most importantly the upload algorithm (standard partSize calculation was wrong) * focus on API users, separated notes to implementors * for BinaryDownloadOptions added note from which jcr properties a client would normally take these values from * added security considerations -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573451#comment-16573451 ] Alexander Klimetschek commented on OAK-7569: AuthorizablePropertiesImpl: Good to be confirmed. When I followed the call hierachy, there were many places that used DEFAULT (IIRC for the UserManagerImpl constructor) and it wasn‘t clear which ones are actually used at normal runtime (and I skipped any test usages). Limitation/documentation: fair enough. So the places that we support are getProperty().[getValue().]getBinary(), as well as the equivalent multivalue methods (if anyone really uses multivalue binary properties ;-). Any other „normal“ Binary read case that I missed? One thought: if we don‘t have a BlobAccessProvider available, could we return a Binary that does not implement BinaryDownload? Then it would be clear to developers they used an unsupported access path. We need documentation for the oak docs anyway, not sure if there is anything yet. Happy to add something. Where would I add this in the navigation tree? > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7692) [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
[
https://issues.apache.org/jira/browse/OAK-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Klimetschek updated OAK-7692:
---
Description:
The upload token's hmac signature (after the #) is not base64 encoded. This
might create problems for clients passing that string around if it can contain
non-ascii characters.
Example:
{noformat}
ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
{noformat}
Code is
[here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148]
Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac.
was:
The upload token's hmac signature (after the #) is not base64 encoded. This
might create problems for clients passing that string around if it can contain
non-ascii characters.
Example:
{noformat}
ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
{noformat}
Code is
[here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148]
Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac for
a string that can be safely passed around.
> [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
> ---
>
> Key: OAK-7692
> URL: https://issues.apache.org/jira/browse/OAK-7692
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: blob-plugins
>Reporter: Alexander Klimetschek
>Priority: Major
>
> The upload token's hmac signature (after the #) is not base64 encoded. This
> might create problems for clients passing that string around if it can
> contain non-ascii characters.
> Example:
> {noformat}
> ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
> {noformat}
> Code is
> [here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148]
> Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7692) [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
[
https://issues.apache.org/jira/browse/OAK-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Klimetschek updated OAK-7692:
---
Description:
The upload token's hmac signature (after the #) is not base64 encoded. This
might create problems for clients passing that string around if it can contain
non-ascii characters.
Example:
{noformat}
ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
{noformat}
Code is
[here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148].
Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac.
was:
The upload token's hmac signature (after the #) is not base64 encoded. This
might create problems for clients passing that string around if it can contain
non-ascii characters.
Example:
{noformat}
ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
{noformat}
Code is
[here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148]
Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac.
> [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
> ---
>
> Key: OAK-7692
> URL: https://issues.apache.org/jira/browse/OAK-7692
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: blob-plugins
>Reporter: Alexander Klimetschek
>Priority: Major
>
> The upload token's hmac signature (after the #) is not base64 encoded. This
> might create problems for clients passing that string around if it can
> contain non-ascii characters.
> Example:
> {noformat}
> ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
> {noformat}
> Code is
> [here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148].
> Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16572552#comment-16572552 ] Alexander Klimetschek commented on OAK-7569: Added OAK-7692 which was found by a team trying out the feature. While [~mattvryan] is out, I can tackle that unless [~amitjain] has time to look into this. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7692) [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
[
https://issues.apache.org/jira/browse/OAK-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Klimetschek updated OAK-7692:
---
Component/s: (was: blob-cloud)
blob-plugins
> [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
> ---
>
> Key: OAK-7692
> URL: https://issues.apache.org/jira/browse/OAK-7692
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: blob-plugins
>Reporter: Alexander Klimetschek
>Priority: Major
>
> The upload token's hmac signature (after the #) is not base64 encoded. This
> might create problems for clients passing that string around if it can
> contain non-ascii characters.
> Example:
> {noformat}
> ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
> {noformat}
> Code is
> [here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148]
> Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac
> for a string that can be safely passed around.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7692) [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
[
https://issues.apache.org/jira/browse/OAK-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Klimetschek updated OAK-7692:
---
Description:
The upload token's hmac signature (after the #) is not base64 encoded. This
might create problems for clients passing that string around if it can contain
non-ascii characters.
Example:
{noformat}
ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
{noformat}
Code is
[here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148]
Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac for
a string that can be safely passed around.
was:
The upload token's hmac signature (after the #) is not base64 encoded. This
might create problems for clients passing that string around if it can contain
non-ascii characters.
Example:
{noformat}
ZDI4Zi1jYzVmLTk2M2EtNGVmMC1hMjEzLTdlYTJjM2MwYWJkYi0xNTMzNjkxNzA3Nzg0IzIwMTgtMDgtMDhUMDE6Mjg6MjcuNzg3Wg==#i�_�\��?��S��,0:�
{noformat}
Code is
[here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148]
Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac for
a string that can be safely passed around.
> [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
> ---
>
> Key: OAK-7692
> URL: https://issues.apache.org/jira/browse/OAK-7692
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: blob-plugins
>Reporter: Alexander Klimetschek
>Priority: Major
>
> The upload token's hmac signature (after the #) is not base64 encoded. This
> might create problems for clients passing that string around if it can
> contain non-ascii characters.
> Example:
> {noformat}
> ZDI4Zi1[...]jcuNzg3Wg==#i�_�\��?��S��,0:�
> {noformat}
> Code is
> [here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148]
> Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac
> for a string that can be safely passed around.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7692) [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
[
https://issues.apache.org/jira/browse/OAK-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Klimetschek updated OAK-7692:
---
Description:
The upload token's hmac signature (after the #) is not base64 encoded. This
might create problems for clients passing that string around if it can contain
non-ascii characters.
Example:
{noformat}
ZDI4Zi1jYzVmLTk2M2EtNGVmMC1hMjEzLTdlYTJjM2MwYWJkYi0xNTMzNjkxNzA3Nzg0IzIwMTgtMDgtMDhUMDE6Mjg6MjcuNzg3Wg==#i�_�\��?��S��,0:�
{noformat}
Code is
[here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148]
Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac for
a string that can be safely passed around.
> [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
> ---
>
> Key: OAK-7692
> URL: https://issues.apache.org/jira/browse/OAK-7692
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: blob-cloud
>Reporter: Alexander Klimetschek
>Priority: Major
>
> The upload token's hmac signature (after the #) is not base64 encoded. This
> might create problems for clients passing that string around if it can
> contain non-ascii characters.
> Example:
> {noformat}
> ZDI4Zi1jYzVmLTk2M2EtNGVmMC1hMjEzLTdlYTJjM2MwYWJkYi0xNTMzNjkxNzA3Nzg0IzIwMTgtMDgtMDhUMDE6Mjg6MjcuNzg3Wg==#i�_�\��?��S��,0:�
> {noformat}
> Code is
> [here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148]
> Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac
> for a string that can be safely passed around.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7692) [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
[
https://issues.apache.org/jira/browse/OAK-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Klimetschek updated OAK-7692:
---
Component/s: blob-cloud
> [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
> ---
>
> Key: OAK-7692
> URL: https://issues.apache.org/jira/browse/OAK-7692
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: blob-cloud
>Reporter: Alexander Klimetschek
>Priority: Major
>
> The upload token's hmac signature (after the #) is not base64 encoded. This
> might create problems for clients passing that string around if it can
> contain non-ascii characters.
> Example:
> {noformat}
> ZDI4Zi1jYzVmLTk2M2EtNGVmMC1hMjEzLTdlYTJjM2MwYWJkYi0xNTMzNjkxNzA3Nzg0IzIwMTgtMDgtMDhUMDE6Mjg6MjcuNzg3Wg==#i�_�\��?��S��,0:�
> {noformat}
> Code is
> [here|https://github.com/mattvryan/jackrabbit-oak/blob/trunk/oak-blob-plugins/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/directaccess/DataRecordUploadToken.java#L147-L148]
> Should probably do a {{Base64.encode()}} of the {{hash}} result of the hmac
> for a string that can be safely passed around.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OAK-7692) [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded
Alexander Klimetschek created OAK-7692: -- Summary: [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded Key: OAK-7692 URL: https://issues.apache.org/jira/browse/OAK-7692 Project: Jackrabbit Oak Issue Type: Technical task Reporter: Alexander Klimetschek -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16572284#comment-16572284 ] Alexander Klimetschek commented on OAK-7569: For reference, [~mreutegg] did the ValueFactoryImpl refactoring mentioned above in OAK-7688. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16572281#comment-16572281 ] Alexander Klimetschek edited comment on OAK-7569 at 8/7/18 8:30 PM: FWIW, I added a plain unit test for oak-jcr: https://github.com/mattvryan/jackrabbit-oak/commit/f45e4ecc7823ee8d50484fdb3d076b88fab429d9 This quickly tests the current problem (plus the upload API) without requiring S3 or Azure blob storage setup. was (Author: alexander.klimetschek): FWIW, I added a plain unit test for oak-jcr: https://github.com/mattvryan/jackrabbit-oak/pull/13 This quickly tests the current problem (plus the upload API) without requiring S3 or Azure blob storage setup. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16572281#comment-16572281 ] Alexander Klimetschek commented on OAK-7569: FWIW, I added a plain unit test for oak-jcr: https://github.com/mattvryan/jackrabbit-oak/pull/13 This quickly tests the current problem (plus the upload API) without requiring S3 or Azure blob storage setup. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571976#comment-16571976 ] Alexander Klimetschek commented on OAK-7569: Ok, the „unlikely“ part is the one I wonder if Oak would be comfortable with. Otherwise I am happy if you can handle approach 1. Regarding NamePathMapper.DEFAULT: I didn‘t say all usages are wrong, and the stats was just an indication. But from looking at the code and call hierarchies, it seems a lot or most of the NamePathMapper usages is simply there to pass it to the static ValueFactoryImpl methods in the end. And it seemed impossible for me to reason about it with the varying requirements of all these different components. One example: if I have a NAME or PATH type property on a user home, will Authorizable.getProperty() handle the right namespace mappings? > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570592#comment-16570592
]
Alexander Klimetschek edited comment on OAK-7569 at 8/6/18 11:22 PM:
-
Option 1 also has 3 different levels if you look at it in detail. From less
complete (a) to full complete (c):
* *a)* Fix just the immediate case in oak-jcr and oak-core as done by Matt in
[this
branch|https://github.com/mattvryan/jackrabbit-oak/tree/direct-binary-access-v2-with-create-binary-value-fix]
** Addresses the normal {{getProperty().getBinary()}} and
{{getProperty().getValue().getBinary()}} usages plus multi-value binary
properties.
** I would do the changes slightly differently and just replace all static
method usage outside ValueFactoryImpl, where possible.
** Con: Leaves open {{ValueFactoryImpl.createValue()}} static method usage in
12 files (3 of which are in tests), with unknown impact.
* *b)* Same as a) but also change the {{createValue()}} usage in all code
paths that don't actually use a {{NamePathMapper.DEFAULT}} and don't have
access to a {{SessionContext}} currently
** No exact numbers, but {{NamePathMapper}} is used in 104 files, and 62 use
{{NamePathMapper.DEFAULT}}. Some might be from different uses of this interface.
** A trick and arguable a hack for this case could be to repurpose
{{NamePathMapper}} and add a new interface method {{getInternalValueFactory()}}
to it, which would return {{ValueFactoryImpl}} (or a new internal interface).
In some / most cases the {{NamePathMapper}} is actually the {{SessionContext}},
and this would be simple to do. The API semantics of {{NamePathMapper}} would
be a bit confusing, but at least it would prevent one from changing all the
methods and classes that pass around {{NamePathMapper}} to pass around
something else.
** Con: This leaves open places that end up using {{NamePathMapper.DEFAULT}}.
* *c)* Rigorously ensure everyone using a {{ValueFactoryImpl}} static method
must use an instance, and must have access to the current {{SessionContext}} or
{{SessionImpl}}, and remove use of {{NamePathMapper.DEFAULT}}
** Would surely be a useful improvement.
** Con: Seems like a massive Oak change as mentioned above.
Note that places (at runtime, not tests) that use {{NamePathMapper.DEFAULT}}
today would have at least the limitation that when a NAME or PATH property is
used, and the session has local namespaces, that these aren't respected,
because the default impl has no access to them.
was (Author: alexander.klimetschek):
Option 1 also has 3 different levels if you look at it in detail. From less
complete (a) to full complete (c):
* *a)* Fix just the immediate case in oak-jcr and oak-core as done by Matt in
[this
PR|https://github.com/mattvryan/jackrabbit-oak/commit/e022f846c0ff113c95910d26584568004beacb66]
(linked in my long comment above already)
** Addresses the normal {{getProperty().getBinary()}} and
{{getProperty().getValue().getBinary()}} usages plus multi-value binary
properties.
** I would do the changes slightly differently and just replace all static
method usage outside ValueFactoryImpl, where possible.
** Con: Leaves open {{ValueFactoryImpl.createValue()}} static method usage in
12 files (3 of which are in tests), with unknown impact.
* *b)* Same as a) but also change the {{createValue()}} usage in all code
paths that don't actually use a {{NamePathMapper.DEFAULT}} and don't have
access to a {{SessionContext}} currently
** No exact numbers, but {{NamePathMapper}} is used in 104 files, and 62 use
{{NamePathMapper.DEFAULT}}. Some might be from different uses of this interface.
** A trick and arguable a hack for this case could be to repurpose
{{NamePathMapper}} and add a new interface method {{getInternalValueFactory()}}
to it, which would return {{ValueFactoryImpl}} (or a new internal interface).
In some / most cases the {{NamePathMapper}} is actually the {{SessionContext}},
and this would be simple to do. The API semantics of {{NamePathMapper}} would
be a bit confusing, but at least it would prevent one from changing all the
methods and classes that pass around {{NamePathMapper}} to pass around
something else.
** Con: This leaves open places that end up using {{NamePathMapper.DEFAULT}}.
* *c)* Rigorously ensure everyone using a {{ValueFactoryImpl}} static method
must use an instance, and must have access to the current {{SessionContext}} or
{{SessionImpl}}, and remove use of {{NamePathMapper.DEFAULT}}
** Would surely be a useful improvement.
** Con: Seems like a massive Oak change as mentioned above.
Note that places (at runtime, not tests) that use {{NamePathMapper.DEFAULT}}
today would have at least the limitation that when a NAME or PATH property is
used, and the session has local namespaces, that these aren't respected,
because the default impl has no access to them.
>
[jira] [Comment Edited] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570592#comment-16570592
]
Alexander Klimetschek edited comment on OAK-7569 at 8/6/18 6:25 PM:
Option 1 also has 3 different levels if you look at it in detail. From less
complete (a) to full complete (c):
* *a)* Fix just the immediate case in oak-jcr and oak-core as done by Matt in
[this
PR|https://github.com/mattvryan/jackrabbit-oak/commit/e022f846c0ff113c95910d26584568004beacb66]
(linked in my long comment above already)
** Addresses the normal {{getProperty().getBinary()}} and
{{getProperty().getValue().getBinary()}} usages plus multi-value binary
properties.
** I would do the changes slightly differently and just replace all static
method usage outside ValueFactoryImpl, where possible.
** Con: Leaves open {{ValueFactoryImpl.createValue()}} static method usage in
12 files (3 of which are in tests), with unknown impact.
* *b)* Same as a) but also change the {{createValue()}} usage in all code
paths that don't actually use a {{NamePathMapper.DEFAULT}} and don't have
access to a {{SessionContext}} currently
** No exact numbers, but {{NamePathMapper}} is used in 104 files, and 62 use
{{NamePathMapper.DEFAULT}}. Some might be from different uses of this interface.
** A trick and arguable a hack for this case could be to repurpose
{{NamePathMapper}} and add a new interface method {{getInternalValueFactory()}}
to it, which would return {{ValueFactoryImpl}} (or a new internal interface).
In some / most cases the {{NamePathMapper}} is actually the {{SessionContext}},
and this would be simple to do. The API semantics of {{NamePathMapper}} would
be a bit confusing, but at least it would prevent one from changing all the
methods and classes that pass around {{NamePathMapper}} to pass around
something else.
** Con: This leaves open places that end up using {{NamePathMapper.DEFAULT}}.
* *c)* Rigorously ensure everyone using a {{ValueFactoryImpl}} static method
must use an instance, and must have access to the current {{SessionContext}} or
{{SessionImpl}}, and remove use of {{NamePathMapper.DEFAULT}}
** Would surely be a useful improvement.
** Con: Seems like a massive Oak change as mentioned above.
Note that places (at runtime, not tests) that use {{NamePathMapper.DEFAULT}}
today would have at least the limitation that when a NAME or PATH property is
used, and the session has local namespaces, that these aren't respected,
because the default impl has no access to them.
was (Author: alexander.klimetschek):
Option 1 also has 3 different levels if you look at it in detail. From less
complete (a) to full complete (c):
* *a)* Fix just the immediate case in oak-jcr and oak-core as done by Matt in
[this
PR|https://github.com/mattvryan/jackrabbit-oak/commit/e022f846c0ff113c95910d26584568004beacb66]
(linked in my long comment above already)
** Addresses the normal {{getProperty().getBinary()}} and
{{getProperty().getValue().getBinary()}} usages plus multi-value binary
properties.
** I would do the changes slightly differently and just replace all static
method usage outside ValueFactoryImpl, where possible.
** Con: Leaves open {{ValueFactoryImpl.createValue()}} static method usage in
12 files (3 of which are in tests), with unknown impact.
* *b)* Same as a) but also change the createValue() usage in all code paths
that don't start with a NamePathMapper.DEFAULT and don't have access to a
SessionContext currently
** No exact numbers, but NamePathMapper is used in 104 files, and 62 use
{{NamePathMapper.DEFAULT}}.
** A trick and arguable a hack for this case could be to repurpose
{{NamePathMapper}} and add a new interface method {{getInternalValueFactory()}}
to it, which would return {{ValueFactoryImpl}} (or a new internal interface).
In some / most cases the {{NamePathMapper}} is actually the {{SessionContext}},
and this would be simple to do. The API semantics of {{NamePathMapper}} would
be a bit confusing, but at least it would prevent one from changing all the
methods and classes that pass around {{NamePathMapper}} to pass around
something else.
** Con: This leaves open places that end up using {{NamePathMapper.DEFAULT}}.
* *c)* Rigorously ensure everyone using a {{ValueFactoryImpl}} static method
must use an instance, and must have access to the current {{SessionContext}} or
{{SessionImpl}}, and remove use of {{NamePathMapper.DEFAULT}}
** Would surely be a useful improvement.
** Con: Seems like a massive Oak change as mentioned above.
Note that places (at runtime, not tests) that use {{NamePathMapper.DEFAULT}}
today would have at least the limitation that when a NAME or PATH property is
used, and the session has local namespaces, that these aren't respected,
because the default impl has no access to them.
> Direct Binary Access
>
>
>
[jira] [Commented] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570592#comment-16570592
]
Alexander Klimetschek commented on OAK-7569:
Option 1 also has 3 different levels if you look at it in detail. From less
complete (a) to full complete (c):
* *a)* Fix just the immediate case in oak-jcr and oak-core as done by Matt in
[this
PR|https://github.com/mattvryan/jackrabbit-oak/commit/e022f846c0ff113c95910d26584568004beacb66]
(linked in my long comment above already)
** Addresses the normal {{getProperty().getBinary()}} and
{{getProperty().getValue().getBinary()}} usages plus multi-value binary
properties.
** I would do the changes slightly differently and just replace all static
method usage outside ValueFactoryImpl, where possible.
** Con: Leaves open {{ValueFactoryImpl.createValue()}} static method usage in
12 files (3 of which are in tests), with unknown impact.
* *b)* Same as a) but also change the createValue() usage in all code paths
that don't start with a NamePathMapper.DEFAULT and don't have access to a
SessionContext currently
** No exact numbers, but NamePathMapper is used in 104 files, and 62 use
{{NamePathMapper.DEFAULT}}.
** A trick and arguable a hack for this case could be to repurpose
{{NamePathMapper}} and add a new interface method {{getInternalValueFactory()}}
to it, which would return {{ValueFactoryImpl}} (or a new internal interface).
In some / most cases the {{NamePathMapper}} is actually the {{SessionContext}},
and this would be simple to do. The API semantics of {{NamePathMapper}} would
be a bit confusing, but at least it would prevent one from changing all the
methods and classes that pass around {{NamePathMapper}} to pass around
something else.
** Con: This leaves open places that end up using {{NamePathMapper.DEFAULT}}.
* *c)* Rigorously ensure everyone using a {{ValueFactoryImpl}} static method
must use an instance, and must have access to the current {{SessionContext}} or
{{SessionImpl}}, and remove use of {{NamePathMapper.DEFAULT}}
** Would surely be a useful improvement.
** Con: Seems like a massive Oak change as mentioned above.
Note that places (at runtime, not tests) that use {{NamePathMapper.DEFAULT}}
today would have at least the limitation that when a NAME or PATH property is
used, and the session has local namespaces, that these aren't respected,
because the default impl has no access to them.
> Direct Binary Access
>
>
> Key: OAK-7569
> URL: https://issues.apache.org/jira/browse/OAK-7569
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: api, blob-cloud, blob-cloud-azure, blob-plugins
>Reporter: Matt Ryan
>Assignee: Matt Ryan
>Priority: Major
> Attachments: OAK-7569-api-javadoc-improvements.patch
>
>
> Provide a direct binary access feature to Oak which allows an authenticated
> client to create or download blobs directly to/from the blob store, assuming
> the authenticated user has appropriate permission to do so. The primary value
> of this feature is that the I/O of transferring large binary files to or from
> the blob store can be offloaded entirely from Oak and performed directly
> between a client application and the blob store.
> This feature is described in more detail [on the Oak
> wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access].
> This feature is similar in functionality to OAK-6575. It adds the capability
> to also upload directly to storage via preauthorized URLs in addition to
> downloading directly from storage via preauthorized URLs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570288#comment-16570288 ] Alexander Klimetschek edited comment on OAK-7569 at 8/6/18 2:57 PM: Sure, but please note that getting the implementation right (getting rid of static ValueFactoryImpl usage) affects 100+ files. Searching for NamePathMapper gives an indication: https://github.com/apache/jackrabbit-oak/search?utf8=%E2%9C%93=NamePathMapper This would involve folks from different parts of Oak and possible take weeks and introduce a lot of risks. Matt and me really looked into that last week but had to give up - especially when you hit NamePathMapper.DEFAULT. I fully agree it would be good for Oak to straighten this, but I guess there is a limit to what amount can be refactored and it would be nice to finally get this feature in. was (Author: alexander.klimetschek): Sure, but please note that getting the implementation right (getting rid of static ValueFactoryImpl usage) affects 100+ files. Searching for NamePathMapper gives an indication: https://github.com/apache/jackrabbit-oak/search?utf8=%E2%9C%93=NamePathMapper This would involve folks from different parts of Oak and possible take weeks and introduce a lot of risks. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570288#comment-16570288 ] Alexander Klimetschek commented on OAK-7569: Sure, but please note that getting the implementation right (getting rid of static ValueFactoryImpl usage) affects 100+ files. Searching for NamePathMapper gives an indication: https://github.com/apache/jackrabbit-oak/search?utf8=%E2%9C%93=NamePathMapper This would involve folks from different parts of Oak and possible take weeks and introduce a lot of risks. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569821#comment-16569821 ] Alexander Klimetschek commented on OAK-7569: Ok, maybe then I might not know Oak‘s process for such API releases and how it relates to semantic versioning. Anyway, any preference on the different options? Do you think 1 is ok or the „full“ static-less ValueFactoryImpl is feasible for an Oak code expert? > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569802#comment-16569802
]
Alexander Klimetschek edited comment on OAK-7569 at 8/6/18 7:14 AM:
h4. Problem
Unfortunately, we could not commit the final agreed change this week, because
[~mattvryan] found a blocking issue: {{BinaryDownload.getURI()}} always returns
null because the regular code path {{node.getProperty().getBinary()}} does not
pass the {{BlobAccessProvider}} instance through to the {{ValueFactoryImpl}}
because it uses a [static method of ValueFactoryImpl in
PropertyImpl.getValue()|https://github.com/apache/jackrabbit-oak/blob/905c9bd0f716778a035a708bbdb8de634f464e66/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/session/PropertyImpl.java#L252-L253].
Looking further, there are actually many static method usages of
{{ValueFactoryImpl}} today, that all need to ensure a {{BlobAccessProvider}} is
available, which is far from trivial.
[~mattvryan] and me think there are 2 solutions:
# fix the static ValueFactoryImpl issue as much as possible (but ALL cases
seem to be impossible to fix without a major Oak refactoring)
# go back to the original API design (2a) or a slight variation thereof (2b)
as a feasible compromise.
More details below.
h4. Details
The unit tests so far had a shortcut where it reused the {{Binary}} used for
writing, hence no one saw the issue earlier. This has been fixed in the latest
changes for the integration tests:
[https://github.com/mattvryan/jackrabbit-oak/pull/12] (also includes a lot of
improvements around the test framework itself).
This issue, that did not exist initially, and was tested, was introduced as a
result of the refactorings during the review - which moved the API to a
ValueFactory extension. At the same time the unit tests changed a bit,
accidentally removing the important test case.
It is possible to fix this particular Value access case, since {{PropertyImpl}}
already has access to the per-session {{ValueFactoryImpl}} instance, and it
just needs to replace all the static usages left by leveraging
{{getValueFactory()}}. Same for some other places in oak-jcr and oak-core that
currently employ a mix of instance and static method access. [~mattvryan] did
an attempt at this here:
[https://github.com/mattvryan/jackrabbit-oak/commit/e022f846c0ff113c95910d26584568004beacb66]
Ideally, to ensure the {{blobAccessProvider}} reference in the
{{ValueFactoryImpl}} is always set to the reference from the whiteboard that
the {{SessionContext}} has, one would have to eliminate the static methods on
{{ValueFactoryImpl}} and replace by instance methods. The class isn't doing
proper encapsulation at this point, which is visible by the fact that
{{NamePathMapper}}, an implementation detail the ValueFactoryImpl relies on, is
actually all over the Oak code base.
*However, there are two static {{ValueFactoryImpl.createValue()}} methods
retrieving a PropertyState and PropertyValue argument respectively (+
NamePathMapper) that are used in different places in oak, including
oak-security-spi.* These are completely unrelated to the binary change. They
don't have any immediate access to the {{SessionContext}} (which provides both
the {{ValueFactoryImpl}} and {{BlobAcccessProvider}} instance). They all get a
{{NamePathMapper}} passed in, which is essentially the same situation that a
component reference must be available for the {{ValueFactoryImpl}} methods to
do their work, but if you follow the call hierarchy it literally explodes.
Meaning a lot of files all over Oak would have to be changed to actually pass
through the {{ValueFactoryImpl}} instance instead of just {{NamePathMapper}}.
But even then, in many places it does not even get any access to the real
{{NamePathMapper}} which {{SessionContext}} implements, but a
{{NamePathMapper.DEFAULT}} is used (which apparently works for some of these
cases?). It might be that some components were never designed to get access to
the {{SessionContext}}, at least it seems it would be a massive refactoring to
ensure that everywhere.
h4. Unsolved Binary cases
How does that affect the direct binary access feature? One example is
[AuthorizablePropertiesImpl|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/security/user/AuthorizablePropertiesImpl.java#L108].
This corresponds to the code path of
{{Authorizable.getProperty("foo").getValue().getBinary()}}, which would never
be able to return anything from {{getURI()}}. Note that it will implement
{{BinaryDownload}}, and the API contract is that {{getURI()}} shall return null
if the feature is not available, but not if the client used the "wrong" way of
retrieving a {{Binary}}.
Other cases are hard to reason about: it's unclear where exactly they could end
up being used by a JCR client.
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569819#comment-16569819 ] Alexander Klimetschek commented on OAK-7569: [~reschke] Yes, if by "here" you mean just talking about the API changes done for this feature in 2.17.5. Generally speaking, there cannot be breaking API changes (major version updates) in jackrabbit-api without disrupting the entire ecosystem. It's important to note the maven-bundle-plugin will not differentiate between the two - it will complain about both. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569814#comment-16569814 ] Alexander Klimetschek commented on OAK-7569: Another approach for 2a) would be to keep 2.17.5 on maven release, but completely ignore it (mark it as broken), and for 2.17.6 just ignore following the semantic version changes (that are only based on stuff added in 2.17.5). > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569814#comment-16569814 ] Alexander Klimetschek edited comment on OAK-7569 at 8/6/18 7:10 AM: Another approach for 2a) would be to keep 2.17.5 on maven central, but completely ignore it (mark it as broken), and for 2.17.6 just ignore following the semantic version changes (that are only based on stuff added in 2.17.5). was (Author: alexander.klimetschek): Another approach for 2a) would be to keep 2.17.5 on maven release, but completely ignore it (mark it as broken), and for 2.17.6 just ignore following the semantic version changes (that are only based on stuff added in 2.17.5). > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569813#comment-16569813
]
Alexander Klimetschek edited comment on OAK-7569 at 8/6/18 7:07 AM:
[~reschke] Please have a look at the details of 2a and 2b (just made some
slight changes). Anything that forces {{org.apache.jackrabbit.api}} to have a
breaking change and have it's package version go from 2.5.0 to 3.0.0 would be a
no go as it would make all existing JCR client bundles using that package (and
that will be most I reckon) incompatible without recompilation. I.e. just the
upgrade to a newer Oak would break all these bundles.
was (Author: alexander.klimetschek):
[~reschke] Please have a look at the details of 2a and 2b (just made some
slight changes). Anything that forces {{org.apache.jackrabbit.api}} to have a
breaking change and have it's package version go from 2.5.0 to 3.0.0 would be a
no go as it would make all existing JCR client bundles using that package (and
that will be most I reckon) incompatible without recompilation.
> Direct Binary Access
>
>
> Key: OAK-7569
> URL: https://issues.apache.org/jira/browse/OAK-7569
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: api, blob-cloud, blob-cloud-azure, blob-plugins
>Reporter: Matt Ryan
>Assignee: Matt Ryan
>Priority: Major
> Attachments: OAK-7569-api-javadoc-improvements.patch
>
>
> Provide a direct binary access feature to Oak which allows an authenticated
> client to create or download blobs directly to/from the blob store, assuming
> the authenticated user has appropriate permission to do so. The primary value
> of this feature is that the I/O of transferring large binary files to or from
> the blob store can be offloaded entirely from Oak and performed directly
> between a client application and the blob store.
> This feature is described in more detail [on the Oak
> wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access].
> This feature is similar in functionality to OAK-6575. It adds the capability
> to also upload directly to storage via preauthorized URLs in addition to
> downloading directly from storage via preauthorized URLs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
