[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588580#comment-16588580 ] Marcel Reutegger commented on OAK-7569: --- Finally committed the remaining changes in Matt's PR https://github.com/apache/jackrabbit-oak/pull/96 to trunk: http://svn.apache.org/r1838617 > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588529#comment-16588529 ] Alexander Klimetschek commented on OAK-7569: [~mreutegg] The javadoc patch is ready in JCR-4355, it could be committed and a new jackrabbit-api released. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575229#comment-16575229 ] Marcel Reutegger commented on OAK-7569: --- Good point, I made the NamePathMapper private in my github branch. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574731#comment-16574731
]
Michael Dürig commented on OAK-7569:
[~mreutegg], from a quick glance at your branch:
* {{PartialValueFactory#namePathMapper}} is protected final while
{{PartialValueFactory#blobAccessProvider}} is private but has a package private
accessor. I don't have a preference for one over the other but would like to
have them exposed in the same way for consistency reasons.
* Looking at {{PartialValueFactory}} and {{ValueFactoryImpl}} I get the
feeling that - with some more refactoring - {{ValueFactoryImpl}} could be moved
to {{oak-jcr}} (where it belongs IMO) and {{PartialValueFactory}} could
encapsulate {{NamePathMapper#DEFAULT}} instead of having the latter passed
around to all consumers. Maybe worth a separate issue...
> Direct Binary Access
>
>
> Key: OAK-7569
> URL: https://issues.apache.org/jira/browse/OAK-7569
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: api, blob-cloud, blob-cloud-azure, blob-plugins
>Reporter: Matt Ryan
>Assignee: Matt Ryan
>Priority: Major
> Attachments: OAK-7569-api-javadoc-improvements.patch
>
>
> Provide a direct binary access feature to Oak which allows an authenticated
> client to create or download blobs directly to/from the blob store, assuming
> the authenticated user has appropriate permission to do so. The primary value
> of this feature is that the I/O of transferring large binary files to or from
> the blob store can be offloaded entirely from Oak and performed directly
> between a client application and the blob store.
> This feature is described in more detail [on the Oak
> wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access].
> This feature is similar in functionality to OAK-6575. It adds the capability
> to also upload directly to storage via preauthorized URLs in addition to
> downloading directly from storage via preauthorized URLs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573485#comment-16573485 ] Marcel Reutegger commented on OAK-7569: --- Today I continued working on this feature on top of Matt's work. The github branch is: https://github.com/mreutegg/jackrabbit-oak/tree/direct-binary-access-v2-mreutegg Direct binary access is now also working for authorizable properties and values obtained through events. One of the tests uncovered an issue that need to be fixed first: OAK-7694 > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573479#comment-16573479 ] Marcel Reutegger commented on OAK-7569: --- bq. We need documentation for the oak docs anyway, not sure if there is anything yet. Happy to add something. Where would I add this in the navigation tree? I would suggest either here: https://jackrabbit.apache.org/oak/docs/plugins/blobstore.html or maybe create a sub page? > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573475#comment-16573475 ] Marcel Reutegger commented on OAK-7569: --- bq. So the places that we support are getProperty().[getValue().]getBinary(), as well as the equivalent multivalue methods (if anyone really uses multivalue binary properties . Any other „normal“ Binary read case that I missed? The feature would be generally available with two exceptions: - Default values obtained from a PropertyDefinition - Restriction values from a ACE > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573451#comment-16573451 ] Alexander Klimetschek commented on OAK-7569: AuthorizablePropertiesImpl: Good to be confirmed. When I followed the call hierachy, there were many places that used DEFAULT (IIRC for the UserManagerImpl constructor) and it wasn‘t clear which ones are actually used at normal runtime (and I skipped any test usages). Limitation/documentation: fair enough. So the places that we support are getProperty().[getValue().]getBinary(), as well as the equivalent multivalue methods (if anyone really uses multivalue binary properties ;-). Any other „normal“ Binary read case that I missed? One thought: if we don‘t have a BlobAccessProvider available, could we return a Binary that does not implement BinaryDownload? Then it would be clear to developers they used an unsupported access path. We need documentation for the oak docs anyway, not sure if there is anything yet. Happy to add something. Where would I add this in the navigation tree? > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572812#comment-16572812 ] Marcel Reutegger commented on OAK-7569: --- bq. „unlikely“ part is the one I wonder if Oak would be comfortable with I'd be comfortable with it when this limitation is mentioned in the documentation. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572808#comment-16572808
]
Marcel Reutegger commented on OAK-7569:
---
bq. One example: if I have a NAME or PATH type property on a user home, will
Authorizable.getProperty() handle the right namespace mappings?
It will, because {{AuthorizablePropertiesImpl}} gets a NamePathMapper bound to
the session and is not using {{NamePathMapper.DEFAULT}}.
> Direct Binary Access
>
>
> Key: OAK-7569
> URL: https://issues.apache.org/jira/browse/OAK-7569
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: api, blob-cloud, blob-cloud-azure, blob-plugins
>Reporter: Matt Ryan
>Assignee: Matt Ryan
>Priority: Major
> Attachments: OAK-7569-api-javadoc-improvements.patch
>
>
> Provide a direct binary access feature to Oak which allows an authenticated
> client to create or download blobs directly to/from the blob store, assuming
> the authenticated user has appropriate permission to do so. The primary value
> of this feature is that the I/O of transferring large binary files to or from
> the blob store can be offloaded entirely from Oak and performed directly
> between a client application and the blob store.
> This feature is described in more detail [on the Oak
> wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access].
> This feature is similar in functionality to OAK-6575. It adds the capability
> to also upload directly to storage via preauthorized URLs in addition to
> downloading directly from storage via preauthorized URLs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572552#comment-16572552 ] Alexander Klimetschek commented on OAK-7569: Added OAK-7692 which was found by a team trying out the feature. While [~mattvryan] is out, I can tackle that unless [~amitjain] has time to look into this. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572284#comment-16572284 ] Alexander Klimetschek commented on OAK-7569: For reference, [~mreutegg] did the ValueFactoryImpl refactoring mentioned above in OAK-7688. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572281#comment-16572281 ] Alexander Klimetschek commented on OAK-7569: FWIW, I added a plain unit test for oak-jcr: https://github.com/mattvryan/jackrabbit-oak/pull/13 This quickly tests the current problem (plus the upload API) without requiring S3 or Azure blob storage setup. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16571976#comment-16571976 ] Alexander Klimetschek commented on OAK-7569: Ok, the „unlikely“ part is the one I wonder if Oak would be comfortable with. Otherwise I am happy if you can handle approach 1. Regarding NamePathMapper.DEFAULT: I didn‘t say all usages are wrong, and the stats was just an indication. But from looking at the code and call hierarchies, it seems a lot or most of the NamePathMapper usages is simply there to pass it to the static ValueFactoryImpl methods in the end. And it seemed impossible for me to reason about it with the varying requirements of all these different components. One example: if I have a NAME or PATH type property on a user home, will Authorizable.getProperty() handle the right namespace mappings? > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16571914#comment-16571914
]
Marcel Reutegger commented on OAK-7569:
---
There are now a few places where a {{PartialValueFactory}} is instantiated
without a {{BlobAccessProvider}}.
In oak-core:
- {{EffectiveNodeTypeImpl}} - the factory creates values uses only internally
to check whether a property can be set.
- {{PropertyDefinitionImpl}} - the factory creates values from properties that
represent default values from the property definition. Those values can be
accessed by client code, though it is rather unlikely that those values are
from the blob store.
- {{TypeEditor}} - the factory creates values only used internally in the type
editor.
- {{AuthorizablePropertiesImpl}} - the factory creates values that can be
accessed by client code.
In oak-jcr:
- {{EventFactory}}, {{QueryResultImpl}}, {{PropertyImpl}}, {{VersionImpl}} -
the factory creates values that can be accessed by client code.
In oak-security-spi:
- {{ACE}} - the factory creates values from a restriction. The values can be
accessed by client code but are unlikely from the blob store.
In oak-store-spi:
- {{ValueFactoryImpl}} - these are the deprecated methods that should be
removed.
> Direct Binary Access
>
>
> Key: OAK-7569
> URL: https://issues.apache.org/jira/browse/OAK-7569
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: api, blob-cloud, blob-cloud-azure, blob-plugins
>Reporter: Matt Ryan
>Assignee: Matt Ryan
>Priority: Major
> Attachments: OAK-7569-api-javadoc-improvements.patch
>
>
> Provide a direct binary access feature to Oak which allows an authenticated
> client to create or download blobs directly to/from the blob store, assuming
> the authenticated user has appropriate permission to do so. The primary value
> of this feature is that the I/O of transferring large binary files to or from
> the blob store can be offloaded entirely from Oak and performed directly
> between a client application and the blob store.
> This feature is described in more detail [on the Oak
> wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access].
> This feature is similar in functionality to OAK-6575. It adds the capability
> to also upload directly to storage via preauthorized URLs in addition to
> downloading directly from storage via preauthorized URLs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16571244#comment-16571244
]
Marcel Reutegger commented on OAK-7569:
---
The {{NamePathMapper.DEFAULT}} should only be used when names and paths have
already been mapped to the internal representation, which is the case in layers
below oak-jcr. E.g. using {{NamePathMapper.DEFAULT}} in oak-core doesn't
necessarily mean it's a problem.
The static methods on {{ValueFactoryImpl}} all wrap a {{Value}} around an
existing {{PropertyState}}. Those methods don't need a {{Root}}, which is
otherwise available in a {{ValueFactoryImpl}} instance. I think it should be
possible to prevent additional technical debt (more static methods) with some
refactoring. I'll prepare a patch to illustrate the approach.
Regarding the options you mentioned:
- b) Simply counting usages of {{NamePathMapper}} is misleading. There are a
few places we need to look at additionally, but usages of the value factory
methods gives a better indication.
- c) Whenever a {{NamePathMapper.DEFAULT}} is used, the code assumes a name has
already been mapped to the internal representation. Using the session mapping
instead would be wrong.
> Direct Binary Access
>
>
> Key: OAK-7569
> URL: https://issues.apache.org/jira/browse/OAK-7569
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: api, blob-cloud, blob-cloud-azure, blob-plugins
>Reporter: Matt Ryan
>Assignee: Matt Ryan
>Priority: Major
> Attachments: OAK-7569-api-javadoc-improvements.patch
>
>
> Provide a direct binary access feature to Oak which allows an authenticated
> client to create or download blobs directly to/from the blob store, assuming
> the authenticated user has appropriate permission to do so. The primary value
> of this feature is that the I/O of transferring large binary files to or from
> the blob store can be offloaded entirely from Oak and performed directly
> between a client application and the blob store.
> This feature is described in more detail [on the Oak
> wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access].
> This feature is similar in functionality to OAK-6575. It adds the capability
> to also upload directly to storage via preauthorized URLs in addition to
> downloading directly from storage via preauthorized URLs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16570592#comment-16570592
]
Alexander Klimetschek commented on OAK-7569:
Option 1 also has 3 different levels if you look at it in detail. From less
complete (a) to full complete (c):
* *a)* Fix just the immediate case in oak-jcr and oak-core as done by Matt in
[this
PR|https://github.com/mattvryan/jackrabbit-oak/commit/e022f846c0ff113c95910d26584568004beacb66]
(linked in my long comment above already)
** Addresses the normal {{getProperty().getBinary()}} and
{{getProperty().getValue().getBinary()}} usages plus multi-value binary
properties.
** I would do the changes slightly differently and just replace all static
method usage outside ValueFactoryImpl, where possible.
** Con: Leaves open {{ValueFactoryImpl.createValue()}} static method usage in
12 files (3 of which are in tests), with unknown impact.
* *b)* Same as a) but also change the createValue() usage in all code paths
that don't start with a NamePathMapper.DEFAULT and don't have access to a
SessionContext currently
** No exact numbers, but NamePathMapper is used in 104 files, and 62 use
{{NamePathMapper.DEFAULT}}.
** A trick and arguable a hack for this case could be to repurpose
{{NamePathMapper}} and add a new interface method {{getInternalValueFactory()}}
to it, which would return {{ValueFactoryImpl}} (or a new internal interface).
In some / most cases the {{NamePathMapper}} is actually the {{SessionContext}},
and this would be simple to do. The API semantics of {{NamePathMapper}} would
be a bit confusing, but at least it would prevent one from changing all the
methods and classes that pass around {{NamePathMapper}} to pass around
something else.
** Con: This leaves open places that end up using {{NamePathMapper.DEFAULT}}.
* *c)* Rigorously ensure everyone using a {{ValueFactoryImpl}} static method
must use an instance, and must have access to the current {{SessionContext}} or
{{SessionImpl}}, and remove use of {{NamePathMapper.DEFAULT}}
** Would surely be a useful improvement.
** Con: Seems like a massive Oak change as mentioned above.
Note that places (at runtime, not tests) that use {{NamePathMapper.DEFAULT}}
today would have at least the limitation that when a NAME or PATH property is
used, and the session has local namespaces, that these aren't respected,
because the default impl has no access to them.
> Direct Binary Access
>
>
> Key: OAK-7569
> URL: https://issues.apache.org/jira/browse/OAK-7569
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: api, blob-cloud, blob-cloud-azure, blob-plugins
>Reporter: Matt Ryan
>Assignee: Matt Ryan
>Priority: Major
> Attachments: OAK-7569-api-javadoc-improvements.patch
>
>
> Provide a direct binary access feature to Oak which allows an authenticated
> client to create or download blobs directly to/from the blob store, assuming
> the authenticated user has appropriate permission to do so. The primary value
> of this feature is that the I/O of transferring large binary files to or from
> the blob store can be offloaded entirely from Oak and performed directly
> between a client application and the blob store.
> This feature is described in more detail [on the Oak
> wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access].
> This feature is similar in functionality to OAK-6575. It adds the capability
> to also upload directly to storage via preauthorized URLs in addition to
> downloading directly from storage via preauthorized URLs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16570288#comment-16570288 ] Alexander Klimetschek commented on OAK-7569: Sure, but please note that getting the implementation right (getting rid of static ValueFactoryImpl usage) affects 100+ files. Searching for NamePathMapper gives an indication: https://github.com/apache/jackrabbit-oak/search?utf8=%E2%9C%93&q=NamePathMapper&type This would involve folks from different parts of Oak and possible take weeks and introduce a lot of risks. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569955#comment-16569955
]
Michael Dürig commented on OAK-7569:
{quote}My preferred option is keeping the current Jackrabbit API and getting
the implementation right.
{quote}
+1
> Direct Binary Access
>
>
> Key: OAK-7569
> URL: https://issues.apache.org/jira/browse/OAK-7569
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: api, blob-cloud, blob-cloud-azure, blob-plugins
>Reporter: Matt Ryan
>Assignee: Matt Ryan
>Priority: Major
> Attachments: OAK-7569-api-javadoc-improvements.patch
>
>
> Provide a direct binary access feature to Oak which allows an authenticated
> client to create or download blobs directly to/from the blob store, assuming
> the authenticated user has appropriate permission to do so. The primary value
> of this feature is that the I/O of transferring large binary files to or from
> the blob store can be offloaded entirely from Oak and performed directly
> between a client application and the blob store.
> This feature is described in more detail [on the Oak
> wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access].
> This feature is similar in functionality to OAK-6575. It adds the capability
> to also upload directly to storage via preauthorized URLs in addition to
> downloading directly from storage via preauthorized URLs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569944#comment-16569944 ] Marcel Reutegger commented on OAK-7569: --- bq. Anyway, any preference on the different options? Do you think 1 is ok or the „full“ static-less ValueFactoryImpl is feasible for an Oak code expert? My preferred option is keeping the current Jackrabbit API and getting the implementation right. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569921#comment-16569921 ] Marcel Reutegger commented on OAK-7569: --- I would not revert the changes right now. The Jackrabbit API changes are already released and I don't see a benefit in reverting the changes immediately. We can decide to keep the Jackrabbit API dependency in Oak at version 2.17.4 if needed or even update to 2.17.5 and not implement the feature at this point. The Oak API changes are in my view unrelated to the oak-jcr implementation issue discussed here and will likely not change anyway. The same applies to the changes to the lower layers already committed by Amit. I think it's fine to keep them and even create an Oak release. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569826#comment-16569826 ] Julian Reschke commented on OAK-7569: - I'm not that familiar with oak-jcr, so I really don't know how to proceed. If there are currently doubts that we can proceed with the original plan, we may want to revert all changes that we currently have both in Jackrabbit trunk and Oak trunk in order to unblock releases of other things, and not to paint us into a corner. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569821#comment-16569821 ] Alexander Klimetschek commented on OAK-7569: Ok, maybe then I might not know Oak‘s process for such API releases and how it relates to semantic versioning. Anyway, any preference on the different options? Do you think 1 is ok or the „full“ static-less ValueFactoryImpl is feasible for an Oak code expert? > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569820#comment-16569820 ] Julian Reschke commented on OAK-7569: - Once again. "unstable" means "unstable". We are calling these releases "unstable" so that we *can* undo changes if needed. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569819#comment-16569819 ] Alexander Klimetschek commented on OAK-7569: [~reschke] Yes, if by "here" you mean just talking about the API changes done for this feature in 2.17.5. Generally speaking, there cannot be breaking API changes (major version updates) in jackrabbit-api without disrupting the entire ecosystem. It's important to note the maven-bundle-plugin will not differentiate between the two - it will complain about both. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569815#comment-16569815 ] Julian Reschke commented on OAK-7569: - [~alexander.klimetschek] AFAIU there is no problem here - we can just undo the changes and release a unstable version. There are no compatibility guarantees here. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569814#comment-16569814 ] Alexander Klimetschek commented on OAK-7569: Another approach for 2a) would be to keep 2.17.5 on maven release, but completely ignore it (mark it as broken), and for 2.17.6 just ignore following the semantic version changes (that are only based on stuff added in 2.17.5). > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569813#comment-16569813
]
Alexander Klimetschek commented on OAK-7569:
[~reschke] Please have a look at the details of 2a and 2b (just made some
slight changes). Anything that forces {{org.apache.jackrabbit.api}} to have a
breaking change and have it's package version go from 2.5.0 to 3.0.0 would be a
no go as it would make all existing JCR client bundles using that package (and
that will be most I reckon) incompatible without recompilation.
> Direct Binary Access
>
>
> Key: OAK-7569
> URL: https://issues.apache.org/jira/browse/OAK-7569
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: api, blob-cloud, blob-cloud-azure, blob-plugins
>Reporter: Matt Ryan
>Assignee: Matt Ryan
>Priority: Major
> Attachments: OAK-7569-api-javadoc-improvements.patch
>
>
> Provide a direct binary access feature to Oak which allows an authenticated
> client to create or download blobs directly to/from the blob store, assuming
> the authenticated user has appropriate permission to do so. The primary value
> of this feature is that the I/O of transferring large binary files to or from
> the blob store can be offloaded entirely from Oak and performed directly
> between a client application and the blob store.
> This feature is described in more detail [on the Oak
> wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access].
> This feature is similar in functionality to OAK-6575. It adds the capability
> to also upload directly to storage via preauthorized URLs in addition to
> downloading directly from storage via preauthorized URLs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569808#comment-16569808 ] Julian Reschke commented on OAK-7569: - We can't (IMHO) remove a release. That said, we *can* make breaking changes in Jackrabbit 2.17.* - there are no compatibility guarantees. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569802#comment-16569802
]
Alexander Klimetschek commented on OAK-7569:
h4. Problem
Unfortunately, we could not commit the final agreed change this week, because
[~mattvryan] found a blocking issue: {{BinaryDownload.getURI()}} always returns
null because the regular code path {{node.getProperty().getBinary()}} does not
pass the {{BlobAccessProvider}} instance through to the {{ValueFactoryImpl}}
because it uses a [static method of ValueFactoryImpl in
PropertyImpl.getValue()|https://github.com/apache/jackrabbit-oak/blob/905c9bd0f716778a035a708bbdb8de634f464e66/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/session/PropertyImpl.java#L252-L253].
h4. Details
The unit tests so far had a shortcut where it reused the {{Binary used for
writing, hence we didn't see it earlier. This has been fixed in the latest
changes for the integration tests:
[https://github.com/mattvryan/jackrabbit-oak/pull/12] (also includes a lot of
improvements around the test framework itself).
This bug was introduced as a result of the refactorings during the review -
which moved the API to a ValueFactory extension. At the same time the unit
tests changed a bit (accidentally).
It is possible to fix this particular Value access case, since {{PropertyImpl}}
already has access to the per-session {{ValueFactoryImpl}} instance, and it
just needs to replace all the static usages left by leveraging
{{getValueFactory()}}. Same for some other places in oak-jcr and oak-core that
currently employ a mix of instance and static method access. [~mattvryan] did
an attempt at this here:
[https://github.com/mattvryan/jackrabbit-oak/commit/e022f846c0ff113c95910d26584568004beacb66]
Ideally, to ensure the {{blobAccessProvider}} reference in the
{{ValueFactoryImpl}} is always set to the reference from the whiteboard that
the {{SessionContext}} has, one would have to eliminate the static methods on
{{ValueFactoryImpl}} and replace by instance methods. The class isn't doing
proper encapsulation at this point, which is visible by the fact that
{{NamePathMapper}}, an implementation detail the ValueFactoryImpl relies on, is
actually all over the Oak code base.
*However, there are two static {{ValueFactoryImpl.createValue()}} methods
retrieving a PropertyState and PropertyValue argument respectively (+
NamePathMapper) that are used in different places in oak, including
oak-security-spi.* These are completely unrelated to the binary change. They
don't have any immediate access to the {{SessionContext}} (which provides both
the {{ValueFactoryImpl}} and {{BlobAcccessProvider}} instance). They all get a
{{NamePathMapper}} passed in, which is essentially the same situation that a
component reference must be available for the {{ValueFactoryImpl}} methods to
do their work, but if you follow the call hierarchy it literally explodes.
Meaning a lot of files all over Oak would have to be changed to actually pass
through the {{ValueFactoryImpl}} instance instead of just {{NamePathMapper}}.
But even then, in many places it does not even get any access to the real
{{NamePathMapper}} which {{SessionContext}} implements, but a
{{NamePathMapper.DEFAULT}} is used (which apparently works for some of these
cases?). It might be that some components were never designed to get access to
the {{SessionContext}}, at least it seems it would be a massive refactoring to
ensure that everywhere.
h4. Unsolved Binary cases
How does that affect the change? One example is {{AuthorizablePropertiesImpl}}.
This corresponds to the code path of
{{Authorizable.getProperty("foo").getValue().getBinary()}}, which would never
be able to return anything from {{getURI()}}. Note that it will implement
{{BinaryDownload}}, and the API contract is that {{getURI()}} shall return null
if the feature is not available, but not if the client used the "wrong" way of
retrieving a {{Binary}}.
Other cases are hard to reason about: it's unclear where exactly they could end
up being used by a JCR client. We tried to analyze it but I am not sure one can
make guarantees... and since the static methods aren't property-type specific,
one cannot exclude binary properties. Furthermore we have to consider future
usage of these static methods (unless they become deprecated or removed).
FWIW, this is the reason in our original patch we did not extended the
{{ValueFactory}} at all but used an extension interface
({{BinaryAccessProvider}} on {{Session}}, as then access to the
{{SessionContext}} and anything else is trivial. The idea of the
{{ValueFactory}} as creator of values sounds good in theory, but the
implementation in Oak using static methods creates values "out of thin air"
with no access to other components.
Also note that this only applies to downloads (BinaryDownload.getU
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16568922#comment-16568922 ] Alexander Klimetschek commented on OAK-7569: Made some improvements to the jackrabbit-api javadocs for this feature: [^OAK-7569-api-javadoc-improvements.patch] I waited on committing them because there is still an issue (see comment below) which _might_ require to rollback/change the API. There has been [no change|https://github.com/apache/jackrabbit/commits/trunk/jackrabbit-api] to jackrabbit-api on trunk after the 2.17.5 release and it might be easier if there is none. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > Attachments: OAK-7569-api-javadoc-improvements.patch > > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561782#comment-16561782 ] Amit Jain commented on OAK-7569: [~mattvryan] Thanks! for your patch. I committed the changes to the blob modules with [http://svn.apache.org/viewvc?rev=1837047&view=rev]. * I did some changes to the * DataRecordAccessProvider: ** Enabling tests only when required properties available ** Using the existing utilities for reading configuration, DataStore initialization ** Using the randomStream method to create streams instead of randomString * Created new ITs for the memory intensive test that should not run by default. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16558245#comment-16558245 ] Marcel Reutegger commented on OAK-7569: --- And some more changes to oak-jcr that reduce the time to run the tests: https://github.com/mattvryan/jackrabbit-oak/pull/11 > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16558155#comment-16558155 ] Marcel Reutegger commented on OAK-7569: --- I also reverted some of your changed in oak-jcr/pom.xml. See https://github.com/mattvryan/jackrabbit-oak/pull/10 > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16558127#comment-16558127
]
Marcel Reutegger commented on OAK-7569:
---
[~mattvryan], I moved the lookup of the {{BlobAccessProvider}} into
{{RepositoryImpl}}. See my PR:
https://github.com/mattvryan/jackrabbit-oak/pull/9
> Direct Binary Access
>
>
> Key: OAK-7569
> URL: https://issues.apache.org/jira/browse/OAK-7569
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: api, blob-cloud, blob-cloud-azure, blob-plugins
>Reporter: Matt Ryan
>Assignee: Matt Ryan
>Priority: Major
>
> Provide a direct binary access feature to Oak which allows an authenticated
> client to create or download blobs directly to/from the blob store, assuming
> the authenticated user has appropriate permission to do so. The primary value
> of this feature is that the I/O of transferring large binary files to or from
> the blob store can be offloaded entirely from Oak and performed directly
> between a client application and the blob store.
> This feature is described in more detail [on the Oak
> wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access].
> This feature is similar in functionality to OAK-6575. It adds the capability
> to also upload directly to storage via preauthorized URLs in addition to
> downloading directly from storage via preauthorized URLs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[
https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16558110#comment-16558110
]
Michael Dürig commented on OAK-7569:
Looking at the changes of {{oak-jcr}} and {{oak-store-spi}} in
https://github.com/mattvryan/jackrabbit-oak/tree/direct-binary-access-v2 . this
looks very good to me now! As a minor point I noticed
{{SessionDelegate.safePerformNullable}}, which seems unreferenced now and I
think could be removed again.
> Direct Binary Access
>
>
> Key: OAK-7569
> URL: https://issues.apache.org/jira/browse/OAK-7569
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: api, blob-cloud, blob-cloud-azure, blob-plugins
>Reporter: Matt Ryan
>Assignee: Matt Ryan
>Priority: Major
>
> Provide a direct binary access feature to Oak which allows an authenticated
> client to create or download blobs directly to/from the blob store, assuming
> the authenticated user has appropriate permission to do so. The primary value
> of this feature is that the I/O of transferring large binary files to or from
> the blob store can be offloaded entirely from Oak and performed directly
> between a client application and the blob store.
> This feature is described in more detail [on the Oak
> wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access].
> This feature is similar in functionality to OAK-6575. It adds the capability
> to also upload directly to storage via preauthorized URLs in addition to
> downloading directly from storage via preauthorized URLs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16558102#comment-16558102 ] Marcel Reutegger commented on OAK-7569: --- I noticed quite a performance impact on session login in an OSGi environment with the current state of the proposal in https://github.com/mattvryan/jackrabbit-oak/tree/direct-binary-access-v2 Because of the service lookup in the SessionContext for the BlobAccessProvider, the login time is now roughly 8 times as high as before. I will move the lookup to the repository. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16558099#comment-16558099 ] Marcel Reutegger commented on OAK-7569: --- Here's a quick summary of the current state for this proposal. - The Jackrabbit API changes discussed in OAK-7589 have been added to jackrabbit-api (JCR-4335) and will be released with Jackrabbit 2.17.5 - The Oak API changes discussed in OAK-7603 have been committed to oak-api - Reviews for the remaining part of the proposal (mainly data/blob store related) are pending review by Amit The remaining code changes can be split into two parts. - Changes in oak-jcr and oak-store-spi depend on the new interfaces introduced with Jackrabbit 2.17.5 and can only be committed when the new Jackrabbit release is public - The remaining changes can already be committed before the Jackrabbit release is public, because they depend only on the new Oak API introduced for this feature. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. > This feature is similar in functionality to OAK-6575. It adds the capability > to also upload directly to storage via preauthorized URLs in addition to > downloading directly from storage via preauthorized URLs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7569) Direct Binary Access
[ https://issues.apache.org/jira/browse/OAK-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16519927#comment-16519927 ] Matt Ryan commented on OAK-7569: A pull request for this issue is available [here|https://github.com/apache/jackrabbit-oak/pull/88]. > Direct Binary Access > > > Key: OAK-7569 > URL: https://issues.apache.org/jira/browse/OAK-7569 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: api, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan >Priority: Major > > Provide a direct binary access feature to Oak which allows an authenticated > client to create or download blobs directly to/from the blob store, assuming > the authenticated user has appropriate permission to do so. The primary value > of this feature is that the I/O of transferring large binary files to or from > the blob store can be offloaded entirely from Oak and performed directly > between a client application and the blob store. > This feature is described in more detail [on the Oak > wiki|https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
