[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780986#comment-17780986 ] Carsten Ziegeler commented on SLING-10899: -- Sounds ok to me, however I think we should add a kill switch to disable in case it creates problems somewhere. > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Priority: Major > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776179#comment-17776179 ] Henry Kuijpers commented on SLING-10899: Do we want to proceed with reusing the JcrValueMap throughout various getValueMap/adaptTo(ValueMap.class) calls on the *same* Resource object? It would be a very nice improvement in terms of performance. My vote: +1 > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Priority: Major > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771007#comment-17771007 ] Henry Kuijpers commented on SLING-10899: I agree! That way, the JcrValueMap's cache is actually useful: You have the same ValueMap across multiple Filter / Servlet / .. calls. > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Priority: Major > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770994#comment-17770994 ] Carsten Ziegeler commented on SLING-10899: -- I looked at some other resource implementations and there the value map is actually cached inside the resource object. maybe it is ok to make this change and cache the value map within the resource and always return the same. > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Priority: Major > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770714#comment-17770714 ] Carsten Ziegeler commented on SLING-10899: -- Throwing some random ideas in here: * it should be possible to add caching via a ResourceDecorator after the fact. With this an app could decide to go this route without any changes in Sling * we could think of adding a new method to Resource which will return a shared ValueMap across all resource objects for the same path within the same resource resolver. (in hindsight getValueMap() should have been that method, but that's too late now). > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Priority: Major > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770499#comment-17770499 ] Carsten Ziegeler commented on SLING-10899: -- Memory consumption is one thing - that can most likely be solved - but change in behaviour is risky. I'm pretty sure that there will be users of Sling which expect that two calls to getResource on the same resolver with the same path return two different objects. And I'm also sure the same is true for getting the value map. So we can't simply change that. One way out seems to be to create new methods to introduce the caching behaviour - but that obviously only works if you then change all usage to the new methods - and here again, I'm sure that there are places where you can't do this. I think the Filter use case could be solved (for example by introducing a method on the request object), but I assume that is only one out of many > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Priority: Major > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770465#comment-17770465 ] Henry Kuijpers commented on SLING-10899: [~joerghoh] [~cziegeler] There is no caching of the adaption, unfortunately, and it's also not possible to do this outside of all the implementations: Whenever ResourceResolver#getResource is called, a new Resource object is returned. Never an existing one(? unless I'm missing something). It would be up to ResourceResolverImpl to return the same one under certain conditions, if I have that right. This would definitely be a nice upgrade (and I agree also a complicated one), as right now, calling getResource() on the same instance of ResourceResolver, will always return a new fresh Resource (which can be any impl, of course). In the case of a JCR Resource, a JcrNodeResource is returned. This JcrNodeResource has 2 ways of obtaining a ValueMap: * getValueMap * adaptTo(ValueMap.class) The getValueMap (coming from AbstractResource) calls adaptTo(ValueMap.class). adaptTo(ValueMap.class) will *always* return a new instance of JcrValueMap, as you can see here: https://github.com/apache/sling-org-apache-sling-jcr-resource/blame/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 It is not possible to influence this, as we have no influence over these internal helper classes and/or any of the other impls. So, this means that: 1. The ResourceResolverImpl is returning a fresh new JcrNodeResource object for the same path *all the time* 2. The JcrNodeResource is returning a fresh new JcrValueMap for every call to getValueMap/adaptTo(ValueMap.class) *all the time* 3. The cache that resides in JcrValueMap is of almost no use, as it will most often have a cache miss. 4. It is not very easily possible to create a wrapper around such resources (because, where would we create them)? 5. Creating a wrapping that could cache the valuemap is not possible, because there are quite a few checks for "instanceof", which then isn't true anymore A few very easy improvements could be: 1. When obtaining the ValueMap for a JcrNodeResource, create the map and save it in an instance field, when calling it the next time, just return the instance field, instead of creating a new JcrValueMap - This will of course all go away when the Resource-object is garbage collected 2. Do not obtain the ValueMap in the adaptTo-method in the Resource impls, but instead, use AdapterFactory for that, but, put some kind of caching mechanism in there (basically a cache that could be similar to the Sling Models cache that is in place) 3. Let the ValueMap-cache (and vielleicht other related stored data) live together with the lifetime of the ResourceResolver, so as soon as it is closed, also remove all the caches In the improvements I don't see something so risky that it could blow the JVM and/or cause too much memory to be claimed or similar. As long as it is implemented well enough. :) About doing the adaption repeatedly: We have no choice! If I have a Filter implementation that needs to access the ValueMap of a Resource, I *need* to call getValueMap and/or adaptTo(ValueMap.class). Both will have their own code inside JcrNodeResource, that bypasses any AdapterFactory or anything else where caching can take place. And *they do not cache anything*. If the next Filter comes, and it has to access the ValueMap too, then the same logic is kicked off and yet another new JcrValueMap (with empty cache) is created, instead of reusing the previous one. It is impossible to obtain a resource only once and reuse it throughout the entire application and all the various phases. Looking at every location independently is totally fine. I know most of the locations that are interesting. But they always involve Sling/Adobe/... code, mainly filters etc etc. And also our own code, but we have no way to pass those Resource objects (and especially their JcrValueMap objects) around. The problem is really inside JcrNodeResource and JcrValueMap. Please have another look at it, based on this summary of the problem. > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Priority: Major > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770344#comment-17770344 ] Carsten Ziegeler commented on SLING-10899: -- Re-reading the ticket, I think it will be dangerous to introduce a change in behaviour. This is better handled on the client side. The risk of breaking existing clients is imho too high > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Priority: Major > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770335#comment-17770335 ] Joerg Hoh commented on SLING-10899: --- [~Henry Kuijpers] I see the problem with the caching of the results of this adaption, as the cache must be limited in size (e.g 100 elements or 500k) , otherwise it can easily blow the JVM. This naturally limits the effectiveness, because depending on the access patterns and the cache eviction strategy you benefit from it only when you access a few resources constantly. Or if your code access only a small amount of resources at a time. I would say that in both cases you as the consumer of the API are in a much better position to decide if you want to do that adaption repeatedly and if it makes sense to cache the resulting ValueMaps in an application-layer cache and thus avoid them. And yes, this definitely makes it much harder to optimize performance, because you need to look at each location independently, and there is no immediate performance boost for each functionality doing such an adaption. But in my opinion the effectiveness of such a cache is not guaranteed, and the memory usage can be significant. > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Priority: Major > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769540#comment-17769540 ] Henry Kuijpers commented on SLING-10899: [~joerghoh] This is the ticket I mentioned at adaptTo(), when you did your session about AEM Backend performance. The thing is: Everytime Resource#adaptTo is called with `ValueMap.class` is parameter, a new JcrValueMap (which has a cache of type Map) is created and returned. Therefore, the cached entries inside that ValueMap are not of much use. As you know, it is not so easy to reuse the same ValueMap in all pieces of the application (as you can only obtain a ValueMap inside your piece of code by doing request.getResource().getValueMap() or request.getResource().adaptTo(ValueMap.class)). This will always yield a new ValueMap, instead of reusing the same one (and reusing the cache, making the cache actually useful). > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Priority: Major > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437282#comment-17437282 ] Carsten Ziegeler commented on SLING-10899: -- The resource resolver does not know anything about the adaptions - if we want to improve we probably would need to change the resource resolver to keep track of all resources it handed out and ensure that it returns the same resource object. That alone is already a heavy change which can lead to problems. Now, once we have the same resource object, we need to deliver the same Modifiable(ValueMap) on adaption. We could of course solve the second one without the first one by just keeping track in jcr resource. But then a refresh is still not catched - again with keeping track of everything we might be able to handle it. Then comes the problem of the deep value maps acting across resources, this is where we most likely will loose And then there is so much code doing modifications directly via the JCR API (sometimes mixed with resource API usage), and thats where we have to give up. Or in other words, I don't think that we can easily improve this area without creating new problems - and that is most likely the reason why the code is how it is today. > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Assignee: Carsten Ziegeler >Priority: Major > Fix For: JCR Resource 3.0.24 > > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437268#comment-17437268 ] Henry Kuijpers commented on SLING-10899: [~cziegeler], I see. However, the performance impact is quite drastic. Not only in our very specific situation, but also in the Sling product itself: With every filter, with every servlet, with every OSGi service, ... accessing the valuemap of the Resource, a new one is created and therefore, fetching properties from it will always reach out to the repository. ValueMap objects are not shared in general, not because technically it's impossible to do so (if you have the reference, you can just use it), but rather because the various APIs etc get a Resource-object to work with (and therefore you will always have to call getValueMap (or adaptTo) to obtain it). I think the impact on performance with this improvement will be quite noticeable. Maybe the ResourceResolver code can be updated, so that it can track resources that have been adapted to a ModifiableValueMap? So that, on commit, it can invalidate the internal caches? > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Assignee: Carsten Ziegeler >Priority: Major > Fix For: JCR Resource 3.0.24 > > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437202#comment-17437202 ] Carsten Ziegeler commented on SLING-10899: -- We don't do anything special for commit/refresh today. The problem exists regardless whether we cache the value map or not. Granted, with caching the value map, the problem might occur more often. But it can already occur today. So we have a couple of problems: there might be stale value maps - but we also support deep tree access/modifications through (modifiable) value maps - which allows for cross resource changes. I'm wondering now if it is really worth caching the value map as it seems to create more problems than it actually solves? In theory we could return the same object for value map and modifiable value map and just make it immutable for value map. This way changes to the modifiable map would be visibile to the read-only value map. But I'Ve also seen code which actually alters a value map and expects that the changes do not end up persisted. So we would break that (rather strange) use case. > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Assignee: Carsten Ziegeler >Priority: Major > Fix For: JCR Resource 3.0.24 > > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437195#comment-17437195 ] Joerg Hoh commented on SLING-10899: --- I am not sure if it's possible to know when the underlying ResourceResolver has performed a commit/refresh and clear the cached ValueMap in that case (if we can handle it on the JCR side, it would be fine as well). > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Assignee: Carsten Ziegeler >Priority: Major > Fix For: JCR Resource 3.0.24 > > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[
https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437160#comment-17437160
]
Carsten Ziegeler commented on SLING-10899:
--
I think we had a lot of discussions around this in the past - if my memory
serves me well, we lazily agreed that an adaption of an object to another
object should return the same instance - if possible. That's why we added the
caching in SlingAdaptable. Of course there are edge cases like adapting to an
InputStream where caching doesn't make sense.
So looking at it from this POV, the current behaviour (returning a new
ValueMap) seems to be a bug - and returning the same ValueMap is more
consistent. For read-only cases this should not matter anyway but improves
performance.
For write cases, it gets more tricky - I'm not 100% sure there - although
having a consistent view of the changed content looks like the way to go, so
again returning the same object seems to be the right thing.
But there is a higher level problem which will not be solved by this - and in
the past we lazily agreed in the past to not solve it: two Resource objects
representing the same resource will result different ValueMap objects - and
therefore (in the write case) potentially different state, like:
{noformat}
Resource a = resolver.getResource("/foo");
ValueMap va = a.adaptTo(ValueMap.class);
Resource b = resolver.getResource("/foo");
ValueMap vb = b.adaptTo(ValueMap.class);
{noformat}
in that case va != vb (same for the modifiable value map)
> Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
> --
>
> Key: SLING-10899
> URL: https://issues.apache.org/jira/browse/SLING-10899
> Project: Sling
> Issue Type: Improvement
> Components: JCR
>Affects Versions: JCR Resource 3.0.22
>Reporter: Henry Kuijpers
>Assignee: Carsten Ziegeler
>Priority: Major
> Fix For: JCR Resource 3.0.24
>
>
> I was performance testing our application. There is a specific part of the
> code where we have a set of ~500 resources and we sort them based on a few
> properties of the resources. Multiple properties are used (which results in
> multiple calls to getValueMap). The sorting is done using a comparator, so
> the logic that determines the order is accessed numerous times.
> We've seen quite a decrease in performance when doing this with Resources
> that are of type JcrNodeResource. Turns out that the result of getValueMap
> (or adaptTo((Value)Map.class)) is not cached.
> As you can see in the adaptTo()-method implementation:
> https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136
> this call bypasses the AdapterFactory framework and also bypasses any
> caching that happens over there.
> What's even more unfortunate, is that JcrValueMap is implementing caching via
> https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49
> . That caching is not leveraged properly, because every call to getValueMap
> returns a new JcrValueMap, instead of reusing a previously created one.
> One change that can be done is maybe converting the logic inside the
> adaptTo()-method to a dedicated AdapterFactory implementation, so that
> caching is done by default?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437156#comment-17437156 ] Henry Kuijpers commented on SLING-10899: One issue I was thinking of is using a modifiablevaluemap or similar, adjusting a few properties, then trying to read those properties again using a previously obtained Resource's ValueMap. In the code before this change, it would return the new and changed properties, because it always made a new ValueMap, now, it doesn't do that anymore. Would that be an issue? Or was the fact that this happened previously, actually a bug? > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Assignee: Carsten Ziegeler >Priority: Major > Fix For: JCR Resource 3.0.24 > > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-10899) Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached
[ https://issues.apache.org/jira/browse/SLING-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437145#comment-17437145 ] Carsten Ziegeler commented on SLING-10899: -- I think it is easier to cache directly in the JcrNodeResource to avoid further indirections. I've applied such a change in https://github.com/apache/sling-org-apache-sling-jcr-resource/commit/7c09c1e99cba3e5545af05028e88aa518f3272d9 > Result of JcrNodeResource#adaptTo(ValueMap.class) should be cached > -- > > Key: SLING-10899 > URL: https://issues.apache.org/jira/browse/SLING-10899 > Project: Sling > Issue Type: Improvement > Components: JCR >Affects Versions: JCR Resource 3.0.22 >Reporter: Henry Kuijpers >Assignee: Carsten Ziegeler >Priority: Major > Fix For: JCR Resource 3.0.24 > > > I was performance testing our application. There is a specific part of the > code where we have a set of ~500 resources and we sort them based on a few > properties of the resources. Multiple properties are used (which results in > multiple calls to getValueMap). The sorting is done using a comparator, so > the logic that determines the order is accessed numerous times. > We've seen quite a decrease in performance when doing this with Resources > that are of type JcrNodeResource. Turns out that the result of getValueMap > (or adaptTo((Value)Map.class)) is not cached. > As you can see in the adaptTo()-method implementation: > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/master/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResource.java#L136 > this call bypasses the AdapterFactory framework and also bypasses any > caching that happens over there. > What's even more unfortunate, is that JcrValueMap is implementing caching via > https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/6e13f4d315ddf2804d2e16c55faee18e805b618e/src/main/java/org/apache/sling/jcr/resource/internal/JcrValueMap.java#L49 > . That caching is not leveraged properly, because every call to getValueMap > returns a new JcrValueMap, instead of reusing a previously created one. > One change that can be done is maybe converting the logic inside the > adaptTo()-method to a dedicated AdapterFactory implementation, so that > caching is done by default? -- This message was sent by Atlassian Jira (v8.3.4#803005)
