Hi, Am Montag, den 22.11.2010, 08:36 +0000 schrieb Ian Boston: > On 22 Nov 2010, at 02:17, Justin Edelson wrote: > > > It should *not* be the case that requesting > > /system/userManager/user/<userid>.json causes listChildren() to be > > called. If it does, that's a bug (but eyeballing the code, this doesn't > > seem possible). Requesting /system/userManager/user.2.json will cause > > listChildren() to be called. > > > at > org.sakaiproject.nakamura.lite.authorizable.AuthorizableManagerImpl.findAuthorizable(AuthorizableManagerImpl.java:290) > at > org.sakaiproject.nakamura.api.lite.jackrabbit.SparseMapUserManager.findAuthorizables(SparseMapUserManager.java:110) > at > org.apache.jackrabbit.core.security.principal.DynamicPrincipalProvider.findUserPrincipals(DynamicPrincipalProvider.java:346) > at > org.apache.jackrabbit.core.security.principal.DynamicPrincipalProvider.findPrincipals(DynamicPrincipalProvider.java:194) > at > org.apache.jackrabbit.core.security.principal.DynamicPrincipalProvider.getPrincipals(DynamicPrincipalProvider.java:217) > at > org.apache.jackrabbit.core.security.principal.PrincipalManagerImpl.getPrincipals(PrincipalManagerImpl.java:114) > at > org.sakaiproject.nakamura.user.resource.SakaiAuthorizableResourceProvider.listChildren(SakaiAuthorizableResourceProvider.java:182) > at > org.apache.sling.jcr.resource.internal.helper.WrappedResourceProvider.listChildren(WrappedResourceProvider.java:65) > at > org.apache.sling.jcr.resource.internal.helper.ResourceProviderEntry$1.seek(ResourceProviderEntry.java:206) > at > org.apache.sling.jcr.resource.internal.helper.ResourceProviderEntry$1.<init>(ResourceProviderEntry.java:179) > at > org.apache.sling.jcr.resource.internal.helper.ResourceProviderEntry.listChildren(ResourceProviderEntry.java:145) > at > org.apache.sling.jcr.resource.internal.JcrResourceResolver.listChildren(JcrResourceResolver.java:819) > at > org.apache.sling.jcr.resource.internal.JcrResourceResolver.getChildInternal(JcrResourceResolver.java:1101) > at > org.apache.sling.jcr.resource.internal.JcrResourceResolver.resolveInternal(JcrResourceResolver.java:1036) > at > org.apache.sling.jcr.resource.internal.JcrResourceResolver.resolve(JcrResourceResolver.java:463) > at > org.apache.sling.engine.impl.request.RequestData.initResource(RequestData.java:196) > at > org.apache.sling.engine.impl.SlingMainServlet.service(SlingMainServlet.java:302) > at > org.apache.sling.engine.impl.SlingMainServlet.service(SlingMainServlet.java:207) > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:389) > > > the SakaiAuthorizableRespourceProvider.listChildren method is identical to > the standard AuthrizableResourceProvider > > > > > In terms of your larger point, ResourceProvider.listChildren() returns > > an Iterator and AuthorizableResourceProvider.listChildren() return value > > simply wraps the result of getPrincipals(), so in and of itself, this > > shouldn't exhaust memory. > > Agreed, and it looks like the iterator is not used, so the only expense is > executing the query which provided the lucene/jackrabbit batch size is small > and the result set is dense from an ACL point of view there should not be a > problem. However if the result set is sparse, and the first entry is > retrieved from the iterator that will cause many items to enter the shared > item cache and block all other operations that need the persistence manager.
But how can we solve this in Sling ? This sounds like a Jackrabbit problem ? Regards Felix > > > > > > > What is probably necessary is to put some guard in place so that only X > > number of items are returned and if more than X results are available, > > support some kind of paging (I don't know what X should be). This would > > be similar to SLING-1308. Actually, why isn't the code from SLING-1308 > > being invoked in your case? That seems very suspicious. > > I have a feeling that infinity on the parent path will invoke listChildren > without limit, which will exercise the iterator. > > > > > Sorry to hear about your bad release. > > One of the dangers of making a platform too flexible and releasing it to > inexperienced hands, there were some hugely complex JCR Queries written that > had no hope of scaling to more than a few users or items of data, but I also > think that our use cases are not well aligned with those of Jackrabbit and > Enterprise Content Management in general. > > Ian > > > > > > > > Justin > > > > > > On 11/21/10 4:05 PM, Ian Boston wrote: > >> Hi, > >> In AuthorizableResourceProvider there is a call to > >> principalManager.getPrincipals(searchType); > >> > >> IIUC that lists all authorizables that match the search type. Eg, all > >> users or all groups or all authorizables. > >> > >> This is probably Ok with 100 users, but not so ok with 25K users and 40K > >> groups, as its called every time to do a call to > >> http://localhost:8080/system/userManager/user/<userid>.json > >> > >> Did I understand correctly or am I reading the code incorrectly ? > >> > >> I think the stack trace is something like > >> AuthorizableResourceProvider.listChildren > >> PrincipalManagerImpl.getPrincipals > >> DefaultPrincipalProvider.getPrincipals > >> DefaultPrincipalProvider.findPrincipals with a null search filter (ie all > >> the known Principals matching that type) > >> > >> Obviously AuthorizableResourceProvider.listChildren *should* list all > >> Authorizables, but since in most systems that identify the users, there > >> could be millions, this is probably not a good idea. > >> > >> AuthorizableResourceProvider should probably not list children as doing so > >> will almost certainly crash the server for anything other than a trivial > >> case. > >> > >> WDYT? > >> Ian > >> > >> > >> BTW, I must apologise for not doing much work on Sling in the last month, > >> our release did not go smoothy and I have been firefighting. (fire still > >> smouldering) > >> > > >
