Re: Parameters in request get lost when using servlet: protocol
Rice Yeh schrieb: Thank you for your solution. But I still feel weird for a brand new request being created for a called block (servlet), at least for a super block (servlet). Having a look at the source code, each new brand request and response are supposed to be stored in its call frame (actually, this feature is not implemented yet, but api for retrieving these two objects are provided). If such design is intended, why not just treat request and response as part of servlet (block) context. And a way for passing a request's attributes or parameters must be provided. My real opinion is that request should be pass through all servlets invoked, instead of creating a new one. You are right, as far Cocoon's internal blocks with Sitemap servlets are involved. In that case one can optimize the calls and provide the original context (request, sitemap parameters etc.) to the callee. The only problem you have is how to handle the different parameter levels you have: you might want to have global but sometimes only local parameters, eg. when setting some value in a called block, you don't want it to override a parameter with the same name in the calling block. The advantage of the servlet protocol internally is that you can mount any servlet as a block, not only cocoon's sitemap servlet. We for example have mounted the Solr servlets (for indexing and searching with lucene) as normal blocks. This is a nice feature that allows you to manage your complex web application solely with Cocoon and Spring. Alex -- Alexander Klimetschek http://www.mindquarry.com
Re: [result][vote] Move CachingSource to cocoon-core
Reinhard Poetz wrote:
Reinhard Poetz wrote:
Because of dependencies on the event-cache block, the caching source
was added to the repository block when it moved out from scratchpad.
After some refactorings I should be able now to move it to cocoon-core
without having to add any new depenendencies there.
As a caching source is of general interest for many of our users (see
several requests on the users list) I want to propose to move it to
cocoon-core.
The proposal has been accepted. It got 7 binding +1 votes and no -1. I
will move the CachingSource to core as soon as time permits.
Done. I also removed all depenencies on Avalon. The CachingSourceFactory can be
configured as Spring bean now. I added two default configurations cached and
async-cached which both inherit from the abstract configuration
org.apache.excalibur.source.SourceFactory/cached::abstract which is also a
good starting point for custom configurations (e.g. you want to switch the used
cache or the validity strategy).
The tests run through but I don't know what the coverage is. I guess not to
high. Unfortunatly it isn't particularily easy to write tests for the async
mode. Since I need the CachingSource in my own apps very soon, I will hopefully
find any problems caused by my refactorings very soon.
- o -
The only behavioural change that I'm aware of was, that I had to remove the
possibility of inline configurations of the source refresher. The problem is
that Spring doesn't make free-style configurations as simple as Avalon does.
Altough it would be possible to mimick it, it is too much work for something
that can be done using a configuration file IMO.
- o -
There were two reasons why the CachingSourceFactory wasn't already in
cocoon-core: First, it supports InspectableSources and second, it supports
EventAwareCaches.
If you want to use InspectableSources, you can use the
InspectableSourcesAwareCachingSourceFactory* from the repository block.
If you want to use an EventAwareCache you can configure the caching source using
the EventAwareCachingSourceValidityStrategy in combination with a cache that
implements EventAware. The strategy can be found in the event-cache block.
* cool name, isn't it? I know that we can do as good as Spring in inventing
lng class names ;-)
--
Reinhard Pötz Independent Consultant, Trainer (IT)-Coach
{Software Engineering, Open Source, Web Applications, Apache Cocoon}
web(log): http://www.poetz.cc
Servlet Services + Shielding
Hi Servlet Hackers, I am switching from our old block servlet config to the current servlet services in trunk and have problems with the configuration. I have two problems: 1) I changed the spring configuration to look like the following snippet, but I get an exception (see below). Debugging showed that the embeddedServlet field in ServletFactoryBean is null. But it should be of type o.a.c.sitemap.SitemapServlet, the declared class of the bean?? bean id=com.mindquarry.dforms.dforms-block class=org.apache.cocoon.sitemap.SitemapServlet servlet:context mount-path=/dforms context-path=blockcontext:/mindquarry-dforms/ servlet:connections entry key=resources value-ref=com.mindquarry.webapp.resources-block / entry key=teams value-ref=com.mindquarry.teamspace.teamspace-block / entry key=tasks value-ref=com.mindquarry.tasks.tasks-block / /servlet:connections /servlet:context /bean Caused by: org.springframework.aop.framework.AopConfigException: Can't proxy null object at org.springframework.aop.framework.ProxyFactory.init(ProxyFactory.java:49) at org.apache.cocoon.servletservice.spring.ServletFactoryBean.getObject(ServletFactoryBean.java:194) at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectFromFactoryBean(AbstractBeanFactory.java:1211) 2) How to configure the ShieldingServletService (my little baby...)? Since you no longer explicitly define the wrapper class for the servlet, I don't see a way to use the ShieldingServletService instead of the standard one. I think those questions and the answers should be documented. As long as the docs are not finished (and public and searchable...), I'd like to put all important information in the Wiki: http://wiki.apache.org/cocoon/ServletService Alex -- Alexander Klimetschek http://www.mindquarry.com
RE: StoreJanitor (was: Re: Moving reduced version of CachingSource to core | Configuration issues)
Hello, Ard Schrijvers wrote: i would be glad to share the code and my ideas, for example about this whole StoreJanitor idea :-) ) Just curious, what did you mean by this whole StoreJanitor idea? Before I say things that are wrong, please consider that the StoreJanitor was invented long before I looked into the cocoon code, so probably a lot of discussion and good ideas has been around which I am not aware of. But still, my ideas about the StoreJanitor (and sorry for the long mail, but perhaps it might contain something useful): 1) How it works and its intention (I think :-) ): The StoreJanitor is originally invented to monitor cocoon's memory useage and does this by checking some memory values every X (default 10) seconds. Beside the fact that I doubt users know that it is quite important to configure the store janitor correctly, I stick to the defaults and use a heapsize of just a little lower then JVM maxmemory. Now, every 10 seconds, the StoreJanitor does a check wether (getJVM().totalMemory() = getMaxHeapSize() (getJVM().freeMemory() getMinFreeMemory()) is true, and if so, the next store is choosen (compared to previoud one) and entries are removed from this store (I saw a post that in trunk not one single store is chosen anymore, but an equal part of all of them is being removed, right?...probably you can configure which stores to use, i don't know) 2) My Observations: When running high traffic sites and render them live (only mod_cache in between which holds pages for 5 to 10 min) like [1] or [2], then checking every X sec for a JVM to be low on memory doesn't make sense to me. At the moment of checking, the JVM might be perfectly sound but just needed some extra memory for a moment, in that case, the Store Janitor is removing items from cache while not needed. Also, when the JVM is really in trouble, but the Store Janitor is not checking for 5 more secthis might be too long for a JVM in a high traffic site when it is low on memory. Problems that result from it are: - Since there is no way to remove cache entries from the used cache impl by the cache's eviction policy, the cache entries from memory are removed by starting from entry 0, whatever this might be in the cache. There is a very likely situation, that at the very next request, the same cache entries are added again. - Ones the JVM gets low on memory, and the StoreJanitor is needed, it is quite likely that from that moment on, the StoreJanitor runs *every* 10 seconds, and keeps removing cache entries which you perhaps don't want to be removed, like compiled stylesheets. 1) suppose, from one store (or since trunk from multiple stores) 10% (default) is removed. This 10% is from the number of memory cache entries. I quite frequently happen to have only 200 entries in memory for each store ( I have added *many* different stores to enable all we wanted in a high traffic environment) and the rest is disk store. Now, suppose, the JVM which has 512 Mb of memory, is low on memory, and removes 10% of 200 entries = 20 entries, helping me zero! These memory entries are my most important ones, so, on the next request, they are either added again, or, from diskcache I have a hit, implying that the cache will put this cache entry in memory again. If I would use 2000 memory items, I am very sure, the 200 items which are cleaned are put back in memory before the next StoreJanitor runs. 2) I am not sure if in trunk you can configure wether the StoreJanitor should leave one store alone, like the DefaultTransientStore. In this store, typically, compiled stylesheets end up, and i18n resource bundles. Since these files are needed virtually on every request, I had rather not that the StoreJanitor removes from this store. I think, the StoreJanitor does so, leaving my critical app in an even worse state, and on the next request, the hardly improved JVM needs to recompile stylesheets and i18n resource bundles. 3) What if the JVM being low is not because of the storesFor example, you have added some component which has some problems you did not know, and, that component is the real reason for you OOM. The StoreJanitor, sees your low memory, and starts removing entries from your perfectly sound cache, leaving you app in a much worse situation then it already was. Your component with memory leak has some more memory it now can fill, and hapily does this, making the StoreJanitor remove more and more entries from cache, untill it ends up with an empty cache. You could blame the wrong component for this behavior. One of these wrong components in use is the event registry for event caching, which made our high traffic sites with 512 Mb crash every two days. Better that I write in another mail what I did to the event cache registry, why I did not yet post about it, and if others are interested and how to include it in the trunk. Bottom line is that there was a
RE: StoreJanitor (was: Re: Moving reduced version of CachingSource to core | Configuration issues)
/snip ?? my mail got sended by accident :Sfinishing it now be implemented quite easily, but might take long start up times) 6) JCSCache has a complex configuration IMO. Therefor, I added default configurations to choose from, for example: store logger=core.store parameter name=region-name value=store/ parameter name=size value=small/ where size might be small, medium, large or huge. I think we have created with in this way a setup for cocoon, where it is harder for unexperienced users to have memory problems when trying to implement larger sites. Hopefully somebody read my mail until here :-) I am curious about what others think, Ard
Re: Servlet Services + Shielding
Further debugging shows that the method ServletFactoryBean.getObject() is
called by Spring (DefaultListableBeanFactory.getObjectFromFactoryBean(...))
*before* the embeddedServlet or any other property was set. So
embeddedServlet is null, which makes the code inside getObject() fail:
public Object getObject() throws Exception {
ProxyFactory proxyFactory = new ProxyFactory(this.embeddedServlet);
proxyFactory.addAdvice(new ServiceInterceptor());
if (this.mountPath != null) {
proxyFactory.addAdvisor(new MountableMixinAdvisor());
}
proxyFactory.addAdvisor(new ServletServiceContextMixinAdvisor());
return proxyFactory.getProxy();
}
We have lots of spring config files now, so I don't see if we have any
special config which triggers this problem. Any idea?
Alex
Alexander Klimetschek schrieb:
1)
I changed the spring configuration to look like the following snippet,
but I get an exception (see below). Debugging showed that the
embeddedServlet field in ServletFactoryBean is null. But it should be of
type o.a.c.sitemap.SitemapServlet, the declared class of the bean??
bean id=com.mindquarry.dforms.dforms-block
class=org.apache.cocoon.sitemap.SitemapServlet
servlet:context mount-path=/dforms
context-path=blockcontext:/mindquarry-dforms/
servlet:connections
entry key=resources
value-ref=com.mindquarry.webapp.resources-block /
entry key=teams
value-ref=com.mindquarry.teamspace.teamspace-block /
entry key=tasks
value-ref=com.mindquarry.tasks.tasks-block /
/servlet:connections
/servlet:context
/bean
Caused by: org.springframework.aop.framework.AopConfigException: Can't
proxy null object
at
org.springframework.aop.framework.ProxyFactory.init(ProxyFactory.java:49)
at
org.apache.cocoon.servletservice.spring.ServletFactoryBean.getObject(ServletFactoryBean.java:194)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getObjectFromFactoryBean(AbstractBeanFactory.java:1211)
--
Alexander Klimetschek
http://www.mindquarry.com
Circular deps in servlet connections (was Re: Servlet Services + Shielding)
Oh, I got it: there was a circular dependency in the servlet connections (A using B, B using A as super). It did work with the old block servlets, but now you get the very unhelpful exception. I don't think we actually need this circular connection in particular. But generally, two solutions are available: Is it possible to allow circular dependencies in the servlet connections? If not, there should be at least some kind of check that prints a better message. Alex -- Alexander Klimetschek http://www.mindquarry.com
Re: Circular deps in servlet connections (was Re: Servlet Services + Shielding)
Alexander Klimetschek napisał(a): Oh, I got it: there was a circular dependency in the servlet connections (A using B, B using A as super). It did work with the old block servlets, but now you get the very unhelpful exception. I don't think we actually need this circular connection in particular. But generally, two solutions are available: Is it possible to allow circular dependencies in the servlet connections? If not, there should be at least some kind of check that prints a better message. My opinion is that circular dependencies should not be allowed and I agree that more meaningful exception is needed here. Could you please create an issue for this? Providing a patch would be even more appreciated. If you are going to work on it please let me know to not duplicate the effort. -- Grzegorz Kossakowski
[jira] Created: (COCOON-2036) Handle circular dependencies in servlet connections
Handle circular dependencies in servlet connections
---
Key: COCOON-2036
URL: https://issues.apache.org/jira/browse/COCOON-2036
Project: Cocoon
Issue Type: Bug
Components: - Servlet service framework
Affects Versions: 2.2-dev (Current SVN)
Reporter: Alexander Klimetschek
Circular dependencies in servlet connections lead to a Spring exception [1] in
[2] that does not provide any help. The previous implementation (block:) did
allow circular dependencies because they were not handled by spring but by
custom code.
Solution would be either to allow them (probably difficult to implement with
spring) or, if not, to provide a helpful warning message, that skips this
problem. The latter could be a check for embeddedServlet==null and, if not,
throw an exception saying you might have a circular dependency in
servlet-foobar.
---
[1]:
The exception is thrown after ServletFactoryBean.getObject() tries to create a
proxy for the embeddedServlet, which is null in the case of a circular
dependency (one of the circle endpoints is created, but the other will be null).
Caused by: org.springframework.aop.framework.AopConfigException: Can't proxy
null object
at
org.springframework.aop.framework.ProxyFactory.init(ProxyFactory.java:49)
at
org.apache.cocoon.servletservice.spring.ServletFactoryBean.getObject(ServletFactoryBean.java:194)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getObjectFromFactoryBean(AbstractBeanFactory.java:1211)
[2]:
public Object getObject() throws Exception {
ProxyFactory proxyFactory = new ProxyFactory(this.embeddedServlet);
proxyFactory.addAdvice(new ServiceInterceptor());
if (this.mountPath != null) {
proxyFactory.addAdvisor(new MountableMixinAdvisor());
}
proxyFactory.addAdvisor(new ServletServiceContextMixinAdvisor());
return proxyFactory.getProxy();
}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: Circular deps in servlet connections (was Re: Servlet Services + Shielding)
Grzegorz Kossakowski schrieb: My opinion is that circular dependencies should not be allowed and I agree that more meaningful exception is needed here. We do need the circular dependency: the basic principle is a super block that runs cforms that are mainly defined in a child block - the model lies in the child block and it contains a selection-list whose source is block:child:/some/matcher - hence the circle, because the selection-list will be resolved inside the super block. Could you please create an issue for this? Providing a patch would be even more appreciated. If you are going to work on it please let me know to not duplicate the effort. https://issues.apache.org/jira/browse/COCOON-2036 How could one solve this with spring? Does spring have a mechanism for setting circular bean references? Alex -- Alexander Klimetschek http://www.mindquarry.com
Re: StoreJanitor
Ard Schrijvers wrote: Before I say things that are wrong, please consider that the StoreJanitor was invented long before I looked into the cocoon code, so probably a lot of discussion and good ideas has been around which I am not aware of. But still, my ideas about the StoreJanitor (and sorry for the long mail, but perhaps it might contain something useful): 1) How it works and its intention (I think :-) ): The StoreJanitor is originally invented to monitor cocoon's memory useage and does this by checking some memory values every X (default 10) seconds. Beside the fact that I doubt users know that it is quite important to configure the store janitor correctly, I stick to the defaults and use a heapsize of just a little lower then JVM maxmemory. Now, every 10 seconds, the StoreJanitor does a check wether (getJVM().totalMemory() = getMaxHeapSize() (getJVM().freeMemory() getMinFreeMemory()) is true, and if so, the next store is choosen (compared to previoud one) and entries are removed from this store (I saw a post that in trunk not one single store is chosen anymore, but an equal part of all of them is being removed, right?...probably you can configure which stores to use, i don't know) AFAICS there are two freeing algorithms in trunk: round-robin and all-stores. 2) My Observations: When running high traffic sites and render them live (only mod_cache in between which holds pages for 5 to 10 min) like [1] or [2], then checking every X sec for a JVM to be low on memory doesn't make sense to me. At the moment of checking, the JVM might be perfectly sound but just needed some extra memory for a moment, in that case, the Store Janitor is removing items from cache while not needed. Also, when the JVM is really in trouble, but the Store Janitor is not checking for 5 more secthis might be too long for a JVM in a high traffic site when it is low on memory. Problems that result from it are: - Since there is no way to remove cache entries from the used cache impl by the cache's eviction policy, the cache entries from memory are removed by starting from entry 0, whatever this might be in the cache. There is a very likely situation, that at the very next request, the same cache entries are added again. - Ones the JVM gets low on memory, and the StoreJanitor is needed, it is quite likely that from that moment on, the StoreJanitor runs *every* 10 seconds, and keeps removing cache entries which you perhaps don't want to be removed, like compiled stylesheets. yep, that's a problem 1) suppose, from one store (or since trunk from multiple stores) 10% (default) is removed. This 10% is from the number of memory cache entries. I quite frequently happen to have only 200 entries in memory for each store ( I have added *many* different stores to enable all we wanted in a high traffic environment) and the rest is disk store. Now, suppose, the JVM which has 512 Mb of memory, is low on memory, and removes 10% of 200 entries = 20 entries, helping me zero! agreed These memory entries are my most important ones, so, on the next request, they are either added again, or, from diskcache I have a hit, implying that the cache will put this cache entry in memory again. If I would use 2000 memory items, I am very sure, the 200 items which are cleaned are put back in memory before the next StoreJanitor runs. 2) I am not sure if in trunk you can configure wether the StoreJanitor should leave one store alone, like the DefaultTransientStore. In this store, typically, compiled stylesheets end up, and i18n resource bundles. Since these files are needed virtually on every request, I had rather not that the StoreJanitor removes from this store. I think, the StoreJanitor does so, leaving my critical app in an even worse state, and on the next request, the hardly improved JVM needs to recompile stylesheets and i18n resource bundles. agreed 3) What if the JVM being low is not because of the storesFor example, you have added some component which has some problems you did not know, and, that component is the real reason for you OOM. The StoreJanitor, sees your low memory, and starts removing entries from your perfectly sound cache, leaving you app in a much worse situation then it already was. Your component with memory leak has some more memory it now can fill, and hapily does this, making the StoreJanitor remove more and more entries from cache, untill it ends up with an empty cache. You could blame the wrong component for this behavior. One of these wrong components in use is the event registry for event caching, which made our high traffic sites with 512 Mb crash every two days. Better that I write in another mail what I did to the event cache registry, why I did not yet post about it, and if others are interested and how to include it in the trunk. Bottom line is that there was a major OOM problem if the registry grows, resulting in a StoreJanitor
Re: [release] Prepare RC1 from trunk (core + many blocks + achetypes + rcl-maven-plugin)
Reinhard Poetz wrote: The limiting factor is that nobody has done it yet. It would be great if you create the artifact and put it into the Apache snapshot repo. Then we have some time to test it and if everything works fine, Marc will I've created a 1.3-SNAPSHOT off [1] and put it in m1-snapshot-repository. I've tested this locally and it seems to be OK. You can test it by modifying the cforms-impl pom : Index: pom.xml === --- pom.xml (revision 525249) +++ pom.xml (working copy) @@ -129,7 +129,7 @@ dependency groupIdxreporter/groupId artifactIdxreporter-expression/artifactId - versionr683/version + version1.3-SNAPSHOT/version /dependency dependency groupIdjunit/groupId Please have a look and let me know in case of problems. Regards Jorg [1] http://svn.cocoondev.org/viewsvn/trunk/xreporter/?rev=684root=xreporter
Re: StoreJanitor
Reinhard Poetz wrote:
P.S. Ard, answering to your mails is very difficult because there are no
line breaks. Is anybody else experiencing the same problem or is it only
me?
Jörg pointed me to the rewrap function of Thunderbird. Using it fixes all my
problems with never ending lines. Thanks Jörg!
--
Reinhard Pötz Independent Consultant, Trainer (IT)-Coach
{Software Engineering, Open Source, Web Applications, Apache Cocoon}
web(log): http://www.poetz.cc
Re: Circular deps in servlet connections (was Re: Servlet Services + Shielding)
Alexander Klimetschek napisał(a): Grzegorz Kossakowski schrieb: We do need the circular dependency: the basic principle is a super block that runs cforms that are mainly defined in a child block - the model lies in the child block and it contains a selection-list whose source is block:child:/some/matcher - hence the circle, because the selection-list will be resolved inside the super block. Let me tell you how i *imagine* servlet-service-fw should deal super calls and tell me how it's different from current situation. This should help us both. Let's have two blocks, one named base and one extending, base is super block for extending. I've imagined (before using super calls) that it would work that way: extending block is requested for someResource and suppose it has not any matching pattern for it, then super (base in our case) block is automatically asked for the same. You don't have to provide any generic matcher that redirects processing to super block. Now someResource is catched by base block but it asks for internalResource by calling servlet:internalResource. Given that base block is called as super of extending, first extending block is asked for this resource (because it may override it) and if it does not provide one, it's taken from base block. In short: I mean that super should work as fallback mechanism. I guess it does not work that way now. However, this would eliminate need for circular dependencies and give cleaner design. What do you think? -- Grzegorz Kossakowski
