[SUMMARY] Pipeline match optimization (Re: [Q] Pipeline best practices)
[EMAIL PROTECTED] wrote: Another issue of having too many pipelines. As i understand in the sitemap is generated into a java class. And the various matchers within a pipeline(s) are really a bunch of if-else if statements. So the more matchers you have (to match to a pipeline or various paths within a pipeline) your going to have a large series of if/else statements. And... since this uses string compares and wildcards and... if there's alot of these if else if statements it will lead to performance issues. It may help to break things up with sub site maps to help limit the searching. Or put the most likely pipelines to get hit first and the least likely last... Yes, and make search hierarchical. Since you can match inside a match, you can also do this. To get to match2-3 you need 8 matches match1-1 | 1 match1-2 | . match1-3 | . match1-4 | . match1-5 | . match2-1 | . match2-2 | . match2-3 v 8 But you can do: match1 | 1 match1-1 ! x match1-2 ! x match1-3 ! x match1-4 ! x match1-5 ! x match2 | . match2-1 | . match2-2 | . match2-3 v 5 And have 5 matches -- Nicola Ken Barozzi [EMAIL PROTECTED] - verba volant, scripta manent - (discussions get forgotten, just code remains) - - Please check that your question has not already been answered in the FAQ before posting. http://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [SUMMARY] Pipeline match optimization (Re: [Q] Pipeline best practices)
The results of this thread from my point of view can be found at http://outerthought.net/wiki/Wiki.jsp?page=DesignPipelines. Are there any other points? Regards, Reinhard -Original Message- From: Nicola Ken Barozzi [mailto:[EMAIL PROTECTED]] Sent: Monday, October 14, 2002 8:57 AM To: [EMAIL PROTECTED] Subject: [SUMMARY] Pipeline match optimization (Re: [Q] Pipeline best practices) [EMAIL PROTECTED] wrote: Another issue of having too many pipelines. As i understand in the sitemap is generated into a java class. And the various matchers within a pipeline(s) are really a bunch of if-else if statements. So the more matchers you have (to match to a pipeline or various paths within a pipeline) your going to have a large series of if/else statements. And... since this uses string compares and wildcards and... if there's alot of these if else if statements it will lead to performance issues. It may help to break things up with sub site maps to help limit the searching. Or put the most likely pipelines to get hit first and the least likely last... Yes, and make search hierarchical. Since you can match inside a match, you can also do this. To get to match2-3 you need 8 matches match1-1 | 1 match1-2 | . match1-3 | . match1-4 | . match1-5 | . match2-1 | . match2-2 | . match2-3 v 8 But you can do: match1 | 1 match1-1 ! x match1-2 ! x match1-3 ! x match1-4 ! x match1-5 ! x match2 | . match2-1 | . match2-2 | . match2-3 v 5 And have 5 matches -- Nicola Ken Barozzi [EMAIL PROTECTED] - verba volant, scripta manent - (discussions get forgotten, just code remains) - - Please check that your question has not already been answered in the FAQ before posting. http://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Please check that your question has not already been answered in the FAQ before posting. http://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [Q] Pipeline best practices
Or put the most likely pipelines to get hit first and the least likely last... That can be problematic if your most used pipelines are the generic matches. Eg, three special cases and 100 general cases: match=fee.foe match=fee.fie match=fee.fum match= fee.* Probably ok with three special cases, but what about 20, or 50...??? - Please check that your question has not already been answered in the FAQ before posting. http://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [Q] Pipeline best practices
To be honest, I circumvented this whole issue by having as few pipelines as possible. Basically I use action or event id's that point to metadata files. The metadata files have all the information how to build a portal page, I then use WSUI (wsui.org) files to describe each of the portlets. So really I only have one pipeline for the portal (external pipeline), one for the portlets (internal pipeline), and some external pipelines to handle gifs, jpegs, css, and js. the sitemap has the additional authentication actions as well as some locale actions so it's a little more compicated than what I described above... So basically I match on a metadata id (using the request parameter matcher), rather than match URL's as I take it that most people do. For that matter I not really matching as my wildcard is really * and instead I'm using a request parameter action. If not found show a 404 message. So my sitemap is probably a 100-150 total lines even though it's serving up 500-600 dynamic pages composed of approximately 700 or so portlets. so it's quite lean and mean. MD In a message dated 10/14/2002 11:50:17 AM Eastern Standard Time, [EMAIL PROTECTED] writes: Or put the most likely pipelines to get hit first and the least likely last... That can be problematic if your most used pipelines are the generic matches. Eg, three special cases and 100 general cases: match=fee.foe match=fee.fie match=fee.fum match= fee.* Probably ok with three special cases, but what about 20, or 50...??? - Please check that your question has not already been answered in the FAQ before posting. TITLE=http://xml.apache.org/cocoon/faq/index.html TARGET=_blankhttp://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Please check that your question has not already been answered in the FAQ before posting. http://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [Q] Pipeline best practices
Can you share your pipeline. Maybe you organize your content in a much smarter way than I can imagine. - Original Message - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Monday, October 14, 2002 3:59 PM Subject: Re: [Q] Pipeline best practices To be honest, I circumvented this whole issue by having as few pipelines as possible. Basically I use action or event id's that point to metadata files. The metadata files have all the information how to build a portal page, I then use WSUI (wsui.org) files to describe each of the portlets. So really I only have one pipeline for the portal (external pipeline), one for the portlets (internal pipeline), and some external pipelines to handle gifs, jpegs, css, and js. the sitemap has the additional authentication actions as well as some locale actions so it's a little more compicated than what I described above... So basically I match on a metadata id (using the request parameter matcher), rather than match URL's as I take it that most people do. For that matter I not really matching as my wildcard is really * and instead I'm using a request parameter action. If not found show a 404 message. So my sitemap is probably a 100-150 total lines even though it's serving up 500-600 dynamic pages composed of approximately 700 or so portlets. so it's quite lean and mean. MD In a message dated 10/14/2002 11:50:17 AM Eastern Standard Time, [EMAIL PROTECTED] writes: Or put the most likely pipelines to get hit first and the least likely last... That can be problematic if your most used pipelines are the generic matches. Eg, three special cases and 100 general cases: match=fee.foe match=fee.fie match=fee.fum match= fee.* Probably ok with three special cases, but what about 20, or 50...??? - Please check that your question has not already been answered in the FAQ before posting. TITLE=http://xml.apache.org/cocoon/faq/index.html TARGET=_blankhttp://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Please check that your question has not already been answered in the FAQ before posting. http://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Please check that your question has not already been answered in the FAQ before posting. http://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[Q] Pipeline best practices
Can someone point me to a document or briefly explain what are the best practices to writing pipeline in regard of multiplicity. For example how many pipelines in a sitemap are reasonable? How do one chooses when to put multiple matchers in one pipeline vs splitting them in separate pipelines. What are the performance implications of (not) splitting components in multiple pipelines? -=Ivelin=- - Please check that your question has not already been answered in the FAQ before posting. http://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [Q] Pipeline best practices
Hi Ivelin, -Original Message- From: Ivelin Ivanov [mailto:[EMAIL PROTECTED]] Sent: Sunday, October 13, 2002 4:25 PM To: [EMAIL PROTECTED] Subject: [Q] Pipeline best practices For example how many pipelines in a sitemap are reasonable? How do one chooses when to put multiple matchers in one pipeline vs splitting them in separate pipelines. Some of the reasons are: - different error handling - different types of pipelines (caching, profiling) - internal-only use - the use of different expires attribute Does anybody know some other reasons? As far as I know there is no difference in performance. Regards, Reinhard - Please check that your question has not already been answered in the FAQ before posting. http://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [Q] Pipeline best practices
Another issue of having too many pipelines. As i understand in the sitemap is generated into a java class. And the various matchers within a pipeline(s) are really a bunch of if-else if statements. So the more matchers you have (to match to a pipeline or various paths within a pipeline) your going to have a large series of if/else statements. And... since this uses string compares and wildcards and... if there's alot of these if else if statements it will lead to performance issues. It may help to break things up with sub site maps to help limit the searching. Or put the most likely pipelines to get hit first and the least likely last... In a message dated Sun, 13 Oct 2002 18:03:02 +0200, [EMAIL PROTECTED] writes: Hi Ivelin, -Original Message- From: Ivelin Ivanov [mailto:[EMAIL PROTECTED]] Sent: Sunday, October 13, 2002 4:25 PM To: [EMAIL PROTECTED] Subject: [Q] Pipeline best practices For example how many pipelines in a sitemap are reasonable? How do one chooses when to put multiple matchers in one pipeline vs splitting them in separate pipelines. Some of the reasons are: - different error handling - different types of pipelines (caching, profiling) - internal-only use - the use of different expires attribute Does anybody know some other reasons? As far as I know there is no difference in performance. Regards, Reinhard - Please check that your question has not already been answered in the FAQ before posting. TITLE=http://xml.apache.org/cocoon/faq/index.html TARGET=_blankhttp://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Please check that your question has not already been answered in the FAQ before posting. http://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [Q] Pipeline best practices
--- [EMAIL PROTECTED] wrote: It may help to break things up with sub site maps to help limit the searching. Or put the most likely pipelines to get hit first and the least likely last... After reading Stefano's thread on dev (?) regarding the profiling work he did revealing that matching may be the most expensive operation, I've been thinking along these exact lines. The second one certainly seems to be the easiest to apply. Seems like search algorithm ideas apply here too - you want to get to a match in the least number of decisions possible, so either subsitemaps, or nested matchers seem the best ways to implement a good decision tree. Seems like laying out url space can help this. Question: wouldn't in theory subsitemaps carry more overhead than nested matchers? Of course there are many reasons to use them any way but this would be a factor in deciding when. I don't see any benefit (besides different configurations as mentioned) of multiple pipeline sections, unless each one has a small number of top level matchers that can help eliminate all options quickly. Geoff Howard __ Do you Yahoo!? Faith Hill - Exclusive Performances, Videos More http://faith.yahoo.com - Please check that your question has not already been answered in the FAQ before posting. http://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
