Re: mod_headers as a Filter
On Wed, Jun 23, 2021 at 8:43 PM Mark Thomas wrote: > > On 07/05/2021 09:33, Rémy Maucherat wrote: > > On Wed, Apr 28, 2021 at 10:45 AM Rémy Maucherat wrote: > > > >> On Wed, Apr 28, 2021 at 9:07 AM Mark Thomas wrote: > >> > >>> I'm wondering if there is merit in a Valve-like mechanism for Coyote. > >>> Name TBD but would look something like: > >>> - callbacks > >>> - after request headers are parsed / before the request is prepared > >>> - after the request is prepared > >>> - before response headers are prepared > >>> - after response headers are prepared / before they are written > >>> - allow multiple "Valves" to be configured > >>> - provide a "default" that doesn't require explicit config > >>> - explicit config can add custom "valves", the default "valve" or any > >>> combination > >>> > >> > >> I thought about it quite a bit in the past, an interceptor [resurrecting > >> the old name] could be a solution to a lot of problems here. So why not. > >> > > > > Thinking about it some more, I would propose investigating either: > > - Adding a new interface in addition to ActionHook to coyote.Request, maybe > > called EventHook (gets a code and the request object). > > - Allowing more than one ActionHook [the codes include plenty of the > > interesting events already especially commit, flush]. User configured hooks > > would be added and called *before* the main coyote hook (obviously). > > Although this sounds easier to implement, this is tricker and error prone > > as the call patterns are funky (also the hook field is duplicated between > > request and response likely for no good reason); basically I would say all > > hook action calls would now go through a helper method on the Request. > > - Something else ;) > > I think a new interface would be cleaner. Some sort of CoyoteListener > configured as elements nested under the Connector. > > Looking into the history of ActionHook, it looks like the intention was > to enable support for custom actions for custom Connectors. I don't > think that was ever used. I switched it to an Enum ages ago mainly as I > was concerned that, with the growing numbers of actions, some Processors > may not be implementing all the actions they needed to. > > Given that we can now define default method implementations in an > interface, I think I prefer an interface with one method per event type > rather than a single method with a parameter that indicates the event type. Ok, no problem with that plan. > > Your email also got me wondering whether it was time to refactor the > Coyote code to remove ActionHook entirely. Ok, otoh that's probably not going to bring much value. Rémy > > I think I have some experimenting to do. > > Mark > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: mod_headers as a Filter
On 07/05/2021 09:33, Rémy Maucherat wrote: On Wed, Apr 28, 2021 at 10:45 AM Rémy Maucherat wrote: On Wed, Apr 28, 2021 at 9:07 AM Mark Thomas wrote: I'm wondering if there is merit in a Valve-like mechanism for Coyote. Name TBD but would look something like: - callbacks - after request headers are parsed / before the request is prepared - after the request is prepared - before response headers are prepared - after response headers are prepared / before they are written - allow multiple "Valves" to be configured - provide a "default" that doesn't require explicit config - explicit config can add custom "valves", the default "valve" or any combination I thought about it quite a bit in the past, an interceptor [resurrecting the old name] could be a solution to a lot of problems here. So why not. Thinking about it some more, I would propose investigating either: - Adding a new interface in addition to ActionHook to coyote.Request, maybe called EventHook (gets a code and the request object). - Allowing more than one ActionHook [the codes include plenty of the interesting events already especially commit, flush]. User configured hooks would be added and called *before* the main coyote hook (obviously). Although this sounds easier to implement, this is tricker and error prone as the call patterns are funky (also the hook field is duplicated between request and response likely for no good reason); basically I would say all hook action calls would now go through a helper method on the Request. - Something else ;) I think a new interface would be cleaner. Some sort of CoyoteListener configured as elements nested under the Connector. Looking into the history of ActionHook, it looks like the intention was to enable support for custom actions for custom Connectors. I don't think that was ever used. I switched it to an Enum ages ago mainly as I was concerned that, with the growing numbers of actions, some Processors may not be implementing all the actions they needed to. Given that we can now define default method implementations in an interface, I think I prefer an interface with one method per event type rather than a single method with a parameter that indicates the event type. Your email also got me wondering whether it was time to refactor the Coyote code to remove ActionHook entirely. I think I have some experimenting to do. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: mod_headers as a Filter
On Wed, Apr 28, 2021 at 10:45 AM Rémy Maucherat wrote: > On Wed, Apr 28, 2021 at 9:07 AM Mark Thomas wrote: > >> I'm wondering if there is merit in a Valve-like mechanism for Coyote. >> Name TBD but would look something like: >> - callbacks >>- after request headers are parsed / before the request is prepared >>- after the request is prepared >>- before response headers are prepared >>- after response headers are prepared / before they are written >> - allow multiple "Valves" to be configured >> - provide a "default" that doesn't require explicit config >> - explicit config can add custom "valves", the default "valve" or any >>combination >> > > I thought about it quite a bit in the past, an interceptor [resurrecting > the old name] could be a solution to a lot of problems here. So why not. > Thinking about it some more, I would propose investigating either: - Adding a new interface in addition to ActionHook to coyote.Request, maybe called EventHook (gets a code and the request object). - Allowing more than one ActionHook [the codes include plenty of the interesting events already especially commit, flush]. User configured hooks would be added and called *before* the main coyote hook (obviously). Although this sounds easier to implement, this is tricker and error prone as the call patterns are funky (also the hook field is duplicated between request and response likely for no good reason); basically I would say all hook action calls would now go through a helper method on the Request. - Something else ;) Rémy > > Rémy > > >> >> I'm leaning towards the latter approach as it has much greater >> flexibility and I can see different users having subtly different >> requirements and this avoids an explosion of configuration attributes on >> a single class. >> >> Mark >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: dev-h...@tomcat.apache.org >> >>
Re: mod_headers as a Filter
On Wed, Apr 28, 2021 at 9:07 AM Mark Thomas wrote: > I'm wondering if there is merit in a Valve-like mechanism for Coyote. > Name TBD but would look something like: > - callbacks >- after request headers are parsed / before the request is prepared >- after the request is prepared >- before response headers are prepared >- after response headers are prepared / before they are written > - allow multiple "Valves" to be configured > - provide a "default" that doesn't require explicit config > - explicit config can add custom "valves", the default "valve" or any >combination > I thought about it quite a bit in the past, an interceptor [resurrecting the old name] could be a solution to a lot of problems here. So why not. Rémy > > I'm leaning towards the latter approach as it has much greater > flexibility and I can see different users having subtly different > requirements and this avoids an explosion of configuration attributes on > a single class. > > Mark > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: mod_headers as a Filter
On 27/04/2021 22:14, Rémy Maucherat wrote: I remember after doing the rewrite valve I got asked a bit about mod_headers because "why not". However, now I recall I found out it would be far less practical. So I very quickly moved on since it was also less useful than rewrite. I would still probably not do it. I'm rapidly coming to the same conclusion. Certainly for a pure Filter approach. And a Valve approach doesn't look much better. I'm currently mulling over an extension point for the Connector. responseHeaderValidationClass or similar. Ideas at this point are: - have one or more callbacks from AbstractProcessor.prepareResponse() - provide a number of standard implementations - NO-OP (same as now) - warn (logs a warning for anything dangerous it finds) - fix (removes/fixes any invalid headers) Alternatively, configuration could be nested inside the connector element which would allow the option for more configuration. We could still have standard implementations. My expectation is that a suitable default would be added (probably not NO-OP) if no explicit configuration was provided. I'm wondering if there is merit in a Valve-like mechanism for Coyote. Name TBD but would look something like: - callbacks - after request headers are parsed / before the request is prepared - after the request is prepared - before response headers are prepared - after response headers are prepared / before they are written - allow multiple "Valves" to be configured - provide a "default" that doesn't require explicit config - explicit config can add custom "valves", the default "valve" or any combination I'm leaning towards the latter approach as it has much greater flexibility and I can see different users having subtly different requirements and this avoids an explosion of configuration attributes on a single class. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: mod_headers as a Filter
On Tue, Apr 27, 2021 at 7:05 PM Mark Thomas wrote: > Hi all, > > I've started to look at this and I am struggling to see a way to > implement something that looks like mod_headers as a Filter. > > Request headers are fairly simple. The process looks something like: > a) take a copy of all the headers received > b) apply all the rules for request headers > c) wrap the request, overriding the various getHeader... methods and > return values appropriate for the modified set of headers > > Response headers are where I am currently stuck. > A similar model to request headers might look like: > a) wrap the response > b) intercept all the headers set by the application > c) apply all the rules for response headers > d) call the appropriate setHeader... methods on the wrapped response > for the modified set of headers > > The problem is that d) (and hence c) needs to be done immediately before > the response is committed and - short of buffering the entire response > body - there is no way to know when that is going to happen. > > Is the answer we need to buffer the entire response body? > > Any cunning ideas on how to detect (in a Filter or wrapped response) > that the response is about to be committed? > > I guess we could try and track bytes (about to be) written and compare > that to the known buffer size. That seems a little fragile on first > impression. > > Another option is to abandon the mod_headers clone aim and do something > simpler along the lines of blocking applications from setting specific > headers and/or header/value combinations. > > Thoughts? > I remember after doing the rewrite valve I got asked a bit about mod_headers because "why not". However, now I recall I found out it would be far less practical. So I very quickly moved on since it was also less useful than rewrite. I would still probably not do it. Rémy > > Mark > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: mod_headers as a Filter
Isnt the response buffer size giving a sufficient hint or callback like (dont rewrite before it is reached or body starts to be read)? Guess filter must force some size if not set but sounds like something to check, no? Le mar. 27 avr. 2021 à 21:32, Raymond Augé a écrit : > Couldn't you add a callback/hook to commit() impl and trigger the rules > during that callback/hook? > > But with that the filter is merely a shell for pushing rules into that > callback/hook registry. > > - Ray > > On Tue, Apr 27, 2021 at 1:04 PM Mark Thomas wrote: > > > Hi all, > > > > I've started to look at this and I am struggling to see a way to > > implement something that looks like mod_headers as a Filter. > > > > Request headers are fairly simple. The process looks something like: > > a) take a copy of all the headers received > > b) apply all the rules for request headers > > c) wrap the request, overriding the various getHeader... methods and > > return values appropriate for the modified set of headers > > > > Response headers are where I am currently stuck. > > A similar model to request headers might look like: > > a) wrap the response > > b) intercept all the headers set by the application > > c) apply all the rules for response headers > > d) call the appropriate setHeader... methods on the wrapped response > > for the modified set of headers > > > > The problem is that d) (and hence c) needs to be done immediately before > > the response is committed and - short of buffering the entire response > > body - there is no way to know when that is going to happen. > > > > Is the answer we need to buffer the entire response body? > > > > Any cunning ideas on how to detect (in a Filter or wrapped response) > > that the response is about to be committed? > > > > I guess we could try and track bytes (about to be) written and compare > > that to the known buffer size. That seems a little fragile on first > > impression. > > > > Another option is to abandon the mod_headers clone aim and do something > > simpler along the lines of blocking applications from setting specific > > headers and/or header/value combinations. > > > > Thoughts? > > > > Mark > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: dev-h...@tomcat.apache.org > > > > > > -- > *Raymond Augé* (@rotty3000) > Senior Software Architect *Liferay, Inc.* (@Liferay) > OSGi Fellow >
Re: mod_headers as a Filter
Couldn't you add a callback/hook to commit() impl and trigger the rules during that callback/hook? But with that the filter is merely a shell for pushing rules into that callback/hook registry. - Ray On Tue, Apr 27, 2021 at 1:04 PM Mark Thomas wrote: > Hi all, > > I've started to look at this and I am struggling to see a way to > implement something that looks like mod_headers as a Filter. > > Request headers are fairly simple. The process looks something like: > a) take a copy of all the headers received > b) apply all the rules for request headers > c) wrap the request, overriding the various getHeader... methods and > return values appropriate for the modified set of headers > > Response headers are where I am currently stuck. > A similar model to request headers might look like: > a) wrap the response > b) intercept all the headers set by the application > c) apply all the rules for response headers > d) call the appropriate setHeader... methods on the wrapped response > for the modified set of headers > > The problem is that d) (and hence c) needs to be done immediately before > the response is committed and - short of buffering the entire response > body - there is no way to know when that is going to happen. > > Is the answer we need to buffer the entire response body? > > Any cunning ideas on how to detect (in a Filter or wrapped response) > that the response is about to be committed? > > I guess we could try and track bytes (about to be) written and compare > that to the known buffer size. That seems a little fragile on first > impression. > > Another option is to abandon the mod_headers clone aim and do something > simpler along the lines of blocking applications from setting specific > headers and/or header/value combinations. > > Thoughts? > > Mark > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > > -- *Raymond Augé* (@rotty3000) Senior Software Architect *Liferay, Inc.* (@Liferay) OSGi Fellow
mod_headers as a Filter
Hi all, I've started to look at this and I am struggling to see a way to implement something that looks like mod_headers as a Filter. Request headers are fairly simple. The process looks something like: a) take a copy of all the headers received b) apply all the rules for request headers c) wrap the request, overriding the various getHeader... methods and return values appropriate for the modified set of headers Response headers are where I am currently stuck. A similar model to request headers might look like: a) wrap the response b) intercept all the headers set by the application c) apply all the rules for response headers d) call the appropriate setHeader... methods on the wrapped response for the modified set of headers The problem is that d) (and hence c) needs to be done immediately before the response is committed and - short of buffering the entire response body - there is no way to know when that is going to happen. Is the answer we need to buffer the entire response body? Any cunning ideas on how to detect (in a Filter or wrapped response) that the response is about to be committed? I guess we could try and track bytes (about to be) written and compare that to the known buffer size. That seems a little fragile on first impression. Another option is to abandon the mod_headers clone aim and do something simpler along the lines of blocking applications from setting specific headers and/or header/value combinations. Thoughts? Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
