On Tue, Jan 27, 2009 at 6:02 AM, William Tam <email.w...@gmail.com> wrote:
> On Mon, Jan 26, 2009 at 11:34 PM, Hadrian Zbarcea <hzbar...@gmail.com> wrote:
>> I think having dots ('.') in property names is not a good idea as they don't
>> get along iirc with some technologies.
>>
>
> Sorry one more thing, the property name is just key to exchange
> property map. I believe we already use dots in exchange property
> names in some places today.
>
We have, in the Camel 2.0 design
(http://camel.apache.org/camel-20-design.html) a bullet, that we
should resolve the header key name issue.
James suggested to use a new naming convention CamelComponentName that I like.
We should really remove all dots and usage with long packagenames etc.
as the name names dont apply with JMS spec and some other transports.
And I think its something we should do prior to 2.0 release as its a
change that affects many components and internal parts in Camel.
I am glad that I kinda started this ball, this thread sure has proven
that we needed a little heads up and consensus on the
header/properties issue in Camel. Keep the ball rolling.
This is early morning and I am not juiced up for long and deep thoughts :)
>> Secondly, my point was that at this time, I am against the idea of
>> separating the headers into two maps. I don't see a compelling reason to do
>> so and we should properly use properties as Roman and I think Claus
>> suggested. I'm confident that we'll find a good way to do it.
>>
>> Hadrian
>>
>>
>> On Jan 26, 2009, at 11:09 PM, William Tam wrote:
>>
>>> On Mon, Jan 26, 2009 at 5:34 PM, Hadrian Zbarcea <hzbar...@gmail.com>
>>> wrote:
>>>>
>>>> On Jan 26, 2009, at 4:17 PM, William Tam wrote:
>>>>
>>>>> On Mon, Jan 26, 2009 at 1:37 PM, Hadrian Zbarcea <hzbar...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> I don't disagree, I was just suggesting that they should then travel as
>>>>>> properties.
>>>>>
>>>>> Why shouldn't they (protocol headers) travel as headers (to avoids
>>>>> unnecessary copying between exchange properties and protocol headers)?
>>>>
>>>> There should be no unnecessary copying, I agree. If a header (such as
>>>> username/password) has broadly known and accepted semantics, the endpoint
>>>> or
>>>> associated policy could set it as a property from start. Similarly, an
>>>> endpoint should look at both properties and headers when sending a
>>>> message.
>>>
>>> I think I finally get it. Your example of username/password are
>>> really properties in my view. E.g, we can have a property
>>> org.apache.camel.headers.username. It is broadly known in Camel.
>>> Sure, we can stick it in exchange properties and let endpoint read it
>>> and create a protocol header with it (like, USER=foo). I am fine with
>>> that idea. That does not make org.apache.camel.headers.username=foo a
>>> protocol header, though. It really isn't.
>>>
>>> I am concerned about true custom protocol header that is totally
>>> unknown to Camel (say, foo=bar). I want my app to be able to define
>>> and insert some header in protocol message header. I argue that
>>> foo=bar should be set in the header not in exchange properties.
>>>
>>>>
>>>>>
>>>>>
>>>>>> Whatever we name them, and whatever mechanism we decide to use,
>>>>>> as pointed out before, we need to distinguish between headers that are
>>>>>> endpoint/protocol specific and have no semantics outside the endpoint
>>>>>> and
>>>>>> headers (which we called properties and didn't use consistently) that
>>>>>> must
>>>>>> be carried over the lifetime of the Exchange.
>>>>>
>>>>> Custom protocol header fits into "no semantics outside the endpoint"
>>>>> (or else it won't get propagated) and so it goes to the exchange
>>>>> properties. How does the next component (in the pipeline) know what
>>>>> one of the properties are intended to be sent in protocol header?
>>>>> Wouldn't it be better to let protocol headers travel as headers so
>>>>> only the "header candidates" will be potentially be sent?
>>>>
>>>> Not sure I understand your question, but I think we should use policies
>>>> for
>>>> handling custom headers.
>>>
>>> Sorry about the typos and grammars. All I am saying is if we put user
>>> defined header in message headers, filter strategy (or policy if you
>>> will) won't have to iterate through the exchange properties for it.
>>> It is already in headers where it belongs. All we need to do is to
>>> apply filters to headers. It is much cleaner. In your example, if
>>> endpoint wants to read a specific property (e.g.
>>> org.apache.camel.headers.username) and make a header out of a
>>> property, it can certainly do so.
>>>
>>> (However, I am not sure there are really that many Camel well known
>>> properties at least in the presence.)
>>>
>>>
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> Hadrian
>>>>>>
>>>>>>
>>>>>> On Jan 26, 2009, at 1:07 PM, William Tam wrote:
>>>>>>
>>>>>>> On Mon, Jan 26, 2009 at 10:35 AM, Hadrian Zbarcea <hzbar...@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> This headers business is a bit of a tricky one. I hit it last year
>>>>>>>> in
>>>>>>>> the
>>>>>>>> context of security.
>>>>>>>>
>>>>>>>> I agree with the view that headers should only exist in the context
>>>>>>>> of
>>>>>>>> an
>>>>>>>> endpoint. I think outside of that there is no guarantee that the
>>>>>>>> semantics
>>>>>>>> of a header is preserved. I am not sure if headers should be
>>>>>>>> propagated
>>>>>>>> from one endpoint to another at all.
>>>>>>>
>>>>>>> There are certainly use cases that protocol headers DO need to be
>>>>>>> propagated between endpoints. If users want to integrate with some
>>>>>>> management and/or security tools like Actional, users are required to
>>>>>>> include custom headers in protocol headers. These custom headers
>>>>>>> travel with messages to allow trust zone enforcement and message
>>>>>>> correlation. They need to be preserved and propagated across hops
>>>>>>> which are potentially over different transport protocols.
>>>>>>>
>>>>>>>> Properties should be used instead.
>>>>>>>> Coming back to security, if http is used for instance there are
>>>>>>>> several
>>>>>>>> ways
>>>>>>>> of handling that. If basic auth is used for instance one gets a
>>>>>>>> user/pass,
>>>>>>>> but that may need to be translated to something else at endpoint
>>>>>>>> boundaries.
>>>>>>>> I don't think that the "Authorization" header should exist outside
>>>>>>>> the
>>>>>>>> http
>>>>>>>> endpoint for instance.
>>>>>>>>
>>>>>>>> Yes, we do propagate properties today, no issue there. But then some
>>>>>>>> policies need to be defined per endpoint to deal with known
>>>>>>>> headers/properties, and camel specific properties should be defined
>>>>>>>> to
>>>>>>>> deal
>>>>>>>> with know headers. Even better, endpoints should set protocol
>>>>>>>> specific
>>>>>>>> headers that are known as required to propagate (such as the auth
>>>>>>>> stuff)
>>>>>>>> as
>>>>>>>> properties from start.
>>>>>>>>
>>>>>>>> My $0.02
>>>>>>>> Hadrian
>>>>>>>>
>>>>>>>>
>>>>>>>> On Jan 26, 2009, at 9:44 AM, Claus Ibsen wrote:
>>>>>>>>
>>>>>>>>> On Mon, Jan 26, 2009 at 3:37 PM, Roman Kalukiewicz
>>>>>>>>> <roman.kalukiew...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Why don't we talk about exchange properties here? My feeling here
>>>>>>>>>> is
>>>>>>>>>> that properties should be used as user-headers, while headers are
>>>>>>>>>> always protocol headers. In fact it works this way right now: If I
>>>>>>>>>> want to keep some value through the whole flow I put it into
>>>>>>>>>> properties.
>>>>>>>>>>
>>>>>>>>>> By current convention if I put something on a header it is sent as
>>>>>>>>>> protocol-specific header (JMS property, HTTP header), and out
>>>>>>>>>> headers
>>>>>>>>>> are filled also with protocol headers (JSM properties of out
>>>>>>>>>> emssage,
>>>>>>>>>> HTTP response headers). In this case headers shouldn't be
>>>>>>>>>> propagated,
>>>>>>>>>> as there is no way to distinguish things propagated, from things
>>>>>>>>>> retrieved. And out headers ARE different than in headers.
>>>>>>>>>>
>>>>>>>>>> It is a matter of naming, but currently headers are (what you call)
>>>>>>>>>> protocol/system headers, while properties are user-headers (work as
>>>>>>>>>> variables). Do we really need to extend it further? If someone mix
>>>>>>>>>> those two concepts then it is problem of documentation, but not
>>>>>>>>>> lack
>>>>>>>>>> of functionality. I would just extend DSL a little to be able to
>>>>>>>>>> retrieve a property (instead using header()).
>>>>>>>>>>
>>>>>>>>>> What do you think, guys? Maybe we should clearly communicate what
>>>>>>>>>> things are for and what are the consequences of using one or
>>>>>>>>>> another.
>>>>>>>>>>
>>>>>>>>>> Roman
>>>>>>>>>>
>>>>>>>>>> PS. Pipeline should propagate all headers of course, but I believe
>>>>>>>>>> an
>>>>>>>>>> endpoint is a place where we shouldn't guarantee that headers will
>>>>>>>>>> be
>>>>>>>>>> propagated by stating it clearly.
>>>>>>>>>
>>>>>>>>> Properties have just lived in the dark and end users does not really
>>>>>>>>> know they exists. We have some builder methods to set/get
>>>>>>>>> properties.
>>>>>>>>> I guess we need to document and maybe make sure the Spring DSL also
>>>>>>>>> has support for accessing properties as well.
>>>>>>>>>
>>>>>>>>> To my knowledge properties is always preserved so I doubt we have an
>>>>>>>>> issue there.
>>>>>>>>>
>>>>>>>>> So users should just start learn using properties as well :)
>>>>>>>>> However then we have the ProducerTemplate that has one liners for
>>>>>>>>> sending an Exchange. We dont have a sendBodyAndProperties method.
>>>>>>>>> But
>>>>>>>>> yet again it has too many methods already. They can just use
>>>>>>>>> send("xxx", Exchange) and have exchange populated with the
>>>>>>>>> properties
>>>>>>>>> of choice.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2009/1/26 Claus Ibsen <claus.ib...@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>> On Sat, Jan 24, 2009 at 9:08 PM, William Tam
>>>>>>>>>>> <email.w...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> What we have stored in Headers today in Camel is both:
>>>>>>>>>>>>> - user headers
>>>>>>>>>>>>> - and system headers (added by Camel itself).
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am starting to be more and more convinced that we should
>>>>>>>>>>>>> separate
>>>>>>>>>>>>> the two.
>>>>>>>>>>>>> So any headers that a users has enforced to be set should be
>>>>>>>>>>>>> kept
>>>>>>>>>>>>> in
>>>>>>>>>>>>> one Map and the others that the components set internally (such
>>>>>>>>>>>>> as
>>>>>>>>>>>>> SQL
>>>>>>>>>>>>> number of rows returned, or whatnot we have, there are many) in
>>>>>>>>>>>>> another Map.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> It means that a component would have to look for header in more
>>>>>>>>>>>> than
>>>>>>>>>>>> one place. Besides, the distinction of user vs system header is
>>>>>>>>>>>> not
>>>>>>>>>>>> always clear. For example, the operation name header for cxf
>>>>>>>>>>>> endpoint
>>>>>>>>>>>> can be set by user but it is also created by cxf component. I
>>>>>>>>>>>> am
>>>>>>>>>>>> sure there are many more examples. There is another header
>>>>>>>>>>>> category:
>>>>>>>>>>>> protocol headers. A protocol header is not really a user or
>>>>>>>>>>>> system
>>>>>>>>>>>> header. Protocol headers are header propagated from protocol
>>>>>>>>>>>> like
>>>>>>>>>>>> HTTP, which we do want to preserve in message header.
>>>>>>>>>>>>
>>>>>>>>>>>>> The user headers is always preserved and copied along in the
>>>>>>>>>>>>> routing.
>>>>>>>>>>>>> User can always clear/remove unwanted headers.
>>>>>>>>>>>>> The system headers should be short lived as they are not really
>>>>>>>>>>>>> useable. So they are "alive" in the next step (process) in the
>>>>>>>>>>>>> route,
>>>>>>>>>>>>> and when the pipeline invokes next route thereafter these
>>>>>>>>>>>>> information
>>>>>>>>>>>>> is cleared.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Separating these will also make the routing/tracing a bit easier
>>>>>>>>>>>>> as
>>>>>>>>>>>>> Users can recognize their own headers instead its mixed with all
>>>>>>>>>>>>> the
>>>>>>>>>>>>> noise the Camel components add.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I wonder we can leverage/extend the HeaderFilterStrategy
>>>>>>>>>>>> mechanism.
>>>>>>>>>>>> Currently, it is only used for filtering unwanted headers (in
>>>>>>>>>>>> both
>>>>>>>>>>>> request and response direction) when we propagate headers between
>>>>>>>>>>>> Camel and external messages (like HTTP). HeaderFilterStrategy
>>>>>>>>>>>> is
>>>>>>>>>>>> (or
>>>>>>>>>>>> will be) associated with an endpoint. We could make
>>>>>>>>>>>> HeaderFilterStrategy available to the exchange object. So, when
>>>>>>>>>>>> an
>>>>>>>>>>>> endpoint creates an exchange, the exchange gets a header filter
>>>>>>>>>>>> strategy. Then, pipeline can do something like this to filter
>>>>>>>>>>>> unwanted header: message.filterHeaders(). The header filter
>>>>>>>>>>>> strategy
>>>>>>>>>>>> is highly customizable for each endpoint (can have a component
>>>>>>>>>>>> wide
>>>>>>>>>>>> default) and it can be looked up from registry.
>>>>>>>>>>>>
>>>>>>>>>>> Good pointers William.
>>>>>>>>>>>
>>>>>>>>>>> Yeah we can revist it after you have moved the header filters to
>>>>>>>>>>> the
>>>>>>>>>>> endpoint.
>>>>>>>>>>>
>>>>>>>>>>> Then we can check up upon how to leverage it as you suggest.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Claus Ibsen
>>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>>
>>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Claus Ibsen
>>>>>>>>> Apache Camel Committer
>>>>>>>>>
>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
