> because this field might be accessed when processing every message. the overhead in key/value property is much higher than an int64 field.
I have the same concern as Andrews mentioned. But a very quick implementation question: properties <https://github.com/apache/incubator-pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L47> field is also part of message-metadata only. So, I believe in your usecase, we want to skip metadata-deserialization and idea is to read only "EventTime" field (field-12) without deserializing message-metada using ByteBuf-offset (eg: read last int64 using end of metadata-index).? Don't you think it will prevent us to add additional String-field in future because then it will be hard to know "EventTime" index? Thanks, Rajan On Wed, Aug 23, 2017 at 3:56 PM, Sijie Guo <guosi...@gmail.com> wrote: > On Wed, Aug 23, 2017 at 3:19 PM, Sahaya Andrews <andr...@apache.org> > wrote: > > > If this is not going to be used by the broker for anything, why can't > > this be set in message property key/value? > > > > Currently it is not used by broker. However, we might leverage field later > on to provide advanced features like event-time based index. It is not > covered by this proposal. > > Also there is another consideration - adding this field to message metadata > is more serialization friendly than putting it into key/value property. > this is important for stream computing, > because this field might be accessed when processing every message. the > overhead in key/value property is much higher than an int64 field. > > Hope this make sense. > > - Sijie > > > > > > Andrews > > > > On Wed, Aug 23, 2017 at 3:03 PM, Matteo Merli <mme...@apache.org> wrote: > > > Proposal looks good to me. > > > > > > Added to wiki: > > > https://github.com/apache/incubator-pulsar/wiki/PIP-5:-Event-time > > > > > > On Wed, Aug 23, 2017 at 2:52 PM Sijie Guo <guosi...@gmail.com> wrote: > > > > > >> Hi pulsar folks, > > >> > > >> Currently Pulsar messages has fields for `publish_time`. However in > some > > >> use cases for example stream computing, `event_time` is required. I'd > > like > > >> to start a proposal to add `event_time` support to pulsar messages. > > >> > > >> This proposal only covers adding `event_time` to pulsar messages and > > make > > >> it available to both producers and subscribers. It doesn't cover > > advanced > > >> features, such as event-time index or rewind subscriptions based on > > >> event-time. > > >> > > >> I created the proposal here: > > >> https://gist.github.com/sijie/60324cc892643961b923593a597109ab > > >> > > >> Please go over and provide your feedback/comments. > > >> > > >> Also, since I don't have any permissions on writing pulsar wiki pages, > > can > > >> any pulsar committers help me add this PIP to the wiki pages if it is > > >> accepted? > > >> > > >> - Sijie > > >> > > > -- > > > Matteo Merli > > > <mme...@apache.org> > > >
