If we follow the current way of doing this format change, it would work the following way:
0. Bump up the magic byte to 2 to indicate the tombstone bit is used. 1. On receiving a ProduceRequest, broker always convert the messages to the configured message.format.version. 1.1 If the message version does not match the configured message.format.version, the broker will either up convert or down convert the message. In that case, users pay the performance cost of re-compression if needed. 1.2 If the message version matches the configured message.format.version, the broker will not do the format conversion and user may save the re-compression cost if the message.format.version is on or above 0.10.0. 2. On receiving a FetchRequest, the broker check the FetchRequest version to see if the consumer supports the configured message.format.version or not. If the consumer does not support it, down conversion is required and zero copy is lost. Otherwise zero copy is used to return the FetchResponse. Notice that setting message.format.version to 0.10.2 is equivalent to always up convert if needed, but that also means to always down convert if there is an old consumer. Thanks, Jiangjie (Becket) Qin On Mon, Nov 14, 2016 at 1:43 PM, Michael Pearce <michael.pea...@ig.com> wrote: > I like the idea of up converting and then just having the logic to look > for tombstones. It makes that quite clean in nature. > > It's quite late here in the UK, so I fully understand / confirm I > understand what you propose could you write it on the kip wiki or fully > describe exactly how you see it working, so uk morning I could read through? > > Thanks all for the input on this it is appreciated. > > > Sent using OWA for iPhone > ________________________________________ > From: Mayuresh Gharat <gharatmayures...@gmail.com> > Sent: Monday, November 14, 2016 9:28:16 PM > To: dev@kafka.apache.org > Subject: Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag > > Hi Michael, > > Just another thing that came up during my discussion with Renu and I wanted > to share it. > > Other thing we can do to handle a mixture of old and new clients is when > once the new broker with this KIP is deployed, the new code should check > the request version from older producer we can up convert it with a > tombstone marker when appending the message to the log. This is similar to > down converting messages for older clients. > > If this is possible then the broker in this case has to rely only on the > tombstone marker for log compaction. Using this approach we preserve the > description of when to update the magic byte as described here : > https://kafka.apache.org/documentation#messageformat (1 byte "magic" > identifier to allow format changes). > > In stage 2, if we don't want open source kafka to make the decision of > deprecation of null value for log compaction (which is the approach I would > prefer as an end state) since there are some concerns on this, individual > companies/orgs can then have a horizontal initiative at their convenience > to move to stage 2 by asking all app users to upgrade to new kafka clients. > > Thanks, > > Mayuresh > > On Mon, Nov 14, 2016 at 11:50 AM, Becket Qin <becket....@gmail.com> wrote: > > > Michael, > > > > Yes, I am OK with stage 1. We can discuss about stage 2 later but this > > sounds really an organization specific decision to deprecate an API. It > > does not seem a general need in open source Kafka to only support > tombstone > > bit , which is a bad thing for people who are still running old clients > in > > the community. This is exactly why we want to have magic byte bump - > there > > should be only one definitive way to interpret a message with a given > magic > > byte. We should avoid re-define the interpretation of a message with an > > existing magic byte. The interpretation of an old message format should > not > > be changed and should always be supported until deprecated. > > > > Thanks, > > > > Jiangjie (Becket) Qin > > > > On Mon, Nov 14, 2016 at 11:28 AM, Mayuresh Gharat < > > gharatmayures...@gmail.com> wrote: > > > > > I am not sure about "If I understand correctly, you want to let the > > broker > > > to reject requests > > > from old clients to ensure everyone in an organization has upgraded, > > > right?" > > > > > > I don't think we will be rejecting requests. What phase2 (stage 2) > meant > > > was we will only do log compaction based on tombstone marker and > nothing > > > else. > > > I am not sure about making this configurable, as a configuration makes > it > > > sound like the broker has the capability to support null and tombstone > > > marker bit, for log compaction, for life long. > > > > > > I agree with Michael's idea of delivering phase 1 (stage1) right now. > It > > > will also be good to put a note saying that we will be deprecating the > > > older way of log compaction in next release. > > > Phase 2 (stage 2) will actually only rely on tombstone marker for doing > > log > > > compaction. > > > So this actually meets our end state of having tombstone marker for log > > > compaction support. > > > > > > Thanks, > > > > > > Mayuresh > > > > > > > > > On Mon, Nov 14, 2016 at 11:09 AM, Michael Pearce < > michael.pea...@ig.com> > > > wrote: > > > > > > > I'm ok with this, but I'd like to at least get phase 1 in, for the > next > > > > release, this is what I'm very keen for, > > > > > > > > As such shall we say we have > > > > Kip-87a that delivers phase 1 > > > > > > > > And then > > > > Kip-87b in release after that delivers phase 2 but has a dependency > on > > > > support of deprecating old api versions > > > > > > > > How does this approach sound? > > > > > > > > > > > > ________________________________________ > > > > From: Becket Qin <becket....@gmail.com> > > > > Sent: Monday, November 14, 2016 6:14:44 PM > > > > To: dev@kafka.apache.org > > > > Subject: Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag > > > > > > > > Hey Michael and Mayuresh, > > > > > > > > If I understand correctly, you want to let the broker to reject > > requests > > > > from old clients to ensure everyone in an organization has upgraded, > > > right? > > > > This is essentially deprecating an old protocol. I agree it is useful > > and > > > > that is why we have that baked in KIP-35. However, we haven't thought > > > > through how to deprecate an API so far. From broker's point of view., > > it > > > > will always support all the old protocols according to Kafka's widely > > > known > > > > compatibility guarantee. If people decide to deprecate an API within > an > > > > organization, we can provide some additional configuration to let > > people > > > do > > > > this, which is not a bad idea but seems better to be discussed in > > another > > > > KIP. > > > > > > > > Thanks, > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > On Mon, Nov 14, 2016 at 8:52 AM, Michael Pearce < > michael.pea...@ig.com > > > > > > > wrote: > > > > > > > > > I agree with Mayuresh. > > > > > > > > > > I don't see how having a magic byte helps here. > > > > > > > > > > What we are saying is that on day 1 after an upgrade both tombstone > > > flag > > > > > or a null value will be treated as a marker to delete on compacted > > > topic. > > > > > > > > > > During this time we expect organisations to migrate themselves over > > > onto > > > > > the new producers and consumers that call the new Api, changing > their > > > > > application logic. during this point producers should start setting > > the > > > > > tombstone marker and consumers to start behaving based on the > > tombstone > > > > > marker. > > > > > > > > > > Only after all producers and consumers are upgraded then we enter > > > phase 2 > > > > > disabling the use of a null value as a delete marker. And the old > > apis > > > > > would not be allowed to be called as the broker now expects all > > clients > > > > to > > > > > be using latest api only. > > > > > > > > > > At this second phase stage only also we can support a null value > in a > > > > > compacted topic without it being treated as deletion. We solely > base > > on > > > > the > > > > > tombstone marker. > > > > > > > > > > > > > > > ________________________________________ > > > > > From: Mayuresh Gharat <gharatmayures...@gmail.com> > > > > > Sent: Saturday, November 12, 2016 2:18:19 AM > > > > > To: dev@kafka.apache.org > > > > > Subject: Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag > > > > > > > > > > I think "In second stage we move on to supporting only the > attribute > > > flag > > > > > for log > > > > > compaction." means that it will no longer support request from > older > > > > > clients. > > > > > > > > > > Thanks, > > > > > > > > > > Mayuresh > > > > > > > > > > On Fri, Nov 11, 2016 at 10:02 AM, Becket Qin <becket....@gmail.com > > > > > > wrote: > > > > > > > > > > > Hey Michael, > > > > > > > > > > > > The way Kafka implements backwards compatibility is to let the > > > brokers > > > > > > support old protocols. So the brokers have to support older > clients > > > > that > > > > > do > > > > > > not understand the new attribute bit. That means we will not be > > able > > > to > > > > > get > > > > > > away with the null value as a tombstone. So we need a good > > definition > > > > of > > > > > > whether we should treat it as a tombstone or not. This is why I > > think > > > > we > > > > > > need a magic value bump. > > > > > > > > > > > > My confusion on the second stage is exactly about "we want to end > > up > > > > just > > > > > > supporting tombstone marker (not both)", or in the original > > > statement: > > > > > "2) > > > > > > In second stage we move on to supporting only the attribute flag > > for > > > > log > > > > > > compaction." - We will always support the null value as a > tombstone > > > as > > > > > long > > > > > > as the message format version (i.e. magic byte) is less than 2 > for > > > the > > > > > > reason I mentioned above. > > > > > > > > > > > > Although the way to interpret the message bytes at producing time > > can > > > > be > > > > > > inferred from the the ProduceRequest version, this version > > > information > > > > > > won't be stored in the broker. So when an old consumer comes, we > > need > > > > to > > > > > > decide whether we have to do down conversion or not.. With a > clear > > > > > existing > > > > > > configuration of message version config (i.e. magic value), we > know > > > for > > > > > > sure when to down convert the messages to adapt to older clients. > > > > > Otherwise > > > > > > we will have to always scan all the messages. It would probably > > work > > > > but > > > > > > relies on guess or inference. > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > On Fri, Nov 11, 2016 at 8:42 AM, Mayuresh Gharat < > > > > > > gharatmayures...@gmail.com > > > > > > > wrote: > > > > > > > > > > > > > Sounds good Michael. > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Mayuresh > > > > > > > > > > > > > > On Fri, Nov 11, 2016 at 12:48 AM, Michael Pearce < > > > > > michael.pea...@ig.com> > > > > > > > wrote: > > > > > > > > > > > > > > > @Mayuresh i don't think you've missed anything - > > > > > > > > > > > > > > > > as per earlier in the discussion. > > > > > > > > > > > > > > > > We're providing new api versions, but not planning on bumping > > the > > > > > magic > > > > > > > > number as there is no structural changes, we are simply using > > up > > > a > > > > > new > > > > > > > > attribute bit (as like adding new compression support just > uses > > > up > > > > > > > > additional attribute bits) > > > > > > > > > > > > > > > > > > > > > > > > I think also difference between null vs 0 length byte array > is > > > > > covered. > > > > > > > > > > > > > > > > @Becket, > > > > > > > > > > > > > > > > The two stage approach is because we want to end up just > > > supporting > > > > > > > > tombstone marker (not both) > > > > > > > > > > > > > > > > But we accept we need to allow organisations and systems a > > period > > > > of > > > > > > > > transition (this is what stage 1 provides) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ________________________________________ > > > > > > > > From: Mayuresh Gharat <gharatmayures...@gmail.com> > > > > > > > > Sent: Thursday, November 10, 2016 8:57 PM > > > > > > > > To: dev@kafka.apache.org > > > > > > > > Subject: Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag > > > > > > > > > > > > > > > > I am not sure if we are bumping magicByte value as the KIP > does > > > not > > > > > > > mention > > > > > > > > it (If I didn't miss anything). > > > > > > > > > > > > > > > > @Nacho : I did not understand what you meant by : Conversion > > from > > > > one > > > > > > to > > > > > > > > the other may be possible but I would rather not have to do > it. > > > > > > > > > > > > > > > > You will have to do the conversion for the scenario that > Becket > > > has > > > > > > > > mentioned above where an old consumer talks to the new > broker, > > > > right? > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Mayuresh > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Nov 10, 2016 at 11:54 AM, Becket Qin < > > > becket....@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Nacho, > > > > > > > > > > > > > > > > > > In Kafka protocol, a negative length is null, a zero length > > > means > > > > > > empty > > > > > > > > > byte array. > > > > > > > > > > > > > > > > > > I am still confused about the two stages. It seems that the > > > > broker > > > > > > only > > > > > > > > > needs to understand three versions of messages. > > > > > > > > > 1. MagicValue=0 - no timestamp, absolute offset, no > tombstone > > > > flag > > > > > > > > > 2. MagicValue=1 - with timestamp, relative offsets, no > > > tombstone > > > > > flag > > > > > > > > (null > > > > > > > > > value = tombstone) > > > > > > > > > 3. MagicValue=2 - with timestamp, relative offsets, > tombstone > > > > flag > > > > > > > > > > > > > > > > > > We are talking about two flavors for 3: > > > > > > > > > 3.1 tombstone flag set = tombstone (Allows a key with null > > > value > > > > in > > > > > > the > > > > > > > > > compacted topics) > > > > > > > > > 3.2 tombstone flag set OR null value = tombstone ( Do not > > > allow a > > > > > key > > > > > > > > with > > > > > > > > > null value in the compacted topics) > > > > > > > > > > > > > > > > > > No matter which flavor we choose, we just need to stick to > > that > > > > way > > > > > > of > > > > > > > > > interpretation, right? Why would we need a second stage? > > > > > > > > > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > > > > > > > On Thu, Nov 10, 2016 at 10:37 AM, Ignacio Solis < > > > iso...@igso.net > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > A quick differentiation I would like to make is null is > not > > > the > > > > > > same > > > > > > > as > > > > > > > > > > size 0. > > > > > > > > > > > > > > > > > > > > Many times these are made equal, but they're not. When > > > > > serializing > > > > > > > > data, > > > > > > > > > > we have to make a choice in null values and many times > > these > > > > are > > > > > > > > > translated > > > > > > > > > > to zero length blobs. This is not really the same thing. > > > > > > > > > > > > > > > > > > > > From this perspective, what can Kafka represent and what > is > > > > > > > considered > > > > > > > > a > > > > > > > > > > valid value? > > > > > > > > > > > > > > > > > > > > If the user sends a byte array of length 0, or if the > > > > serializer > > > > > > > sends > > > > > > > > > > something of length 0, this should be a valid value. It > is > > > not > > > > > > > kafka's > > > > > > > > > job > > > > > > > > > > to determine what the user is trying to send. For all we > > > know, > > > > > the > > > > > > > user > > > > > > > > > has > > > > > > > > > > a really good compression serializer that sends 0 bytes > > when > > > > > > nothing > > > > > > > > has > > > > > > > > > > changed. > > > > > > > > > > > > > > > > > > > > If the user is allowed to send null then some behavior > > should > > > > be > > > > > > > > defined. > > > > > > > > > > However, this should semantically be different than > > sending a > > > > > > > command. > > > > > > > > It > > > > > > > > > > is possible for a null value could signify some form of > > > delete, > > > > > > like > > > > > > > > > > "delete all messages with this key". However, if kafka > has > > a > > > > goal > > > > > > to > > > > > > > > > write > > > > > > > > > > good, readable code, then this should not be allowed. > > > > > > > > > > > > > > > > > > > > A delete or a purge is a call that can have certain side > > > > effects > > > > > or > > > > > > > > > > encounter errors that are unrelated to a send call. > > > > > > > > > > > > > > > > > > > > A couple of bullets from the Kafka style guide ( > > > > > > > > > > http://kafka.apache.org/coding-guide.html ): > > > > > > > > > > > > > > > > > > > > - Clear code is preferable to comments. When possible > make > > > your > > > > > > > naming > > > > > > > > so > > > > > > > > > > good you don't need comments. > > > > > > > > > > - Logging, configuration, and public APIs are our "UI". > > Make > > > > them > > > > > > > > pretty, > > > > > > > > > > consistent, and usable. > > > > > > > > > > > > > > > > > > > > If you want sendTombstone() functionality make the > protocol > > > > > reflect > > > > > > > > that. > > > > > > > > > > > > > > > > > > > > Right now we're struggling because we did not have a > clear > > > > > > separation > > > > > > > > of > > > > > > > > > > concerns. > > > > > > > > > > > > > > > > > > > > I don't have a good solution for deployment. I'm in favor > > of > > > a > > > > > > harsh > > > > > > > > > road: > > > > > > > > > > - Add the flag/variable/header for tombstone > > > > > > > > > > - Add an API for tombstone behavior if needed > > > > > > > > > > - Error if somebody tries to API send null > > > > > > > > > > - Accept if somebody tries to API send byte[] length 0 > > > > > > > > > > - Rev broker version > > > > > > > > > > - broker accept flag/variable/header as tombstone > > > > > > > > > > - broker accept zero length values as normal messages > > > > > > > > > > > > > > > > > > > > The more gentle road would be: > > > > > > > > > > - Add the flag/variable/header for tombstone > > > > > > > > > > - Add an API for tombstone behavior if needed > > > > > > > > > > - warn/deprecate/etc if somebody sends null > > > > > > > > > > - broker accepts flags/variable/headers and zero length > > > values > > > > as > > > > > > > > > > tombstones > > > > > > > > > > > > > > > > > > > > Conversion from one to the other may be possible but I > > would > > > > > > rather > > > > > > > > not > > > > > > > > > > have to do it. > > > > > > > > > > > > > > > > > > > > Nacho > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 9, 2016 at 9:03 PM, radai < > > > > > radai.rosenbl...@gmail.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > my personal opinion - a log compacted topic is > basically > > a > > > > > > > kv-store, > > > > > > > > > so a > > > > > > > > > > > map API. > > > > > > > > > > > map.put(key, null) is not the same as map.remove(key), > > > which > > > > to > > > > > > me > > > > > > > > > means > > > > > > > > > > a > > > > > > > > > > > null value should not represent a delete. a delete > should > > > be > > > > > > > explicit > > > > > > > > > > > (meaning flag). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 9, 2016 at 11:01 AM, Mayuresh Gharat < > > > > > > > > > > > gharatmayures...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > I see the reasoning and might be inclined to agree a > > bit > > > : > > > > > > > > > > > > If we go to stage 2, the only difference is that we > can > > > > > > > > theoretically > > > > > > > > > > > > support a null value non-tombstone message in a log > > > > compacted > > > > > > > > topic, > > > > > > > > > > but > > > > > > > > > > > I > > > > > > > > > > > > am not sure if that has any use case. > > > > > > > > > > > > > > > > > > > > > > > > But as an end goal I see that kafka should clearly > > > specify > > > > > what > > > > > > > it > > > > > > > > > > means > > > > > > > > > > > by > > > > > > > > > > > > a tombstone : is it the attribute flag OR is it the > > null > > > > > value. > > > > > > > If > > > > > > > > we > > > > > > > > > > > just > > > > > > > > > > > > do stage 1, I don't think we are defining the > end-goal > > > > > > > completely. > > > > > > > > > > > > Again this is more about semantics of correctness of > > end > > > > > state. > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > Mayuresh > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 9, 2016 at 10:49 AM, Becket Qin < > > > > > > > becket....@gmail.com> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if we need the second stage. Wouldn't > > it > > > be > > > > > > > enough > > > > > > > > to > > > > > > > > > > say > > > > > > > > > > > > > that a message is a tombstone if one of the > following > > > is > > > > > > true? > > > > > > > > > > > > > 1. tombstone flag is set. > > > > > > > > > > > > > 2. value is null. > > > > > > > > > > > > > > > > > > > > > > > > > > If we go to stage 2, the only difference is that we > > can > > > > > > > > > theoretically > > > > > > > > > > > > > support a null value non-tombstone message in a log > > > > > compacted > > > > > > > > > topic, > > > > > > > > > > > but > > > > > > > > > > > > I > > > > > > > > > > > > > am not sure if that has any use case. > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 9, 2016 at 9:23 AM, Mayuresh Gharat < > > > > > > > > > > > > > gharatmayures...@gmail.com> > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > I think it will be a good idea. +1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > Mayuresh > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 9, 2016 at 9:13 AM, Michael Pearce < > > > > > > > > > > > michael.pea...@ig.com> > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 Mayuresh, I think this is a good > > > > solution/strategy. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Shall we update the KIP with this? > > Becket/Jun/Joel > > > > any > > > > > > > > comments > > > > > > > > > > to > > > > > > > > > > > > add > > > > > > > > > > > > > > > before we do? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 08/11/2016, 17:29, "Mayuresh Gharat" < > > > > > > > > > > > gharatmayures...@gmail.com> > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think the migration can be done in 2 > > stages : > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 1) In first stage the broker should > > understand > > > > the > > > > > > > > > attribute > > > > > > > > > > > flag > > > > > > > > > > > > > as > > > > > > > > > > > > > > > well > > > > > > > > > > > > > > > as Null for the value for log compaction. > > > > > > > > > > > > > > > 2) In second stage we move on to supporting > > > only > > > > > the > > > > > > > > > > attribute > > > > > > > > > > > > flag > > > > > > > > > > > > > > > for log > > > > > > > > > > > > > > > compaction. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I agree with Becket that for older clients > > > > > > (consumers) > > > > > > > > the > > > > > > > > > > > broker > > > > > > > > > > > > > > might > > > > > > > > > > > > > > > have to down convert a message that has the > > > > > attribute > > > > > > > > flag > > > > > > > > > > set > > > > > > > > > > > > for > > > > > > > > > > > > > > log > > > > > > > > > > > > > > > compacting but has a non null value. But > this > > > > > should > > > > > > be > > > > > > > > in > > > > > > > > > > > first > > > > > > > > > > > > > > stage. > > > > > > > > > > > > > > > Once all the clients have upgraded (clients > > > start > > > > > > > > > recognizing > > > > > > > > > > > the > > > > > > > > > > > > > > > attribute > > > > > > > > > > > > > > > flag), we can move the broker to stage 2. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Mayuresh > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 8, 2016 at 12:06 AM, Michael > > > Pearce < > > > > > > > > > > > > > > michael.pea...@ig.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Also we can add further guidance: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > To avoid the below caveat to > organisations > > > by > > > > > > > > promoting > > > > > > > > > of > > > > > > > > > > > > > > > upgrading all > > > > > > > > > > > > > > > > consumers first before relying on > producing > > > > > > tombstone > > > > > > > > > > > messages > > > > > > > > > > > > > with > > > > > > > > > > > > > > > data > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sent using OWA for iPhone > > > > > > > > > > > > > > > > ________________________________________ > > > > > > > > > > > > > > > > From: Michael Pearce > > > > > > > > > > > > > > > > Sent: Tuesday, November 8, 2016 8:03:32 > AM > > > > > > > > > > > > > > > > To: dev@kafka.apache.org > > > > > > > > > > > > > > > > Subject: Re: [DISCUSS] KIP-87 - Add > > > Compaction > > > > > > > > Tombstone > > > > > > > > > > Flag > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks Jun on the feedback, I think I > > > > understand > > > > > > the > > > > > > > > > > > > issue/point > > > > > > > > > > > > > > now. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We def can add that on older client > version > > > if > > > > > > > > tombstone > > > > > > > > > > > marker > > > > > > > > > > > > > > make > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > value null to preserve behaviour. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > There is one caveats to this: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > * we have to be clear that data is lost > if > > > > > reading > > > > > > > via > > > > > > > > > old > > > > > > > > > > > > > > > client/message > > > > > > > > > > > > > > > > format - I don't think this is a big > issue > > as > > > > > > mostly > > > > > > > > the > > > > > > > > > > > > idea/use > > > > > > > > > > > > > > > case is > > > > > > > > > > > > > > > > around meta data transport as such would > > only > > > > be > > > > > as > > > > > > > bad > > > > > > > > > as > > > > > > > > > > > > > current > > > > > > > > > > > > > > > situation > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Re having configurable broker this was to > > > > handle > > > > > > > cases > > > > > > > > > like > > > > > > > > > > > you > > > > > > > > > > > > > > > described > > > > > > > > > > > > > > > > but in another way by allowing > organisation > > > > > choose > > > > > > > the > > > > > > > > > > > > behaviour > > > > > > > > > > > > > of > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > compaction per broker or per topic so > they > > > > could > > > > > > > manage > > > > > > > > > > their > > > > > > > > > > > > > > > transition to > > > > > > > > > > > > > > > > using tombstone markers. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On hind sight it maybe easier to just > > upgrade > > > > and > > > > > > > > > downgrade > > > > > > > > > > > the > > > > > > > > > > > > > > > messages > > > > > > > > > > > > > > > > on version as you propose. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sent using OWA for iPhone > > > > > > > > > > > > > > > > ________________________________________ > > > > > > > > > > > > > > > > From: Jun Rao <j...@confluent.io> > > > > > > > > > > > > > > > > Sent: Tuesday, November 8, 2016 12:34:41 > AM > > > > > > > > > > > > > > > > To: dev@kafka.apache.org > > > > > > > > > > > > > > > > Subject: Re: [DISCUSS] KIP-87 - Add > > > Compaction > > > > > > > > Tombstone > > > > > > > > > > Flag > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > For the use case, one potential use case > is > > > for > > > > > > > schema > > > > > > > > > > > > > > registration. > > > > > > > > > > > > > > > For > > > > > > > > > > > > > > > > example, in Avro, a null value > corresponds > > > to a > > > > > > Null > > > > > > > > > > schema. > > > > > > > > > > > > So, > > > > > > > > > > > > > if > > > > > > > > > > > > > > > you > > > > > > > > > > > > > > > > want to be able to keep the schema id in > a > > > > delete > > > > > > > > > message, > > > > > > > > > > > the > > > > > > > > > > > > > > value > > > > > > > > > > > > > > > can't > > > > > > > > > > > > > > > > be null. We could get around this issue > by > > > > > > > specializing > > > > > > > > > > null > > > > > > > > > > > > > value > > > > > > > > > > > > > > > during > > > > > > > > > > > > > > > > schema registration though. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Now for the proposed changes. We probably > > > > should > > > > > > > > preserve > > > > > > > > > > > > client > > > > > > > > > > > > > > > > compatibility. If a client application is > > > > > sending a > > > > > > > > null > > > > > > > > > > > value > > > > > > > > > > > > > to a > > > > > > > > > > > > > > > > compacted topic, ideally, it should work > > the > > > > same > > > > > > > after > > > > > > > > > the > > > > > > > > > > > > > client > > > > > > > > > > > > > > > > upgrades. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure about making the tombstone > > > marker > > > > > > > > > > configurable, > > > > > > > > > > > > > > > especially at > > > > > > > > > > > > > > > > the topic level. Should we allow users to > > > > change > > > > > > the > > > > > > > > > config > > > > > > > > > > > > > values > > > > > > > > > > > > > > > back and > > > > > > > > > > > > > > > > forth, and what would be the implication? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 7, 2016 at 10:48 AM, Becket > > Qin < > > > > > > > > > > > > > becket....@gmail.com> > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Michael, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, changing the logic in the log > > cleaner > > > > > makes > > > > > > > > sense. > > > > > > > > > > > There > > > > > > > > > > > > > > > could be > > > > > > > > > > > > > > > > some > > > > > > > > > > > > > > > > > other thing worth thinking (e.g. the > > > message > > > > > size > > > > > > > > > change > > > > > > > > > > > > after > > > > > > > > > > > > > > > > conversion), > > > > > > > > > > > > > > > > > though. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario I was thinking is the > > > following: > > > > > > > > > > > > > > > > > Imagine a distributed caching system > > built > > > on > > > > > top > > > > > > > of > > > > > > > > > > > Kafka. A > > > > > > > > > > > > > > user > > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > > consuming from a topic and it is > > guaranteed > > > > > that > > > > > > if > > > > > > > > the > > > > > > > > > > > user > > > > > > > > > > > > > > > consume to > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > log end it will get the latest value > for > > > all > > > > > the > > > > > > > > keys. > > > > > > > > > > > > > Currently > > > > > > > > > > > > > > > if the > > > > > > > > > > > > > > > > > consumer sees a null value it knows the > > key > > > > has > > > > > > > been > > > > > > > > > > > removed. > > > > > > > > > > > > > Now > > > > > > > > > > > > > > > let's > > > > > > > > > > > > > > > > say > > > > > > > > > > > > > > > > > we rolled out this change. And the > > producer > > > > > > > applies a > > > > > > > > > > > message > > > > > > > > > > > > > > with > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > tombstone flag set, but the value was > not > > > > null. > > > > > > > When > > > > > > > > we > > > > > > > > > > > > append > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > message > > > > > > > > > > > > > > > > > to the log I suppose we will not do the > > > down > > > > > > > > conversion > > > > > > > > > > if > > > > > > > > > > > > the > > > > > > > > > > > > > > > broker has > > > > > > > > > > > > > > > > > set the message.format.version to the > > > latest. > > > > > > > Because > > > > > > > > > the > > > > > > > > > > > log > > > > > > > > > > > > > > > cleaner > > > > > > > > > > > > > > > > won't > > > > > > > > > > > > > > > > > touch the active log segment, so that > > > message > > > > > > will > > > > > > > be > > > > > > > > > > > sitting > > > > > > > > > > > > > in > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > active > > > > > > > > > > > > > > > > > segment as is. Now when a consumer that > > > > hasn't > > > > > > > > upgraded > > > > > > > > > > yet > > > > > > > > > > > > > > > consumes that > > > > > > > > > > > > > > > > > tombstone message in the active > segment, > > it > > > > > seems > > > > > > > > that > > > > > > > > > > the > > > > > > > > > > > > > broker > > > > > > > > > > > > > > > will > > > > > > > > > > > > > > > > need > > > > > > > > > > > > > > > > > to down convert that message to remove > > the > > > > > value, > > > > > > > > > right? > > > > > > > > > > In > > > > > > > > > > > > > this > > > > > > > > > > > > > > > case, we > > > > > > > > > > > > > > > > > cannot wait for the log cleaner to do > the > > > > down > > > > > > > > > conversion > > > > > > > > > > > > > because > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > > message may have already been consumed > > > before > > > > > the > > > > > > > log > > > > > > > > > > > > > compaction > > > > > > > > > > > > > > > happens. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 7, 2016 at 9:59 AM, Michael > > > > Pearce > > > > > < > > > > > > > > > > > > > > > michael.pea...@ig.com> > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Becket, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We were thinking more about having > the > > > > logic > > > > > > > that’s > > > > > > > > > in > > > > > > > > > > > the > > > > > > > > > > > > > > method > > > > > > > > > > > > > > > > > > shouldRetainMessage configurable via > > > > > > > > > > > > > http://kafka.apache.org/ > > > > > > > > > > > > > > > > > > documentation.html#brokerconfigs > at a > > > > > > > > broker/topic > > > > > > > > > > > level. > > > > > > > > > > > > > And > > > > > > > > > > > > > > > then > > > > > > > > > > > > > > > > > scrap > > > > > > > > > > > > > > > > > > auto converting the message, and > allow > > > > > > > > organisations > > > > > > > > > to > > > > > > > > > > > > > manage > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > rollout > > > > > > > > > > > > > > > > > > of enabling of the feature. > > > > > > > > > > > > > > > > > > (this isn’t in documentation but in > > > > response > > > > > to > > > > > > > the > > > > > > > > > > > > > discussion > > > > > > > > > > > > > > > thread > > > > > > > > > > > > > > > > as > > > > > > > > > > > > > > > > > > an alternative approach to roll out > the > > > > > > feature) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Does this make any more sense? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > Mike > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 11/3/16, 2:27 PM, "Becket Qin" < > > > > > > > > > > becket....@gmail.com> > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Michael, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Do you mean using a new > > configuration > > > > it > > > > > is > > > > > > > > just > > > > > > > > > > the > > > > > > > > > > > > > > exiting > > > > > > > > > > > > > > > > > > message.format.version config? It > > > seems > > > > > the > > > > > > > > > > > > > > > message.format.version > > > > > > > > > > > > > > > > > > config > > > > > > > > > > > > > > > > > > is enough in this case. And the > > > default > > > > > > value > > > > > > > > > would > > > > > > > > > > > > > always > > > > > > > > > > > > > > > be the > > > > > > > > > > > > > > > > > > latest > > > > > > > > > > > > > > > > > > version. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Message version migration would > > be > > > > > > handled > > > > > > > as > > > > > > > > > > like > > > > > > > > > > > in > > > > > > > > > > > > > > > KIP-32 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Also just want to confirm on > this. > > > > Today > > > > > if > > > > > > > an > > > > > > > > > old > > > > > > > > > > > > > consumer > > > > > > > > > > > > > > > > consumes > > > > > > > > > > > > > > > > > a > > > > > > > > > > > > > > > > > > log > > > > > > > > > > > > > > > > > > compacted topic and sees an empty > > > > value, > > > > > it > > > > > > > > knows > > > > > > > > > > > that > > > > > > > > > > > > > is a > > > > > > > > > > > > > > > > > tombstone. > > > > > > > > > > > > > > > > > > After we start to use the > attribute > > > > bit, > > > > > a > > > > > > > > > > tombstone > > > > > > > > > > > > > > message > > > > > > > > > > > > > > > can > > > > > > > > > > > > > > > > > have a > > > > > > > > > > > > > > > > > > non-empty value. So by "like in > > > KIP-32" > > > > > you > > > > > > > > mean > > > > > > > > > we > > > > > > > > > > > > will > > > > > > > > > > > > > > > remove the > > > > > > > > > > > > > > > > > > value > > > > > > > > > > > > > > > > > > to down convert the message if > the > > > > > consumer > > > > > > > > > version > > > > > > > > > > > is > > > > > > > > > > > > > old, > > > > > > > > > > > > > > > right? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 2, 2016 at 1:37 AM, > > > Michael > > > > > > > Pearce > > > > > > > > < > > > > > > > > > > > > > > > > > michael.pea...@ig.com> > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Joel , et al. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Any comments on the below idea > to > > > > > handle > > > > > > > roll > > > > > > > > > > out / > > > > > > > > > > > > > > > compatibility > > > > > > > > > > > > > > > > > of > > > > > > > > > > > > > > > > > > this > > > > > > > > > > > > > > > > > > > feature, using a configuration? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Does it make sense/clear? > > > > > > > > > > > > > > > > > > > Does it add value? > > > > > > > > > > > > > > > > > > > Do we want to enforce flag by > > > > default, > > > > > or > > > > > > > > value > > > > > > > > > > by > > > > > > > > > > > > > > > default, or > > > > > > > > > > > > > > > > > both? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Cheers > > > > > > > > > > > > > > > > > > > Mike > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 10/27/16, 4:47 PM, "Michael > > > > Pearce" > > > > > < > > > > > > > > > > > > > > > michael.pea...@ig.com> > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, James, I think this > > is > > > a > > > > > > really > > > > > > > > > good > > > > > > > > > > > > > addition > > > > > > > > > > > > > > > to the > > > > > > > > > > > > > > > > > KIP > > > > > > > > > > > > > > > > > > > details, please feel free to > > amend > > > > the > > > > > > > > wiki/add > > > > > > > > > > the > > > > > > > > > > > > use > > > > > > > > > > > > > > > cases, > > > > > > > > > > > > > > > > also > > > > > > > > > > > > > > > > > > if any > > > > > > > > > > > > > > > > > > > others you think of. I > definitely > > > > think > > > > > > its > > > > > > > > > > > > worthwhile > > > > > > > > > > > > > > > > documenting. > > > > > > > > > > > > > > > > > > If you > > > > > > > > > > > > > > > > > > > can’t let me know ill add them > > next > > > > > week > > > > > > > > (just > > > > > > > > > > > > leaving > > > > > > > > > > > > > > for > > > > > > > > > > > > > > > a long > > > > > > > > > > > > > > > > > > weekend > > > > > > > > > > > > > > > > > > > off) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Re Joel and others comments > > > about > > > > > > > upgrade > > > > > > > > > and > > > > > > > > > > > > > > > compatibility. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Rather than trying to auto > > > manage > > > > > > this. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Actually maybe we make a > > > > > > configuration > > > > > > > > > > option, > > > > > > > > > > > > both > > > > > > > > > > > > > > at > > > > > > > > > > > > > > > server > > > > > > > > > > > > > > > > > > and per > > > > > > > > > > > > > > > > > > > topic level to control the > > behavior > > > > of > > > > > > how > > > > > > > > the > > > > > > > > > > > server > > > > > > > > > > > > > > logic > > > > > > > > > > > > > > > > should > > > > > > > > > > > > > > > > > > work out > > > > > > > > > > > > > > > > > > > if the record, is a tombstone > > > record > > > > . > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > e.g. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > key = > > > compation.tombstone.marker > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > value options: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > value (continues to use > > null > > > > > value > > > > > > as > > > > > > > > > > > tombstone > > > > > > > > > > > > > > > marker) > > > > > > > > > > > > > > > > > > > flag (expects to use the > > > > tombstone > > > > > > > flag) > > > > > > > > > > > > > > > > > > > value_or_flag (if either is > > > true > > > > it > > > > > > > > treats > > > > > > > > > > the > > > > > > > > > > > > > record > > > > > > > > > > > > > > > as a > > > > > > > > > > > > > > > > > > tombstone) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This way on upgrade users > can > > > > keep > > > > > > > > current > > > > > > > > > > > > > behavior, > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > slowly > > > > > > > > > > > > > > > > > > > migrate to the new. Having a > > > > transition > > > > > > > > period > > > > > > > > > of > > > > > > > > > > > > using > > > > > > > > > > > > > > > > > > value_or_flag, > > > > > > > > > > > > > > > > > > > finally having flag only if an > > > > > > organization > > > > > > > > > > wishes > > > > > > > > > > > to > > > > > > > > > > > > > use > > > > > > > > > > > > > > > null > > > > > > > > > > > > > > > > > values > > > > > > > > > > > > > > > > > > > without it being treated as a > > > > tombstone > > > > > > > > marker > > > > > > > > > > (use > > > > > > > > > > > > > case > > > > > > > > > > > > > > > noted > > > > > > > > > > > > > > > > > below) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having it both global > broker > > > > level > > > > > > and > > > > > > > > > topic > > > > > > > > > > > > > override > > > > > > > > > > > > > > > also > > > > > > > > > > > > > > > > > > allows some > > > > > > > > > > > > > > > > > > > flexibility here. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Cheers > > > > > > > > > > > > > > > > > > > Mike > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 10/27/16, 8:03 AM, > "James > > > > > Cheng" < > > > > > > > > > > > > > > > wushuja...@gmail.com> > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This KIP would > definitely > > > > > > address a > > > > > > > > gap > > > > > > > > > > in > > > > > > > > > > > > the > > > > > > > > > > > > > > > current > > > > > > > > > > > > > > > > > > > functionality, where you > > currently > > > > > can't > > > > > > > > have a > > > > > > > > > > > > > tombstone > > > > > > > > > > > > > > > with > > > > > > > > > > > > > > > > any > > > > > > > > > > > > > > > > > > > associated content. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > That said, I'd like to > > talk > > > > > about > > > > > > > use > > > > > > > > > > > cases, > > > > > > > > > > > > to > > > > > > > > > > > > > > > make sure > > > > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > > > > this is in fact useful. The KIP > > > > should > > > > > be > > > > > > > > > updated > > > > > > > > > > > > with > > > > > > > > > > > > > > > whatever > > > > > > > > > > > > > > > > use > > > > > > > > > > > > > > > > > > cases > > > > > > > > > > > > > > > > > > > we come up with. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > First of all, an > > > observation: > > > > > > When > > > > > > > we > > > > > > > > > > speak > > > > > > > > > > > > > about > > > > > > > > > > > > > > > log > > > > > > > > > > > > > > > > > > compaction, > > > > > > > > > > > > > > > > > > > we typically think of "the > latest > > > > > message > > > > > > > > for a > > > > > > > > > > key > > > > > > > > > > > > is > > > > > > > > > > > > > > > retained". > > > > > > > > > > > > > > > > > In > > > > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > > > > respect, a delete tombstone > > (i.e. a > > > > > > message > > > > > > > > > with > > > > > > > > > > a > > > > > > > > > > > > null > > > > > > > > > > > > > > > payload) > > > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > > > treated > > > > > > > > > > > > > > > > > > > the same as any other Kafka > > > message: > > > > > the > > > > > > > > latest > > > > > > > > > > > > message > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > > retained. > > > > > > > > > > > > > > > > > > It > > > > > > > > > > > > > > > > > > > doesn't matter whether the > latest > > > > > message > > > > > > > is > > > > > > > > > > null, > > > > > > > > > > > or > > > > > > > > > > > > > if > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > latest > > > > > > > > > > > > > > > > > > message > > > > > > > > > > > > > > > > > > > has actual content. In all > cases, > > > the > > > > > > last > > > > > > > > > > message > > > > > > > > > > > is > > > > > > > > > > > > > > > retained. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The only way a delete > > > > tombstone > > > > > > is > > > > > > > > > > treated > > > > > > > > > > > > > > > differently > > > > > > > > > > > > > > > > from > > > > > > > > > > > > > > > > > > other > > > > > > > > > > > > > > > > > > > Kafka messages is that it > > > > automatically > > > > > > > > > > disappears > > > > > > > > > > > > > after > > > > > > > > > > > > > > a > > > > > > > > > > > > > > > while. > > > > > > > > > > > > > > > > > > The time > > > > > > > > > > > > > > > > > > > of deletion is specified using > > > > > > > > > > delete.retention.ms > > > > > > > > > > > . > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So what we're really > > > talking > > > > > > about > > > > > > > > is, > > > > > > > > > do > > > > > > > > > > > we > > > > > > > > > > > > > want > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > support > > > > > > > > > > > > > > > > > > > messages in a log-compacted > topic > > > > that > > > > > > > > > > auto-delete > > > > > > > > > > > > > > > themselves > > > > > > > > > > > > > > > > after > > > > > > > > > > > > > > > > > > a while? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In a thread from 2015, > > > there > > > > > was > > > > > > a > > > > > > > > > > > discussion > > > > > > > > > > > > > on > > > > > > > > > > > > > > > > > first-class > > > > > > > > > > > > > > > > > > > support of headers between > Roger > > > > > Hoover, > > > > > > > > Felix > > > > > > > > > > GV, > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > Rao, and > > > > > > > > > > > > > > > > I. > > > > > > > > > > > > > > > > > > See > > > > > > > > > > > > > > > > > > > thread at > > > > https://groups.google.com/d/ > > > > > > > > > > > > > > > msg/confluent-platform/ > > > > > > > > > > > > > > > > > > > 8xPbjyUE_7E/yQ1AeCufL_gJ < > > > > > > > > > > > > https://groups.google.com/d/ > > > > > > > > > > > > > > > > > > > msg/confluent-platform/ > > > > > > > > > 8xPbjyUE_7E/yQ1AeCufL_gJ> > > > > > > > > > > . > > > > > > > > > > > > In > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > thread, > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > raised a good question that I > > > didn't > > > > > > have a > > > > > > > > > good > > > > > > > > > > > > answer > > > > > > > > > > > > > > > for at > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > time: If > > > > > > > > > > > > > > > > > > > a message is going to > auto-delete > > > > > itself > > > > > > > > after > > > > > > > > > a > > > > > > > > > > > > while, > > > > > > > > > > > > > > how > > > > > > > > > > > > > > > > > > important was > > > > > > > > > > > > > > > > > > > the message? That is, what > > > > information > > > > > > did > > > > > > > > the > > > > > > > > > > > > message > > > > > > > > > > > > > > > contain > > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > > > was > > > > > > > > > > > > > > > > > > > important *for a while* but not > > so > > > > > > > important > > > > > > > > > that > > > > > > > > > > > it > > > > > > > > > > > > > > > needed to be > > > > > > > > > > > > > > > > > > kept > > > > > > > > > > > > > > > > > > > around forever? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Some use cases that I > can > > > > think > > > > > > of: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 1) Tracability. I would > > > like > > > > to > > > > > > > know > > > > > > > > > who > > > > > > > > > > > > issued > > > > > > > > > > > > > > > this > > > > > > > > > > > > > > > > delete > > > > > > > > > > > > > > > > > > > tombstone. It might include the > > > > > hostname, > > > > > > > IP > > > > > > > > of > > > > > > > > > > the > > > > > > > > > > > > > > > producer of > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > delete. > > > > > > > > > > > > > > > > > > > 2) Timestamps. I would > > like > > > > to > > > > > > know > > > > > > > > > when > > > > > > > > > > > this > > > > > > > > > > > > > > > delete was > > > > > > > > > > > > > > > > > > issued. > > > > > > > > > > > > > > > > > > > This use case is already > > addressed > > > by > > > > > the > > > > > > > > > > > > availability > > > > > > > > > > > > > of > > > > > > > > > > > > > > > > > per-message > > > > > > > > > > > > > > > > > > > timestamps that came in 0.10.0 > > > > > > > > > > > > > > > > > > > 3) Data provenance. I > > hope > > > > I'm > > > > > > > using > > > > > > > > > this > > > > > > > > > > > > > phrase > > > > > > > > > > > > > > > > correctly, > > > > > > > > > > > > > > > > > > but > > > > > > > > > > > > > > > > > > > what I mean is, where did this > > > delete > > > > > > come > > > > > > > > > from? > > > > > > > > > > > What > > > > > > > > > > > > > > > processing > > > > > > > > > > > > > > > > > job > > > > > > > > > > > > > > > > > > > emitted it? What input to the > > > > > processing > > > > > > > job > > > > > > > > > > caused > > > > > > > > > > > > > this > > > > > > > > > > > > > > > delete > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > be > > > > > > > > > > > > > > > > > > > produced? For example, if a > > record > > > in > > > > > > > topic A > > > > > > > > > was > > > > > > > > > > > > > > > processed and > > > > > > > > > > > > > > > > > > caused a > > > > > > > > > > > > > > > > > > > delete tombstone to be emitted > to > > > > topic > > > > > > B, > > > > > > > I > > > > > > > > > > might > > > > > > > > > > > > like > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > offset > > > > > > > > > > > > > > > > > > of the > > > > > > > > > > > > > > > > > > > topic A message to be attached > to > > > the > > > > > > > topic B > > > > > > > > > > > > message. > > > > > > > > > > > > > > > > > > > 4) Distributed tracing > > for > > > > > stream > > > > > > > > > > > topologies. > > > > > > > > > > > > > > This > > > > > > > > > > > > > > > might > > > > > > > > > > > > > > > > > be a > > > > > > > > > > > > > > > > > > > slight repeat of the above use > > > cases. > > > > > In > > > > > > > the > > > > > > > > > > > > > > microservices > > > > > > > > > > > > > > > world, > > > > > > > > > > > > > > > > > we > > > > > > > > > > > > > > > > > > can > > > > > > > > > > > > > > > > > > > generate call-graphs of > > webservices > > > > > using > > > > > > > > tools > > > > > > > > > > > like > > > > > > > > > > > > > > > Zipkin/ > > > > > > > > > > > > > > > > > > opentracing.io > > > > > > > > > > > > > > > > > > > <http://opentracing.io/>, or > > > > something > > > > > > > > > homegrown > > > > > > > > > > > > like > > > > > > > > > > > > > > > > > > > https://engineering.linkedin. > > > > > > > > > > > > > > com/distributed-service-call- > > > > > > > > > > > > > > > > > > > graph/real-time-distributed- > > > > > > > > > > > > > tracing-website-performance- > > > > > > > > > > > > > > > > > and-efficiency > > > > > > > > > > > > > > > > > > < > > > > > > > > > > > > > > > > > > > https://engineering.linkedin. > > > > > > > > > > > > > > com/distributed-service-call- > > > > > > > > > > > > > > > > > > > graph/real-time-distributed- > > > > > > > > > > > > > tracing-website-performance- > > > > > > > > > > > > > > > > > > and-efficiency>. > > > > > > > > > > > > > > > > > > > I can imagine that you might > want > > > to > > > > do > > > > > > > > > something > > > > > > > > > > > > > similar > > > > > > > > > > > > > > > for > > > > > > > > > > > > > > > > > stream > > > > > > > > > > > > > > > > > > > processing topologies, where > > stream > > > > > > > > processing > > > > > > > > > > jobs > > > > > > > > > > > > > carry > > > > > > > > > > > > > > > along > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > > forward > > > > > > > > > > > > > > > > > > > along a globally unique > > identifier, > > > > > and a > > > > > > > > > > > distributed > > > > > > > > > > > > > > > topology > > > > > > > > > > > > > > > > > graph > > > > > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > > > > generated. > > > > > > > > > > > > > > > > > > > 5) Cases where > > processing a > > > > > > delete > > > > > > > > > > requires > > > > > > > > > > > > > data > > > > > > > > > > > > > > > that is > > > > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > > > > available in the message key. > I'm > > > not > > > > > > sure > > > > > > > I > > > > > > > > > > have a > > > > > > > > > > > > > good > > > > > > > > > > > > > > > example > > > > > > > > > > > > > > > > of > > > > > > > > > > > > > > > > > > this, > > > > > > > > > > > > > > > > > > > though. One hand-wavy example > > might > > > > be > > > > > > > where > > > > > > > > I > > > > > > > > > am > > > > > > > > > > > > > > > publishing > > > > > > > > > > > > > > > > > > documents into > > > > > > > > > > > > > > > > > > > Kafka where the documentId is > the > > > > > message > > > > > > > > key, > > > > > > > > > > and > > > > > > > > > > > > the > > > > > > > > > > > > > > text > > > > > > > > > > > > > > > > > contents > > > > > > > > > > > > > > > > > > of the > > > > > > > > > > > > > > > > > > > document are in the message > body. > > > > And I > > > > > > > have > > > > > > > > a > > > > > > > > > > > > > consuming > > > > > > > > > > > > > > > job that > > > > > > > > > > > > > > > > > > does some > > > > > > > > > > > > > > > > > > > analytics on the message body. > If > > > > that > > > > > > > > document > > > > > > > > > > > gets > > > > > > > > > > > > > > > deleted, > > > > > > > > > > > > > > > > then > > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > consuming job might need the > > > original > > > > > > > message > > > > > > > > > > body > > > > > > > > > > > in > > > > > > > > > > > > > > > order to > > > > > > > > > > > > > > > > > > "delete" > > > > > > > > > > > > > > > > > > > that message's impact from the > > > > > analytics. > > > > > > > But > > > > > > > > > I'm > > > > > > > > > > > not > > > > > > > > > > > > > > sure > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > > a > > > > > > > > > > > > > > > > > > great > > > > > > > > > > > > > > > > > > > example. If the consumer was > > > worried > > > > > > about > > > > > > > > > that, > > > > > > > > > > > the > > > > > > > > > > > > > > > consumer > > > > > > > > > > > > > > > > would > > > > > > > > > > > > > > > > > > > probably keep the original > > message > > > > > > around, > > > > > > > > > stored > > > > > > > > > > > by > > > > > > > > > > > > > > > primary key. > > > > > > > > > > > > > > > > > > And then > > > > > > > > > > > > > > > > > > > all it would need from a delete > > > > message > > > > > > > would > > > > > > > > > be > > > > > > > > > > > the > > > > > > > > > > > > > > > primary key > > > > > > > > > > > > > > > > of > > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > message. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Do people think these > are > > > > valid > > > > > > use > > > > > > > > > > cases? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What are other use > cases > > > that > > > > > > > people > > > > > > > > > can > > > > > > > > > > > > think > > > > > > > > > > > > > > of? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -James > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Oct 26, 2016, at > > 3:46 > > > > PM, > > > > > > > > Mayuresh > > > > > > > > > > > > Gharat > > > > > > > > > > > > > < > > > > > > > > > > > > > > > > > > > gharatmayures...@gmail.com> > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 @Joel. > > > > > > > > > > > > > > > > > > > > I think a clear > > migration > > > > > plan > > > > > > of > > > > > > > > > > > upgrading > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > > downgrading of > > > > > > > > > > > > > > > > > > > server and > > > > > > > > > > > > > > > > > > > > clients along with > > > handling > > > > > of > > > > > > > > issues > > > > > > > > > > > that > > > > > > > > > > > > > Joel > > > > > > > > > > > > > > > > > mentioned, > > > > > > > > > > > > > > > > > > on > > > > > > > > > > > > > > > > > > > the KIP would > > > > > > > > > > > > > > > > > > > > be really great. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Mayuresh > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 26, 2016 > at > > > > 3:31 > > > > > > PM, > > > > > > > > Joel > > > > > > > > > > > > Koshy < > > > > > > > > > > > > > > > > > > jjkosh...@gmail.com> > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> I'm not sure why it > > > would > > > > be > > > > > > > > useful, > > > > > > > > > > but > > > > > > > > > > > > it > > > > > > > > > > > > > > > should be > > > > > > > > > > > > > > > > > > > theoretically > > > > > > > > > > > > > > > > > > > >> possible if the > > > attribute > > > > > bit > > > > > > > > alone > > > > > > > > > is > > > > > > > > > > > > > enough > > > > > > > > > > > > > > > to mark > > > > > > > > > > > > > > > > a > > > > > > > > > > > > > > > > > > > tombstone. OTOH, we > > > > > > > > > > > > > > > > > > > >> could consider that > as > > > > > invalid > > > > > > > if > > > > > > > > we > > > > > > > > > > > wish. > > > > > > > > > > > > > > > These are > > > > > > > > > > > > > > > > > > relevant > > > > > > > > > > > > > > > > > > > details that > > > > > > > > > > > > > > > > > > > >> I think should be > > added > > > to > > > > > the > > > > > > > > KIP. > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > >> Also, in the few odd > > > > > scenarios > > > > > > > > that > > > > > > > > > I > > > > > > > > > > > > > > mentioned > > > > > > > > > > > > > > > we > > > > > > > > > > > > > > > > > should > > > > > > > > > > > > > > > > > > also > > > > > > > > > > > > > > > > > > > consider > > > > > > > > > > > > > > > > > > > >> that fetches could > be > > > > coming > > > > > > > from > > > > > > > > > > other > > > > > > > > > > > > > > > > > yet-to-be-upgraded > > > > > > > > > > > > > > > > > > > brokers in a > > > > > > > > > > > > > > > > > > > >> cluster that is > being > > > > > > upgraded. > > > > > > > So > > > > > > > > > we > > > > > > > > > > > > would > > > > > > > > > > > > > > > probably > > > > > > > > > > > > > > > > > want > > > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > continue to > > > > > > > > > > > > > > > > > > > >> support nulls as > > > > tombstones > > > > > or > > > > > > > > > > > > down-convert > > > > > > > > > > > > > in > > > > > > > > > > > > > > > a way > > > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > > > we > > > > > > > > > > > > > > > > > > > are sure works > > > > > > > > > > > > > > > > > > > >> with least surprise > to > > > > > > fetchers. > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > >> There is a slightly > > > vague > > > > > > > > statement > > > > > > > > > > > under > > > > > > > > > > > > > > > > > "Compatibility, > > > > > > > > > > > > > > > > > > > Deprecation, and > > > > > > > > > > > > > > > > > > > >> Migration Plan" that > > > could > > > > > > > benefit > > > > > > > > > > more > > > > > > > > > > > > > > details: > > > > > > > > > > > > > > > > *Logic > > > > > > > > > > > > > > > > > > would > > > > > > > > > > > > > > > > > > > base on > > > > > > > > > > > > > > > > > > > >> current behavior of > > null > > > > > value > > > > > > > or > > > > > > > > if > > > > > > > > > > > > > tombstone > > > > > > > > > > > > > > > flag > > > > > > > > > > > > > > > > set > > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > true, as such > > > > > > > > > > > > > > > > > > > >> wouldn't impact any > > > > existing > > > > > > > flows > > > > > > > > > > > simply > > > > > > > > > > > > > > allow > > > > > > > > > > > > > > > new > > > > > > > > > > > > > > > > > > producers > > > > > > > > > > > > > > > > > > > to make use > > > > > > > > > > > > > > > > > > > >> of the feature*. It > is > > > > > unclear > > > > > > > to > > > > > > > > me > > > > > > > > > > > based > > > > > > > > > > > > > on > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > > > whether you > > > > > > > > > > > > > > > > > > > would > > > > > > > > > > > > > > > > > > > >> interpret null as a > > > > > tombstone > > > > > > if > > > > > > > > the > > > > > > > > > > > > > tombstone > > > > > > > > > > > > > > > > attribute > > > > > > > > > > > > > > > > > > bit is > > > > > > > > > > > > > > > > > > > off. > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > >> On Wed, Oct 26, 2016 > > at > > > > 3:10 > > > > > > PM, > > > > > > > > > > Xavier > > > > > > > > > > > > > > Léauté < > > > > > > > > > > > > > > > > > > > xav...@confluent.io> > > > > > > > > > > > > > > > > > > > >> wrote: > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > >>> Does this mean that > > > > > starting > > > > > > > with > > > > > > > > > V4 > > > > > > > > > > > > > requests > > > > > > > > > > > > > > > we > > > > > > > > > > > > > > > > would > > > > > > > > > > > > > > > > > > allow > > > > > > > > > > > > > > > > > > > storing null > > > > > > > > > > > > > > > > > > > >>> messages in > compacted > > > > > topics? > > > > > > > The > > > > > > > > > KIP > > > > > > > > > > > > > should > > > > > > > > > > > > > > > probably > > > > > > > > > > > > > > > > > > clarify > > > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > >> behavior > > > > > > > > > > > > > > > > > > > >>> for null messages > > where > > > > the > > > > > > > > > tombstone > > > > > > > > > > > > flag > > > > > > > > > > > > > is > > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > net. > > > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > >>> On Wed, Oct 26, > 2016 > > at > > > > > 1:32 > > > > > > AM > > > > > > > > > > Magnus > > > > > > > > > > > > > > > Edenhill < > > > > > > > > > > > > > > > > > > > mag...@edenhill.se> > > > > > > > > > > > > > > > > > > > >>> wrote: > > > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > >>>> 2016-10-25 21:36 > > > > GMT+02:00 > > > > > > > Nacho > > > > > > > > > > Solis > > > > > > > > > > > > > > > > > > > <nso...@linkedin.com.invalid>: > > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > > >>>>> I think you > > probably > > > > > > require > > > > > > > a > > > > > > > > > > > > MagicByte > > > > > > > > > > > > > > > bump if > > > > > > > > > > > > > > > > you > > > > > > > > > > > > > > > > > > expect > > > > > > > > > > > > > > > > > > > correct > > > > > > > > > > > > > > > > > > > >>>>> behavior of the > > > system > > > > > as a > > > > > > > > > whole. > > > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > > >>>>> From a client > > > > perspective > > > > > > you > > > > > > > > > want > > > > > > > > > > to > > > > > > > > > > > > > make > > > > > > > > > > > > > > > sure > > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > > > when you > > > > > > > > > > > > > > > > > > > >> deliver a > > > > > > > > > > > > > > > > > > > >>>>> message that the > > > broker > > > > > > > > supports > > > > > > > > > > the > > > > > > > > > > > > > > feature > > > > > > > > > > > > > > > you're > > > > > > > > > > > > > > > > > > expecting > > > > > > > > > > > > > > > > > > > >>>>> (compaction). > So, > > > > > > depending > > > > > > > on > > > > > > > > > the > > > > > > > > > > > > > > behavior > > > > > > > > > > > > > > > of the > > > > > > > > > > > > > > > > > > broker on > > > > > > > > > > > > > > > > > > > >>>> encountering > > > > > > > > > > > > > > > > > > > >>>>> a previously > > > undefined > > > > > bit > > > > > > > > flag I > > > > > > > > > > > would > > > > > > > > > > > > > > > suggest > > > > > > > > > > > > > > > > > making > > > > > > > > > > > > > > > > > > some > > > > > > > > > > > > > > > > > > > change to > > > > > > > > > > > > > > > > > > > >>>> make > > > > > > > > > > > > > > > > > > > >>>>> certain that > > > flag-based > > > > > > > > > compaction > > > > > > > > > > is > > > > > > > > > > > > > > > supported. > > > > > > > > > > > > > > > > I'm > > > > > > > > > > > > > > > > > > going > > > > > > > > > > > > > > > > > > > to guess > > > > > > > > > > > > > > > > > > > >>> that > > > > > > > > > > > > > > > > > > > >>>>> the MagicByte > would > > > do > > > > > > this. > > > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > > >>>> I dont believe > this > > is > > > > > > needed > > > > > > > > > since > > > > > > > > > > it > > > > > > > > > > > > is > > > > > > > > > > > > > > > already > > > > > > > > > > > > > > > > > > attributed > > > > > > > > > > > > > > > > > > > through > > > > > > > > > > > > > > > > > > > >> the > > > > > > > > > > > > > > > > > > > >>>> request's API > > version. > > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > > >>>> Producer: > > > > > > > > > > > > > > > > > > > >>>> * if a client > sends > > > > > > > > ProduceRequest > > > > > > > > > > V4 > > > > > > > > > > > > then > > > > > > > > > > > > > > > > > > attributes.bit5 > > > > > > > > > > > > > > > > > > > indicates a > > > > > > > > > > > > > > > > > > > >>>> tombstone > > > > > > > > > > > > > > > > > > > >>>> * if a clients > sends > > > > > > > > > ProduceRequest > > > > > > > > > > > <V4 > > > > > > > > > > > > > then > > > > > > > > > > > > > > > > > > attributes.bit5 > > > > > > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > > > > >> ignored > > > > > > > > > > > > > > > > > > > >>>> and value==null > > > > indicates > > > > > a > > > > > > > > > > tombstone > > > > > > > > > > > > > > > > > > > >>>> * in both cases > the > > > > > on-disk > > > > > > > > > messages > > > > > > > > > > > are > > > > > > > > > > > > > > > stored with > > > > > > > > > > > > > > > > > > > attributes.bit5 > > > > > > > > > > > > > > > > > > > >> (I > > > > > > > > > > > > > > > > > > > >>>> assume?) > > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > > >>>> Consumer: > > > > > > > > > > > > > > > > > > > >>>> * if a clients > sends > > > > > > > > FetchRequest > > > > > > > > > V4 > > > > > > > > > > > > > > messages > > > > > > > > > > > > > > > are > > > > > > > > > > > > > > > > > > > sendfile():ed > > > > > > > > > > > > > > > > > > > >> directly > > > > > > > > > > > > > > > > > > > >>>> from disk (with > > > > > > > attributes.bit5) > > > > > > > > > > > > > > > > > > > >>>> * if a client > sends > > > > > > > FetchRequest > > > > > > > > > <V4 > > > > > > > > > > > > > > messages > > > > > > > > > > > > > > > are > > > > > > > > > > > > > > > > > > slowpathed > > > > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > > > >>>> translated from > > > > > > > attributes.bit5 > > > > > > > > to > > > > > > > > > > > > > > value=null > > > > > > > > > > > > > > > as > > > > > > > > > > > > > > > > > > required. > > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > > >>>> That's my > > > understanding > > > > > > > anyway, > > > > > > > > > > please > > > > > > > > > > > > > > > correct me if > > > > > > > > > > > > > > > > > I'm > > > > > > > > > > > > > > > > > > > wrong. > > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > > >>>> /Magnus > > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > > >>>>> On Tue, Oct 25, > > 2016 > > > at > > > > > > 10:17 > > > > > > > > AM, > > > > > > > > > > > > Magnus > > > > > > > > > > > > > > > Edenhill < > > > > > > > > > > > > > > > > > > > >> mag...@edenhill.se> > > > > > > > > > > > > > > > > > > > >>>>> wrote: > > > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > > >>>>>> It is safe to > > assume > > > > > that > > > > > > a > > > > > > > > > > > previously > > > > > > > > > > > > > > > undefined > > > > > > > > > > > > > > > > > > attributes > > > > > > > > > > > > > > > > > > > bit > > > > > > > > > > > > > > > > > > > >> will > > > > > > > > > > > > > > > > > > > >>> be > > > > > > > > > > > > > > > > > > > >>>>>> unset in > protocol > > > > > requests > > > > > > > > from > > > > > > > > > > > > existing > > > > > > > > > > > > > > > clients, > > > > > > > > > > > > > > > > if > > > > > > > > > > > > > > > > > > not, > > > > > > > > > > > > > > > > > > > such a > > > > > > > > > > > > > > > > > > > >>> client > > > > > > > > > > > > > > > > > > > >>>>> is > > > > > > > > > > > > > > > > > > > >>>>>> already > violating > > > the > > > > > > > protocol > > > > > > > > > and > > > > > > > > > > > > needs > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > be > > > > > > > > > > > > > > > > > fixed. > > > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > > > >>>>>> So I dont see a > > need > > > > > for a > > > > > > > > > > MagicByte > > > > > > > > > > > > > bump, > > > > > > > > > > > > > > > both > > > > > > > > > > > > > > > > > > broker and > > > > > > > > > > > > > > > > > > > client > > > > > > > > > > > > > > > > > > > >> has > > > > > > > > > > > > > > > > > > > >>>> the > > > > > > > > > > > > > > > > > > > >>>>>> information it > > needs > > > > to > > > > > > > > > construct > > > > > > > > > > or > > > > > > > > > > > > > parse > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > message > > > > > > > > > > > > > > > > > > > according to > > > > > > > > > > > > > > > > > > > >>>>> request > > > > > > > > > > > > > > > > > > > >>>>>> version. > > > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > > > >>>>>> 2016-10-25 18:48 > > > > > GMT+02:00 > > > > > > > > > Michael > > > > > > > > > > > > > Pearce > > > > > > > > > > > > > > < > > > > > > > > > > > > > > > > > > > michael.pea...@ig.com>: > > > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>> Hi Magnus, > > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>> I was wondering > > if > > > I > > > > > even > > > > > > > > > needed > > > > > > > > > > to > > > > > > > > > > > > > > change > > > > > > > > > > > > > > > those > > > > > > > > > > > > > > > > > > also, as > > > > > > > > > > > > > > > > > > > >>> technically > > > > > > > > > > > > > > > > > > > >>>>>>> we’re just > making > > > use > > > > > of > > > > > > a > > > > > > > > non > > > > > > > > > > used > > > > > > > > > > > > > > > attribute > > > > > > > > > > > > > > > > bit, > > > > > > > > > > > > > > > > > > but im > > > > > > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > > > > >> 100% > > > > > > > > > > > > > > > > > > > >>>> that > > > > > > > > > > > > > > > > > > > >>>>>> it > > > > > > > > > > > > > > > > > > > >>>>>>> be always false > > > > > > currently. > > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>> If someone can > > say > > > > 100% > > > > > > it > > > > > > > > will > > > > > > > > > > > > already > > > > > > > > > > > > > > be > > > > > > > > > > > > > > > set > > > > > > > > > > > > > > > > > false > > > > > > > > > > > > > > > > > > with > > > > > > > > > > > > > > > > > > > current > > > > > > > > > > > > > > > > > > > >>> and > > > > > > > > > > > > > > > > > > > >>>>>>> historic bit > wise > > > > > masking > > > > > > > > > > > techniques > > > > > > > > > > > > > used > > > > > > > > > > > > > > > over > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > time, > > > > > > > > > > > > > > > > > > > we could > > > > > > > > > > > > > > > > > > > >>> do > > > > > > > > > > > > > > > > > > > >>>>> away > > > > > > > > > > > > > > > > > > > >>>>>>> with both, and > > > simply > > > > > > just > > > > > > > > > start > > > > > > > > > > to > > > > > > > > > > > > use > > > > > > > > > > > > > > it. > > > > > > > > > > > > > > > > > > Unfortunately > > > > > > > > > > > > > > > > > > > I don’t > > > > > > > > > > > > > > > > > > > >>>> have > > > > > > > > > > > > > > > > > > > >>>>>> that > > > > > > > > > > > > > > > > > > > >>>>>>> historic > > knowledge > > > so > > > > > was > > > > > > > > > hoping > > > > > > > > > > it > > > > > > > > > > > > > would > > > > > > > > > > > > > > > be > > > > > > > > > > > > > > > > > flagged > > > > > > > > > > > > > > > > > > up in > > > > > > > > > > > > > > > > > > > this > > > > > > > > > > > > > > > > > > > >>>>>> discussion > > > > > > > > > > > > > > > > > > > >>>>>>> thread ? > > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>> Cheers > > > > > > > > > > > > > > > > > > > >>>>>>> Mike > > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>> On 10/25/16, > 5:36 > > > PM, > > > > > > > "Magnus > > > > > > > > > > > > > Edenhill" < > > > > > > > > > > > > > > > > > > > mag...@edenhill.se> > > > > > > > > > > > > > > > > > > > >>> wrote: > > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>> Hi Michael, > > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>> With the > > version > > > > > bumps > > > > > > > for > > > > > > > > > > > Produce > > > > > > > > > > > > > and > > > > > > > > > > > > > > > Fetch > > > > > > > > > > > > > > > > > > requests, > > > > > > > > > > > > > > > > > > > do you > > > > > > > > > > > > > > > > > > > >>>>> really > > > > > > > > > > > > > > > > > > > >>>>>>> need > > > > > > > > > > > > > > > > > > > >>>>>>> to bump > > > MagicByte > > > > > too? > > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>> Regards, > > > > > > > > > > > > > > > > > > > >>>>>>> Magnus > > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>> 2016-10-25 > > 18:09 > > > > > > > GMT+02:00 > > > > > > > > > > > Michael > > > > > > > > > > > > > > > Pearce < > > > > > > > > > > > > > > > > > > > >>> > > michael.pea...@ig.com > > > > > > > > > > > > > > > > > > > >>>>> : > > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>>> Hi All, > > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>>> I would like > to > > > > > discuss > > > > > > > the > > > > > > > > > > > > following > > > > > > > > > > > > > > KIP > > > > > > > > > > > > > > > > > proposal: > > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > https://cwiki.apache.org/ > > > > > > > > > > > > > > > > > > confluence/display/KAFKA/KIP- > > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > 87+-+Add+Compaction+Tombstone+ > > > > > > > > > > > Flag > > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>>> This is off > the > > > back > > > > > of > > > > > > > the > > > > > > > > > > > > discussion > > > > > > > > > > > > > > on > > > > > > > > > > > > > > > KIP-82 > > > > > > > > > > > > > > > > > / > > > > > > > > > > > > > > > > > > KIP > > > > > > > > > > > > > > > > > > > >>> meeting > > > > > > > > > > > > > > > > > > > >>>>>>> where it > > > > > > > > > > > > > > > > > > > >>>>>>>> was agreed to > > > > separate > > > > > > > this > > > > > > > > > > issue > > > > > > > > > > > > and > > > > > > > > > > > > > > > feature. > > > > > > > > > > > > > > > > > See: > > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > http://mail-archives.apache > > > > > > > > . > > > > > > > > > > > > > > > > > > org/mod_mbox/kafka-dev/201610 > > > > > > > > > > > > > > > > > > > . > > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > mbox/%3cCAJS3ho8OcR== > > > > > > > > > > > > > > > EcxsJ8OP99pD2hz=iiGecWsv- > > > > > > > > > > > > > > > > > > > >>>>>>>> EZsBsNyDcKr= > > > > > > > > g...@mail.gmail.com% > > > > > > > > > 3e > > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>>> Thanks > > > > > > > > > > > > > > > > > > > >>>>>>>> Mike > > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>>> The > information > > > > > > contained > > > > > > > in > > > > > > > > > > this > > > > > > > > > > > > > email > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > > strictly > > > > > > > > > > > > > > > > > > > >>>> confidential > > > > > > > > > > > > > > > > > > > >>>>>> and > > > > > > > > > > > > > > > > > > > >>>>>>> for > > > > > > > > > > > > > > > > > > > >>>>>>>> the use of the > > > > > addressee > > > > > > > > only, > > > > > > > > > > > > unless > > > > > > > > > > > > > > > otherwise > > > > > > > > > > > > > > > > > > indicated. > > > > > > > > > > > > > > > > > > > >> If > > > > > > > > > > > > > > > > > > > >>>> you > > > > > > > > > > > > > > > > > > > >>>>>>> are not > > > > > > > > > > > > > > > > > > > >>>>>>>> the intended > > > > > recipient, > > > > > > > > please > > > > > > > > > > do > > > > > > > > > > > > not > > > > > > > > > > > > > > > read, > > > > > > > > > > > > > > > > copy, > > > > > > > > > > > > > > > > > > use or > > > > > > > > > > > > > > > > > > > >>>> disclose > > > > > > > > > > > > > > > > > > > >>>>>> to > > > > > > > > > > > > > > > > > > > >>>>>>> others > > > > > > > > > > > > > > > > > > > >>>>>>>> this message > or > > > any > > > > > > > > > attachment. > > > > > > > > > > > > Please > > > > > > > > > > > > > > > also > > > > > > > > > > > > > > > > notify > > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > >> sender > > > > > > > > > > > > > > > > > > > >>>> by > > > > > > > > > > > > > > > > > > > >>>>>>> replying > > > > > > > > > > > > > > > > > > > >>>>>>>> to this email > or > > > by > > > > > > > > telephone > > > > > > > > > > > > (+44(020 > > > > > > > > > > > > > > > 7896 > > > > > > > > > > > > > > > > 0011) > > > > > > > > > > > > > > > > > > and then > > > > > > > > > > > > > > > > > > > >>>> delete > > > > > > > > > > > > > > > > > > > >>>>>>> the email > > > > > > > > > > > > > > > > > > > >>>>>>>> and any copies > > of > > > > it. > > > > > > > > > Opinions, > > > > > > > > > > > > > > > conclusion (etc) > > > > > > > > > > > > > > > > > > that do > > > > > > > > > > > > > > > > > > > >> not > > > > > > > > > > > > > > > > > > > >>>>> relate > > > > > > > > > > > > > > > > > > > >>>>>>> to the > > > > > > > > > > > > > > > > > > > >>>>>>>> official > > business > > > of > > > > > > this > > > > > > > > > > company > > > > > > > > > > > > > shall > > > > > > > > > > > > > > be > > > > > > > > > > > > > > > > > > understood as > > > > > > > > > > > > > > > > > > > >>>> neither > > > > > > > > > > > > > > > > > > > >>>>>>> given nor > > > > > > > > > > > > > > > > > > > >>>>>>>> endorsed by > it. > > IG > > > > is > > > > > a > > > > > > > > > trading > > > > > > > > > > > name > > > > > > > > > > > > > of > > > > > > > > > > > > > > IG > > > > > > > > > > > > > > > > Markets > > > > > > > > > > > > > > > > > > Limited > > > > > > > > > > > > > > > > > > > >> (a > > > > > > > > > > > > > > > > > > > >>>>>> company > > > > > > > > > > > > > > > > > > > >>>>>>>> registered in > > > > England > > > > > > and > > > > > > > > > Wales, > > > > > > > > > > > > > company > > > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > > > 04008957) > > > > > > > > > > > > > > > > > > > >> and > > > > > > > > > > > > > > > > > > > >>>> IG > > > > > > > > > > > > > > > > > > > >>>>>>> Index > > > > > > > > > > > > > > > > > > > >>>>>>>> Limited (a > > company > > > > > > > > registered > > > > > > > > > in > > > > > > > > > > > > > England > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > Wales, > > > > > > > > > > > > > > > > > > > company > > > > > > > > > > > > > > > > > > > >>>>> number > > > > > > > > > > > > > > > > > > > >>>>>>>> 01190902). > > > > Registered > > > > > > > > address > > > > > > > > > at > > > > > > > > > > > > > Cannon > > > > > > > > > > > > > > > Bridge > > > > > > > > > > > > > > > > > > House, 25 > > > > > > > > > > > > > > > > > > > >>>> Dowgate > > > > > > > > > > > > > > > > > > > >>>>>>> Hill, > > > > > > > > > > > > > > > > > > > >>>>>>>> London EC4R > 2YA. > > > > Both > > > > > IG > > > > > > > > > Markets > > > > > > > > > > > > > Limited > > > > > > > > > > > > > > > > (register > > > > > > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > > > > >>>> 195355) > > > > > > > > > > > > > > > > > > > >>>>>>> and IG > > > > > > > > > > > > > > > > > > > >>>>>>>> Index Limited > > > > > (register > > > > > > > > number > > > > > > > > > > > > 114059) > > > > > > > > > > > > > > are > > > > > > > > > > > > > > > > > > authorised and > > > > > > > > > > > > > > > > > > > >>>>> regulated > > > > > > > > > > > > > > > > > > > >>>>>>> by the > > > > > > > > > > > > > > > > > > > >>>>>>>> Financial > > Conduct > > > > > > > Authority. > > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>>> The information > > > > > contained > > > > > > > in > > > > > > > > > this > > > > > > > > > > > > email > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > strictly > > > > > > > > > > > > > > > > > > > confidential > > > > > > > > > > > > > > > > > > > >>> and > > > > > > > > > > > > > > > > > > > >>>>> for > > > > > > > > > > > > > > > > > > > >>>>>>> the use of the > > > > > addressee > > > > > > > > only, > > > > > > > > > > > unless > > > > > > > > > > > > > > > otherwise > > > > > > > > > > > > > > > > > > indicated. > > > > > > > > > > > > > > > > > > > If you > > > > > > > > > > > > > > > > > > > >>> are > > > > > > > > > > > > > > > > > > > >>>>> not > > > > > > > > > > > > > > > > > > > >>>>>>> the intended > > > > recipient, > > > > > > > > please > > > > > > > > > do > > > > > > > > > > > not > > > > > > > > > > > > > > > read, copy, > > > > > > > > > > > > > > > > > > use or > > > > > > > > > > > > > > > > > > > disclose > > > > > > > > > > > > > > > > > > > >>> to > > > > > > > > > > > > > > > > > > > >>>>>> others > > > > > > > > > > > > > > > > > > > >>>>>>> this message or > > any > > > > > > > > attachment. > > > > > > > > > > > > Please > > > > > > > > > > > > > > also > > > > > > > > > > > > > > > > notify > > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > sender by > > > > > > > > > > > > > > > > > > > >>>>> replying > > > > > > > > > > > > > > > > > > > >>>>>>> to this email > or > > by > > > > > > > telephone > > > > > > > > > > > > (+44(020 > > > > > > > > > > > > > > > 7896 0011) > > > > > > > > > > > > > > > > > > and then > > > > > > > > > > > > > > > > > > > delete > > > > > > > > > > > > > > > > > > > >>> the > > > > > > > > > > > > > > > > > > > >>>>>> email > > > > > > > > > > > > > > > > > > > >>>>>>> and any copies > of > > > it. > > > > > > > > Opinions, > > > > > > > > > > > > > > conclusion > > > > > > > > > > > > > > > (etc) > > > > > > > > > > > > > > > > > > that do > > > > > > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > > > > >> relate > > > > > > > > > > > > > > > > > > > >>>> to > > > > > > > > > > > > > > > > > > > >>>>>> the > > > > > > > > > > > > > > > > > > > >>>>>>> official > business > > > of > > > > > this > > > > > > > > > company > > > > > > > > > > > > shall > > > > > > > > > > > > > > be > > > > > > > > > > > > > > > > > > understood as > > > > > > > > > > > > > > > > > > > neither > > > > > > > > > > > > > > > > > > > >>>> given > > > > > > > > > > > > > > > > > > > >>>>>> nor > > > > > > > > > > > > > > > > > > > >>>>>>> endorsed by it. > > IG > > > > is a > > > > > > > > trading > > > > > > > > > > > name > > > > > > > > > > > > of > > > > > > > > > > > > > > IG > > > > > > > > > > > > > > > > Markets > > > > > > > > > > > > > > > > > > Limited > > > > > > > > > > > > > > > > > > > (a > > > > > > > > > > > > > > > > > > > >>> company > > > > > > > > > > > > > > > > > > > >>>>>>> registered in > > > England > > > > > and > > > > > > > > > Wales, > > > > > > > > > > > > > company > > > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > > > 04008957) > > > > > > > > > > > > > > > > > > > and IG > > > > > > > > > > > > > > > > > > > >>>> Index > > > > > > > > > > > > > > > > > > > >>>>>>> Limited (a > > company > > > > > > > registered > > > > > > > > > in > > > > > > > > > > > > > England > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > Wales, > > > > > > > > > > > > > > > > > > company > > > > > > > > > > > > > > > > > > > >> number > > > > > > > > > > > > > > > > > > > >>>>>>> 01190902). > > > Registered > > > > > > > address > > > > > > > > > at > > > > > > > > > > > > Cannon > > > > > > > > > > > > > > > Bridge > > > > > > > > > > > > > > > > > > House, 25 > > > > > > > > > > > > > > > > > > > Dowgate > > > > > > > > > > > > > > > > > > > >>>> Hill, > > > > > > > > > > > > > > > > > > > >>>>>>> London EC4R > 2YA. > > > Both > > > > > IG > > > > > > > > > Markets > > > > > > > > > > > > > Limited > > > > > > > > > > > > > > > > (register > > > > > > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > > > > 195355) > > > > > > > > > > > > > > > > > > > >>> and > > > > > > > > > > > > > > > > > > > >>>>> IG > > > > > > > > > > > > > > > > > > > >>>>>>> Index Limited > > > > (register > > > > > > > > number > > > > > > > > > > > > 114059) > > > > > > > > > > > > > > are > > > > > > > > > > > > > > > > > > authorised and > > > > > > > > > > > > > > > > > > > >> regulated > > > > > > > > > > > > > > > > > > > >>>> by > > > > > > > > > > > > > > > > > > > >>>>>> the > > > > > > > > > > > > > > > > > > > >>>>>>> Financial > Conduct > > > > > > > Authority. > > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > > >>>>> -- > > > > > > > > > > > > > > > > > > > >>>>> Nacho (Ignacio) > > Solis > > > > > > > > > > > > > > > > > > > >>>>> Kafka > > > > > > > > > > > > > > > > > > > >>>>> > > nso...@linkedin.com > > > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > -Regards, > > > > > > > > > > > > > > > > > > > > Mayuresh R. Gharat > > > > > > > > > > > > > > > > > > > > (862) 250-7125 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The information contained > in > > > this > > > > > > email > > > > > > > > is > > > > > > > > > > > > strictly > > > > > > > > > > > > > > > > > confidential > > > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > > > for the use of the addressee > > only, > > > > > unless > > > > > > > > > > otherwise > > > > > > > > > > > > > > > indicated. If > > > > > > > > > > > > > > > > > > you are > > > > > > > > > > > > > > > > > > > not the intended recipient, > > please > > > do > > > > > not > > > > > > > > read, > > > > > > > > > > > copy, > > > > > > > > > > > > > use > > > > > > > > > > > > > > > or > > > > > > > > > > > > > > > > > > disclose to > > > > > > > > > > > > > > > > > > > others this message or any > > > > attachment. > > > > > > > Please > > > > > > > > > > also > > > > > > > > > > > > > notify > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > sender > > > > > > > > > > > > > > > > > > by > > > > > > > > > > > > > > > > > > > replying to this email or by > > > > telephone > > > > > > > > (+44(020 > > > > > > > > > > > 7896 > > > > > > > > > > > > > > 0011) > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > then > > > > > > > > > > > > > > > > > > delete > > > > > > > > > > > > > > > > > > > the email and any copies of it. > > > > > Opinions, > > > > > > > > > > > conclusion > > > > > > > > > > > > > > (etc) > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > do > > > > > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > > > > relate to the official business > > of > > > > this > > > > > > > > company > > > > > > > > > > > shall > > > > > > > > > > > > > be > > > > > > > > > > > > > > > > understood > > > > > > > > > > > > > > > > > > as > > > > > > > > > > > > > > > > > > > neither given nor endorsed by > it. > > > IG > > > > > is a > > > > > > > > > trading > > > > > > > > > > > > name > > > > > > > > > > > > > of > > > > > > > > > > > > > > > IG > > > > > > > > > > > > > > > > > Markets > > > > > > > > > > > > > > > > > > > Limited (a company registered > in > > > > > England > > > > > > > and > > > > > > > > > > Wales, > > > > > > > > > > > > > > company > > > > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > > > > 04008957) and IG Index Limited > (a > > > > > company > > > > > > > > > > > registered > > > > > > > > > > > > in > > > > > > > > > > > > > > > England > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > > Wales, > > > > > > > > > > > > > > > > > > > company number 01190902). > > > Registered > > > > > > > address > > > > > > > > at > > > > > > > > > > > > Cannon > > > > > > > > > > > > > > > Bridge > > > > > > > > > > > > > > > > > House, > > > > > > > > > > > > > > > > > > 25 > > > > > > > > > > > > > > > > > > > Dowgate Hill, London EC4R 2YA. > > Both > > > > IG > > > > > > > > Markets > > > > > > > > > > > > Limited > > > > > > > > > > > > > > > (register > > > > > > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > > > > 195355) and IG Index Limited > > > > (register > > > > > > > number > > > > > > > > > > > 114059) > > > > > > > > > > > > > are > > > > > > > > > > > > > > > > > authorised > > > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > > > regulated by the Financial > > Conduct > > > > > > > Authority. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The information contained in this > email > > > is > > > > > > > strictly > > > > > > > > > > > > > > confidential > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > for > > > > > > > > > > > > > > > > > > the use of the addressee only, unless > > > > > otherwise > > > > > > > > > > > indicated. > > > > > > > > > > > > If > > > > > > > > > > > > > > > you are > > > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > > > the intended recipient, please do not > > > read, > > > > > > copy, > > > > > > > > use > > > > > > > > > > or > > > > > > > > > > > > > > > disclose to > > > > > > > > > > > > > > > > > others > > > > > > > > > > > > > > > > > > this message or any attachment. > Please > > > also > > > > > > > notify > > > > > > > > > the > > > > > > > > > > > > sender > > > > > > > > > > > > > > by > > > > > > > > > > > > > > > > replying > > > > > > > > > > > > > > > > > > to this email or by telephone > (+44(020 > > > 7896 > > > > > > 0011) > > > > > > > > and > > > > > > > > > > > then > > > > > > > > > > > > > > > delete the > > > > > > > > > > > > > > > > > email > > > > > > > > > > > > > > > > > > and any copies of it. Opinions, > > > conclusion > > > > > > (etc) > > > > > > > > that > > > > > > > > > > do > > > > > > > > > > > > not > > > > > > > > > > > > > > > relate to > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > official business of this company > shall > > > be > > > > > > > > understood > > > > > > > > > > as > > > > > > > > > > > > > > neither > > > > > > > > > > > > > > > given > > > > > > > > > > > > > > > > > nor > > > > > > > > > > > > > > > > > > endorsed by it. IG is a trading name > of > > > IG > > > > > > > Markets > > > > > > > > > > > Limited > > > > > > > > > > > > (a > > > > > > > > > > > > > > > company > > > > > > > > > > > > > > > > > > registered in England and Wales, > > company > > > > > number > > > > > > > > > > 04008957) > > > > > > > > > > > > and > > > > > > > > > > > > > > IG > > > > > > > > > > > > > > > Index > > > > > > > > > > > > > > > > > > Limited (a company registered in > > England > > > > and > > > > > > > Wales, > > > > > > > > > > > company > > > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > > > 01190902). Registered address at > Cannon > > > > > Bridge > > > > > > > > House, > > > > > > > > > > 25 > > > > > > > > > > > > > > Dowgate > > > > > > > > > > > > > > > Hill, > > > > > > > > > > > > > > > > > > London EC4R 2YA. Both IG Markets > > Limited > > > > > > > (register > > > > > > > > > > number > > > > > > > > > > > > > > > 195355) and > > > > > > > > > > > > > > > > IG > > > > > > > > > > > > > > > > > > Index Limited (register number > 114059) > > > are > > > > > > > > authorised > > > > > > > > > > and > > > > > > > > > > > > > > > regulated by > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > Financial Conduct Authority. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The information contained in this email > is > > > > > strictly > > > > > > > > > > > > confidential > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > for > > > > > > > > > > > > > > > > the use of the addressee only, unless > > > otherwise > > > > > > > > > indicated. > > > > > > > > > > If > > > > > > > > > > > > you > > > > > > > > > > > > > > > are not > > > > > > > > > > > > > > > > the intended recipient, please do not > read, > > > > copy, > > > > > > use > > > > > > > > or > > > > > > > > > > > > disclose > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > others > > > > > > > > > > > > > > > > this message or any attachment. Please > also > > > > > notify > > > > > > > the > > > > > > > > > > sender > > > > > > > > > > > > by > > > > > > > > > > > > > > > replying > > > > > > > > > > > > > > > > to this email or by telephone (+44(020 > 7896 > > > > 0011) > > > > > > and > > > > > > > > > then > > > > > > > > > > > > delete > > > > > > > > > > > > > > > the email > > > > > > > > > > > > > > > > and any copies of it. Opinions, > conclusion > > > > (etc) > > > > > > that > > > > > > > > do > > > > > > > > > > not > > > > > > > > > > > > > relate > > > > > > > > > > > > > > > to the > > > > > > > > > > > > > > > > official business of this company shall > be > > > > > > understood > > > > > > > > as > > > > > > > > > > > > neither > > > > > > > > > > > > > > > given nor > > > > > > > > > > > > > > > > endorsed by it. IG is a trading name of > IG > > > > > Markets > > > > > > > > > Limited > > > > > > > > > > (a > > > > > > > > > > > > > > company > > > > > > > > > > > > > > > > registered in England and Wales, company > > > number > > > > > > > > 04008957) > > > > > > > > > > and > > > > > > > > > > > > IG > > > > > > > > > > > > > > > Index > > > > > > > > > > > > > > > > Limited (a company registered in England > > and > > > > > Wales, > > > > > > > > > company > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > 01190902). Registered address at Cannon > > > Bridge > > > > > > House, > > > > > > > > 25 > > > > > > > > > > > > Dowgate > > > > > > > > > > > > > > > Hill, > > > > > > > > > > > > > > > > London EC4R 2YA. Both IG Markets Limited > > > > > (register > > > > > > > > number > > > > > > > > > > > > 195355) > > > > > > > > > > > > > > > and IG > > > > > > > > > > > > > > > > Index Limited (register number 114059) > are > > > > > > authorised > > > > > > > > and > > > > > > > > > > > > > regulated > > > > > > > > > > > > > > > by the > > > > > > > > > > > > > > > > Financial Conduct Authority. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > -Regards, > > > > > > > > > > > > > > > Mayuresh R. Gharat > > > > > > > > > > > > > > > (862) 250-7125 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The information contained in this email is > > strictly > > > > > > > > > confidential > > > > > > > > > > > and > > > > > > > > > > > > > for > > > > > > > > > > > > > > > the use of the addressee only, unless otherwise > > > > > > indicated. > > > > > > > If > > > > > > > > > you > > > > > > > > > > > are > > > > > > > > > > > > > not > > > > > > > > > > > > > > > the intended recipient, please do not read, > copy, > > > use > > > > > or > > > > > > > > > disclose > > > > > > > > > > > to > > > > > > > > > > > > > > others > > > > > > > > > > > > > > > this message or any attachment. Please also > > notify > > > > the > > > > > > > sender > > > > > > > > > by > > > > > > > > > > > > > replying > > > > > > > > > > > > > > > to this email or by telephone (+44(020 7896 > 0011) > > > and > > > > > > then > > > > > > > > > delete > > > > > > > > > > > the > > > > > > > > > > > > > > email > > > > > > > > > > > > > > > and any copies of it. Opinions, conclusion > (etc) > > > that > > > > > do > > > > > > > not > > > > > > > > > > relate > > > > > > > > > > > > to > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > official business of this company shall be > > > understood > > > > > as > > > > > > > > > neither > > > > > > > > > > > > given > > > > > > > > > > > > > > nor > > > > > > > > > > > > > > > endorsed by it. IG is a trading name of IG > > Markets > > > > > > Limited > > > > > > > (a > > > > > > > > > > > company > > > > > > > > > > > > > > > registered in England and Wales, company number > > > > > 04008957) > > > > > > > and > > > > > > > > > IG > > > > > > > > > > > > Index > > > > > > > > > > > > > > > Limited (a company registered in England and > > Wales, > > > > > > company > > > > > > > > > > number > > > > > > > > > > > > > > > 01190902). Registered address at Cannon Bridge > > > House, > > > > > 25 > > > > > > > > > Dowgate > > > > > > > > > > > > Hill, > > > > > > > > > > > > > > > London EC4R 2YA. Both IG Markets Limited > > (register > > > > > number > > > > > > > > > 195355) > > > > > > > > > > > and > > > > > > > > > > > > > IG > > > > > > > > > > > > > > > Index Limited (register number 114059) are > > > authorised > > > > > and > > > > > > > > > > regulated > > > > > > > > > > > > by > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > Financial Conduct Authority. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > -Regards, > > > > > > > > > > > > > > Mayuresh R. Gharat > > > > > > > > > > > > > > (862) 250-7125 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > -Regards, > > > > > > > > > > > > Mayuresh R. Gharat > > > > > > > > > > > > (862) 250-7125 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Nacho - Ignacio Solis - iso...@igso.net > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > -Regards, > > > > > > > > Mayuresh R. Gharat > > > > > > > > (862) 250-7125 > > > > > > > > The information contained in this email is strictly > > confidential > > > > and > > > > > > for > > > > > > > > the use of the addressee only, unless otherwise indicated. If > > you > > > > are > > > > > > not > > > > > > > > the intended recipient, please do not read, copy, use or > > disclose > > > > to > > > > > > > others > > > > > > > > this message or any attachment. Please also notify the sender > > by > > > > > > replying > > > > > > > > to this email or by telephone (+44(020 7896 0011) and then > > delete > > > > the > > > > > > > email > > > > > > > > and any copies of it. Opinions, conclusion (etc) that do not > > > relate > > > > > to > > > > > > > the > > > > > > > > official business of this company shall be understood as > > neither > > > > > given > > > > > > > nor > > > > > > > > endorsed by it. IG is a trading name of IG Markets Limited (a > > > > company > > > > > > > > registered in England and Wales, company number 04008957) and > > IG > > > > > Index > > > > > > > > Limited (a company registered in England and Wales, company > > > number > > > > > > > > 01190902). Registered address at Cannon Bridge House, 25 > > Dowgate > > > > > Hill, > > > > > > > > London EC4R 2YA. Both IG Markets Limited (register number > > 195355) > > > > and > > > > > > IG > > > > > > > > Index Limited (register number 114059) are authorised and > > > regulated > > > > > by > > > > > > > the > > > > > > > > Financial Conduct Authority. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > - > > ... > > > > [Message clipped] > > > > > -- > -Regards, > Mayuresh R. Gharat > (862) 250-7125 > The information contained in this email is strictly confidential and for > the use of the addressee only, unless otherwise indicated. If you are not > the intended recipient, please do not read, copy, use or disclose to others > this message or any attachment. Please also notify the sender by replying > to this email or by telephone (+44(020 7896 0011) and then delete the email > and any copies of it. Opinions, conclusion (etc) that do not relate to the > official business of this company shall be understood as neither given nor > endorsed by it. IG is a trading name of IG Markets Limited (a company > registered in England and Wales, company number 04008957) and IG Index > Limited (a company registered in England and Wales, company number > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, > London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG > Index Limited (register number 114059) are authorised and regulated by the > Financial Conduct Authority. >
