Michael, Yes, I am OK with stage 1. We can discuss about stage 2 later but this sounds really an organization specific decision to deprecate an API. It does not seem a general need in open source Kafka to only support tombstone bit , which is a bad thing for people who are still running old clients in the community. This is exactly why we want to have magic byte bump - there should be only one definitive way to interpret a message with a given magic byte. We should avoid re-define the interpretation of a message with an existing magic byte. The interpretation of an old message format should not be changed and should always be supported until deprecated.
Thanks, Jiangjie (Becket) Qin On Mon, Nov 14, 2016 at 11:28 AM, Mayuresh Gharat < gharatmayures...@gmail.com> wrote: > I am not sure about "If I understand correctly, you want to let the broker > to reject requests > from old clients to ensure everyone in an organization has upgraded, > right?" > > I don't think we will be rejecting requests. What phase2 (stage 2) meant > was we will only do log compaction based on tombstone marker and nothing > else. > I am not sure about making this configurable, as a configuration makes it > sound like the broker has the capability to support null and tombstone > marker bit, for log compaction, for life long. > > I agree with Michael's idea of delivering phase 1 (stage1) right now. It > will also be good to put a note saying that we will be deprecating the > older way of log compaction in next release. > Phase 2 (stage 2) will actually only rely on tombstone marker for doing log > compaction. > So this actually meets our end state of having tombstone marker for log > compaction support. > > Thanks, > > Mayuresh > > > On Mon, Nov 14, 2016 at 11:09 AM, Michael Pearce <michael.pea...@ig.com> > wrote: > > > I'm ok with this, but I'd like to at least get phase 1 in, for the next > > release, this is what I'm very keen for, > > > > As such shall we say we have > > Kip-87a that delivers phase 1 > > > > And then > > Kip-87b in release after that delivers phase 2 but has a dependency on > > support of deprecating old api versions > > > > How does this approach sound? > > > > > > ________________________________________ > > From: Becket Qin <becket....@gmail.com> > > Sent: Monday, November 14, 2016 6:14:44 PM > > To: dev@kafka.apache.org > > Subject: Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag > > > > Hey Michael and Mayuresh, > > > > If I understand correctly, you want to let the broker to reject requests > > from old clients to ensure everyone in an organization has upgraded, > right? > > This is essentially deprecating an old protocol. I agree it is useful and > > that is why we have that baked in KIP-35. However, we haven't thought > > through how to deprecate an API so far. From broker's point of view., it > > will always support all the old protocols according to Kafka's widely > known > > compatibility guarantee. If people decide to deprecate an API within an > > organization, we can provide some additional configuration to let people > do > > this, which is not a bad idea but seems better to be discussed in another > > KIP. > > > > Thanks, > > > > Jiangjie (Becket) Qin > > > > On Mon, Nov 14, 2016 at 8:52 AM, Michael Pearce <michael.pea...@ig.com> > > wrote: > > > > > I agree with Mayuresh. > > > > > > I don't see how having a magic byte helps here. > > > > > > What we are saying is that on day 1 after an upgrade both tombstone > flag > > > or a null value will be treated as a marker to delete on compacted > topic. > > > > > > During this time we expect organisations to migrate themselves over > onto > > > the new producers and consumers that call the new Api, changing their > > > application logic. during this point producers should start setting the > > > tombstone marker and consumers to start behaving based on the tombstone > > > marker. > > > > > > Only after all producers and consumers are upgraded then we enter > phase 2 > > > disabling the use of a null value as a delete marker. And the old apis > > > would not be allowed to be called as the broker now expects all clients > > to > > > be using latest api only. > > > > > > At this second phase stage only also we can support a null value in a > > > compacted topic without it being treated as deletion. We solely base on > > the > > > tombstone marker. > > > > > > > > > ________________________________________ > > > From: Mayuresh Gharat <gharatmayures...@gmail.com> > > > Sent: Saturday, November 12, 2016 2:18:19 AM > > > To: dev@kafka.apache.org > > > Subject: Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag > > > > > > I think "In second stage we move on to supporting only the attribute > flag > > > for log > > > compaction." means that it will no longer support request from older > > > clients. > > > > > > Thanks, > > > > > > Mayuresh > > > > > > On Fri, Nov 11, 2016 at 10:02 AM, Becket Qin <becket....@gmail.com> > > wrote: > > > > > > > Hey Michael, > > > > > > > > The way Kafka implements backwards compatibility is to let the > brokers > > > > support old protocols. So the brokers have to support older clients > > that > > > do > > > > not understand the new attribute bit. That means we will not be able > to > > > get > > > > away with the null value as a tombstone. So we need a good definition > > of > > > > whether we should treat it as a tombstone or not. This is why I think > > we > > > > need a magic value bump. > > > > > > > > My confusion on the second stage is exactly about "we want to end up > > just > > > > supporting tombstone marker (not both)", or in the original > statement: > > > "2) > > > > In second stage we move on to supporting only the attribute flag for > > log > > > > compaction." - We will always support the null value as a tombstone > as > > > long > > > > as the message format version (i.e. magic byte) is less than 2 for > the > > > > reason I mentioned above. > > > > > > > > Although the way to interpret the message bytes at producing time can > > be > > > > inferred from the the ProduceRequest version, this version > information > > > > won't be stored in the broker. So when an old consumer comes, we need > > to > > > > decide whether we have to do down conversion or not.. With a clear > > > existing > > > > configuration of message version config (i.e. magic value), we know > for > > > > sure when to down convert the messages to adapt to older clients. > > > Otherwise > > > > we will have to always scan all the messages. It would probably work > > but > > > > relies on guess or inference. > > > > > > > > Thanks, > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > On Fri, Nov 11, 2016 at 8:42 AM, Mayuresh Gharat < > > > > gharatmayures...@gmail.com > > > > > wrote: > > > > > > > > > Sounds good Michael. > > > > > > > > > > Thanks, > > > > > > > > > > Mayuresh > > > > > > > > > > On Fri, Nov 11, 2016 at 12:48 AM, Michael Pearce < > > > michael.pea...@ig.com> > > > > > wrote: > > > > > > > > > > > @Mayuresh i don't think you've missed anything - > > > > > > > > > > > > as per earlier in the discussion. > > > > > > > > > > > > We're providing new api versions, but not planning on bumping the > > > magic > > > > > > number as there is no structural changes, we are simply using up > a > > > new > > > > > > attribute bit (as like adding new compression support just uses > up > > > > > > additional attribute bits) > > > > > > > > > > > > > > > > > > I think also difference between null vs 0 length byte array is > > > covered. > > > > > > > > > > > > @Becket, > > > > > > > > > > > > The two stage approach is because we want to end up just > supporting > > > > > > tombstone marker (not both) > > > > > > > > > > > > But we accept we need to allow organisations and systems a period > > of > > > > > > transition (this is what stage 1 provides) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ________________________________________ > > > > > > From: Mayuresh Gharat <gharatmayures...@gmail.com> > > > > > > Sent: Thursday, November 10, 2016 8:57 PM > > > > > > To: dev@kafka.apache.org > > > > > > Subject: Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag > > > > > > > > > > > > I am not sure if we are bumping magicByte value as the KIP does > not > > > > > mention > > > > > > it (If I didn't miss anything). > > > > > > > > > > > > @Nacho : I did not understand what you meant by : Conversion from > > one > > > > to > > > > > > the other may be possible but I would rather not have to do it. > > > > > > > > > > > > You will have to do the conversion for the scenario that Becket > has > > > > > > mentioned above where an old consumer talks to the new broker, > > right? > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Mayuresh > > > > > > > > > > > > > > > > > > On Thu, Nov 10, 2016 at 11:54 AM, Becket Qin < > becket....@gmail.com > > > > > > > > wrote: > > > > > > > > > > > > > Nacho, > > > > > > > > > > > > > > In Kafka protocol, a negative length is null, a zero length > means > > > > empty > > > > > > > byte array. > > > > > > > > > > > > > > I am still confused about the two stages. It seems that the > > broker > > > > only > > > > > > > needs to understand three versions of messages. > > > > > > > 1. MagicValue=0 - no timestamp, absolute offset, no tombstone > > flag > > > > > > > 2. MagicValue=1 - with timestamp, relative offsets, no > tombstone > > > flag > > > > > > (null > > > > > > > value = tombstone) > > > > > > > 3. MagicValue=2 - with timestamp, relative offsets, tombstone > > flag > > > > > > > > > > > > > > We are talking about two flavors for 3: > > > > > > > 3.1 tombstone flag set = tombstone (Allows a key with null > value > > in > > > > the > > > > > > > compacted topics) > > > > > > > 3.2 tombstone flag set OR null value = tombstone ( Do not > allow a > > > key > > > > > > with > > > > > > > null value in the compacted topics) > > > > > > > > > > > > > > No matter which flavor we choose, we just need to stick to that > > way > > > > of > > > > > > > interpretation, right? Why would we need a second stage? > > > > > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > > > On Thu, Nov 10, 2016 at 10:37 AM, Ignacio Solis < > iso...@igso.net > > > > > > > > wrote: > > > > > > > > > > > > > > > A quick differentiation I would like to make is null is not > the > > > > same > > > > > as > > > > > > > > size 0. > > > > > > > > > > > > > > > > Many times these are made equal, but they're not. When > > > serializing > > > > > > data, > > > > > > > > we have to make a choice in null values and many times these > > are > > > > > > > translated > > > > > > > > to zero length blobs. This is not really the same thing. > > > > > > > > > > > > > > > > From this perspective, what can Kafka represent and what is > > > > > considered > > > > > > a > > > > > > > > valid value? > > > > > > > > > > > > > > > > If the user sends a byte array of length 0, or if the > > serializer > > > > > sends > > > > > > > > something of length 0, this should be a valid value. It is > not > > > > > kafka's > > > > > > > job > > > > > > > > to determine what the user is trying to send. For all we > know, > > > the > > > > > user > > > > > > > has > > > > > > > > a really good compression serializer that sends 0 bytes when > > > > nothing > > > > > > has > > > > > > > > changed. > > > > > > > > > > > > > > > > If the user is allowed to send null then some behavior should > > be > > > > > > defined. > > > > > > > > However, this should semantically be different than sending a > > > > > command. > > > > > > It > > > > > > > > is possible for a null value could signify some form of > delete, > > > > like > > > > > > > > "delete all messages with this key". However, if kafka has a > > goal > > > > to > > > > > > > write > > > > > > > > good, readable code, then this should not be allowed. > > > > > > > > > > > > > > > > A delete or a purge is a call that can have certain side > > effects > > > or > > > > > > > > encounter errors that are unrelated to a send call. > > > > > > > > > > > > > > > > A couple of bullets from the Kafka style guide ( > > > > > > > > http://kafka.apache.org/coding-guide.html ): > > > > > > > > > > > > > > > > - Clear code is preferable to comments. When possible make > your > > > > > naming > > > > > > so > > > > > > > > good you don't need comments. > > > > > > > > - Logging, configuration, and public APIs are our "UI". Make > > them > > > > > > pretty, > > > > > > > > consistent, and usable. > > > > > > > > > > > > > > > > If you want sendTombstone() functionality make the protocol > > > reflect > > > > > > that. > > > > > > > > > > > > > > > > Right now we're struggling because we did not have a clear > > > > separation > > > > > > of > > > > > > > > concerns. > > > > > > > > > > > > > > > > I don't have a good solution for deployment. I'm in favor of > a > > > > harsh > > > > > > > road: > > > > > > > > - Add the flag/variable/header for tombstone > > > > > > > > - Add an API for tombstone behavior if needed > > > > > > > > - Error if somebody tries to API send null > > > > > > > > - Accept if somebody tries to API send byte[] length 0 > > > > > > > > - Rev broker version > > > > > > > > - broker accept flag/variable/header as tombstone > > > > > > > > - broker accept zero length values as normal messages > > > > > > > > > > > > > > > > The more gentle road would be: > > > > > > > > - Add the flag/variable/header for tombstone > > > > > > > > - Add an API for tombstone behavior if needed > > > > > > > > - warn/deprecate/etc if somebody sends null > > > > > > > > - broker accepts flags/variable/headers and zero length > values > > as > > > > > > > > tombstones > > > > > > > > > > > > > > > > Conversion from one to the other may be possible but I would > > > > rather > > > > > > not > > > > > > > > have to do it. > > > > > > > > > > > > > > > > Nacho > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 9, 2016 at 9:03 PM, radai < > > > radai.rosenbl...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > > > my personal opinion - a log compacted topic is basically a > > > > > kv-store, > > > > > > > so a > > > > > > > > > map API. > > > > > > > > > map.put(key, null) is not the same as map.remove(key), > which > > to > > > > me > > > > > > > means > > > > > > > > a > > > > > > > > > null value should not represent a delete. a delete should > be > > > > > explicit > > > > > > > > > (meaning flag). > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 9, 2016 at 11:01 AM, Mayuresh Gharat < > > > > > > > > > gharatmayures...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > I see the reasoning and might be inclined to agree a bit > : > > > > > > > > > > If we go to stage 2, the only difference is that we can > > > > > > theoretically > > > > > > > > > > support a null value non-tombstone message in a log > > compacted > > > > > > topic, > > > > > > > > but > > > > > > > > > I > > > > > > > > > > am not sure if that has any use case. > > > > > > > > > > > > > > > > > > > > But as an end goal I see that kafka should clearly > specify > > > what > > > > > it > > > > > > > > means > > > > > > > > > by > > > > > > > > > > a tombstone : is it the attribute flag OR is it the null > > > value. > > > > > If > > > > > > we > > > > > > > > > just > > > > > > > > > > do stage 1, I don't think we are defining the end-goal > > > > > completely. > > > > > > > > > > Again this is more about semantics of correctness of end > > > state. > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > Mayuresh > > > > > > > > > > > > > > > > > > > > On Wed, Nov 9, 2016 at 10:49 AM, Becket Qin < > > > > > becket....@gmail.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > I am not sure if we need the second stage. Wouldn't it > be > > > > > enough > > > > > > to > > > > > > > > say > > > > > > > > > > > that a message is a tombstone if one of the following > is > > > > true? > > > > > > > > > > > 1. tombstone flag is set. > > > > > > > > > > > 2. value is null. > > > > > > > > > > > > > > > > > > > > > > If we go to stage 2, the only difference is that we can > > > > > > > theoretically > > > > > > > > > > > support a null value non-tombstone message in a log > > > compacted > > > > > > > topic, > > > > > > > > > but > > > > > > > > > > I > > > > > > > > > > > am not sure if that has any use case. > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 9, 2016 at 9:23 AM, Mayuresh Gharat < > > > > > > > > > > > gharatmayures...@gmail.com> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > I think it will be a good idea. +1 > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > Mayuresh > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 9, 2016 at 9:13 AM, Michael Pearce < > > > > > > > > > michael.pea...@ig.com> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > +1 Mayuresh, I think this is a good > > solution/strategy. > > > > > > > > > > > > > > > > > > > > > > > > > > Shall we update the KIP with this? Becket/Jun/Joel > > any > > > > > > comments > > > > > > > > to > > > > > > > > > > add > > > > > > > > > > > > > before we do? > > > > > > > > > > > > > > > > > > > > > > > > > > On 08/11/2016, 17:29, "Mayuresh Gharat" < > > > > > > > > > gharatmayures...@gmail.com> > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > I think the migration can be done in 2 stages : > > > > > > > > > > > > > > > > > > > > > > > > > > 1) In first stage the broker should understand > > the > > > > > > > attribute > > > > > > > > > flag > > > > > > > > > > > as > > > > > > > > > > > > > well > > > > > > > > > > > > > as Null for the value for log compaction. > > > > > > > > > > > > > 2) In second stage we move on to supporting > only > > > the > > > > > > > > attribute > > > > > > > > > > flag > > > > > > > > > > > > > for log > > > > > > > > > > > > > compaction. > > > > > > > > > > > > > > > > > > > > > > > > > > I agree with Becket that for older clients > > > > (consumers) > > > > > > the > > > > > > > > > broker > > > > > > > > > > > > might > > > > > > > > > > > > > have to down convert a message that has the > > > attribute > > > > > > flag > > > > > > > > set > > > > > > > > > > for > > > > > > > > > > > > log > > > > > > > > > > > > > compacting but has a non null value. But this > > > should > > > > be > > > > > > in > > > > > > > > > first > > > > > > > > > > > > stage. > > > > > > > > > > > > > Once all the clients have upgraded (clients > start > > > > > > > recognizing > > > > > > > > > the > > > > > > > > > > > > > attribute > > > > > > > > > > > > > flag), we can move the broker to stage 2. > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > Mayuresh > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 8, 2016 at 12:06 AM, Michael > Pearce < > > > > > > > > > > > > michael.pea...@ig.com > > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Also we can add further guidance: > > > > > > > > > > > > > > > > > > > > > > > > > > > > To avoid the below caveat to organisations > by > > > > > > promoting > > > > > > > of > > > > > > > > > > > > > upgrading all > > > > > > > > > > > > > > consumers first before relying on producing > > > > tombstone > > > > > > > > > messages > > > > > > > > > > > with > > > > > > > > > > > > > data > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sent using OWA for iPhone > > > > > > > > > > > > > > ________________________________________ > > > > > > > > > > > > > > From: Michael Pearce > > > > > > > > > > > > > > Sent: Tuesday, November 8, 2016 8:03:32 AM > > > > > > > > > > > > > > To: dev@kafka.apache.org > > > > > > > > > > > > > > Subject: Re: [DISCUSS] KIP-87 - Add > Compaction > > > > > > Tombstone > > > > > > > > Flag > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks Jun on the feedback, I think I > > understand > > > > the > > > > > > > > > > issue/point > > > > > > > > > > > > now. > > > > > > > > > > > > > > > > > > > > > > > > > > > > We def can add that on older client version > if > > > > > > tombstone > > > > > > > > > marker > > > > > > > > > > > > make > > > > > > > > > > > > > the > > > > > > > > > > > > > > value null to preserve behaviour. > > > > > > > > > > > > > > > > > > > > > > > > > > > > There is one caveats to this: > > > > > > > > > > > > > > > > > > > > > > > > > > > > * we have to be clear that data is lost if > > > reading > > > > > via > > > > > > > old > > > > > > > > > > > > > client/message > > > > > > > > > > > > > > format - I don't think this is a big issue as > > > > mostly > > > > > > the > > > > > > > > > > idea/use > > > > > > > > > > > > > case is > > > > > > > > > > > > > > around meta data transport as such would only > > be > > > as > > > > > bad > > > > > > > as > > > > > > > > > > > current > > > > > > > > > > > > > situation > > > > > > > > > > > > > > > > > > > > > > > > > > > > Re having configurable broker this was to > > handle > > > > > cases > > > > > > > like > > > > > > > > > you > > > > > > > > > > > > > described > > > > > > > > > > > > > > but in another way by allowing organisation > > > choose > > > > > the > > > > > > > > > > behaviour > > > > > > > > > > > of > > > > > > > > > > > > > the > > > > > > > > > > > > > > compaction per broker or per topic so they > > could > > > > > manage > > > > > > > > their > > > > > > > > > > > > > transition to > > > > > > > > > > > > > > using tombstone markers. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On hind sight it maybe easier to just upgrade > > and > > > > > > > downgrade > > > > > > > > > the > > > > > > > > > > > > > messages > > > > > > > > > > > > > > on version as you propose. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sent using OWA for iPhone > > > > > > > > > > > > > > ________________________________________ > > > > > > > > > > > > > > From: Jun Rao <j...@confluent.io> > > > > > > > > > > > > > > Sent: Tuesday, November 8, 2016 12:34:41 AM > > > > > > > > > > > > > > To: dev@kafka.apache.org > > > > > > > > > > > > > > Subject: Re: [DISCUSS] KIP-87 - Add > Compaction > > > > > > Tombstone > > > > > > > > Flag > > > > > > > > > > > > > > > > > > > > > > > > > > > > For the use case, one potential use case is > for > > > > > schema > > > > > > > > > > > > registration. > > > > > > > > > > > > > For > > > > > > > > > > > > > > example, in Avro, a null value corresponds > to a > > > > Null > > > > > > > > schema. > > > > > > > > > > So, > > > > > > > > > > > if > > > > > > > > > > > > > you > > > > > > > > > > > > > > want to be able to keep the schema id in a > > delete > > > > > > > message, > > > > > > > > > the > > > > > > > > > > > > value > > > > > > > > > > > > > can't > > > > > > > > > > > > > > be null. We could get around this issue by > > > > > specializing > > > > > > > > null > > > > > > > > > > > value > > > > > > > > > > > > > during > > > > > > > > > > > > > > schema registration though. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Now for the proposed changes. We probably > > should > > > > > > preserve > > > > > > > > > > client > > > > > > > > > > > > > > compatibility. If a client application is > > > sending a > > > > > > null > > > > > > > > > value > > > > > > > > > > > to a > > > > > > > > > > > > > > compacted topic, ideally, it should work the > > same > > > > > after > > > > > > > the > > > > > > > > > > > client > > > > > > > > > > > > > > upgrades. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure about making the tombstone > marker > > > > > > > > configurable, > > > > > > > > > > > > > especially at > > > > > > > > > > > > > > the topic level. Should we allow users to > > change > > > > the > > > > > > > config > > > > > > > > > > > values > > > > > > > > > > > > > back and > > > > > > > > > > > > > > forth, and what would be the implication? > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 7, 2016 at 10:48 AM, Becket Qin < > > > > > > > > > > > becket....@gmail.com> > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Michael, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, changing the logic in the log cleaner > > > makes > > > > > > sense. > > > > > > > > > There > > > > > > > > > > > > > could be > > > > > > > > > > > > > > some > > > > > > > > > > > > > > > other thing worth thinking (e.g. the > message > > > size > > > > > > > change > > > > > > > > > > after > > > > > > > > > > > > > > conversion), > > > > > > > > > > > > > > > though. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario I was thinking is the > following: > > > > > > > > > > > > > > > Imagine a distributed caching system built > on > > > top > > > > > of > > > > > > > > > Kafka. A > > > > > > > > > > > > user > > > > > > > > > > > > > is > > > > > > > > > > > > > > > consuming from a topic and it is guaranteed > > > that > > > > if > > > > > > the > > > > > > > > > user > > > > > > > > > > > > > consume to > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > log end it will get the latest value for > all > > > the > > > > > > keys. > > > > > > > > > > > Currently > > > > > > > > > > > > > if the > > > > > > > > > > > > > > > consumer sees a null value it knows the key > > has > > > > > been > > > > > > > > > removed. > > > > > > > > > > > Now > > > > > > > > > > > > > let's > > > > > > > > > > > > > > say > > > > > > > > > > > > > > > we rolled out this change. And the producer > > > > > applies a > > > > > > > > > message > > > > > > > > > > > > with > > > > > > > > > > > > > the > > > > > > > > > > > > > > > tombstone flag set, but the value was not > > null. > > > > > When > > > > > > we > > > > > > > > > > append > > > > > > > > > > > > that > > > > > > > > > > > > > > message > > > > > > > > > > > > > > > to the log I suppose we will not do the > down > > > > > > conversion > > > > > > > > if > > > > > > > > > > the > > > > > > > > > > > > > broker has > > > > > > > > > > > > > > > set the message.format.version to the > latest. > > > > > Because > > > > > > > the > > > > > > > > > log > > > > > > > > > > > > > cleaner > > > > > > > > > > > > > > won't > > > > > > > > > > > > > > > touch the active log segment, so that > message > > > > will > > > > > be > > > > > > > > > sitting > > > > > > > > > > > in > > > > > > > > > > > > > the > > > > > > > > > > > > > > active > > > > > > > > > > > > > > > segment as is. Now when a consumer that > > hasn't > > > > > > upgraded > > > > > > > > yet > > > > > > > > > > > > > consumes that > > > > > > > > > > > > > > > tombstone message in the active segment, it > > > seems > > > > > > that > > > > > > > > the > > > > > > > > > > > broker > > > > > > > > > > > > > will > > > > > > > > > > > > > > need > > > > > > > > > > > > > > > to down convert that message to remove the > > > value, > > > > > > > right? > > > > > > > > In > > > > > > > > > > > this > > > > > > > > > > > > > case, we > > > > > > > > > > > > > > > cannot wait for the log cleaner to do the > > down > > > > > > > conversion > > > > > > > > > > > because > > > > > > > > > > > > > that > > > > > > > > > > > > > > > message may have already been consumed > before > > > the > > > > > log > > > > > > > > > > > compaction > > > > > > > > > > > > > happens. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 7, 2016 at 9:59 AM, Michael > > Pearce > > > < > > > > > > > > > > > > > michael.pea...@ig.com> > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Becket, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We were thinking more about having the > > logic > > > > > that’s > > > > > > > in > > > > > > > > > the > > > > > > > > > > > > method > > > > > > > > > > > > > > > > shouldRetainMessage configurable via > > > > > > > > > > > http://kafka.apache.org/ > > > > > > > > > > > > > > > > documentation.html#brokerconfigs at a > > > > > > broker/topic > > > > > > > > > level. > > > > > > > > > > > And > > > > > > > > > > > > > then > > > > > > > > > > > > > > > scrap > > > > > > > > > > > > > > > > auto converting the message, and allow > > > > > > organisations > > > > > > > to > > > > > > > > > > > manage > > > > > > > > > > > > > the > > > > > > > > > > > > > > > rollout > > > > > > > > > > > > > > > > of enabling of the feature. > > > > > > > > > > > > > > > > (this isn’t in documentation but in > > response > > > to > > > > > the > > > > > > > > > > > discussion > > > > > > > > > > > > > thread > > > > > > > > > > > > > > as > > > > > > > > > > > > > > > > an alternative approach to roll out the > > > > feature) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Does this make any more sense? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > Mike > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 11/3/16, 2:27 PM, "Becket Qin" < > > > > > > > > becket....@gmail.com> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Michael, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Do you mean using a new configuration > > it > > > is > > > > > > just > > > > > > > > the > > > > > > > > > > > > exiting > > > > > > > > > > > > > > > > message.format.version config? It > seems > > > the > > > > > > > > > > > > > message.format.version > > > > > > > > > > > > > > > > config > > > > > > > > > > > > > > > > is enough in this case. And the > default > > > > value > > > > > > > would > > > > > > > > > > > always > > > > > > > > > > > > > be the > > > > > > > > > > > > > > > > latest > > > > > > > > > > > > > > > > version. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Message version migration would be > > > > handled > > > > > as > > > > > > > > like > > > > > > > > > in > > > > > > > > > > > > > KIP-32 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Also just want to confirm on this. > > Today > > > if > > > > > an > > > > > > > old > > > > > > > > > > > consumer > > > > > > > > > > > > > > consumes > > > > > > > > > > > > > > > a > > > > > > > > > > > > > > > > log > > > > > > > > > > > > > > > > compacted topic and sees an empty > > value, > > > it > > > > > > knows > > > > > > > > > that > > > > > > > > > > > is a > > > > > > > > > > > > > > > tombstone. > > > > > > > > > > > > > > > > After we start to use the attribute > > bit, > > > a > > > > > > > > tombstone > > > > > > > > > > > > message > > > > > > > > > > > > > can > > > > > > > > > > > > > > > have a > > > > > > > > > > > > > > > > non-empty value. So by "like in > KIP-32" > > > you > > > > > > mean > > > > > > > we > > > > > > > > > > will > > > > > > > > > > > > > remove the > > > > > > > > > > > > > > > > value > > > > > > > > > > > > > > > > to down convert the message if the > > > consumer > > > > > > > version > > > > > > > > > is > > > > > > > > > > > old, > > > > > > > > > > > > > right? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 2, 2016 at 1:37 AM, > Michael > > > > > Pearce > > > > > > < > > > > > > > > > > > > > > > michael.pea...@ig.com> > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Joel , et al. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Any comments on the below idea to > > > handle > > > > > roll > > > > > > > > out / > > > > > > > > > > > > > compatibility > > > > > > > > > > > > > > > of > > > > > > > > > > > > > > > > this > > > > > > > > > > > > > > > > > feature, using a configuration? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Does it make sense/clear? > > > > > > > > > > > > > > > > > Does it add value? > > > > > > > > > > > > > > > > > Do we want to enforce flag by > > default, > > > or > > > > > > value > > > > > > > > by > > > > > > > > > > > > > default, or > > > > > > > > > > > > > > > both? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Cheers > > > > > > > > > > > > > > > > > Mike > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 10/27/16, 4:47 PM, "Michael > > Pearce" > > > < > > > > > > > > > > > > > michael.pea...@ig.com> > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, James, I think this is > a > > > > really > > > > > > > good > > > > > > > > > > > addition > > > > > > > > > > > > > to the > > > > > > > > > > > > > > > KIP > > > > > > > > > > > > > > > > > details, please feel free to amend > > the > > > > > > wiki/add > > > > > > > > the > > > > > > > > > > use > > > > > > > > > > > > > cases, > > > > > > > > > > > > > > also > > > > > > > > > > > > > > > > if any > > > > > > > > > > > > > > > > > others you think of. I definitely > > think > > > > its > > > > > > > > > > worthwhile > > > > > > > > > > > > > > documenting. > > > > > > > > > > > > > > > > If you > > > > > > > > > > > > > > > > > can’t let me know ill add them next > > > week > > > > > > (just > > > > > > > > > > leaving > > > > > > > > > > > > for > > > > > > > > > > > > > a long > > > > > > > > > > > > > > > > weekend > > > > > > > > > > > > > > > > > off) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Re Joel and others comments > about > > > > > upgrade > > > > > > > and > > > > > > > > > > > > > compatibility. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Rather than trying to auto > manage > > > > this. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Actually maybe we make a > > > > configuration > > > > > > > > option, > > > > > > > > > > both > > > > > > > > > > > > at > > > > > > > > > > > > > server > > > > > > > > > > > > > > > > and per > > > > > > > > > > > > > > > > > topic level to control the behavior > > of > > > > how > > > > > > the > > > > > > > > > server > > > > > > > > > > > > logic > > > > > > > > > > > > > > should > > > > > > > > > > > > > > > > work out > > > > > > > > > > > > > > > > > if the record, is a tombstone > record > > . > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > e.g. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > key = > compation.tombstone.marker > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > value options: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > value (continues to use null > > > value > > > > as > > > > > > > > > tombstone > > > > > > > > > > > > > marker) > > > > > > > > > > > > > > > > > flag (expects to use the > > tombstone > > > > > flag) > > > > > > > > > > > > > > > > > value_or_flag (if either is > true > > it > > > > > > treats > > > > > > > > the > > > > > > > > > > > record > > > > > > > > > > > > > as a > > > > > > > > > > > > > > > > tombstone) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This way on upgrade users can > > keep > > > > > > current > > > > > > > > > > > behavior, > > > > > > > > > > > > > and > > > > > > > > > > > > > > slowly > > > > > > > > > > > > > > > > > migrate to the new. Having a > > transition > > > > > > period > > > > > > > of > > > > > > > > > > using > > > > > > > > > > > > > > > > value_or_flag, > > > > > > > > > > > > > > > > > finally having flag only if an > > > > organization > > > > > > > > wishes > > > > > > > > > to > > > > > > > > > > > use > > > > > > > > > > > > > null > > > > > > > > > > > > > > > values > > > > > > > > > > > > > > > > > without it being treated as a > > tombstone > > > > > > marker > > > > > > > > (use > > > > > > > > > > > case > > > > > > > > > > > > > noted > > > > > > > > > > > > > > > below) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having it both global broker > > level > > > > and > > > > > > > topic > > > > > > > > > > > override > > > > > > > > > > > > > also > > > > > > > > > > > > > > > > allows some > > > > > > > > > > > > > > > > > flexibility here. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Cheers > > > > > > > > > > > > > > > > > Mike > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 10/27/16, 8:03 AM, "James > > > Cheng" < > > > > > > > > > > > > > wushuja...@gmail.com> > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This KIP would definitely > > > > address a > > > > > > gap > > > > > > > > in > > > > > > > > > > the > > > > > > > > > > > > > current > > > > > > > > > > > > > > > > > functionality, where you currently > > > can't > > > > > > have a > > > > > > > > > > > tombstone > > > > > > > > > > > > > with > > > > > > > > > > > > > > any > > > > > > > > > > > > > > > > > associated content. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > That said, I'd like to talk > > > about > > > > > use > > > > > > > > > cases, > > > > > > > > > > to > > > > > > > > > > > > > make sure > > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > > this is in fact useful. The KIP > > should > > > be > > > > > > > updated > > > > > > > > > > with > > > > > > > > > > > > > whatever > > > > > > > > > > > > > > use > > > > > > > > > > > > > > > > cases > > > > > > > > > > > > > > > > > we come up with. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > First of all, an > observation: > > > > When > > > > > we > > > > > > > > speak > > > > > > > > > > > about > > > > > > > > > > > > > log > > > > > > > > > > > > > > > > compaction, > > > > > > > > > > > > > > > > > we typically think of "the latest > > > message > > > > > > for a > > > > > > > > key > > > > > > > > > > is > > > > > > > > > > > > > retained". > > > > > > > > > > > > > > > In > > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > > respect, a delete tombstone (i.e. a > > > > message > > > > > > > with > > > > > > > > a > > > > > > > > > > null > > > > > > > > > > > > > payload) > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > treated > > > > > > > > > > > > > > > > > the same as any other Kafka > message: > > > the > > > > > > latest > > > > > > > > > > message > > > > > > > > > > > > is > > > > > > > > > > > > > > > retained. > > > > > > > > > > > > > > > > It > > > > > > > > > > > > > > > > > doesn't matter whether the latest > > > message > > > > > is > > > > > > > > null, > > > > > > > > > or > > > > > > > > > > > if > > > > > > > > > > > > > the > > > > > > > > > > > > > > latest > > > > > > > > > > > > > > > > message > > > > > > > > > > > > > > > > > has actual content. In all cases, > the > > > > last > > > > > > > > message > > > > > > > > > is > > > > > > > > > > > > > retained. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The only way a delete > > tombstone > > > > is > > > > > > > > treated > > > > > > > > > > > > > differently > > > > > > > > > > > > > > from > > > > > > > > > > > > > > > > other > > > > > > > > > > > > > > > > > Kafka messages is that it > > automatically > > > > > > > > disappears > > > > > > > > > > > after > > > > > > > > > > > > a > > > > > > > > > > > > > while. > > > > > > > > > > > > > > > > The time > > > > > > > > > > > > > > > > > of deletion is specified using > > > > > > > > delete.retention.ms > > > > > > > > > . > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So what we're really > talking > > > > about > > > > > > is, > > > > > > > do > > > > > > > > > we > > > > > > > > > > > want > > > > > > > > > > > > > to > > > > > > > > > > > > > > > support > > > > > > > > > > > > > > > > > messages in a log-compacted topic > > that > > > > > > > > auto-delete > > > > > > > > > > > > > themselves > > > > > > > > > > > > > > after > > > > > > > > > > > > > > > > a while? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In a thread from 2015, > there > > > was > > > > a > > > > > > > > > discussion > > > > > > > > > > > on > > > > > > > > > > > > > > > first-class > > > > > > > > > > > > > > > > > support of headers between Roger > > > Hoover, > > > > > > Felix > > > > > > > > GV, > > > > > > > > > > Jun > > > > > > > > > > > > > Rao, and > > > > > > > > > > > > > > I. > > > > > > > > > > > > > > > > See > > > > > > > > > > > > > > > > > thread at > > https://groups.google.com/d/ > > > > > > > > > > > > > msg/confluent-platform/ > > > > > > > > > > > > > > > > > 8xPbjyUE_7E/yQ1AeCufL_gJ < > > > > > > > > > > https://groups.google.com/d/ > > > > > > > > > > > > > > > > > msg/confluent-platform/ > > > > > > > 8xPbjyUE_7E/yQ1AeCufL_gJ> > > > > > > > > . > > > > > > > > > > In > > > > > > > > > > > > that > > > > > > > > > > > > > > thread, > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > raised a good question that I > didn't > > > > have a > > > > > > > good > > > > > > > > > > answer > > > > > > > > > > > > > for at > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > time: If > > > > > > > > > > > > > > > > > a message is going to auto-delete > > > itself > > > > > > after > > > > > > > a > > > > > > > > > > while, > > > > > > > > > > > > how > > > > > > > > > > > > > > > > important was > > > > > > > > > > > > > > > > > the message? That is, what > > information > > > > did > > > > > > the > > > > > > > > > > message > > > > > > > > > > > > > contain > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > was > > > > > > > > > > > > > > > > > important *for a while* but not so > > > > > important > > > > > > > that > > > > > > > > > it > > > > > > > > > > > > > needed to be > > > > > > > > > > > > > > > > kept > > > > > > > > > > > > > > > > > around forever? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Some use cases that I can > > think > > > > of: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 1) Tracability. I would > like > > to > > > > > know > > > > > > > who > > > > > > > > > > issued > > > > > > > > > > > > > this > > > > > > > > > > > > > > delete > > > > > > > > > > > > > > > > > tombstone. It might include the > > > hostname, > > > > > IP > > > > > > of > > > > > > > > the > > > > > > > > > > > > > producer of > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > delete. > > > > > > > > > > > > > > > > > 2) Timestamps. I would like > > to > > > > know > > > > > > > when > > > > > > > > > this > > > > > > > > > > > > > delete was > > > > > > > > > > > > > > > > issued. > > > > > > > > > > > > > > > > > This use case is already addressed > by > > > the > > > > > > > > > > availability > > > > > > > > > > > of > > > > > > > > > > > > > > > per-message > > > > > > > > > > > > > > > > > timestamps that came in 0.10.0 > > > > > > > > > > > > > > > > > 3) Data provenance. I hope > > I'm > > > > > using > > > > > > > this > > > > > > > > > > > phrase > > > > > > > > > > > > > > correctly, > > > > > > > > > > > > > > > > but > > > > > > > > > > > > > > > > > what I mean is, where did this > delete > > > > come > > > > > > > from? > > > > > > > > > What > > > > > > > > > > > > > processing > > > > > > > > > > > > > > > job > > > > > > > > > > > > > > > > > emitted it? What input to the > > > processing > > > > > job > > > > > > > > caused > > > > > > > > > > > this > > > > > > > > > > > > > delete > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > be > > > > > > > > > > > > > > > > > produced? For example, if a record > in > > > > > topic A > > > > > > > was > > > > > > > > > > > > > processed and > > > > > > > > > > > > > > > > caused a > > > > > > > > > > > > > > > > > delete tombstone to be emitted to > > topic > > > > B, > > > > > I > > > > > > > > might > > > > > > > > > > like > > > > > > > > > > > > the > > > > > > > > > > > > > > offset > > > > > > > > > > > > > > > > of the > > > > > > > > > > > > > > > > > topic A message to be attached to > the > > > > > topic B > > > > > > > > > > message. > > > > > > > > > > > > > > > > > 4) Distributed tracing for > > > stream > > > > > > > > > topologies. > > > > > > > > > > > > This > > > > > > > > > > > > > might > > > > > > > > > > > > > > > be a > > > > > > > > > > > > > > > > > slight repeat of the above use > cases. > > > In > > > > > the > > > > > > > > > > > > microservices > > > > > > > > > > > > > world, > > > > > > > > > > > > > > > we > > > > > > > > > > > > > > > > can > > > > > > > > > > > > > > > > > generate call-graphs of webservices > > > using > > > > > > tools > > > > > > > > > like > > > > > > > > > > > > > Zipkin/ > > > > > > > > > > > > > > > > opentracing.io > > > > > > > > > > > > > > > > > <http://opentracing.io/>, or > > something > > > > > > > homegrown > > > > > > > > > > like > > > > > > > > > > > > > > > > > https://engineering.linkedin. > > > > > > > > > > > > com/distributed-service-call- > > > > > > > > > > > > > > > > > graph/real-time-distributed- > > > > > > > > > > > tracing-website-performance- > > > > > > > > > > > > > > > and-efficiency > > > > > > > > > > > > > > > > < > > > > > > > > > > > > > > > > > https://engineering.linkedin. > > > > > > > > > > > > com/distributed-service-call- > > > > > > > > > > > > > > > > > graph/real-time-distributed- > > > > > > > > > > > tracing-website-performance- > > > > > > > > > > > > > > > > and-efficiency>. > > > > > > > > > > > > > > > > > I can imagine that you might want > to > > do > > > > > > > something > > > > > > > > > > > similar > > > > > > > > > > > > > for > > > > > > > > > > > > > > > stream > > > > > > > > > > > > > > > > > processing topologies, where stream > > > > > > processing > > > > > > > > jobs > > > > > > > > > > > carry > > > > > > > > > > > > > along > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > forward > > > > > > > > > > > > > > > > > along a globally unique identifier, > > > and a > > > > > > > > > distributed > > > > > > > > > > > > > topology > > > > > > > > > > > > > > > graph > > > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > > generated. > > > > > > > > > > > > > > > > > 5) Cases where processing a > > > > delete > > > > > > > > requires > > > > > > > > > > > data > > > > > > > > > > > > > that is > > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > > available in the message key. I'm > not > > > > sure > > > > > I > > > > > > > > have a > > > > > > > > > > > good > > > > > > > > > > > > > example > > > > > > > > > > > > > > of > > > > > > > > > > > > > > > > this, > > > > > > > > > > > > > > > > > though. One hand-wavy example might > > be > > > > > where > > > > > > I > > > > > > > am > > > > > > > > > > > > > publishing > > > > > > > > > > > > > > > > documents into > > > > > > > > > > > > > > > > > Kafka where the documentId is the > > > message > > > > > > key, > > > > > > > > and > > > > > > > > > > the > > > > > > > > > > > > text > > > > > > > > > > > > > > > contents > > > > > > > > > > > > > > > > of the > > > > > > > > > > > > > > > > > document are in the message body. > > And I > > > > > have > > > > > > a > > > > > > > > > > > consuming > > > > > > > > > > > > > job that > > > > > > > > > > > > > > > > does some > > > > > > > > > > > > > > > > > analytics on the message body. If > > that > > > > > > document > > > > > > > > > gets > > > > > > > > > > > > > deleted, > > > > > > > > > > > > > > then > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > consuming job might need the > original > > > > > message > > > > > > > > body > > > > > > > > > in > > > > > > > > > > > > > order to > > > > > > > > > > > > > > > > "delete" > > > > > > > > > > > > > > > > > that message's impact from the > > > analytics. > > > > > But > > > > > > > I'm > > > > > > > > > not > > > > > > > > > > > > sure > > > > > > > > > > > > > that > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > a > > > > > > > > > > > > > > > > great > > > > > > > > > > > > > > > > > example. If the consumer was > worried > > > > about > > > > > > > that, > > > > > > > > > the > > > > > > > > > > > > > consumer > > > > > > > > > > > > > > would > > > > > > > > > > > > > > > > > probably keep the original message > > > > around, > > > > > > > stored > > > > > > > > > by > > > > > > > > > > > > > primary key. > > > > > > > > > > > > > > > > And then > > > > > > > > > > > > > > > > > all it would need from a delete > > message > > > > > would > > > > > > > be > > > > > > > > > the > > > > > > > > > > > > > primary key > > > > > > > > > > > > > > of > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > message. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Do people think these are > > valid > > > > use > > > > > > > > cases? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What are other use cases > that > > > > > people > > > > > > > can > > > > > > > > > > think > > > > > > > > > > > > of? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -James > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Oct 26, 2016, at 3:46 > > PM, > > > > > > Mayuresh > > > > > > > > > > Gharat > > > > > > > > > > > < > > > > > > > > > > > > > > > > > gharatmayures...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 @Joel. > > > > > > > > > > > > > > > > > > I think a clear migration > > > plan > > > > of > > > > > > > > > upgrading > > > > > > > > > > > and > > > > > > > > > > > > > > > > downgrading of > > > > > > > > > > > > > > > > > server and > > > > > > > > > > > > > > > > > > clients along with > handling > > > of > > > > > > issues > > > > > > > > > that > > > > > > > > > > > Joel > > > > > > > > > > > > > > > mentioned, > > > > > > > > > > > > > > > > on > > > > > > > > > > > > > > > > > the KIP would > > > > > > > > > > > > > > > > > > be really great. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Mayuresh > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 26, 2016 at > > 3:31 > > > > PM, > > > > > > Joel > > > > > > > > > > Koshy < > > > > > > > > > > > > > > > > jjkosh...@gmail.com> > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> I'm not sure why it > would > > be > > > > > > useful, > > > > > > > > but > > > > > > > > > > it > > > > > > > > > > > > > should be > > > > > > > > > > > > > > > > > theoretically > > > > > > > > > > > > > > > > > >> possible if the > attribute > > > bit > > > > > > alone > > > > > > > is > > > > > > > > > > > enough > > > > > > > > > > > > > to mark > > > > > > > > > > > > > > a > > > > > > > > > > > > > > > > > tombstone. OTOH, we > > > > > > > > > > > > > > > > > >> could consider that as > > > invalid > > > > > if > > > > > > we > > > > > > > > > wish. > > > > > > > > > > > > > These are > > > > > > > > > > > > > > > > relevant > > > > > > > > > > > > > > > > > details that > > > > > > > > > > > > > > > > > >> I think should be added > to > > > the > > > > > > KIP. > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> Also, in the few odd > > > scenarios > > > > > > that > > > > > > > I > > > > > > > > > > > > mentioned > > > > > > > > > > > > > we > > > > > > > > > > > > > > > should > > > > > > > > > > > > > > > > also > > > > > > > > > > > > > > > > > consider > > > > > > > > > > > > > > > > > >> that fetches could be > > coming > > > > > from > > > > > > > > other > > > > > > > > > > > > > > > yet-to-be-upgraded > > > > > > > > > > > > > > > > > brokers in a > > > > > > > > > > > > > > > > > >> cluster that is being > > > > upgraded. > > > > > So > > > > > > > we > > > > > > > > > > would > > > > > > > > > > > > > probably > > > > > > > > > > > > > > > want > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > continue to > > > > > > > > > > > > > > > > > >> support nulls as > > tombstones > > > or > > > > > > > > > > down-convert > > > > > > > > > > > in > > > > > > > > > > > > > a way > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > we > > > > > > > > > > > > > > > > > are sure works > > > > > > > > > > > > > > > > > >> with least surprise to > > > > fetchers. > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> There is a slightly > vague > > > > > > statement > > > > > > > > > under > > > > > > > > > > > > > > > "Compatibility, > > > > > > > > > > > > > > > > > Deprecation, and > > > > > > > > > > > > > > > > > >> Migration Plan" that > could > > > > > benefit > > > > > > > > more > > > > > > > > > > > > details: > > > > > > > > > > > > > > *Logic > > > > > > > > > > > > > > > > would > > > > > > > > > > > > > > > > > base on > > > > > > > > > > > > > > > > > >> current behavior of null > > > value > > > > > or > > > > > > if > > > > > > > > > > > tombstone > > > > > > > > > > > > > flag > > > > > > > > > > > > > > set > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > true, as such > > > > > > > > > > > > > > > > > >> wouldn't impact any > > existing > > > > > flows > > > > > > > > > simply > > > > > > > > > > > > allow > > > > > > > > > > > > > new > > > > > > > > > > > > > > > > producers > > > > > > > > > > > > > > > > > to make use > > > > > > > > > > > > > > > > > >> of the feature*. It is > > > unclear > > > > > to > > > > > > me > > > > > > > > > based > > > > > > > > > > > on > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > whether you > > > > > > > > > > > > > > > > > would > > > > > > > > > > > > > > > > > >> interpret null as a > > > tombstone > > > > if > > > > > > the > > > > > > > > > > > tombstone > > > > > > > > > > > > > > attribute > > > > > > > > > > > > > > > > bit is > > > > > > > > > > > > > > > > > off. > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> On Wed, Oct 26, 2016 at > > 3:10 > > > > PM, > > > > > > > > Xavier > > > > > > > > > > > > Léauté < > > > > > > > > > > > > > > > > > xav...@confluent.io> > > > > > > > > > > > > > > > > > >> wrote: > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >>> Does this mean that > > > starting > > > > > with > > > > > > > V4 > > > > > > > > > > > requests > > > > > > > > > > > > > we > > > > > > > > > > > > > > would > > > > > > > > > > > > > > > > allow > > > > > > > > > > > > > > > > > storing null > > > > > > > > > > > > > > > > > >>> messages in compacted > > > topics? > > > > > The > > > > > > > KIP > > > > > > > > > > > should > > > > > > > > > > > > > probably > > > > > > > > > > > > > > > > clarify > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > >> behavior > > > > > > > > > > > > > > > > > >>> for null messages where > > the > > > > > > > tombstone > > > > > > > > > > flag > > > > > > > > > > > is > > > > > > > > > > > > > not > > > > > > > > > > > > > > net. > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > >>> On Wed, Oct 26, 2016 at > > > 1:32 > > > > AM > > > > > > > > Magnus > > > > > > > > > > > > > Edenhill < > > > > > > > > > > > > > > > > > mag...@edenhill.se> > > > > > > > > > > > > > > > > > >>> wrote: > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > >>>> 2016-10-25 21:36 > > GMT+02:00 > > > > > Nacho > > > > > > > > Solis > > > > > > > > > > > > > > > > > <nso...@linkedin.com.invalid>: > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>>> I think you probably > > > > require > > > > > a > > > > > > > > > > MagicByte > > > > > > > > > > > > > bump if > > > > > > > > > > > > > > you > > > > > > > > > > > > > > > > expect > > > > > > > > > > > > > > > > > correct > > > > > > > > > > > > > > > > > >>>>> behavior of the > system > > > as a > > > > > > > whole. > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > >>>>> From a client > > perspective > > > > you > > > > > > > want > > > > > > > > to > > > > > > > > > > > make > > > > > > > > > > > > > sure > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > when you > > > > > > > > > > > > > > > > > >> deliver a > > > > > > > > > > > > > > > > > >>>>> message that the > broker > > > > > > supports > > > > > > > > the > > > > > > > > > > > > feature > > > > > > > > > > > > > you're > > > > > > > > > > > > > > > > expecting > > > > > > > > > > > > > > > > > >>>>> (compaction). So, > > > > depending > > > > > on > > > > > > > the > > > > > > > > > > > > behavior > > > > > > > > > > > > > of the > > > > > > > > > > > > > > > > broker on > > > > > > > > > > > > > > > > > >>>> encountering > > > > > > > > > > > > > > > > > >>>>> a previously > undefined > > > bit > > > > > > flag I > > > > > > > > > would > > > > > > > > > > > > > suggest > > > > > > > > > > > > > > > making > > > > > > > > > > > > > > > > some > > > > > > > > > > > > > > > > > change to > > > > > > > > > > > > > > > > > >>>> make > > > > > > > > > > > > > > > > > >>>>> certain that > flag-based > > > > > > > compaction > > > > > > > > is > > > > > > > > > > > > > supported. > > > > > > > > > > > > > > I'm > > > > > > > > > > > > > > > > going > > > > > > > > > > > > > > > > > to guess > > > > > > > > > > > > > > > > > >>> that > > > > > > > > > > > > > > > > > >>>>> the MagicByte would > do > > > > this. > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> I dont believe this is > > > > needed > > > > > > > since > > > > > > > > it > > > > > > > > > > is > > > > > > > > > > > > > already > > > > > > > > > > > > > > > > attributed > > > > > > > > > > > > > > > > > through > > > > > > > > > > > > > > > > > >> the > > > > > > > > > > > > > > > > > >>>> request's API version. > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> Producer: > > > > > > > > > > > > > > > > > >>>> * if a client sends > > > > > > ProduceRequest > > > > > > > > V4 > > > > > > > > > > then > > > > > > > > > > > > > > > > attributes.bit5 > > > > > > > > > > > > > > > > > indicates a > > > > > > > > > > > > > > > > > >>>> tombstone > > > > > > > > > > > > > > > > > >>>> * if a clients sends > > > > > > > ProduceRequest > > > > > > > > > <V4 > > > > > > > > > > > then > > > > > > > > > > > > > > > > attributes.bit5 > > > > > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > > >> ignored > > > > > > > > > > > > > > > > > >>>> and value==null > > indicates > > > a > > > > > > > > tombstone > > > > > > > > > > > > > > > > > >>>> * in both cases the > > > on-disk > > > > > > > messages > > > > > > > > > are > > > > > > > > > > > > > stored with > > > > > > > > > > > > > > > > > attributes.bit5 > > > > > > > > > > > > > > > > > >> (I > > > > > > > > > > > > > > > > > >>>> assume?) > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> Consumer: > > > > > > > > > > > > > > > > > >>>> * if a clients sends > > > > > > FetchRequest > > > > > > > V4 > > > > > > > > > > > > messages > > > > > > > > > > > > > are > > > > > > > > > > > > > > > > > sendfile():ed > > > > > > > > > > > > > > > > > >> directly > > > > > > > > > > > > > > > > > >>>> from disk (with > > > > > attributes.bit5) > > > > > > > > > > > > > > > > > >>>> * if a client sends > > > > > FetchRequest > > > > > > > <V4 > > > > > > > > > > > > messages > > > > > > > > > > > > > are > > > > > > > > > > > > > > > > slowpathed > > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > >>>> translated from > > > > > attributes.bit5 > > > > > > to > > > > > > > > > > > > value=null > > > > > > > > > > > > > as > > > > > > > > > > > > > > > > required. > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> That's my > understanding > > > > > anyway, > > > > > > > > please > > > > > > > > > > > > > correct me if > > > > > > > > > > > > > > > I'm > > > > > > > > > > > > > > > > > wrong. > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> /Magnus > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>>> On Tue, Oct 25, 2016 > at > > > > 10:17 > > > > > > AM, > > > > > > > > > > Magnus > > > > > > > > > > > > > Edenhill < > > > > > > > > > > > > > > > > > >> mag...@edenhill.se> > > > > > > > > > > > > > > > > > >>>>> wrote: > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > >>>>>> It is safe to assume > > > that > > > > a > > > > > > > > > previously > > > > > > > > > > > > > undefined > > > > > > > > > > > > > > > > attributes > > > > > > > > > > > > > > > > > bit > > > > > > > > > > > > > > > > > >> will > > > > > > > > > > > > > > > > > >>> be > > > > > > > > > > > > > > > > > >>>>>> unset in protocol > > > requests > > > > > > from > > > > > > > > > > existing > > > > > > > > > > > > > clients, > > > > > > > > > > > > > > if > > > > > > > > > > > > > > > > not, > > > > > > > > > > > > > > > > > such a > > > > > > > > > > > > > > > > > >>> client > > > > > > > > > > > > > > > > > >>>>> is > > > > > > > > > > > > > > > > > >>>>>> already violating > the > > > > > protocol > > > > > > > and > > > > > > > > > > needs > > > > > > > > > > > > to > > > > > > > > > > > > > be > > > > > > > > > > > > > > > fixed. > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > >>>>>> So I dont see a need > > > for a > > > > > > > > MagicByte > > > > > > > > > > > bump, > > > > > > > > > > > > > both > > > > > > > > > > > > > > > > broker and > > > > > > > > > > > > > > > > > client > > > > > > > > > > > > > > > > > >> has > > > > > > > > > > > > > > > > > >>>> the > > > > > > > > > > > > > > > > > >>>>>> information it needs > > to > > > > > > > construct > > > > > > > > or > > > > > > > > > > > parse > > > > > > > > > > > > > the > > > > > > > > > > > > > > > message > > > > > > > > > > > > > > > > > according to > > > > > > > > > > > > > > > > > >>>>> request > > > > > > > > > > > > > > > > > >>>>>> version. > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > >>>>>> 2016-10-25 18:48 > > > GMT+02:00 > > > > > > > Michael > > > > > > > > > > > Pearce > > > > > > > > > > > > < > > > > > > > > > > > > > > > > > michael.pea...@ig.com>: > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > >>>>>>> Hi Magnus, > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > >>>>>>> I was wondering if > I > > > even > > > > > > > needed > > > > > > > > to > > > > > > > > > > > > change > > > > > > > > > > > > > those > > > > > > > > > > > > > > > > also, as > > > > > > > > > > > > > > > > > >>> technically > > > > > > > > > > > > > > > > > >>>>>>> we’re just making > use > > > of > > > > a > > > > > > non > > > > > > > > used > > > > > > > > > > > > > attribute > > > > > > > > > > > > > > bit, > > > > > > > > > > > > > > > > but im > > > > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > > >> 100% > > > > > > > > > > > > > > > > > >>>> that > > > > > > > > > > > > > > > > > >>>>>> it > > > > > > > > > > > > > > > > > >>>>>>> be always false > > > > currently. > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > >>>>>>> If someone can say > > 100% > > > > it > > > > > > will > > > > > > > > > > already > > > > > > > > > > > > be > > > > > > > > > > > > > set > > > > > > > > > > > > > > > false > > > > > > > > > > > > > > > > with > > > > > > > > > > > > > > > > > current > > > > > > > > > > > > > > > > > >>> and > > > > > > > > > > > > > > > > > >>>>>>> historic bit wise > > > masking > > > > > > > > > techniques > > > > > > > > > > > used > > > > > > > > > > > > > over > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > time, > > > > > > > > > > > > > > > > > we could > > > > > > > > > > > > > > > > > >>> do > > > > > > > > > > > > > > > > > >>>>> away > > > > > > > > > > > > > > > > > >>>>>>> with both, and > simply > > > > just > > > > > > > start > > > > > > > > to > > > > > > > > > > use > > > > > > > > > > > > it. > > > > > > > > > > > > > > > > Unfortunately > > > > > > > > > > > > > > > > > I don’t > > > > > > > > > > > > > > > > > >>>> have > > > > > > > > > > > > > > > > > >>>>>> that > > > > > > > > > > > > > > > > > >>>>>>> historic knowledge > so > > > was > > > > > > > hoping > > > > > > > > it > > > > > > > > > > > would > > > > > > > > > > > > > be > > > > > > > > > > > > > > > flagged > > > > > > > > > > > > > > > > up in > > > > > > > > > > > > > > > > > this > > > > > > > > > > > > > > > > > >>>>>> discussion > > > > > > > > > > > > > > > > > >>>>>>> thread ? > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > >>>>>>> Cheers > > > > > > > > > > > > > > > > > >>>>>>> Mike > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > >>>>>>> On 10/25/16, 5:36 > PM, > > > > > "Magnus > > > > > > > > > > > Edenhill" < > > > > > > > > > > > > > > > > > mag...@edenhill.se> > > > > > > > > > > > > > > > > > >>> wrote: > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > >>>>>>> Hi Michael, > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > >>>>>>> With the version > > > bumps > > > > > for > > > > > > > > > Produce > > > > > > > > > > > and > > > > > > > > > > > > > Fetch > > > > > > > > > > > > > > > > requests, > > > > > > > > > > > > > > > > > do you > > > > > > > > > > > > > > > > > >>>>> really > > > > > > > > > > > > > > > > > >>>>>>> need > > > > > > > > > > > > > > > > > >>>>>>> to bump > MagicByte > > > too? > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > >>>>>>> Regards, > > > > > > > > > > > > > > > > > >>>>>>> Magnus > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > >>>>>>> 2016-10-25 18:09 > > > > > GMT+02:00 > > > > > > > > > Michael > > > > > > > > > > > > > Pearce < > > > > > > > > > > > > > > > > > >>> michael.pea...@ig.com > > > > > > > > > > > > > > > > > >>>>> : > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > >>>>>>>> Hi All, > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > >>>>>>>> I would like to > > > discuss > > > > > the > > > > > > > > > > following > > > > > > > > > > > > KIP > > > > > > > > > > > > > > > proposal: > > > > > > > > > > > > > > > > > >>>>>>>> > > > > https://cwiki.apache.org/ > > > > > > > > > > > > > > > > confluence/display/KAFKA/KIP- > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > 87+-+Add+Compaction+Tombstone+ > > > > > > > > > Flag > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > >>>>>>>> This is off the > back > > > of > > > > > the > > > > > > > > > > discussion > > > > > > > > > > > > on > > > > > > > > > > > > > KIP-82 > > > > > > > > > > > > > > > / > > > > > > > > > > > > > > > > KIP > > > > > > > > > > > > > > > > > >>> meeting > > > > > > > > > > > > > > > > > >>>>>>> where it > > > > > > > > > > > > > > > > > >>>>>>>> was agreed to > > separate > > > > > this > > > > > > > > issue > > > > > > > > > > and > > > > > > > > > > > > > feature. > > > > > > > > > > > > > > > See: > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > http://mail-archives.apache > > > > > > . > > > > > > > > > > > > > > > > org/mod_mbox/kafka-dev/201610 > > > > > > > > > > > > > > > > > . > > > > > > > > > > > > > > > > > >>>>>>>> > > mbox/%3cCAJS3ho8OcR== > > > > > > > > > > > > > EcxsJ8OP99pD2hz=iiGecWsv- > > > > > > > > > > > > > > > > > >>>>>>>> EZsBsNyDcKr= > > > > > > g...@mail.gmail.com% > > > > > > > 3e > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > >>>>>>>> Thanks > > > > > > > > > > > > > > > > > >>>>>>>> Mike > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > >>>>>>>> The information > > > > contained > > > > > in > > > > > > > > this > > > > > > > > > > > email > > > > > > > > > > > > is > > > > > > > > > > > > > > > strictly > > > > > > > > > > > > > > > > > >>>> confidential > > > > > > > > > > > > > > > > > >>>>>> and > > > > > > > > > > > > > > > > > >>>>>>> for > > > > > > > > > > > > > > > > > >>>>>>>> the use of the > > > addressee > > > > > > only, > > > > > > > > > > unless > > > > > > > > > > > > > otherwise > > > > > > > > > > > > > > > > indicated. > > > > > > > > > > > > > > > > > >> If > > > > > > > > > > > > > > > > > >>>> you > > > > > > > > > > > > > > > > > >>>>>>> are not > > > > > > > > > > > > > > > > > >>>>>>>> the intended > > > recipient, > > > > > > please > > > > > > > > do > > > > > > > > > > not > > > > > > > > > > > > > read, > > > > > > > > > > > > > > copy, > > > > > > > > > > > > > > > > use or > > > > > > > > > > > > > > > > > >>>> disclose > > > > > > > > > > > > > > > > > >>>>>> to > > > > > > > > > > > > > > > > > >>>>>>> others > > > > > > > > > > > > > > > > > >>>>>>>> this message or > any > > > > > > > attachment. > > > > > > > > > > Please > > > > > > > > > > > > > also > > > > > > > > > > > > > > notify > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > >> sender > > > > > > > > > > > > > > > > > >>>> by > > > > > > > > > > > > > > > > > >>>>>>> replying > > > > > > > > > > > > > > > > > >>>>>>>> to this email or > by > > > > > > telephone > > > > > > > > > > (+44(020 > > > > > > > > > > > > > 7896 > > > > > > > > > > > > > > 0011) > > > > > > > > > > > > > > > > and then > > > > > > > > > > > > > > > > > >>>> delete > > > > > > > > > > > > > > > > > >>>>>>> the email > > > > > > > > > > > > > > > > > >>>>>>>> and any copies of > > it. > > > > > > > Opinions, > > > > > > > > > > > > > conclusion (etc) > > > > > > > > > > > > > > > > that do > > > > > > > > > > > > > > > > > >> not > > > > > > > > > > > > > > > > > >>>>> relate > > > > > > > > > > > > > > > > > >>>>>>> to the > > > > > > > > > > > > > > > > > >>>>>>>> official business > of > > > > this > > > > > > > > company > > > > > > > > > > > shall > > > > > > > > > > > > be > > > > > > > > > > > > > > > > understood as > > > > > > > > > > > > > > > > > >>>> neither > > > > > > > > > > > > > > > > > >>>>>>> given nor > > > > > > > > > > > > > > > > > >>>>>>>> endorsed by it. IG > > is > > > a > > > > > > > trading > > > > > > > > > name > > > > > > > > > > > of > > > > > > > > > > > > IG > > > > > > > > > > > > > > Markets > > > > > > > > > > > > > > > > Limited > > > > > > > > > > > > > > > > > >> (a > > > > > > > > > > > > > > > > > >>>>>> company > > > > > > > > > > > > > > > > > >>>>>>>> registered in > > England > > > > and > > > > > > > Wales, > > > > > > > > > > > company > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > 04008957) > > > > > > > > > > > > > > > > > >> and > > > > > > > > > > > > > > > > > >>>> IG > > > > > > > > > > > > > > > > > >>>>>>> Index > > > > > > > > > > > > > > > > > >>>>>>>> Limited (a company > > > > > > registered > > > > > > > in > > > > > > > > > > > England > > > > > > > > > > > > > and > > > > > > > > > > > > > > > Wales, > > > > > > > > > > > > > > > > > company > > > > > > > > > > > > > > > > > >>>>> number > > > > > > > > > > > > > > > > > >>>>>>>> 01190902). > > Registered > > > > > > address > > > > > > > at > > > > > > > > > > > Cannon > > > > > > > > > > > > > Bridge > > > > > > > > > > > > > > > > House, 25 > > > > > > > > > > > > > > > > > >>>> Dowgate > > > > > > > > > > > > > > > > > >>>>>>> Hill, > > > > > > > > > > > > > > > > > >>>>>>>> London EC4R 2YA. > > Both > > > IG > > > > > > > Markets > > > > > > > > > > > Limited > > > > > > > > > > > > > > (register > > > > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > > >>>> 195355) > > > > > > > > > > > > > > > > > >>>>>>> and IG > > > > > > > > > > > > > > > > > >>>>>>>> Index Limited > > > (register > > > > > > number > > > > > > > > > > 114059) > > > > > > > > > > > > are > > > > > > > > > > > > > > > > authorised and > > > > > > > > > > > > > > > > > >>>>> regulated > > > > > > > > > > > > > > > > > >>>>>>> by the > > > > > > > > > > > > > > > > > >>>>>>>> Financial Conduct > > > > > Authority. > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > >>>>>>> The information > > > contained > > > > > in > > > > > > > this > > > > > > > > > > email > > > > > > > > > > > > is > > > > > > > > > > > > > > strictly > > > > > > > > > > > > > > > > > confidential > > > > > > > > > > > > > > > > > >>> and > > > > > > > > > > > > > > > > > >>>>> for > > > > > > > > > > > > > > > > > >>>>>>> the use of the > > > addressee > > > > > > only, > > > > > > > > > unless > > > > > > > > > > > > > otherwise > > > > > > > > > > > > > > > > indicated. > > > > > > > > > > > > > > > > > If you > > > > > > > > > > > > > > > > > >>> are > > > > > > > > > > > > > > > > > >>>>> not > > > > > > > > > > > > > > > > > >>>>>>> the intended > > recipient, > > > > > > please > > > > > > > do > > > > > > > > > not > > > > > > > > > > > > > read, copy, > > > > > > > > > > > > > > > > use or > > > > > > > > > > > > > > > > > disclose > > > > > > > > > > > > > > > > > >>> to > > > > > > > > > > > > > > > > > >>>>>> others > > > > > > > > > > > > > > > > > >>>>>>> this message or any > > > > > > attachment. > > > > > > > > > > Please > > > > > > > > > > > > also > > > > > > > > > > > > > > notify > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > sender by > > > > > > > > > > > > > > > > > >>>>> replying > > > > > > > > > > > > > > > > > >>>>>>> to this email or by > > > > > telephone > > > > > > > > > > (+44(020 > > > > > > > > > > > > > 7896 0011) > > > > > > > > > > > > > > > > and then > > > > > > > > > > > > > > > > > delete > > > > > > > > > > > > > > > > > >>> the > > > > > > > > > > > > > > > > > >>>>>> email > > > > > > > > > > > > > > > > > >>>>>>> and any copies of > it. > > > > > > Opinions, > > > > > > > > > > > > conclusion > > > > > > > > > > > > > (etc) > > > > > > > > > > > > > > > > that do > > > > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > > >> relate > > > > > > > > > > > > > > > > > >>>> to > > > > > > > > > > > > > > > > > >>>>>> the > > > > > > > > > > > > > > > > > >>>>>>> official business > of > > > this > > > > > > > company > > > > > > > > > > shall > > > > > > > > > > > > be > > > > > > > > > > > > > > > > understood as > > > > > > > > > > > > > > > > > neither > > > > > > > > > > > > > > > > > >>>> given > > > > > > > > > > > > > > > > > >>>>>> nor > > > > > > > > > > > > > > > > > >>>>>>> endorsed by it. IG > > is a > > > > > > trading > > > > > > > > > name > > > > > > > > > > of > > > > > > > > > > > > IG > > > > > > > > > > > > > > Markets > > > > > > > > > > > > > > > > Limited > > > > > > > > > > > > > > > > > (a > > > > > > > > > > > > > > > > > >>> company > > > > > > > > > > > > > > > > > >>>>>>> registered in > England > > > and > > > > > > > Wales, > > > > > > > > > > > company > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > 04008957) > > > > > > > > > > > > > > > > > and IG > > > > > > > > > > > > > > > > > >>>> Index > > > > > > > > > > > > > > > > > >>>>>>> Limited (a company > > > > > registered > > > > > > > in > > > > > > > > > > > England > > > > > > > > > > > > > and > > > > > > > > > > > > > > Wales, > > > > > > > > > > > > > > > > company > > > > > > > > > > > > > > > > > >> number > > > > > > > > > > > > > > > > > >>>>>>> 01190902). > Registered > > > > > address > > > > > > > at > > > > > > > > > > Cannon > > > > > > > > > > > > > Bridge > > > > > > > > > > > > > > > > House, 25 > > > > > > > > > > > > > > > > > Dowgate > > > > > > > > > > > > > > > > > >>>> Hill, > > > > > > > > > > > > > > > > > >>>>>>> London EC4R 2YA. > Both > > > IG > > > > > > > Markets > > > > > > > > > > > Limited > > > > > > > > > > > > > > (register > > > > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > > 195355) > > > > > > > > > > > > > > > > > >>> and > > > > > > > > > > > > > > > > > >>>>> IG > > > > > > > > > > > > > > > > > >>>>>>> Index Limited > > (register > > > > > > number > > > > > > > > > > 114059) > > > > > > > > > > > > are > > > > > > > > > > > > > > > > authorised and > > > > > > > > > > > > > > > > > >> regulated > > > > > > > > > > > > > > > > > >>>> by > > > > > > > > > > > > > > > > > >>>>>> the > > > > > > > > > > > > > > > > > >>>>>>> Financial Conduct > > > > > Authority. > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > >>>>> -- > > > > > > > > > > > > > > > > > >>>>> Nacho (Ignacio) Solis > > > > > > > > > > > > > > > > > >>>>> Kafka > > > > > > > > > > > > > > > > > >>>>> nso...@linkedin.com > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > -Regards, > > > > > > > > > > > > > > > > > > Mayuresh R. Gharat > > > > > > > > > > > > > > > > > > (862) 250-7125 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The information contained in > this > > > > email > > > > > > is > > > > > > > > > > strictly > > > > > > > > > > > > > > > confidential > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > for the use of the addressee only, > > > unless > > > > > > > > otherwise > > > > > > > > > > > > > indicated. If > > > > > > > > > > > > > > > > you are > > > > > > > > > > > > > > > > > not the intended recipient, please > do > > > not > > > > > > read, > > > > > > > > > copy, > > > > > > > > > > > use > > > > > > > > > > > > > or > > > > > > > > > > > > > > > > disclose to > > > > > > > > > > > > > > > > > others this message or any > > attachment. > > > > > Please > > > > > > > > also > > > > > > > > > > > notify > > > > > > > > > > > > > the > > > > > > > > > > > > > > > sender > > > > > > > > > > > > > > > > by > > > > > > > > > > > > > > > > > replying to this email or by > > telephone > > > > > > (+44(020 > > > > > > > > > 7896 > > > > > > > > > > > > 0011) > > > > > > > > > > > > > and > > > > > > > > > > > > > > then > > > > > > > > > > > > > > > > delete > > > > > > > > > > > > > > > > > the email and any copies of it. > > > Opinions, > > > > > > > > > conclusion > > > > > > > > > > > > (etc) > > > > > > > > > > > > > that > > > > > > > > > > > > > > do > > > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > > relate to the official business of > > this > > > > > > company > > > > > > > > > shall > > > > > > > > > > > be > > > > > > > > > > > > > > understood > > > > > > > > > > > > > > > > as > > > > > > > > > > > > > > > > > neither given nor endorsed by it. > IG > > > is a > > > > > > > trading > > > > > > > > > > name > > > > > > > > > > > of > > > > > > > > > > > > > IG > > > > > > > > > > > > > > > Markets > > > > > > > > > > > > > > > > > Limited (a company registered in > > > England > > > > > and > > > > > > > > Wales, > > > > > > > > > > > > company > > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > > 04008957) and IG Index Limited (a > > > company > > > > > > > > > registered > > > > > > > > > > in > > > > > > > > > > > > > England > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > Wales, > > > > > > > > > > > > > > > > > company number 01190902). > Registered > > > > > address > > > > > > at > > > > > > > > > > Cannon > > > > > > > > > > > > > Bridge > > > > > > > > > > > > > > > House, > > > > > > > > > > > > > > > > 25 > > > > > > > > > > > > > > > > > Dowgate Hill, London EC4R 2YA. Both > > IG > > > > > > Markets > > > > > > > > > > Limited > > > > > > > > > > > > > (register > > > > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > > 195355) and IG Index Limited > > (register > > > > > number > > > > > > > > > 114059) > > > > > > > > > > > are > > > > > > > > > > > > > > > authorised > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > regulated by the Financial Conduct > > > > > Authority. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The information contained in this email > is > > > > > strictly > > > > > > > > > > > > confidential > > > > > > > > > > > > > and > > > > > > > > > > > > > > for > > > > > > > > > > > > > > > > the use of the addressee only, unless > > > otherwise > > > > > > > > > indicated. > > > > > > > > > > If > > > > > > > > > > > > > you are > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > the intended recipient, please do not > read, > > > > copy, > > > > > > use > > > > > > > > or > > > > > > > > > > > > > disclose to > > > > > > > > > > > > > > > others > > > > > > > > > > > > > > > > this message or any attachment. Please > also > > > > > notify > > > > > > > the > > > > > > > > > > sender > > > > > > > > > > > > by > > > > > > > > > > > > > > replying > > > > > > > > > > > > > > > > to this email or by telephone (+44(020 > 7896 > > > > 0011) > > > > > > and > > > > > > > > > then > > > > > > > > > > > > > delete the > > > > > > > > > > > > > > > email > > > > > > > > > > > > > > > > and any copies of it. Opinions, > conclusion > > > > (etc) > > > > > > that > > > > > > > > do > > > > > > > > > > not > > > > > > > > > > > > > relate to > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > official business of this company shall > be > > > > > > understood > > > > > > > > as > > > > > > > > > > > > neither > > > > > > > > > > > > > given > > > > > > > > > > > > > > > nor > > > > > > > > > > > > > > > > endorsed by it. IG is a trading name of > IG > > > > > Markets > > > > > > > > > Limited > > > > > > > > > > (a > > > > > > > > > > > > > company > > > > > > > > > > > > > > > > registered in England and Wales, company > > > number > > > > > > > > 04008957) > > > > > > > > > > and > > > > > > > > > > > > IG > > > > > > > > > > > > > Index > > > > > > > > > > > > > > > > Limited (a company registered in England > > and > > > > > Wales, > > > > > > > > > company > > > > > > > > > > > > > number > > > > > > > > > > > > > > > > 01190902). Registered address at Cannon > > > Bridge > > > > > > House, > > > > > > > > 25 > > > > > > > > > > > > Dowgate > > > > > > > > > > > > > Hill, > > > > > > > > > > > > > > > > London EC4R 2YA. Both IG Markets Limited > > > > > (register > > > > > > > > number > > > > > > > > > > > > > 195355) and > > > > > > > > > > > > > > IG > > > > > > > > > > > > > > > > Index Limited (register number 114059) > are > > > > > > authorised > > > > > > > > and > > > > > > > > > > > > > regulated by > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > Financial Conduct Authority. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The information contained in this email is > > > strictly > > > > > > > > > > confidential > > > > > > > > > > > > and > > > > > > > > > > > > > for > > > > > > > > > > > > > > the use of the addressee only, unless > otherwise > > > > > > > indicated. > > > > > > > > If > > > > > > > > > > you > > > > > > > > > > > > > are not > > > > > > > > > > > > > > the intended recipient, please do not read, > > copy, > > > > use > > > > > > or > > > > > > > > > > disclose > > > > > > > > > > > > to > > > > > > > > > > > > > others > > > > > > > > > > > > > > this message or any attachment. Please also > > > notify > > > > > the > > > > > > > > sender > > > > > > > > > > by > > > > > > > > > > > > > replying > > > > > > > > > > > > > > to this email or by telephone (+44(020 7896 > > 0011) > > > > and > > > > > > > then > > > > > > > > > > delete > > > > > > > > > > > > > the email > > > > > > > > > > > > > > and any copies of it. Opinions, conclusion > > (etc) > > > > that > > > > > > do > > > > > > > > not > > > > > > > > > > > relate > > > > > > > > > > > > > to the > > > > > > > > > > > > > > official business of this company shall be > > > > understood > > > > > > as > > > > > > > > > > neither > > > > > > > > > > > > > given nor > > > > > > > > > > > > > > endorsed by it. IG is a trading name of IG > > > Markets > > > > > > > Limited > > > > > > > > (a > > > > > > > > > > > > company > > > > > > > > > > > > > > registered in England and Wales, company > number > > > > > > 04008957) > > > > > > > > and > > > > > > > > > > IG > > > > > > > > > > > > > Index > > > > > > > > > > > > > > Limited (a company registered in England and > > > Wales, > > > > > > > company > > > > > > > > > > > number > > > > > > > > > > > > > > 01190902). Registered address at Cannon > Bridge > > > > House, > > > > > > 25 > > > > > > > > > > Dowgate > > > > > > > > > > > > > Hill, > > > > > > > > > > > > > > London EC4R 2YA. Both IG Markets Limited > > > (register > > > > > > number > > > > > > > > > > 195355) > > > > > > > > > > > > > and IG > > > > > > > > > > > > > > Index Limited (register number 114059) are > > > > authorised > > > > > > and > > > > > > > > > > > regulated > > > > > > > > > > > > > by the > > > > > > > > > > > > > > Financial Conduct Authority. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > -Regards, > > > > > > > > > > > > > Mayuresh R. Gharat > > > > > > > > > > > > > (862) 250-7125 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The information contained in this email is strictly > > > > > > > confidential > > > > > > > > > and > > > > > > > > > > > for > > > > > > > > > > > > > the use of the addressee only, unless otherwise > > > > indicated. > > > > > If > > > > > > > you > > > > > > > > > are > > > > > > > > > > > not > > > > > > > > > > > > > the intended recipient, please do not read, copy, > use > > > or > > > > > > > disclose > > > > > > > > > to > > > > > > > > > > > > others > > > > > > > > > > > > > this message or any attachment. Please also notify > > the > > > > > sender > > > > > > > by > > > > > > > > > > > replying > > > > > > > > > > > > > to this email or by telephone (+44(020 7896 0011) > and > > > > then > > > > > > > delete > > > > > > > > > the > > > > > > > > > > > > email > > > > > > > > > > > > > and any copies of it. Opinions, conclusion (etc) > that > > > do > > > > > not > > > > > > > > relate > > > > > > > > > > to > > > > > > > > > > > > the > > > > > > > > > > > > > official business of this company shall be > understood > > > as > > > > > > > neither > > > > > > > > > > given > > > > > > > > > > > > nor > > > > > > > > > > > > > endorsed by it. IG is a trading name of IG Markets > > > > Limited > > > > > (a > > > > > > > > > company > > > > > > > > > > > > > registered in England and Wales, company number > > > 04008957) > > > > > and > > > > > > > IG > > > > > > > > > > Index > > > > > > > > > > > > > Limited (a company registered in England and Wales, > > > > company > > > > > > > > number > > > > > > > > > > > > > 01190902). Registered address at Cannon Bridge > House, > > > 25 > > > > > > > Dowgate > > > > > > > > > > Hill, > > > > > > > > > > > > > London EC4R 2YA. Both IG Markets Limited (register > > > number > > > > > > > 195355) > > > > > > > > > and > > > > > > > > > > > IG > > > > > > > > > > > > > Index Limited (register number 114059) are > authorised > > > and > > > > > > > > regulated > > > > > > > > > > by > > > > > > > > > > > > the > > > > > > > > > > > > > Financial Conduct Authority. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > -Regards, > > > > > > > > > > > > Mayuresh R. Gharat > > > > > > > > > > > > (862) 250-7125 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > -Regards, > > > > > > > > > > Mayuresh R. Gharat > > > > > > > > > > (862) 250-7125 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Nacho - Ignacio Solis - iso...@igso.net > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > -Regards, > > > > > > Mayuresh R. Gharat > > > > > > (862) 250-7125 > > > > > > The information contained in this email is strictly confidential > > and > > > > for > > > > > > the use of the addressee only, unless otherwise indicated. If you > > are > > > > not > > > > > > the intended recipient, please do not read, copy, use or disclose > > to > > > > > others > > > > > > this message or any attachment. Please also notify the sender by > > > > replying > > > > > > to this email or by telephone (+44(020 7896 0011) and then delete > > the > > > > > email > > > > > > and any copies of it. Opinions, conclusion (etc) that do not > relate > > > to > > > > > the > > > > > > official business of this company shall be understood as neither > > > given > > > > > nor > > > > > > endorsed by it. IG is a trading name of IG Markets Limited (a > > company > > > > > > registered in England and Wales, company number 04008957) and IG > > > Index > > > > > > Limited (a company registered in England and Wales, company > number > > > > > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate > > > Hill, > > > > > > London EC4R 2YA. Both IG Markets Limited (register number 195355) > > and > > > > IG > > > > > > Index Limited (register number 114059) are authorised and > regulated > > > by > > > > > the > > > > > > Financial Conduct Authority. > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > -Regards, > > > > > Mayuresh R. Gharat > > > > > (862) 250-7125 > > > > > > > > > > > > > > > > > > > > > -- > > > -Regards, > > > Mayuresh R. Gharat > > > (862) 250-7125 > > > The information contained in this email is strictly confidential and > for > > > the use of the addressee only, unless otherwise indicated. If you are > not > > > the intended recipient, please do not read, copy, use or disclose to > > others > > > this message or any attachment. Please also notify the sender by > replying > > > to this email or by telephone (+44(020 7896 0011) and then delete the > > email > > > and any copies of it. Opinions, conclusion (etc) that do not relate to > > the > > > official business of this company shall be understood as neither given > > nor > > > endorsed by it. IG is a trading name of IG Markets Limited (a company > > > registered in England and Wales, company number 04008957) and IG Index > > > Limited (a company registered in England and Wales, company number > > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, > > > London EC4R 2YA. Both IG Markets Limited (register number 195355) and > IG > > > Index Limited (register number 114059) are authorised and regulated by > > the > > > Financial Conduct Authority. > > > > > The information contained in this email is strictly confidential and for > > the use of the addressee only, unless otherwise indicated. If you are not > > the intended recipient, please do not read, copy, use or disclose to > others > > this message or any attachment. Please also notify the sender by replying > > to this email or by telephone (+44(020 7896 0011) and then delete the > email > > and any copies of it. Opinions, conclusion (etc) that do not relate to > the > > official business of this company shall be understood as neither given > nor > > endorsed by it. IG is a trading name of IG Markets Limited (a company > > registered in England and Wales, company number 04008957) and IG Index > > Limited (a company registered in England and Wales, company number > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, > > London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG > > Index Limited (register number 114059) are authorised and regulated by > the > > Financial Conduct Authority. > > > > > > -- > -Regards, > Mayuresh R. Gharat > (862) 250-7125 >
