[jira] [Updated] (ATLAS-2064) Compress Hook Messages Posted to Kafka Atlas Topic
[ https://issues.apache.org/jira/browse/ATLAS-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Madhan Neethiraj updated ATLAS-2064: Fix Version/s: (was: trunk) > Compress Hook Messages Posted to Kafka Atlas Topic > -- > > Key: ATLAS-2064 > URL: https://issues.apache.org/jira/browse/ATLAS-2064 > Project: Atlas > Issue Type: Improvement > Components: atlas-intg >Affects Versions: 0.8-incubating >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Attachments: ATLAS-2064-compressed-hook-message-envelope.patch > > > *Background* > Messages posted by hooks to Atlas Kafka topics are JSON format. > Kafka imposes a 1MB limit on the message size. > Occasionally, depending on operations performed, this threshold is reached. > This results in messages being dropped. > The entities are thus not reflected in Atlas. > *Solution* > Applying compression to these message will alleviate the problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2064) Compress Hook Messages Posted to Kafka Atlas Topic
[ https://issues.apache.org/jira/browse/ATLAS-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Mestry updated ATLAS-2064: --- Summary: Compress Hook Messages Posted to Kafka Atlas Topic (was: Compress Hook Messages Posted to Kafka) > Compress Hook Messages Posted to Kafka Atlas Topic > -- > > Key: ATLAS-2064 > URL: https://issues.apache.org/jira/browse/ATLAS-2064 > Project: Atlas > Issue Type: Improvement > Components: atlas-intg >Affects Versions: 0.8-incubating >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry > Fix For: trunk > > Attachments: ATLAS-2064-compressed-hook-message-envelope.patch > > > *Background* > Messages posted by hooks to Atlas Kafka topics are JSON format. > Kafka imposes a 1MB limit on the message size. > Occasionally, depending on operations performed, this threshold is reached. > This results in messages being dropped. > The entities are thus not reflected in Atlas. > *Solution* > Applying compression to these message will alleviate the problem. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
