[ https://issues.apache.org/jira/browse/YARN-6426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Junping Du updated YARN-6426: ----------------------------- Target Version/s: 2.9.0, 3.0.0-beta1, 2.8.3 (was: 2.9.0, 2.8.1, 3.0.0-beta1) > Compress ZK YARN keys to scale up (especially AppStateData > ---------------------------------------------------------- > > Key: YARN-6426 > URL: https://issues.apache.org/jira/browse/YARN-6426 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn > Affects Versions: 3.0.0-alpha2 > Reporter: Roni Burd > Assignee: Roni Burd > Labels: patch > Attachments: zkcompression.patch > > > ZK today stores the protobuf files uncompressed. This is not an issue except > that if a customer job has thousands of files, AppStateData will store the > user context as a string with multiple URLs and it is easy to get to 1MB or > more. > This can put unnecessary strain on ZK and make the process slow. > The proposal is to simply compress protobufs before sending them to ZK -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org