[
https://issues.apache.org/jira/browse/KAFKA-9685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manikumar resolved KAFKA-9685.
------------------------------
Fix Version/s: 2.6.0
Resolution: Fixed
Issue resolved by pull request 8261
[https://github.com/apache/kafka/pull/8261]
> Solve Set concatenation perf issue in AclAuthorizer
> ---------------------------------------------------
>
> Key: KAFKA-9685
> URL: https://issues.apache.org/jira/browse/KAFKA-9685
> Project: Kafka
> Issue Type: Improvement
> Components: security
> Affects Versions: 1.1.0
> Reporter: Jiao Zhang
> Priority: Minor
> Fix For: 2.6.0
>
>
> In version 1.1,
> [https://github.com/apache/kafka/blob/71b1e19fc60b5e1f9bba33025737ec2b7fb1c2aa/core/src/main/scala/kafka/security/auth/SimpleAclAuthorizer.scala#L110]
> the logic for checking acls is preparing a merged acl Set with
> {code:java}
> acls = getAcls(new Resource(resource.resourceType,
> Resource.WildCardResource)) ++ getAcls(resource);{code}
> and then pass it as aclMatch's parameter.
> We found scala's Set ++ operation is very slow for example in the case that
> the Set on right hand of ++ has more than 100 entries.
> And the bad performance of ++ is due to iterating every entry of the Set on
> right hand of ++, in which the calculation of HashCode seems heavy.
> The performance of 'authorize' is important as each request delivered to
> broker goes through the logic, that's the reason we can't leave it as-is
> although the change for this proposal seems trivial.
> Here is the approach. We propose to solve this issue by introducing a new
> class 'AclSets' which takes multiple Sets as parameters and do 'find' against
> them one by one.
> {code:java}
> class AclSets(sets: Set[Acl]*){
> def find(p: Acl => Boolean): Option[Acl] =
> sets.flatMap(_.find(p)).headOption
> def isEmpty: Boolean = !sets.exists(_.nonEmpty)
> }
> {code}
> This approach avoid the Set ++ operation like following,
> {code:java}
> val acls = new AclSets(getAcls(new Resource(resource.resourceType,
> Resource.WildCardResource)), getAcls(resource)){code}
> and thus outperforms a lot compared to old logic.
> The benchmark result(we did the test with kafka version 1.1) shows notable
> difference under the condition:
> 1. set on left consists of 60 entries
> 2. set of right consists of 30 entries
> 3. search for absent entry (so that all entries are iterated)
> Benchmark Results is as following.
> Mode Cnt Score
> Error Units
> ScalaSetConcatination.Set thrpt 3 281.974 ± 140.029 ops/ms
> ScalaSetConcatination.AclSets thrpt 3 887.426 ± 40.261 ops/ms
> As the upstream also use the similar ++ operation,
> [https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/security/authorizer/AclAuthorizer.scala#L360]
> we think it's necessary to fix this issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
