[
https://issues.apache.org/jira/browse/IMPALA-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039337#comment-17039337
]
Csaba Ringhofer commented on IMPALA-9387:
-----------------------------------------
I tried to profile the Ranger process:
{code}
JvmTop 0.8.0 alpha - 19:28:10, amd64, 4 cpus, Linux 4.4.0-171, load avg 1.89
http://code.google.com/p/jvmtop
Profiling PID 19422: org.apache.ranger.server.tomcat.Embedded
47.07% ( 4.66s) org.springframework.cglib.proxy.MethodProxy.invoke()
27.23% ( 2.70s) org.postgresql.core.VisibleBufferedInputStream.readMore()
8.67% ( 0.86s) org.postgresql.core.PGStream.flush()
5.82% ( 0.58s) ....mchange.v2.c3p0.impl.NewProxyConnection.prepareState()
4.52% ( 0.45s) ....springframework.orm.jpa.SharedEntityManagerCreator$S()
2.88% ( 0.29s) ....mchange.v2.c3p0.impl.NewPooledConnection.markActiveR()
1.18% ( 0.12s) ....google.gson.internal.bind.ReflectiveTypeAdapterFacto()
1.07% ( 0.11s) org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom()
0.57% ( 0.06s) ....codehaus.jackson.map.ser.BeanPropertyWriter.serializ()
0.49% ( 0.05s) ....springframework.beans.factory.support.AbstractBeanFa()
0.49% ( 0.05s) ....mchange.v2.async.ThreadPoolAsynchronousRunner.postRu()
{code}
Postgres seems to be involved and this makes sense - if Ranger updates the
database for every grant/revoke, then db access can become the bottleneck.
io top also mentioned Ranger during the test:
{code}
5510 be/4 postgres 54.35 K/s 326.10 K/s 0.00 % 0.24 % postgres: hiveuser
ranger_home_csringhofer_impala2 127.0.0.1(58798) idle in transaction
{code}
> AuthorizationStmtTest is very slow
> ----------------------------------
>
> Key: IMPALA-9387
> URL: https://issues.apache.org/jira/browse/IMPALA-9387
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Csaba Ringhofer
> Priority: Major
> Attachments: AuthorizationStmtTest-Ranger-CPU-Sample.png,
> AuthorizationStmtTest-Ranger-cpu-profile.nps
>
>
> AuthorizationStmtTest takes around a hour to run in the jenkines runs I
> recently saw:
> 02:54:32.772 [INFO] Tests run: 58, Failures: 0, Errors: 0, Skipped: 0, Time
> elapsed: 3,436.751 s - in
> org.apache.impala.authorization.AuthorizationStmtTest
> This is extremely slow compared to other FE tests. I ran these tests with
> Sentry several times in the recent past, and they were below 1 min, so I
> think that Ranger tests are the culprit here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
