-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30599/
-----------------------------------------------------------
Review request for Ambari, Nate Cole and Tom Beerbower.
Bugs: AMBARI-9334
https://issues.apache.org/jira/browse/AMBARI-9334
Repository: ambari
Description
-------
StageDAO.findByCommandStatuses performs a cartesion product between
host_role_command and stage in order to determine if there are any stages that
are in progress (ie at least one of the stages's commands is in progress). This
was causing a major performance issue on Postgres, especially since it's run on
a thread every x seconds.
Because stage uses a compound primary key, there is no simple way to extract a
DISTINCT set of stages without doing the JOIN. It's also not feasible to simple
request the commands and extract the stage off of them for the following reason:
- There could be 1000's of commands in a large cluster
- The HostRoleCommandEntity binds lazily to its StageEntity
Adding a clustered index did increase performace by a decent amount on
Postgres, but oddly enough not on MySQL. I believe this is due to the fact that
on MySQL the primary key counts as the clustered index already and a table can
only have 1 clustered index.
At the end of the day, my solution does the following:
- Checks to see if any commands are in progress; if there are none, don't even
try the stage lookup.
- If there are commands that are in progress, use a slighly more efficient JPQL
query that optimizes the compiled SQL
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessor.java
1ef161a
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
4db1496
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionManager.java
c8ed235
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
9cc1075
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleStatus.java
4d38b06
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
dca8808
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/StageDAO.java
8b1cfb3
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java
8e08727
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
058fe9b
ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java
3c80b4e
Diff: https://reviews.apache.org/r/30599/diff/
Testing
-------
Some new tests written to cover the cases where there are many stages and many
commands.
mvn clean test
Thanks,
Jonathan Hurley
