Bill Farner created AURORA-116:
----------------------------------
Summary: Improve efficiency of saving host attributes (or avoid
saving host attributes)
Key: AURORA-116
URL: https://issues.apache.org/jira/browse/AURORA-116
Project: Aurora
Issue Type: Task
Components: Scheduler
Reporter: Bill Farner
Priority: Critical
The scheduler performs multiple write operations for every resource offer, to
save slave attributes:
{noformat}
public void resourceOffers(SchedulerDriver driver, List<Offer> offers) {
Preconditions.checkState(registered, "Must be registered before receiving
offers.");
for (final Offer offer : offers) {
log(Level.FINE, "Received offer: %s", offer);
resourceOffers.incrementAndGet();
storage.write(new MutateWork.NoResult.Quiet() {
@Override protected void execute(MutableStoreProvider storeProvider) {
storeProvider.getAttributeStore().saveHostAttributes(Conversions.getAttributes(offer));
}
});
{noformat}
This can unnecessarily block the singly-threaded message dispatch in the
scheduler driver. An incremental improvement would be to aggregate all slave
info and save it in one write operation. Better yet would be to perform writes
asynchronously (taking care to not break task scheduling, since attributes are
expected to be present). Even better yet, it would be great to determine if we
can avoid storing host attributes.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
