[
https://issues.apache.org/jira/browse/AURORA-137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15679560#comment-15679560
]
Bill Farner commented on AURORA-137:
------------------------------------
The actual number of write activity for attributes would be useful data to
decide whether {{BatchWorker}} is useful here. Echoing one of Stephan's points
- {{DbAttributeStore.saveHostAttributes()}} _should_ only be triggering a write
when attributes have actually changed, so i'd expect attribute log entries to
rapidly quiesce.
Relevant snippet from {{DbAttributeStore. saveHostAttributes()}}:
{code}
Optional<IHostAttributes> existing =
getHostAttributes(hostAttributes.getHost());
if (existing.equals(Optional.of(hostAttributes))) {
return false;
...
{code}
I think my original goal with this ticket was to eliminate the call to
{{storage.write()}} when handling {{resourceOffers()}}, and instead store host
attributes when a scheduling match is found (and we're already in a write
transaction). This would satisfy the current purpose of host attributes
(satisfying diversity constraints) while avoiding holding a highly-contended
lock unnecessarily.
{quote}
have noticed contention on storage write lock
{quote}
[~mnurolahzade] is there any data you can share on this? Presumably the
existing {{scheduler_resource_offers_*}} no longer give insight, as the method
body is asynchronous. Perhaps another low-effort action item is to extract the
synchronous stage of that method so that the {{@Timed}} stats are meaningful.
> Save host attributes only when a task is being scheduled
> --------------------------------------------------------
>
> Key: AURORA-137
> URL: https://issues.apache.org/jira/browse/AURORA-137
> Project: Aurora
> Issue Type: Story
> Components: Scheduler
> Reporter: Bill Farner
> Priority: Minor
>
> The scheduler currently aggressively saves host attributes when handling
> {{resourceOffers}}, however it seems tractable for this to only happen when a
> task is actually scheduled. Context: the scheduler stores host attributes to
> satisfy scheduling constraints (like host/rack diversity). Doing this would
> allow us to avoid waiting for the storage write lock, and handle
> {{resourceOffers}} in a more deterministic time frame.
> One caveat with this approach is that the Offer would need to be plumbed into
> {{SchedulingFilterImpl}} in a way so as to ensure that the attributes are
> available for the offer being inspected. In other words, we need to avoid
> the chicken and egg of trying to read the attributes for a host when this is
> the first offer ever received for it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
