> On Aug. 30, 2016, 12:37 a.m., Stephan Erb wrote:
> > Were you able to test this on a cluster with concurrent modifications? 
> > Right now, I am not confident this is working as most ouf our thrift 
> > responses contain unordered sequences. I would therefore suspect that we 
> > get a different hash each time, even though the content itself has not 
> > changed.
> > 
> > Considering it is working in some cases, do we have to do something to get 
> > some speedups for the Aurora UI as well?
> 
> Zameer Manji wrote:
>     Could you describe what you mean by 'concurrent modifications'?
>     
>     I understand your concern that it doesn't work, but empirical interaction 
> in vagrant via `curl` shows the same ETag is set for the same data. I will 
> investigate further and construct a test case that demonstrates this.
>     
>     I'm not sure how the UI works, but yes we would need to do something to 
> store the ETag in a response and send that back if we want a `304` response.
> 
> Stephan Erb wrote:
>     I can imagine that whenever we modify our in-memory task store in a way 
> triggers a resizing of its unordered data structures, the result might be 
> ordered differently.
>     
>     For example, when we cache a call `getTasksWithoutConfig(myQuery)` 
> returning 100 instances, the order of instances might change if we schedule 
> 1000 completely unrelated instances not even matched by our query.
> 
> Zameer Manji wrote:
>     You are correct, in some cases a write unrelated to the data we are 
> trying to retrieve will uncessarily alter the ETag. However in cases where 
> there has been no write the ETag is constant.
>     
>     I guess this shows there is a flaw in this hashing method, however I'm 
> unaware of another method of computing an ETag without altering storage or 
> changing how we use thrift.
>     
>     Do you think these flaws hinder this change?

Actually after some experimentation, I have eliminated most of these instances 
where the ETag changes by having the API layer sort the results by the natural 
comparator (provided by thrift generated code).

For example in `getTasksWithoutConfigs` if I sort the tasks, then the Etag is 
constant between reboots and other writes done.

I'm willing to write e2e tests that ensure this behaviour if that's what's 
required.


- Zameer


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51513/#review147270
-----------------------------------------------------------


On Aug. 29, 2016, 6:12 p.m., Zameer Manji wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51513/
> -----------------------------------------------------------
> 
> (Updated Aug. 29, 2016, 6:12 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Stephan Erb.
> 
> 
> Bugs: AURORA-1757
>     https://issues.apache.org/jira/browse/AURORA-1757
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This patch enhances the Aurora API by producing an `ETag` header for each API
> request. It also consumes etag values in API requests via the `If-None-Match`
> header and produces a HTTP 304 if the request would produce a body with the 
> same
> etag value.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md 1819eaa20cf5014228643a1e120316d646cc2824 
>   
> src/main/java/org/apache/aurora/scheduler/http/api/TContentAwareServlet.java 
> 1634cb88ac09c778c5bb277ca902f4ca35dd6c9d 
>   src/test/java/org/apache/aurora/scheduler/http/api/ApiIT.java 
> 0a3ff05586c87e0ab2cc20470e99b5dd609f7039 
> 
> Diff: https://reviews.apache.org/r/51513/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Zameer Manji
> 
>

Reply via email to