> On Oct. 1, 2014, 5:42 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStore.java, 
> > line 140
> > <https://reviews.apache.org/r/26232/diff/1/?file=710141#file710141line140>
> >
> >     I'm not sure how i feel about this.  On one hand, it's nice that SQL 
> > handles a lot of the complexity.  On the other, it would be _really_ nice 
> > to see deleted job update UUIDs in the log.  Perhaps this function can 
> > return some details about the deleted updates?

I have refactored to return a set of IDs and log them in the pruner.


> On Oct. 1, 2014, 5:42 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPruner.java,
> >  line 63
> > <https://reviews.apache.org/r/26232/diff/1/?file=710139#file710139line63>
> >
> >     Nit: starting this with "Pruning completed" breaks my brain a little.  
> > How about "Pruning job update history."

Reworded.


> On Oct. 1, 2014, 5:42 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPruner.java,
> >  line 43
> > <https://reviews.apache.org/r/26232/diff/1/?file=710139#file710139line43>
> >
> >     maxUpdatesPerJob?

Renamed.


> On Oct. 1, 2014, 5:42 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java, line 324
> > <https://reviews.apache.org/r/26232/diff/1/?file=710137#file710137line324>
> >
> >     Does this work?  I think something needs to 'pull' on the binding (by 
> > injecting) for anything to happen.

Not really. Fixed.


> On Oct. 1, 2014, 5:42 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java, line 161
> > <https://reviews.apache.org/r/26232/diff/1/?file=710137#file710137line161>
> >
> >     We also need time-based retention.  How about 1 month by default?
> 
> Maxim Khutornenko wrote:
>     Thought about that but was not sure the extra complexity is worth it. 
> Also, don't we want to show update history in the UI regardless of how old it 
> is?
> 
> Bill Farner wrote:
>     Given that it currently needs to fit in memory, i think not.
>     
>     For example: if i create a job to test something out, then kill it and 
> never return.  Probably shouldn't retain those updates forever.
> 
> David McLaughlin wrote:
>     Wouldn't the updates in that case be killed by the job garbage 
> collection/cascading deletion of the job key?
> 
> Bill Farner wrote:
>     If so, that warrants validation in an integration test between 
> TaskHistoryPruner, storage, and DbJobUpdateStore.
> 
> Maxim Khutornenko wrote:
>     We don't cascade deletes into the job_keys table. In fact, we don't 
> actually delete from job_keys at all at this point. We should probably 
> address this when converting TaskStore into H2.
>     
>     | if i create a job to test something out, then kill it and never return. 
>  Probably shouldn't retain those updates forever.
>     Good point. This alone warrants adding time-based cleanup.

Added support for time-based pruning.


- Maxim


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26232/#review55088
-----------------------------------------------------------


On Oct. 1, 2014, 5:26 p.m., Maxim Khutornenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26232/
> -----------------------------------------------------------
> 
> (Updated Oct. 1, 2014, 5:26 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin and Bill Farner.
> 
> 
> Bugs: AURORA-743
>     https://issues.apache.org/jira/browse/AURORA-743
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> The pruner runs on periodic basis and trims completed updates up to the 
> guranteed per job retention count.
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java 
> aa45d27391b1786ca3d5e8c928045f1b6f3cf5ef 
>   src/main/java/org/apache/aurora/scheduler/async/HistoryPruner.java 
> ebae58a04e8857c5f26d4b57c27dfcda9e14c82c 
>   src/main/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPruner.java 
> PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/storage/JobUpdateStore.java 
> c3abffe575e801cebec3572cf4aceac83a238b55 
>   src/main/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStore.java 
> 3db0114c5432a5b7d0b01d97c75494be9e3e99a3 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.java
>  c583e085e0458835d51ebf740a3b5f01b428bb25 
>   
> src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java 
> 66c91644677e7176ccf53dcfcf29a6792ec398bc 
>   
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml
>  631ab2543dadb77b7fd1cae96adc9c0bd2f94b39 
>   src/test/java/org/apache/aurora/scheduler/async/HistoryPrunerTest.java 
> 011d9ec5d84c658c2b2e39bb0b3f9d20b5440cee 
>   
> src/test/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPrunerTest.java
>  PRE-CREATION 
>   
> src/test/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStoreTest.java
>  1b1160f349d64bbcd4d20103a82f3b8eb1ca75d9 
> 
> Diff: https://reviews.apache.org/r/26232/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew -Pq build
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Reply via email to