Re: Review Request 47373: Batching explicit task reconciliation calls

[Aurora ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47373/#review133387
---



Master (485da81) is red with this patch.
  ./build-support/jenkins/build.sh

   with temporary_dir() as checkpoint_root:
 te = AuroraExecutor(
 >   
runner_provider=make_provider(checkpoint_root),
 
sandbox_provider=DefaultTestSandboxProvider())
 
 
src/test/python/apache/aurora/executor/test_thermos_executor.py:580: 
 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
 
src/test/python/apache/aurora/executor/test_thermos_executor.py:193: in 
make_provider
 pex_location=thermos_runner_path(),
 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
 
 build = True
 
 def thermos_runner_path(build=True):
   if not build:
 return getattr(thermos_runner_path, 'value', 
None)
 
   if not hasattr(thermos_runner_path, 'value'):
 pex_dir = safe_mkdtemp()
 >   assert subprocess.call(["./pants", 
"--pants-distdir=%s" % pex_dir, "binary",
   
"src/main/python/apache/thermos/runner:thermos_runner"]) == 0
 E   assert 1 == 0
 E+  where 1 = (['./pants', '--pants-distdir=/tmp/user/10021/tmp3nI6ot', 
'binary', 'src/main/python/apache/thermos/runner:thermos_runner'])
 E+where  = subprocess.call
 
 
src/test/python/apache/aurora/executor/test_thermos_executor.py:185: 
AssertionError
 -- Captured stderr call --
 Traceback (most recent call last):
   File 
"/home/jenkins/.cache/pants/setup/bootstrap-Linux-x86_64/0.0.80/bin/pants", 
line 7, in 
 from pants.bin.pants_exe import main
 ImportError: No module named pants.bin.pants_exe
  generated xml file: 
/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/test-results/415337499eb72578eab327a6487c1f5c9452b3d6.xml
 
  16 failed, 625 passed, 6 skipped, 1 warnings, 8 
error in 116.80 seconds 
 
FAILURE


   Waiting for background workers to finish.
17:30:39 02:47   [complete]
   FAILURE


I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On May 16, 2016, 5:13 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47373/
> ---
> 
> (Updated May 16, 2016, 5:13 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Stephan Erb.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This helps alleviate master task reconciliation pressure in large clusters 
> that _may_ lead to unresponsiveness and subsequent loss of slaves given a 
> combination of other load factors.
> 
> Default values are chosen somewhat randomly but have been battle tested. 
> Happy to consider other suggestions.
> 
> 
> Diffs
> -
> 
>   
> src/main/java/org/apache/aurora/scheduler/reconciliation/ReconciliationModule.java
>  cccee083fc2e5f204c91a9d397beb451d4d9df40 
>   
> src/main/java/org/apache/aurora/scheduler/reconciliation/TaskReconciler.java 
> 57d20618fe9eda076936d159172d5e133ec6597b 
>   
> src/test/java/org/apache/aurora/scheduler/reconciliation/TaskReconcilerTest.java
>  5b4b3ac9781c9b572bbc9507c238dfa3b0177b45 
>   
> src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java 
> 9082a31d4d2ff454775f5160a474e31261628929 
> 
> Diff: https://reviews.apache.org/r/47373/diff/
> 
> 
> Testing
> ---
> 
> ./gradlew -Pq build
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 47373: Batching explicit task reconciliation calls

[Maxim Khutornenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47373/
---

(Updated May 16, 2016, 5:13 p.m.)


Review request for Aurora, Joshua Cohen and Stephan Erb.


Changes
---

Stephan's comments.


Repository: aurora


Description
---

This helps alleviate master task reconciliation pressure in large clusters that 
_may_ lead to unresponsiveness and subsequent loss of slaves given a 
combination of other load factors.

Default values are chosen somewhat randomly but have been battle tested. Happy 
to consider other suggestions.


Diffs (updated)
-

  
src/main/java/org/apache/aurora/scheduler/reconciliation/ReconciliationModule.java
 cccee083fc2e5f204c91a9d397beb451d4d9df40 
  src/main/java/org/apache/aurora/scheduler/reconciliation/TaskReconciler.java 
57d20618fe9eda076936d159172d5e133ec6597b 
  
src/test/java/org/apache/aurora/scheduler/reconciliation/TaskReconcilerTest.java
 5b4b3ac9781c9b572bbc9507c238dfa3b0177b45 
  src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java 
9082a31d4d2ff454775f5160a474e31261628929 

Diff: https://reviews.apache.org/r/47373/diff/


Testing
---

./gradlew -Pq build


Thanks,

Maxim Khutornenko

Re: Review Request 47373: Batching explicit task reconciliation calls

[Joshua Cohen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47373/#review133359
---


Ship it!




Ship It!

- Joshua Cohen


On May 13, 2016, 11:23 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47373/
> ---
> 
> (Updated May 13, 2016, 11:23 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Stephan Erb.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This helps alleviate master task reconciliation pressure in large clusters 
> that _may_ lead to unresponsiveness and subsequent loss of slaves given a 
> combination of other load factors.
> 
> Default values are chosen somewhat randomly but have been battle tested. 
> Happy to consider other suggestions.
> 
> 
> Diffs
> -
> 
>   
> src/main/java/org/apache/aurora/scheduler/reconciliation/ReconciliationModule.java
>  cccee083fc2e5f204c91a9d397beb451d4d9df40 
>   
> src/main/java/org/apache/aurora/scheduler/reconciliation/TaskReconciler.java 
> 57d20618fe9eda076936d159172d5e133ec6597b 
>   
> src/test/java/org/apache/aurora/scheduler/reconciliation/TaskReconcilerTest.java
>  5b4b3ac9781c9b572bbc9507c238dfa3b0177b45 
>   
> src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java 
> 9082a31d4d2ff454775f5160a474e31261628929 
> 
> Diff: https://reviews.apache.org/r/47373/diff/
> 
> 
> Testing
> ---
> 
> ./gradlew -Pq build
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 47373: Batching explicit task reconciliation calls

[Stephan Erb]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47373/#review133271
---


Ship it!




Looks good to me!


src/main/java/org/apache/aurora/scheduler/reconciliation/TaskReconciler.java 
(line 138)


Is that really copy necessary?


- Stephan Erb


On Mai 14, 2016, 1:23 vorm., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47373/
> ---
> 
> (Updated Mai 14, 2016, 1:23 vorm.)
> 
> 
> Review request for Aurora, Joshua Cohen and Stephan Erb.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This helps alleviate master task reconciliation pressure in large clusters 
> that _may_ lead to unresponsiveness and subsequent loss of slaves given a 
> combination of other load factors.
> 
> Default values are chosen somewhat randomly but have been battle tested. 
> Happy to consider other suggestions.
> 
> 
> Diffs
> -
> 
>   
> src/main/java/org/apache/aurora/scheduler/reconciliation/ReconciliationModule.java
>  cccee083fc2e5f204c91a9d397beb451d4d9df40 
>   
> src/main/java/org/apache/aurora/scheduler/reconciliation/TaskReconciler.java 
> 57d20618fe9eda076936d159172d5e133ec6597b 
>   
> src/test/java/org/apache/aurora/scheduler/reconciliation/TaskReconcilerTest.java
>  5b4b3ac9781c9b572bbc9507c238dfa3b0177b45 
>   
> src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java 
> 9082a31d4d2ff454775f5160a474e31261628929 
> 
> Diff: https://reviews.apache.org/r/47373/diff/
> 
> 
> Testing
> ---
> 
> ./gradlew -Pq build
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 47373: Batching explicit task reconciliation calls

[Aurora ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47373/#review133230
---


Ship it!




Master (485da81) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On May 13, 2016, 11:23 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47373/
> ---
> 
> (Updated May 13, 2016, 11:23 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Stephan Erb.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This helps alleviate master task reconciliation pressure in large clusters 
> that _may_ lead to unresponsiveness and subsequent loss of slaves given a 
> combination of other load factors.
> 
> Default values are chosen somewhat randomly but have been battle tested. 
> Happy to consider other suggestions.
> 
> 
> Diffs
> -
> 
>   
> src/main/java/org/apache/aurora/scheduler/reconciliation/ReconciliationModule.java
>  cccee083fc2e5f204c91a9d397beb451d4d9df40 
>   
> src/main/java/org/apache/aurora/scheduler/reconciliation/TaskReconciler.java 
> 57d20618fe9eda076936d159172d5e133ec6597b 
>   
> src/test/java/org/apache/aurora/scheduler/reconciliation/TaskReconcilerTest.java
>  5b4b3ac9781c9b572bbc9507c238dfa3b0177b45 
>   
> src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java 
> 9082a31d4d2ff454775f5160a474e31261628929 
> 
> Diff: https://reviews.apache.org/r/47373/diff/
> 
> 
> Testing
> ---
> 
> ./gradlew -Pq build
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>