----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/8965/#review15406 -----------------------------------------------------------
Any line limit rules for Java? I've seen 100 and 120 as rules in the past. hadoop/NOTES <https://reviews.apache.org/r/8965/#comment33178> s/JAVA/Java hadoop/TUTORIAL.sh <https://reviews.apache.org/r/8965/#comment33179> Reword: Build mesos executor package to be downloaded and executed on mesos slaves. hadoop/TUTORIAL.sh <https://reviews.apache.org/r/8965/#comment33182> $ rm -rf cloudera # Only for cdh3. hadoop/TUTORIAL.sh <https://reviews.apache.org/r/8965/#comment33180> end with a period hadoop/TUTORIAL.sh <https://reviews.apache.org/r/8965/#comment33181> s/spacei/space Also mention that this is only for cdh3 hadoop/TUTORIAL.sh <https://reviews.apache.org/r/8965/#comment33183> end with period hadoop/TUTORIAL.sh <https://reviews.apache.org/r/8965/#comment33206> We should include a comment with respect to the slaves needing to have >= 5 cores, >= X RAM, >= Y DISK. And how they can change that by reducing the slots. Even with reducing, we still will require 3 cores. This might be problematic for people. So a comment here and/or in our config patch would be great. hadoop/mapred-site.xml.patch <https://reviews.apache.org/r/8965/#comment33212> what is this file:///? Add an explanation? hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java <https://reviews.apache.org/r/8965/#comment33227> Can you kill this member variable? hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java <https://reviews.apache.org/r/8965/#comment33218> argument formatting hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java <https://reviews.apache.org/r/8965/#comment33217> Can you add a log line to each callback? namely: registered, reregistered, disconnected, frameworkMessage, shutdown This should prove useful for debugging. hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java <https://reviews.apache.org/r/8965/#comment33219> fits on one line? hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java <https://reviews.apache.org/r/8965/#comment33220> Can you add a comment as to why we need this? hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java <https://reviews.apache.org/r/8965/#comment33221> LOG calls take a throwable as a second argument, so replace with: LOG.fatal("Failed to start task tracker", e); // Kill the e.printStackTrace(); hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java <https://reviews.apache.org/r/8965/#comment33222> ditto hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java <https://reviews.apache.org/r/8965/#comment33223> Capitalize: TaskTracker Run Thread hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java <https://reviews.apache.org/r/8965/#comment33224> Inline the @Override for anonymous classes @Override public void run() { } hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java <https://reviews.apache.org/r/8965/#comment33225> Strange line wrapping. hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java <https://reviews.apache.org/r/8965/#comment33226> Pass e as an argument instead, and use "Failed to sleep" style string. hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java <https://reviews.apache.org/r/8965/#comment33228> ditto from above, kill the member variable and just do: new Thread() { ... }.start(); hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java <https://reviews.apache.org/r/8965/#comment33229> Ditto for passing e. hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java <https://reviews.apache.org/r/8965/#comment33230> Ditto for passing e. hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/8965/#comment33231> This never gets set to true, is it needed? hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/8965/#comment33233> Revise my TODO: s/to run a TaskTracker/to run a default (4 slot) TaskTracker hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/8965/#comment33234> Maybe just say "corresponding idle task trackers." instead? hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/8965/#comment33235> I think this 'if' statement no longer makes sense, can you remove it? hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/8965/#comment33236> This log message doesn't make sense, given the status task ID is completed, not pending.. hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/8965/#comment33237> How is this different than not setting the user? hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/8965/#comment33240> Do we need the identifier? hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/8965/#comment33232> running is always false here hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/8965/#comment33242> Can you update my TODO here to be just: TODO(bmahler): Consider allowing non-mesos TaskTrackers. I'm pretty sure the second sentence is no longer applicable, since I refactored. hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/8965/#comment33244> What a beautiful, informative, and eloquent log message!! I wonder who added this ;) hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/8965/#comment33245> Kill my TODO here, I ended up fixing this. hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/8965/#comment33246> Can you move this try/catch into the if local block? Also, pass e as an argument and use "Failed to find.." - Ben Mahler On Jan. 15, 2013, 10:29 p.m., Vinod Kone wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/8965/ > ----------------------------------------------------------- > > (Updated Jan. 15, 2013, 10:29 p.m.) > > > Review request for mesos, Benjamin Hindman, Matei Zaharia, and Ben Mahler. > > > Description > ------- > > Features: > > --> Mesos scheduler is a thin shim that delegates the actual scheduling of > task trackers to the underlying scheduler. > --> Mesos executor is a thin shim that starts/stops task tracker properly. > --> Doesn't depend on mesos specific patches (status updates, task tracker > instrumentation) for hadoop. > --> Builds mesos executor bundle that can be used when running on a real > mesos cluster. > > > Diffs > ----- > > hadoop/Makefile.am 46d16f7c03cda217d1851d78515cad96f60cba2b > hadoop/NOTES PRE-CREATION > hadoop/TUTORIAL.sh fca22c5b7bd485a5d18c99cb6666fab6bb0a06bb > hadoop/hadoop-0.20.2-cdh3u3.patch 84a948f23122a25fb41dfe497a73bff0c8e0867d > hadoop/hadoop-0.20.2-cdh3u3_hadoop-env.sh.patch > b1731e27b1f59d8dc913d259d8f32a69f46ac0b0 > hadoop/hadoop-0.20.2-cdh3u3_mesos.patch > f93e53e286525829bdc2a3da71eefab8f093e5b8 > hadoop/hadoop-0.20.205.0.patch cab488e402343bdbd5d0c4101785b9a48331f85e > hadoop/hadoop-0.20.205.0_hadoop-env.sh.patch > 1b8f4d06323a3ca3f624d9374066e29be5a18acd > hadoop/mapred-site.xml.patch c2d6397d8fc1d662afd606e1069ff469c3556e19 > hadoop/mesos-executor ce8dcccf7eaecd35dfa041a78aa9760dc2ce451c > hadoop/mesos/src/java/org/apache/hadoop/mapred/FrameworkExecutor.java > 9238fe53a718c47ff4bbe2d555d1c072fa8bd6e2 > hadoop/mesos/src/java/org/apache/hadoop/mapred/FrameworkScheduler.java > 754b198a5c7baacc92ce2adc96da15aa97c47ad1 > hadoop/mesos/src/java/org/apache/hadoop/mapred/HadoopFrameworkMessage.java > 1e9510873bab962250afbaf81a5c63a6b191e299 > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java > PRE-CREATION > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java > 1408f300a43eab043b40e4d313e6632cb21fd7b2 > > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosTaskTrackerInstrumentation.java > 8c517d80e54555e12f906a219b12eb572ed7e3a3 > > Diff: https://reviews.apache.org/r/8965/diff/ > > > Testing > ------- > > make hadoop-0.20.2-cdh3u3 > make hadoop-0.20.205.0 > > Also, verified that the generator executor bundle is self-contained and works > as expected for non-local runs. > > > Thanks, > > Vinod Kone > >
