> On March 13, 2013, 8:03 p.m., Thomas Marshall wrote: > > After applying this patch, I get the following output: > > > > > make check > > ... > > [ RUN ] SlaveRecoveryTest/0.ReconnectExecutor > > Checkpointing SlaveInfo to > > '/tmp/SlaveRecoveryTest_0_ReconnectExecutor_7r5qCw/meta/slaves/201303131255-1883310602-49297-1281-0/slave.info' > > Checkpointing FrameworkInfo to > > '/tmp/SlaveRecoveryTest_0_ReconnectExecutor_7r5qCw/meta/slaves/201303131255-1883310602-49297-1281-0/frameworks/201303131255-1883310602-49297-1281-0000/framework.info' > > Checkpointing 'scheduler(14)@10.10.65.112:49297' to > > /tmp/SlaveRecoveryTest_0_ReconnectExecutor_7r5qCw/meta/slaves/201303131255-1883310602-49297-1281-0/frameworks/201303131255-1883310602-49297-1281-0000/framework.pid > > Checkpointing ExecutorInfo to > > '/tmp/SlaveRecoveryTest_0_ReconnectExecutor_7r5qCw/meta/slaves/201303131255-1883310602-49297-1281-0/frameworks/201303131255-1883310602-49297-1281-0000/executors/88f6493c-9dd2-46b3-86b7-1f89d6b57ad3/executor.info' > > Checkpointing Task to > > '/tmp/SlaveRecoveryTest_0_ReconnectExecutor_7r5qCw/meta/slaves/201303131255-1883310602-49297-1281-0/frameworks/201303131255-1883310602-49297-1281-0000/executors/88f6493c-9dd2-46b3-86b7-1f89d6b57ad3/runs/96eb66d8-612e-47d2-8fec-7f0c980c0ced/tasks/88f6493c-9dd2-46b3-86b7-1f89d6b57ad3/task.info' > > Checkpointing forked pid 1334 > > Checkpointing '1334' to > > /tmp/SlaveRecoveryTest_0_ReconnectExecutor_7r5qCw/meta/slaves/201303131255-1883310602-49297-1281-0/frameworks/201303131255-1883310602-49297-1281-0000/executors/88f6493c-9dd2-46b3-86b7-1f89d6b57ad3/runs/96eb66d8-612e-47d2-8fec-7f0c980c0ced/pids/forked.pid > > Fetching resources into > > /tmp/SlaveRecoveryTest_0_ReconnectExecutor_7r5qCw/slaves/201303131255-1883310602-49297-1281-0/frameworks/201303131255-1883310602-49297-1281-0000/executors/88f6493c-9dd2-46b3-86b7-1f89d6b57ad3/runs/96eb66d8-612e-47d2-8fec-7f0c980c0ced > > Checkpointing 'executor(1)@10.10.65.112:49325' to > > /tmp/SlaveRecoveryTest_0_ReconnectExecutor_7r5qCw/meta/slaves/201303131255-1883310602-49297-1281-0/frameworks/201303131255-1883310602-49297-1281-0000/executors/88f6493c-9dd2-46b3-86b7-1f89d6b57ad3/runs/96eb66d8-612e-47d2-8fec-7f0c980c0ced/pids/libprocess.pid > > Registered executor on thomass-macbook-pro-4.local > > Starting task 88f6493c-9dd2-46b3-86b7-1f89d6b57ad3 > > sh -c 'sleep 1000' > > Re-registered executor on thomass-macbook-pro-4.local > > Waited on process 1350, returned status 15 > > [ OK ] SlaveRecoveryTest/0.ReconnectExecutor (138 ms) > > [ RUN ] SlaveRecoveryTest/0.RecoverUnregisteredExecutor > > Checkpointing SlaveInfo to > > '/tmp/SlaveRecoveryTest_0_RecoverUnregisteredExecutor_CuAC1i/meta/slaves/201303131255-1883310602-49297-1281-0/slave.info' > > Checkpointing FrameworkInfo to > > '/tmp/SlaveRecoveryTest_0_RecoverUnregisteredExecutor_CuAC1i/meta/slaves/201303131255-1883310602-49297-1281-0/frameworks/201303131255-1883310602-49297-1281-0000/framework.info' > > Checkpointing 'scheduler(15)@10.10.65.112:49297' to > > /tmp/SlaveRecoveryTest_0_RecoverUnregisteredExecutor_CuAC1i/meta/slaves/201303131255-1883310602-49297-1281-0/frameworks/201303131255-1883310602-49297-1281-0000/framework.pid > > Checkpointing ExecutorInfo to > > '/tmp/SlaveRecoveryTest_0_RecoverUnregisteredExecutor_CuAC1i/meta/slaves/201303131255-1883310602-49297-1281-0/frameworks/201303131255-1883310602-49297-1281-0000/executors/8fe83fa2-e257-4a2e-8436-50bd6704980e/executor.info' > > Checkpointing Task to > > '/tmp/SlaveRecoveryTest_0_RecoverUnregisteredExecutor_CuAC1i/meta/slaves/201303131255-1883310602-49297-1281-0/frameworks/201303131255-1883310602-49297-1281-0000/executors/8fe83fa2-e257-4a2e-8436-50bd6704980e/runs/f1ec30f0-f9ff-4868-b6f4-ef0ca3a43522/tasks/8fe83fa2-e257-4a2e-8436-50bd6704980e/task.info' > > Checkpointing forked pid 1351 > > Checkpointing '1351' to > > /tmp/SlaveRecoveryTest_0_RecoverUnregisteredExecutor_CuAC1i/meta/slaves/201303131255-1883310602-49297-1281-0/frameworks/201303131255-1883310602-49297-1281-0000/executors/8fe83fa2-e257-4a2e-8436-50bd6704980e/runs/f1ec30f0-f9ff-4868-b6f4-ef0ca3a43522/pids/forked.pid > > Fetching resources into > > /tmp/SlaveRecoveryTest_0_RecoverUnregisteredExecutor_CuAC1i/slaves/201303131255-1883310602-49297-1281-0/frameworks/201303131255-1883310602-49297-1281-0000/executors/8fe83fa2-e257-4a2e-8436-50bd6704980e/runs/f1ec30f0-f9ff-4868-b6f4-ef0ca3a43522 > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > ../../src/tests/slave_recovery_tests.cpp:634: Failure > > Failed > > Waited too long for 'statusUpdateCall' > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > dyld: DYLD_ environment variables being ignored because main executable > > (/bin/ps) is setuid or setgid > > ... > > > > > > The "dyld: DYLD_ environment variables being ignored because main > > executable (/bin/ps) is setuid or setgid" also occurs if, for example, if I > > run mesos-master.sh, mesos-slave.sh, and test-framework, the slave will > > output this error after the framework exits, and if I don't kill it almost > > immediately it crashes my entire machine. Again, running this on OSX > > Mountain Lion. > > Thomas Marshall wrote: > Additionally, updating to this patch doesn't seem to have any effect on > the original problem when running on Ubuntu.
Looks like, one of the bugs here is the fact that 'ps' on MacOSX 10.8 reports session id as "0" for every process on the system! This manifests as isolation module trying to kill all the processes (via "killtree.sh") when it tries to kill an executor! I have tested with a fix in killtree.sh, to guard against session ids being "0". This unbreaks the test. Still digging on the reason behind the 'dyld' warnings above! - Vinod ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9907/#review17822 ----------------------------------------------------------- On March 13, 2013, 7:13 p.m., Vinod Kone wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/9907/ > ----------------------------------------------------------- > > (Updated March 13, 2013, 7:13 p.m.) > > > Review request for mesos, Benjamin Hindman, Thomas Marshall, and Ben Mahler. > > > Description > ------- > > See summary. > > > Diffs > ----- > > src/tests/slave_recovery_tests.cpp 2a2fa281bba8e00e93a770828f5a93aff5539cf4 > > Diff: https://reviews.apache.org/r/9907/diff/ > > > Testing > ------- > > make check > > > Thanks, > > Vinod Kone > >
