Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-19 Thread Aurora ReviewBot

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/#review201579
---


Ship it!




Master (ad0bc5f) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On April 19, 2018, 9:49 p.m., Sameer Brenn wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66697/
> ---
> 
> (Updated April 19, 2018, 9:49 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> My team has some scripts to start devel shards which create tunnels:
> 
> ```
> aurora task ssh -L 8002:http --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> aurora task ssh -L 9002:health --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> ```
> 
> We use fixed local port numbers because that way we can run dependent 
> services locally that look for locally-running copies of the
> same service on a fixed port, but then those requests get tunnelled through 
> to the devel shard.
> 
> When the devel shard is restarted, however, the tunnel is still running so 
> the subsequent call to create a new tunnel fails because
> it can't bind to the fixed port.
> 
> If we save the SSH process PID to a file, we can then kill existing tunnel to 
> the old instance before starting up the new tunnel to the
> new instance.
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/cli/task.py 
> 652a545072f161dbf854b3d6d273809b09d142e8 
>   src/test/python/apache/aurora/client/cli/test_task.py 
> a543d4a101c58149f8af265257d061ff5032049c 
> 
> 
> Diff: https://reviews.apache.org/r/66697/diff/5/
> 
> 
> Testing
> ---
> 
> ```
> $ ./pants test src/test/python/apache/aurora/client::
> ```
> 
> And when applying the same patch to our local repo at Twitter:
> 
> ```
> $ ./pants run 
> twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
> task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
> "smf1/sbrenn/devel/proxyapp/0" &
> $ ps -p `cat /tmp/p`
>   PID TTY   TIME CMD
> 34729 ttys0000:00.05 ssh -t -n -N -L 
> 8005:smf1-aki-27-sr1.prod.twitter.com:31794 
> sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
> /var/lib/mesos/slaves/*/frameworks/*/exec
> ```
> 
> 
> Thanks,
> 
> Sameer Brenn
> 
>



Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-19 Thread Sameer Brenn via Review Board


> On April 19, 2018, 8:52 p.m., Santhosh Kumar Shanmugham wrote:
> > @ReviewBot retry
> 
> Santhosh Kumar Shanmugham wrote:
> Actually, looks like some style issues.

Oops sorry.. should be fixed now.


- Sameer


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/#review201559
---


On April 19, 2018, 9:49 p.m., Sameer Brenn wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66697/
> ---
> 
> (Updated April 19, 2018, 9:49 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> My team has some scripts to start devel shards which create tunnels:
> 
> ```
> aurora task ssh -L 8002:http --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> aurora task ssh -L 9002:health --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> ```
> 
> We use fixed local port numbers because that way we can run dependent 
> services locally that look for locally-running copies of the
> same service on a fixed port, but then those requests get tunnelled through 
> to the devel shard.
> 
> When the devel shard is restarted, however, the tunnel is still running so 
> the subsequent call to create a new tunnel fails because
> it can't bind to the fixed port.
> 
> If we save the SSH process PID to a file, we can then kill existing tunnel to 
> the old instance before starting up the new tunnel to the
> new instance.
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/cli/task.py 
> 652a545072f161dbf854b3d6d273809b09d142e8 
>   src/test/python/apache/aurora/client/cli/test_task.py 
> a543d4a101c58149f8af265257d061ff5032049c 
> 
> 
> Diff: https://reviews.apache.org/r/66697/diff/5/
> 
> 
> Testing
> ---
> 
> ```
> $ ./pants test src/test/python/apache/aurora/client::
> ```
> 
> And when applying the same patch to our local repo at Twitter:
> 
> ```
> $ ./pants run 
> twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
> task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
> "smf1/sbrenn/devel/proxyapp/0" &
> $ ps -p `cat /tmp/p`
>   PID TTY   TIME CMD
> 34729 ttys0000:00.05 ssh -t -n -N -L 
> 8005:smf1-aki-27-sr1.prod.twitter.com:31794 
> sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
> /var/lib/mesos/slaves/*/frameworks/*/exec
> ```
> 
> 
> Thanks,
> 
> Sameer Brenn
> 
>



Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-19 Thread Sameer Brenn via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/
---

(Updated April 19, 2018, 9:49 p.m.)


Review request for Aurora, Joshua Cohen and Zameer Manji.


Changes
---

@ReviewBot retry


Repository: aurora


Description
---

My team has some scripts to start devel shards which create tunnels:

```
aurora task ssh -L 8002:http --ssh-options "-f -N" "$DC/$USER/devel/proxyapp/0"
aurora task ssh -L 9002:health --ssh-options "-f -N" 
"$DC/$USER/devel/proxyapp/0"
```

We use fixed local port numbers because that way we can run dependent services 
locally that look for locally-running copies of the
same service on a fixed port, but then those requests get tunnelled through to 
the devel shard.

When the devel shard is restarted, however, the tunnel is still running so the 
subsequent call to create a new tunnel fails because
it can't bind to the fixed port.

If we save the SSH process PID to a file, we can then kill existing tunnel to 
the old instance before starting up the new tunnel to the
new instance.


Diffs (updated)
-

  src/main/python/apache/aurora/client/cli/task.py 
652a545072f161dbf854b3d6d273809b09d142e8 
  src/test/python/apache/aurora/client/cli/test_task.py 
a543d4a101c58149f8af265257d061ff5032049c 


Diff: https://reviews.apache.org/r/66697/diff/5/

Changes: https://reviews.apache.org/r/66697/diff/4-5/


Testing
---

```
$ ./pants test src/test/python/apache/aurora/client::
```

And when applying the same patch to our local repo at Twitter:

```
$ ./pants run 
twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
"smf1/sbrenn/devel/proxyapp/0" &
$ ps -p `cat /tmp/p`
  PID TTY   TIME CMD
34729 ttys0000:00.05 ssh -t -n -N -L 
8005:smf1-aki-27-sr1.prod.twitter.com:31794 
sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
/var/lib/mesos/slaves/*/frameworks/*/exec
```


Thanks,

Sameer Brenn



Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-19 Thread Santhosh Kumar Shanmugham

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/#review201559
---



@ReviewBot retry

- Santhosh Kumar Shanmugham


On April 19, 2018, 12:30 p.m., Sameer Brenn wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66697/
> ---
> 
> (Updated April 19, 2018, 12:30 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> My team has some scripts to start devel shards which create tunnels:
> 
> ```
> aurora task ssh -L 8002:http --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> aurora task ssh -L 9002:health --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> ```
> 
> We use fixed local port numbers because that way we can run dependent 
> services locally that look for locally-running copies of the
> same service on a fixed port, but then those requests get tunnelled through 
> to the devel shard.
> 
> When the devel shard is restarted, however, the tunnel is still running so 
> the subsequent call to create a new tunnel fails because
> it can't bind to the fixed port.
> 
> If we save the SSH process PID to a file, we can then kill existing tunnel to 
> the old instance before starting up the new tunnel to the
> new instance.
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/cli/task.py 
> 652a545072f161dbf854b3d6d273809b09d142e8 
>   src/test/python/apache/aurora/client/cli/test_task.py 
> a543d4a101c58149f8af265257d061ff5032049c 
> 
> 
> Diff: https://reviews.apache.org/r/66697/diff/4/
> 
> 
> Testing
> ---
> 
> ```
> $ ./pants test src/test/python/apache/aurora/client::
> ```
> 
> And when applying the same patch to our local repo at Twitter:
> 
> ```
> $ ./pants run 
> twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
> task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
> "smf1/sbrenn/devel/proxyapp/0" &
> $ ps -p `cat /tmp/p`
>   PID TTY   TIME CMD
> 34729 ttys0000:00.05 ssh -t -n -N -L 
> 8005:smf1-aki-27-sr1.prod.twitter.com:31794 
> sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
> /var/lib/mesos/slaves/*/frameworks/*/exec
> ```
> 
> 
> Thanks,
> 
> Sameer Brenn
> 
>



Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-19 Thread Santhosh Kumar Shanmugham


> On April 19, 2018, 1:52 p.m., Santhosh Kumar Shanmugham wrote:
> > @ReviewBot retry

Actually, looks like some style issues.


- Santhosh Kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/#review201559
---


On April 19, 2018, 12:30 p.m., Sameer Brenn wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66697/
> ---
> 
> (Updated April 19, 2018, 12:30 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> My team has some scripts to start devel shards which create tunnels:
> 
> ```
> aurora task ssh -L 8002:http --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> aurora task ssh -L 9002:health --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> ```
> 
> We use fixed local port numbers because that way we can run dependent 
> services locally that look for locally-running copies of the
> same service on a fixed port, but then those requests get tunnelled through 
> to the devel shard.
> 
> When the devel shard is restarted, however, the tunnel is still running so 
> the subsequent call to create a new tunnel fails because
> it can't bind to the fixed port.
> 
> If we save the SSH process PID to a file, we can then kill existing tunnel to 
> the old instance before starting up the new tunnel to the
> new instance.
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/cli/task.py 
> 652a545072f161dbf854b3d6d273809b09d142e8 
>   src/test/python/apache/aurora/client/cli/test_task.py 
> a543d4a101c58149f8af265257d061ff5032049c 
> 
> 
> Diff: https://reviews.apache.org/r/66697/diff/4/
> 
> 
> Testing
> ---
> 
> ```
> $ ./pants test src/test/python/apache/aurora/client::
> ```
> 
> And when applying the same patch to our local repo at Twitter:
> 
> ```
> $ ./pants run 
> twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
> task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
> "smf1/sbrenn/devel/proxyapp/0" &
> $ ps -p `cat /tmp/p`
>   PID TTY   TIME CMD
> 34729 ttys0000:00.05 ssh -t -n -N -L 
> 8005:smf1-aki-27-sr1.prod.twitter.com:31794 
> sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
> /var/lib/mesos/slaves/*/frameworks/*/exec
> ```
> 
> 
> Thanks,
> 
> Sameer Brenn
> 
>



Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-19 Thread Aurora ReviewBot

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/#review201555
---



Master (ad0bc5f) is red with this patch.
  ./build-support/jenkins/build.sh

 class MesosContext(Struct):
--- 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/cli/test_inspect.py:before
 2018-04-19 20:04:35.102783
+++ 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/cli/test_inspect.py:after
  2018-04-19 20:13:32.129150
@@ -22,9 +22,9 @@
 from apache.aurora.config.schema.base import Job
 from apache.thermos.config.schema_base import MB, Process, Resources, Task
 
+from .util import AuroraClientCommandTest
+
 from gen.apache.aurora.api.constants import AURORA_EXECUTOR_NAME
-
-from .util import AuroraClientCommandTest
 
 
 class TestInspectCommand(AuroraClientCommandTest):

20:13:32 00:03   [complete]
   SUCCESS
+ ./build-support/python/checkstyle-check

20:13:36 00:00 [main]
   (To run a reporting server: ./pants server)
20:13:36 00:00   [setup]
20:13:36 00:00 [parse]
   Executing tasks in goals: lint
20:13:36 00:00   [lint]
20:13:36 00:00 [python-eval]
20:13:36 00:00 [pythonstyle]
20:13:36 00:00   [cache]
   No cached artifacts for 48 targets.
   Invalidated 48 targets.
T001:ERROR   src/test/python/apache/aurora/client/cli/test_task.py:114 Class 
globals must be UPPER_SNAKE_CASED
 |  mocked_pid = 12312


FAILURE: 1 Python Style issues found. You may try `./pants fmt `


20:13:56 00:20   [complete]
   FAILURE


I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On April 19, 2018, 7:30 p.m., Sameer Brenn wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66697/
> ---
> 
> (Updated April 19, 2018, 7:30 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> My team has some scripts to start devel shards which create tunnels:
> 
> ```
> aurora task ssh -L 8002:http --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> aurora task ssh -L 9002:health --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> ```
> 
> We use fixed local port numbers because that way we can run dependent 
> services locally that look for locally-running copies of the
> same service on a fixed port, but then those requests get tunnelled through 
> to the devel shard.
> 
> When the devel shard is restarted, however, the tunnel is still running so 
> the subsequent call to create a new tunnel fails because
> it can't bind to the fixed port.
> 
> If we save the SSH process PID to a file, we can then kill existing tunnel to 
> the old instance before starting up the new tunnel to the
> new instance.
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/cli/task.py 
> 652a545072f161dbf854b3d6d273809b09d142e8 
>   src/test/python/apache/aurora/client/cli/test_task.py 
> a543d4a101c58149f8af265257d061ff5032049c 
> 
> 
> Diff: https://reviews.apache.org/r/66697/diff/4/
> 
> 
> Testing
> ---
> 
> ```
> $ ./pants test src/test/python/apache/aurora/client::
> ```
> 
> And when applying the same patch to our local repo at Twitter:
> 
> ```
> $ ./pants run 
> twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
> task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
> "smf1/sbrenn/devel/proxyapp/0" &
> $ ps -p `cat /tmp/p`
>   PID TTY   TIME CMD
> 34729 ttys0000:00.05 ssh -t -n -N -L 
> 8005:smf1-aki-27-sr1.prod.twitter.com:31794 
> sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
> /var/lib/mesos/slaves/*/frameworks/*/exec
> ```
> 
> 
> Thanks,
> 
> Sameer Brenn
> 
>



Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-19 Thread Sameer Brenn via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/
---

(Updated April 19, 2018, 7:30 p.m.)


Review request for Aurora, Joshua Cohen and Zameer Manji.


Changes
---

Move mocked_pid to a constant


Repository: aurora


Description
---

My team has some scripts to start devel shards which create tunnels:

```
aurora task ssh -L 8002:http --ssh-options "-f -N" "$DC/$USER/devel/proxyapp/0"
aurora task ssh -L 9002:health --ssh-options "-f -N" 
"$DC/$USER/devel/proxyapp/0"
```

We use fixed local port numbers because that way we can run dependent services 
locally that look for locally-running copies of the
same service on a fixed port, but then those requests get tunnelled through to 
the devel shard.

When the devel shard is restarted, however, the tunnel is still running so the 
subsequent call to create a new tunnel fails because
it can't bind to the fixed port.

If we save the SSH process PID to a file, we can then kill existing tunnel to 
the old instance before starting up the new tunnel to the
new instance.


Diffs (updated)
-

  src/main/python/apache/aurora/client/cli/task.py 
652a545072f161dbf854b3d6d273809b09d142e8 
  src/test/python/apache/aurora/client/cli/test_task.py 
a543d4a101c58149f8af265257d061ff5032049c 


Diff: https://reviews.apache.org/r/66697/diff/4/

Changes: https://reviews.apache.org/r/66697/diff/3-4/


Testing
---

```
$ ./pants test src/test/python/apache/aurora/client::
```

And when applying the same patch to our local repo at Twitter:

```
$ ./pants run 
twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
"smf1/sbrenn/devel/proxyapp/0" &
$ ps -p `cat /tmp/p`
  PID TTY   TIME CMD
34729 ttys0000:00.05 ssh -t -n -N -L 
8005:smf1-aki-27-sr1.prod.twitter.com:31794 
sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
/var/lib/mesos/slaves/*/frameworks/*/exec
```


Thanks,

Sameer Brenn



Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-19 Thread Sameer Brenn via Review Board


> On April 19, 2018, 6:01 p.m., Santhosh Kumar Shanmugham wrote:
> > src/test/python/apache/aurora/client/cli/test_task.py
> > Lines 175 (patched)
> > 
> >
> > Pull `12312` into a constant.

Thanks, done!


- Sameer


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/#review201483
---


On April 19, 2018, 7:30 p.m., Sameer Brenn wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66697/
> ---
> 
> (Updated April 19, 2018, 7:30 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> My team has some scripts to start devel shards which create tunnels:
> 
> ```
> aurora task ssh -L 8002:http --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> aurora task ssh -L 9002:health --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> ```
> 
> We use fixed local port numbers because that way we can run dependent 
> services locally that look for locally-running copies of the
> same service on a fixed port, but then those requests get tunnelled through 
> to the devel shard.
> 
> When the devel shard is restarted, however, the tunnel is still running so 
> the subsequent call to create a new tunnel fails because
> it can't bind to the fixed port.
> 
> If we save the SSH process PID to a file, we can then kill existing tunnel to 
> the old instance before starting up the new tunnel to the
> new instance.
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/cli/task.py 
> 652a545072f161dbf854b3d6d273809b09d142e8 
>   src/test/python/apache/aurora/client/cli/test_task.py 
> a543d4a101c58149f8af265257d061ff5032049c 
> 
> 
> Diff: https://reviews.apache.org/r/66697/diff/4/
> 
> 
> Testing
> ---
> 
> ```
> $ ./pants test src/test/python/apache/aurora/client::
> ```
> 
> And when applying the same patch to our local repo at Twitter:
> 
> ```
> $ ./pants run 
> twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
> task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
> "smf1/sbrenn/devel/proxyapp/0" &
> $ ps -p `cat /tmp/p`
>   PID TTY   TIME CMD
> 34729 ttys0000:00.05 ssh -t -n -N -L 
> 8005:smf1-aki-27-sr1.prod.twitter.com:31794 
> sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
> /var/lib/mesos/slaves/*/frameworks/*/exec
> ```
> 
> 
> Thanks,
> 
> Sameer Brenn
> 
>



Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-19 Thread Santhosh Kumar Shanmugham

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/#review201483
---


Ship it!





src/test/python/apache/aurora/client/cli/test_task.py
Lines 175 (patched)


Pull `12312` into a constant.


- Santhosh Kumar Shanmugham


On April 18, 2018, 2:54 p.m., Sameer Brenn wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66697/
> ---
> 
> (Updated April 18, 2018, 2:54 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> My team has some scripts to start devel shards which create tunnels:
> 
> ```
> aurora task ssh -L 8002:http --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> aurora task ssh -L 9002:health --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> ```
> 
> We use fixed local port numbers because that way we can run dependent 
> services locally that look for locally-running copies of the
> same service on a fixed port, but then those requests get tunnelled through 
> to the devel shard.
> 
> When the devel shard is restarted, however, the tunnel is still running so 
> the subsequent call to create a new tunnel fails because
> it can't bind to the fixed port.
> 
> If we save the SSH process PID to a file, we can then kill existing tunnel to 
> the old instance before starting up the new tunnel to the
> new instance.
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/cli/task.py 
> 652a545072f161dbf854b3d6d273809b09d142e8 
>   src/test/python/apache/aurora/client/cli/test_task.py 
> a543d4a101c58149f8af265257d061ff5032049c 
> 
> 
> Diff: https://reviews.apache.org/r/66697/diff/3/
> 
> 
> Testing
> ---
> 
> ```
> $ ./pants test src/test/python/apache/aurora/client::
> ```
> 
> And when applying the same patch to our local repo at Twitter:
> 
> ```
> $ ./pants run 
> twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
> task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
> "smf1/sbrenn/devel/proxyapp/0" &
> $ ps -p `cat /tmp/p`
>   PID TTY   TIME CMD
> 34729 ttys0000:00.05 ssh -t -n -N -L 
> 8005:smf1-aki-27-sr1.prod.twitter.com:31794 
> sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
> /var/lib/mesos/slaves/*/frameworks/*/exec
> ```
> 
> 
> Thanks,
> 
> Sameer Brenn
> 
>



Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-18 Thread Sameer Brenn via Review Board


> On April 18, 2018, 9:57 p.m., Santhosh Kumar Shanmugham wrote:
> > Have you considered this approach? 
> > https://www.g-loaded.eu/2006/11/24/auto-closing-ssh-tunnels/

Hi Santhosh-- thanks for the link, I hadn't looked at that before. 

Unfortunately, I don't believe that will work for us. 

As far as I can tell from that doc, we would need to connect to the tunnel 
within a fixed amount of time, and then when that connection goes away, the 
tunnel is shut down.

In our case, the dependent services make HTTP calls to the remote service over 
the tunnel, so we want the tunnel to stick around after the first HTTP call 
completes so that the dependent service can make subsequent HTTP calls. We 
might stop and restart the dependent service running locally (because we're 
actively developing on it) so we can't guarantee that the dependent service 
will continue to make requests at any particular rate, either.


- Sameer


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/#review201471
---


On April 18, 2018, 9:54 p.m., Sameer Brenn wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66697/
> ---
> 
> (Updated April 18, 2018, 9:54 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> My team has some scripts to start devel shards which create tunnels:
> 
> ```
> aurora task ssh -L 8002:http --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> aurora task ssh -L 9002:health --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> ```
> 
> We use fixed local port numbers because that way we can run dependent 
> services locally that look for locally-running copies of the
> same service on a fixed port, but then those requests get tunnelled through 
> to the devel shard.
> 
> When the devel shard is restarted, however, the tunnel is still running so 
> the subsequent call to create a new tunnel fails because
> it can't bind to the fixed port.
> 
> If we save the SSH process PID to a file, we can then kill existing tunnel to 
> the old instance before starting up the new tunnel to the
> new instance.
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/cli/task.py 
> 652a545072f161dbf854b3d6d273809b09d142e8 
>   src/test/python/apache/aurora/client/cli/test_task.py 
> a543d4a101c58149f8af265257d061ff5032049c 
> 
> 
> Diff: https://reviews.apache.org/r/66697/diff/3/
> 
> 
> Testing
> ---
> 
> ```
> $ ./pants test src/test/python/apache/aurora/client::
> ```
> 
> And when applying the same patch to our local repo at Twitter:
> 
> ```
> $ ./pants run 
> twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
> task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
> "smf1/sbrenn/devel/proxyapp/0" &
> $ ps -p `cat /tmp/p`
>   PID TTY   TIME CMD
> 34729 ttys0000:00.05 ssh -t -n -N -L 
> 8005:smf1-aki-27-sr1.prod.twitter.com:31794 
> sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
> /var/lib/mesos/slaves/*/frameworks/*/exec
> ```
> 
> 
> Thanks,
> 
> Sameer Brenn
> 
>



Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-18 Thread Aurora ReviewBot

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/#review201479
---


Ship it!




Master (ad0bc5f) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On April 18, 2018, 9:54 p.m., Sameer Brenn wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66697/
> ---
> 
> (Updated April 18, 2018, 9:54 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> My team has some scripts to start devel shards which create tunnels:
> 
> ```
> aurora task ssh -L 8002:http --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> aurora task ssh -L 9002:health --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> ```
> 
> We use fixed local port numbers because that way we can run dependent 
> services locally that look for locally-running copies of the
> same service on a fixed port, but then those requests get tunnelled through 
> to the devel shard.
> 
> When the devel shard is restarted, however, the tunnel is still running so 
> the subsequent call to create a new tunnel fails because
> it can't bind to the fixed port.
> 
> If we save the SSH process PID to a file, we can then kill existing tunnel to 
> the old instance before starting up the new tunnel to the
> new instance.
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/cli/task.py 
> 652a545072f161dbf854b3d6d273809b09d142e8 
>   src/test/python/apache/aurora/client/cli/test_task.py 
> a543d4a101c58149f8af265257d061ff5032049c 
> 
> 
> Diff: https://reviews.apache.org/r/66697/diff/3/
> 
> 
> Testing
> ---
> 
> ```
> $ ./pants test src/test/python/apache/aurora/client::
> ```
> 
> And when applying the same patch to our local repo at Twitter:
> 
> ```
> $ ./pants run 
> twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
> task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
> "smf1/sbrenn/devel/proxyapp/0" &
> $ ps -p `cat /tmp/p`
>   PID TTY   TIME CMD
> 34729 ttys0000:00.05 ssh -t -n -N -L 
> 8005:smf1-aki-27-sr1.prod.twitter.com:31794 
> sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
> /var/lib/mesos/slaves/*/frameworks/*/exec
> ```
> 
> 
> Thanks,
> 
> Sameer Brenn
> 
>



Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-18 Thread Santhosh Kumar Shanmugham

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/#review201471
---



Have you considered this approach? 
https://www.g-loaded.eu/2006/11/24/auto-closing-ssh-tunnels/

- Santhosh Kumar Shanmugham


On April 18, 2018, 2:54 p.m., Sameer Brenn wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66697/
> ---
> 
> (Updated April 18, 2018, 2:54 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> My team has some scripts to start devel shards which create tunnels:
> 
> ```
> aurora task ssh -L 8002:http --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> aurora task ssh -L 9002:health --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> ```
> 
> We use fixed local port numbers because that way we can run dependent 
> services locally that look for locally-running copies of the
> same service on a fixed port, but then those requests get tunnelled through 
> to the devel shard.
> 
> When the devel shard is restarted, however, the tunnel is still running so 
> the subsequent call to create a new tunnel fails because
> it can't bind to the fixed port.
> 
> If we save the SSH process PID to a file, we can then kill existing tunnel to 
> the old instance before starting up the new tunnel to the
> new instance.
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/cli/task.py 
> 652a545072f161dbf854b3d6d273809b09d142e8 
>   src/test/python/apache/aurora/client/cli/test_task.py 
> a543d4a101c58149f8af265257d061ff5032049c 
> 
> 
> Diff: https://reviews.apache.org/r/66697/diff/3/
> 
> 
> Testing
> ---
> 
> ```
> $ ./pants test src/test/python/apache/aurora/client::
> ```
> 
> And when applying the same patch to our local repo at Twitter:
> 
> ```
> $ ./pants run 
> twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
> task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
> "smf1/sbrenn/devel/proxyapp/0" &
> $ ps -p `cat /tmp/p`
>   PID TTY   TIME CMD
> 34729 ttys0000:00.05 ssh -t -n -N -L 
> 8005:smf1-aki-27-sr1.prod.twitter.com:31794 
> sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
> /var/lib/mesos/slaves/*/frameworks/*/exec
> ```
> 
> 
> Thanks,
> 
> Sameer Brenn
> 
>



Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-18 Thread Sameer Brenn via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/
---

(Updated April 18, 2018, 9:54 p.m.)


Review request for Aurora, Joshua Cohen and Zameer Manji.


Changes
---

@ReviewBot retry


Repository: aurora


Description
---

My team has some scripts to start devel shards which create tunnels:

```
aurora task ssh -L 8002:http --ssh-options "-f -N" "$DC/$USER/devel/proxyapp/0"
aurora task ssh -L 9002:health --ssh-options "-f -N" 
"$DC/$USER/devel/proxyapp/0"
```

We use fixed local port numbers because that way we can run dependent services 
locally that look for locally-running copies of the
same service on a fixed port, but then those requests get tunnelled through to 
the devel shard.

When the devel shard is restarted, however, the tunnel is still running so the 
subsequent call to create a new tunnel fails because
it can't bind to the fixed port.

If we save the SSH process PID to a file, we can then kill existing tunnel to 
the old instance before starting up the new tunnel to the
new instance.


Diffs (updated)
-

  src/main/python/apache/aurora/client/cli/task.py 
652a545072f161dbf854b3d6d273809b09d142e8 
  src/test/python/apache/aurora/client/cli/test_task.py 
a543d4a101c58149f8af265257d061ff5032049c 


Diff: https://reviews.apache.org/r/66697/diff/3/

Changes: https://reviews.apache.org/r/66697/diff/2-3/


Testing
---

```
$ ./pants test src/test/python/apache/aurora/client::
```

And when applying the same patch to our local repo at Twitter:

```
$ ./pants run 
twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
"smf1/sbrenn/devel/proxyapp/0" &
$ ps -p `cat /tmp/p`
  PID TTY   TIME CMD
34729 ttys0000:00.05 ssh -t -n -N -L 
8005:smf1-aki-27-sr1.prod.twitter.com:31794 
sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
/var/lib/mesos/slaves/*/frameworks/*/exec
```


Thanks,

Sameer Brenn



Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-18 Thread Sameer Brenn via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/
---

(Updated April 18, 2018, 9:28 p.m.)


Review request for Aurora, Joshua Cohen and Zameer Manji.


Changes
---

@ReviewBot retry


Repository: aurora


Description
---

My team has some scripts to start devel shards which create tunnels:

```
aurora task ssh -L 8002:http --ssh-options "-f -N" "$DC/$USER/devel/proxyapp/0"
aurora task ssh -L 9002:health --ssh-options "-f -N" 
"$DC/$USER/devel/proxyapp/0"
```

We use fixed local port numbers because that way we can run dependent services 
locally that look for locally-running copies of the
same service on a fixed port, but then those requests get tunnelled through to 
the devel shard.

When the devel shard is restarted, however, the tunnel is still running so the 
subsequent call to create a new tunnel fails because
it can't bind to the fixed port.

If we save the SSH process PID to a file, we can then kill existing tunnel to 
the old instance before starting up the new tunnel to the
new instance.


Diffs (updated)
-

  src/main/python/apache/aurora/client/cli/task.py 
652a545072f161dbf854b3d6d273809b09d142e8 
  src/test/python/apache/aurora/client/cli/test_task.py 
a543d4a101c58149f8af265257d061ff5032049c 


Diff: https://reviews.apache.org/r/66697/diff/2/

Changes: https://reviews.apache.org/r/66697/diff/1-2/


Testing
---

```
$ ./pants test src/test/python/apache/aurora/client::
```

And when applying the same patch to our local repo at Twitter:

```
$ ./pants run 
twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
"smf1/sbrenn/devel/proxyapp/0" &
$ ps -p `cat /tmp/p`
  PID TTY   TIME CMD
34729 ttys0000:00.05 ssh -t -n -N -L 
8005:smf1-aki-27-sr1.prod.twitter.com:31794 
sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
/var/lib/mesos/slaves/*/frameworks/*/exec
```


Thanks,

Sameer Brenn



Re: Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-18 Thread Aurora ReviewBot

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/#review201464
---



Master (ad0bc5f) is red with this patch.
  ./build-support/jenkins/build.sh

+from .util import AuroraClientCommandTest
+
 from gen.apache.aurora.api.constants import AURORA_EXECUTOR_NAME
-
-from .util import AuroraClientCommandTest
 
 
 class TestInspectCommand(AuroraClientCommandTest):

21:18:40 00:04   [complete]
   SUCCESS
+ ./build-support/python/checkstyle-check

21:18:44 00:00 [main]
   (To run a reporting server: ./pants server)
21:18:44 00:00   [setup]
21:18:45 00:01 [parse]
   Executing tasks in goals: lint
21:18:45 00:01   [lint]
21:18:45 00:01 [python-eval]
21:18:45 00:01 [pythonstyle]
21:18:45 00:01   [cache]
   No cached artifacts for 48 targets.
   Invalidated 48 targets.
E501:ERROR   
PythonFile(src/test/python/apache/aurora/client/cli/test_task.py):153 line too 
long (137 > 100 characters)
 |cmd.execute(['task', 'ssh', 
'--pid-file={}'.format(pid_file.name), '--ssh-options=-v', 
'west/bozo/test/hello/1', '--command=ls'])


T100:ERROR   src/main/python/apache/aurora/client/cli/task.py:163 Indentation 
of 4 instead of 2
 |with open(context.options.pid_file, "w") as f:

T100:ERROR   src/main/python/apache/aurora/client/cli/task.py:164 Indentation 
of 4 instead of 2
 |f.write(str(process.pid))


FAILURE: 3 Python Style issues found. You may try `./pants fmt `


21:19:05 00:21   [complete]
   FAILURE


I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On April 18, 2018, 8:55 p.m., Sameer Brenn wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66697/
> ---
> 
> (Updated April 18, 2018, 8:55 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> My team has some scripts to start devel shards which create tunnels:
> 
> ```
> aurora task ssh -L 8002:http --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> aurora task ssh -L 9002:health --ssh-options "-f -N" 
> "$DC/$USER/devel/proxyapp/0"
> ```
> 
> We use fixed local port numbers because that way we can run dependent 
> services locally that look for locally-running copies of the
> same service on a fixed port, but then those requests get tunnelled through 
> to the devel shard.
> 
> When the devel shard is restarted, however, the tunnel is still running so 
> the subsequent call to create a new tunnel fails because
> it can't bind to the fixed port.
> 
> If we save the SSH process PID to a file, we can then kill existing tunnel to 
> the old instance before starting up the new tunnel to the
> new instance.
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/cli/task.py 
> 652a545072f161dbf854b3d6d273809b09d142e8 
>   src/test/python/apache/aurora/client/cli/test_task.py 
> a543d4a101c58149f8af265257d061ff5032049c 
> 
> 
> Diff: https://reviews.apache.org/r/66697/diff/1/
> 
> 
> Testing
> ---
> 
> ```
> $ ./pants test src/test/python/apache/aurora/client::
> ```
> 
> And when applying the same patch to our local repo at Twitter:
> 
> ```
> $ ./pants run 
> twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
> task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
> "smf1/sbrenn/devel/proxyapp/0" &
> $ ps -p `cat /tmp/p`
>   PID TTY   TIME CMD
> 34729 ttys0000:00.05 ssh -t -n -N -L 
> 8005:smf1-aki-27-sr1.prod.twitter.com:31794 
> sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
> /var/lib/mesos/slaves/*/frameworks/*/exec
> ```
> 
> 
> Thanks,
> 
> Sameer Brenn
> 
>



Review Request 66697: Add --pid-file flag to `aurora task ssh` to write the PID of the underlying SSH command to a specified file.

2018-04-18 Thread Sameer Brenn via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66697/
---

Review request for Aurora, Joshua Cohen and Zameer Manji.


Repository: aurora


Description
---

My team has some scripts to start devel shards which create tunnels:

```
aurora task ssh -L 8002:http --ssh-options "-f -N" "$DC/$USER/devel/proxyapp/0"
aurora task ssh -L 9002:health --ssh-options "-f -N" 
"$DC/$USER/devel/proxyapp/0"
```

We use fixed local port numbers because that way we can run dependent services 
locally that look for locally-running copies of the
same service on a fixed port, but then those requests get tunnelled through to 
the devel shard.

When the devel shard is restarted, however, the tunnel is still running so the 
subsequent call to create a new tunnel fails because
it can't bind to the fixed port.

If we save the SSH process PID to a file, we can then kill existing tunnel to 
the old instance before starting up the new tunnel to the
new instance.


Diffs
-

  src/main/python/apache/aurora/client/cli/task.py 
652a545072f161dbf854b3d6d273809b09d142e8 
  src/test/python/apache/aurora/client/cli/test_task.py 
a543d4a101c58149f8af265257d061ff5032049c 


Diff: https://reviews.apache.org/r/66697/diff/1/


Testing
---

```
$ ./pants test src/test/python/apache/aurora/client::
```

And when applying the same patch to our local repo at Twitter:

```
$ ./pants run 
twitter/src/main/python/twitter/aurora/client/cli_internal:aurora_internal -- 
task ssh -L 8005:http --ssh-options "-n -N" --pid-file /tmp/p 
"smf1/sbrenn/devel/proxyapp/0" &
$ ps -p `cat /tmp/p`
  PID TTY   TIME CMD
34729 ttys0000:00.05 ssh -t -n -N -L 
8005:smf1-aki-27-sr1.prod.twitter.com:31794 
sbr...@smf1-aki-27-sr1.prod.twitter.com cd 
/var/lib/mesos/slaves/*/frameworks/*/exec
```


Thanks,

Sameer Brenn