Am Mo., 23. März 2020 um 11:14 Uhr schrieb Michael Albinus <michael.albi...@gmx.de>: > > Michael Albinus <michael.albi...@gmx.de> writes: > > Hi Philipp, > > >> This uses the current master of Tramp. All these results are > >> statistically significant (on my machine, that is). I think at least > >> start-file-process and make-process should definitely switch to the > >> connectionless approach (or at least offer it as option), that would > >> make language servers or Flymake backends over SSH possible. > >> I'm also happy to contribute patches! > > > > I plan to provide a detailed analysis what happens in Tramp when running > > an asynchronous process. Based on this, we could discuss > > improvements. With commit 4c3c175a63823d851c7e9326f4f70c81b9c0110e I've > > done already a first step this direction. > > Well, I have applied (start-file-process "" nil "true") with a current > buffer being already remote. This avoids the initialization of the > connection, and should show us just the commands Tramp has sent. You can > do the same, analyzing the Tramp debug buffer for entries with Tramp > level (6). > > --8<---------------cut here---------------start------------->8--- > 10:04:29.011033 tramp-maybe-open-connection (6) # /bin/sh -i > 10:04:29.012136 tramp-wait-for-regexp (6) # > #$ > --8<---------------cut here---------------end--------------->8--- > > This opens a local shell in the connection buffer. Whether this is needed > or not might be discussed, it is just a general approach for *all* > different Tramp methods. However, it takes just 0.05 sec before raising > the next command, the performance gain wouldn't be too much.
It might not seem like much, but for the use case of Flymake/LSP we talk about launching a process after every keystroke. Blocking user input for longer than a few milliseconds is a non-starter. For example, according to https://pavelfatin.com/typing-with-pleasure/, Emacs has an average typing latency of 50 ms. This is already way slower than e.g. Vim, or IntelliJ. This first step alone would already double the latency! > > --8<---------------cut here---------------start------------->8--- > 10:04:29.060983 tramp-send-command (6) # exec ssh -q -o ControlMaster=auto > -o ControlPath='tramp.%C' -o ControlPersist=no -e none detlef > 10:04:29.161427 tramp-process-actions (6) # > Last login: Mon Mar 23 10:02:09 2020 from 192.168.178.30 > detlef:~> > --8<---------------cut here---------------end--------------->8--- > > This starts the remote connection with the home shell, and waiting for > the shell prompt. The command in question is not called, because at this > point, Tramp doesn't know whether it must interact for whatever reason, > for example providing a password. Tramps knows this only once the prompt > has been seen. That's in indeed an issue, for the connectionless approach SSH will need to be started with -o BatchMode=yes so that it doesn't ask for passwords. > > --8<---------------cut here---------------start------------->8--- > 10:04:29.163495 tramp-send-command (6) # rm -f ~/.editrc.tramp > 10:04:29.181094 tramp-wait-for-regexp (6) # > detlef:~> > 10:04:29.181402 tramp-send-command (6) # test -e ~/.editrc && mv -f ~/.editrc > ~/.editrc.tramp > 10:04:29.212769 tramp-wait-for-regexp (6) # > detlef:~> > 10:04:29.213056 tramp-send-command (6) # echo 'edit off' >~/.editrc > 10:04:29.214819 tramp-wait-for-regexp (6) # > detlef:~> > --8<---------------cut here---------------end--------------->8--- > > Tramp wants to avoid line editing, so it provides a simple > ~/.editrc. Some shells send annoying escape sequences otherwise. If the > remote shell isn't /bin/sh, but .../zsh or .../bash, Tramp doesn't apply > this, because the shell invocation cares already about. That seems like a compatibility measure for "exotic" systems that could/should be optional. > > --8<---------------cut here---------------start------------->8--- > 10:04:29.215306 tramp-send-command (6) # exec env TERM='dumb' > INSIDE_EMACS='28.0.50,tramp:2.5.0-pre' ENV='' HISTFILE=~/.tramp_history > PROMPT_COMMAND='' PS1=\#\$\ PS2='' PS3='' /bin/sh > 10:04:29.231797 tramp-wait-for-regexp (6) # > #$ > --8<---------------cut here---------------end--------------->8--- > > Now Tramp opens the remote shell, /bin/sh this case. Why does it need a shell, instead of just running the command directly? > > --8<---------------cut here---------------start------------->8--- > 10:04:29.232118 tramp-send-command (6) # rm -f ~/.editrc > 10:04:29.250077 tramp-wait-for-regexp (6) # > #$ > 10:04:29.250411 tramp-send-command (6) # test -e ~/.editrc.tramp && mv -f > ~/.editrc.tramp ~/.editrc > 10:04:29.264064 tramp-wait-for-regexp (6) # > #$ > --8<---------------cut here---------------end--------------->8--- > > ~/.editrc is reset. > > --8<---------------cut here---------------start------------->8--- > 10:04:29.264890 tramp-send-command (6) # (cd ~/) 2>/dev/null; echo > tramp_exit_status $? > 10:04:29.275592 tramp-wait-for-regexp (6) # > tramp_exit_status 0 > #$ > --8<---------------cut here---------------end--------------->8--- > > Tramp checks, whether the tilde ~/ is expanded. Maybe this is not needed > for asynchronous processes. Yes, it seems all of these checks work around some weird/broken/exotic behavior, which might be OK compatibility-wise, but should be optional. > > --8<---------------cut here---------------start------------->8--- > 10:04:29.276401 tramp-send-command (6) # stty -inlcr -onlcr -echo kill '^U' > erase '^H' > 10:04:29.283313 tramp-wait-for-regexp (6) # > #$ > 10:04:29.285092 tramp-wait-for-regexp (6) # > foo > #$ > 10:04:29.285543 tramp-send-command (6) # > PS1=///bbca37f3c9f6555a2319eae4da499ab9\#\$ PS2='' PS3='' PROMPT_COMMAND='' > 10:04:29.287075 tramp-wait-for-regexp (6) # > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.288034 tramp-send-command (6) # echo \"`uname -sr`\" 2>/dev/null; > echo tramp_exit_status $? > 10:04:29.291794 tramp-wait-for-regexp (6) # > "Linux 5.3.0-40-generic" > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.293108 tramp-send-command (6) # (echo foo ; echo bar) > 10:04:29.303182 tramp-wait-for-regexp (6) # > foo > bar > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.303632 tramp-send-command (6) # set +o vi +o emacs > 10:04:29.305080 tramp-wait-for-regexp (6) # > ///bbca37f3c9f6555a2319eae4da499ab9#$ > --8<---------------cut here---------------end--------------->8--- > > These are some interactive settings for the shell. Maybe we could get > rid of some. > > --8<---------------cut here---------------start------------->8--- > 10:04:29.306664 tramp-send-command (6) # echo \"`getconf PATH 2>/dev/null`\" > 2>/dev/null; echo tramp_exit_status $? > 10:04:29.344451 tramp-wait-for-regexp (6) # > "/bin:/usr/bin" > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.345910 tramp-send-command (6) # /bin/sh -l -c 'echo > 101183ec1d014197807aa97f43793ee7 \"$PATH\"' 2>/dev/null; echo > tramp_exit_status $? > 10:04:29.464168 tramp-wait-for-regexp (6) # > 101183ec1d014197807aa97f43793ee7 > "/home/albinus/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > --8<---------------cut here---------------end--------------->8--- > > Tramp checks for the values of $PATH to be set. Needed for commands > which are not absolute. Why? The remote shell is capable of searching the PATH itself, no? > > --8<---------------cut here---------------start------------->8--- > 10:04:29.466752 tramp-send-command (6) # test -d /home/albinus/bin > 2>/dev/null; echo tramp_exit_status $? > 10:04:29.468028 tramp-wait-for-regexp (6) # > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.470670 tramp-send-command (6) # test -d /usr/local/sbin 2>/dev/null; > echo tramp_exit_status $? > 10:04:29.472209 tramp-wait-for-regexp (6) # > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.475370 tramp-send-command (6) # test -d /usr/local/bin 2>/dev/null; > echo tramp_exit_status $? > 10:04:29.476956 tramp-wait-for-regexp (6) # > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.480232 tramp-send-command (6) # test -d /usr/sbin 2>/dev/null; echo > tramp_exit_status $? > 10:04:29.481974 tramp-wait-for-regexp (6) # > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.486268 tramp-send-command (6) # test -d /usr/bin 2>/dev/null; echo > tramp_exit_status $? > 10:04:29.488413 tramp-wait-for-regexp (6) # > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.492653 tramp-send-command (6) # test -d /sbin 2>/dev/null; echo > tramp_exit_status $? > 10:04:29.494734 tramp-wait-for-regexp (6) # > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.499274 tramp-send-command (6) # test -d /bin 2>/dev/null; echo > tramp_exit_status $? > 10:04:29.501362 tramp-wait-for-regexp (6) # > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.505625 tramp-send-command (6) # test -d /usr/games 2>/dev/null; echo > tramp_exit_status $? > 10:04:29.508350 tramp-wait-for-regexp (6) # > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.512642 tramp-send-command (6) # test -d /usr/local/games > 2>/dev/null; echo tramp_exit_status $? > 10:04:29.514700 tramp-wait-for-regexp (6) # > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.518718 tramp-send-command (6) # test -d /snap/bin 2>/dev/null; echo > tramp_exit_status $? > 10:04:29.741115 tramp-wait-for-regexp (6) # > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.744624 tramp-send-command (6) # test -d /local/bin 2>/dev/null; echo > tramp_exit_status $? > 10:04:29.746228 tramp-wait-for-regexp (6) # > tramp_exit_status 1 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.748851 tramp-send-command (6) # test -d /local/freeware/bin > 2>/dev/null; echo tramp_exit_status $? > 10:04:29.753279 tramp-wait-for-regexp (6) # > tramp_exit_status 1 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.757459 tramp-send-command (6) # test -d /local/gnu/bin 2>/dev/null; > echo tramp_exit_status $? > 10:04:29.764677 tramp-wait-for-regexp (6) # > tramp_exit_status 1 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.805413 tramp-send-command (6) # test -d /usr/freeware/bin > 2>/dev/null; echo tramp_exit_status $? > 10:04:29.807191 tramp-wait-for-regexp (6) # > tramp_exit_status 1 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.843228 tramp-send-command (6) # test -d /usr/pkg/bin 2>/dev/null; > echo tramp_exit_status $? > 10:04:29.856355 tramp-wait-for-regexp (6) # > tramp_exit_status 1 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.884514 tramp-send-command (6) # test -d /usr/contrib/bin > 2>/dev/null; echo tramp_exit_status $? > 10:04:29.887926 tramp-wait-for-regexp (6) # > tramp_exit_status 1 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.900189 tramp-send-command (6) # test -d /opt/bin 2>/dev/null; echo > tramp_exit_status $? > 10:04:29.901665 tramp-wait-for-regexp (6) # > tramp_exit_status 1 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.904255 tramp-send-command (6) # test -d /opt/sbin 2>/dev/null; echo > tramp_exit_status $? > 10:04:29.905553 tramp-wait-for-regexp (6) # > tramp_exit_status 1 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.908146 tramp-send-command (6) # test -d /opt/local/bin 2>/dev/null; > echo tramp_exit_status $? > 10:04:29.911510 tramp-wait-for-regexp (6) # > tramp_exit_status 1 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.919287 tramp-send-command (6) # > PATH=/home/albinus/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin; > export PATH > 10:04:29.923720 tramp-wait-for-regexp (6) # > ///bbca37f3c9f6555a2319eae4da499ab9#$ > --8<---------------cut here---------------end--------------->8--- > > Tramp tests every element of $PATH, whether it exists. I believe we > could get rid of these checks. In this test case, it would save 0.5 sec. That's an enormous amount of time! > > --8<---------------cut here---------------start------------->8--- > 10:04:29.924141 tramp-send-command (6) # mesg n 2>/dev/null; biff n > 2>/dev/null > 10:04:29.926596 tramp-wait-for-regexp (6) # > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.926945 tramp-send-command (6) # stty tab0 > 10:04:29.929181 tramp-wait-for-regexp (6) # > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.929508 tramp-send-command (6) # stty iutf8 2>/dev/null > 10:04:29.931592 tramp-wait-for-regexp (6) # > ///bbca37f3c9f6555a2319eae4da499ab9#$ > --8<---------------cut here---------------end--------------->8--- > > Some further interactive shell settings. > > --8<---------------cut here---------------start------------->8--- > 10:04:29.932319 tramp-send-command (6) # echo \"`tty`\" 2>/dev/null; echo > tramp_exit_status $? > 10:04:29.934489 tramp-wait-for-regexp (6) # > "/dev/pts/9" > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > --8<---------------cut here---------------end--------------->8--- > > Tramp determines the tty name. > > --8<---------------cut here---------------start------------->8--- > 10:04:29.939209 tramp-send-command (6) # while read var val; do export > $var="$val"; done <<'101183ec1d014197807aa97f43793ee7' > LC_ALL en_US.utf8 > TMPDIR $HOME > ENV '' > TMOUT 0 > LC_CTYPE '' > PAGER cat > 101183ec1d014197807aa97f43793ee7 > 10:04:29.940929 tramp-wait-for-regexp (6) # > ///bbca37f3c9f6555a2319eae4da499ab9#$ > 10:04:29.941354 tramp-send-command (6) # unset CDPATH HISTORY MAIL MAILCHECK > MAILPATH autocorrect correct > 10:04:29.942930 tramp-wait-for-regexp (6) # > ///bbca37f3c9f6555a2319eae4da499ab9#$ > --8<---------------cut here---------------end--------------->8--- > > Tramp sets shel environment variables, as dictated by > tramp-remote-process-environment and process-environment. This could be done in a single command: ssh "foo=bar exec cmd..." > > --8<---------------cut here---------------start------------->8--- > 10:04:29.944731 tramp-send-command (6) # echo $$ 2>/dev/null; echo > tramp_exit_status $? > 10:04:29.946021 tramp-wait-for-regexp (6) # > 22453 > tramp_exit_status 0 > ///bbca37f3c9f6555a2319eae4da499ab9#$ > --8<---------------cut here---------------end--------------->8--- > > Tramp determines its own pid. Needed for kill-process, if called. > > --8<---------------cut here---------------start------------->8--- > 10:04:29.947139 tramp-send-command (6) # cd /home/albinus/ && exec env > PS1\=/ssh\:detlef\:/home/albinus/\ \#\$\ true > --8<---------------cut here---------------end--------------->8--- > > Finally, Tramp changes the default directory to the one the current > buffer keeps (/home/albinus), and calls "true". The setting of $PS1 is > useful for shells called this way; it doesn't hurt for other commands. > > Well, some room for improvements. I'll try to change the obvious > candidates. Otherwise, comments welcome. Thanks, but I can only say that without my approach or a similar one this can't get significantly faster. The only possible really fast approach I'm aware of is starting exactly one process asynchronously and never calling accept-process-output.
