curl SFTP transfer from Cygwin on Win10 to Ubuntu 18.04 fails with Unknown host key type: 1835008
curl issue https://github.com/curl/curl/issues/7057 was closed with: "This seems to be purely a libssh2 issue and not a curl one." Curl reports "libssh2/1.7.0" On the same system, ssh reports " OpenSSH_8.5p1, OpenSSL 1.1.1f 31 Mar 2020" The curl code in https://github.com/curl/curl/blob/master/lib/vssh/libssh2.c has a number of defines to control what type of host keys it will accept, including LIBSSH2_KNOWNHOST_KEY_ED25519 Was the curl built with this set? Details are in the curl issue, but here they are again. - Here is the curl failure: : curl -vvv -s -T t.cpp sftp://bvoris@nucnuc/tmp/t2.cpp * STATE: INIT => CONNECT handle 0x800085338; line 1634 (connection #-5000) * Added connection 0. The cache now contains 1 members * STATE: CONNECT => RESOLVING handle 0x800085338; line 1680 (connection #0) * family0 == v4, family1 == v6 * Trying 192.168.1.5:22... * STATE: RESOLVING => CONNECTING handle 0x800085338; line 1762 (connection #0) * Connected to nucnuc (192.168.1.5) port 22 (#0) * STATE: CONNECTING => PROTOCONNECT handle 0x800085338; line 1825 (connection #0) * SFTP 0x8000847c8 state change from SSH_STOP to SSH_INIT * Found host nucnuc in /home/BVoris/.ssh/known_hosts * Unknown host key type: 1835008 * SFTP 0x8000847c8 state change from SSH_INIT to SSH_SESSION_FREE * SFTP 0x8000847c8 state change from SSH_SESSION_FREE to SSH_STOP * multi_done * The cache now contains 0 members * SSH DISCONNECT starts now * SSH DISCONNECT is done * Closing connection 0 - The curl/libcurl version: curl 7.76.1 (x86_64-pc-cygwin) libcurl/7.76.1 OpenSSL/1.1.1f zlib/1.2.11 brotli/1.0.9 zstd/1.4.9 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.0.4) libssh2/1.7.0 nghttp2/1.37.0 Release-Date: 2021-04-14 Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp Features: alt-svc AsynchDNS brotli Debug GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz Metalink NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP TrackMemory UnixSockets zstd - The known_hosts entry from the client: nucnuc ssh-ed25519 C3NzaC1lZDI1NTE5ICmjvQ5jehz5Jwt1PDGJBSgcXVhoMRnbn/E2p3srSK+c - curl is run on CYGWIN_NT-10.0 3.2.0(0.340/5/3) 2021-03-29 08:42 x86_64 Cygwin The target system has: OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017 -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
[no subject]
-- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Test: {mingw64-{i686,x86_64}-,}gcc-11.1.0-0.1
Am 13.05.2021 um 21:33 schrieb Hans-Bernhard Bröker: Am 13.05.2021 um 10:57 schrieb Thomas Wolff: The crash vanishes after removing a few lines from a conditional (if block) where the condition is false. A conditions that's always false, or one that's false during the execution of a particular test case? False during execution. This smells like wrong calculation of a relative jump (Intel "short jump") by the optimizer. If it were that simple, the problematic change should stand out like the proverbial sore thumb when comparing assembly listings of the two cases. Does it? Not really. As the problem only occurs with -O2, I'd need to check the result of gcc -S -O2, but with -O2, code is stirred so much it's hardly recognizable. The conditional jump to skip the (dynamically false) conditional is even a jump backwards in this case... -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Test: {mingw64-{i686,x86_64}-,}gcc-11.1.0-0.1
Am 13.05.2021 um 10:57 schrieb Thomas Wolff: The crash vanishes after removing a few lines from a conditional (if block) where the condition is false. A conditions that's always false, or one that's false during the execution of a particular test case? This smells like wrong calculation of a relative jump (Intel "short jump") by the optimizer. If it were that simple, the problematic change should stand out like the proverbial sore thumb when comparing assembly listings of the two cases. Does it? -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: [PATCHES] cygport
Jon Turney writes: > Let me add a 'R-b' for "stop generating packages for obsoletions", > which I almost wasted time writing again. I've just updated the branch with two new things I found a need for: https://repo.or.cz/cygport/rpm-style.git/shortlog/refs/heads/to-upstream The patch Jon was voting for is the first commit on that branch. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf rackAttack V1.04R1: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada
[ANNOUNCEMENT] Updated: postgresql-12.7-1
Version 12.7-1 of packages libecpg-compat3 libecpg-devel libecpg6 libpgtypes3 libpq-devel libpq5 postgresql postgresql-client postgresql-contrib postgresql-devel postgresql-doc postgresql-plperl postgresql-plpython are available in the Cygwin distribution: CHANGES This is the latest upstream 12.x release, upstream and security info on https://www.postgresql.org/about/news/postgresql-133-127-1112-1017-and-9622-released-2210/ Migration to Version 12.x http://www.postgresql.org/docs/12/static/release-12.html ADVISE for major version UPGRADE http://www.postgresql.org/support/versioning/ Major releases usually change the internal format of system tables and data files. These changes are often complex, so we do not maintain backward compatibility of all stored data. A dump/reload of the database or use of the pg_upgrade module is required for major upgrades. DESCRIPTION PostgreSQL is a powerful, open source object-relational database system. It has a proven architecture that has earned it a strong reputation for reliability, data integrity, and correctness. It is fully ACID compliant, has full support for foreign keys, joins, views, triggers, and stored procedures (in multiple languages). It includes most SQL:2008 data types HOMEPAGE http://www.postgresql.org Marco Atzeri If you have questions or comments, please send them to the cygwin mailing list at: cygwin (at) cygwin (dot) com . -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Updated: postgresql-12.7-1
Version 12.7-1 of packages libecpg-compat3 libecpg-devel libecpg6 libpgtypes3 libpq-devel libpq5 postgresql postgresql-client postgresql-contrib postgresql-devel postgresql-doc postgresql-plperl postgresql-plpython are available in the Cygwin distribution: CHANGES This is the latest upstream 12.x release, upstream and security info on https://www.postgresql.org/about/news/postgresql-133-127-1112-1017-and-9622-released-2210/ Migration to Version 12.x http://www.postgresql.org/docs/12/static/release-12.html ADVISE for major version UPGRADE http://www.postgresql.org/support/versioning/ Major releases usually change the internal format of system tables and data files. These changes are often complex, so we do not maintain backward compatibility of all stored data. A dump/reload of the database or use of the pg_upgrade module is required for major upgrades. DESCRIPTION PostgreSQL is a powerful, open source object-relational database system. It has a proven architecture that has earned it a strong reputation for reliability, data integrity, and correctness. It is fully ACID compliant, has full support for foreign keys, joins, views, triggers, and stored procedures (in multiple languages). It includes most SQL:2008 data types HOMEPAGE http://www.postgresql.org Marco Atzeri If you have questions or comments, please send them to the cygwin mailing list at: cygwin (at) cygwin (dot) com .
Re: [PATCH setup] Add new option '--compact-os'
Achim Gratz wrote: ... Deciding the compression and compression type by the extension is prone to miss a lot of real-world things, though, so you'd hope that that was recognized by the compact OS code itself instead of working around it? Yes. The compression only succeeds if the number of clusters could be reduced. Otherwise it fails with ERROR_COMPRESSION_NOT_BENEFICIAL and leaves the file as is. Regards, Christian
Re: [PATCH setup] Add new option '--compact-os'
On 12/05/2021 18:50, Christian Franke wrote: Jon Turney wrote: On 08/05/2021 21:03, Christian Franke wrote: ... +#include "compactos.h" + +#ifndef FSCTL_SET_EXTERNAL_BACKING There should be a comment here saying "not yet provided by w32api" or similar. ... or we wait for a release of w32api headers with the patch mentioned above :-) No, I think this way is better, since I build the setup releases on Fedora, and so don't have any control about when the w32api package I'm building against gets updated (and furthermore it's an old Fedora at the moment, since the x86 MinGW toolchain in recent Fedora isn't built with SJLJ exception handling...)
Re: [PATCH setup] Add new option '--compact-os'
Christian Franke wrote: Corinna Vinschen via Cygwin-apps wrote: ... When running a shell script, certain executables (especially coreutils, gawk, sed, grep, find) are not so very infrequently accessed. Is this compression really feasible for these binaries? Did you compare shell script performance with non-compressed, XPRESS16K and LZX compressed /bin dir? Good point. Now I did a test with a ./configure script run after reboot: There was significant difference with /bin/*.exe (only) uncompressed, NTFS-, XPRESS16K- or LZX-compressed. Time was always around 23s. Of course this should be: "... . There was *no* significant difference ...", sorry.
Re: [PATCH setup] Add new option '--compact-os'
Corinna Vinschen via Cygwin-apps wrote:
On May 12 16:14, Jon Turney wrote:
On 08/05/2021 21:03, Christian Franke wrote:
[...]
+bool io_stream_cygfile::compact_os_is_available = (OSMajorVersion () >= 10);
The documentation seems a bit vague, but are we really expecting this to
work on Windows 10 1507?
I think this could even work under 8.1 from what I can see on MSDN.
I skipped all Win8*, so I didn't test with 8.1 :-)
This page says "Available starting with Windows 10":
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/ns-ntifs-_file_provider_external_info_v0
It also says "Header: ntifs.h" but in recent "Windows Kits" all required
defines are in winioctl.h.
These defines are enabled even for '>= _WIN32_WINNT_WIN7'. According to
a test I did some time ago, Win7 could not read these files.
+{
+ const char * const p = name.c_str();
+ if (!(!strncmp (p, "/bin/", 5) || !strncmp (p, "/sbin/", 6) || !strncmp (p,
"/usr/", 5)))
+return true; /* File is not in R/O tree. */
+ const size_t len = name.size(); /* >= 5 */
+ if (!strcmp (p + (len - 4), ".dll") || !strcmp (p + (len - 3), ".so"))
+return true; /* Rebase will open file for writing which uncompresses the
file. */
+ if (!strcmp (p + (len - 3), ".gz") || !strcmp (p + (len - 3), ".xz"))
+return true; /* File is already compressed. */
Is this an assertion that there are no .bz2, .lzma, .zst etc. files in the
install?
Another question is this: FILE_PROVIDER_COMPRESSION_LZX
"This algorithm is designed to be highly compact, and provides for small
footprint for infrequently accessed data."
When running a shell script, certain executables (especially coreutils,
gawk, sed, grep, find) are not so very infrequently accessed. Is this
compression really feasible for these binaries? Did you compare shell
script performance with non-compressed, XPRESS16K and LZX compressed
/bin dir?
Good point. Now I did a test with a ./configure script run after reboot:
There was significant difference with /bin/*.exe (only) uncompressed,
NTFS-, XPRESS16K- or LZX-compressed. Time was always around 23s.
Here a read speed test with fast and slow storage and a 10+ years old
i7-2600K (4C/8T). The 256MiB test file was generated by concatenating
various EXE files. All file accesses were the first after reboot. AV
(defender) was turned off:
Compression MiB T1 T2 T3,T4
==
None 256 0.69s 10.1s <0.02s
NTFS 159 1.03s 8.1s <0.02s
XPRESS4K 138 -
XPRESS8K 128 -
XPRESS16K 123 0.64s 5.4s <0.02s
LZX 97 0.79s 4.8s <0.02s
T1,T2: Read whole file: time dd if=FILE bs=FILESIZE of=/dev/null
T3,T4: Read last byte: time dd if=FILE bs=1 skip=FILESIZE-1 of=/dev/null
T1,T3: SATA SSD, raw read speed with dd bs=1M: ~520MB/s
T2,T4: USB3 flash drive via USB2, raw read speed: ~27MB/s
As expected, compression helps to improve 'virtual' read speed on slow
storage. Otherwise, it depends on storage speed, CPU speed, system load, ...
As unexpected (for me), even LZX seems to be suitable for random reads
which are done when EXE files are preloaded or paged-in.
If the files were already cached, all read times were similar: ~0.135s
for the whole file.
For more flexibility, I will provide a new version of the patch with
'--compact-os ALGORITHM' option.
Thanks,
Christian
Re: [PATCH v6] Cygwin: rewrite cmdline parser
Oh, I should also mention that this patch does *not* fix globbing with
Windows paths. Although stuff like (in node-js):
child_process.spawnSync("C:\\cygwin64\\bin\\echo.exe",
["C:\\Qt\\*"], {encoding: 'utf-8'})
is correctly fed to glob(3), the glob() function does not perform any
interpretation of Windows paths. The older parser have the same
behavior and I didn't change that, seeing there are two ways this can
play out:
* Cygwin goes to extend glob() with the ability to do Windows paths
under a new flag. Globify() is modified to use that flag too.
* Cygwin does not extend glob(), so globify() is modified to skip on
is_dos_path much like how it already skips with a strpbrk.
Regards,
Artoria2e5
[PATCH v6] Cygwin: rewrite cmdline parser
This commit rewrites the cmdline parser to achieve the following:
* MSVCRT compatibility. Except for the single-quote handling (an
extension for compatibility with old Cygwin), the parser now
interprets option boundaries exactly like MSVCR since 2008. This fixes
the issue where our escaping does not work with our own parsing.
* Clarity. Since globify() is no longer responsible for handling the
opening and closing of quotes, the code is much simpler.
* Sanity. The GLOB_NOCHECK flag is removed, so a failed glob correctly
returns the literal value. Without the change, anything path-like
would be garbled by globify's escaping.
* A memory leak in the @file expansion is removed by rewriting it to use
a stack of buffers. This also simplifies the code since we no longer
have to move stuff. The "downside" is that tokens can no longer cross
file boundaries.
Some clarifications are made in the documentation for when globs are not
expanded.
The change fixes two complaints of mine:
* That cygwin is incompatible with its own escape.[1]
* That there is no way to echo `C:\"` from win32.[2]
[1]: https://cygwin.com/pipermail/cygwin/2020-June/245162.html
[2]: https://cygwin.com/pipermail/cygwin/2019-October/242790.html
(It's never the point to spawn cygwin32 from cygwin64. Consistency
matters: with yourself always, and with the outside world when you are
supposed to.)
This is the sixth version of the patch, having fixed issues with
compilation, rebased to the latest version, and tested by replacing
cygwin1.dll. I provide all my patches to Cygwin,
including this one and all future ones, under the 2-clause BSD license.
---
winsup/cygwin/dcrt0.cc | 299 +
winsup/cygwin/winf.cc| 351 ++-
winsup/cygwin/winf.h | 4 +-
winsup/cygwin/winsup.h | 7 +-
winsup/doc/cygwinenv.xml | 8 +-
winsup/doc/faq-api.xml | 2 +-
6 files changed, 367 insertions(+), 304 deletions(-)
diff --git a/winsup/cygwin/dcrt0.cc b/winsup/cygwin/dcrt0.cc
index 6f4723bb0..8998075a6 100644
--- a/winsup/cygwin/dcrt0.cc
+++ b/winsup/cygwin/dcrt0.cc
@@ -10,7 +10,6 @@ details. */
#include "miscfuncs.h"
#include
#include
-#include "glob.h"
#include
#include
#include
@@ -35,6 +34,7 @@ details. */
#include "cygxdr.h"
#include
#include "ntdll.h"
+#include "winf.h"
#define MAX_AT_FILE_LEVEL 10
@@ -77,292 +77,6 @@ do_global_ctors (void (**in_pfunc)(), int force)
(*pfunc) ();
}
-/*
- * Replaces @file in the command line with the contents of the file.
- * There may be multiple @file's in a single command line
- * A \@file is replaced with @file so that echo \@foo would print
- * @foo and not the contents of foo.
- */
-static bool __stdcall
-insert_file (char *name, char *)
-{
- HANDLE f;
- DWORD size;
- tmp_pathbuf tp;
-
- PWCHAR wname = tp.w_get ();
- sys_mbstowcs (wname, NT_MAX_PATH, name + 1);
- f = CreateFileW (wname,
- GENERIC_READ,/* open for reading */
- FILE_SHARE_VALID_FLAGS, /* share for reading*/
- _none_nih, /* default security */
- OPEN_EXISTING, /* existing file only */
- FILE_ATTRIBUTE_NORMAL, /* normal file */
- NULL); /* no attr. template*/
-
- if (f == INVALID_HANDLE_VALUE)
-{
- debug_printf ("couldn't open file '%s', %E", name);
- return false;
-}
-
- /* This only supports files up to about 4 billion bytes in
- size. I am making the bold assumption that this is big
- enough for this feature */
- size = GetFileSize (f, NULL);
- if (size == 0x)
-{
- CloseHandle (f);
- debug_printf ("couldn't get file size for '%s', %E", name);
- return false;
-}
-
- int new_size = strlen (cmd) + size + 2;
- char *tmp = (char *) malloc (new_size);
- if (!tmp)
-{
- CloseHandle (f);
- debug_printf ("malloc failed, %E");
- return false;
-}
-
- /* realloc passed as it should */
- DWORD rf_read;
- BOOL rf_result;
- rf_result = ReadFile (f, tmp, size, _read, NULL);
- CloseHandle (f);
- if (!rf_result || (rf_read != size))
-{
- free (tmp);
- debug_printf ("ReadFile failed, %E");
- return false;
-}
-
- tmp[size++] = ' ';
- strcpy (tmp + size, cmd);
- cmd = tmp;
- return true;
-}
-
-static inline int
-isquote (char c)
-{
- char ch = c;
- return ch == '"' || ch == '\'';
-}
-
-/* Step over a run of characters delimited by quotes */
-static /*__inline*/ char *
-quoted (char *cmd, int winshell)
-{
- char *p;
- char quote = *cmd;
-
- if (!winshell)
-{
- char *p;
- strcpy (cmd, cmd + 1);
- if (*(p = strchrnul (cmd, quote)))
- strcpy (p, p + 1);
- return p;
-}
-
- const char *s = quote == '\'' ? "'" : "\\\"";
- /* This must have been run from a Windows shell, so preserve
-
Re: [ANNOUNCEMENT] Test: {mingw64-{i686,x86_64}-,}gcc-11.1.0-0.1
Am 12.05.2021 um 20:53 schrieb Achim Gratz: Thomas Wolff writes: Am 10.05.2021 um 21:13 schrieb Achim Gratz: The native and mingw-w64 cross compilers have been updated for both architectures to the latest upstream release version: gcc-11.1.0-0.1 Are there any known problems with gcc 11? My program crashes if compiled with gcc -O2; gcc -O1 works, gcc 10 also works. None that I'd know of, especially given that very vague symptoms. While I guess it's possible that there's a bug in the optimizer, I'd first attempt to rule out undefined behaviour in the program being compiled. The crash vanishes after removing a few lines from a conditional (if block) where the condition is false. This smells like wrong calculation of a relative jump (Intel "short jump") by the optimizer. Regards, Achim. -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
