Re: Subversion crashes on list with an empty format 3 repo
Am 2016-02-07 um 01:22 schrieb Daniel Shahaf: Michael Osipov wrote on Fri, Feb 05, 2016 at 23:34:16 +0100: Am 2016-02-05 um 22:10 schrieb Ivan Zhakov: 'vn: E160033: Invalid name for FS type 'fsfs ]]] We know now that the repo is mangled. The output of the last line is broken btw. It should read "svn:... 'fsfs'" but it reads "'vn:...'fsfs". Shall a I create a JIRA issue for that? Yes, please. We should be escaping the \r character rather than printing it literally. Done: https://issues.apache.org/jira/browse/SVN-4620 The root problem is that we just use the equivalent of apr_psprintf("The value is '%s'", value) to enter replaceables into error messages; we don't escape the argument to %s, nor do we use a format code (like the %r and {!r} of Python's string formatting or the %q of the printf(1) builtin of bash and zsh) that implements single-quoting-and-escaping itself. >>> print("The value is '%s'." % ('fsfs\x0D',)) '.e value is 'fsfs >>> print("The value is %r." % ('fsfs\x0D',)) The value is 'fsfs\r'. Does 'svnadmin create' use CRLF for the fs-type file on windows? It should probably use LF to make the format entirely platform-independent. Not, it does not. Just created a repo on Windows 10. All internal files are LF line ending. Additionally, I just created a VM with Windows 7 SP1 32 bit and installed Subversion 1.9.3 and the test repo. The svn client crashes here too. For what it's worth, I wouldn't call it a "crash": it's not an abnormal program terminal (such as an exception or a signal) but a metadata validation doing its job. I do not understand why it is not failing for you. Can you retry with the binaries provided by CollabNet? Or any other idea why there is a disparity? Subversion doesn't request binary mode for opening/reading the fs-type file (doesn't request APR_BINARY which, I imagine, is a portable wrapper for the "b" mode flag of fopen()). Could that explain why you see different binaries behaving differently? I am not sure that I really understand the question. It shouldn't be any different for different binaries on the same OS as long as you don't have defines which change the behavior at runtime. Are there any? Isn't it worth to file any issue which makes Subversion to able to read repos in either file encoding but write only Unix style? Michael
Re: Subversion crashes on list with an empty format 3 repo
Am 2016-02-09 um 21:12 schrieb Branko Čibej: On 09.02.2016 17:42, Michael Osipov wrote: Isn't it worth to file any issue which makes Subversion to able to read repos in either file encoding but write only Unix style? We should read files that we expect the user to edit -- e.g., fsfs.conf -- in text mode. Revision files, however, are not text, and there's no reason to add complexity to FSFS to "solve" cases where some tool decides to mangle things that look like line endings in those files. Agreed. Maybe 'svnadmin verify' could handle such issues and print an approriate warning. In this specific case, if you have a Subversion repository inside a Git repo, well, just make sure your Git options don't mangle the repository contents. This is what I was about to do. I need to file an issue with Maven Wagon anyway. Michael
Re: Subversion crashes on list with an empty format 3 repo
On 09.02.2016 17:42, Michael Osipov wrote: > Isn't it worth to file any issue which makes Subversion to able to > read repos in either file encoding but write only Unix style? We should read files that we expect the user to edit -- e.g., fsfs.conf -- in text mode. Revision files, however, are not text, and there's no reason to add complexity to FSFS to "solve" cases where some tool decides to mangle things that look like line endings in those files. In this specific case, if you have a Subversion repository inside a Git repo, well, just make sure your Git options don't mangle the repository contents. -- Brane
Re: Subversion crashes on list with an empty format 3 repo
Bert Huijben wrote on Sun, Feb 07, 2016 at 15:38:42 +0100: > It is probably nice to fix this specific case as it is above the FS > layer, but I don't think we should really start to look at fsfs as EOL > agnostic. I wasn't proposing to make FSFS EOL agnostic. I was proposing to make the error message ungarbled.
RE: Subversion crashes on list with an empty format 3 repo
> -Original Message- > From: Daniel Shahaf [mailto:danie...@apache.org] > Sent: zondag 7 februari 2016 01:22 > To: Michael Osipov <micha...@apache.org> > Cc: Ivan Zhakov <i...@visualsvn.com>; users@subversion.apache.org > Subject: Re: Subversion crashes on list with an empty format 3 repo > > Michael Osipov wrote on Fri, Feb 05, 2016 at 23:34:16 +0100: > > Am 2016-02-05 um 22:10 schrieb Ivan Zhakov: > > >'vn: E160033: Invalid name for FS type 'fsfs > > >]]] > > We know now that the repo is mangled. The output of the last line is > broken > > btw. It should read "svn:... 'fsfs'" but it reads "'vn:...'fsfs". Shall a I > > create a JIRA issue for that? > > Yes, please. We should be escaping the \r character rather than > printing it literally. > > The root problem is that we just use the equivalent of apr_psprintf("The > value is '%s'", value) to enter replaceables into error messages; we > don't escape the argument to %s, nor do we use a format code (like > the %r and {!r} of Python's string formatting or the %q of the printf(1) > builtin of bash and zsh) that implements single-quoting-and-escaping itself. > > >>> print("The value is '%s'." % ('fsfs\x0D',)) > '.e value is 'fsfs > >>> print("The value is %r." % ('fsfs\x0D',)) > The value is 'fsfs\r'. > > > I have unzipped the repo on: > > FreeBSD bsd10 10.2-RELEASE FreeBSD 10.2-RELEASE #0 r28: Wed Aug > 12 > > 15:26:37 UTC 2015 > r...@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC > > amd64 > > > > and I receive the very same error as you do. > > That's not surprising: the fs-type file has a CRLF line ending, and on > FreeBSD the CR wouldn't be removed. > > Does 'svnadmin create' use CRLF for the fs-type file on windows? It > should probably use LF to make the format entirely platform-independent. No, it uses a LF only, like on other platforms. (I think that should be clear about this bug report... We fail when we have a file with that byte sequence we never create ourselves). We open most text files as binary in the libsvn_* code on Windows. (Note that we use the other default in our python testsuite) It is probably nice to fix this specific case as it is above the FS layer, but I don't think we should really start to look at fsfs as EOL agnostic. Bert
Re: Subversion crashes on list with an empty format 3 repo
Michael Osipov wrote on Fri, Feb 05, 2016 at 23:34:16 +0100: > Am 2016-02-05 um 22:10 schrieb Ivan Zhakov: > >'vn: E160033: Invalid name for FS type 'fsfs > >]]] > We know now that the repo is mangled. The output of the last line is broken > btw. It should read "svn:... 'fsfs'" but it reads "'vn:...'fsfs". Shall a I > create a JIRA issue for that? Yes, please. We should be escaping the \r character rather than printing it literally. The root problem is that we just use the equivalent of apr_psprintf("The value is '%s'", value) to enter replaceables into error messages; we don't escape the argument to %s, nor do we use a format code (like the %r and {!r} of Python's string formatting or the %q of the printf(1) builtin of bash and zsh) that implements single-quoting-and-escaping itself. >>> print("The value is '%s'." % ('fsfs\x0D',)) '.e value is 'fsfs >>> print("The value is %r." % ('fsfs\x0D',)) The value is 'fsfs\r'. > I have unzipped the repo on: > FreeBSD bsd10 10.2-RELEASE FreeBSD 10.2-RELEASE #0 r28: Wed Aug 12 > 15:26:37 UTC 2015 r...@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC > amd64 > > and I receive the very same error as you do. That's not surprising: the fs-type file has a CRLF line ending, and on FreeBSD the CR wouldn't be removed. Does 'svnadmin create' use CRLF for the fs-type file on windows? It should probably use LF to make the format entirely platform-independent. > Additionally, I just created a VM with Windows 7 SP1 32 bit and installed > Subversion 1.9.3 and the test repo. The svn client crashes here too. For what it's worth, I wouldn't call it a "crash": it's not an abnormal program terminal (such as an exception or a signal) but a metadata validation doing its job. > I do not understand why it is not failing for you. Can you retry with the > binaries provided by CollabNet? Or any other idea why there is a disparity? Subversion doesn't request binary mode for opening/reading the fs-type file (doesn't request APR_BINARY which, I imagine, is a portable wrapper for the "b" mode flag of fopen()). Could that explain why you see different binaries behaving differently? Cheers, Daniel
Re: Subversion crashes on list with an empty format 3 repo
Am 2016-02-06 um 11:43 schrieb Ivan Zhakov: On 6 February 2016 at 01:49, Michael Osipovwrote: Am 2016-02-05 um 22:10 schrieb Ivan Zhakov: On 5 February 2016 at 23:48, Michael Osipov wrote: Am 2016-02-05 um 20:31 schrieb Ivan Zhakov: On 5 February 2016 at 21:38, Michael Osipov wrote: Hi folks, Subversion advised me to report this. The following command was performed: svn --non-interactive list file:///D:/Entwicklung/Projekte/maven-wagon/wagon-providers/wagon-scm/src/test/resources/test-repo-svn The repo is available here [1], the crash files are here [2]. The issue can always be reproduced, it does not require to run the tests with Maven on Maven Wagon. I cannot reproduce: [[[ svn --non-interactive file:///D:/Ivan/temp/repos/test-repo-svn dummy C:\Program Files\VisualSVN Server\bin>svn --version svn, version 1.9.3 (r1718519) compiled Dec 15 2015, 11:23:08 on x86-microsoft-windows6.1.7601 After further testing, I think I know why it does not fail for you. You have downloaded a tarball snapshot of the repo via Gitweb. I have retried with that one and Subversion does not crash. Aifter that, I ran SHA1 on all files, cloned via Git and from the tarball, they are different because Git transforms the db files and aligned the line ending. svnadmin verify crashes too. These might shred the repo: core.autocrlf=true core.eol=native core.safecrlf=true I have uploaded both repos to [1] as ZIP files. Can you kindly retry with a cloned repo or the ZIP files and report back? [1] http://home.apache.org/~michaelo/svn-crash-2016-02-05/ I downloaded archive and now I'm getting error about wrong FS type as expected: [[[ C:\Program Files\VisualSVN Server\bin> --non-interactive file:///D:/Ivan/temp/repos/test-repo-svn svn: E170013: Unable to connect to a repository at URL 'file:///D:/Ivan/temp/repos/test-repo-svn' svn: E180001: Unable to open repository 'file:///D:/Ivan/temp/repos/test-repo-svn' 'vn: E160033: Invalid name for FS type 'fsfs ]]] Picking up my previous response, I just tried binaries from VisualSVN (probably the one you are using) and from SlikSVN. Both are working...is that now an issue I have to raise with CollabNet or some hidden bug in Subversion itself? It could be some issue in CollabNet distribution, but we cannot investigate without debug symbols of Collabnet Subversion binaries. thank you Ivan, I posted the issue in the CollabNet forums [1] and will wait for a response. I will keep you updated. Michael [1] https://subversion.open.collab.net/ds/viewMessage.do?dsForumId=4=573294
Re: Subversion crashes on list with an empty format 3 repo
On 6 February 2016 at 01:49, Michael Osipovwrote: > Am 2016-02-05 um 22:10 schrieb Ivan Zhakov: >> >> On 5 February 2016 at 23:48, Michael Osipov wrote: >>> >>> Am 2016-02-05 um 20:31 schrieb Ivan Zhakov: On 5 February 2016 at 21:38, Michael Osipov wrote: > > > Hi folks, > > Subversion advised me to report this. > > The following command was performed: > svn --non-interactive list > > > file:///D:/Entwicklung/Projekte/maven-wagon/wagon-providers/wagon-scm/src/test/resources/test-repo-svn > > The repo is available here [1], the crash files are here [2]. > The issue can always be reproduced, it does not require to run the > tests > with Maven on Maven Wagon. > I cannot reproduce: [[[ svn --non-interactive file:///D:/Ivan/temp/repos/test-repo-svn dummy C:\Program Files\VisualSVN Server\bin>svn --version svn, version 1.9.3 (r1718519) compiled Dec 15 2015, 11:23:08 on x86-microsoft-windows6.1.7601 >>> >>> >>> >>> After further testing, I think I know why it does not fail for you. You >>> have >>> downloaded a tarball snapshot of the repo via Gitweb. I have retried with >>> that one and Subversion does not crash. Aifter that, I ran SHA1 on all >>> files, cloned via Git and from the tarball, they are different because >>> Git >>> transforms the db files and aligned the line ending. >>> >>> svnadmin verify crashes too. >>> >>> These might shred the repo: >>> core.autocrlf=true >>> core.eol=native >>> core.safecrlf=true >>> >>> I have uploaded both repos to [1] as ZIP files. >>> >>> Can you kindly retry with a cloned repo or the ZIP files and report back? >>> >>> [1] http://home.apache.org/~michaelo/svn-crash-2016-02-05/ >>> >> I downloaded archive and now I'm getting error about wrong FS type as >> expected: >> [[[ >> C:\Program Files\VisualSVN Server\bin> --non-interactive >> file:///D:/Ivan/temp/repos/test-repo-svn >> svn: E170013: Unable to connect to a repository at URL >> 'file:///D:/Ivan/temp/repos/test-repo-svn' >> svn: E180001: Unable to open repository >> 'file:///D:/Ivan/temp/repos/test-repo-svn' >> 'vn: E160033: Invalid name for FS type 'fsfs >> ]]] > > > Picking up my previous response, I just tried binaries from VisualSVN > (probably the one you are using) and from SlikSVN. Both are working...is > that now an issue I have to raise with CollabNet or some hidden bug in > Subversion itself? > It could be some issue in CollabNet distribution, but we cannot investigate without debug symbols of Collabnet Subversion binaries. -- Ivan Zhakov
Re: Subversion crashes on list with an empty format 3 repo
On 5 February 2016 at 21:38, Michael Osipovwrote: > Hi folks, > > Subversion advised me to report this. > > The following command was performed: > svn --non-interactive list > file:///D:/Entwicklung/Projekte/maven-wagon/wagon-providers/wagon-scm/src/test/resources/test-repo-svn > > The repo is available here [1], the crash files are here [2]. > The issue can always be reproduced, it does not require to run the tests > with Maven on Maven Wagon. > I cannot reproduce: [[[ svn --non-interactive file:///D:/Ivan/temp/repos/test-repo-svn dummy C:\Program Files\VisualSVN Server\bin>svn --version svn, version 1.9.3 (r1718519) compiled Dec 15 2015, 11:23:08 on x86-microsoft-windows6.1.7601 Copyright (C) 2015 The Apache Software Foundation. This software consists of contributions made by many people; see the NOTICE file for more information. Subversion is open source software, see http://subversion.apache.org/ The following repository access (RA) modules are available: * ra_svn : Module for accessing a repository using the svn network protocol. - handles 'svn' scheme * ra_local : Module for accessing a repository on local disk. - handles 'file' scheme * ra_serf : Module for accessing a repository via WebDAV protocol using serf. - using serf 1.3.8 (compiled with 1.3.8) - handles 'http' scheme - handles 'https' scheme The following authentication credential caches are available: * Wincrypt cache in C:\Users\ivan\AppData\Roaming\Subversion ]]] -- Ivan Zhakov
Re: Subversion crashes on list with an empty format 3 repo
On 5 February 2016 at 23:48, Michael Osipovwrote: > Am 2016-02-05 um 20:31 schrieb Ivan Zhakov: >> >> On 5 February 2016 at 21:38, Michael Osipov wrote: >>> >>> Hi folks, >>> >>> Subversion advised me to report this. >>> >>> The following command was performed: >>> svn --non-interactive list >>> >>> file:///D:/Entwicklung/Projekte/maven-wagon/wagon-providers/wagon-scm/src/test/resources/test-repo-svn >>> >>> The repo is available here [1], the crash files are here [2]. >>> The issue can always be reproduced, it does not require to run the tests >>> with Maven on Maven Wagon. >>> >> I cannot reproduce: >> [[[ >> svn --non-interactive file:///D:/Ivan/temp/repos/test-repo-svn >> dummy >> >> C:\Program Files\VisualSVN Server\bin>svn --version >> svn, version 1.9.3 (r1718519) >> compiled Dec 15 2015, 11:23:08 on x86-microsoft-windows6.1.7601 > > > After further testing, I think I know why it does not fail for you. You have > downloaded a tarball snapshot of the repo via Gitweb. I have retried with > that one and Subversion does not crash. Aifter that, I ran SHA1 on all > files, cloned via Git and from the tarball, they are different because Git > transforms the db files and aligned the line ending. > > svnadmin verify crashes too. > > These might shred the repo: > core.autocrlf=true > core.eol=native > core.safecrlf=true > > I have uploaded both repos to [1] as ZIP files. > > Can you kindly retry with a cloned repo or the ZIP files and report back? > > [1] http://home.apache.org/~michaelo/svn-crash-2016-02-05/ > I downloaded archive and now I'm getting error about wrong FS type as expected: [[[ C:\Program Files\VisualSVN Server\bin> --non-interactive file:///D:/Ivan/temp/repos/test-repo-svn svn: E170013: Unable to connect to a repository at URL 'file:///D:/Ivan/temp/repos/test-repo-svn' svn: E180001: Unable to open repository 'file:///D:/Ivan/temp/repos/test-repo-svn' 'vn: E160033: Invalid name for FS type 'fsfs ]]] -- Ivan Zhakov
Re: Subversion crashes on list with an empty format 3 repo
Am 2016-02-05 um 22:10 schrieb Ivan Zhakov: On 5 February 2016 at 23:48, Michael Osipovwrote: Am 2016-02-05 um 20:31 schrieb Ivan Zhakov: On 5 February 2016 at 21:38, Michael Osipov wrote: Hi folks, Subversion advised me to report this. The following command was performed: svn --non-interactive list file:///D:/Entwicklung/Projekte/maven-wagon/wagon-providers/wagon-scm/src/test/resources/test-repo-svn The repo is available here [1], the crash files are here [2]. The issue can always be reproduced, it does not require to run the tests with Maven on Maven Wagon. I cannot reproduce: [[[ svn --non-interactive file:///D:/Ivan/temp/repos/test-repo-svn dummy C:\Program Files\VisualSVN Server\bin>svn --version svn, version 1.9.3 (r1718519) compiled Dec 15 2015, 11:23:08 on x86-microsoft-windows6.1.7601 After further testing, I think I know why it does not fail for you. You have downloaded a tarball snapshot of the repo via Gitweb. I have retried with that one and Subversion does not crash. Aifter that, I ran SHA1 on all files, cloned via Git and from the tarball, they are different because Git transforms the db files and aligned the line ending. svnadmin verify crashes too. These might shred the repo: core.autocrlf=true core.eol=native core.safecrlf=true I have uploaded both repos to [1] as ZIP files. Can you kindly retry with a cloned repo or the ZIP files and report back? [1] http://home.apache.org/~michaelo/svn-crash-2016-02-05/ I downloaded archive and now I'm getting error about wrong FS type as expected: [[[ C:\Program Files\VisualSVN Server\bin> --non-interactive file:///D:/Ivan/temp/repos/test-repo-svn svn: E170013: Unable to connect to a repository at URL 'file:///D:/Ivan/temp/repos/test-repo-svn' svn: E180001: Unable to open repository 'file:///D:/Ivan/temp/repos/test-repo-svn' 'vn: E160033: Invalid name for FS type 'fsfs ]]] We know now that the repo is mangled. The output of the last line is broken btw. It should read "svn:... 'fsfs'" but it reads "'vn:...'fsfs". Shall a I create a JIRA issue for that? I have unzipped the repo on: FreeBSD bsd10 10.2-RELEASE FreeBSD 10.2-RELEASE #0 r28: Wed Aug 12 15:26:37 UTC 2015 r...@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 and I receive the very same error as you do. Additionally, I just created a VM with Windows 7 SP1 32 bit and installed Subversion 1.9.3 and the test repo. The svn client crashes here too. I do not understand why it is not failing for you. Can you retry with the binaries provided by CollabNet? Or any other idea why there is a disparity? Addendum: I have tried the MaxSVN distro recently announced on this list: D:\Entwicklung\Projekte\test-repo-svn>C:\Users\mosipov\Documents\maxsvn_1.9.3-1\svn --version svn, version 1.9.3 (r1718519) compiled Jan 19 2016, 21:32:35 on x86_64-microsoft-windows10.0.10240 ... The following authentication credential caches are available: * Wincrypt cache in C:\Users\mosipov\AppData\Roaming\Subversion D:\Entwicklung\Projekte\test-repo-svn>C:\Users\mosipov\Documents\maxsvn_1.9.3-1\svn ls file:///D:/Entwicklung/Projekte/test-repo-svn svn: E170013: Unable to connect to a repository at URL 'file:///D:/Entwicklung/Projekte/test-repo-svn' svn: E180001: Unable to open repository 'file:///D:/Entwicklung/Projekte/test-repo-svn' 'vn: E160033: Unknown FS type 'fsfs Are the CollabNet binaries incorrectly compiled? Michael
Re: Subversion crashes on list with an empty format 3 repo
Am 2016-02-05 um 22:10 schrieb Ivan Zhakov: On 5 February 2016 at 23:48, Michael Osipovwrote: Am 2016-02-05 um 20:31 schrieb Ivan Zhakov: On 5 February 2016 at 21:38, Michael Osipov wrote: Hi folks, Subversion advised me to report this. The following command was performed: svn --non-interactive list file:///D:/Entwicklung/Projekte/maven-wagon/wagon-providers/wagon-scm/src/test/resources/test-repo-svn The repo is available here [1], the crash files are here [2]. The issue can always be reproduced, it does not require to run the tests with Maven on Maven Wagon. I cannot reproduce: [[[ svn --non-interactive file:///D:/Ivan/temp/repos/test-repo-svn dummy C:\Program Files\VisualSVN Server\bin>svn --version svn, version 1.9.3 (r1718519) compiled Dec 15 2015, 11:23:08 on x86-microsoft-windows6.1.7601 After further testing, I think I know why it does not fail for you. You have downloaded a tarball snapshot of the repo via Gitweb. I have retried with that one and Subversion does not crash. Aifter that, I ran SHA1 on all files, cloned via Git and from the tarball, they are different because Git transforms the db files and aligned the line ending. svnadmin verify crashes too. These might shred the repo: core.autocrlf=true core.eol=native core.safecrlf=true I have uploaded both repos to [1] as ZIP files. Can you kindly retry with a cloned repo or the ZIP files and report back? [1] http://home.apache.org/~michaelo/svn-crash-2016-02-05/ I downloaded archive and now I'm getting error about wrong FS type as expected: [[[ C:\Program Files\VisualSVN Server\bin> --non-interactive file:///D:/Ivan/temp/repos/test-repo-svn svn: E170013: Unable to connect to a repository at URL 'file:///D:/Ivan/temp/repos/test-repo-svn' svn: E180001: Unable to open repository 'file:///D:/Ivan/temp/repos/test-repo-svn' 'vn: E160033: Invalid name for FS type 'fsfs ]]] Picking up my previous response, I just tried binaries from VisualSVN (probably the one you are using) and from SlikSVN. Both are working...is that now an issue I have to raise with CollabNet or some hidden bug in Subversion itself? Michael