Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2024-02-01 Thread Nathan Hartman
On Thu, Feb 1, 2024 at 5:26 PM Daniel Sahlberg wrote: > > Gentlemen, > > It seems you have both had your say in what flaws there has been in the > process. Can we please leave this part of the discussion and continue on the > technical issues? I'd hate for this discussion to turn to

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2024-02-01 Thread Daniel Sahlberg
Gentlemen, It seems you have both had your say in what flaws there has been in the process. Can we please leave this part of the discussion and continue on the technical issues? I'd hate for this discussion to turn to pie-throwing where someone in the end feel offended and leave the community. We

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2024-01-18 Thread Evgeny Kotkov via dev
Daniel Sahlberg writes: > As far as I understand, the point of multi-hash is to keep the WC format > between versions (so older clients can continue to use the WC). Just as a minor note, the working copies created using the implementation on the `pristine-checksum-salt` branch don't multi-hash

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2024-01-18 Thread Branko Čibej
On 18.01.2024 08:43, Daniel Sahlberg wrote: As far as I understand, the point of multi-hash is to keep the WC format between versions (so older clients can continue to use the WC). I need some help to understand how that would work in practice. Let's say that 1.15 adds SHAABC, 1.16 adds

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2024-01-18 Thread Evgeny Kotkov via dev
Daniel Shahaf writes: > Procedurally, the long hiatus is counterproductive. This reminds me that the substantive discussion of your veto ended with my email from 8 Feb 2023 that had four direct questions to you and was left without an answer: `` > That's not how design discussions work.

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2024-01-17 Thread Daniel Sahlberg
@Karl Fogel , @Evgeny Kotkov Any chance for a comment on the questions in this thread? I've also added my own comment below. Kind regards, Daniel Den sön 14 jan. 2024 kl 00:56 skrev Nathan Hartman : > On Fri, Jan 12, 2024 at 3:51 PM Johan Corveleyn wrote: > >> On Fri, Jan 12, 2024 at

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2024-01-13 Thread Nathan Hartman
On Sat, Jan 13, 2024 at 3:56 PM Nathan Hartman wrote: > Pros: Future-proofing against the real and perceived brokenness of any > hash types. > I meant to write: Pros: Future-proofing against the real and perceived brokenness of any hash types, or the deprecation and later removal of their

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2024-01-13 Thread Nathan Hartman
On Fri, Jan 12, 2024 at 3:51 PM Johan Corveleyn wrote: > On Fri, Jan 12, 2024 at 12:37 PM Daniel Shahaf > wrote: > ... > > Procedurally, the long hiatus is counterproductive. Neither kfogel nor > > I had the context in our heads, and the cache misses took their toll in > > tuits and in

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2024-01-13 Thread Daniel Sahlberg
Den lör 13 jan. 2024 kl 00:50 skrev Johan Corveleyn : > On Fri, Jan 12, 2024 at 12:37 PM Daniel Shahaf > wrote: > ... > > Procedurally, the long hiatus is counterproductive. Neither kfogel nor > > I had the context in our heads, and the cache misses took their toll in > > tuits and in wallclock

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2024-01-12 Thread Johan Corveleyn
On Fri, Jan 12, 2024 at 12:37 PM Daniel Shahaf wrote: ... > Procedurally, the long hiatus is counterproductive. Neither kfogel nor > I had the context in our heads, and the cache misses took their toll in > tuits and in wallclock time. Furthermore, I have less spare time for > dev@ discussions

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2024-01-12 Thread Daniel Shahaf
-dev/202212.mbox/%3C904aded6-5ef0-4123-ade0-e23a3bb56726%40app.fastmail.com%3E >>>> Date: Fri, 20 Jan 2023 12:15:24 +0000 >>>> From: Daniel Shahaf >>>> To: dev@subversion.apache.org >>>> Subject: Re: Switching from SHA1 to a checksum type without kn

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2024-01-04 Thread Karl Fogel
On 04 Jan 2024, Daniel Shahaf wrote: Acknowledging receipt. I'll reply substantively when I have the time to swap in the context. Thanks. Yeah, I went through the same context-swapping-in process yesterday before posting! Best regards, -Karl Evgeny's work is on this branch...

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2024-01-04 Thread Daniel Shahaf
Karl Fogel wrote on Wed, 03 Jan 2024 22:13 +00:00: > On 01 Apr 2023, Evgeny Kotkov via dev wrote: >>Daniel Shahaf writes: >> >>> What's the question or action item to/for me? Thanks. >> >>I'm afraid I don't fully understand your question. As you >>probably remember, the change is blocked by

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2024-01-03 Thread Karl Fogel
On 01 Apr 2023, Evgeny Kotkov via dev wrote: Daniel Shahaf writes: What's the question or action item to/for me? Thanks. I'm afraid I don't fully understand your question. As you probably remember, the change is blocked by your veto. To my knowledge, this veto hasn't been revoked as of

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-04-01 Thread Evgeny Kotkov via dev
Daniel Shahaf writes: > What's the question or action item to/for me? Thanks. I'm afraid I don't fully understand your question. As you probably remember, the change is blocked by your veto. To my knowledge, this veto hasn't been revoked as of now, and I simply mentioned that in my email.

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-03-31 Thread Daniel Shahaf
Evgeny Kotkov via dev wrote on Wed, 22 Mar 2023 15:23 +00:00: > This change is still being blocked by a veto, but if danielsh changes his > mind and if there won't be other objections, I'm ready to complete the few > remaining bits and merge it to trunk. What's the question or action item to/for

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-03-22 Thread Evgeny Kotkov via dev
Evgeny Kotkov writes: > > Now, how hard would this be to actually implement? > > To have a more or less accurate estimate, I went ahead and prepared the > first-cut implementation of an approach that makes the pristine checksum > kind configurable in a working copy. > > The current

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-02-07 Thread Evgeny Kotkov via dev
Daniel Shahaf writes: > Look, it's pretty simple. You said "We should do Y because it > addresses X". You didn't explain why X needs to be addressed, didn't > consider what alternatives there are to Y, didn't consider any cons that > Y may have… and when people had questions, you just began to

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-02-06 Thread Daniel Shahaf
Evgeny Kotkov via dev wrote on Sun, Jan 29, 2023 at 16:37:20 +0300: > Daniel Shahaf writes: > > > > (I'm not saying that the above rules have to be used in this particular > > > case > > > and that a veto is invalid, but still thought it’s worth mentioning.) > > > > > > > I vetoed the change

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-02-06 Thread Daniel Shahaf
Karl Fogel wrote on Mon, Jan 30, 2023 at 17:26:03 -0600: > On 29 Jan 2023, Evgeny Kotkov via dev wrote: > > I have *absolutely* no idea where "being railroaded through" comes > > from. Really, it's a wrong way of portraying and thinking about the > > events that have happened so far. > > > >

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-02-06 Thread Daniel Shahaf
Evgeny Kotkov via dev wrote on Sun, Jan 29, 2023 at 16:36:12 +0300: > Daniel Shahaf writes: > > > > That could happen after a public disclosure of a pair of executable > > > files/scripts where the forged version allows for remote code execution. > > > Or maybe something similar with a file

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-31 Thread Karl Fogel
On 31 Jan 2023, Daniel Shahaf wrote: Karl Fogel wrote on Mon, 30 Jan 2023 23:26 +00:00: Daniel, given what's in Evgeny's branch now, could you summarize your current technical objections if any? Certainly, but I won't have time to do so today. Oh, my gosh, I'd be the last person to ever

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-31 Thread Daniel Shahaf
Karl Fogel wrote on Mon, 30 Jan 2023 23:26 +00:00: > Daniel, given what's in Evgeny's branch now, could you summarize > your current technical objections if any? Certainly, but I won't have time to do so today.

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-30 Thread Karl Fogel
On 29 Jan 2023, Evgeny Kotkov via dev wrote: I have *absolutely* no idea where "being railroaded through" comes from. Really, it's a wrong way of portraying and thinking about the events that have happened so far. Reiterating over those events: I wrote an email containing my thoughts and

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-29 Thread Evgeny Kotkov via dev
Daniel Shahaf writes: > > (I'm not saying that the above rules have to be used in this particular case > > and that a veto is invalid, but still thought it’s worth mentioning.) > > > > I vetoed the change because it hadn't been designed on the dev@ list, > had not garnered dev@'s consensus, and

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-29 Thread Evgeny Kotkov via dev
Daniel Shahaf writes: > > That could happen after a public disclosure of a pair of executable > > files/scripts where the forged version allows for remote code execution. > > Or maybe something similar with a file format that is often stored in > > repositories and that can be executed or used

Glossary of attacks (was: Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format)

2023-01-26 Thread Daniel Shahaf
Definitions of attacks: 1. Collision attack: Given h(), find x₁, x₂ such that h(x₁) == h(x₂). 2. Second preimage attack: Given h() and x, find x′ such that h(x) == h(x′). 3. First preimage attack: Given h() and y, find x such that h(x) == y. 4. Chosen prefix attack: Given

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-26 Thread Daniel Shahaf
Evgeny Kotkov via dev wrote on Mon, Jan 23, 2023 at 02:28:50 +0300: > Daniel Shahaf writes: > > > > I can complete the work on this branch and bring it to a production-ready > > > state, assuming there are no objections. > > > > Your assumption is counterfactual: > > > >

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-22 Thread Evgeny Kotkov via dev
Daniel Shahaf writes: > > I can complete the work on this branch and bring it to a production-ready > > state, assuming there are no objections. > > Your assumption is counterfactual: > >

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-22 Thread Nathan Hartman
Replying to multiple parts of this thread... On Sat, Jan 21, 2023 at 12:58 PM Karl Fogel wrote: > *nod* This issue isn't important enough to me to continue the > conversation -- I'd like for new hash algorithms to be possible, > and I think Evgeny's work on it is worthwhile, but I don't feel >

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-22 Thread Daniel Shahaf
[ tl;dr: See last paragraph for a concrete question about ra_serf. ] Karl Fogel wrote on Fri, 20 Jan 2023 17:18 +00:00: > Yes. A hash is considered "broken" the moment security researches > can generate a collision. Consider the following uses of hash functions in our code: - FSFS rep-cache

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-22 Thread Daniel Shahaf
[See below a proposal that libsvn_wc not use any fixed hash function.] Martin Edgar Furter Rathod wrote on Sat, 21 Jan 2023 05:22 +00:00: > On 20.01.23 22:48, Karl Fogel wrote: >> On 20 Jan 2023, Nathan Hartman wrote: >>> We already can't store files with identical SHA1 hashes, but AFAIK the >>>

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-22 Thread Daniel Shahaf
To be clear, I wasn't vetoing changing the hash algorithm. I was vetoing making a change without discussion. If there is discussion and it results in consensus to change the algorithm, that'll be absolutely fine by me. Daniel Karl Fogel wrote on Sat, 21 Jan 2023 17:58 +00:00: > *nod* This

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-21 Thread Karl Fogel
*nod* This issue isn't important enough to me to continue the conversation -- I'd like for new hash algorithms to be possible, and I think Evgeny's work on it is worthwhile, but I don't feel nearly as strongly about this as I feel about making the new pristineless working copies available in

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-21 Thread Daniel Shahaf
Karl Fogel wrote on Fri, Jan 20, 2023 at 11:18:56 -0600: > On 20 Jan 2023, Nathan Hartman wrote: > > Taking a step back, this discussion started because pristine-free WCs > > are IIUC more dependent on comparing hashes than pristineful WCs, and > > therefore a hash collision could have more impact

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-21 Thread Daniel Shahaf
Karl Fogel wrote on Fri, Jan 20, 2023 at 11:09:11 -0600: > On 20 Jan 2023, Daniel Shahaf wrote: > > Evgeny Kotkov via dev wrote on Thu, 19 Jan 2023 18:52 +00:00: > > > I can complete the work on this branch and bring it to a > > > production-ready > > > state, assuming there are no objections. > >

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-20 Thread Martin Edgar Furter Rathod
On 20.01.23 22:48, Karl Fogel wrote: On 20 Jan 2023, Nathan Hartman wrote: We already can't store files with identical SHA1 hashes, but AFAIK the only meaningful impact we've ever heard is that security researchers cannot track files they generate with deliberate collisions. The same would

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-20 Thread Karl Fogel
On 20 Jan 2023, Nathan Hartman wrote: Taking a step back, this discussion started because pristine-free WCs are IIUC more dependent on comparing hashes than pristineful WCs, and therefore a hash collision could have more impact in a pristine-free WC. "Guarantees" were mentioned, but I think

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-20 Thread Karl Fogel
On 20 Jan 2023, Daniel Shahaf wrote: Evgeny Kotkov via dev wrote on Thu, 19 Jan 2023 18:52 +00:00: I can complete the work on this branch and bring it to a production-ready state, assuming there are no objections. Your assumption is counterfactual:

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-20 Thread Daniel Shahaf
Nathan Hartman wrote on Fri, 20 Jan 2023 14:51 +00:00: > 1. Pros/cons of switching from SHA1 to another hash. ⋮ > Do we need to switch from SHA1 to another hash? One con that was > already mentioned [1] is that we'll never really be able to switch > away from SHA1, as there are existing clients,

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-20 Thread Nathan Hartman
On Fri, Jan 20, 2023 at 9:51 AM Nathan Hartman wrote: > > On Fri, Jan 20, 2023 at 7:18 AM Daniel Shahaf wrote: > > > > Evgeny Kotkov via dev wrote on Thu, 19 Jan 2023 18:52 +00:00: > > > I can complete the work on this branch and bring it to a production-ready > > > state, assuming there are no

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-20 Thread Nathan Hartman
On Fri, Jan 20, 2023 at 7:18 AM Daniel Shahaf wrote: > > Evgeny Kotkov via dev wrote on Thu, 19 Jan 2023 18:52 +00:00: > > I can complete the work on this branch and bring it to a production-ready > > state, assuming there are no objections. > > Your assumption is counterfactual: > >

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-20 Thread Daniel Shahaf
Evgeny Kotkov via dev wrote on Thu, 19 Jan 2023 18:52 +00:00: > I can complete the work on this branch and bring it to a production-ready > state, assuming there are no objections. Your assumption is counterfactual:

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-19 Thread Karl Fogel
On 19 Jan 2023, Evgeny Kotkov wrote: To have a more or less accurate estimate, I went ahead and prepared the first-cut implementation of an approach that makes the pristine checksum kind configurable in a working copy. The current implementation passes all tests in my environment and seems

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-19 Thread Karl Fogel
On 19 Jan 2023, Daniel Shahaf wrote: https://subversion.apache.org/security/sha1-advisory.txt That's a well-written advisory. I was surprised to see that there is no date on it, though -- from looking at the page, one would have no quick way of knowing the date it was published (although

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-19 Thread Evgeny Kotkov via dev
Karl Fogel writes: > Now, how hard would this be to actually implement? To have a more or less accurate estimate, I went ahead and prepared the first-cut implementation of an approach that makes the pristine checksum kind configurable in a working copy. The current implementation passes all

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2023-01-19 Thread Daniel Shahaf
Karl Fogel wrote on Thu, Dec 29, 2022 at 17:35:44 -0600: > On 29 Dec 2022, Evgeny Kotkov wrote: > > Karl Fogel writes: > > > > > Now, how hard would this be to actually implement? > > > > I plan to take a more detailed look at that, but I'm currently on > > vacation for the New Year holidays. >

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2022-12-29 Thread Karl Fogel
On 29 Dec 2022, Evgeny Kotkov wrote: Karl Fogel writes: Now, how hard would this be to actually implement? I plan to take a more detailed look at that, but I'm currently on vacation for the New Year holidays. That's great to hear, Evgeny. In the meantime, enjoy your vacation! Best

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2022-12-29 Thread Branko Čibej
On 28.12.2022 13:34, Daniel Sahlberg wrote: Since we need to be backwards compatible with older v1 clients, can this check ever be removed (before Subversion 2)? The case you're citing is specific to the repository, you could easily have a repository format that uses different hashes. The

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2022-12-29 Thread Evgeny Kotkov via dev
Karl Fogel writes: > Now, how hard would this be to actually implement? I plan to take a more detailed look at that, but I'm currently on vacation for the New Year holidays. Thanks, Evgeny Kotkov

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2022-12-28 Thread Karl Fogel
On 28 Dec 2022, Branko Čibej wrote: My point was that we shouldn't have to worry about format bumps as much any more because we have infrastructure in the client for supporting multiple WC formats. That includes optional pristines, different hashes, compressed pristines, etc. etc. Thank you

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2022-12-28 Thread Daniel Sahlberg
Den ons 28 dec. 2022 kl 08:48 skrev Branko Čibej : > On 27.12.2022 02:56, Karl Fogel wrote: > > Now, how hard would this be to actually implement? The > pristineless-format WC upgrade is an opportunity to make other format > changes, but I'd hate to block the release of pristineless working

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2022-12-27 Thread Branko Čibej
On 27.12.2022 02:56, Karl Fogel wrote: Now, how hard would this be to actually implement?  The pristineless-format WC upgrade is an opportunity to make other format changes, but I'd hate to block the release of pristineless working copies on this... My point was that we shouldn't have to

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format

2022-12-26 Thread Karl Fogel
On 20 Dec 2022, Evgeny Kotkov via dev wrote: [Moving discussion to a new thread] We currently have a problem that a working copy relies on the checksum type with known collisions (SHA1). A solution to that problem is to switch to a different checksum type without known collisions in one of

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format (was: Re: Getting to first release of pristines-on-demand feature (#525).)

2022-12-20 Thread Daniel Shahaf
Evgeny Kotkov via dev wrote on Tue, Dec 20, 2022 at 11:14:00 +0300: > [Moving discussion to a new thread] > > We currently have a problem that a working copy relies on the checksum type > with known collisions (SHA1). A solution to that problem Why is libsvn_wc's use of SHA-1 a problem? What's

Re: Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format (was: Re: Getting to first release of pristines-on-demand feature (#525).)

2022-12-20 Thread Branko Čibej
On 20.12.2022 09:14, Evgeny Kotkov wrote: 2) We already need a working copy format bump for the pristines-on-demand feature. So using that format bump to solve the SHA1 issue might reduce the overall number of required bumps for users (assuming that we'll still need to switch from

Switching from SHA1 to a checksum type without known collisions in 1.15 working copy format (was: Re: Getting to first release of pristines-on-demand feature (#525).)

2022-12-20 Thread Evgeny Kotkov via dev
Karl Fogel writes: > > While here, I would like to raise a topic of incorporating a switch from > > SHA1 to a different checksum type (without known collisions) for the new > > working copy format. This topic is relevant to the pristines-on-demand > > branch, because the new "is the file