Re: [fossil-users] Proposed roadmap for Fossil 2.0

[Ron Aaron]

I'm happy to see you thinking along those lines.

>From a performance standpoint, I would rather see Fossil adopt the
BLAKE2 hash, as it is one of the fastest of the SHA3 finalists, and has
adjustable output hash size.


On 27/02/2017 3:48, Richard Hipp wrote:
> On 2/26/17, Tony Papadimitriou  wrote:
>> how urgent is the need to
>> transition away from SHA1?
>>
> From a technical standpoint, it is not very urgent, in my assessment.
>
> However, from a PR standpoint, I think it needs to happen quickly.
>
> It can also be a big PR win if we are able to boast that Fossil
> transitioned away from SHA1 painlessly, quickly, and efficiently and
> without breaking any legacy.
>


*Ron Aaron | * CTO Aaron High-Tech, Ltd  | +1
425.296.0766 / +972 52.652.5543 | GnuPG Key: 91F92EB8

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Proposed roadmap for Fossil 2.0

[Richard Hipp]

On 2/26/17, Tony Papadimitriou  wrote:
>
> how urgent is the need to
> transition away from SHA1?
>

From a technical standpoint, it is not very urgent, in my assessment.

However, from a PR standpoint, I think it needs to happen quickly.

It can also be a big PR win if we are able to boast that Fossil
transitioned away from SHA1 painlessly, quickly, and efficiently and
without breaking any legacy.

-- 
D. Richard Hipp
d...@sqlite.org
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Proposed roadmap for Fossil 2.0

[Tony Papadimitriou]

Leaving aside for a moment the consequences in general of the presumed 
imminent SHA1 collapse (and some of the valid points already made by Linus 
regarding Git):


If FOSSIL will refuse (and I actually tried it with those two same SHA1 
PDFs) to accept a file (commit, push, pull) with the same SHA1 as any of 
those already in the repo (not sure about the unversioned case, however), 
how is it possible for someone to inject a 'bad' file with the same SHA1 as 
a 'good' file already in the repo?


The only ways I can imagine (and please add more if you see them) are:

* Deconstruct the repo, replace the specific file(s) with the 'bad' one(s) 
and reconstruct.  But, this would be in the user's local copy, and s/he 
would not be able to push those changes to the other side (again, because 
the given SHA1 already exists, and the file with the same SHA1 will not be 
retransmitted/reloaded).  The injection will not propagate beyond the 
attacker's machine.


* Know the 'good' file before it's actually committed, prepare a 'bad' same 
SHA1 replacement, and commit it before the 'good' has a chance, locking it 
out.  (Rather impossible even for clairvoyant people -- and even if, it 
would most likely be noticed more easily than replacing a dormant file 
nobody bothers with!)


* Be the administrator of a site (like chiselapp for example -- I do not 
mean to insinuate anything, I simply do not know of another public example) 
and go through the deconstruct-replace-reconstruct process replacing good 
with bad.  This is the only scenario I see which will affect the general 
public -- specifically, those cloning the injected repo from scratch. 
However, this again (because of no same SHA1 reloading) will not affect the 
local copies of the contributors, when pulling/syncing -- or any of the 
clones done before the injection.  This is the only one I would worry about 
at a theoretical level.


So, unless my assumptions above are incorrect, how urgent is the need to 
transition away from SHA1?


Also, the two example PDF files with the same SHA1 still have different MD5 
which fossil apparently already uses, and this (MD5) could be used as an 
alternate verifier for each artifact without changing anything else.  I 
believe it will be really-really difficult (for the foreseeable future at 
least) for someone to come up with a 'bad' file with both SHA1 and MD5 being 
the same.  Don't tell me MD5 is broken.  One would still need to match both 
SHA1 and MD5 to inject -- not easy!


I'm certainly not against transitioning to a more secure hash *eventually* 
but I doubt there is such an immediate need (until the Easter deadline, for 
example) for making what seems to be a rather serious update that (and this 
my biggest concern) may introduce (an avalanche[?] of) bugs, and possibly 
even risk the integrity of our current repos until fully bug-free.  (I for 
one would be reluctant to try it for actual work until enough other people 
have used it for some time without problems.)  So, I think it could be done 
in a more relaxed timeframe that will also give time to brainstorm the best 
possible general solution that will work easily even in the event of another 
hash function replacement in the future (e.g., what if SHA3 is already being 
prepped by Google for summer announcement?) while maintaining backwards 
compatibility to the greatest extent possible.  It's also interesting to 
take some time to see how others will try to deal with this problem and get 
ideas.


As for the proposal, although it sounds OK on first reading, the 'unknowns' 
are a bit worrisome, particularly the syncing between different versions --  
you can't really get the whole population to switch at the exact same time.


And, I'm not sure it's the minimum (i.e., less chance for new bugs) solution 
possible.  I believe the example I gave with the MD5 is safe enough 
temporary 'hack' for the foreseeable future with less possibility of bugs as 
it will not switch to a new hash, simply use the second one for extra 
verification (and it doesn't have to be MD5, you can use SHA3 but in a 
similar context -- simply MD5 is already there).


My 0.01 eurocent!

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Google Security Blog: Announcing the first SHA1 collision

[K. Fossil user]

Hello,

Does this mean that it is not so hard to adapt SHA algorithm to a better one ?:D

DRH suspected that it would be hard :D :D :D
Of course I don't agree with DRH ; I will never agree with him about security 
discuss either ... :-|
Thank to "sgbeal". :-)  
Best Regards

K.

  De : Stephan Beal 
 À : Fossil SCM user's discussion  
 Envoyé le : Dimanche 26 février 2017 21h58
 Objet : Re: [fossil-users] Google Security Blog: Announcing the first SHA1 
collision
   
On Sun, Feb 26, 2017 at 10:34 PM, Richard Hipp  wrote:

And in any event, I don't think centralization is a factor here.
Fossil is better positioned than Git or Mercurial to transition to a
different hash algorithm because the Fossil implementation uses a
relational database as its backing store.  Git and Hg, in contrast,
both use bespoke pile-of-files database formats which, I suspect, will
be more difficult to adapt.


just FYI, Linus' own words on the topic, posted yesterday:
https://plus.google.com/u/0/+LinusTorvalds/posts/7tp2gYWQugL
-- 
- stephan beal
http://wanderinghorse.net/home/stephan/"Freedom is sloppy. But since tyranny's 
the only guaranteed byproduct of those who insist on a perfect world, freedom 
will have to do." -- Bigby Wolf

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


   ___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Google Security Blog: Announcing the first SHA1 collision

[Stephan Beal]

On Sun, Feb 26, 2017 at 10:34 PM, Richard Hipp  wrote:

> And in any event, I don't think centralization is a factor here.
> Fossil is better positioned than Git or Mercurial to transition to a
> different hash algorithm because the Fossil implementation uses a
> relational database as its backing store.  Git and Hg, in contrast,
> both use bespoke pile-of-files database formats which, I suspect, will
> be more difficult to adapt.
>

just FYI, Linus' own words on the topic, posted yesterday:

https://plus.google.com/u/0/+LinusTorvalds/posts/7tp2gYWQugL

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Google Security Blog: Announcing the first SHA1 collision

[Richard Hipp]

On 2/23/17, Warren Young  wrote:
>
> I think Fossil is in a much better position to do this sort of migration
> than, say, Git, due to its semi-centralized nature.

Though they are technically distinct, in the minds of many users Git
and GitHub are the same thing.  And GitHub is highly centralized.  So
it is reasonable to argue that Git(Hub) is more centralized than
Fossil.

And in any event, I don't think centralization is a factor here.
Fossil is better positioned than Git or Mercurial to transition to a
different hash algorithm because the Fossil implementation uses a
relational database as its backing store.  Git and Hg, in contrast,
both use bespoke pile-of-files database formats which, I suspect, will
be more difficult to adapt.

-- 
D. Richard Hipp
d...@sqlite.org
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Google Security Blog: Announcing the first SHA1 collision

[Ron W]

On Thu, Feb 23, 2017 at 11:23 PM,  wrote:
>
> Date: Fri, 24 Feb 2017 04:23:06 + (UTC)
> From: "K. Fossil user" 
> To: Fossil SCM user's discussion 
> Subject:
> 2/ semi?
>
> > « I think Fossil is in a much better position to do this sort of
> migration than, say, Git, due to its semi-centralized nature »
> This would convince people to use Git not Fossil ...
>
> Git is more secure than Fossil (first reason to use a VCS)Git could be
> centralized or not. I am wondering if Fossil could be centralized... Now
> you've said that it is semi-centralized by NATURE.
>

git and Fossil are equally decentralized. Both are DVCSs.

The "semi-centralized nature" really refers to the git community coalescing
around huge repository services like GitHub.

Fossil can also be organised around repository services. chisselapp.org is
a dedicated Fossil repository service. There are some repository services,
like SourceForge, that off several VCS options, including git and Fossil,

FYI, for most organizational purposes, projects tend to "revolve" around a
"central" master repository (or a central cluster of redundant master
repositories). This is equally true for both git and Fossil.

However, truly peer-to-peer relations between developer repositories can be
setup. This setup is basically the same as a central cluster, except that
each member of the cluster is used directly by members of the development
team.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Google Security Blog: Announcing the first SHA1 collision

[Ron W]

On Thu, Feb 23, 2017 at 7:02 PM, 
wrote:
>
> Date: Thu, 23 Feb 2017 17:01:56 -0700
> From: Warren Young 
> Subject: Re: [fossil-users] Google Security Blog: Announcing the first
> SHA1 collision
>
> The PHC scheme would allow Fossil to migrate to something stronger in a
> backwards-compatible fashion:
>
>https://github.com/P-H-C/phc-string-format/blob/master/phc-sf-spec.md
>
> That is, if the hash argument in the F, P, and Q cards is not 40
> characters and it has a suitable prefix, it’s a new-style hash, else it’s a
> legacy SHA-1 hash.
>

The PHC scheme is conceptually good, but is not friendly for use by command
line tools like Fossil or git. This is mostly because it uses $ as its
field introducer, so will quoting. Also, the Base64 encoding relies on both
upper and lowercase letters, so is more prone to typographical errors.

I suggest a simpler scheme that provides the benefits of PHC in a more
command line friendly way.

Use ^ as the prefix and data introducers. The prefix would have a 1
character field for the artifact type, followed by the nonce. Then a second
^ separates the prefix from the data, which will be the hash. Base64
encoding would make the hash string use fewer characters while continuing
to use the hexadecimal encoding would be less prone to typographical errors.

Example: ^m1234567890^ab4c90e2.

m is the artifact type. Suggest m for manifest, c for control, etc.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] Proposed roadmap for Fossil 2.0

[Richard Hipp]

This message is cross-posted to fossil-users and fossil-dev.
Follow-ups should go to fossil-dev only, please.  Thanks.

I propose that the next release of Fossil be called "Fossil 2.0", that
it occur before Easter (2017-04-16), and that it have the following
features:

(1) Fossil 2.0 is backwards compatible with Fossil 1.x.  Fossil 2.0
can push and pull from a Fossil 1.x server.  Fossil 2.0 can read and
write Fossil 1.x repositories, though only after having run "fossil
rebuild".  The upgrade path is to first overwrite the older fossil 1.x
executable with a new fossil 2.0 executable, then run "fossil all
rebuild".

(2) Artifacts can be identified via multiple hash algorithms.  The
initial implementation will support SHA1 and SHA3-228.  (For brevity,
SHA3-228 will hereafter be referred to as K228.)

(3) The low-level file formats
(https://www.fossil-scm.org/fossil/doc/trunk/www/fileformat.wiki) are
unchanged except that the artifact hashes are allowed to be longer
than 40 hex digits for alternative hash algorithms.  For K228, the
hashes are 56 hex digits long.  Other hash algorithms may be supported
in future releases as long as each hash algorithm has a unique hash
length, thus enabling Fossil to figure out which algorithm is being
used simply by looking at the length of the hash.

(4) All artifact hashes within a single well-formed structure artifact
must use the same algorithm.  This restriction does not apply to the
MD5 hash used by the R-card and the Z-card.

(5) Every repository will have a preferred hash algorithm.  The
preferred hash algorithm can be changed by running "fossil rebuild"
with appropriate options. The artifact hashes displayed in the web
interface and on command-line output will be computed using the
preferred hash algorithm.  This means that the displayed hash names
for legacy check-ins will change when the hash algorithm is changed.
However, references to the old hash values will still be correctly
resolved.

For example, the current tip of trunk in the Fossil self-hosting
repository is named using a SHA1 hash as:
ccdafa2a93e7bcefa1b4d0ea7474f9ce84c690f2.  If the hash algorithm is
changed to K228, then this check-in will afterwards be displayed as
3c658054301feb7e1cd25b66e32c94ffbf48d0b2f67310d33fb79a50.  However,
you will still be able to access the check-in using the
"https://www.fossil-scm.org/fossil/info/ccdafa2a93e7bcef; URL and you
will still be able to update to that check-in by typing "fossil update
ccdafa2a".  In this way, a repository can transition from one hash
algorithm to another without breaking any legacy hyperlinks.

(6) Repositories can be configured to reject check-ins and other
structure artifacts that occur after a selected cut-off date and which
use the SHA1 hash algorithm.

(7) To implement the above, the BLOB.UUID field will be removed from
the repository database.  In its place, a new table will be added,
tentatively declared as follows:

 CREATE TABLE hname(
hash TEXT,
alg ANY,
rid INTEGER REFERENCES blob(rid),
aux ANY,
PRIMARY KEY(hash,alg)
 ) WITHOUT ROWID;
 CREATE INDEX hname_rid ON hname(rid);

In Fossil 1.x, there was a 1-to-1 correspondence between hash values
and artifacts.  Since it supports multiple hash algorithms, Fossil 2.0
now has a many-to-one relationship between hash values and artifacts,
and so the hash values need to be stored in a separate table.  The
"alg" field will be a numeric 0 for the preferred hash, and some other
code (yet to be decided) for alternative hashes.  Note that this new
table can also store git-style artifact hashes which would facilitate
creating a Fossil-to-Git bridge that enables a Fossil server to
directly respond to push/pull requests from Git clients using the Git
wire protocol.  The "aux" field is included in anticipation of this
Fossil-to-Git bridge.  For now, the "aux" field will always be NULL.
This Fossil-to-Git bridge will not be available in the first release
but might be a feature added in subsequent releases.

I believe that most of the work in creating Fossil 2.0 will involve
going through the source code, locating queries that use BLOB.UUID,
and revising those queries to use the HNAME table instead.

Unknowns:

(8) Is it possible for two Fossil servers to sync if they are using
different preferred hash algorithms?   This is a desired goal, but I
do not yet understand how hard that will be.

(9) Can a Fossil 1.x client push/pull/clone from a Fossil 2.0 server,
assuming the repository uses SHA1 has it preferred hash algorithm?
This is desirable, but I am willing to sacrifice this capability in
order to reduce complexity.

(10) Should Keccak hashes that are not part of the SHA3 standard
(example: Keccak[196]) be supported?  K196 is desirable in that its
hash length is 48 bytes, only 8 bytes longer than SHA1.

Feedback is welcomed and encouraged, though let's keep the discussion
on fossil-dev and off of fossil-users if possible.  Thanks.
-- 
D. Richard 

Re: [fossil-users] Google Security Blog: Announcing the first SHA1 collision

[Ron W]

On Fri, Feb 24, 2017 at 5:54 PM, 
wrote:
>
> Date: Fri, 24 Feb 2017 20:38:48 +0100
> From: Joerg Sonnenberger 
> Subject: Re: [fossil-users] Google Security Blog: Announcing the first
> SHA1 collision
>
> On Fri, Feb 24, 2017 at 10:32:20AM -0800, bch wrote:
> > Are you saing:
> >
> > contenthash = sha256(content);
> > identifier = sha256 (contenthash . blobtype . conentsize . content);
> >
> > "blobtype" == cardtype ?
>
> Yes.
>

Wouldn't it be artifact type? (manifest, control, etc.)  rather than card
type?
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users