Re: [Wikitech-l] SHA-1 hash officially broken

[Strainu]

2017-02-24 21:29 GMT+02:00 Florian Schmidt :
> About the git part: There's an interesting statement from Linus [1] with 
> which I can totally agree (and additionally this one[2]). So probably even 
> this is not a "high priority, do nothing else" task :P

Just FYI, this was confirmed in [3]

>
> [1] http://marc.info/?l=git=148787047422954
> [2] 
> https://public-inbox.org/git/pine.lnx.4.58.0504291221250.18...@ppc970.osdl.org/
[3] https://plus.google.com/+LinusTorvalds/posts/7tp2gYWQugL

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] SHA-1 hash officially broken

[Gergo Tisza]

On Fri, Feb 24, 2017 at 11:20 AM, Chad  wrote:

> But, like you said (and Dan pointed out on the task), this is far from a
> drop
> everything situation...we just need to start making some inroads to this
> sooner rather than later. Since it's broken, making it faster is an
> optimization
> problem so the writing is on the wall.
>

Well, technically it was broken in 2012, optimization is what we are seeing
now.
Google claims files generated with this technique are detectable and they
will publish the detection library soon, so in the short term we just need
to install that as a filter for file uploads.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] SHA-1 hash officially broken

[Florian Schmidt]

About the git part: There's an interesting statement from Linus [1] with which 
I can totally agree (and additionally this one[2]). So probably even this is 
not a "high priority, do nothing else" task :P

[1] http://marc.info/?l=git=148787047422954
[2] 
https://public-inbox.org/git/pine.lnx.4.58.0504291221250.18...@ppc970.osdl.org/

-Ursprüngliche Nachricht-
Von: Wikitech-l [mailto:wikitech-l-boun...@lists.wikimedia.org] Im Auftrag von 
Chad
Gesendet: Freitag, 24. Februar 2017 20:21
An: Wikimedia developers <wikitech-l@lists.wikimedia.org>
Betreff: Re: [Wikitech-l] SHA-1 hash officially broken

On Fri, Feb 24, 2017 at 10:57 AM Brion Vibber <bvib...@wikimedia.org> wrote:

> Added a tracking/epic task on phab:
> https://phabricator.wikimedia.org/T158986
>
> Attach any specific things to that as subtasks, y'all.
>
>
Thanks for getting the discussion started. I was talking about this yesterday 
with some people and yeah, file and revision sha1s immediately came to mind.

Also possibly a future issue: Git--but we'll mostly want to wait for various 
upstreams to make progress here. There's some interesting thoughts over at this 
fork[0] (h/t to Tim)

But, like you said (and Dan pointed out on the task), this is far from a drop 
everything situation...we just need to start making some inroads to this sooner 
rather than later. Since it's broken, making it faster is an optimization 
problem so the writing is on the wall.

-Chad

[0]
https://github.com/peff/git/commit/b55aa200a0b3af58dc9e0accadc719bb47573ac3
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] SHA-1 hash officially broken

[Chad]

On Fri, Feb 24, 2017 at 10:57 AM Brion Vibber  wrote:

> Added a tracking/epic task on phab:
> https://phabricator.wikimedia.org/T158986
>
> Attach any specific things to that as subtasks, y'all.
>
>
Thanks for getting the discussion started. I was talking about this
yesterday
with some people and yeah, file and revision sha1s immediately came to
mind.

Also possibly a future issue: Git--but we'll mostly want to wait for various
upstreams to make progress here. There's some interesting thoughts over
at this fork[0] (h/t to Tim)

But, like you said (and Dan pointed out on the task), this is far from a
drop
everything situation...we just need to start making some inroads to this
sooner rather than later. Since it's broken, making it faster is an
optimization
problem so the writing is on the wall.

-Chad

[0]
https://github.com/peff/git/commit/b55aa200a0b3af58dc9e0accadc719bb47573ac3
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] SHA-1 hash officially broken

[Brion Vibber]

Added a tracking/epic task on phab:
https://phabricator.wikimedia.org/T158986

Attach any specific things to that as subtasks, y'all.

-- brion


On Fri, Feb 24, 2017 at 10:51 AM, Florian Schmidt <
florian.schmidt.wel...@t-online.de> wrote:

> > but this doesn't need to be a "drop everything" hurry task.
>
> Totally agree! :)
>
> Thanks for the information with the page on mediawiki.org. I'll take a
> look and probably edit it, if I see something :P
>
> Best,
> Florian
>
> -Ursprüngliche Nachricht-
> Von: Wikitech-l [mailto:wikitech-l-boun...@lists.wikimedia.org] Im
> Auftrag von Brion Vibber
> Gesendet: Freitag, 24. Februar 2017 19:48
> An: Wikimedia developers <wikitech-l@lists.wikimedia.org>
> Betreff: Re: [Wikitech-l] SHA-1 hash officially broken
>
> On Fri, Feb 24, 2017 at 10:34 AM, Florian Schmidt <
> florian.schmidt.wel...@t-online.de> wrote:
>
> > I searched in phabricator, if we've a task already, but couldn't find
> any.
> > However, as the phabricator search and me aren't really good friends,
> > it's possible, that the search wasn't as honest to me, as I would wish
> > and I missed something, so I ask on this list :) Do we've a task
> > already to track the work on this topic? A short github search[1]
> > showed some usages of sha1 (at least the string), so I suspect, that
> > there're some places where we use it, right?
> >
> > [1]
> > https://github.com/wikimedia/mediawiki/search?utf8=%E2%9C%93=SHA1
>
>
> Usages are listed at https://www.mediawiki.org/wiki/Manual:Hashing --
> I've added a "purpose" column on the DB fields list. There might be a
> couple we missed, so feel free to edit!
>
>
> It looks like img_sha1 and fa_storage_key are the biggest practical
> dangers of collision, in that pairs of images could be created such that,
> say, one is a cute kitten and the other is a shock image, such that
> deletion and undeletion might surface the wrong file from what the
> undeleting admin expected.
>
> However since the attack requires creating a common prefix and suffix for
> the matching pair of files, there's little or no danger of an attack that
> replaces legitimate files already uploaded by someone else.
>
> I would probably recommend migrating rev_sha1 and img_sha1's usages in
> collision detection over to a rev_sha256 and img_sha256 columns, and
> reworking filearchve to use a sha-256 hash for content addressing of new
> files, but this doesn't need to be a "drop everything" hurry task.
>
> -- brion
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] SHA-1 hash officially broken

[Florian Schmidt]

> but this doesn't need to be a "drop everything" hurry task.

Totally agree! :)

Thanks for the information with the page on mediawiki.org. I'll take a look and 
probably edit it, if I see something :P

Best,
Florian

-Ursprüngliche Nachricht-
Von: Wikitech-l [mailto:wikitech-l-boun...@lists.wikimedia.org] Im Auftrag von 
Brion Vibber
Gesendet: Freitag, 24. Februar 2017 19:48
An: Wikimedia developers <wikitech-l@lists.wikimedia.org>
Betreff: Re: [Wikitech-l] SHA-1 hash officially broken

On Fri, Feb 24, 2017 at 10:34 AM, Florian Schmidt < 
florian.schmidt.wel...@t-online.de> wrote:

> I searched in phabricator, if we've a task already, but couldn't find any.
> However, as the phabricator search and me aren't really good friends, 
> it's possible, that the search wasn't as honest to me, as I would wish 
> and I missed something, so I ask on this list :) Do we've a task 
> already to track the work on this topic? A short github search[1] 
> showed some usages of sha1 (at least the string), so I suspect, that 
> there're some places where we use it, right?
>
> [1] 
> https://github.com/wikimedia/mediawiki/search?utf8=%E2%9C%93=SHA1


Usages are listed at https://www.mediawiki.org/wiki/Manual:Hashing -- I've 
added a "purpose" column on the DB fields list. There might be a couple we 
missed, so feel free to edit!


It looks like img_sha1 and fa_storage_key are the biggest practical dangers of 
collision, in that pairs of images could be created such that, say, one is a 
cute kitten and the other is a shock image, such that deletion and undeletion 
might surface the wrong file from what the undeleting admin expected.

However since the attack requires creating a common prefix and suffix for the 
matching pair of files, there's little or no danger of an attack that replaces 
legitimate files already uploaded by someone else.

I would probably recommend migrating rev_sha1 and img_sha1's usages in 
collision detection over to a rev_sha256 and img_sha256 columns, and reworking 
filearchve to use a sha-256 hash for content addressing of new files, but this 
doesn't need to be a "drop everything" hurry task.

-- brion
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] SHA-1 hash officially broken

[Brion Vibber]

On Fri, Feb 24, 2017 at 10:34 AM, Florian Schmidt <
florian.schmidt.wel...@t-online.de> wrote:

> I searched in phabricator, if we've a task already, but couldn't find any.
> However, as the phabricator search and me aren't really good friends, it's
> possible, that the search wasn't as honest to me, as I would wish and I
> missed something, so I ask on this list :) Do we've a task already to track
> the work on this topic? A short github search[1] showed some usages of sha1
> (at least the string), so I suspect, that there're some places where we use
> it, right?
>
> [1] https://github.com/wikimedia/mediawiki/search?utf8=%E2%9C%93=SHA1


Usages are listed at https://www.mediawiki.org/wiki/Manual:Hashing -- I've
added a "purpose" column on the DB fields list. There might be a couple we
missed, so feel free to edit!


It looks like img_sha1 and fa_storage_key are the biggest practical dangers
of collision, in that pairs of images could be created such that, say, one
is a cute kitten and the other is a shock image, such that deletion and
undeletion might surface the wrong file from what the undeleting admin
expected.

However since the attack requires creating a common prefix and suffix for
the matching pair of files, there's little or no danger of an attack that
replaces legitimate files already uploaded by someone else.

I would probably recommend migrating rev_sha1 and img_sha1's usages in
collision detection over to a rev_sha256 and img_sha256 columns, and
reworking filearchve to use a sha-256 hash for content addressing of new
files, but this doesn't need to be a "drop everything" hurry task.

-- brion
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] SHA-1 hash officially broken

[Florian Schmidt]

I searched in phabricator, if we've a task already, but couldn't find any. 
However, as the phabricator search and me aren't really good friends, it's 
possible, that the search wasn't as honest to me, as I would wish and I missed 
something, so I ask on this list :) Do we've a task already to track the work 
on this topic? A short github search[1] showed some usages of sha1 (at least 
the string), so I suspect, that there're some places where we use it, right?

[1] https://github.com/wikimedia/mediawiki/search?utf8=%E2%9C%93=SHA1

Best,
Florian

-Ursprüngliche Nachricht-
Von: Wikitech-l [mailto:wikitech-l-boun...@lists.wikimedia.org] Im Auftrag von 
Brion Vibber
Gesendet: Freitag, 24. Februar 2017 18:57
An: Wikimedia-tech list <wikitech-l@lists.wikimedia.org>
Betreff: [Wikitech-l] SHA-1 hash officially broken

Google security have announced that they have a working collision attack 
against the SHA-1 hash:

https://security.googleblog.com/2017/02/announcing-first-sha1-collision.html

It's highly recommended to move to sha-256 where doable.

Note that MediaWiki uses sha-1 in a number of places; in some such as revision 
hashes it's advisory for tools only, but in other places like deleted files 
(filearchive table) we use it for addressing, and should consider steps to 
mitigate attacks swapping in alternate files during deletion/undeletion.

-- brion
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] SHA-1 hash officially broken

[Brion Vibber]

Google security have announced that they have a working collision attack
against the SHA-1 hash:

https://security.googleblog.com/2017/02/announcing-first-sha1-collision.html

It's highly recommended to move to sha-256 where doable.

Note that MediaWiki uses sha-1 in a number of places; in some such as
revision hashes it's advisory for tools only, but in other places like
deleted files (filearchive table) we use it for addressing, and should
consider steps to mitigate attacks swapping in alternate files during
deletion/undeletion.

-- brion
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l