Re: git send-email does not work with Google anymore?!

[ankostis]

On 5 October 2017 at 15:43, Paolo Bonzini  wrote:
>
> On 05/10/2017 12:52, Lars Schneider wrote:
> > Hi,
> >
> > I used to use the Google SMTP server to send my patches to the list with
> > the following config:
> >
> > [sendemail]
> > smtpencryption = tls
> > smtpserver = smtp.gmail.com
> > smtpuser = larsxschnei...@gmail.com
> > smtpserverport = 587
> > from = larsxschnei...@gmail.com
> > chainreplyto = false
> > suppresscc = self
> >
> > Apparently that stopped working today. I get this error:
> >
> > (mbox) Adding cc: Lars Schneider  from line 
> > 'From: Lars Schneider '
> > Password for 'smtp://larsxschnei...@gmail.com@smtp.gmail.com:587':
> > 5.7.14  > 5.7.14 ...> Please log in via your web browser and
> > 5.7.14 then try again.
> > 5.7.14  Learn more at
> > 5.7.14  https://support.google.com/mail/answer/78754 ... - gsmtp
> >
> > Of couse I tried to log in via web browser etc. Does anyone else use
> > Google as SMTP server? If yes, does it work for you?

I have been using SMTP+GMail rather a lot, recently.
The situation is a mess, and it becomes worse by the month!

Till recently, you had to perform various steps to achieve this:

1. Got to your web-gmail, Settings | Forwarding and POP/IMAP | Enable
IMAP Access

2. Enable access for "Less secure apps":
   https://support.google.com/accounts/answer/6010255?hl=en

3. Try to SMTP-login.

   BUT if the IP had not access web-GMail recently,
   you got the message you report.
   So you had to:
   - reconstruct the URL+token intermingled in the SMTP-error message,
   - paste it in your browser, and
   - do a web-login to prove you are a human.

You had to perform step 3 regularly.


Now, if you had 2-factor-auth enabled, you could use
"app-passwords" to log into your gmail account:
https://support.google.com/accounts/answer/185833?hl=en

But this seems to have stopped working; I clicked the link in the page below
  https://support.google.com/accounts/answer/6010255?hl=en
and for my account it tells me "less secure apps" access is unavailable :-(



>
> It's probably a good idea to set up two-factor authentication and add an
> app-specific password for "git send-email" (at least that's what I do).
>
> The password can be stored encrypted using the OS keychain, or if you
> don't want/have one (e.g. you are running headless) you could check if
> your distro installs git-credential-netrc.  Create a ~/.netrc.gpg file,
> where the decrypted content should look like
>
> machine smtp.gmail.com login larsxschnei...@gmail.com password mypassword
>
> Then do
>
> git config --global credential.helper netrc
>
> and you're all set. :)

I believe the best way forward is to use OAuth.



Ciao,
  Kostis

Would it make sense to add `--depth=0` feature?

[ankostis]

Would it be feasible/sensical to support clone/fetch with --depth=0?
The point is to download the tags/commit objects but not the actual
trees and blobs,
creating a "ledger" acting as a promise for the remote'e repo content?

Is there some estimation of the effort required to do such thing?

Best,
  Kostis

Re: CYGWIN git cannot install python packages with pip

[ankostis]

On 2 May 2017 at 20:08, ankostis <ankos...@gmail.com> wrote:
> On Windows, with Cygwin-git 02.12.2-1 the python command:
>
> pip install git+https://github.com/...`
>
> fails to work with the following error:
>
> $ pip install git+https://github.com/ipython/traitlets/
> Collecting git+https://github.com/ipython/traitlets/
> Cloning https://github.com/ipython/traitlets/ to c:\users\username\appdata
> \local\temp\pip-kjwxq_oy-build
> fatal: Invalid path '/cygdrive/d/Work/ALLINONES2/co2mpas_AIO-v1.6.0/CO2MPA
> S/C:\Users\username\AppData\Local\Temp\pip-kjwxq_oy-build': No such file or di
> rectory
> git clone -q https://github.com/ipython/traitlets/ C:\Users\user\AppData\L
> ocal\Temp\pip-kjwxq_oy-build" failed with error code 128 in None
>
>
>
> Git-2.8.3 had no such problem.
>
> Any ideas?

Reporting for those bitten by this, latest working CYGWIN timemachine
release is on 16 March 2017:
  
http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/2017/04/16/142104/index.html

Re: CYGWIN git cannot install python packages with pip

[ankostis]

On 4 May 2017 at 13:26, Johannes Schindelin <johannes.schinde...@gmx.de> wrote:
> Hi,
>
> On Thu, 4 May 2017, ankostis wrote:
>
>> On 4 May 2017 at 11:47, Adam Dinwoodie <a...@dinwoodie.org> wrote:
>> Judging from the error-message, it somehow concatenates input & output
>> paths.
>> Isn't this something to research about?
>
> It is something to research about. But by you, not by me.

You got a point there.

> I need to focus
> on bigger-impact issues, because I do not scale, and Git for Windows'
> users generally do not experience the problem you described.

Git for Windows uses MINGW indeed.
But there are still CYGWIN users out in the wild
running the vanilla Windows python installation.

Actually this issue has a big impact on those of us because
recent `setuptools` tools allow for packages to specify project dependencies
using the `git+https://...` url-scheme.
So the problem may appear to unaware users just trying to install
a packages the regular way, with `pip install foo.package`

Great thanks,
  Kostis

FYI, cross-posting also to CYGWIN list.

Re: CYGWIN git cannot install python packages with pip

[ankostis]

On 4 May 2017 at 11:47, Adam Dinwoodie  wrote:
> Confirmed: the Cygwin project as a general rule doesn't support this
> sort of mixing of Windows and Cygwin tools.  Either use Python and Git
> packages both provided by Cygwin, or both provided by Windows.
>
> Mixing and matching does work sometimes -- as was apparently the case
> with Cygwin Git v2.8.3-1 -- but it requires care and you're generally on
> your own with it.

MSYS2's git-2.12.1 also works fine, so there must be something
different in this build of git that breaks this long-standing capability.

Judging from the error-message, it somehow concatenates input & output
paths.
Isn't this something to research about?

Regards, for your continuous effort,
  Kostis

Re: CYGWIN git cannot install python packages with pip

[ankostis]

On 3 May 2017 11:47, Johannes Schindelin <johannes.schinde...@gmx.de> wrote:
> On Tue, 2 May 2017, ankostis wrote:
>
>> On Windows, with Cygwin-git 02.12.2-1 the python command:
>> [...]
>
> You forgot to mention what python/pip you use (is it the Cygwin one, or a
> standalone one?). Intentional?

I have tested it on using WinPython[1] 3.6.1 & 3.5.3.
WinPython is built out of the "standard" python on Windows.

Judging by the error, the problem is on the temporary folder
used as target.  For some reason, Git uses $HOME
as a prefix to the temporary folder selected by `pip`.

My home in the above case above is this:
/cygdrive/d/Work/ALLINONES2/co2mpas_AIO-v1.6.0/CO2MPAS


Thanks,
  Kostis

[1] http://winpython.github.io/

Re: CYGWIN git cannot install python packages with pip

[ankostis]

Note that MSYS2 Git-12.2.1 also has no such problem.

On 2 May 2017 at 20:08, ankostis <ankos...@gmail.com> wrote:
> On Windows, with Cygwin-git 02.12.2-1 the python command:
>
> pip install git+https://github.com/...`
>
> fails to work with the following error:
>
> ...
>
> Git-2.8.3 had no such problem.
>
> Any ideas?

CYGWIN git cannot install python packages with pip

[ankostis]

On Windows, with Cygwin-git 02.12.2-1 the python command:

pip install git+https://github.com/...`

fails to work with the following error:

$ pip install git+https://github.com/ipython/traitlets/
Collecting git+https://github.com/ipython/traitlets/
Cloning https://github.com/ipython/traitlets/ to c:\users\username\appdata
\local\temp\pip-kjwxq_oy-build
fatal: Invalid path '/cygdrive/d/Work/ALLINONES2/co2mpas_AIO-v1.6.0/CO2MPA
S/C:\Users\username\AppData\Local\Temp\pip-kjwxq_oy-build': No such file or di
rectory
git clone -q https://github.com/ipython/traitlets/ C:\Users\user\AppData\L
ocal\Temp\pip-kjwxq_oy-build" failed with error code 128 in None



Git-2.8.3 had no such problem.

Any ideas?

Re: [PATCH 00/10] RFC Partial Clone and Fetch

[ankostis]

Dear Jeff

I read most of the valuable references you provided
but could not find something along the lines describing inline.


On 8 March 2017 at 19:50,   wrote:
> From: Jeff Hostetler 
>
>
> [RFC] Partial Clone and Fetch
> =
>
> This is a WIP RFC for a partial clone and fetch feature wherein the client
> can request that the server omit various blobs from the packfile during
> clone and fetch.  Clients can later request omitted blobs (either from a
> modified upload-pack-like request to the server or via a completely
> independent mechanism).

Is it foreseen the server to *decide* with partial objects to serve
And the cloning-client still to work ok?

My case in mind is storing confidential files in Git (server)
that I want to publicize them to partial-cloning clients,
for non-repudiation, by sending out trees and commits alone
(or any non-sensitive blobs).

A possible UI would be to rely on a `.gitattributes` to specify
which objects are to be upheld.


Apologies if I'm intruding with an unrelated feature requests.
  Kostis

Re: RFC: Another proposed hash function transition plan

[ankostis]

On 13 March 2017 at 18:48, Jonathan Nieder  wrote:
>
> Hi,
>
> The Keccak Team wrote:
>
> > We have read your transition plan to move away from SHA-1 and noticed
> > your intent to use SHA3-256 as the new hash function in the new Git
> > repository format and protocol. Although this is a valid choice, we
> > think that the new SHA-3 standard proposes alternatives that may also be
> > interesting for your use cases.  As designers of the Keccak function
> > family, we thought we could jump in the mail thread and present these
> > alternatives.
>
> I indeed had some reservations about SHA3-256's performance.  The main
> hash function we had in mind to compare against is blake2bp-256.  This
> overview of other functions to compare against should end up being
> very helpful.

What if some of us need this extra difficulty, and don't mind about
the performance tax,
because we need to refer to hashes 10 or 30 years from now,
or even in the Post Quantum era?

Thanks,
  Kostis

Unconventional roles of git

[ankostis]

On 26 February 2017 at 02:13, Jason Cooper  wrote:
> As someone looking to deploy (and having previously deployed) git in
> unconventional roles, I'd like to add ...

We are developing a distributed storage for type approval files regarding all
vehicles registered in Europe.[1]  To ensure integrity even after 10 or 30
years, the hash of a commit of these files (as contained in a tag) are
to be printed on the the paper certificates.

- Can you provide some hints for other similar unconventional roles of git?
- Any other comment on the above usage of git are welcomed.

Kind Regards,
  Kostis Anagnostopoulos

[1] https://co2mpas.io

Re: SHA1 collisions found

[ankostis]

On 24 February 2017 at 18:23, Jason Cooper  wrote:
> Hi Ian,
>
> On Fri, Feb 24, 2017 at 03:13:37PM +, Ian Jackson wrote:
>> Joey Hess writes ("SHA1 collisions found"):
>> > https://shattered.io/static/shattered.pdf
>> > https://freedom-to-tinker.com/2017/02/23/rip-sha-1/
>> >
>> > IIRC someone has been working on parameterizing git's SHA1 assumptions
>> > so a repository could eventually use a more secure hash. How far has
>> > that gotten? There are still many "40" constants in git.git HEAD.
>>
>> I have been thinking about how to do a transition from SHA1 to another
>> hash function.
>>
>> I have concluded that:
>>
>>  * We can should avoid expecting everyone to rewrite all their
>>history.
>
> Agreed.
>
>>  * Unfortunately, because the data formats (particularly, the commit
>>header) are not in practice extensible (because of the way existing
>>code parses them), it is not useful to try generate new data (new
>>commits etc.) containing both new hashes and old hashes: old
>>clients will mishandle the new data.
>
> My thought here is:
>
>  a) re-hash blobs with sha256, hardlink to sha1 objects
>  b) create new tree objects which are mirrors of each sha1 tree object,
> but purely sha256
>  c) mirror commits, but they are also purely sha256
>  d) future PGP signed tags would sign both hashes (or include both?)


IMHO that is a great idea that needs more attention.
You get to keep 2 or more hash-functions for extra security in a PQ world.

And to keep sketches for the future so far,
SHA-3 must be always one of the new hashes.
Actually, you can get rid of SHA-1 completely, and land on Linus's
current sketches for the way ahead.

Thanks,
  Kostis
>
> Which would end up something like:
>
>   .git/
> \... #usual files
> \objects
>   \ef
> \3c39f7522dc55a24f64da9febcfac71e984366
> \objects-sha2_256
>   \72
> \604fd2de5f25c89d692b01081af93bcf00d2af34549d8d1bdeb68bc048932
> \info
>   \...
> \info-sha2_256
>   \refs #uses sha256 commit identifiers
>
> Basically, keep the sha256 stuff out of the way for legacy clients, and
> new clients will still be able to use it.
>
> There shouldn't be a need to re-sign old signed tags if the underlying
> objects are counter-hashed.  There might need to be some transition
> info, though.
>
> Say a new client does 'git tag -v tags/v3.16' in the kernel tree.  I would
> expect it to check the sha1 hashes, verify the PGP signed tag, and then
> also check the sha256 counter-hashes of the relevant objects.
>
> thx,
>
> Jason.

Re: SHA1 collisions found

[ankostis]

On 24 February 2017 at 21:32, Junio C Hamano <gits...@pobox.com> wrote:
> ankostis <ankos...@gmail.com> writes:
>
>> Let's assume that git is retroffited to always support the "default"
>> SHA-3, but support additionally more hash-funcs.
>> If in the future SHA-3 also gets defeated, it would be highly unlikely
>> that the same math would also break e.g. Blake.
>> So certain high-profile repos might choose for extra security 2 or more 
>> hashes.
>
> I think you are conflating two unrelated things.

I believe the two distinct things you refer to below are these:

  a. storing objects in filesystem and accessing them
 by name (e.g. from cmdline), and

  b. cross-referencing inside the objects (trees, tags, notes),

correct?

If not, then please ignore my answers, below.


>  * How are these "2 or more hashes" actually used?  Are you going to
>add three "parent " line to a commit with just one parent, each
>line storing the different hashes?

Yes, in all places where references are involved (tags, notes).
Based on what what the git-hackers have written so far, this might be doable.

To ensure integrity in the case of crypto-failures, all objects must
cross-reference each other with multiple hashes.
Of course this extra security would stop as soon as you reach "old"
history (unless you re-write it).


>How will such a commit object
>be named---does it have three names and do you plan to have three
>copies of .git/refs/heads/master somehow, each of which have
>SHA-1, SHA-3 and Blake, and let any one hash to identify the
>object?

Yes, based on Jason Cooper's idea, above, objects would be stored
under all names in the filesystem using hard links (although this
might not work nice on Windows).


>I suspect you are not going to do so; instead, you would use a
>very long string that is a concatenation of these three hashes as
>if it is an output from a single hash function that produces a
>long result.
>
>So I think the most natural way to do the "2 or more for extra
>security" is to allow us to use a very long hash.  It does not
>help to allow an object to be referred to with any of these 2 or
>more hashes at the same time.

If hard-linking all names is doable, then most restrictions above are
gone, correct?


>  * If employing 2 or more hashes by combining into one may enhance
>the security, that is wonderful.  But we want to discourage
>people from inventing their own combinations left and right and
>end up fragmenting the world.  If a project that begins with
>SHA-1 only naming is forked to two (or more) and each fork uses
>different hashes, merging them back will become harder than
>necessary unless you support all these hashes forks used.

Agree on discouraging people's inventions.

That is why I believe that some HASH (e.g. SHA-3) must be the blessed one.
All git >= 3.x.x must support at least this one (for naming and
cross-referencing between objects).


> Having said all that, the way to figure out the hash used in the way
> we spell the object name may not be the best place to discourage
> people from using random hashes of their choice.  But I think we
> want to avoid doing something that would actively encourage
> fragmentation.

I guess the "blessed SHA-3 will discourage people using the other
names., untill the next crypto-crack.

Re: SHA1 collisions found

[ankostis]

On 24 February 2017 at 20:20, Junio C Hamano  wrote:
> Stefan Beller  writes:
>
>> On Fri, Feb 24, 2017 at 10:14 AM, Junio C Hamano  wrote:
>>
>>> you are inviting people to start using
>>>
>>> md5,54ddf8d47340e048166c45f439ce65fd
>>>
>>> as object names.
>>
>> which might even be okay for specific subsets of operations.
>> (e.g. all local work including staging things, making local "fixup" commits)
>>
>> The addressing scheme should not be too hardcoded, we should rather
>> treat it similar to the cipher schemes in pgp. The additional complexity that
>> we have is the longevity of existence of things, though.
>
> The not-so-well-hidden agenda was exactly that we _SHOULD_ not
> mimick PGP.  They do not have a requirement to encourage everybody
> to use the same thing because each message is encrypted/signed
> independently, i.e. they do not have to chain things like we do.

But there is a scenario where supporting more hashes, in parallel, is
beneficial:

Let's assume that git is retroffited to always support the "default"
SHA-3, but support additionally more hash-funcs.
If in the future SHA-3 also gets defeated, it would be highly unlikely
that the same math would also break e.g. Blake.
So certain high-profile repos might choose for extra security 2 or more hashes.

Apologies if I'm misusing the list,
  Kostis

Re: SHA1 collisions found

[ankostis]

On 24 February 2017 at 16:13, Ian Jackson
 wrote:
>
> Joey Hess writes ("SHA1 collisions found"):
> > https://shattered.io/static/shattered.pdf
> > https://freedom-to-tinker.com/2017/02/23/rip-sha-1/
> >
> > IIRC someone has been working on parameterizing git's SHA1 assumptions
> > so a repository could eventually use a more secure hash. How far has
> > that gotten? There are still many "40" constants in git.git HEAD.
>
> I have been thinking about how to do a transition from SHA1 to another
> hash function.
>
> I have concluded that:
>
>  * We can should avoid expecting everyone to rewrite all their
>history.
>
>  * Unfortunately, because the data formats (particularly, the commit
>header) are not in practice extensible (because of the way existing
>code parses them), it is not useful to try generate new data (new
>commits etc.) containing both new hashes and old hashes: old
>clients will mishandle the new data.
>
>  * Therefore the transition needs to be done by giving every object
>two names (old and new hash function).  Objects may refer to each
>other by either name, but must pick one.  The usual shape of
>project histories will be a pile of old commits referring to each
>other by old names, surmounted by new commits referrring to each
>other by new names.
>
>  * It is not possible to solve this problem without extending the
>object name format.  Therefore all software which calls git and
>expects to handle object names will need to be updated.
>
> I have been writing a more detailed transition plan.  I hope to post
> this within a few days.

It would be great to have a rough plan of the transition to a new hash
function.

We are writing a git-based application to store electronic-files for
legislative purposes for EU.
And one of the great questions we face is about git's SHA-1 validity in 5
or 20 years of time from now.

Is it possible to have an assessment of the situation for this transition?

Best regards for your efforts,
  Kostis