Re: [git-users] git checkout file with custom mtime

2019-01-09 Thread Daniel Fanjul
The mtime of the files in my working copy change when I amend or
rebase or checkout different branches or in general when I use git
commands. I carefully store the mtime when these files are generated
or overwritten and restore it when it is going to be read. The tool I
use is https://packages.ubuntu.com/bionic/metastore and does not have
stale information.

I do not know how git lfs will work here, it handles large contents
but I doubt it will handle the mtimes as I need.

-- 
You received this message because you are subscribed to the Google Groups "Git 
for human beings" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to git-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [git-users] git checkout file with custom mtime

2019-01-09 Thread Konstantin Khomoutov
On Sat, Jan 05, 2019 at 10:44:47PM +0100, Daniel Fanjul wrote:
> I'm on Ubuntu. I do not use LFS. I track mods and saved games of
> Skyrim with git, TESV.exe sorts the saved games only by their mtime. I
> know it is not the most usual use case for git.
> 
> I agree with that viewpoint and I like the way git works right now, I
> do not want to change that. Checking out the saved games and then
> fixing the mtime works but forces a lot of unneeded I/O.

You might find this overview of git-annex vs git lfs useful:
https://lwn.net/Articles/774125/

Also see this recent thread on the Git list:
https://public-inbox.org/git/79fd2b4e-243c-a9f5-3485-2954fb0f5...@aixigo.de/

-- 
You received this message because you are subscribed to the Google Groups "Git 
for human beings" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to git-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [git-users] git checkout file with custom mtime

2019-01-09 Thread Philip Oakley

Hi Daniel,

I think between us we have two wrongs and a right.

On 06/01/2019 14:11, Daniel Fanjul wrote:

Yes, I know about it but it does not work completely for me:

Examples:
$ git update-index --no-assume-unchanged saves/* && touch saves/* &&
time git status saves/*
20 seconds reading file contents that did not actually change.

$ git update-index --assume-unchanged saves/* && touch saves/* && time
git status saves/*
0,5 seconds, that's good, the assume-unchanged flag works as intended.


This one is a "wrong" in that you said to git that it can 
'--assume-unchanged' and yet you then went and touched it, which is 
wrong of you. I would agree that the manual pages are not clear enough 
about the viewpoint that is taken about the option (who promised who 
'what'?)




$ git update-index --no-assume-unchanged saves/* && time git status saves/*
20 seconds again. This is only a partial solution, as I have to remove
the assume-unchanged flag eventually.

'git update-index --refresh' does not improve:
$ git update-index --assume-unchanged saves/* && touch saves/* && git
update-index --refresh saves/* && git update-index
--no-assume-unchanged saves/* && time git status saves/*
20 seconds again.


I was wrong here. I'd misunderstood the manual regarding the stat info. 
I'd thought git was just getting the file time and size info, and if the 
indicators of the file were good (same size), then it would simply 
update the stat times. Instead it looks to be doing a full file read and 
refreshing where differences are found.




'git update-index --really-refresh' does not improve:
$ git update-index --assume-unchanged saves/* && touch saves/* && git
update-index --really-refresh saves/* && git update-index
--no-assume-unchanged saves/* && time git status saves/*
20 seconds again.

The problem with the assume-unchanged flag for me is that git ignores
the mtime stored in the index, and when commands like 'git status' or
'git commit saves' have to check the contents then they have to reread
the file.

I need it not to ignore the mtime but to store the proper mtime
without rereading the files. A command like 'git checkout --mtime
$mymtime HEAD -- $myfile' in a clean working copy would be able to
store the mtime in the index without touching the contents of the
file. We can use GIT_COMMITTER_DATE for 'git commit', another
environment variable could make sense here too.


I'd agree that this would be good. However I'm not sure if it is not an 
XY-problem.


I can't see how you get into the situation in the first place. Who 
changed the file time stamp, when, and how does the other tool have 
stale file time information? As best I understand it Git avoids changing 
file time stamps (because of build tool issues) when there are no 
changes to a file during a checkout, so who's data went stale when?


--

Philip



On Sun, Jan 6, 2019 at 1:53 PM Philip Oakley  wrote:

Hi Daniel

(trimmed the cc, removing the Git List as they won't have seen these
HTML emails.. I'd only added it to confirm their address and forgot to
remove it, doh!)


Yes, the 'git update-index --assume-unchanged' is confusing because
actually it is a _user_ promise not to change things, so that git can be
parsimonious in the checking of the file status, rather than telling Git
not to look at the file ! There is a thread I'm contributing to about
getting the man page to be more plain speaking about that


I've just had a re-read of the git update-index man page
https://git-scm.com/docs/git-update-index#_using_refresh and it looks
like the --refresh option is what you need as soon as you update the mtimes.

Check all occurrences of --refresh on the man page to see the
interaction with other flags, and the quiet/error options.


See what you think and have a try. Hope it helps.

Philip

On 05/01/2019 21:44, Daniel Fanjul wrote:

I'm on Ubuntu. I do not use LFS. I track mods and saved games of
Skyrim with git, TESV.exe sorts the saved games only by their mtime. I
know it is not the most usual use case for git.

I agree with that viewpoint and I like the way git works right now, I
do not want to change that. Checking out the saved games and then
fixing the mtime works but forces a lot of unneeded I/O.

I forgot to mention that 'git update-index --assume-unchanged' does
not solve this well enough. Eventually 'git status' rereads the file
when that flag is removed. A better way for my use case would be being
able to set the proper mtime without forcing a rehash of the file that
yields the same object.

Thanks for your reply.



--
You received this message because you are subscribed to the Google Groups "Git for 
human beings" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to git-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [git-users] git checkout file with custom mtime

2019-01-06 Thread Daniel Fanjul
Yes, I know about it but it does not work completely for me:

Examples:
$ git update-index --no-assume-unchanged saves/* && touch saves/* &&
time git status saves/*
20 seconds reading file contents that did not actually change.

$ git update-index --assume-unchanged saves/* && touch saves/* && time
git status saves/*
0,5 seconds, that's good, the assume-unchanged flag works as intended.

$ git update-index --no-assume-unchanged saves/* && time git status saves/*
20 seconds again. This is only a partial solution, as I have to remove
the assume-unchanged flag eventually.

'git update-index --refresh' does not improve:
$ git update-index --assume-unchanged saves/* && touch saves/* && git
update-index --refresh saves/* && git update-index
--no-assume-unchanged saves/* && time git status saves/*
20 seconds again.

'git update-index --really-refresh' does not improve:
$ git update-index --assume-unchanged saves/* && touch saves/* && git
update-index --really-refresh saves/* && git update-index
--no-assume-unchanged saves/* && time git status saves/*
20 seconds again.

The problem with the assume-unchanged flag for me is that git ignores
the mtime stored in the index, and when commands like 'git status' or
'git commit saves' have to check the contents then they have to reread
the file.

I need it not to ignore the mtime but to store the proper mtime
without rereading the files. A command like 'git checkout --mtime
$mymtime HEAD -- $myfile' in a clean working copy would be able to
store the mtime in the index without touching the contents of the
file. We can use GIT_COMMITTER_DATE for 'git commit', another
environment variable could make sense here too.


On Sun, Jan 6, 2019 at 1:53 PM Philip Oakley  wrote:
>
> Hi Daniel
>
> (trimmed the cc, removing the Git List as they won't have seen these
> HTML emails.. I'd only added it to confirm their address and forgot to
> remove it, doh!)
>
>
> Yes, the 'git update-index --assume-unchanged' is confusing because
> actually it is a _user_ promise not to change things, so that git can be
> parsimonious in the checking of the file status, rather than telling Git
> not to look at the file ! There is a thread I'm contributing to about
> getting the man page to be more plain speaking about that
>
>
> I've just had a re-read of the git update-index man page
> https://git-scm.com/docs/git-update-index#_using_refresh and it looks
> like the --refresh option is what you need as soon as you update the mtimes.
>
> Check all occurrences of --refresh on the man page to see the
> interaction with other flags, and the quiet/error options.
>
>
> See what you think and have a try. Hope it helps.
>
> Philip
>
> On 05/01/2019 21:44, Daniel Fanjul wrote:
> > I'm on Ubuntu. I do not use LFS. I track mods and saved games of
> > Skyrim with git, TESV.exe sorts the saved games only by their mtime. I
> > know it is not the most usual use case for git.
> >
> > I agree with that viewpoint and I like the way git works right now, I
> > do not want to change that. Checking out the saved games and then
> > fixing the mtime works but forces a lot of unneeded I/O.
> >
> > I forgot to mention that 'git update-index --assume-unchanged' does
> > not solve this well enough. Eventually 'git status' rereads the file
> > when that flag is removed. A better way for my use case would be being
> > able to set the proper mtime without forcing a rehash of the file that
> > yields the same object.
> >
> > Thanks for your reply.
> >

-- 
You received this message because you are subscribed to the Google Groups "Git 
for human beings" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to git-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [git-users] git checkout file with custom mtime

2019-01-06 Thread Philip Oakley

Hi Daniel

(trimmed the cc, removing the Git List as they won't have seen these 
HTML emails.. I'd only added it to confirm their address and forgot to 
remove it, doh!)



Yes, the 'git update-index --assume-unchanged' is confusing because 
actually it is a _user_ promise not to change things, so that git can be 
parsimonious in the checking of the file status, rather than telling Git 
not to look at the file ! There is a thread I'm contributing to about 
getting the man page to be more plain speaking about that



I've just had a re-read of theĀ git update-index man page 
https://git-scm.com/docs/git-update-index#_using_refresh and it looks 
like the --refresh option is what you need as soon as you update the mtimes.


Check all occurrences of --refresh on the man page to see the 
interaction with other flags, and the quiet/error options.



See what you think and have a try. Hope it helps.

Philip

On 05/01/2019 21:44, Daniel Fanjul wrote:

I'm on Ubuntu. I do not use LFS. I track mods and saved games of
Skyrim with git, TESV.exe sorts the saved games only by their mtime. I
know it is not the most usual use case for git.

I agree with that viewpoint and I like the way git works right now, I
do not want to change that. Checking out the saved games and then
fixing the mtime works but forces a lot of unneeded I/O.

I forgot to mention that 'git update-index --assume-unchanged' does
not solve this well enough. Eventually 'git status' rereads the file
when that flag is removed. A better way for my use case would be being
able to set the proper mtime without forcing a rehash of the file that
yields the same object.

Thanks for your reply.



--
You received this message because you are subscribed to the Google Groups "Git for 
human beings" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to git-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [git-users] git checkout file with custom mtime

2019-01-05 Thread Daniel Fanjul
I'm on Ubuntu. I do not use LFS. I track mods and saved games of
Skyrim with git, TESV.exe sorts the saved games only by their mtime. I
know it is not the most usual use case for git.

I agree with that viewpoint and I like the way git works right now, I
do not want to change that. Checking out the saved games and then
fixing the mtime works but forces a lot of unneeded I/O.

I forgot to mention that 'git update-index --assume-unchanged' does
not solve this well enough. Eventually 'git status' rereads the file
when that flag is removed. A better way for my use case would be being
able to set the proper mtime without forcing a rehash of the file that
yields the same object.

Thanks for your reply.

-- 
You received this message because you are subscribed to the Google Groups "Git 
for human beings" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to git-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [git-users] git checkout file with custom mtime

2019-01-05 Thread Philip Oakley

Daniel,


Do you use the Git LFS (Large File System) add-on?, are you on Windows 
or Linux?, and what tools need mtime (or is it something about the 
process of using the tool..)?


The Git viewpoint is that the mtime shouldn't be important for the 
version storage & control aspects, though it maybe for the external 
compiler tooling, so they do tend to try to keep the mtime/ctime consistent.


I'm not aware of specific capability to do what you ask, but it may be 
worth discussing this on the git mailing list "Git List 
" (which only accepts 100% plain text, no HTML, 
messages). The mailing list archive is at 
https://public-inbox.org/git/?q= where you can search for mtime/ctime 
discussions.


There will be a Git developer conference at the end of the month, so it 
is worth raining it soonish, even if it becomes an add on the fires via 
a post checkout hook that updates the mtimes from a stored file of 
'true' mtimes (plus updates the index's view of those mtimes.


Philip

On 05/01/2019 13:33, Daniel Fanjul wrote:

Hi all,

I have some large files tracked in git and I have to track their mtime 
because of some legacy software. With another tool I save and restore 
their mtime. When I restore their mtime git status rereads the files 
to update the mtime in the index. I would like to improve that because 
there are too many files, the whole I/O is too slow and the whole 
process is triggered too often.


I would like a way to tell git to checkout a file and set a given 
mtime at the same time so the index is updated with the mtime but the 
file is not rewritten because the working copy is clean. This would 
solve my problem. Do you know a way to do this?


Do you know any other way to handle this properly?

Thanks in advance, and happy new year,
Daniel.
--
You received this message because you are subscribed to the Google 
Groups "Git for human beings" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to git-users+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Git for 
human beings" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to git-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.