Re: File extension conflict when working with git and latex
On 2014-03-21 17.13, Matthias Beyer wrote: > Hi, > > I struggled with a really nasty issue today (and yesterday): > > I work on a semester project paper which is written in latex and > therefor version-controlled with git. We compile the document using > pdflatex, we automate the compiling with make. > > pdflatex generates some temporary files, which are required if you > call it the second and third time (it is a top-down compiler, which > needs to be called several times). One of the file extensions is > ".idx", which (kind of) conflicts with file extensions in .git/. > > Unfortunetely, we wrote our `make clean` task recursively. I think you > can imagine what went wrong: The clean-task corrupted the repository, > as it removed .idx files from within .git/. > > I lost work because of this ugly name collision. > > My question now: > > I know, I can fix this by fixing the clean task in my Makefile. But > maybe someone somewhere on this world doesn't know the git internals > as good as me (and, of course, my coworker). Is there _any chance > at all_ that this gets mentioned somewhere, so others don't fall into > this pit? > The idx files are mentioned here: http://www.git-scm.com/book/en/Git-Internals-Packfiles (Otherwise, some random thoughts:) - stay out of the .git directory, whatever you do (I lost some data because my faulty scripts using sed to replace "s/this/that/" and went into the .git directory) - Don't delete "read-only" files (The .idx files are all "r--r--r--") Or in other words, do not use "rf -f" in a script, unless you know exactly what you and the script are doing. - Replace "rm -f" whith "mv /tmp" could be an option - As mentioned before, "git clean" can be an option, I used it once when the "make clean", "make distclean", "make reallydistclean" did not work as expected by me, some auto-generated config files where still not deleted. - Make backups of your repo (frequently) - push the repo to a backup repo (frequently) - Use $GIT_DIR to move the repo out of your work tree: https://www.kernel.org/pub/software/scm/git/docs/git.html - Prepare a patch that renames the *.idx files used by Git into *.gitidx, *.gidx, *.packidx, (and the *.pack into *.packdata ??) Hopefully you did not loose too much data, and if you have an idea how to improve things, you may consider sending a patch, improvements are possible and welcome. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: File extension conflict when working with git and latex
On 03/21/2014 07:32 PM, Junio C Hamano wrote: > Matthias Beyer writes: > >> I know, I can fix this by fixing the clean task in my Makefile. But >> maybe someone somewhere on this world doesn't know the git internals >> as good as me (and, of course, my coworker). Is there _any chance >> at all_ that this gets mentioned somewhere, so others don't fall into >> this pit? > > Surely, we are here to please ;-) All of us want to make sure > newbies do not shoot themselves in the foot. > > But the problem is what exactly should be mentioned. With a fresh > wound with your LaTeX project still in your mind, you may be tempted > to special case ".idx", but other newbies may inflict the same kind > of hurt on themselves with different "find" patterns, e.g. > > $ find . -name '[0-9a-f]*[0-9a-f]' -type f -print | xargs rm -f > > when they know their project creates hexadecimal-numbered temoprary > files, or whatever other pattern that match the files they do not > care about, that also happens to match whatever is in $GIT_DIR. The > only common caution that helps us to make sure "others do not fall > into this pit" is "Files and directories in $GIT_DIR are used to > record your work; do not muck with them unless you know what you are > doing e.g. manually repairing a corrupt repository", but that is a > bit lame, isn't it? It might help some people to show the "find" incantation for skipping files under .git, which I think would be something like find . -name .git -prune -o \( [...] \) -print0 | xargs -0 rm -f where "[...]" is whatever condition the user would otherwise use. Michael -- Michael Haggerty mhag...@alum.mit.edu http://softwareswirl.blogspot.com/ -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: File extension conflict when working with git and latex
Matthias Beyer writes: > I know, I can fix this by fixing the clean task in my Makefile. But > maybe someone somewhere on this world doesn't know the git internals > as good as me (and, of course, my coworker). Is there _any chance > at all_ that this gets mentioned somewhere, so others don't fall into > this pit? Surely, we are here to please ;-) All of us want to make sure newbies do not shoot themselves in the foot. But the problem is what exactly should be mentioned. With a fresh wound with your LaTeX project still in your mind, you may be tempted to special case ".idx", but other newbies may inflict the same kind of hurt on themselves with different "find" patterns, e.g. $ find . -name '[0-9a-f]*[0-9a-f]' -type f -print | xargs rm -f when they know their project creates hexadecimal-numbered temoprary files, or whatever other pattern that match the files they do not care about, that also happens to match whatever is in $GIT_DIR. The only common caution that helps us to make sure "others do not fall into this pit" is "Files and directories in $GIT_DIR are used to record your work; do not muck with them unless you know what you are doing e.g. manually repairing a corrupt repository", but that is a bit lame, isn't it? It is tempting to suggest "git clean '*.idx'", but that is a good fit in the Makefile only when you know everybody involved in the project works in a checkout from Git, not from a tarball extract, and does not apply to projects in general. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: File extension conflict when working with git and latex
On Fri, Mar 21, 2014 at 05:46:51PM +0100, Matthias Beyer wrote: > Hi Simon, > > I think so. I executed: > > git fsck # reports N missing blobs, commits, trees and dangling stuff > git index-pack ... > git fsck # reports only dangling commits and blobs > > I don't know if this means that the repository is fixed now? Hello Matthias, If nothing else was deleted, then yes, the repository should be fine now. Regards Simon -- + privacy is necessary + using gnupg http://gnupg.org + public key id: 0x92FEFDB7E44C32F9 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: File extension conflict when working with git and latex
On 21-03-2014 17:36:28, Simon Ruderich wrote: > On Fri, Mar 21, 2014 at 05:13:24PM +0100, Matthias Beyer wrote: > > Unfortunetely, we wrote our `make clean` task recursively. I think you > > can imagine what went wrong: The clean-task corrupted the repository, > > as it removed .idx files from within .git/. > > > > I lost work because of this ugly name collision. > > Hello Matthias, > > You can recreate the .idx files by running > > git index-pack .git/objects/pack/pack-.pack > > for each pack file. Hi Simon, I think so. I executed: git fsck # reports N missing blobs, commits, trees and dangling stuff git index-pack ... git fsck # reports only dangling commits and blobs I don't know if this means that the repository is fixed now? -- Mit freundlichen Grüßen, Kind regards, Matthias Beyer Proudly sent with mutt. Happily signed with gnupg. pgpxEdzzwXXTm.pgp Description: PGP signature
Re: File extension conflict when working with git and latex
On Fri, Mar 21, 2014 at 05:13:24PM +0100, Matthias Beyer wrote: > Unfortunetely, we wrote our `make clean` task recursively. I think you > can imagine what went wrong: The clean-task corrupted the repository, > as it removed .idx files from within .git/. > > I lost work because of this ugly name collision. Hello Matthias, You can recreate the .idx files by running git index-pack .git/objects/pack/pack-.pack for each pack file. Regards Simon -- + privacy is necessary + using gnupg http://gnupg.org + public key id: 0x92FEFDB7E44C32F9 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
