Re: RFC: Native clean/smudge filter for UTF-16 files

On Sun, Dec 03, 2017 at 07:48:01PM +0100, Lars Schneider wrote: > > - if core.convertEncoding is true, then for any file with an > > encoding=foo attribute, internally run iconv(foo, utf8) in > > convert_to_git(), and likewise iconv(utf8, foo) in > > convert_to_working_tree. > > > > - I'm

Re: RFC: Native clean/smudge filter for UTF-16 files

-16 >> >> There was a previous discussion on the topic and Jonathan already suggested >> a "native" clean/smudge filter in 2010 [4]. Also the "encoding" attribute >> is already present but, as far as I can tell, is only used by the git gui >&g

Re: RFC: Native clean/smudge filter for UTF-16 files

Jeff King writes: > So anyway, that is an alternate strategy, but I think I like "canonical > in-repo text is utf-8" approach a lot more, since then git operations > work consistently. There are still a few rough edges (e.g., I'm not sure Sounds like a good way forward. > if you

Re: RFC: Native clean/smudge filter for UTF-16 files

t be a problem from my > point of view as UTF-16 is only relevant on Windows anyways. The attribute > could look like this: > > *.txttext encoding=utf-16 > > There was a previous discussion on the topic and Jonathan already suggested > a "native" c

Re: RFC: Native clean/smudge filter for UTF-16 files

F-16 is no good option > either as Git and Git related tools (e.g. GitHub) consider the files binary > and consequently do not render diffs. > > The obvious solution is to setup a clean/smudge filter like this [1]: > [filter "winutf16"] > clean = iconv -f

RFC: Native clean/smudge filter for UTF-16 files

the files binary and consequently do not render diffs. The obvious solution is to setup a clean/smudge filter like this [1]: [filter "winutf16"] clean = iconv -f utf-16 -t utf-8 smudge = iconv -f utf-8 -t utf-16 In general this works well but the "per-file"

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

> On 09 Jan 2017, at 21:44, Stefan Beller <sbel...@google.com> wrote: > > On Mon, Nov 14, 2016 at 1:09 PM, Lars Schneider > <larsxschnei...@gmail.com> wrote: >> Hi, >> >> Git always performs a clean/smudge filter on files in sequential order

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

On Mon, Nov 14, 2016 at 1:09 PM, Lars Schneider <larsxschnei...@gmail.com> wrote: > Hi, > > Git always performs a clean/smudge filter on files in sequential order. > Sometimes a filter operation can take a noticeable amount of time. > This blocks the entire Git process. >

Re: [PATCH] docs: warn about possible '=' in clean/smudge filter process values

Jeff King <p...@peff.net> writes: > On Sat, Dec 03, 2016 at 08:45:16PM +0100, larsxschnei...@gmail.com wrote: > >> From: Lars Schneider <larsxschnei...@gmail.com> >> >> A pathname value in a clean/smudge filter process "key=value" pair can >&

Re: [PATCH] docs: warn about possible '=' in clean/smudge filter process values

On Sat, Dec 03, 2016 at 08:45:16PM +0100, larsxschnei...@gmail.com wrote: > From: Lars Schneider <larsxschnei...@gmail.com> > > A pathname value in a clean/smudge filter process "key=value" pair can > contain the '=' character (introduced in edcc858). Make th

[PATCH] docs: warn about possible '=' in clean/smudge filter process values

From: Lars Schneider <larsxschnei...@gmail.com> A pathname value in a clean/smudge filter process "key=value" pair can contain the '=' character (introduced in edcc858). Make the user aware of this issue in the docs, add a corresponding test case, and fix the issue in filter proc

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

Lars Schneider writes: > What way do you think is better from a maintenance point of view? > I prefer option 2 but I fear that these "special" values could confuse > future readers of the code. I recall getting confused by the redefinition of the meaning of return

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

rect. This way Git doesn't need to deal with threading... > > I think you need to be careful about three things (at least; there > may be more): > > ... > > * Done naively, it will lead to unmaintainable code, like this: > > ... Hi Junio, I started to work on th

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

> On 17 Nov 2016, at 00:46, Junio C Hamano wrote: > > Jakub Narębski writes: > >>> I intend to implement this feature only for the new long running filter >>> process protocol. OK with you? >> >> If I remember and understand it correctly, current version

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

Jakub Narębski writes: >> I intend to implement this feature only for the new long running filter >> process protocol. OK with you? > > If I remember and understand it correctly, current version of long > running process protocol processes files sequentially, one by one: > git

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

W dniu 16.11.2016 o 10:53, Lars Schneider pisze: > On 15 Nov 2016, at 19:03, Junio C Hamano wrote: >> Lars Schneider writes: >> The filter itself would need to be aware of parallelism if it lives for multiple objects, right? >>> >>> Correct.

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

Lars Schneider writes: >> On 16 Nov 2016, at 19:15, Junio C Hamano wrote: >> >> Lars Schneider writes: >> * You'd need to rein in the maximum parallelism somehow, as you do not want to see hundreds of competing

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

> On 16 Nov 2016, at 19:15, Junio C Hamano wrote: > > Lars Schneider writes: > >>> * You'd need to rein in the maximum parallelism somehow, as you do >>> not want to see hundreds of competing filter processes starting >>> only to tell the main

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

Lars Schneider writes: >> * You'd need to rein in the maximum parallelism somehow, as you do >> not want to see hundreds of competing filter processes starting >> only to tell the main loop over an index with hundreds of entries >> that they are delayed checkouts.

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

On 15 Nov 2016, at 19:03, Junio C Hamano wrote: > Lars Schneider writes: > >>> The filter itself would need to be aware of parallelism >>> if it lives for multiple objects, right? >> >> Correct. This way Git doesn't need to deal with threading...

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

Lars Schneider wrote: > > On 15 Nov 2016, at 02:03, Eric Wong wrote: > > > Anyways, I'll plan on doing something similar (in Perl) with the > > synchronous parts of public-inbox which relies on "cat-file --batch" > > at some point... (my rotational

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

Lars Schneider writes: >> The filter itself would need to be aware of parallelism >> if it lives for multiple objects, right? > > Correct. This way Git doesn't need to deal with threading... I think you need to be careful about three things (at least; there may be

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

> On 15 Nov 2016, at 02:03, Eric Wong <e...@80x24.org> wrote: > > Lars Schneider <larsxschnei...@gmail.com> wrote: >> Hi, >> >> Git always performs a clean/smudge filter on files in sequential order. >> Sometimes a filter operation can take

Re: RFC: Enable delayed responses to Git clean/smudge filter requests

Lars Schneider <larsxschnei...@gmail.com> wrote: > Hi, > > Git always performs a clean/smudge filter on files in sequential order. > Sometimes a filter operation can take a noticeable amount of time. > This blocks the entire Git process. I have the same problem in many pl

RFC: Enable delayed responses to Git clean/smudge filter requests

Hi, Git always performs a clean/smudge filter on files in sequential order. Sometimes a filter operation can take a noticeable amount of time. This blocks the entire Git process. I would like to give a filter process the possibility to answer Git with "I got your request, I am proce

Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter

ttps://myrepo.git >>> >>> If no credential helper is setup, then Git asks the user for credentials. >>> Afterwards Git starts downloading stuff. At some point Git will run my >>> smudge filter on some files and in my case the smudge filter needs the >>>

Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter

word. > > > > > > Hm.. either I don't understand you or I expressed myself unclear. > > > > Let's say a user runs: > > > > $ git clone https://myrepo.git > > > > If no credential helper is setup, then Git asks the user for credentials

Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter

The credentials are > > > > totally orthogonal to the filters, and I would rather not shove them > > > > into the protocol. It's an extra process, but with the new multi-use > > > > smudge filter, it's one per git invocation, not one per file. > > > >

Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter

I would rather not shove them >>> into the protocol. It's an extra process, but with the new multi-use >>> smudge filter, it's one per git invocation, not one per file. >> >> The trouble with "git credential" is that it works only if the credential >> helper is setup c

Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter

On Fri, Nov 11, 2016 at 10:28:56AM +0100, Lars Schneider wrote: > > Yeah, that is the solution I was going to suggest. The credentials are > > totally orthogonal to the filters, and I would rather not shove them > > into the protocol. It's an extra process, but with the new mu

Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter

he credentials are > totally orthogonal to the filters, and I would rather not shove them > into the protocol. It's an extra process, but with the new multi-use > smudge filter, it's one per git invocation, not one per file. The trouble with "git credential" is that it

Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter

tials interactively, it grabs the terminal directly, so > it work even if stdin/stdout are used for the protocol. Yeah, that is the solution I was going to suggest. The credentials are totally orthogonal to the filters, and I would rather not shove them into the protocol. It's an extra process,

Re: [RFC] Add way to make Git credentials accessible from clean/smudge filter

Lars Schneider writes: > I haven't looked at an implemenation approach at all. I wonder if this could > be OK from a conceptional point of view or if there are obvious security > problems that I am missing. Did you consider just running "git credential" from the

[RFC] Add way to make Git credentials accessible from clean/smudge filter

Hi, we just implemented the first "real-world" user of the new clean/smudge "filter protocol" interface (see "convert: add filter..process option" edcc858 for details) and the results are fantastic. Filtering 12,000 files in my artificial test repo is m

Re: How to simulate a real checkout to test a new smudge filter?

On 9/10/16, Jakub Narębski <jna...@gmail.com> wrote: > You would need post-checkout hook together with clean / smudge filters > (though you could get by without smudge filter, at least in theory...). > The `post-checkout` hook could run e.g. "git checkout -- '*.conf'"

Re: How to simulate a real checkout to test a new smudge filter?

t checkin it's the same as committing a > file into the repository. I was wrong, I'm sorry. My mistake. You would need post-checkout hook together with clean / smudge filters (though you could get by without smudge filter, at least in theory...). The `post-checkout` hook could run e.g. "git

Re: How to simulate a real checkout to test a new smudge filter?

john smith writes: > ... get it. The problem is that in the scenario presented in my last > e-mail clean filter is run in the situation which doesn't like a > checkin to me. Is `git checkout ' doing a *checkin*" under the > hood so that the clean filter is called? What does

Re: How to simulate a real checkout to test a new smudge filter?

On 9/10/16, Junio C Hamano wrote: > The clean and smudge operations should look _only_ at the contents > they are filtering, and nothing else, and the clean/smudge filtering > mechanism is designed to support that use case. It is not designed > to do things like embedding the

Re: How to simulate a real checkout to test a new smudge filter?

john smith writes: > So it seems that clean filter is only run when checking out paths but > not when checking out branches. Is my thinking correct? If the new branch you are going to and your current branch record different contents for a path, that path will have to be

Re: How to simulate a real checkout to test a new smudge filter?

1. could be run automatically >> >> 2. do not modify files as shown by `git show HEAD:' and >> therefore no files are reported as modified by git status and also >> there are not conflicts when merging master into work/home branch. >> >> I have problems because w

Re: How to simulate a real checkout to test a new smudge filter?

own by `git show HEAD:' and > therefore no files are reported as modified by git status and also > there are not conflicts when merging master into work/home branch. > > I have problems because with point 1 because apparently smudge filter > is not run automatically every time when branch is changed

Re: How to simulate a real checkout to test a new smudge filter?

john smith venit, vidit, dixit 06.09.2016 23:01: > On 9/6/16, Torsten Bögershausen <tbo...@web.de> wrote: >> On 06.09.16 19:47, john smith wrote: >>> I am looking for a way to force smudge filter to run by simulating a >>> real life checkout. Let's say I

Re: How to simulate a real checkout to test a new smudge filter?

On 9/6/16, Torsten Bögershausen <tbo...@web.de> wrote: > On 06.09.16 19:47, john smith wrote: >> I am looking for a way to force smudge filter to run by simulating a >> real life checkout. Let's say I just created a new branch and did not >> modify any files but want t

Re: How to simulate a real checkout to test a new smudge filter?

On 06.09.16 19:47, john smith wrote: > I am looking for a way to force smudge filter to run by simulating a > real life checkout. Let's say I just created a new branch and did not > modify any files but want to test my new smudge filter. According to > some answers suc

How to simulate a real checkout to test a new smudge filter?

I am looking for a way to force smudge filter to run by simulating a real life checkout. Let's say I just created a new branch and did not modify any files but want to test my new smudge filter. According to some answers such as https://stackoverflow.com/questions/22909620/git-smudge-clean-filter

Re: [RFC] Long running Git clean/smudge filter

> On 10 Jul 2016, at 17:10, Joey Hess wrote: > > larsxschnei...@gmail.com wrote: >> (2) Joey's topic, which is the base for my patch, looks stalled for more than >> 2 weeks: >> http://thread.gmane.org/gmane.comp.version-control.git/297994/focus=298006 >> I would be happy to

Re: [RFC] Long running Git clean/smudge filter

larsxschnei...@gmail.com wrote: > (2) Joey's topic, which is the base for my patch, looks stalled for more than > 2 weeks: > http://thread.gmane.org/gmane.comp.version-control.git/297994/focus=298006 > I would be happy to address Junio's comments and post a reroll. However, > I don't want to

[RFC] Long running Git clean/smudge filter

s. If the filter command is there then we just take the existing and running process. In case of a clean filter we send the filename of the file to clean via pipe to the filter process. The filter process reads the file, generates the cleaned version and sends it back to Git via pipe. In case of a

Re: git smudge filter fails

Stephen Morton writes: > It's perhaps beyond the scope of my original question, but for > situations where I need a "last change date" embedded in a file (e.g. > because a protocol standard requires it), is there any recommended way > to do so? We've the hard way that

Re: git smudge filter fails

substitutions. > I will raise an Issue with ProGit. It's perhaps beyond the scope of my original question, but for situations where I need a "last change date" embedded in a file (e.g. because a protocol standard requires it), is there any recommended way to do so? We've the hard way that hardco

Re: git smudge filter fails

Jeff King writes: > On Thu, Mar 10, 2016 at 09:45:19AM -0500, Stephen Morton wrote: > >> I am a bit confused because this is basically the example used in >> ProGit [1] and it is fundamentally broken. In fact, if I understand >> correctly, this means that smudge filters cannot be

Re: git smudge filter fails

On Thu, Mar 10, 2016 at 09:45:19AM -0500, Stephen Morton wrote: > I am a bit confused because this is basically the example used in > ProGit [1] and it is fundamentally broken. In fact, if I understand > correctly, this means that smudge filters cannot be relied upon to > provide any 'keyword

Re: git smudge filter fails

I am a bit confused because this is basically the example used in ProGit [1] and it is fundamentally broken. In fact, if I understand correctly, this means that smudge filters cannot be relied upon to provide any 'keyword expansion' type tasks because they will all by nature have to query the file

Re: git smudge filter fails

On Wed, Mar 09, 2016 at 01:29:31PM -0500, Stephen Morton wrote: > git config --local filter.dater.smudge 'myDate=`git log > --pretty=format:"%cd" --date=iso -1 -- %f`; sed -e > "s/\(\\$\)Date[^\\$]*\\$/\1Date: $myDate \\$/g"' Your filter is running "git log" without a revision parameter, which

git smudge filter fails

A git smudge filter, at least one that relies on the results from 'git log' does not seem to work on file A when doing a 'git update' from a revision where file A doesn't exist to a revision where it does exist. Below is a simple recipe to reproduce. This appears to me to be a bug. If not, why

Re: Merging branches with smudge filter

[Please bottom-reply on this list] Leonardo venit, vidit, dixit 08.02.2016 18:52: > Hi. I understand what you mean, but if that's the case, I don't get > how every file was merged successfully despite the encryption, except > two of them. I thought the smudge filter was supposed to b

Re: Merging branches with smudge filter

Leonardo writes: > Hi, everybody. I'm new to git and I'd like to keep track of some codes > we have here in our company. They have some sensitive information I > would like to keep private. After some googling, I found some > solutions that encrypt/decrypt the files using

Re: Merging branches with smudge filter

Hi. I understand what you mean, but if that's the case, I don't get how every file was merged successfully despite the encryption, except two of them. I thought the smudge filter was supposed to be applied to every blob before any git operation, thus exposing the clean source code. Well

Merging branches with smudge filter

Hi, everybody. I'm new to git and I'd like to keep track of some codes we have here in our company. They have some sensitive information I would like to keep private. After some googling, I found some solutions that encrypt/decrypt the files using filters as they're committed/checked out. I've

[PATCH v2] convert: legitimately disable clean/smudge filter with an empty override

est_config/test_config_global to set configs (Thanks Eric) * use test_must_be_empty to check for empty err file (Thanks Eric) Cheers, Lars Lars Schneider (1): convert: legitimately disable clean/smudge filter with an empty override convert.c | 2 +- t/t0021-conversion.sh | 16 +

[PATCH v2] convert: legitimately disable clean/smudge filter with an empty override

l print an error for every file that is affected by the filter. Teach Git to consider an empty clean/smudge filter as legitimately disabled and do not print an error message if the filter is not required. Signed-off-by: Lars Schneider <larsxschnei...@gmail.com> --- convert.c | 2 +-

Re: [PATCH v2] convert: legitimately disable clean/smudge filter with an empty override

quired=false) then Git will run successfully. > However, Git will print an error for every file that is affected by the > filter. > > Teach Git to consider an empty clean/smudge filter as legitimately disabled > and do not print an error message if the filter is not required. > > Sig

Re: [PATCH v2] convert: legitimately disable clean/smudge filter with an empty override

Junio C Hamano writes: > Instead, teach apply_filter() to treat an empty string given as a > filter means the input must be returned as-is without conversion, > and the operation must always succeed. Ugh, that was a non-sentence. Instead, teach apply_filter() to treat an

Re: [PATCH v2] convert: legitimately disable clean/smudge filter with an empty override

and >> filter..clean) is set to an empty string ("") and the filter driver >> is >> not required (filter..required=false) then Git will run successfully. >> However, Git will print an error for every file that is affected by the >> filter. >> >> Teac

Re: [PATCH] convert: legitimately disable clean/smudge filter with an empty override

> However, Git will print an error for every file that is affected by the > filter. > > Teach Git to consider an empty clean/smudge filter as legitimately disabled > and do not print an error message if the filter is not required. That makes more sense to me. -- To unsubscribe from t

Re: [PATCH] convert: legitimately disable clean/smudge filter with an empty override

larsxschnei...@gmail.com writes: > - if (ca.drv) { > + if (ca.drv && ca.drv->smudge && strlen(ca.drv->smudge)) { You are not interested in its length, but if it is an empty string or not, so I'd tweak this like so: > + if (ca.drv && ca.drv->smudge && *ca.drv->smudge) { -- To

Re: [PATCH] convert: legitimately disable clean/smudge filter with an empty override

On 25 Jan 2016, at 02:25, Junio C Hamano <gits...@pobox.com> wrote: > larsxschnei...@gmail.com writes: > >> From: Lars Schneider <larsxschnei...@gmail.com> >> >> A clean/smudge filter can be disabled if set to an empty string. However, >> Git will tr

Re: [PATCH] convert: legitimately disable clean/smudge filter with an empty override

On 24 Jan 2016, at 22:45, Jeff King <p...@peff.net> wrote: > On Sun, Jan 24, 2016 at 01:22:50PM +0100, larsxschnei...@gmail.com wrote: > >> From: Lars Schneider <larsxschnei...@gmail.com> >> >> A clean/smudge filter can be disabled if set to an empty str

Re: [PATCH] convert: legitimately disable clean/smudge filter with an empty override

On 24 Jan 2016, at 22:35, Eric Sunshine wrote: > On Sun, Jan 24, 2016 at 10:06 AM, Torsten Bögershausen wrote: >> On 24.01.16 13:22, larsxschnei...@gmail.com wrote: >>> From: Lars Schneider >>> diff --git a/convert.c

Re: [PATCH] convert: legitimately disable clean/smudge filter with an empty override

On 24 Jan 2016, at 16:06, Torsten Bögershausen <tbo...@web.de> wrote: > On 24.01.16 13:22, larsxschnei...@gmail.com wrote: >> From: Lars Schneider <larsxschnei...@gmail.com> > Some minor nits inside: >> >> A clean/smudge filter can be disabled if set to

Re: [PATCH] convert: legitimately disable clean/smudge filter with an empty override

larsxschnei...@gmail.com writes: > From: Lars Schneider <larsxschnei...@gmail.com> > > A clean/smudge filter can be disabled if set to an empty string. However, > Git will try to run the empty string as command which results in a error > message per processed file. The a

[PATCH] convert: legitimately disable clean/smudge filter with an empty override

From: Lars Schneider <larsxschnei...@gmail.com> A clean/smudge filter can be disabled if set to an empty string. However, Git will try to run the empty string as command which results in a error message per processed file. Teach Git to consider an empty clean/smudge filter as legiti

Re: [PATCH] convert: legitimately disable clean/smudge filter with an empty override

On 24.01.16 13:22, larsxschnei...@gmail.com wrote: > From: Lars Schneider <larsxschnei...@gmail.com> Some minor nits inside: > > A clean/smudge filter can be disabled if set to an empty string. "set to an empty string" refers to "git config" (in op

Re: [PATCH] convert: legitimately disable clean/smudge filter with an empty override

On Sun, Jan 24, 2016 at 01:22:50PM +0100, larsxschnei...@gmail.com wrote: > From: Lars Schneider <larsxschnei...@gmail.com> > > A clean/smudge filter can be disabled if set to an empty string. However, > Git will try to run the empty string as command which results in a e

Re: [PATCH] convert: legitimately disable clean/smudge filter with an empty override

On Sun, Jan 24, 2016 at 10:06 AM, Torsten Bögershausen wrote: > On 24.01.16 13:22, larsxschnei...@gmail.com wrote: >> From: Lars Schneider >> diff --git a/convert.c b/convert.c >> @@ -786,7 +786,7 @@ int convert_to_git(const char *path, const char *src,

git $Id$ smudge filter

Currently, I've got a perl script that modifies the Id line in a smudge filter: [filter ident-line] smudge = /usr/local/bin/githook_ident-filter.pl %f The problem I've noticed with smudge filters is that it leaves the repo dirty. How do I fix this? I am basically trying to replicate

Re: git $Id$ smudge filter

shawn wilson ag4ve...@gmail.com writes: Currently, I've got a perl script that modifies the Id line in a smudge filter: [filter ident-line] smudge = /usr/local/bin/githook_ident-filter.pl %f The problem I've noticed with smudge filters is that it leaves the repo dirty. How do I fix

smudge filter

Can someone take a look at this and let me know what I'm doing wrong? Also, what's the best way to test filters? I can't really do -Debug or really even print various output. https://github.com/ag4ve/github-test -- To unsubscribe from this list: send the line unsubscribe git in the body of a