[GSoC][PATCH v5 1/7] clone: test for our behavior on odd objects/* content

2019-03-30 Thread Matheus Tavares
a...@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason [matheus.bernardino: improved and split tests in more than one patch] Helped-by: Matheus Tavares Signed-off-by: Matheus Tavares --- t/t5604-clone-reference.sh | 111 + 1 file changed, 111 insertions(+)

[GSoC][PATCH v5 6/7] clone: use dir-iterator to avoid explicit dir traversal

2019-03-30 Thread Matheus Tavares
error on readdir or stat, inside dir_iterator_advance. Previously it would just print a warning for errors on stat and ignore errors on readdir, which isn't nice because a local git clone could succeed even though the .git/objects copy didn't fully succeed. Signed-off-by: Mathe

[GSoC][PATCH v5 3/7] dir-iterator: add flags parameter to dir_iterator_begin

2019-03-30 Thread Matheus Tavares
DIR_ITERATOR_FOLLOW_SYMLINKS, which makes the iteration follow symlinks to directories and include its contents in the iteration. These new flags will be used in a subsequent patch. Also adjust refs/files-backend.c to the new dir_iterator_begin signature. Signed-off-by: Matheus Tavares --- dir-iterator.c | 28

[GSoC][PATCH v5 0/7] clone: dir-iterator refactoring with tests

2019-03-30 Thread Matheus Tavares
and may not be supported in all platforms git is being built. Now the same effect is achieved using real_pathdup() + link(). v4: https://public-inbox.org/git/20190322232237.13293-1-matheus.bernard...@usp.br/ Matheus Tavares (6): clone: better handle symlinked files at .git/objects/ dir-iter

[GSoC][PATCH v5 4/7] clone: copy hidden paths at local clone

2019-03-30 Thread Matheus Tavares
' and '..' only but it ended up accidentally skipping all directories starting with '.'. Besides being more natural, the new behaviour is more permissive to the user. Also adjust tests to reflect this behaviour change. Signed-off-by: Matheus Tavares Signed-off-by: Ævar Arnf

[GSoC][PATCH v5 5/7] clone: extract function from copy_or_link_directory

2019-03-30 Thread Matheus Tavares
Extract dir creation code snippet from copy_or_link_directory to its own function named mkdir_if_missing. This change will help to remove copy_or_link_directory's explicit recursion, which will be done in a following patch. Also makes the code more readable. Signed-off-by: Matheus Ta

[GSoC][PATCH v5 7/7] clone: replace strcmp by fspathcmp

2019-03-30 Thread Matheus Tavares
Replace the use of strcmp by fspathcmp at copy_or_link_directory, which is more permissive/friendly to case-insensitive file systems. Signed-off-by: Matheus Tavares Suggested-by: Nguyễn Thái Ngọc Duy --- builtin/clone.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a

[GSoC][PATCH v5 2/7] clone: better handle symlinked files at .git/objects/

2019-03-30 Thread Matheus Tavares
int to dereferenced paths, instead of the symlinks themselves. Also, add tests for symlinked files at .git/objects/. Note: Git won't create symlinks at .git/objects itself, but it's better to handle this case and be friendly with users who manually create them. Signed-off-by: Matheus

Re: Make the git codebase thread-safe

2019-04-01 Thread Matheus Tavares
] ./chrome/browser/about_flags.cc (same with ./DEPS) [3] third_party/sqlite/amalgamation/sqlite3.c (7.5M) Best, Matheus Tavares

[GSoC][PATCH v6 02/10] clone: better handle symlinked files at .git/objects/

2019-05-02 Thread Matheus Tavares
int to dereferenced paths, instead of the symlinks themselves. Also, add tests for symlinked files at .git/objects/. Note: Git won't create symlinks at .git/objects itself, but it's better to handle this case and be friendly with users who manually create them. Signed-off-by: Matheus

[GSoC][PATCH v6 03/10] dir-iterator: add tests for dir-iterator API

2019-05-02 Thread Matheus Tavares
Ferreira [matheus.bernardino: update to use test-tool and some minor aesthetics] Helped-by: Matheus Tavares Signed-off-by: Matheus Tavares --- Makefile | 1 + t/helper/test-dir-iterator.c | 33 ++ t/helper/test-tool.c | 1 + t/helper/test-tool.h

[GSoC][PATCH v6 01/10] clone: test for our behavior on odd objects/* content

2019-05-02 Thread Matheus Tavares
a...@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason [matheus.bernardino: improved and split tests in more than one patch] Helped-by: Matheus Tavares Signed-off-by: Matheus Tavares --- t/t5604-clone-reference.sh | 111 + 1 file changed, 111 insertions(+)

[GSoC][PATCH v6 04/10] dir-iterator: use warning_errno when possible

2019-05-02 Thread Matheus Tavares
Change warning(..., strerror(errno)) by warning_errno(...). This helps to unify warning display besides simplifying a bit the code. Also, improve warning messages by surrounding paths with quotation marks and using more meaningful statements. Signed-off-by: Matheus Tavares --- dir-iterator.c

[GSoC][PATCH v6 00/10] clone: dir-iterator refactoring with tests

2019-05-02 Thread Matheus Tavares
ts for dir-iterator API Matheus Tavares (8): clone: better handle symlinked files at .git/objects/ dir-iterator: use warning_errno when possible dir-iterator: refactor state machine model dir-iterator: add flags parameter to dir_iterator_begin clone: copy hidden paths at local clone clon

[GSoC][PATCH v6 06/10] dir-iterator: add flags parameter to dir_iterator_begin

2019-05-02 Thread Matheus Tavares
Signed-off-by: Matheus Tavares --- refs/files_backend.c is currently using no flags at the place it calls dir_iterator_begin(), to keep the same behavior it previously had. But as ITER_ERROR will now only be possibly returned by dir_iterator_avance() when DIR_ITERATOR_PEDANTIC is used a

[GSoC][PATCH v6 07/10] clone: copy hidden paths at local clone

2019-05-02 Thread Matheus Tavares
' and '..' only but it ended up accidentally skipping all directories starting with '.'. Besides being more natural, the new behaviour is more permissive to the user. Also adjust tests to reflect this behaviour change. Signed-off-by: Matheus Tavares Signed-off-by: Ævar Arnf

[GSoC][PATCH v6 10/10] clone: replace strcmp by fspathcmp

2019-05-02 Thread Matheus Tavares
Replace the use of strcmp by fspathcmp at copy_or_link_directory, which is more permissive/friendly to case-insensitive file systems. Signed-off-by: Matheus Tavares Suggested-by: Nguyễn Thái Ngọc Duy --- builtin/clone.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a

[GSoC][PATCH v6 09/10] clone: use dir-iterator to avoid explicit dir traversal

2019-05-02 Thread Matheus Tavares
recursive symlinks will be detected and skipped. This is another behavior improvement, since the current version would continue to copy the same content over and over until stat() returned an ELOOP error. Signed-off-by: Matheus Tavares --- builtin/cl

[GSoC][PATCH v6 05/10] dir-iterator: refactor state machine model

2019-05-02 Thread Matheus Tavares
) failures. Original-patch-by: Daniel Ferreira Signed-off-by: Matheus Tavares --- When dir_iterator_begin() fails at refs/files-backend.c, I used the same idea Daniel proposed, which is to initialize an empty iterator with empty_ref_iterator_begin(). Still, I'm not sure wether we shouldn&#x

[GSoC][PATCH v6 08/10] clone: extract function from copy_or_link_directory

2019-05-02 Thread Matheus Tavares
Extract dir creation code snippet from copy_or_link_directory to its own function named mkdir_if_missing. This change will help to remove copy_or_link_directory's explicit recursion, which will be done in a following patch. Also makes the code more readable. Signed-off-by: Matheus Ta

Re: [GSoC] Some #leftoverbits for anyone looking for little projects

2019-05-20 Thread Matheus Tavares
Hi, Ævar > Give "rebase -i" some option so when you "reword" the patch is > included in the message. > > I keep going to the shell because I have no idea what change I'm > describing. I have the same problem, so I wanted to try solving this. The patch bellow creates a "rebase.verboseCommit" confi

[GSoC][PATCH v7 00/10] clone: dir-iterator refactoring with tests

2019-06-18 Thread Matheus Tavares
lds/547451528 Daniel Ferreira (1): dir-iterator: add tests for dir-iterator API Matheus Tavares (8): clone: better handle symlinked files at .git/objects/ dir-iterator: use warning_errno when possible dir-iterator: refactor state machine model dir-iterator: add flags parameter to dir_itera

[GSoC][PATCH v7 02/10] clone: better handle symlinked files at .git/objects/

2019-06-18 Thread Matheus Tavares
int to dereferenced paths, instead of the symlinks themselves. Also, add tests for symlinked files at .git/objects/. Note: Git won't create symlinks at .git/objects itself, but it's better to handle this case and be friendly with users who manually create them. Signed-off-by: Matheus

[GSoC][PATCH v7 01/10] clone: test for our behavior on odd objects/* content

2019-06-18 Thread Matheus Tavares
a...@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason [matheus.bernardino: improved and split tests in more than one patch] Helped-by: Matheus Tavares Signed-off-by: Matheus Tavares --- t/t5604-clone-reference.sh | 111 + 1 file changed, 111 insertions(+)

[GSoC][PATCH v7 05/10] dir-iterator: refactor state machine model

2019-06-18 Thread Matheus Tavares
) failures. Original-patch-by: Daniel Ferreira Signed-off-by: Matheus Tavares --- dir-iterator.c | 234 ++- dir-iterator.h | 15 ++- refs/files-backend.c | 17 ++- t/helper/test-dir-iterator.c | 5 + t/t0066-dir-iterator.sh

[GSoC][PATCH v7 09/10] clone: use dir-iterator to avoid explicit dir traversal

2019-06-18 Thread Matheus Tavares
recursive symlinks will be detected and skipped. This is another behavior improvement, since the current version would continue to copy the same content over and over until stat() returned an ELOOP error. Signed-off-by: Matheus Tavares --- builtin/cl

[GSoC][PATCH v7 06/10] dir-iterator: add flags parameter to dir_iterator_begin

2019-06-18 Thread Matheus Tavares
Signed-off-by: Matheus Tavares --- dir-iterator.c | 82 +-- dir-iterator.h | 51 ++- refs/files-backend.c | 2 +- t/helper/test-dir-iterator.c | 34 ++--- t/t0066-dir-iterator.sh

[GSoC][PATCH v7 03/10] dir-iterator: add tests for dir-iterator API

2019-06-18 Thread Matheus Tavares
Ferreira [matheus.bernardino: update to use test-tool and some minor aesthetics] Helped-by: Matheus Tavares Signed-off-by: Matheus Tavares --- Makefile | 1 + t/helper/test-dir-iterator.c | 33 ++ t/helper/test-tool.c | 1 + t/helper/test-tool.h

[GSoC][PATCH v7 04/10] dir-iterator: use warning_errno when possible

2019-06-18 Thread Matheus Tavares
Change warning(..., strerror(errno)) by warning_errno(...). This helps to unify warning display besides simplifying a bit the code. Also, improve warning messages by surrounding paths with quotation marks and using more meaningful statements. Signed-off-by: Matheus Tavares --- dir-iterator.c

[GSoC][PATCH v7 08/10] clone: extract function from copy_or_link_directory

2019-06-18 Thread Matheus Tavares
Extract dir creation code snippet from copy_or_link_directory to its own function named mkdir_if_missing. This change will help to remove copy_or_link_directory's explicit recursion, which will be done in a following patch. Also makes the code more readable. Signed-off-by: Matheus Ta

[GSoC][PATCH v7 07/10] clone: copy hidden paths at local clone

2019-06-18 Thread Matheus Tavares
' and '..' only but it ended up accidentally skipping all directories starting with '.'. Besides being more natural, the new behaviour is more permissive to the user. Also adjust tests to reflect this behaviour change. Signed-off-by: Matheus Tavares Signed-off-by: Ævar Arnf

[GSoC][PATCH v7 10/10] clone: replace strcmp by fspathcmp

2019-06-18 Thread Matheus Tavares
Replace the use of strcmp by fspathcmp at copy_or_link_directory, which is more permissive/friendly to case-insensitive file systems. Suggested-by: Nguyễn Thái Ngọc Duy Signed-off-by: Matheus Tavares --- builtin/clone.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a

[GSoC][PATCH v8 01/10] clone: test for our behavior on odd objects/* content

2019-07-10 Thread Matheus Tavares
a...@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason [matheus.bernardino: improved and split tests in more than one patch] Helped-by: Matheus Tavares Signed-off-by: Matheus Tavares --- t/t5604-clone-reference.sh | 111 + 1 file changed, 111 insertions(+)

[GSoC][PATCH v8 00/10] clone: dir-iterator refactoring with tests

2019-07-10 Thread Matheus Tavares
r.qro.7.76.6.1907041136530...@tvgsbejvaqbjf.bet/ travis build: https://travis-ci.org/matheustavares/git/builds/557047597 Daniel Ferreira (1): dir-iterator: add tests for dir-iterator API Matheus Tavares (8): clone: better handle symlinked files at .git/objects/ dir-iterator: use warning_errno when possible

[GSoC][PATCH v8 02/10] clone: better handle symlinked files at .git/objects/

2019-07-10 Thread Matheus Tavares
int to dereferenced paths, instead of the symlinks themselves. Also, add tests for symlinked files at .git/objects/. Note: Git won't create symlinks at .git/objects itself, but it's better to handle this case and be friendly with users who manually create them. Signed-off-by: Matheus

[GSoC][PATCH v8 03/10] dir-iterator: add tests for dir-iterator API

2019-07-10 Thread Matheus Tavares
Ferreira [matheus.bernardino: update to use test-tool and some minor aesthetics] Helped-by: Matheus Tavares Signed-off-by: Matheus Tavares --- Makefile | 1 + t/helper/test-dir-iterator.c | 33 ++ t/helper/test-tool.c | 1 + t/helper/test-tool.h

[GSoC][PATCH v8 04/10] dir-iterator: use warning_errno when possible

2019-07-10 Thread Matheus Tavares
Change warning(..., strerror(errno)) by warning_errno(...). This helps to unify warning display besides simplifying a bit the code. Also, improve warning messages by surrounding paths with quotation marks and using more meaningful statements. Signed-off-by: Matheus Tavares --- dir-iterator.c

[GSoC][PATCH v8 05/10] dir-iterator: refactor state machine model

2019-07-10 Thread Matheus Tavares
) failures. Original-patch-by: Daniel Ferreira Signed-off-by: Matheus Tavares --- dir-iterator.c | 234 ++- dir-iterator.h | 17 ++- refs/files-backend.c | 17 ++- t/helper/test-dir-iterator.c | 5 + t/t0066-dir-iterator.sh

[GSoC][PATCH v8 09/10] clone: use dir-iterator to avoid explicit dir traversal

2019-07-10 Thread Matheus Tavares
error on readdir or stat inside dir_iterator_advance. Previously it would just print a warning for errors on stat and ignore errors on readdir, which isn't nice because a local git clone could succeed even though the .git/objects copy didn't fully succeed. Signed-off-by: Matheus Tavares --

[GSoC][PATCH v8 06/10] dir-iterator: add flags parameter to dir_iterator_begin

2019-07-10 Thread Matheus Tavares
Signed-off-by: Matheus Tavares --- dir-iterator.c | 56 + dir-iterator.h | 55 - refs/files-backend.c | 2 +- t/helper/test-dir-iterator.c | 34 +++ t/t0066-dir-iterator.sh

[GSoC][PATCH v8 07/10] clone: copy hidden paths at local clone

2019-07-10 Thread Matheus Tavares
' and '..' only but it ended up accidentally skipping all directories starting with '.'. Besides being more natural, the new behaviour is more permissive to the user. Also adjust tests to reflect this behaviour change. Signed-off-by: Matheus Tavares Signed-off-by: Ævar Arnf

[GSoC][PATCH v8 10/10] clone: replace strcmp by fspathcmp

2019-07-10 Thread Matheus Tavares
Replace the use of strcmp by fspathcmp at copy_or_link_directory, which is more permissive/friendly to case-insensitive file systems. Suggested-by: Nguyễn Thái Ngọc Duy Signed-off-by: Matheus Tavares --- builtin/clone.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a

[GSoC][PATCH v8 08/10] clone: extract function from copy_or_link_directory

2019-07-10 Thread Matheus Tavares
Extract dir creation code snippet from copy_or_link_directory to its own function named mkdir_if_missing. This change will help to remove copy_or_link_directory's explicit recursion, which will be done in a following patch. Also makes the code more readable. Signed-off-by: Matheus Ta

[GSoC][RFC WIP PATCH 0/3] grep: allow parallelism in zlib inflation

2019-07-19 Thread Matheus Tavares
lab.io/posts/week-6-working-at-zlib-inflation#multithreading-zlib-inflation [2]: chromium’s repo at commit 03ae96f (“Add filters testing at DSF=2”, 04-06-2019), after a 'git gc' execution. travis build: https://travis-ci.org/matheustavares/git/builds/561200398 Matheus Tavares (3):

[GSoC][RFC WIP PATCH 1/3] object-store: make read_object_file_extended() thread-safe

2019-07-19 Thread Matheus Tavares
s possible as phase I and phase III of unpack_entry() may execute concurrently. Signed-off-by: Matheus Tavares --- object-store.h | 4 packfile.c | 7 +++ sha1-file.c| 56 +- 3 files changed, 62 insertions(+), 5 deletions(-) diff

[GSoC][RFC WIP PATCH 2/3] grep: remove locks on object reading operations

2019-07-19 Thread Matheus Tavares
git-grep has locks around the object reading operations it performs. But these have an internal lock now, which can be enabled through enable_obj_read_lock(). So let's use it and drop git-grep's ones. Signed-off-by: Matheus Tavares --- builtin/grep.c | 18 +++--- grep.c

[GSoC][RFC WIP PATCH 3/3] grep: re-enable threads in non-worktree case

2019-07-19 Thread Matheus Tavares
after 2 warmup runs. All tests were executed on a i7-7700HQ with 16GB of RAM and SSD. [1]: chromium’s repo at commit 03ae96f (“Add filters testing at DSF=2”, 04-06-2019), after a 'git gc' execution. Signed-off-by: Matheus Tavares --- builtin/grep.c | 2 +- 1 file changed, 1 ins

[GSoC][PATCH] grep: fix worktree case in submodules

2019-07-30 Thread Matheus Tavares
nside grep_submodule(). Also, add tests to ensure that the desired behavior is performed. Reported-by: Daniel Zaoui Signed-off-by: Matheus Tavares --- builtin/grep.c | 10 ++ t/t7814-grep-recurse-submodules.sh | 21 + 2 files changed, 27 insertions(

[GSoC][PATCH v2] grep: fix worktree case in submodules

2019-07-31 Thread Matheus Tavares
nside grep_submodule(). Also, add tests to ensure that the desired behavior is performed. Reported-by: Daniel Zaoui Signed-off-by: Matheus Tavares --- Changes in v2: - repositioned the '/* ignored */' comment to avoid ambiguity - joined `git clean` and `git reset` invokations i

[GSoC][PATCH v3] grep: fix worktree case in submodules

2019-07-31 Thread Matheus Tavares
nside grep_submodule(). Also, add tests to ensure that the desired behavior is performed. Reported-by: Daniel Zaoui Signed-off-by: Matheus Tavares --- Changes in v3: - Replaced the "\* ignored *\" comment by a more meaningful note at the top of grep_submodule() builtin/grep.c

[GSoC][RFC PATCH] clone: use dir-iterator to avoid explicit dir traversal

2019-02-13 Thread Matheus Tavares
Replace usage of opendir/readdir/closedir API to traverse directories recursively, at copy_or_link_directory function, by the dir-iterator API. Signed-off-by: Matheus Tavares --- This is my microproject for GSoC 2019. It's still a RFC because I have some questions. Any help will be

[GSoC][PATCH 0/2] clone: convert explicit traversal to

2019-02-15 Thread Matheus Tavares
Convert explicit recursive dir traversal at copy_or_link_directory of clone.c to use dir-iterator API. This is my microproject for GSoC 2019. Idea taken from https://git.github.io/SoC-2019-Microprojects/#use-dir-iterator-to-avoid-explicit-recursive-directory-traversal Matheus Tavares (2

[GSoC][PATCH 2/2] clone: use dir-iterator to avoid explicit dir traversal

2019-02-15 Thread Matheus Tavares
Replace usage of opendir/readdir/closedir API to traverse directories recursively, at copy_or_link_directory function, by the dir-iterator API. Signed-off-by: Matheus Tavares --- builtin/clone.c | 39 +++ 1 file changed, 19 insertions(+), 20 deletions

[GSoC][PATCH 1/2] clone: extract function from copy_or_link_directory

2019-02-15 Thread Matheus Tavares
e more readable. Signed-off-by: Matheus Tavares --- builtin/clone.c | 27 +++ 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index 50bde99618..2a1cc4dab9 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -392,6 +392,24

[GSoC][PATCH 0/3] clone: convert explicit dir traversal to dir-iterator

2019-02-23 Thread Matheus Tavares
-to-avoid-explicit-recursive-directory-traversal Build: https://travis-ci.org/MatheusBernardino/git/builds/497512561 Matheus Tavares (3): dir-iterator: add pedantic option to dir_iterator_begin clone: extract function from copy_or_link_directory clone: use dir-iterator to avoid explicit dir

[GSoC][PATCH 3/3] clone: use dir-iterator to avoid explicit dir traversal

2019-02-23 Thread Matheus Tavares
irs since hidden dirs/files are not expected at .git/objects. - Now, copy_or_link_directory will call die() in case of an error on openddir, readdir or lstat, inside dir_iterator_advance. That means it will abort in case of an error trying to fetch any iteration entry. Signed-off-by:

[GSoC][PATCH 1/3] dir-iterator: add pedantic option to dir_iterator_begin

2019-02-23 Thread Matheus Tavares
r the next entry. Also adjust refs/files-backend.c to the new dir_iterator_begin signature. Signed-off-by: Matheus Tavares --- Changes in v2: - Added in v2 dir-iterator.c | 23 +-- dir-iterator.h | 16 +--- refs/files-backend.c | 2 +- 3 files ch

[GSoC][PATCH 2/3] clone: extract function from copy_or_link_directory

2019-02-23 Thread Matheus Tavares
Extract dir creation code snippet from copy_or_link_directory to its own function named mkdir_if_missing. This change will help removing copy_or_link_directory's explicit recursion, which will be done in a following patch. Also makes code more readable. Signed-off-by: Matheus Tavares --- Ch

[WIP RFC PATCH v2 0/5] clone: dir iterator refactoring with tests

2019-02-25 Thread Matheus Tavares
.gmail.com/ And also addresses the comments I made in WIP's v1. [1]: https://public-inbox.org/git/20190223190309.6728-1-matheus.bernard...@usp.br/ Matheus Tavares (4): dir-iterator: add flags parameter to dir_iterator_begin clone: copy hidden paths at local clone clone: e

[WIP RFC PATCH v2 2/5] clone: test for our behavior on odd objects/* content

2019-02-25 Thread Matheus Tavares
From: Ævar Arnfjörð Bjarmason We've implicitly supported .git/objects/* content of symlinks since approximately forever, and when we do a copy of the repo we transfer those over, but aren't very consistent about other random stuff we find depending on if it's a "hidden" file or not. Let's add a

[WIP RFC PATCH v2 5/5] clone: use dir-iterator to avoid explicit dir traversal

2019-02-25 Thread Matheus Tavares
error on readdir or stat, inside dir_iterator_advance. Previously it would just print a warning for errors on stat and ignore errors on readdir, which isn't nice because a local git clone would end up successfully even though the .git/objects copy didn't fully succeeded. Signed-off-by: Mathe

[WIP RFC PATCH v2 1/5] dir-iterator: add flags parameter to dir_iterator_begin

2019-02-25 Thread Matheus Tavares
DIR_ITERATOR_FOLLOW_SYMLINKS, which makes the iteration follow symlinks to directories and include its contents in the iteration. These new flags will be used in a subsequent patch. Also adjust refs/files-backend.c to the new dir_iterator_begin signature. Signed-off-by: Matheus Tavares --- dir-iterator.c | 28

[WIP RFC PATCH v2 3/5] clone: copy hidden paths at local clone

2019-02-25 Thread Matheus Tavares
'..' only but it ended up accidentally skipping all directories starting with '.'. Besides being more natural, the new behaviour is more permissive to the user. Also adjusted tests to reflect this change. Signed-off-by: Matheus Tavares Signed-off-by: Ævar Arnfjörð B

[WIP RFC PATCH v2 4/5] clone: extract function from copy_or_link_directory

2019-02-25 Thread Matheus Tavares
Extract dir creation code snippet from copy_or_link_directory to its own function named mkdir_if_missing. This change will help removing copy_or_link_directory's explicit recursion, which will be done in a following patch. Also makes code more readable. Signed-off-by: Matheus Ta

Re: [RFC PATCH v3 1/5] clone: test for our behavior on odd objects/* content

2019-03-12 Thread Matheus Tavares
Hi, Ævar First of all, I must apologize for my very late reply. I just got back from a trip and only now have been able to look again at this series. On Fri, Mar 1, 2019 at 10:49 AM Ævar Arnfjörð Bjarmason wrote: > > > On Thu, Feb 28 2019, Matheus Tavares Bernardino wrote: >

[GSoC][PATCH v4 6/7] clone: use dir-iterator to avoid explicit dir traversal

2019-03-22 Thread Matheus Tavares
error on readdir or stat, inside dir_iterator_advance. Previously it would just print a warning for errors on stat and ignore errors on readdir, which isn't nice because a local git clone would end up successfully even though the .git/objects copy didn't fully succeeded. Signed-off-by: Mathe

[GSoC][PATCH v4 3/7] dir-iterator: add flags parameter to dir_iterator_begin

2019-03-22 Thread Matheus Tavares
DIR_ITERATOR_FOLLOW_SYMLINKS, which makes the iteration follow symlinks to directories and include its contents in the iteration. These new flags will be used in a subsequent patch. Also adjust refs/files-backend.c to the new dir_iterator_begin signature. Signed-off-by: Matheus Tavares --- dir-iterator.c | 28

[GSoC][PATCH v4 7/7] clone: Replace strcmp by fspathcmp

2019-03-22 Thread Matheus Tavares
Replace the use of strcmp by fspathcmp at copy_or_link_directory, which is more permissive/friendly to case-insensitive file systems. Signed-off-by: Matheus Tavares Suggested-by: Nguyễn Thái Ngọc Duy --- builtin/clone.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a

[GSoC][PATCH v4 2/7] clone: better handle symlinked files at .git/objects/

2019-03-22 Thread Matheus Tavares
his case and be friendly with users who manually create them. Signed-off-by: Matheus Tavares Signed-off-by: Ævar Arnfjörð Bjarmason Co-authored-by: Ævar Arnfjörð Bjarmason --- builtin/clone.c| 2 +- t/t5604-clone-reference.sh | 26 +++--- 2 files changed, 20 insert

[GSoC][PATCH v4 1/7] clone: test for our behavior on odd objects/* content

2019-03-22 Thread Matheus Tavares
il.com/ Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Matheus Tavares Helped-by: Matheus Tavares --- t/t5604-clone-reference.sh | 116 + 1 file changed, 116 insertions(+) diff --git a/t/t5604-clone-reference.sh b/t/t5604-clone-reference.sh

[GSoC][PATCH v4 0/7] clone: dir-iterator refactoring with tests

2019-03-22 Thread Matheus Tavares
e usages of the string 'link' for 'symlink' just to avoid confusion with 'hardlinks' which are also known just by 'links'. v3: https://public-inbox.org/git/20190226122829.19178-1-ava...@gmail.com/ Matheus Tavares (6): clone: better handle symlinked fil

[GSoC][PATCH v4 4/7] clone: copy hidden paths at local clone

2019-03-22 Thread Matheus Tavares
' and '..' only but it ended up accidentally skipping all directories starting with '.'. Besides being more natural, the new behaviour is more permissive to the user. Also adjust tests to reflect this behaviour change. Signed-off-by: Matheus Tavares Signed-off-by: Ævar Arnf

[GSoC][PATCH v4 5/7] clone: extract function from copy_or_link_directory

2019-03-22 Thread Matheus Tavares
Extract dir creation code snippet from copy_or_link_directory to its own function named mkdir_if_missing. This change will help removing copy_or_link_directory's explicit recursion, which will be done in a following patch. Also makes code more readable. Signed-off-by: Matheus Ta

[GSoC][PATCH 4/4] grep: re-enable threads in some non-worktree cases

2019-08-10 Thread Matheus Tavares
]: chromium’s repo at commit 03ae96f (“Add filters testing at DSF=2”, 04-06-2019), after a 'git gc' execution. Signed-off-by: Matheus Tavares --- Documentation/git-grep.txt | 12 builtin/grep.c | 3 ++- 2 files changed, 14 insertions(+), 1 deletion(-) di

[GSoC][PATCH 1/4] object-store: add lock to read_object_file_extended()

2019-08-10 Thread Matheus Tavares
, but now it is since phase I and phase III of unpack_entry() may execute concurrently. Signed-off-by: Matheus Tavares --- object-store.h | 4 packfile.c | 7 ++ sha1-file.c| 61 +- 3 files changed, 67 insertions(+), 5 deletions(-

[GSoC][PATCH 0/4] grep: re-enable threads when cached, w/ parallel inflation

2019-08-10 Thread Matheus Tavares
. [1]: https://matheustavares.gitlab.io/posts/week-6-working-at-zlib-inflation#multithreading-zlib-inflation [2]: chromium’s repo at commit 03ae96f (“Add filters testing at DSF=2”, 04-06-2019), after a 'git gc' execution. travis build: https://travis-ci.org/matheustav

[GSoC][PATCH 2/4] grep: allow locks to be enabled individually

2019-08-10 Thread Matheus Tavares
ne in the following patch. Signed-off-by: Matheus Tavares --- builtin/grep.c | 2 +- grep.c | 4 ++-- grep.h | 8 ++-- 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/builtin/grep.c b/builtin/grep.c index 560051784e..a871bad8ad 100644 --- a/builtin/grep.c +++ b

[GSoC][PATCH 3/4] grep: disable grep_read_mutex when possible

2019-08-10 Thread Matheus Tavares
git-grep uses 'grep_read_mutex' to protect some object reading operations. But these have their own internal lock now, which ensure a better performance (with more parallel regions). So, disable the former when it's possible to use the latter, with enable_obj_read_lock(). Signed-

[PATCH 0/2] config: make config_with_options() handle any repo

2019-08-26 Thread Matheus Tavares
This patchset makes more config.c functions handle arbitrary repos and use that to remove an add_to_alternates_memory() call in submodule-config.c. This should hopefully benefit performance and memory. Matheus Tavares (2): config: allow config_with_options() to handle any repo submodule: pass

[PATCH 1/2] config: allow config_with_options() to handle any repo

2019-08-26 Thread Matheus Tavares
ehind NO_THE_REPOSITORY_COMPATIBILITY_MACROS. Finally, adjust documentation and add a rule to coccinelle to reflect the change. Note: the following patch will take care of actually using the added function in place of config_with_options() at config_from_gitmodules(). Signed-off-by: Matheus Ta

[PATCH 2/2] submodule: pass repo instead of adding to alternates list

2019-08-26 Thread Matheus Tavares
27;d right after use whereas the_repository's alternates list doesn't). While we are here, let's also adjust the comment on top of config_from_gitmodules() to be explicit that it also handles the case where .gitmodules is not present at the working tree. Signed-off-by: Matheus T

[RFC PATCH 0/3] grep: don't add subrepos to in-memory alternates

2019-09-17 Thread Matheus Tavares
_file()? (Again, for paths inside the submodule, read_object_file() won't be able to find the object as we won't be adding to alternates anymore.) Matheus Tavares (3): diff: use the given repo at diff_populate_filespec() object: allow parse_object_or_die() to handle any repo

[RFC PATCH 1/3] diff: use the given repo at diff_populate_filespec()

2019-09-17 Thread Matheus Tavares
diff_populate_filespec() accepts an struct repository "r" as parameter but implicitly uses the_repository when calling read_object_file(). To be more consistent, replace this call with repo_read_object_file(r, ...). Signed-off-by: Matheus Tavares --- diff.c | 2 +- 1 file changed, 1

[RFC PATCH 2/3] object: allow parse_object_or_die() to handle any repo

2019-09-17 Thread Matheus Tavares
r the third parameter as it's the default when NULL is given. This will be used in the following patch, to allow git-grep to parse submodule's objects. Signed-off-by: Matheus Tavares --- builtin/grep.c | 4 ++-- builtin/prune.c | 4 ++-- bundle.c| 13 - objec

[RFC PATCH 3/3] grep: don't add submodules to the alternates list

2019-09-17 Thread Matheus Tavares
will also help future patches to re-enable threads in non-worktree grep as less mutual exclusions will be needed. E.g. grep_submodule()'s call to parse_object_or_die() won't conflict with other calls to the same function by worker threads (as they should be referencing different '

[PATCH v2 01/11] grep: fix race conditions on userdiff calls

2019-09-29 Thread Matheus Tavares
acing grep_read_mutex for an internal object reading lock and allowing parallel inflation during object reading. Signed-off-by: Matheus Tavares --- grep.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/grep.c b/grep.c index cd952ef5d3..b29946def2 100644 --- a/grep.c +++ b/g

[PATCH v2 00/11] grep: improve threading and fix race conditions

2019-09-29 Thread Matheus Tavares
race condition fixes and being able to run --textconv and --recurse-submodules threaded now. Matheus Tavares (11): grep: fix race conditions on userdiff calls grep: fix race conditions at grep_submodule() grep: fix racy calls in grep_objects() replace-object: make replace operations thread

[PATCH v2 03/11] grep: fix racy calls in grep_objects()

2019-09-29 Thread Matheus Tavares
s with the call to gitmodules_config_oid() which also has parse_object() in its call stack. Fix that protecting both call with the said grep_read_mutex. Signed-off-by: Matheus Tavares --- builtin/grep.c | 5 + 1 file changed, 5 insertions(+) diff --git a/builtin/grep.c b/builtin/grep.c index

[PATCH v2 05/11] object-store: allow threaded access to object reading

2019-09-29 Thread Matheus Tavares
object reading lock only works in conjunction with the 'struct raw_object_store's replace_mutex. Otherwise, there would still be racy spots in object reading functions. Signed-off-by: Matheus Tavares --- object-store.h | 35 +++ packfile.c |

[PATCH v2 06/11] grep: replace grep_read_mutex by internal obj read lock

2019-09-29 Thread Matheus Tavares
some of them will require additional modifications to be removed from the critical section, this will be done in its own patch. Signed-off-by: Matheus Tavares --- builtin/grep.c | 46 -- grep.c | 39 +++

[PATCH v2 04/11] replace-object: make replace operations thread-safe

2019-09-29 Thread Matheus Tavares
;s not really worthy since the warning doesn't represent a real problem. Instead, to make sure we don't get false positives (at ThreadSanitizer, at least) an entry for the respective function is added to .tsan-suppressions. Signed-off-by: Matheus Tavares --- .tsan-suppressions | 6

[PATCH v2 02/11] grep: fix race conditions at grep_submodule()

2019-09-29 Thread Matheus Tavares
ow. But we better protect ourselves against this possibility, anyway. So, to solve these issues, move both of these function calls into the critical section of grep_read_mutex. Signed-off-by: Matheus Tavares --- builtin/grep.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git

[PATCH v2 08/11] grep: allow submodule functions to run in parallel

2019-09-29 Thread Matheus Tavares
racy with object readings in the superproject. So it must be kept protected for now. Let's add a "NEEDSWORK" to it, informing why it cannot be removed from the critical section yet. - Finally, add_to_alternates_memory() must be kept protected by the same reason of the above ite

[PATCH v2 11/11] grep: move driver pre-load out of critical section

2019-09-29 Thread Matheus Tavares
--- Out of critical section | 5.7852s [1]: Tests performed on an i7-7700HQ with 16GB of RAM and SSD, running Manjaro Linux. [2]: chromium’s repo at commit 03ae96f (“Add filters testing at DSF=2”, 04-06-2019), after a 'git gc' execution. Signed-off-by: Matheus Tavares ---

[PATCH v2 09/11] grep: protect packed_git [re-]initialization

2019-09-29 Thread Matheus Tavares
Again, all paths to it in git-grep are already protected by obj_read_lock() but it may suffer from the same problem in the future. So let's also internally protect it with obj_read_lock() (which is a recursive mutex). Signed-off-by: Matheus Tavares --- builtin/grep.c | 8 ++-- packfi

[PATCH v2 07/11] submodule-config: add skip_if_read option to repo_read_gitmodules()

2019-09-29 Thread Matheus Tavares
les() adding an 'skip_if_read' which allow both internal and external callers to access this functionality. This simplifies a little the code. The added option will also be used in the following patch. Signed-off-by: Matheus Tavares --- builtin/grep.c | 2 +-

[PATCH v2 10/11] grep: re-enable threads in non-worktree case

2019-09-29 Thread Matheus Tavares
ons, not being able to perform them in parallel can cause performance drops. To inform the users about this (and other threading detais), let's also add a "NOTES ON THREADS" section to Documentation/git-grep.txt. [1]: chromium’s repo at commit 03ae96f (“Add filters testin

[PATCH] squash! grep: replace grep_read_mutex by internal obj read lock

2019-10-01 Thread Matheus Tavares
Signed-off-by: Matheus Tavares --- This is just a small fixup to be squashed into patch 6: with multiple locks, the locking order must be consistent across all critical sections to avoid dead-lock. Since grep_attr_lock() is called before obj_read_lock() in grep_source_load_driver(), it must also

Re: [GSoC][PATCH v4 3/7] dir-iterator: add flags parameter to dir_iterator_begin

2019-03-29 Thread Matheus Tavares Bernardino
On Thu, Mar 28, 2019 at 7:19 PM Thomas Gummerer wrote: > > On 03/22, Matheus Tavares wrote: > > Add the possibility of giving flags to dir_iterator_begin to initialize > > a dir-iterator with special options. > > > > Currently possible flags are DIR_

Re: [GSoC][PATCH v4 1/7] clone: test for our behavior on odd objects/* content

2019-03-29 Thread Matheus Tavares Bernardino
On Thu, Mar 28, 2019 at 6:49 PM Thomas Gummerer wrote: > > On 03/22, Matheus Tavares wrote: > > From: Ævar Arnfjörð Bjarmason > > > > Add tests for what happens when we perform a local clone on a repo > > containing odd files at .git/object directory, such as

  1   2   >