On Mon, Apr 1, 2019 at 10:07 PM Duy Nguyen wrote:
>
> On Tue, Apr 2, 2019 at 7:52 AM Matheus Tavares
> wrote:
> > I downloaded chromium to give it a try and got (on a machine with i7 and
> > SSD, running Manjaro Linux):
> >
> > - 17s on blame for a file with long history[2]
> > - 2m on blame for
Duy Nguyen writes:
> On Tue, Apr 2, 2019 at 5:30 PM David Kastrup wrote:
>>
>> Duy Nguyen writes:
>>
>> > On Tue, Apr 2, 2019 at 7:52 AM Matheus Tavares
>> > wrote:
>> >> I downloaded chromium to give it a try and got (on a machine with i7 and
>> >> SSD, running Manjaro Linux):
>> >>
>> >> - 1
On Tue, Apr 2, 2019 at 5:30 PM David Kastrup wrote:
>
> Duy Nguyen writes:
>
> > On Tue, Apr 2, 2019 at 7:52 AM Matheus Tavares
> > wrote:
> >> I downloaded chromium to give it a try and got (on a machine with i7 and
> >> SSD, running Manjaro Linux):
> >>
> >> - 17s on blame for a file with long
Duy Nguyen writes:
> On Tue, Apr 2, 2019 at 7:52 AM Matheus Tavares
> wrote:
>> I downloaded chromium to give it a try and got (on a machine with i7 and
>> SSD, running Manjaro Linux):
>>
>> - 17s on blame for a file with long history[2]
>> - 2m on blame for a huge file[3]
>> - 15s on log for bo
On Tue, Apr 2, 2019 at 7:52 AM Matheus Tavares
wrote:
> I downloaded chromium to give it a try and got (on a machine with i7 and
> SSD, running Manjaro Linux):
>
> - 17s on blame for a file with long history[2]
> - 2m on blame for a huge file[3]
> - 15s on log for both [2] and [3]
> - 1s for git s
Hi,
I am planning to work on making pack access thread-safe as my GSoC
project, and after that, parallelize git blame or checkout. Or even use
the thread-safe pack access to improve the already parallel grep or
pack-objects.
With this in mind, I would like to know if the problem discussed in this
It shouldn't be hard for us to run some tests with this patch applied.
Will report back in a day or two.
On Tue, Feb 18, 2014 at 9:55 AM, Junio C Hamano wrote:
> Duy Nguyen writes:
>
>> On Sat, Feb 15, 2014 at 8:15 AM, Zachary Turner wrote:
>> ...
>>> 2) Use TLS as you suggest and have one fd
Duy Nguyen writes:
> On Sat, Feb 15, 2014 at 8:15 AM, Zachary Turner wrote:
> ...
>> 2) Use TLS as you suggest and have one fd per pack thread. Probably
>> the most complicated code change (at least for me, being a first-time
>> contributor)
>
> It's not so complicated. I suggested a patch [1]
On Sat, Feb 15, 2014 at 8:15 AM, Zachary Turner wrote:
> Even if we make that change to use TLS for this case, the
> implementation of pread() will still change in such a way that the
> semantics of pread() are different on Windows. Is that ok?
>
> Just to summarize, here's the viable approaches
Even if we make that change to use TLS for this case, the
implementation of pread() will still change in such a way that the
semantics of pread() are different on Windows. Is that ok?
Just to summarize, here's the viable approaches I've seen discussed so far:
1) Use _WINVER at compile time to se
On Sat, Feb 15, 2014 at 7:50 AM, Stefan Zager wrote:
> On Fri, Feb 14, 2014 at 4:45 PM, Duy Nguyen wrote:
>> On Sat, Feb 15, 2014 at 2:16 AM, Zachary Turner wrote:
>>> (Gah, sorry if you're receiving multiple emails to your personal
>>> addresses, I need to get used to manually setting Plain-tex
On Fri, Feb 14, 2014 at 4:45 PM, Duy Nguyen wrote:
> On Sat, Feb 15, 2014 at 2:16 AM, Zachary Turner wrote:
>> (Gah, sorry if you're receiving multiple emails to your personal
>> addresses, I need to get used to manually setting Plain-text mode
>> every time I send a message).
>>
>> For the mixed
On Sat, Feb 15, 2014 at 2:16 AM, Zachary Turner wrote:
> (Gah, sorry if you're receiving multiple emails to your personal
> addresses, I need to get used to manually setting Plain-text mode
> every time I send a message).
>
> For the mixed read, we wouldn't be looking for another caller of
> pread
Am 14.02.2014 20:16, schrieb Zachary Turner:
> For the mixed read, we wouldn't be looking for another caller of
> pread() (since it doesn't care what the file pointer is), but instead
> a caller of read() or lseek() (since those do depend on the current
> file pointer). In index-pack.c, I see two
On Fri, Feb 14, 2014 at 11:04 AM, Karsten Blees wrote:
>
> Damn...you're right, multi-threaded git-index-pack works fine, but some tests
> fail badly. Mixed reads would have to be from git_mmap, which is the only
> other caller of pread().
msysgit used git_mmap() as defined in compat/win32mmap.
On Fri, Feb 14, 2014 at 11:15 AM, Zachary Turner wrote:
> For the mixed read, we wouldn't be looking for another caller of pread()
> (since it doesn't care what the file pointer is), but instead a caller of
> read() or lseek(). In index-pack.c, I see two possible culprits:
>
> 1) A call to xread(
(Gah, sorry if you're receiving multiple emails to your personal
addresses, I need to get used to manually setting Plain-text mode
every time I send a message).
For the mixed read, we wouldn't be looking for another caller of
pread() (since it doesn't care what the file pointer is), but instead
a
Am 14.02.2014 00:09, schrieb Zachary Turner:
> To elaborate a little bit more, you can verify with a sample program
> that ReadFile with OVERLAPPED does in fact modify the HANDLE's file
> position. The documentation doesn't actually state one way or
> another. My original attempt at a patch didn
To elaborate a little bit more, you can verify with a sample program
that ReadFile with OVERLAPPED does in fact modify the HANDLE's file
position. The documentation doesn't actually state one way or
another. My original attempt at a patch didn't have the ReOpenFile,
and we experienced regular re
On Thu, Feb 13, 2014 at 2:51 PM, Karsten Blees wrote:
> Am 13.02.2014 19:38, schrieb Zachary Turner:
>
>> The only reason ReOpenFile is necessary at
>> all is because some code somewhere is mixing read-styles against the same
>> fd.
>>
>
> I don't understand...ReadFile with OVERLAPPED parameter do
Am 13.02.2014 19:38, schrieb Zachary Turner:
> The only reason ReOpenFile is necessary at
> all is because some code somewhere is mixing read-styles against the same
> fd.
>
I don't understand...ReadFile with OVERLAPPED parameter doesn't modify the
HANDLE's file position, so you should be abl
Karsten Blees gmail.com> writes:
>
> Am 12.02.2014 19:37, schrieb Erik Faye-Lund:
> > On Wed, Feb 12, 2014 at 7:34 PM, Stefan Zager google.com>
wrote:
> >> On Wed, Feb 12, 2014 at 10:27 AM, Erik Faye-Lund
gmail.com> wrote:
> >>> On Wed, Feb 12, 2014 at 7:20 PM, Stefan Zager google.com>
wro
On Thu, Feb 13, 2014 at 12:27 AM, Johannes Sixt wrote:
> Am 2/12/2014 20:30, schrieb Stefan Zager:
>> On Wed, Feb 12, 2014 at 11:22 AM, Karsten Blees
>> wrote:
>>> Am 12.02.2014 19:37, schrieb Erik Faye-Lund:
ReOpenFile, that's fantastic. Thanks a lot!
>>>
>>> ...but should be loaded dynami
On Thu, Feb 13, 2014 at 06:34:39PM +0900, Mike Hommey wrote:
> On Thu, Feb 13, 2014 at 07:04:02AM +0100, David Kastrup wrote:
> > Mike Hommey writes:
> >
> > > On Wed, Feb 12, 2014 at 08:15:24PM +0100, David Kastrup wrote:
> > >> Stefan Zager writes:
> > >>
> > >> > On Wed, Feb 12, 2014 at 10:5
On Thu, Feb 13, 2014 at 07:04:02AM +0100, David Kastrup wrote:
> Mike Hommey writes:
>
> > On Wed, Feb 12, 2014 at 08:15:24PM +0100, David Kastrup wrote:
> >> Stefan Zager writes:
> >>
> >> > On Wed, Feb 12, 2014 at 10:50 AM, David Kastrup wrote:
> >> >
> >> >> Really, give the above patch a t
Johannes Sixt writes:
> Am 2/12/2014 20:30, schrieb Stefan Zager:
>> On Wed, Feb 12, 2014 at 11:22 AM, Karsten Blees
>> wrote:
>>> Am 12.02.2014 19:37, schrieb Erik Faye-Lund:
ReOpenFile, that's fantastic. Thanks a lot!
>>>
>>> ...but should be loaded dynamically via GetProcAddress, or are
David Kastrup writes:
> Again, that's with an SSD and ext4 filesystem on GNU/Linux, and there
> are no improvements in system time (I/O) except for patch 4 of the
> series which helps perhaps 20% or so.
>
> So the benefits of the patch will come into play mostly for big, bad
> files on Windows: o
Mike Hommey writes:
> On Wed, Feb 12, 2014 at 08:15:24PM +0100, David Kastrup wrote:
>> Stefan Zager writes:
>>
>> > On Wed, Feb 12, 2014 at 10:50 AM, David Kastrup wrote:
>> >
>> >> Really, give the above patch a try. I am taking longer to finish it
>> >> than anticipated (with a lot due to
Am 2/12/2014 20:30, schrieb Stefan Zager:
> On Wed, Feb 12, 2014 at 11:22 AM, Karsten Blees
> wrote:
>> Am 12.02.2014 19:37, schrieb Erik Faye-Lund:
>>> ReOpenFile, that's fantastic. Thanks a lot!
>>
>> ...but should be loaded dynamically via GetProcAddress, or are we ready to
>> drop XP support
On Tue, Feb 11, 2014 at 05:54:51PM -0800, Stefan Zager wrote:
> To this end, I'd like to start submitting patches that make the code
> base generally more thread-safe and thread-friendly. Right after this
> email, I'm going to send the first such patch, which makes the global
> list of pack files
Am 13.02.2014 00:03, schrieb Mike Hommey:
> On Wed, Feb 12, 2014 at 12:00:19PM +0100, Karsten Blees wrote:
>> Am 12.02.2014 04:43, schrieb Duy Nguyen:
>>> On Wed, Feb 12, 2014 at 9:02 AM, Robin H. Johnson
>>> wrote:
On Tue, Feb 11, 2014 at 05:54:51PM -0800, Stefan Zager wrote:
> We in t
On Wed, Feb 12, 2014 at 12:00:19PM +0100, Karsten Blees wrote:
> Am 12.02.2014 04:43, schrieb Duy Nguyen:
> > On Wed, Feb 12, 2014 at 9:02 AM, Robin H. Johnson
> > wrote:
> >> On Tue, Feb 11, 2014 at 05:54:51PM -0800, Stefan Zager wrote:
> >>> We in the chromium project have a keen interest in a
On Wed, Feb 12, 2014 at 08:15:24PM +0100, David Kastrup wrote:
> Stefan Zager writes:
>
> > On Wed, Feb 12, 2014 at 10:50 AM, David Kastrup wrote:
> >
> >> Really, give the above patch a try. I am taking longer to finish it
> >> than anticipated (with a lot due to procrastination but that is,
>
On Wed, Feb 12, 2014 at 12:27 PM, Stefan Zager wrote:
> On Wed, Feb 12, 2014 at 12:06 PM, Junio C Hamano wrote:
>> Stefan Zager writes:
>>
>> Calls to write (and preparation of data to be written) will then
>> remain single-threaded, but it sounds like that codepath is not the
>> bottleneck in y
On Wed, Feb 12, 2014 at 12:06 PM, Junio C Hamano wrote:
> Stefan Zager writes:
>
>> ... I used the Very Sleepy profiler
>> to see where all the time was spent on Windows: 55% of the time was
>> spent in OpenFile, and 25% in CloseFile (both in win32).
>
> This is somewhat interesting.
>
> When we
Stefan Zager writes:
> ... I used the Very Sleepy profiler
> to see where all the time was spent on Windows: 55% of the time was
> spent in OpenFile, and 25% in CloseFile (both in win32).
This is somewhat interesting.
When we check things out, checkout_paths() has a list of paths to be
checked
On Wed, Feb 12, 2014 at 11:22 AM, Karsten Blees wrote:
> Am 12.02.2014 19:37, schrieb Erik Faye-Lund:
>> On Wed, Feb 12, 2014 at 7:34 PM, Stefan Zager wrote:
>>> On Wed, Feb 12, 2014 at 10:27 AM, Erik Faye-Lund
>>> wrote:
On Wed, Feb 12, 2014 at 7:20 PM, Stefan Zager wrote:
>
> I
Am 12.02.2014 19:37, schrieb Erik Faye-Lund:
> On Wed, Feb 12, 2014 at 7:34 PM, Stefan Zager wrote:
>> On Wed, Feb 12, 2014 at 10:27 AM, Erik Faye-Lund wrote:
>>> On Wed, Feb 12, 2014 at 7:20 PM, Stefan Zager wrote:
I don't want to steal the thunder of my coworker, who wrote the
i
Stefan Zager writes:
> On Wed, Feb 12, 2014 at 10:50 AM, David Kastrup wrote:
>
>> Really, give the above patch a try. I am taking longer to finish it
>> than anticipated (with a lot due to procrastination but that is,
>> unfortunately, a large part of my workflow), and it's cutting into my
>>
On Wed, Feb 12, 2014 at 10:50 AM, David Kastrup wrote:
> Stefan Zager writes:
>
>> Anything on Windows that touches a lot of files is miserable due to
>> the usual file system slowness on Windows, and luafv.sys (the UAC file
>> virtualization driver) seems to make it much worse.
>
> There is an o
Stefan Zager writes:
> On Tue, Feb 11, 2014 at 6:11 PM, Duy Nguyen wrote:
>>
>> I have no comments about thread safety improvements (well, not yet).
>> If you have investigated about git performance on chromium
>> repositories, could you please sum it up? Threading may be an option
>> to improve
On Wed, Feb 12, 2014 at 7:34 PM, Stefan Zager wrote:
> On Wed, Feb 12, 2014 at 10:27 AM, Erik Faye-Lund wrote:
>> On Wed, Feb 12, 2014 at 7:20 PM, Stefan Zager wrote:
>>>
>>> I don't want to steal the thunder of my coworker, who wrote the
>>> implementation. He plans to submit it upstream soon-
On Wed, Feb 12, 2014 at 10:33 AM, Matthieu Moy
wrote:
> Stefan Zager writes:
>
>> I'm optimistic that parallelizing the stat calls will yield a further
>> improvement.
>
> It has already been mentionned in the thread, but in case you overlooked
> it: did you look at core.preloadindex? It seems at
On Wed, Feb 12, 2014 at 10:27 AM, Erik Faye-Lund wrote:
> On Wed, Feb 12, 2014 at 7:20 PM, Stefan Zager wrote:
>>
>> I don't want to steal the thunder of my coworker, who wrote the
>> implementation. He plans to submit it upstream soon-ish. It relies
>> on using the lpOverlapped argument to Rea
Stefan Zager writes:
> I'm optimistic that parallelizing the stat calls will yield a further
> improvement.
It has already been mentionned in the thread, but in case you overlooked
it: did you look at core.preloadindex? It seems at least very close to
what you want.
--
Matthieu Moy
http://www-
On Wed, Feb 12, 2014 at 7:20 PM, Stefan Zager wrote:
> On Wed, Feb 12, 2014 at 3:59 AM, Erik Faye-Lund wrote:
>> On Wed, Feb 12, 2014 at 2:54 AM, Stefan Zager wrote:
>>>
>>> We are particularly concerned with the performance of msysgit, and we
>>> have already chalked up a significant performanc
On Wed, Feb 12, 2014 at 3:59 AM, Erik Faye-Lund wrote:
> On Wed, Feb 12, 2014 at 2:54 AM, Stefan Zager wrote:
>>
>> We are particularly concerned with the performance of msysgit, and we
>> have already chalked up a significant performance gain by turning on
>> the threading code in pack-objects (
On Tue, Feb 11, 2014 at 7:43 PM, Duy Nguyen wrote:
>
> From v1.9 shallow clone should work for all push/pull/clone... so
> history depth does not matter (on the client side). As for
> gentoo-x86's large worktree, using index v4 and avoid full-tree
> operations (e.g. "status .", not "status"..) sho
On Tue, Feb 11, 2014 at 6:11 PM, Duy Nguyen wrote:
>
> I have no comments about thread safety improvements (well, not yet).
> If you have investigated about git performance on chromium
> repositories, could you please sum it up? Threading may be an option
> to improve performance, but it's probabl
On Wed, Feb 12, 2014 at 2:54 AM, Stefan Zager wrote:
> We in the chromium project have a keen interest in adding threading to
> git in the pursuit of performance for lengthy operations (checkout,
> status, blame, ...). Our motivation comes from hitting some
> performance walls when working with r
Am 12.02.2014 04:43, schrieb Duy Nguyen:
> On Wed, Feb 12, 2014 at 9:02 AM, Robin H. Johnson wrote:
>> On Tue, Feb 11, 2014 at 05:54:51PM -0800, Stefan Zager wrote:
>>> We in the chromium project have a keen interest in adding threading to
>>> git in the pursuit of performance for lengthy operati
On Wed, Feb 12, 2014 at 9:02 AM, Robin H. Johnson wrote:
> On Tue, Feb 11, 2014 at 05:54:51PM -0800, Stefan Zager wrote:
>> We in the chromium project have a keen interest in adding threading to
>> git in the pursuit of performance for lengthy operations (checkout,
>> status, blame, ...). Our mo
On Wed, Feb 12, 2014 at 8:54 AM, Stefan Zager wrote:
> We in the chromium project have a keen interest in adding threading to
> git in the pursuit of performance for lengthy operations (checkout,
> status, blame, ...). Our motivation comes from hitting some
> performance walls when working with r
On Tue, Feb 11, 2014 at 05:54:51PM -0800, Stefan Zager wrote:
> We in the chromium project have a keen interest in adding threading to
> git in the pursuit of performance for lengthy operations (checkout,
> status, blame, ...). Our motivation comes from hitting some
> performance walls when worki
We in the chromium project have a keen interest in adding threading to
git in the pursuit of performance for lengthy operations (checkout,
status, blame, ...). Our motivation comes from hitting some
performance walls when working with repositories the size of chromium
and blink:
https://chromium.
55 matches
Mail list logo
