Re: [fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-24 Thread Jan Nijtmans 
2014-02-21 13:56 GMT+01:00 Jan Nijtmans :
> I would say it's a GO. Any objections?


The Speed-up as suggested by Samuel Debionne is merged
to trunk now. I noticed considerable speedup on Windows and
Cygwin (up to 40%). I tested it on Ubuntu as well, but there
the speedup was not impressive, just a few %.

Thanks!
  Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-21 Thread Jan Nijtmans 
2014-02-21 12:43 GMT+01:00 Jan Nijtmans :
> The only operations this is expected giving a speedup is
> in "fossil extra" or "fossil addremove":  Functions which
> traverse the checked-out filesystem.

Measured on Cygwin64, on checked-out/built fossil repo (mtime-changes=0).

Original timing (current trunk):
$ time fossil extra
real0m0.349s
user0m0.078s
sys 0m0.233s

"use-t-type" branch timing:
$ time fossil extra
real0m0.211s
user0m0.015s
sys 0m0.109s

The same with mtime-changes=1:

Original timing (current trunk):
$ time fossil extra
real0m0.329s
user0m0.045s
sys 0m0.280s

"use-t-type" branch timing:
$ time fossil extra
real0m0.159s
user0m0.062s
sys 0m0.093s

That's a little more than the 36% gain Samuel reported.

I would say it's a GO. Any objections?

Regards,
   Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-21 Thread Jan Nijtmans 
2014-02-21 12:40 GMT+01:00 Stephan Beal :
> Not bad. Try "./fossil rebuild" - that one does a lot more i/o (and is
> harmless unless you have added your own tables to the repo db).


That wouldn't help. This improvement is for accessing the local
checkout, "fossil rebuild" is not an operation touching that.

The only operations this is expected giving a speedup is
in "fossil extra" or "fossil addremove":  Functions which
traverse the checked-out filesystem.

Regards,
   Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-21 Thread Stephan Beal 
On Fri, Feb 21, 2014 at 11:56 AM, Samuel Debionne <
samuel.debio...@ujf-grenoble.fr> wrote:

> $./fossil extra
>
> without _DIRENT_HAVE_D_TYPE : 30s - x1.0
> with _DIRENT_HAVE_D_TYPE : 22s - x1.36
>
> Nothing phenomenal but not bad neither, no ?
>


Not bad. Try "./fossil rebuild" - that one does a lot more i/o (and is
harmless unless you have added your own tables to the repo db).


-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-21 Thread Samuel Debionne 
> rebuild/deconstruct are very i/o intensive, and it might be easier
> to see the degree of change in longer-running ops like those.

I'm not very familiar with this commands...

Here are some raw performance numbers. The repository is a new (empty)
one where 10K of files are copied that are not versioned. To avoid cache
side effects, the command has been run multiple time
until the perf numbers stabilized.

$./fossil extra

without _DIRENT_HAVE_D_TYPE : 30s - x1.0
with _DIRENT_HAVE_D_TYPE : 22s - x1.36

Nothing phenomenal but not bad neither, no ?
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-21 Thread Samuel Debionne 
Le 20/02/2014 16:23, Jan Nijtmans a écrit :
> 2014-02-20 15:17 GMT+01:00 Samuel Debionne :
>> While debugging vfile.c I noticed that vfile_scan does not use
>> "dirent.d_type" to dispatch between folder and regular file/link but
>> uses an addition getStat call. I came across this thread that claims a
>> high speedup for command like "git status" with this kind of optimization :
>>
>> https://groups.google.com/forum/#!searchin/msysgit/d_type/msysgit/PGCFol99xUo/vjEPrajM_QsJ
>>
>> Has this been ever considered ? Worth a try ?
> 
> Something like:
>  
> ?
> 
> It looks like _DIRENT_HAVE_D_TYPE is set for MSC
> (win/include/dirent.h) and on WinGW-w64, but not bare
> MinGW. I have no idea whether's it's worth or not.

The mingwrt-4.0 release notes say:

"Redefine the structure for dirent and _wdirent to include d_type and be
  copesetic to Microsoft attributes."

http://sourceforge.net/p/mingw/mingw-org-wsl/ci/4.0.0/tree/NEWS

Unfortunaley I think there is a bug that I have just reported here :

http://sourceforge.net/p/mingw/bugs/2185/

Fortunately the workaround is quite easy.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-21 Thread Stephan Beal 
On Fri, Feb 21, 2014 at 9:18 AM, Samuel Debionne <
samuel.debio...@ujf-grenoble.fr> wrote:

> I'll give it a try and report some perf numbers for some commands on the
> fossil repo:
> fossil status
> fossil extra
>
> Does that sound correct ?
>

rebuild/deconstruct are very i/o intensive, and it might be easier to see
the degree of change in longer-running ops like those.

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-21 Thread Samuel Debionne 
> Something like:
>  
> ?

Yes, this is exactly what I had in mind. The thread on git suggest that
a speedup from x2-x6 on Windows (because the file system and mingw
support the d_type).

I'll give it a try and report some perf numbers for some commands on the
fossil repo:
fossil status
fossil extra

Does that sound correct ?

I can confirm that _DIRENT_HAVE_D_TYPE is NOT defined for the latest
MingW (but seems to be set).
<>___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-20 Thread Jan Nijtmans 
2014-02-20 15:17 GMT+01:00 Samuel Debionne :
> While debugging vfile.c I noticed that vfile_scan does not use
> "dirent.d_type" to dispatch between folder and regular file/link but
> uses an addition getStat call. I came across this thread that claims a
> high speedup for command like "git status" with this kind of optimization :
>
> https://groups.google.com/forum/#!searchin/msysgit/d_type/msysgit/PGCFol99xUo/vjEPrajM_QsJ
>
> Has this been ever considered ? Worth a try ?

Something like:
 
?

It looks like _DIRENT_HAVE_D_TYPE is set for MSC
(win/include/dirent.h) and on WinGW-w64, but not bare
MinGW. I have no idea whether's it's worth or not.

Regards,
Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-20 Thread Jan Nijtmans 
2014-02-20 15:17 GMT+01:00 Samuel Debionne :
> While debugging vfile.c I noticed that vfile_scan does not use
> "dirent.d_type" to dispatch between folder and regular file/link but
> uses an addition getStat call. I came across this thread that claims a
> high speedup for command like "git status" with this kind of optimization :
>
> https://groups.google.com/forum/#!searchin/msysgit/d_type/msysgit/PGCFol99xUo/vjEPrajM_QsJ
>
> Has this been ever considered ? Worth a try ?

See:


> This member is a BSD extension. The symbol _DIRENT_HAVE_D_TYPE is defined if 
> this member is available.
> On systems where it is used, it corresponds to the file type bits in the 
> st_mode member of struct stat.
> If the value cannot be determine the member value is DT_UNKNOWN

As far as I know it is never considered. I simply have no idea how
wide-spread it is.

Regards,
  Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-20 Thread Samuel Debionne 
While debugging vfile.c I noticed that vfile_scan does not use
"dirent.d_type" to dispatch between folder and regular file/link but
uses an addition getStat call. I came across this thread that claims a
high speedup for command like "git status" with this kind of optimization :

https://groups.google.com/forum/#!searchin/msysgit/d_type/msysgit/PGCFol99xUo/vjEPrajM_QsJ

Has this been ever considered ? Worth a try ?
Samuel
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-20 Thread Samuel Debionne 
Hello Jan,

> @Samuel, could you please as some comments to
> MinGW's bug report here:
> 
> This MinGW bug already has 5 duplications, and it is
> already open for more than 4 months. It's really time
> they bring out a fix for it...

With great pleasure ! And thanks to the pointers to those bug reports,
interesting reads indeed. If only I had found them before, that would
have saved my morning...

Samuel
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-20 Thread Jan Nijtmans 
2014-02-20 14:46 GMT+01:00 Jan Nijtmans :
> 2014-02-20 14:39 GMT+01:00 Samuel Debionne :
>> Hello all,
>>
>> Compiling Fossil with the latest MinGW32 (that includes MinGW Runtime >
>> 4.0) seems not to require _USE_32BIT_TIME_T.
> Yes it's a regression in MinGW, see:
> 
> and
> 
>
> Regards,
>  Jan Nijtmans


@Samuel, could you please as some comments to
MinGW's bug report here:

This MinGW bug already has 5 duplications, and it is
already open for more than 4 months. It's really time
they bring out a fix for it...

Thanks!
  Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-20 Thread Jan Nijtmans 
2014-02-20 14:39 GMT+01:00 Samuel Debionne :
> Hello all,
>
> Compiling Fossil with the latest MinGW32 (that includes MinGW Runtime >
> 4.0) seems not to require _USE_32BIT_TIME_T.
> Actually Fossil builds without error but will crash at runtime with any
> commands that browses the filesystem tree (vfile_scan). This is because
> the definition of _wdirent in fossil and in mingw does not match (one
> using 32 bits time and the other 64 bits). This may be a regression of
> MinGW Runtime...

Did you try it on Windows XP too?  -D_USE_32BIT_TIME_T is needed to
stay compatible with Windows XP. Starting with Vista everything works.

Yes it's a regression in MinGW, see:

and


Regards,
 Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] Latest MinGW does not need _USE_32BIT_TIME_T anymore

2014-02-20 Thread Samuel Debionne 
Hello all,

Compiling Fossil with the latest MinGW32 (that includes MinGW Runtime >
4.0) seems not to require _USE_32BIT_TIME_T.
Actually Fossil builds without error but will crash at runtime with any
commands that browses the filesystem tree (vfile_scan). This is because
the definition of _wdirent in fossil and in mingw does not match (one
using 32 bits time and the other 64 bits). This may be a regression of
MinGW Runtime...

AFAIK there is no easy fix for that in "config.h" since
_USE_32BIT_TIME_T must be defined before any inclusion of MinGW headers.
That means that __MINGW_MAJOR_VERSION cannot be used to conditionally
define _USE_32BIT_TIME_T. In other words, this would have been handy :

/* Make sure that in Win32 MinGW builds, _USE_32BIT_TIME_T is always
defined. */
#if defined(_WIN32) && !defined(_WIN64) && !defined(_MSC_VER) &&
!defined(_USE_32BIT_TIME_T)
#  if (__MINGW_MAJOR_VERSION < 4)
#define _USE_32BIT_TIME_T
#  endif
#endif

but won't work...

Samuel
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users