Bug#855359: [Pkg-phototools-devel] Bug#855359: darktable: 2.2.1 binary very slow compared to self-compiled

2017-03-13 Thread David Bremner 
Matthias Bodenbinder  writes:

> Am 06.03.2017 um 20:45 schrieb David Bremner:

> Now I got your point. I downloaded 2.2.3-2 and tested it. The speed
> improvement is minimal: 19,6 s for the whole pixel pipeline vs. 21,9 s
> for version 2.2.3-1 and compared to 14 s for self-compiled.

I managed to roughly duplicate this behaviour on my laptop which is an
i7-6600U. In fact the speed difference is more dramatic, 51s for the
binary package and about 30s for the "self compiled" version (with
-march=native).

There is a small improvement (from 59s to 51s on my laptop) in -4 due to
using CMAKE_BUILD_TYPE=Release. I'm not sure if further improvements are
possible by tweaking buildflags, that will require some more test builds.

Bug#855359: [Pkg-phototools-devel] Bug#855359: darktable: 2.2.1 binary very slow compared to self-compiled

2017-03-06 Thread Matthias Bodenbinder 
Am 06.03.2017 um 20:45 schrieb David Bremner:
> Matthias Bodenbinder  writes:
> 
>> Am 06.03.2017 um 13:07 schrieb David Bremner:
>>> Matthias Bodenbinder  writes:
>>>
>>> Is that 32% difference with 2.2.3-1 or with 2.2.3-2?
>>>
>>> d
>>>
>>
>> Yes, this is the diff between DT 2.2.3 self-compiled vs. debian
>> experimental. But the diff is basically the same for DT 2.2.1
>> self-compiled vs. debian teating. I tested both scenarios.
> 
> That doesn't actually answer my question.
> 
> 

Now I got your point. I downloaded 2.2.3-2 and tested it. The speed improvement 
is minimal:
19,6 s for the whole pixel pipeline vs. 21,9 s for version 2.2.3-1 and compared 
to 14 s for self-compiled.

Matthias

Bug#855359: [Pkg-phototools-devel] Bug#855359: darktable: 2.2.1 binary very slow compared to self-compiled

2017-03-06 Thread David Bremner 
Matthias Bodenbinder  writes:

> Am 06.03.2017 um 13:07 schrieb David Bremner:
>> Matthias Bodenbinder  writes:
>> 
>> Is that 32% difference with 2.2.3-1 or with 2.2.3-2?
>> 
>> d
>> 
>
> Yes, this is the diff between DT 2.2.3 self-compiled vs. debian
> experimental. But the diff is basically the same for DT 2.2.1
> self-compiled vs. debian teating. I tested both scenarios.

That doesn't actually answer my question.

Bug#855359: [Pkg-phototools-devel] Bug#855359: darktable: 2.2.1 binary very slow compared to self-compiled

2017-03-06 Thread Matthias Bodenbinder 
Am 06.03.2017 um 13:07 schrieb David Bremner:
> Matthias Bodenbinder  writes:
> 
>>
>> I am wondering if it has to do with the fact that I am using the latest Kaby 
>> Lake CPU i7-7700k? 
>>
>> The Equalizer module in my case improves from 6,9 s to 3,0 s and non-local 
>> means noise reduction improves from 10,3 s to 8,7 s. 
>>
>> They both sum up to 17,2 s with the debian testing binary and 11,7 s with my 
>> self-compiled binary. This is a 32% difference. 
>>
>> What else can I do to help debugging? 
> 
> Is that 32% difference with 2.2.3-1 or with 2.2.3-2?
> 
> d
> 

Yes, this is the diff between DT 2.2.3 self-compiled vs. debian experimental. 
But the diff is basically the same for DT 2.2.1 self-compiled vs. debian 
teating. I tested both scenarios.

Bug#855359: [Pkg-phototools-devel] Bug#855359: darktable: 2.2.1 binary very slow compared to self-compiled

2017-03-06 Thread David Bremner 
Matthias Bodenbinder  writes:

>
> I am wondering if it has to do with the fact that I am using the latest Kaby 
> Lake CPU i7-7700k? 
>
> The Equalizer module in my case improves from 6,9 s to 3,0 s and non-local 
> means noise reduction improves from 10,3 s to 8,7 s. 
>
> They both sum up to 17,2 s with the debian testing binary and 11,7 s with my 
> self-compiled binary. This is a 32% difference. 
>
> What else can I do to help debugging? 

Is that 32% difference with 2.2.3-1 or with 2.2.3-2?

d

Bug#855359: [Pkg-phototools-devel] Bug#855359: darktable: 2.2.1 binary very slow compared to self-compiled

2017-03-05 Thread Matthias Bodenbinder 
Am 05.03.2017 um 18:41 schrieb David Bremner:
> I'm not able to reproduce the large difference between the debian
> package and compiling with
> 
>  ./build.sh --disable-gnome-keyring --build-type Release
> 
> On my machine (an older i7), I get an average of 40s to export with
> 2.2.3-2 (compiled with -O3, just uploaded) and an average of 39s with
> the build.sh produced binary.  In both cases the time is dominated by
> the non-local means noise reduction (about 24s).
>
> Looking only at the time in the equalizer module, the improvement is a
> slightly larger (as a percentage) 7.9 seconds for build.sh versions 8.5
> seconds for the debian package. Given we can't use -march=native or
> -mtune=native, that's probably an acceptable slowdown.

This is really interesting. And puzzling. 

I am wondering if it has to do with the fact that I am using the latest Kaby 
Lake CPU i7-7700k? 

The Equalizer module in my case improves from 6,9 s to 3,0 s and non-local 
means noise reduction improves from 10,3 s to 8,7 s. 

They both sum up to 17,2 s with the debian testing binary and 11,7 s with my 
self-compiled binary. This is a 32% difference. 

What else can I do to help debugging? 

Matthias

Bug#855359: [Pkg-phototools-devel] Bug#855359: darktable: 2.2.1 binary very slow compared to self-compiled

2017-03-05 Thread David Bremner 
Matthias Bodenbinder  writes:

> Am 28.02.2017 um 13:25 schrieb David Bremner:
>> Matthias Bodenbinder  writes:

> 1) download this RAW file: http://www.mirada.ch/bench.SRW
> 2) the xmp file for my test is attached to this email (bench.SRW.xmp)
> 3) put both files in the same directory
> 4) disable opencl in the darktable GUI
> 5) run "darktable-cli bench.SRW test.jpg --core -d perf -d opencl"

I'm not able to reproduce the large difference between the debian
package and compiling with

 ./build.sh --disable-gnome-keyring --build-type Release

On my machine (an older i7), I get an average of 40s to export with
2.2.3-2 (compiled with -O3, just uploaded) and an average of 39s with
the build.sh produced binary.  In both cases the time is dominated by
the non-local means noise reduction (about 24s).

Looking only at the time in the equalizer module, the improvement is a
slightly larger (as a percentage) 7.9 seconds for build.sh versions 8.5
seconds for the debian package. Given we can't use -march=native or
-mtune=native, that's probably an acceptable slowdown.

d

Bug#855359: [Pkg-phototools-devel] Bug#855359: darktable: 2.2.1 binary very slow compared to self-compiled

2017-02-28 Thread Matthias Bodenbinder 
Am 28.02.2017 um 13:25 schrieb David Bremner:
> Matthias Bodenbinder  writes:
> 
>>
>> I can provide DT 2 and DT 3 as deb files (approx. 9 MB each).
>>
> 
> That's not needed (or helpful). But did you some place provide the
> actual benchmark you are running, including the data?
> 
> I don't use the equalizer, so I suspect my images won't show such a
> dramatic difference.
> 
> d

I am basically following the benchmark described in this posting: 
https://www.mail-archive.com/darktable-user@lists.darktable.org/msg01636.html

1) download this RAW file: http://www.mirada.ch/bench.SRW
2) the xmp file for my test is attached to this email (bench.SRW.xmp)
3) put both files in the same directory
4) disable opencl in the darktable GUI
5) run "darktable-cli bench.SRW test.jpg --core -d perf -d opencl"

Matthias


 http://www.w3.org/1999/02/22-rdf-syntax-ns#;>
  http://ns.adobe.com/xap/1.0/mm/;
xmlns:xmp="http://ns.adobe.com/xap/1.0/;
xmlns:darktable="http://darktable.sf.net/;
   xmpMM:DerivedFrom="bench.SRW"
   xmp:Rating="1"
   darktable:xmp_version="2"
   darktable:raw_params="0"
   darktable:auto_presets_applied="1"
   darktable:history_end="17">

Bug#855359: [Pkg-phototools-devel] Bug#855359: darktable: 2.2.1 binary very slow compared to self-compiled

2017-02-28 Thread David Bremner 
Matthias Bodenbinder  writes:

>
> I can provide DT 2 and DT 3 as deb files (approx. 9 MB each).
>

That's not needed (or helpful). But did you some place provide the
actual benchmark you are running, including the data?

I don't use the equalizer, so I suspect my images won't show such a
dramatic difference.

d

Bug#855359: [Pkg-phototools-devel] Bug#855359: darktable: 2.2.1 binary very slow compared to self-compiled

2017-02-20 Thread Matthias Bodenbinder 
Am 20.02.2017 um 00:36 schrieb David Bremner:
> Matthias Bodenbinder  writes:
> 
>> And by the way, this are the commands I use to compile DT:
>>
>> ./build.sh --disable-gnome-keyring --prefix /home/software/darktable 
>> --build-type Release
>> cd build
>> echo "darktable 2.2.1" > description-pak
>> checkinstall --default --install=no --pkgname=darktable-mbo 
>> --pkgversion=$version --docdir=$INST/share/doc
>>
>> Matthias
> 
> The debian package does not use build.sh. I looks like build.sh does not
> pass -DBINARY_PACKAGE_BUILD=1 to cmake.
> 
> To test this, you could run something like
> 
> mkdir -p dtbuild && cd dtbuild && cmake  
> -DCMAKE_INSTALL_PREFIX=/opt/darktable \
>   -DCMAKE_BUILD_TYPE=Release  -DBINARY_PACKAGE_BUILD=1 .. 
> 
> then I guess your same checkinstall should work.
> 
> d
> 

Ok. I did the test. 

I tested 3 different DT binaries:

DT 1: binary from debian testing
DT 2: self-compiled with default DT build.sh
DT 3: self-compiled following your cmake commands

The result is interesting. The time shown is the average of 5 consecutive runs. 
The variance between each run is less than a second:

DT 1: binary from debian testing: 22 s
DT 2: self-compiled with default DT build.sh: 15 s
DT 3: self-compiled following your cmake commands   : 19 s

So your cmake variant is basicyll in the middle. 

I can provide DT 2 and DT 3 as deb files (approx. 9 MB each).

Matthias

Bug#855359: [Pkg-phototools-devel] Bug#855359: darktable: 2.2.1 binary very slow compared to self-compiled

2017-02-20 Thread David Bremner 
matth...@bodenbinder.de writes:

>
> Hi,
>
> I do not understand your point. My checkinstall is working fine. It is 
> just that my binaries are a lot faster than the debian testing binaries.
>
> Matthias

I was suggesting that you compile with the same cmake options that the debian
package uses. That will help confirm whether there is anything that can
be done about the situation.

d

Bug#855359: [Pkg-phototools-devel] Bug#855359: darktable: 2.2.1 binary very slow compared to self-compiled

2017-02-20 Thread Roman Lebedev 
On Mon, Feb 20, 2017 at 10:08 AM,   wrote:
> Am 2017-02-20 0:36, schrieb David Bremner:
>>
>> Matthias Bodenbinder  writes:
>>
>>> And by the way, this are the commands I use to compile DT:
>>>
>>> ./build.sh --disable-gnome-keyring --prefix /home/software/darktable
>>> --build-type Release
>>> cd build
>>> echo "darktable 2.2.1" > description-pak
>>> checkinstall --default --install=no --pkgname=darktable-mbo
>>> --pkgversion=$version --docdir=$INST/share/doc
>>>
>>> Matthias
>>
>>
>> The debian package does not use build.sh. I looks like build.sh does not
>> pass -DBINARY_PACKAGE_BUILD=1 to cmake.
>>
>> To test this, you could run something like
>>
>> mkdir -p dtbuild && cd dtbuild && cmake
>> -DCMAKE_INSTALL_PREFIX=/opt/darktable \
>>   -DCMAKE_BUILD_TYPE=Release  -DBINARY_PACKAGE_BUILD=1 ..
>>
>> then I guess your same checkinstall should work.
>>
>> d
>
>
> Hi,
>
> I do not understand your point. My checkinstall is working fine. It is just
> that my binaries are a lot faster than the debian testing binaries.
darktable custom-built binaries are optimized for whatever the host computer
is used during build.
debian packages do not have that luxury, they need to run on a wide variety
of hardware, so they are built with most generic optimizations possible

It is expected that self-compiled binaries may be faster than the .deb's

> Matthias
Roman.

Bug#855359: [Pkg-phototools-devel] Bug#855359: darktable: 2.2.1 binary very slow compared to self-compiled

2017-02-19 Thread matthias 

Am 2017-02-20 0:36, schrieb David Bremner:

Matthias Bodenbinder  writes:


And by the way, this are the commands I use to compile DT:

./build.sh --disable-gnome-keyring --prefix /home/software/darktable 
--build-type Release

cd build
echo "darktable 2.2.1" > description-pak
checkinstall --default --install=no --pkgname=darktable-mbo 
--pkgversion=$version --docdir=$INST/share/doc


Matthias


The debian package does not use build.sh. I looks like build.sh does 
not

pass -DBINARY_PACKAGE_BUILD=1 to cmake.

To test this, you could run something like

mkdir -p dtbuild && cd dtbuild && cmake  
-DCMAKE_INSTALL_PREFIX=/opt/darktable \

  -DCMAKE_BUILD_TYPE=Release  -DBINARY_PACKAGE_BUILD=1 ..

then I guess your same checkinstall should work.

d


Hi,

I do not understand your point. My checkinstall is working fine. It is 
just that my binaries are a lot faster than the debian testing binaries.


Matthias

Bug#855359: [Pkg-phototools-devel] Bug#855359: darktable: 2.2.1 binary very slow compared to self-compiled

2017-02-19 Thread David Bremner 
Matthias Bodenbinder  writes:

> And by the way, this are the commands I use to compile DT:
>
> ./build.sh --disable-gnome-keyring --prefix /home/software/darktable 
> --build-type Release
> cd build
> echo "darktable 2.2.1" > description-pak
> checkinstall --default --install=no --pkgname=darktable-mbo 
> --pkgversion=$version --docdir=$INST/share/doc
>
> Matthias

The debian package does not use build.sh. I looks like build.sh does not
pass -DBINARY_PACKAGE_BUILD=1 to cmake.

To test this, you could run something like

mkdir -p dtbuild && cd dtbuild && cmake  -DCMAKE_INSTALL_PREFIX=/opt/darktable \
  -DCMAKE_BUILD_TYPE=Release  -DBINARY_PACKAGE_BUILD=1 .. 

then I guess your same checkinstall should work.

d