[darktable-user] Re: CPU Recommendations

[Matthias Bodenbinder]

Am 07.02.2018 um 03:12 schrieb Michael Rasmussen:
> On Tue, 6 Feb 2018 17:52:48 -0800
> Scott  wrote:
> 
>> [default_process_tiling_opencl_ptp] couldn't run process_cl() for
>> module 'atrous' in tiling mode: 0
>> [opencl_pixelpipe] could not run module 'atrous' on gpu. falling back
>> to cpu path
> I sometimes sees this error as well if I have Google Chrome open.
> Closing Google Chrome and running 'atrous' on gpu succeeds.
> 
> I have not been able yet to test whether enabled hardware optimization
> in Google Chrome or not have any influence on this.
> 

I had similar isues with picture viewer geeqie. The issue was that with the 
geeqie running in parallel to DT, DT reached a point where it could not 
allocate enough GPU RAM anymore. So DT starts to use the CPU only. And DT would 
revover from that. Once the errors occurs DT sticks to the CPU for the rest of 
the session. A restart of DT was needed to get opencl back.  

Here is an example how that looked like in the DT log:

[opencl_denoiseprofile] couldn't enqueue kernel! -4
[opencl_pixelpipe] failed to run module 'denoiseprofile'. fall back to cpu path

I fixed this for me with "opencl_memory_headroom=300" in 
$HOME/.config/darktable/darktablerc . With this setting DT at least recovers 
from the memory error so that the following exports start using opencl again 
without restart.

Matthias


darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org

Re: [darktable-user] CPU Recommendations

[Michael Rasmussen]

On Tue, 6 Feb 2018 17:52:48 -0800
Scott  wrote:

> [default_process_tiling_opencl_ptp] couldn't run process_cl() for
> module 'atrous' in tiling mode: 0
> [opencl_pixelpipe] could not run module 'atrous' on gpu. falling back
> to cpu path
I sometimes sees this error as well if I have Google Chrome open.
Closing Google Chrome and running 'atrous' on gpu succeeds.

I have not been able yet to test whether enabled hardware optimization
in Google Chrome or not have any influence on this.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael  rasmussen  cc
http://pgp.mit.edu:11371/pks/lookup?op=get=0xD3C9A00E
mir  datanom  net
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE501F51C
mir  miras  org
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE3E80917
--
/usr/games/fortune -es says:
Sigh.  I like to think it's just the Linux people who want to be on
the "leading edge" so bad they walk right off the precipice.
(Craig E. Groeschel)


pgp_fLyVzuILB.pgp
Description: OpenPGP digital signature

Re: [darktable-user] CPU Recommendations

[Scott]

Just to add one more data point, another older system:

Darktable 2.4.1 on Ubuntu 17.10
Intel Core i5-3570K @ 3.40GHz (4 cores)
GeForce GTX 650 Ti BOOST
16 GB RAM

$ darktable-cli bench.SRW test.jpg --core --disable-opencl -d perf
[dev_process_export] pixel pipeline processing took 29.503 secs (101.837 CPU)

$ darktable-cli bench.SRW test.jpg --core -d perf -d opencl
[dev_process_export] pixel pipeline processing took 17.418 secs (36.790 CPU)
[opencl_summary_statistics] device 'GeForce GTX 650 Ti BOOST' (0): 545
out of 546 events were successful and 1 events lost

A couple of notes:
With no opencl, over half of the time was spent on the non-local means
denoise (15.28 seconds).  The equalizer was 6.8 seconds.
With opencl, the non-local means reduced to 6.56 seconds.  The
equalizer actually took longer at 7.6 seconds because this is the
opencl event that was lost on my machine:

[default_process_tiling_cl_ptp] use tiling on module 'atrous' for
image with full size 5490 x 3660
[default_process_tiling_cl_ptp] (3 x 1) tiles with max dimensions 2756
x 3660 and overlap 256
[default_process_tiling_cl_ptp] tile (0, 0) with 2756 x 3660 at origin [0, 0]
[opencl_atrous] couldn't enqueue kernel! -4
[default_process_tiling_opencl_ptp] couldn't run process_cl() for
module 'atrous' in tiling mode: 0
[opencl_pixelpipe] could not run module 'atrous' on gpu. falling back
to cpu path
[dev_pixelpipe] took 7.646 secs (25.516 CPU) processed `equalizer' on
CPU with tiling, blended on CPU [export]


On Tue, Feb 6, 2018 at 2:25 PM, Tobias Ellinghaus  wrote:
> Am Dienstag, 6. Februar 2018, 14:32:28 CET schrieb Guillermo Rozas:
>> But the manual explicitly says that "there can be multiple parallel
>> pixelpipes doing file exports and there can be multiple parallel
>> pixelpipes generating thumbnails". How are them allocated? With
>> "multiple" it refers to different queues from consecutive runs of the
>> export panel or thumbnails generator, or files from a single queue can
>> run in parallel if more than one device is available?
>
> Neither nor. Only one export job is run at a time, and for the running job
> only one image is processed at a time. Even when there are several GPUs that
> could do the computations you would still need huge amounts of memory.
>
>> Best regards,
>> Guillermo
>
> Tobias
>
>> On Tue, Feb 6, 2018 at 10:23 AM, Roman Lebedev  wrote:
>> > More like not really, that is *only* about darkroom editing (it will
>> > do main and preview pipes in parallel)
>> > It won't make any difference whatsoever for export.
>>
>> 
>> darktable user mailing list
>> to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org
>

darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org

Re: [darktable-user] CPU Recommendations

[Tobias Ellinghaus]

Am Dienstag, 6. Februar 2018, 14:32:28 CET schrieb Guillermo Rozas:
> But the manual explicitly says that "there can be multiple parallel
> pixelpipes doing file exports and there can be multiple parallel
> pixelpipes generating thumbnails". How are them allocated? With
> "multiple" it refers to different queues from consecutive runs of the
> export panel or thumbnails generator, or files from a single queue can
> run in parallel if more than one device is available?

Neither nor. Only one export job is run at a time, and for the running job 
only one image is processed at a time. Even when there are several GPUs that 
could do the computations you would still need huge amounts of memory.

> Best regards,
> Guillermo

Tobias

> On Tue, Feb 6, 2018 at 10:23 AM, Roman Lebedev  wrote:
> > More like not really, that is *only* about darkroom editing (it will
> > do main and preview pipes in parallel)
> > It won't make any difference whatsoever for export.
> 
> 
> darktable user mailing list
> to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org



signature.asc
Description: This is a digitally signed message part.

[darktable-user] Re: CPU Recommendations

[darktable]

Older system:

AMD 8350 4ghz (8 cores)
GeForce GTX 650 Ti
12Gb RAM

darktable-cli bench.SRW test1.jpg --core --disable-opencl -d perf
[dev_process_export] pixel pipeline processing took 29.812 secs (223.700 CPU)


darktable-cli bench.SRW test2.jpg --core -d perf -d opencl
[dev_process_export] pixel pipeline processing took 18.718 secs (24.033 CPU)
[opencl_summary_statistics] device 'GeForce GTX 650 Ti' (0): 1251 out of 1251 
events were successful and 0 events lost

-- 
sknahT

vyS

darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org

[darktable-user] Re: CPU Recommendations

[Matthias Bodenbinder]

Am 06.02.2018 um 16:41 schrieb Jean-Luc Lacroix:
> CPU i7-4790K
> GPU GeForce GTX 970 (1664 CUDA cores, 4 GB)
> RAM 24 GB
> 
> $ wget http://www.mirada.ch/bench.SRW
> $ wget http://www.mirada.ch/bench.SRW.xmp
> 
> $ darktable-cli bench.SRW test.jpg --core --disable-opencl -d perf
 pixel pipeline processing took *28.824* secs (223.044 CPU)
> 
> $ darktable-cli bench.SRW test.jpg --core -d perf -d opencl
 pixel pipeline processing took *5.235* secs (11.148 CPU)
> 
> Jean-Luc

This is funny. I have been using the exact same benchmark so far.
But I havent run it recently without opencl so that I obvioulsy underestimated 
the opencl performance with my claim that it just gives 30 % imporvement. In 
fact it is more like 50 % with my hardware:

CPU i7-7700K
GeForce GTX 750 Ti
RAM 32 GB

without opencl:
[dev_process_export] pixel pipeline processing took 14,018 secs (109,334 CPU)

with opencl:
[dev_process_export] pixel pipeline processing took 9,312 secs (11,503 CPU)

Matthias


darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org

Re: [darktable-user] Re: CPU Recommendations

[Michael Rasmussen]

On Tue, 6 Feb 2018 21:16:24 +0100
Michael Rasmussen  wrote:

> CPU AMD Phenom(tm) II X6 1045T Processor
Forgot to mention:
CPU overclocked from 2.7 GHz to 3.2 GHz.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael  rasmussen  cc
http://pgp.mit.edu:11371/pks/lookup?op=get=0xD3C9A00E
mir  datanom  net
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE501F51C
mir  miras  org
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE3E80917
--
/usr/games/fortune -es says:
Q:  What do you have when you have a lawyer buried up to his neck
in sand? A: Not enough sand.


pgpU509JfRsCa.pgp
Description: OpenPGP digital signature

Re: [darktable-user] Re: CPU Recommendations

[Michael Rasmussen]

CPU AMD Phenom(tm) II X6 1045T Processor
GPU GeForce GTX 1050
RAM 16GB

$ darktable-cli bench.SRW test.jpg --core --disable-opencl -d perf
 >>> pixel pipeline processing took 39,803 secs (221,981 CPU)

$ darktable-cli bench.SRW test.jpg --core -d perf -d opencl
 >>> pixel pipeline processing took 8,350 secs (16,093 CPU)

OpenCL can compensate for the lack of a 'descent' processor.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael  rasmussen  cc
http://pgp.mit.edu:11371/pks/lookup?op=get=0xD3C9A00E
mir  datanom  net
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE501F51C
mir  miras  org
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE3E80917
--
/usr/games/fortune -es says:
Linux poses a real challenge for those with a taste for late-night
hacking (and/or conversations with God).
-- Matt Welsh


pgpr3XxE4nG0P.pgp
Description: OpenPGP digital signature

Re: [darktable-user] Re: CPU Recommendations

[Denis Testemale]

Hello,
based on Jean-Luc samples files:

CPU i7 5820K
GPU GeForce GTX 750Ti
RAM 32Go

$ darktable-cli bench.SRW test.jpg --core --disable-opencl -d perf
 >>> pixel pipeline processing took 18,813 secs (168,012 CPU)

$ darktable-cli bench.SRW test.jpg --core -d perf -d opencl
 >>> pixel pipeline processing took 10,370 secs (17,896 CPU)

Cheers
denis


Le 06/02/2018 à 16:41, Jean-Luc Lacroix a écrit :
> CPU i7-4790K
> GPU GeForce GTX 970 (1664 CUDA cores, 4 GB)
> RAM 24 GB
> 
> $ wget http://www.mirada.ch/bench.SRW
> $ wget http://www.mirada.ch/bench.SRW.xmp
> 
> $ darktable-cli bench.SRW test.jpg --core --disable-opencl -d perf
 pixel pipeline processing took *28.824* secs (223.044 CPU)
> 
> $ darktable-cli bench.SRW test.jpg --core -d perf -d opencl
 pixel pipeline processing took *5.235* secs (11.148 CPU)
> 
> Jean-Luc
> 
> 
> On 06/02/18 16:03, Matthias Bodenbinder wrote:
>> Am 04.02.2018 um 23:38 schrieb Robert Bieber:
>>> Oh yeah, I'm just talking about the price of the CPU, not the whole
>>> PC. I'm planning on doing all out on RAM and solid state drives as
>>> well. I guess GPU might be of some concern as well, how has GPU
>>> processing in dark table on Linux been going lately? I know it
>>> exists, but I'm always scared of anything to do with graphics drivers
>>> on Linux
>>
>> I am using i7-7700k with NVIDIA GTX 750Ti. There is a 30% performance
>> improvement in the pixelpipe when using NVIDIA with opencl compared to
>> CPU only. And the GTX 750Ti is not the fastet card. So I  guess that
>> you can speed up DT easily by 50% with the proper graphics card.
>>
>> Matthias
>>
>> 
>>
>> darktable user mailing list
>> to unsubscribe send a mail to
>> darktable-user+unsubscr...@lists.darktable.org
>>
> 
> 
> darktable user mailing list
> to unsubscribe send a mail to
> darktable-user+unsubscr...@lists.darktable.org
> 


darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org

Aw: [darktable-user] Re: CPU Recommendations

[Michael Fritze]

Hi,

 

I am planning a new system myself, thanks to meltdown/spectre I am waiting a bit longer. I think fully fixed core will not be available this year so I'm going to build a system now.

 

Graphics Card Speed according to https://www.videocardbenchmark.net/gpu_list.php

750 Ti: 3712

1080 Ti: 13774


 

My guess is that a newer card would be much faster.

 

BR, Micha.


Gesendet: Dienstag, 06. Februar 2018 um 16:03 Uhr
Von: "Matthias Bodenbinder" 
An: darktable-user@lists.darktable.org
Betreff: [darktable-user] Re: CPU Recommendations

Am 04.02.2018 um 23:38 schrieb Robert Bieber:
> Oh yeah, I'm just talking about the price of the CPU, not the whole PC. I'm planning on doing all out on RAM and solid state drives as well. I guess GPU might be of some concern as well, how has GPU processing in dark table on Linux been going lately? I know it exists, but I'm always scared of anything to do with graphics drivers on Linux

I am using i7-7700k with NVIDIA GTX 750Ti. There is a 30% performance improvement in the pixelpipe when using NVIDIA with opencl compared to CPU only. And the GTX 750Ti is not the fastet card. So I guess that you can speed up DT easily by 50% with the proper graphics card.

Matthias


darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org
 





darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org

Re: [darktable-user] Re: CPU Recommendations

[Jean-Luc Lacroix]

CPU i7-4790K
GPU GeForce GTX 970 (1664 CUDA cores, 4 GB)
RAM 24 GB

$ wget http://www.mirada.ch/bench.SRW
$ wget http://www.mirada.ch/bench.SRW.xmp

$ darktable-cli bench.SRW test.jpg --core --disable-opencl -d perf
>>> pixel pipeline processing took *28.824* secs (223.044 CPU)

$ darktable-cli bench.SRW test.jpg --core -d perf -d opencl
>>> pixel pipeline processing took *5.235* secs (11.148 CPU)

Jean-Luc


On 06/02/18 16:03, Matthias Bodenbinder wrote:

Am 04.02.2018 um 23:38 schrieb Robert Bieber:

Oh yeah, I'm just talking about the price of the CPU, not the whole PC. I'm 
planning on doing all out on RAM and solid state drives as well. I guess GPU 
might be of some concern as well, how has GPU processing in dark table on Linux 
been going lately? I know it exists, but I'm always scared of anything to do 
with graphics drivers on Linux


I am using i7-7700k with NVIDIA GTX 750Ti. There is a 30% performance 
improvement in the pixelpipe when using NVIDIA with opencl compared to CPU 
only. And the GTX 750Ti is not the fastet card. So I  guess that you can speed 
up DT easily by 50% with the proper graphics card.

Matthias


darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org



darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org

[darktable-user] Re: CPU Recommendations

[Matthias Bodenbinder]

Am 04.02.2018 um 23:38 schrieb Robert Bieber:
> Oh yeah, I'm just talking about the price of the CPU, not the whole PC. I'm 
> planning on doing all out on RAM and solid state drives as well. I guess GPU 
> might be of some concern as well, how has GPU processing in dark table on 
> Linux been going lately? I know it exists, but I'm always scared of anything 
> to do with graphics drivers on Linux

I am using i7-7700k with NVIDIA GTX 750Ti. There is a 30% performance 
improvement in the pixelpipe when using NVIDIA with opencl compared to CPU 
only. And the GTX 750Ti is not the fastet card. So I  guess that you can speed 
up DT easily by 50% with the proper graphics card. 

Matthias


darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org

Re: [darktable-user] CPU Recommendations

[Guillermo Rozas]

But the manual explicitly says that "there can be multiple parallel
pixelpipes doing file exports and there can be multiple parallel
pixelpipes generating thumbnails". How are them allocated? With
"multiple" it refers to different queues from consecutive runs of the
export panel or thumbnails generator, or files from a single queue can
run in parallel if more than one device is available?

Best regards,
Guillermo

On Tue, Feb 6, 2018 at 10:23 AM, Roman Lebedev  wrote:
> More like not really, that is *only* about darkroom editing (it will
> do main and preview pipes in parallel)
> It won't make any difference whatsoever for export.
>

darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org

Re: [darktable-user] CPU Recommendations

[Roman Lebedev]

On Tue, Feb 6, 2018 at 4:18 PM, KOVÁCS István  wrote:
>
>
> On 6 Feb 2018 09:18, "Jean-Luc Lacroix"  wrote:
>
> Just out of curiosity: if I add another GPU card, would DT be able to use
> that additional GPU power?
>
> Cheers.
>
> Jean-Luc
>
>
> Yes.
More like not really, that is *only* about darkroom editing (it will
do main and preview pipes in parallel)
It won't make any difference whatsoever for export.

> https://www.darktable.org/usermanual/en/darktable_and_opencl_multiple_devices.html
>
> Kofa
Roman.

> 
> darktable user mailing list to unsubscribe send a mail to
> darktable-user+unsubscr...@lists.darktable.org

darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org

Re: [darktable-user] CPU Recommendations

[KOVÁCS István]

On 6 Feb 2018 09:18, "Jean-Luc Lacroix"  wrote:

Just out of curiosity: if I add another GPU card, would DT be able to use
that additional GPU power?

Cheers.

Jean-Luc


Yes.
https://www.darktable.org/usermanual/en/darktable_and_opencl_multiple_devices.html

Kofa


darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org

Re: [darktable-user] Focuspoints

[Tobias Ellinghaus]

Am Montag, 5. Februar 2018, 19:38:50 CET schrieb ternaryd:
> On Mon, 05 Feb 2018 15:00:19 +0100
> 
> Tobias Ellinghaus  wrote:
> > Am Montag, 5. Februar 2018, 08:49:34 CET
> 
> schrieb ternaryd:
> > No that is not possible in any clean way. It
> > has been asked for a few times in the past
> > but so far no one was interested enough to
> > provide a patch that would add that.
> 
> May I ask, what would break?

Nothing I hope. It's just that none of the devs think this is a feature worth 
the time and effort to write code for. So either someone who actually wants it 
steps up and provides the code, or it won't happen.

> > PS: In case you wonder, the dirty way is to
> > add a Lua guide to the cropping module and
> > from there call to command line exiv2/exiftool
> > to get the data and then draw it.
> 
> I was thinking more of an analogy to the
> indication of sharpness in lighttable (ctrl-z).
> This would mean a similar drawing, but a way
> simpler task of finding the spots, which must
> be really hard when qualifying image detail.
> 
> darktable user mailing list
> to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org



signature.asc
Description: This is a digitally signed message part.

Re: [darktable-user] CPU Recommendations

[Jean-Luc Lacroix]

I currently have a Nvidia GTX 970 that works flawlessly with its 
proprietary driver and OpenCL on a i7 Debian box.


Just out of curiosity: if I add another GPU card, would DT be able to 
use that additional GPU power?


Cheers.

Jean-Luc


On 05/02/18 17:50, Peter Mc Donough wrote:

Am 05.02.2018 um 02:35 schrieb Šarūnas:

On 02/04/2018 05:38 PM, Robert Bieber wrote:

Oh yeah, I'm just talking about the price of the CPU, not the whole PC.
I'm planning on doing all out on RAM and solid state drives as well. I
guess GPU might be of some concern as well, how has GPU processing in
dark table on Linux been going lately? I know it exists, but I'm always
scared of anything to do with graphics drivers on Linux


GPU might just be the upgrade with the biggest impact for RAW
processing. To have that impact it will need not only the drivers to
work, but also the corresponding OpenCL part...
...


This is exactly what you should keep in mind. As much RAM as you can 
afford, and an openCL capable graphics card. 16GB RAM and a 1TB SSD work 
nicely in my computer.


I run darktable with an 4-core AMD processor and an AMD-RX 460 graphics 
card in Ubuntu 16.04.3 with the amdgpu-pro driver (just the openCL part)


I copied some time ago a sort of test-suite, which shows that photo 
export is more than four times faster on my PC with openCL enabled.


1.899 secs without openCL
0.442 secs with openCL

You usually do not export just one photo, and openCL should also 
accelerate other processing tasks.


If you wish I can send you the zipped test per mail.

cu
Peter
 


darktable user mailing list
to unsubscribe send a mail to 
darktable-user+unsubscr...@lists.darktable.org




darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org