Re: [fpc-pascal] Why is FPImage dog slow in reading jpeg's?
On Mon, 30 May 2011 23:24:30 +0200 (CEST) Michael Van Canneyt mich...@freepascal.org wrote: On Mon, 30 May 2011, Felipe Monteiro de Carvalho wrote: On Mon, May 30, 2011 at 10:16 PM, Marco van de Voort mar...@stack.nl wrote: http://bugs.freepascal.org/view.php?id=17621 Indeed, I was remembering that too =) And also, I have a program which loads a huge image (20MB) using TPortableNetworkGraphics and it takes minutes to load the image. I have simply supposed that it was because of the file system usage, but now I am starting to think that maybe the fcl-image pallete causes the slowness. So I have to suggest: Is there a convincing reason for having Pallete set to true by default? What do you think about the trivial solution of setting it off by default? It is now off by default. Committed in revision 17613. Great. :) Mattias ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Why is FPImage dog slow in reading jpeg's?
Try the following code:
Img:=TFPMemoryImage.Create(0,0);
Img.UsePalette:=False;
Img.loadFromFile(FileList[i]);
That should work much faster. The reason is probably that the default image
created by the reader uses a palette, which is of course dog slow when reading
large images.
Michael.
On Mon, 30 May 2011, Ben Smith wrote:
Hi,
I wrote this console utility app in 5 minutes. All it is supposed to
do, is iterate over *.jpg images (65 at the moment) in a folder (no
recursive searches), and generate a snipped of HTML that I need to
paste into my wives website.
I first thought my utility app is in some recursive loop or something
because the output didn't appear instantly - like I thought it would.
I added some debug output between the generate HTML. To my surprise
the FPImage code reading a 35Kb jpg file takes +- 8 seconds and a 65KB
file takes +- 30 seconds - on a Quad Core machine! WTF???
Is this normal? Is FPImage really that slow reading small roughly
650x450 pixel jpg images? Or am I doing something unbelievably
stupid? :-/
The reason I'm reading in the jpg files is so I can get their Width
and Height values. Avg. jpeg file sizes are between 35-45KB, and
largest file size is 65KB.
Here is the relevant code snippet
var
imgreader: TFPReaderJPEG;
stream: TFileStream;
img:TFPCustomImage;
writeln('processing file: ', filelist[i]);
stream := TFileStream.Create(filelist[i], fmOpenRead);
writeln(1);
imgreader := TFPReaderJPEG.Create;
writeln(2, FormatDateTime(' hh:mm:ss.zz', Now) + ' (start reading)');
img := imgreader.ImageRead(stream, nil);
writeln(3, FormatDateTime(' hh:mm:ss.zz', Now) + ' (done)');
Here is the program output... Note the timestamps between start
reading and done reading (lines marked with '2' and '3').
---[ start ]---
$ ./project1 img/babies/full
Finished search. Found 65 matches
---
div id=thumbnails
table cellspacing=10
tr align=center valign=middel !-- Row starts --
td
processing file: img/babies/full/babies.01.jpg
1
2 16:00:30.210 (start reading)
3 16:00:38.338 (done)
img src=img/babies/full/babies.01.jpg alt= id=screen1
class=imglarge onmouseout=reducethumb(1); return false;
onclick=reducethumb(1); return false;
img src=img/babies/thumbs/babies.01.jpg alt= class=imgsmall
id=thumb1 onclick=expandthumb(1, 650, 413);
/td
/tr
tr align=center valign=middel !-- Row starts --
td
processing file: img/babies/full/babies.02.jpg
1
2 16:00:38.338 (start reading)
3 16:01:10.447 (done)
img src=img/babies/full/babies.02.jpg alt= id=screen2
class=imglarge onmouseout=reducethumb(2); return false;
onclick=reducethumb(2); return false;
img src=img/babies/thumbs/babies.02.jpg alt= class=imgsmall
id=thumb2 onclick=expandthumb(2, 650, 435);
/td
/tr
tr align=center valign=middel !-- Row starts --
td
processing file: img/babies/full/babies.03.jpg
1
2 16:01:10.447 (start reading)
3 16:01:28.859 (done)
img src=img/babies/full/babies.03.jpg alt= id=screen3
class=imglarge onmouseout=reducethumb(3); return false;
onclick=reducethumb(3); return false;
img src=img/babies/thumbs/babies.03.jpg alt= class=imgsmall
id=thumb3 onclick=expandthumb(3, 405, 650);
/td
/tr
tr align=center valign=middel !-- Row starts --
td
processing file: img/babies/full/babies.04.jpg
1
2 16:01:28.859 (start reading)
snip
---[ end ]---
--
Ben.
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Why is FPImage dog slow in reading jpeg's?
On Mon, May 30, 2011 at 4:16 PM, Michael wrote: snip That should work much faster. The reason is probably that the default image Ah, thanks Michael that did the trick. I knew I had to be doing something wrong. -- Ben. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Why is FPImage dog slow in reading jpeg's?
On Mon, May 30, 2011 16:16, michael.vancann...@wisa.be wrote:
Try the following code:
Img:=TFPMemoryImage.Create(0,0);
Img.UsePalette:=False;
Img.loadFromFile(FileList[i]);
That should work much faster. The reason is probably that the default
image
created by the reader uses a palette, which is of course dog slow when
reading
large images.
Interesting. What is supposed to be a large image? I ask because the
original post mentioned 35 kB and 65 kB (and indirectly also dimensions
around 600x400 if I understand the program output correctly) which I
certainly wouldn't consider as large (and the mentioned time necessary
for reading it surely not appropriate regardless of using palette or not).
I guess that some optimization may be reasonable because such pictures
could be loaded and displayed (including conversion from true colour to
256 colours palette) faster even on a 486 machine...
Tomas
On Mon, 30 May 2011, Ben Smith wrote:
Hi,
I wrote this console utility app in 5 minutes. All it is supposed to
do, is iterate over *.jpg images (65 at the moment) in a folder (no
recursive searches), and generate a snipped of HTML that I need to
paste into my wives website.
I first thought my utility app is in some recursive loop or something
because the output didn't appear instantly - like I thought it would.
I added some debug output between the generate HTML. To my surprise
the FPImage code reading a 35Kb jpg file takes +- 8 seconds and a 65KB
file takes +- 30 seconds - on a Quad Core machine! WTF???
Is this normal? Is FPImage really that slow reading small roughly
650x450 pixel jpg images? Or am I doing something unbelievably
stupid? :-/
The reason I'm reading in the jpg files is so I can get their Width
and Height values. Avg. jpeg file sizes are between 35-45KB, and
largest file size is 65KB.
Here is the relevant code snippet
var
imgreader: TFPReaderJPEG;
stream: TFileStream;
img:TFPCustomImage;
writeln('processing file: ', filelist[i]);
stream := TFileStream.Create(filelist[i], fmOpenRead);
writeln(1);
imgreader := TFPReaderJPEG.Create;
writeln(2, FormatDateTime(' hh:mm:ss.zz', Now) + ' (start reading)');
img := imgreader.ImageRead(stream, nil);
writeln(3, FormatDateTime(' hh:mm:ss.zz', Now) + ' (done)');
Here is the program output... Note the timestamps between start
reading and done reading (lines marked with '2' and '3').
---[ start ]---
$ ./project1 img/babies/full
Finished search. Found 65 matches
---
div id=thumbnails
table cellspacing=10
tr align=center valign=middel !-- Row starts --
td
processing file: img/babies/full/babies.01.jpg
1
2 16:00:30.210 (start reading)
3 16:00:38.338 (done)
img src=img/babies/full/babies.01.jpg alt= id=screen1
class=imglarge onmouseout=reducethumb(1); return false;
onclick=reducethumb(1); return false;
img src=img/babies/thumbs/babies.01.jpg alt= class=imgsmall
id=thumb1 onclick=expandthumb(1, 650, 413);
/td
/tr
tr align=center valign=middel !-- Row starts --
td
processing file: img/babies/full/babies.02.jpg
1
2 16:00:38.338 (start reading)
3 16:01:10.447 (done)
img src=img/babies/full/babies.02.jpg alt= id=screen2
class=imglarge onmouseout=reducethumb(2); return false;
onclick=reducethumb(2); return false;
img src=img/babies/thumbs/babies.02.jpg alt= class=imgsmall
id=thumb2 onclick=expandthumb(2, 650, 435);
/td
/tr
tr align=center valign=middel !-- Row starts --
td
processing file: img/babies/full/babies.03.jpg
1
2 16:01:10.447 (start reading)
3 16:01:28.859 (done)
img src=img/babies/full/babies.03.jpg alt= id=screen3
class=imglarge onmouseout=reducethumb(3); return false;
onclick=reducethumb(3); return false;
img src=img/babies/thumbs/babies.03.jpg alt= class=imgsmall
id=thumb3 onclick=expandthumb(3, 405, 650);
/td
/tr
tr align=center valign=middel !-- Row starts --
td
processing file: img/babies/full/babies.04.jpg
1
2 16:01:28.859 (start reading)
snip
---[ end ]---
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Why is FPImage dog slow in reading jpeg's?
On Mon, 30 May 2011, Tomas Hajny wrote: On Mon, May 30, 2011 16:16, michael.vancann...@wisa.be wrote: Try the following code: Img:=TFPMemoryImage.Create(0,0); Img.UsePalette:=False; Img.loadFromFile(FileList[i]); That should work much faster. The reason is probably that the default image created by the reader uses a palette, which is of course dog slow when reading large images. Interesting. What is supposed to be a large image? I ask because the original post mentioned 35 kB and 65 kB (and indirectly also dimensions around 600x400 if I understand the program output correctly) which I certainly wouldn't consider as large (and the mentioned time necessary for reading it surely not appropriate regardless of using palette or not). I guess that some optimization may be reasonable because such pictures could be loaded and displayed (including conversion from true colour to 256 colours palette) faster even on a 486 machine... It depends on the colors used. The default palette algorithm simply allocates a slot per found color in the jpg and returns the index of the slow. For a picture, this will create a huge palette, in which the search for a color is in essence linear, so n^2 lookups. So, dog slow. You could obviously speed this up by creating a palette yourself, and limiting the number of colors. Or, best for pictures is not to use a palette, which is what my solution entails. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Why is FPImage dog slow in reading jpeg's?
Interesting. What is supposed to be a large image? I ask because the original post mentioned 35 kB and 65 kB (and indirectly also dimensions around 600x400 if I understand the program output correctly) which I I suppose he was referring to the color palette, and not the physical image size. I guess that is why The Gimp limits png images to 256 colors when I saved it as Indexed (which then generates a color palette). So I guess it is a standard issue with Index/Palette images thus limit your colors, or don't use a palette. I guess one change in FPImage (depending on how often this occurs obviously), could be to default to UsePalette = False instead. -- Ben. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Why is FPImage dog slow in reading jpeg's?
Tomas Hajny xhaj...@hajny.biz hat am 30. Mai 2011 um 16:50 geschrieben: On Mon, May 30, 2011 16:16, michael.vancann...@wisa.be wrote: Try the following code: Img:=TFPMemoryImage.Create(0,0); Img.UsePalette:=False; Img.loadFromFile(FileList[i]); That should work much faster. The reason is probably that the default image created by the reader uses a palette, which is of course dog slow when reading large images. Interesting. What is supposed to be a large image? I ask because the original post mentioned 35 kB and 65 kB (and indirectly also dimensions around 600x400 if I understand the program output correctly) which I certainly wouldn't consider as large (and the mentioned time necessary for reading it surely not appropriate regardless of using palette or not). I guess that some optimization may be reasonable because such pictures could be loaded and displayed (including conversion from true colour to 256 colours palette) faster even on a 486 machine... Is there any size, where a palette image is fast? I think this is a newbie trap and the default for UsePalette should be changed. Mattias___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Why is FPImage dog slow in reading jpeg's?
michael.vancann...@wisa.be wrote: Try the following code: Img:=TFPMemoryImage.Create(0,0); Img.UsePalette:=False; Img.loadFromFile(FileList[i]); That should work much faster. The reason is probably that the default image created by the reader uses a palette, which is of course dog slow when reading large images. Pardon me for jumping in, but is there an equivalent fix for slow operations in a TLazIntfImage? -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Why is FPImage dog slow in reading jpeg's?
On 30 May 2011, at 16:59, michael.vancann...@wisa.be wrote: It depends on the colors used. The default palette algorithm simply allocates a slot per found color in the jpg and returns the index of the slow. For a picture, this will create a huge palette, in which the search for a color is in essence linear, so n^2 lookups. Isn't it possible to automatically disable the palette automatically for images with a bit depth of 16 bits or more? I'm not aware of any image format that uses palettes at those bit depths (and even if there are such formats, they are at least not common in general use cases). Jonas___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Why is FPImage dog slow in reading jpeg's?
In our previous episode, Mattias Gaertner said: for reading it surely not appropriate regardless of using palette or not). I guess that some optimization may be reasonable because such pictures could be loaded and displayed (including conversion from true colour to 256 colours palette) faster even on a 486 machine... Is there any size, where a palette image is fast? I think this is a newbie trap and the default for UsePalette should be changed. Not just newbies :-) http://bugs.freepascal.org/view.php?id=17621 ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Why is FPImage dog slow in reading jpeg's?
On Mon, May 30, 2011 at 10:16 PM, Marco van de Voort mar...@stack.nlwrote: http://bugs.freepascal.org/view.php?id=17621 Indeed, I was remembering that too =) And also, I have a program which loads a huge image (20MB) using TPortableNetworkGraphics and it takes minutes to load the image. I have simply supposed that it was because of the file system usage, but now I am starting to think that maybe the fcl-image pallete causes the slowness. So I have to suggest: Is there a convincing reason for having Pallete set to true by default? What do you think about the trivial solution of setting it off by default? -- Felipe Monteiro de Carvalho ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Why is FPImage dog slow in reading jpeg's?
On Mon, May 30, 2011 at 10:22 PM, Felipe Monteiro de Carvalho wrote: So I have to suggest: Is there a convincing reason for having Pallete set to true by default? What do you think about the trivial solution of setting it off by default? It seems I have opened a hornets nest. :) But from the many replies, it seems everybody expects UsePalette to be False by default (instead of what it is now True). Maybe a new default should be considered. -- Ben. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Why is FPImage dog slow in reading jpeg's?
On Mon, 30 May 2011, Felipe Monteiro de Carvalho wrote: On Mon, May 30, 2011 at 10:16 PM, Marco van de Voort mar...@stack.nl wrote: http://bugs.freepascal.org/view.php?id=17621 Indeed, I was remembering that too =) And also, I have a program which loads a huge image (20MB) using TPortableNetworkGraphics and it takes minutes to load the image. I have simply supposed that it was because of the file system usage, but now I am starting to think that maybe the fcl-image pallete causes the slowness. So I have to suggest: Is there a convincing reason for having Pallete set to true by default? What do you think about the trivial solution of setting it off by default? It is now off by default. Committed in revision 17613. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
