Incidentally, some speed-ups can be obtained in ImageMagick by using the
'clone' command appropriately, duplicating the image in memory, (and
then appropriate 'delete's when done with it), rather than repeated
'convert' commands, which re-load it again from disk each time.
See eg the answer #2, by Mark Setchell, in this thread
http://stackoverflow.com/questions/25995030/image-magick-resize-on-server-side-taking-3-minutes-to-process-to-10-different-s
... but I suspect you guys know all this already, and this is advice to
my elders and betters as to how to suck eggs. :-)
- JH.
On 05/10/2014 23:35, James Heald wrote:
Thanks Brian,
I just want to say, I really do appreciate the trouble you're taking
over this.
For me as a user/uploader, what I find most frustrating is if we've
found a really good high-resolution copy of an image (or perhaps worked
really hard to clean it up), only to find that after treatment through
WP's scalers, it ends up looking significantly worse than the first
low-res image.
(I also don't understand why we get jaggies on SVG images, but that may
be a separate issue).
Part of the trouble is that we've got used to using an extremely
low-quality Image Magick option designed for throwaway thumbnails, and
instead we're using it for what ought to be the best production-quality
reference images that we serve at each resolution.
If you're telling me this is how it has to be, then I suppose we have to
live with that. But it's not really good enough.
But with these really big images, which typically aren't accessed
particularly often, I don't see why it's not at least possible to
produce best-quality rescalings at 2048px, running when server resources
allow, but not compromising on the quality, and then simply store them
-- they only need to be generated once, so surely it makes sense to get
them right.
(I suspect it probably gives better results to generate the 2048px in
best quality, and then generate lower-res images from that as required
cutting whatever corners necessary; rather than baking artefacts in at
2048px, because I doubt they would get unbaked after that, however good
the algorithms used).
As for the memory issue, surely there must be low-memory implementations
even of the best filters? A Lanczos filter falls off pretty rapidly
with distance, so it must be possible to produce a finite-length near
equivalent, and then process the image as overlapping tiles. Even if
the algorithm is using an FFT for the filtering, it can surely do it on
only part of the image at a time (if needs be).
In terms of timing, as I understand it from previous threads from Fae,
at the moment downsized versions are automatically requested as soon as
the image is uploaded -- which means GWToolset can bring Commons to its
knees if not properly throttled. But surely this could be modified so
that the initial reduced-sizing is only generated when resources are
available, unless a user has requested it. (Or alternatively, a
quick-but-nasty resizing could be generated initially, with improved
resizing queued to be generated when resources permit).
But thanks again for all your work on this,
All best,
James.
On 04/10/2014 14:14, Brian Wolff wrote:
Hi James, thanks for the feedback, its exactly the type of feedback i was
looking for.
I did see your comments on the vp earlier. I hadn't responded because i
wanted to do a little research first to better understand what filters
do,
and what our options are in the various tools we use (i'm definitely
not an
expert on image manipulation or signal processing, more just gluing
together pieces other people have made. However learning about this stuff
has been quite interesting)
So at first glance:
Our needs are:
*resize images relatively quickly (resizing done on demand at the moment
for first request. If it takes over a minute, that is very bad)
*resize images in under 512mb
Vips has some interpolation options. In my testing they did not seem to
give very good results. We are probably going to need to use vips for big
files in order to meet the above listed needs. In the current system we
make an intermediary thumbnail ~2048px, and then shrink further from
that.
We could perhaps explore using fancier algorithms on the second step.
Ill respond more fully at VP, when i have a more firm grip on the
different
options we can take.
--bawolff
On Oct 4, 2014 5:55 AM, "James Heald" <[email protected]> wrote:
Hi Brian,
Thanks for taking this on.
I've put some comments up in response to your post at
https://commons.wikimedia.org/wiki/Commons:Village_pump#Experimenting_with_using_VIPS_to_render_large_tiff_files
The short version is that many of these big files are maps or
engravings,
and really need an anti-aliassing filter (eg a Lanczos filter) when they
are reduced, otherwise we get nasty "jaggies" and other aliassing
artefacts.
Unfortunately, it seems that VIPS does not do any anti-aliassing (and
nor
does the Image Magick 'thumbnail' option we currently use).
It's a shame when we have very good high-resolution originals, to be
presenting downsized versions which look so poor.
For comparison, I have put up copies of a couple of the images rescaled
to 1280px using Image Magick's "resize" command, to show what potentially
can be achievable at this resolution, if we allow a few more clock
cycles.
All best,
James.
On 02/10/2014 22:57, Brian Wolff wrote:
Hi everyone.
tl;dr: Can we do https://gerrit.wikimedia.org/r/164476
Now that the pre-requisite patches for using VIPS with tiff has been
merged (Woo!), lets umm use it.
So for those who don't know what vips is, vips is an alternative to
image magick which can scale certain file formats in essentially
constant memory (Or probably to be pedantic, linear in the number of
pixels in the resulting file, instead of linear in the number of
pixels in the source). This means we would be able to make thumbnails
no matter how big the source file is. Which is good because we have
lots of very high resolution tiff files, such as [[File:Zoomit2.tif]]
and [[File:Zentralbibliothek Zürich - Mittelalterliche Stadt -
000005203.tif]]. We already use VIPS to scale png files larger than 20
megapixels, and non-progressive jpeg files can be scaled efficiently
with image magick, so tiff is the current pain point in terms of
scaling limits (although GIF is also painful).
I would like to propose the following:
First we experiment with turning it on for files > 50 megapixels.
Currently we do not even try to render such files, so I doubt this
will cause any community angst. To that end I proposed a patch (
https://gerrit.wikimedia.org/r/164476 ) that uses the following
settings:
array(
'conditions' => array(
'mimeType' => 'image/tiff',
'minShrinkFactor' => 1.2,
'minArea' => 5e7,
),
'sharpen' => array( 'sigma' => 0.8 ),
)
This will turn the feature on for big files (which currently do not
render), and also enable sharpening (Most tiff images benefit from it
and the community has asked for it repeatedly, I think its less
disruptive to enable sharpening at the same time as VIPS, instead of
two separate changes to tiff rendering).
I would propose we let that sit for a little bit. We should than have
a community discussion (With the commons community, since its hard to
have a discussion with every community, and commons (+esp. Glams) are
the people who care the most about this) to see if the community likes
that. Hopefully if all is well we could move to stage 2, which would
be something like:
array(
'conditions' => array(
'mimeType' => 'image/tiff',
'minShrinkFactor' => 1.2,
),
'sharpen' => array( 'sigma' => 0.8 ),
),
array(
'conditions' => array(
'mimeType' => 'image/tiff',
),
),
Anyways, thoughts. Does this sound like a good plan? Someone want to
be bold and deploy my change ;)
--bawolff
_______________________________________________
Multimedia mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/multimedia
_______________________________________________
Multimedia mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/multimedia
_______________________________________________
Multimedia mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/multimedia
_______________________________________________
Multimedia mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/multimedia
