On Thu, 06 Sep 2018 13:03:03 -0500 Federico Mena Quintero <feder...@gnome.org> wrote:
> On Wed, 2018-09-05 at 17:28 +0100, Emmanuele Bassi via gtk-devel-list > wrote: > > > In the near future, I'll very likely deprecate most of GdkPixbuf's > > API, except for the I/O operations; I'd also be happy to seal off > > most of its internals, within the ABI stability promise, to avoid > > leakage of internal state. > > Related to this - I just wrote a braindump on gdk-pixbuf issues and > historical baggage. I hope it's useful for people trying to clean up > this swamp: > > https://people.gnome.org/~federico/blog/my-gdk-pixbuf-braindump.html This was an interesting read. It touched many of my own ideas and concerns. Animation I actually think the concept of animations in gdk-pixbuf is okay, so it's pretty much the same in abydos, with the main difference that abydos fully supports vector animation. Apart from the frame based view of animations abydos also support continuous animations (which it seems like gdk-pixbuf also does). It's mostly the API of gdk-pixbuf that is suboptimal. Apart from GIF there are a few formats common in the art scene that supports animation. It's not just flip book animations but also palette cycling or simply one blinking color. There are also formats that show the current time (or depend on other variables that can change over time). Ideally an image viewer should update those then appropriate, so they can also be considered animations. (As you might have noticed I'm quite interested in the odd corner cases.) Mutable pixbufs The problem with mutable pixbufs is pretty much the same problem described in the documentation for abydos_get_image_surface(). The user can promise to either not modify pixels or not use the image for anything else after the modification (this is of course the fastest alternative and currently the default). Or the user can request the pixels to be copied or transferred, in which case it's safe to both modify the pixels and use the image for something else. This solution is a bit awkward, but since I think the copy/transfer scenarios are actually corner cases (which I still want to support) the main use case should remain as fast as possible, and also the one not requiring extra typing. Pluggable loaders Pluggable loaders it's something I very much appreciate since I can write my own loaders for the additional formats I care about (ten or so) and use common image viewers to view them. Otherwise I would need to write the whole viewer myself as well. The only alternatives are image viewers on original hardware (emulators may or may not cut it since the image viewers often use hardware tricks that are difficult to emulate and most of them doesn't even try very hard to correct the aspect ratio). Or try to get some strange and glitchy Windows application running in Wine, often with a suboptimal result since it doesn't support all details I care about (such as a correct aspect ratio and palette cycling). Some art groups release their own viewers, that are usually quite good. But they usually only support the subset of features their own paint applications support. One viewer to view them all would be better. But all your favourite viewers to view them all would be the best IMHO. Multi page files Multi page TIFF images and SVG images with multiple icons inside are conceptually the same to abydos. The same API could support both. Limitation of supported image formats It's clear that GTK+ doesn't need to load that many image formats. Most applications doesn't either. And it's clear that more supported formats is a bigger security risk. (But I would like to add that this isn't limited to image loading plugins, in general more software is less secure than less software). For image viewers I think the priorities are different. Most would opt for more more more. (It's not like people would opt for less games on steam just because that would reduce the attack surface, to draw a parallel.) Also, distros would probably package the more exotic loaders in different packages, not installed by default. In case applications shouldn't be allowed to read any kind of image, a filtering could be installed that only let blessed MIME types through. It could easily be added to a wrapper around abydos. Apart from the security aspect I think it's a good thing if an application can open anything you throw at it. In the word processor example you mentioned it's especially true IMHO. Personally I use Lyx, and it gives you the ability to teach it about new formats if you just tell it how to convert it to a format it already knows about. I use that a lot and find it very useful and time saving. But it's a problem if you expect others to be able to open it (which I don't, I export it to PDF instead). So for any kind of format intended for data exchange it should be limited, and clearly defined, what you can embed in it. In the end it seams to boil down to this. You can only pick two: 1) comprehensive image format support. 2) security. 3) speed. I think everybody could be happy if the choice could be made at run-time. Progressive loading I also doubt there are some actual use case that needs progressive loading. A progress bar would probably do in the cases it takes a little time to load an image I think. In abydos that API is always supported, but it's not required for plugins to support it. In case they don't abydos will buffer the bytes, call plugin with all data available, then invoke the callbacks to the user supplied functions. I'm quite sure it wouldn't break anything if gdk-pixbuf did the same. It could be mentioned that some image formats cannot be progressively loaded anyway since crucial data is placed at the very end (or they have no predefined height, just add rows until there is no more data). The feature to request another size than the reported size was mainly added to support vector images at larger scale I think. I'm even quite sure it wasn't intended for taking a short path then a smaller image was requested. Once I suggested that if an image contained an embedded thumbnail, that could be returned if the user requested that size (or smaller), especially then creating thumbnails. I was told no, images should always be loaded at their largest size and then scaled down. By the way. Back 1998 I don't think loading images over a slow internet connection was the only use case for progressive loading. You could definitely see images load progressively even then loading from disk too (and I'm not even talking about floppy disks). Non-raster image formats For non-raster data abydos solves the problem by being agnostic to what an image can be. It's simply anything that can be rendered using cairo. This made it especially easy to make plugins for SVG and PDF since librsvg and poppler has this exact same approach. Meta data Then it comes to meta data abydos is almost useless, it only cares about meta data that affects viewing the image. Trying do do more will either be overly ambitious or half assed. I think there is no middle ground. For general purpose meta data extraction libextractor is most useful (but not perfect). And I don't think this is something GTK+ needs at all. For loading one format and saving to another users might expect the meta data to be preserved to some degree, but to what degree is completely arbitrary and I guess it's impossible to find one that makes everybody happy. I think it's better do to what netpbm does, simply ditch all meta data. But I'm glad you mentioned color space support. That's something I have to think about. MIME sniffing I was pretty sure the custom MIME sniffing was already removed from gdk-pixbuf (according to another answer it seems to be a compile time option). Otherwise it's done twice. Because then I write a loader I need to add the corresponding MIME type to the shared MIME database as well for it to work. In case it has been removed, I think the pattern field (as well as the extension field) in all loaders should be left empty. (I assume the ABI isn't intended to be stable in that direction.) Rust I also like the idea of writing plugins in Rust. My experience with Rust is very limited (and a bit dated). But to me Rust seams to be best suited for large applications. It's quite possible to create very tiny binaries, but as soon as you use one something from a library tons of stuff gets dragged in (as with any language). And it gets very noticeable since everything is statically linked. It seams you have better experience with Rust and I look forward to see what you come up with. The main question, at least for me, is: Should the image loading library that GTK+ uses to load images be a general purpose image loading library also usable for image viewing applications? It seams clear to me that the general opinion points to no. And that the goals of abydos doesn't match the needs for GTK+. The problem with the current situation is that creators of image viewing applications perceive gdk-pixbuf as a general purpose image loading library and uses it for that purpose. One could say that the interest of GTK+ developers are in conflict with the interest of users of image viewing applications. In that case the obvious solution is to clearly state that gdk-pixbuf is not a general purpose image loading library, but only intended for loading icons for GUI elements. (And possibly let the whole thing die in time for GTK+ 5.) > Magnus - thanks for showing us Abydos. The parts of the API that > intersect with gdk-pixbuf's look okay. I think you may find this > useful: > > https://developer.gnome.org/programming-guidelines/stable/index.html.en > > I think your API could improve with some of the material there around > memory management and error reporting. Internally, the code needs > checks in all the malloc()-like functions; some of the arithmetic > (especially in assertions) needs to avoid overflows. I couldn't find > any tests. The SVG plug-in needs to handle error results from > librsvg. The Magick plug-in for PNG and JPEG has no error handling; > also you may want to call cairo_surface_mark_dirty() when you are > done writing pixels to it. The netpbm plugin does a bunch of system > calls and doesn't check for errors. Yes, the code is a bit sketchy right now. I'm still in a phase then I consider throwing away and rewriting large parts. I will definitely add better error handling before 1.0 (and well before that hopefully). I'm mainly focusing on getting the API right at the moment. But I appreciate you point these things out (I'll write them down so I don't forget about them). There are tests (no comprehensive test suit though), but they all rely on non-free images, so I decided to leave that out for now. This is a bit off topic. But are there actually modern (non real time) systems that can recover from a state there malloc() has failed? At least in Linux it will basically only happen if you run out of memory addresses (which will not happen on a 64-bit system at least). And if that happens you're long since screwed anyway. Since the actual allocation will happen then you write to the memory, malloc() will continue to give you memory that doesn't exist. I just assumed any modern non real time system takes this approach. (And if one aims for true real time compatibility it opens up a whole new can of worms.) > The parts of your API that deal with pagination / variants / animation > frames - I'm not sure if a gdk-pixbuf replacement would need them, but > I'm coming from the viewpoint of my braindump above. It's not pagination, as in chopping up an image for printing it on several pages. The concept of pages (in lack of a better term I guess) is about files containing multiple images with an otherwise unspecified relationship. Such as SVG files containing multiple icons. The API is also designed so that everything related to pages, layers, variants and animations can just be ignored, both by applications and plugins, then not needed. (And the extra bytes it adds to the binary is not overwhelming.) Anyway, thank you very much for your valuable feedback! _______________________________________________ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list