Re: [External] : Re: Surface the ImageLoader API
Two thoughts come to mind. 1. While I agree that it isn't necessary to do everything all at once, I don't agree that a full featured Image I/O API is off the table as a future enhancement. Whatever you end up proposing to do here should ideally fit into what we would want to do in the future in this space. At the very least we need to have a discussion about how the subset you propose would or would not fit in. I'm not suggesting that we need to plan for every possible unknown (and in some cases unknowable) future need, but there are some obvious things we might want to do in this space. Writing images and dealing with image meta-data (both of which you listed) are two of the main ones. We need to make sure that we don't preclude any likely future direction. Otherwise we could be left with an API that would need to be deprecated and replaced in order to fit into a future Image I/O-like API. 2. Whether or not the ImageLoader implementation classes might be good API isn't really the point. I rather doubt that defining a lookalike API in the public javafx.scene.image package that merely wraps those classes is what we want. Maybe you'll end up with an API that is somewhat close to that, but starting with an implementation to create the proposed API is backwards. One of the big things to consider is how image readers are registered: SPI? explicit registration API? something else? As for next steps, the first sentence of your latest email is a good start at a brief summary of what you are looking for: > ...a feature to support third-party image format plugins, with the intention of decoding images for use in the JavaFX scene graph. I'd recommend fleshing this out a bit, possibly with some mocked up API (high-level). If you haven't already looked at it, the JEP template [1] is good for describing non-trivial features such as what you propose. While we don't generally do JEPs for JavaFX APIs any more, the template can be a useful way to organize the information needed to evaluate the proposal. It gets you think about thinks like motivation, goals / non-goals, API, alternatives, testing, etc. If there is general buy-in that a pluggable image loader is worth doing, the main things to hash out before this can proceed are the two I mentioned above: 1) whether and how it would fit into a larger Image reader/writer API; and 2) what form the API should take. -- Kevin [1] https://openjdk.java.net/jeps/2 On 6/4/2021 3:10 PM, Michael Strauß wrote: Let me restate my intentions to be more clear: What I'm asking for is a feature to support third-party image format plugins, with the intention of decoding images for use in the JavaFX scene graph. I think the following things should remain explicit non-goals: - writing images - transcoding images - reading or writing metadata There's Java2D for that, no need to duplicate existing functionality. For its intended purpose, I do think that the existing internal implementation is actually a good API, because it is geared towards the problem that it is trying to solve. Expanding the scope of the existing feature to include any of these additional things doesn't seem like a sensible idea to me. I would like to understand whether you disagree with my assessment of the intended purpose of this feature. Do you think that there is a "larger picture" that would change the goals and non-goals? You seem to imply that extending the existing feature is the wrong approach, and that there is a better approach which requires more effort. Can you share any of insights that lead to this conclusion? I haven't been able to find this information in JBS. Am Fr., 4. Juni 2021 um 22:03 Uhr schrieb Kevin Rushforth : Characterizing a new feature in terms of exposing internal interfaces isn't really the right way to think about it. The fact that there might be internal classes that do something similar to what you propose isn't really relevant (other than later down the road when it comes time to discuss the implementation). Rather you should start with a description of the enhancement you would like to see: additional image loading capability, in this case. Then a discussion of the API would be in order. What's really being asked for here is a feature like Java2D's Image I/O with pluggable image formats. We spent some time looking at that a few years ago. Doing this right would be a very large effort. I wouldn't be in favor of a piecemeal approach without understanding the larger picture. So I don't support this proposal as stated. -- Kevin
Re: Surface the ImageLoader API
Let me restate my intentions to be more clear: What I'm asking for is a feature to support third-party image format plugins, with the intention of decoding images for use in the JavaFX scene graph. I think the following things should remain explicit non-goals: - writing images - transcoding images - reading or writing metadata There's Java2D for that, no need to duplicate existing functionality. For its intended purpose, I do think that the existing internal implementation is actually a good API, because it is geared towards the problem that it is trying to solve. Expanding the scope of the existing feature to include any of these additional things doesn't seem like a sensible idea to me. I would like to understand whether you disagree with my assessment of the intended purpose of this feature. Do you think that there is a "larger picture" that would change the goals and non-goals? You seem to imply that extending the existing feature is the wrong approach, and that there is a better approach which requires more effort. Can you share any of insights that lead to this conclusion? I haven't been able to find this information in JBS. Am Fr., 4. Juni 2021 um 22:03 Uhr schrieb Kevin Rushforth : > > Characterizing a new feature in terms of exposing internal interfaces > isn't really the right way to think about it. The fact that there might > be internal classes that do something similar to what you propose isn't > really relevant (other than later down the road when it comes time to > discuss the implementation). > > Rather you should start with a description of the enhancement you would > like to see: additional image loading capability, in this case. Then a > discussion of the API would be in order. > > What's really being asked for here is a feature like Java2D's Image I/O > with pluggable image formats. We spent some time looking at that a few > years ago. Doing this right would be a very large effort. I wouldn't be > in favor of a piecemeal approach without understanding the larger picture. > > So I don't support this proposal as stated. > > -- Kevin
Re: Surface the ImageLoader API
Characterizing a new feature in terms of exposing internal interfaces
isn't really the right way to think about it. The fact that there might
be internal classes that do something similar to what you propose isn't
really relevant (other than later down the road when it comes time to
discuss the implementation).
Rather you should start with a description of the enhancement you would
like to see: additional image loading capability, in this case. Then a
discussion of the API would be in order.
What's really being asked for here is a feature like Java2D's Image I/O
with pluggable image formats. We spent some time looking at that a few
years ago. Doing this right would be a very large effort. I wouldn't be
in favor of a piecemeal approach without understanding the larger picture.
So I don't support this proposal as stated.
-- Kevin
On 6/4/2021 9:39 AM, Michael Strauß wrote:
I've been trying to add SVG support to a JavaFX application (which is
the image format I'm encountering most often in app projects), but
that's quite hard because the ImageLoader API is pretty much closed
for extension.
I'm proposing to open up the ImageLoader API to enable third-party
developers to create image loaders for emerging image formats like SVG
or WebP.
There will need to be some minor API and behavioral changes:
1. `ImageLoader.load` requires a new parameter "pixelScale":
current: ImageFrame load(int, int width, int height, boolean, boolean)
new: ImageFrame load(int, int width, int height, double pixelScale,
boolean, boolean)
This parameter is important for vector image loaders like SVG. If an
image loader is invoked with `width==0` and `height==0`, the
implementation should return an image with its natural resolution.
Vector image loaders need to know the requested pixel scale in order
to render the image with the correct pixel density.
2. `ImageFormatDescription.Signature` must be able to accept `null`
values in the byte array:
current: public Signature(byte...) {...}
new: public Signature(Byte...) {...}
A `null` value will match any byte at the corresponding position in
the image file header. This is necessary to match file headers that
contain variable data, like the WebP image header:
byte 0-3: 'R', 'I', 'F', 'F'
byte 4-7:
byte 8-11: 'W', 'E', 'B', 'P'
Note that in this example, the corresponding signature description would be:
{ 'R', 'I', 'F', 'F', null, null, null, null, 'W', 'E', 'B', 'P' }
3. Enable file extension matching (in addition to signature matching)
with the following semantics:
Any valid `ImageFormatDescription` MUST contain at least one file
extension, and MAY contain any number of signatures and MIME types (or
none at all). When `ImageStorage` chooses an image loader for a file,
it first tries to match the file by signature, and if it doesn't
match, it tries to match by file extension (or, if the source is a
"data" URI, it tries to match by MIME type).
The reason for this is that some image formats like SVG cannot be
meaningfully matched by signature; in this case, the SVG image format
description would not contain a signature description and would always
fall back to matching by file extension or MIME type.
I'd welcome any opinions on this proposal.
Re: Surface the ImageLoader API
I should note that these proposed changes were sufficient for me to implement an SVG image loader and a WebP image loader using third-party libraries.
Surface the ImageLoader API
I've been trying to add SVG support to a JavaFX application (which is
the image format I'm encountering most often in app projects), but
that's quite hard because the ImageLoader API is pretty much closed
for extension.
I'm proposing to open up the ImageLoader API to enable third-party
developers to create image loaders for emerging image formats like SVG
or WebP.
There will need to be some minor API and behavioral changes:
1. `ImageLoader.load` requires a new parameter "pixelScale":
current: ImageFrame load(int, int width, int height, boolean, boolean)
new: ImageFrame load(int, int width, int height, double pixelScale,
boolean, boolean)
This parameter is important for vector image loaders like SVG. If an
image loader is invoked with `width==0` and `height==0`, the
implementation should return an image with its natural resolution.
Vector image loaders need to know the requested pixel scale in order
to render the image with the correct pixel density.
2. `ImageFormatDescription.Signature` must be able to accept `null`
values in the byte array:
current: public Signature(byte...) {...}
new: public Signature(Byte...) {...}
A `null` value will match any byte at the corresponding position in
the image file header. This is necessary to match file headers that
contain variable data, like the WebP image header:
byte 0-3: 'R', 'I', 'F', 'F'
byte 4-7:
byte 8-11: 'W', 'E', 'B', 'P'
Note that in this example, the corresponding signature description would be:
{ 'R', 'I', 'F', 'F', null, null, null, null, 'W', 'E', 'B', 'P' }
3. Enable file extension matching (in addition to signature matching)
with the following semantics:
Any valid `ImageFormatDescription` MUST contain at least one file
extension, and MAY contain any number of signatures and MIME types (or
none at all). When `ImageStorage` chooses an image loader for a file,
it first tries to match the file by signature, and if it doesn't
match, it tries to match by file extension (or, if the source is a
"data" URI, it tries to match by MIME type).
The reason for this is that some image formats like SVG cannot be
meaningfully matched by signature; in this case, the SVG image format
description would not contain a signature description and would always
fall back to matching by file extension or MIME type.
I'd welcome any opinions on this proposal.
