That sounds useful, but is your example code complete? Don't you need to declare the "image" variable with the
appropriate generic type?
Also, do we get a lot of compiler warnings if you want to ignore the generics in other code? What's the pain point if
you don't care what the types of the images are? (I suppose the most common case is that people have an "Image" without
even realizing that it is an MRI in the first place so this is only for those who want to manipulate MRIs)...
...jim
On 3/29/17 1:22 PM, Sergey Bylokhov wrote:
Hi, Jim, Phil.
I have started to use MRI and a new Robot in some of the tests and wonder why
the MultiResolutionImage does not use
generics for getResolutionVariants()?
So for example if we have an API like:
http://download.java.net/java/jdk9/docs/api/java/awt/Robot.html#createMultiResolutionScreenCapture-java.awt.Rectangle-
We cannot specify that the MRI contains BufferedImage(not just an image). So
the users will need to check it via
instance of and cast.
So in the worse case the user will get something like this:
=====
Test.java
MultiResolutionImage image =
robot.createMultiResolutionScreenCapture(rect);
List<Image> resolutionVariants = image.getResolutionVariants();
if (resolutionVariants.size() > 1) {
Image tmp = resolutionVariants.get(1);
if (tmp instanceof BufferedImage) {
capture = (BufferedImage) tmp;
}
} else {
Image tmp = resolutionVariants.get(0);
if (tmp instanceof BufferedImage) {
capture = (BufferedImage) tmp;
}
}
=====
I am not sure but for writing the tests, I feel something like this will be
simpler:
=====
Robot.java
public synchronized MultiResolutionImage<BufferedImage>
createMultiResolutionScreenCapture(Rectangle screenRect) {
//
Test.java
MultiResolutionImage image =
robot.createMultiResolutionScreenCapture(rect);
List<BufferedImage> resolutionVariants =
image.getResolutionVariants();
if (resolutionVariants.size() > 1) {
capture = resolutionVariants.get(1);
} else {
capture = resolutionVariants.get(0);
}
=====
Or even this:
capture = robot.createCompatibleScreenCapture(rect);
Hi Alexandr,
Sorry that this fell into the cracks. Here are some fairly minor updates:
AbstractMRI - getGraphics() should include "getGraphics() not supported on
Multi-Resolution Images" as the exception
description text.
AbstractMRI - getBaseImage() should have doc comments:
/**
* Return the base image representing the best version of the image
* for rendering at the default width and height.
* @return the base image of the set of multi-resolution images
*/
(Does it need an @since given that the entire interface is @since 1.9?)
BaseMRI - the doc comments (both the class comment and the constructor comments) don't
start with "/**" so they aren't
really doc comments, but they look good so make them so.
BaseMRI - class comment - "The implementation will return the first ... satisfy the
rendering request." Add another
sentence right there "The last image in the array will be returned if no
suitable image is found that is larger than
the rendering request."
BaseMRI - move "For best effect ..." should be moved to a new paragraph and
mention that no exception will be thrown
if the images are not sorted as suggested.
RenderingHints - comments:
DEFAULT, SIZE_FIT, DPI_FIT - "an image resolution variant is chosen ..." (add
"an")
DEFAULT - "... which may depend on the policies of the platform"
SIZE_FIT, DPI_FIT "... on the DPI of ..." (add "the")
SunHints - descriptor texts:
SIZE_FIT, DPI_FIT "based on the DPI" (add "the")
MRCachedImage - remind me what this is used for?
MRCachedImage.getResolutionVariant - use ceil or round instead of (int) cast?
ceil() would match the actions of
MRToolkitImage better. Also note following comment about precision with SG2D.
SG2D/MRI - the interface declares the values as float, the usage in SG2D
computes values in double and then truncates
them to int to pass to the interface - should we upgrade the interface to
double for completeness? If not, then the
usage in SG2D should at least pass in float precision.
SG2D.getResolutionVariant - using destRegionWH to calculate destImageWH can involve
a lot of "do some math and then
later have additional code undo it". Would it be faster to just compute
destImageWH directly, as in:
float destImageWidth, destImageHeight;
if (BASE) {
destImageWH = srcWH;
} else if (DPI) {
destImageWH = (float) abs(srcWH * devScale);
} else {
double destRegionWidth, destRegionHeight;
if (type) {
...
}
destImageWH = (float) abs(srcWH * destRegionWH / swh);
}
Image rv = img.getRV(destImageWH);
On the other hand, the last "else" is probably the most common case so this
doesn't save anything in the common case,
but it avoids a bunch of math that could introduce rounding error for the
BASE/DPI cases (even though it is done in
doubles).
Is there intent to have the default case be mapped to DPI_FIT for Windows?
MultiResolutionRenderinHints.java - should have "Test" appended to the name
MRRHTest - why not render to a BufferedImage and avoid Robot? Could it tap
into internal APIs to create a
BImg/compatibleImage with a given devScale?
MRRHTest - why allow up to delta=50 on the comparison?
MRRHTest - since the scale factor comes from a dialog, we depend on running
this on a HiDPI display to fully test the
hints? Could using "scaled compatible images" allow testing this more
definitively?
MRRHTest - remove println before pushing?
MRRHTest - probably shouldn't have a "right answer" for DEFAULT - it should
probably just pass if the operation
doesn't throw an exception?
...jim
On 4/15/15 7:04 AM, Alexander Scherbatiy wrote:
Hello,
Could you review the updated fix:
http://cr.openjdk.java.net/~alexsch/8029339/webrev.08/
- SunGraphics2D is updated to calculate the resolution variant size
according to the _BASE, _DPI_FIT, and _SIZE_FIT resolution rendering hint
- MultiResolutionRenderingHints test is added
Thanks,
Alexandr.
On 4/7/2015 1:02 PM, Jim Graham wrote:
This is an interesting suggestion that would let us keep the
implementation of the various hints centralized and simplify the
interface to the part of the mechanism that is most tied in to the
implementation specifics of the database of media variants - which is
housed in the MRI-implementing object.
I'm not sure we ever identified any need to customize the logic of
"what size is needed for this render operation" beyond what we
expressed in the hints, and given that the platform defaults may not
be easy to express to an arbitrary implementation, it is probably
better to put that part of the logic in SG2D, controlled by the easy
to express hints and keep the current API both simple to implement and
flexible to use. Even if there was a need to customize that part of
the operation (the "what size is relevant to this rendering operation"
decision) beyond what the hints suggest, it would be inappropriate to
tie that in to the "find me media" aspect of the MRI interface
anyway. So, best to keep them separate and have the hints affect what
SG2D does and have the MRI focused on just storing (possibly creating)
and managing/finding the variants.
...jim
On 4/1/15 6:46 AM, Alexander Scherbatiy wrote:
On 3/27/2015 12:48 AM, Jim Graham wrote:
Hi Alexander,
http://cr.openjdk.java.net/~alexsch/8029339/webrev.07/
I have updated the fix according to the comments except
RenderingHints.
RenderingHints are supposed to be set on Graphics2D and they have
keys/values which are not relevant to the getResolutionVariant() method.
Graphics objects chooses an alternative according to the set
rendering hints.
May be what SG2D should do just calculate dest image size for the
given resolution variant hint (_BASE - base image size, _SIZE_FIT -
including scale factor and graphics transform (Mac algorithm),
_DPI_FIT - scaled according to the system DPI) and then pass them to
the getResolutionVariant() method?
Thanks,
Alexandr.
MRI.java:
Who is the intended audience for the recommendation in the interface
to cache image variants? Are we asking the developers who call the
methods on the interface to cache the answers? That would be unwise
because the list of variants may change over time for some MRIs. Are
we speaking to the implementer? If so, then I think that it is
unnecessary to remind implementers of basic implementation
strategies like "cache complicated objects".
How about this wording for the getRV() method? - "Gets a specific
image that is the best variant to represent this logical image at
the indicated size and screen resolution."
Perhaps we should clarify in the doc comments for the dimension
arguments in getRV() that the dimensions are measured in pixels?
line 56 - delete blank line between doc comment and method declaration
Also, it is probably overkill to document that the interface
getVariants() method returns an unmodifiable list. Responsible
implementations would probably use an unmodifiable list, but I don't
think it should be required by the interface. We do need to specify
that it isn't required to be modifiable so that a caller doesn't
expect to be able to modify it, but that is a looser spec. How
about "Gets a readable list of all resolution variants. Note that
many implementations might return an unmodifiable list."?
AbstractMIR.java:
"provides a default implementation for the MRI" or "provides default
implementations of several methods in the <MRI> interface and the
<Image> class"? Actually, I'll note that it provides none of the
implementations of the MRI methods so maybe it should be "provides
default implementations of several <Image> methods for classes that
want to implement the <MRI> interface"?
In the doc example - wrap the list to make it unmodifiable
The doc example could be made 2 or 3 lines shorter by simply
assuming the base image is in index 0 (it's just an example so I'm
not sure the flexibility needs to be a part of the example).
getGraphics is allowed by the Image interface to return null.
Actually, the exact wording is that it can only be called for
"offscreen images" and a MRI is technically not "an offscreen
image". Perhaps we should return null here since modifying the base
image is unlikely to modify the variants and arguably it would be
required by the wording in the method docs (even if the base image
was an offscreen, the MRI produced from it stops being an offscreen)?
Are all of the empty "@Inherit" comments needed? I thought
inheritance was the default?
BaseMRI.java:
"This class is [an] array-based implementation of the {AMRI} class"
(missing "an" and "the")
We should probably include the comment about the sorting of the
images in the class comments as well as document that the algorithm
is a simple scan from the beginning for the first image large enough
(and default == last image). The algorithm also might not work very
well if the images are not monotonically both wider and taller. How
about adding this to the class comments:
"The implementation will return the first image variant in the array
that is large enough to satisfy the rendering request. For best
effect the array of images should be sorted with each image being
both wider and taller than the previous image. The base image need
not be the first image in the array."
getVariants() - you need to return an unmodifiable list. asList()
returns a list that "writes back" to the array. You need to use
Collections.unmodifiableList(Arrays.asList(array)).
RenderingHints.java:
Where do we process this hint? Don't we need to pass it to the
getVariant() method?
I don't understand the hint values other than the one that always
uses the base image. Default I guess gives us the ability to use
the Mac algorithm of "next larger size" on Mac and "based on Screen
DPI" on Windows, but the 3rd value "ON" is so vague as to not have
any meaning. Perhaps we should just delete it, but then do we just
always do the Mac method? Or do we vaguely have our internal images
have a platform-specific method and the Abstract/Base classes just
do what they want with no control from the user? In FX we are also
still struggling with this issue and we may likely just do what the
Mac does in all cases, but perhaps AWT needs to "behave like the
platform" more? If we are going to have actual values, then we need
to have them do something, which means:
- SG2D needs to track the hint just like we do the other hints that
affect our processing
- MRI.getVariant() needs to have the hint as an argument
- BaseMRI should probably do something with that hint
- hint values should include:
- ..._DEFAULT - implementation gets to decide
- ..._BASE - always use the base image
- ..._SIZE_FIT - Mac algorithm of smallest image that is big enough
- ..._DPI_FIT - choose based on DPI of the screen, ignoring
transform
line 978 - missing blank line between fields
SG2D.java:
- The interface says that you will be passing in the "logical DPI"
of the display, but here you are actually passing in the screen's
scale factor.
BaseMRITest.java:
- testBaseMRI also passes in a scale rather than a DPI to the MRI
method.
- How does the modification test pass when the implementation
doesn't use unmodifiable lists?
MRITest.java:
- also uses scale rather than DPI in several places
...jim
On 3/13/15 6:34 AM, Alexander Scherbatiy wrote:
Hello,
Could you review the proposed API based on MultiresolutionImage
interface:
http://cr.openjdk.java.net/~alexsch/8029339/webrev.06/
- return unmodifiable list comment is added to the
getResolutionVariants() method javadoc in MultiresolutionImage
interface
- base image size arguments are removed form the
getResolutionVariant(...) method in MultiresolutionImage interface
- BaseMultiResolutionImage class that allows to create a
multi-resolution image based on resolution variant array is added
- the test for the BaseMultiResolutionImage is added
Thanks,
Alexandr.
On 2/14/2015 3:23 AM, Jim Graham wrote:
The second solution looks good. I'd make it standard practice
(perhaps even mentioned in the documentation) to return unmodifiable
lists from the getVariants() method. The Collections class provides
easy methods to create these lists, and it sends a clear message to
the caller that the list was provided for them to read, but not write
to. Otherwise they may add a new image to the list you provided them
and wonder why it wasn't showing up. Also, an unmodifiable list can
be cached and reused for subsequent calls without having to create a
new list every time.
In getResolutionVariant() was there a reason why the base dimensions
were provided as float? The destination dimensions make sense as
float since they could be the result of a scale, but the source
dimensions are typically getWidth/getHeight() on the base image. A
related question would be if they are needed at all, since the
implementation should probably already be aware of what the base
image
is and what its dimensions are. I'm guessing they are provided
because the implementation in SG2D already knows them and it makes it
easier to forward the implementation on to a shared (static?) method?
With respect to default implementations, I take it that the
BaseMRI is
along the pattern that we see in Swing for Base classes. Would it be
helpful to provide an implementation (in addition or instead) that
allows a developer to take a bunch of images and quickly make an MRI
without having to override anything? The implementations of
getBaseImage() and getResolutionVariants() are pretty straightforward
and a fairly reasonable default algorithm can be provided for
getRV(dimensions). This question is more of an idle question for my
own curiosity than a stumbling block...
...jim
On 1/22/2015 6:49 AM, Alexander Scherbatiy wrote:
Hi Phil,
I have prepared two versions of the proposed API:
I) Resolution variants are added directly to the Image:
http://cr.openjdk.java.net/~alexsch/8029339/list/webrev.00
II) MultiResolutionImage interface is used:
http://cr.openjdk.java.net/~alexsch/8029339/webrev.05
It could help to decide which way should be chosen for the the
multi-resolution image support.
Below are some comments:
1. High level goal:
Introduce an API that allows to create and handle an image
with
resolution variants.
2. What is not subject of the provided API
- Scale naming convention for high-resolution images
- Providing pixel scale factor for the screen/window
3. Use cases
3.1 Loading and drawing high-resolution icons in IntelliJ IDEA
A high-resolution image is loaded from resources and stored in
JBHiDPIScaledImage class which is a subclass of the buffered image.
The high-resolution image is used to create a disabled icon
in the
IconLoader.getDisabledIcon(icon) method.
https://github.com/JetBrains/intellij-community/blob/master/platform/util/src/com/intellij/openapi/util/IconLoader.java
3.2 Loading and drawing high-resolution icons in NetBeans
NetBeans does not have support for the high-resolution icons
loading.
It loads an icon from the file system using
Toolkit.getDefaultToolkit().getImage(url) method or from resources
by ImageReader and store it in ToolTipImage class which is
subclass of the buffered image.
ImageUtilities.createDisabledIcon(icon) method creates a
disabled
icon by applying RGBImageFilter to the icon.
http://hg.netbeans.org/main/file/97dcf49eb4a7/openide.util/src/org/openide/util/ImageUtilities.java
3.3 Loading system icons in JDK 1.8
JDK requests icons from the native system for system L&Fs and
applies filters for them.
See for example AquaUtils.generateLightenedImage() method:
http://hg.openjdk.java.net/jdk9/client/jdk/file/e6f48c4fad38/src/java.desktop/macosx/classes/com/apple/laf/AquaUtils.java
4. HiDPI support for Images on different OSes
4.1 Mac OS X
Cocoa API contains NSImage that allows to work with image
representations: add/remove/get all representations.
It picks up an image with necessary resolution based on the
screen backing store pixel scale factor and applied transforms.
https://developer.apple.com/library/mac/documentation/Cocoa/Reference/ApplicationKit/Classes/NSImage_Class/Reference/Reference.html
4.2 Linux
GTK+ 3 API has gtkcssimagescaled lib (it seems that it is not
public/stable)
that parses the -gtk-scaled css property and draws a
GtkCssImage
according to the given scale factor.
I have not found information about the HiDPI support in Xlib.
4.3 Windows
I have only found the tutorial that suggests to select and
draw a
bitmap using the queried DPI
and scale the coordinates for drawing a rectangular frame
http://msdn.microsoft.com/en-us/library/dd464659%28v=vs.85%29.aspx
Windows also provides the horizontal and vertical DPI of the
desktop
http://msdn.microsoft.com/en-us/library/windows/apps/dd371316
5. Pseudo API
Below are some ways which illustrates how multi-resolution
images
can be created and used.
5.1 Resolution variants are stored directly in Image class.
To query a resolution variant it needs to compare the
resolution
variant width/height
with the requested high-resolution size.
------------
public abstract class Image {
public void addResolutionVariant(Image image) {...}
public List<Image> getResolutionVariants() {...}
}
------------
// create a disabled image with resolution variants
Image disabledImage = getDisabledImage(image);
for (Image rv : image.getResolutionVariants()) {
disabledImage.addResolutionVariant(getDisabledImage(rv));
}
------------
This approach requires that all resolution variants have been
created even not of them are really used.
5.2 Resolution variants are stored in a separate object that
allows to create them by demand.
To query a resolution variant it needs to compare the
resolution
variant scale factor
with the requested scale (that can include both screen DPI
scale
and applied transforms).
------------
public abstract class Image {
public static interface ResolutionVariant {
Image getImage();
float getScaleFactor();
}
public void addResolutionVariant(ResolutionVariant
resolutionVariant) {...}
public List<ResolutionVariant> getResolutionVariants()
{...}
}
------------
// create a disabled image with resolution variants
Image disabledImage = getDisabledImage(image);
for (Image.ResolutionVariant rv :
image.getResolutionVariants()) {
disabledImage.addResolutionVariant(new
Image.ResolutionVariant() {
public Image getImage() {
return getDisabledImage(rv.getImage());
}
public float getScaleFactor() {
return rv.getScaleFactor();
}
});
}
------------
It does not have problem if a predefined set of images is
provided
(like image.png and im...@2x.png on the file system).
This does not cover cases where a resolution variant can be
created
using the exact requested size (like loading icons from the native
system).
A resolution variant can be queried based on a scale factor and
applied transforms.
5.3 The provided example allows to create a resolution variant
using the requested high-resolution image size.
------------
public interface MultiResolutionImage {
Image getResolutionVariant(float width, float height);
}
------------
// create a multi-resolution image
Image mrImage = new AbstractMultiResolutionImage() {
public Image getResolutionVariant(float width, float
height) {
// create and return a resolution variant with
exact
requested width/height size
}
protected Image getBaseImage() {
return baseImage;
}
};
------------
// create a disabled image with resolution variants
Image disabledImage = null;
if (image instanceof MultiResolutionImage) {
final MultiResolutionImage mrImage = (MultiResolutionImage)
image;
disabledImage = new AbstractMultiResolutionImage(){
public Image getResolutionVariant(float width, float
height) {
return
getDisabledImage(mrImage.getResolutionVariant(width, height));
}
protected Image getBaseImage() {
return getDisabledImage(mrImage);
}
};
} else {
disabledImage = getDisabledImage(image);
}
------------
Thanks,
Alexandr.
