Hi Jim,

 Let's divide the discussion into two part.

 1. Where it is better to hold resolution variants?
    Putting resolution variants in Image class brings some questions like:
  - Some type of images do not need to have resolution variants
  - Should resolution variants have the same type as the base image?
- getResolutionVariants() method can return copy of the original list so add/removeRV methods should be also added.

There are pros and cons for placing resolution variants to Image class or to a separate intreface.

2. Using scale factor/image sizes/scaled image sizes to retreive a resolution variant.

May be it is better to have a structure that provide all necessary information to query the resolution variant: scale factor, draw area width/height, transformed area width/height?

  For example:
  ---------------------
    public interface MultiResolutionImage {

        interface DrawAreaInfo {

            float getScaleFactor();
            float getAreaWidth();
            float getAreaHeight();
            float getTransformedAreaWidth();
            float getTransformedAreaHeight();
        }

        public Image getResolutionVariant(DrawAreaInfo drawAreaInfo) ;
        public List<Image> getResolutionVariants();
    }
  ---------------------

The MultiResolutionImage default implementation could allow to use different strategies like scale factor/transfom/OS based to query a resolution variant. The OS based strategy can be used by default.

Thanks,
Alexandr.


On 2/13/2014 4:42 AM, Jim Graham wrote:
On 2/12/14 5:59 AM, Alexander Scherbatiy wrote:
On 2/8/2014 4:19 AM, Jim Graham wrote:
The primary thing that I was concerned about was the presence of
integers in the API when Windows uses non-integer multiples
      It would make sense to pass real numbers to the
getResolutionVariant() method  if the difference between resolution
variants sizes is 1.
      It seems that it is not a common case.

I was thinking of other API that is related to this, such as the API that queries the scaling factor from a SurfaceManager. I seem to remember some integer return values in that, but Windows might have the answer 1.4 or 1.8, depending on the screen scaling factor that was determined from the UI.

In terms of the getResolutionVariant() method here, those non-integer screen scaling factors don't directly impact this API. But, we have some issues with the use of integers there from other sources:

- That API assumes that the caller will determine the pixel size needed, but the actual media choice is determined with different techniques on Mac and Windows so this means that the caller will have to worry about platform conventions. Is that the right tradeoff?

- The technique recommended for Mac involves computing the precise size desired using the current transform, which may be a floating point value, so the integer values used in this API are already approximations and there is no documentation on how to generate the proper integer. In particular, the current code in SG2D naively uses a cast to integer to determine the values to supply which means a transformed size of W+0.5 will be truncated to W and the lower resolution image will be used. Does that conform to Mac guidelines? Do they require the truncated size to reach W+1 before the next size is used? Passing in float or double values would sidestep all of that since then the comparisons would be done with full precision, but as long as we can determine a "best practices compatible with all platforms" rule on how to round to integers, then integers are OK there.

- The Windows document you cite below suggests that the determination should be made by looking at the Screen DPI and choosing the next higher media variant based on that screen DPI. They do not specify choosing media based on the current transform as is done for Mac. If we stick with supplying values that are used to determine which media to use, then on Windows we should not take the transform into account, but instead query the SurfaceManager for the scale factor and only transform by those values (even if the current transform was manually overridden to identity).

There are pros and cons to both approaches.

Mac ensures that you are always using the best media for any given render operation.

But, Windows ensure more consistency in the face of other scaling.

The thing to consider is that if you have a 500x500 image with a 1000x1000 variant and you rendering it at 500x500 and then 501x501, that first jump will be fairly jarring as the scaled version of the 1000x1000 will not look precisely like the original 500x500 did. With @2x images only, this effect is minimized so the advantage of always using "the best media for a given render operation" may outweigh the inconsistency issue. But, on Windows where the media are 1.4x or 1.8x in size, a downscaled image will start to show more interpolation noise and so the balance of the two choices may shift more towards not wanting a jarring shift.

We might want one or more of the following:

- Developer chooses policy (TX_AWARE, DPI_AWARE, ALWAYS_LARGEST, NONE, PLATFORM) where the last policy would use TX_AWARE on Mac and DPI_AWARE on Windows

- We create our own policy and always use it (TX_AWARE?  or DPI_AWARE?)

- We create our own policy that dynamically chooses one of the above strategies depending on platform or available media or ???

- We could create an optional interface for them to install their own algorithm as well. I think it would work best as a delegate interface that one installs into Image so that it can be used with any image without having to subclass (it wouldn't really have much to do for BufferedImages or VolatileImages, though):

class Image {
    void setResolutionHelper(ImageResolutionHelper foo);
    List<Image> getResolutionVariants();
}

or:

class Graphics {
     void setResolutionHelper(ImageResolutionHelper foo);
}

or - anywhere else it could be installed more centrally (per App context)?

and the interface would be something like one of these variants:

interface ImageResolutionHelper {
    // This version would prevent substituting a random image:
// They have to return an index into the List<Image> for that image...
    public int chooseVariant(Image img, double dpi, number w, number h);

or:

    // This version would allow substituting an arbitrary image:
    public Image getVariant(Image img, double dpi, number w, number h);
}

Since they would be in full control of the policy, though, we would unfortunately always have to call this, there would be no more testing in SG2D to see "if" we need to deal with DPI, though perhaps we could document some internal conditions in which we do not call it for common cases (but that would have to be well agreed not to get in the way of reasonable uses of the API and well documented)?

Note that we would have to do a security audit to make sure that random image substitution couldn't allow any sort of "screen phishing" exploit.

            ...jim

and also what policy they use for choosing scaled images.

I don't see a mention of taking the current transform into account,
just physical issues like screen DPI and form factor. They talk about
resolution plateaus and in their recommendations section they tell the
developer to use a particular property that tells them the screen
resolution to figure out which image to load if they are loading
manually.  There is no discussion about dynamically loading multiple
versions of the image based on a dynamic program-applied transform
factor as is done on MacOS.

Also, they tell developers to draw images to a specific size rather
than using auto-sizing.  That begs the question as to how they
interpret a call to draw an image just using a location in the
presence of various DPI factors.
      There is an interesting doc that describes how to write DPI-aware
Win32 applications:
http://msdn.microsoft.com/en-us/library/dd464646%28v=vs.85%29.aspx
      It is suggested to handle WM_DPICHANGED message,  load the graphic
that has slightly greater resolution to the current DPI and use StretchBlt
      to scale down the image.

     Thanks,
     Alexandr.


            ...jim

On 2/7/14 3:00 AM, Alexander Scherbatiy wrote:
On 1/22/2014 6:40 AM, Jim Graham wrote:
Hi Alexander,

Before we get too far down the road on this API, I think we understand
the way in which MacOS processes multi-resolution images for HiDPI
screens, but have we investigated the processes that Windows uses
under Windows 8?  My impression is that Windows 8 has included a
number of new techniques for dealing with the high resolution displays
that it will run on in the Windows tablet and mobile industries and
that these will also come into play as 4K displays (already available)
become more common on the desktop.  We should make sure that what we
come up with here can provide native compatibility with either
platform's policies and standard practices.

If you've investigated the MS policies I'd like to see a summary so
that we can consider them as we review this API...
    There is the Windows Guidelines for scaling to pixel density:
http://msdn.microsoft.com/en-us/library/windows/apps/hh465362.aspx
which says that Windows has automatic resource loading that supports
three version of images scaling (100%, 140%, and 180%)
   --------------------------------
Without scaling, as the pixel density of a display device increases, the
physical sizes of objects on screen get smaller.
When UI would otherwise be too small to touch and when text gets too
small to read,
Windows scales the system and app UI to one of the following scaling
plateaus:

     1.0 (100%, no scaling is applied)
     1.4 (140% scaling)
     1.8 (180% scaling)

Windows determines which scaling plateau to use based on the physical
screen size, the screen resolution, the DPI of the screen, and form
factor.

Use resource loading for bitmap images in the app package For bitmap
images stored
in the app package, provide a separate image for each scaling
factor(100%, 140%, and 180%),
and name your image files using the "scale" naming convention described
below.
Windows loads the right image for the current scale automatically.
   --------------------------------

  The image name convention for the various scales is:
    images/logo.scale-100.png
    images/logo.scale-140.png
    images/logo.scale-180.png

The 'ms-appx:///images/logo.png' uri is used to load the image in an
application.

If we want to support this in the same way as it is done for Mac OS X
   the WToolkit should return MultiResolution image in case if the
loaded image has .scale-* qualifiers.
   The Graphics class can request an image with necessary resolution
from the MultiResolution image.

   It seems that nothing should be changed in the MultiResolution
interface in this case.

    Thanks,
    Alexandr.

            ...jim

On 1/14/14 2:54 AM, Alexander Scherbatiy wrote:

Hello,

Could you review the fix:
   bug: https://bugs.openjdk.java.net/browse/JDK-8029339
   webrev: http://cr.openjdk.java.net/~alexsch/8029339/webrev.00

   This is a proposal to introduce an API that allows to create a
custom
multi resolution image.

I. It seems reasonable that the API should provide two basic
operations:

1. Get the resolution variant based on the requested image width and
height:
     - Image getResolutionVariant(int width, int height)

Usually the system provides the scale factor which represents the
number of pixels corresponding to each linear unit on the display.
However, it has sense to combine the scale factor and the current
transformations to get the actual image size to be displayed.

  2. Get all provided resolution variants:
    - List<Image> getResolutionVariants()

   There are several uses cases:
    - Create a new multi-resolution image based on the given
multi-resolution image.
    - Pass to the native system the multi-resolution image. For
example,
a use can set to the system the custom multi-resolution cursor.

II. There are some possible ways where the new API can be added

  1. java.awt.Image.

   The 2 new methods can be added to the Image class. A user can
override
   the getResolutionVariant() and getResolutionVariants() methods to
provide the resolution variants
or there can be default implementations of these methods if a user
puts resolution variants
   to the list in the sorted order.

   To check that the image has resolution variants the following
statement can be used: image.getResolutionVariants().size() != 1

The disadvantage is that there is an overhead that the Image class
should contain the List object and not all
   images can have resolution variants.

  2. Introduce new MultiResolutionImage interface.

   A user should extend Image class and implement the
MultiResolutionImage interface.

   For example:
   ---------------------
     public class CustomMultiResolutionImage extends BufferedImage
             implements MultiResolutionImage {

         Image highResolutionImage;

         public CustomMultiResolutionImage(BufferedImage baseImage,
                 BufferedImage highResolutionImage) {
             super(baseImage.getWidth(), baseImage.getHeight(),
baseImage.getType());
             this.highResolutionImage = highResolutionImage;
             Graphics g = getGraphics();
             g.drawImage(baseImage, 0, 0, null);
             g.dispose();
         }

         @Override
         public Image getResolutionVariant(int width, int height) {
             return ((width <= getWidth() && height <= getHeight()))
                     ? this : highResolutionImage;
         }

         @Override
         public List<Image> getResolutionVariants() {
             return Arrays.asList(this, highResolutionImage);
         }
     }
   ---------------------

  The current fix adds the MultiResolutionImage interface and public
resolution variant rendering hints.

Thanks,
Alexandr.




Reply via email to