[webkit-changes] [213764] trunk

Sat, 11 Mar 2017 19:01:41 -0800

Title: [213764] trunk
Revision
213764
Author
[email protected]
Date
2017-03-11 19:00:46 -0800 (Sat, 11 Mar 2017)

Log Message

Enable async image decoding for large images
https://bugs.webkit.org/show_bug.cgi?id=165039

Patch by Said Abou-Hallawa <[email protected]> on 2017-03-11
Reviewed by Simon Fraser.

Source/WebCore:

When BitmapImage::draw() is called for a large image, we are going to request async
image decoding for the native image instead of drawing it. If a lower resolution
native image is available for this, it is going to be drawn. Otherwise nothing will
be drawn. In both cases, a repaint will be scheduled for the image observer. This
should improve the image first time paint and the scrolling scenarios. It also makes
the scrolling more responsive by removing the decoding step from the main thread.

For now we are going to disable the asynchronous image decoding for the webkit test
runner because drawing the image does not block the page rendering anymore. An image
can be repainted later when its frame is ready for painting. This can cause a test
to fail because the webkit test runner may capture an image for the page before painting
all the images. The asynchronous image decoding can to be explicitly enabled from
the test page. Once the specs of the image 'async' attribute and 'ready' event is
approved, this should be revisited. It is important to test what we ship and eventually
async image decoding should be enabled in the webkit test runner.

* loader/cache/CachedImage.h: Change the default of LargeImageAsyncDecoding and AnimatedImageAsyncDecoding
to be false. This change fixes a layout test which creates an CachedImage inside an ImageDocument. The
CachedImage in this case does not have a loader so getting the values of these options from the settings
which is false for the DRT/WTR did not happen.
* platform/graphics/BitmapImage.cpp:
(WebCore::BitmapImage::destroyDecodedData): ImageSource::hasDecodingQueue() is renamed to ImageSource::hasAsyncDecodingQueue().
(WebCore::BitmapImage::frameImageAtIndex): Use String::utf8().data() instead of String::characters8().
(WebCore::BitmapImage::draw): If drawing the current frame is called while it is being
decoded, draw the current native image if the current frame was decoded but for a
different size  and and will not invoke decoding while painting. If the frame is being
decoded and there isn't a decoded frame, return without drawing but set a flag that
that this image needs a repaint.
(WebCore::BitmapImage::shouldUseAsyncDecodingForLargeImage): Renaming a function.
(WebCore::BitmapImage::shouldUseAsyncDecodingForAnimatedImage): Ditto.
(WebCore::BitmapImage::internalStartAnimation): Use String::utf8().data() instead of String::characters8().
(WebCore::BitmapImage::advanceAnimation): Ditto.
(WebCore::BitmapImage::internalAdvanceAnimation): Ditto.
(WebCore::BitmapImage::newFrameNativeImageAvailableAtIndex): Now this callback can be
called form the ImageFrameCache when finishing a frame of an animated image or the
frame of a large image. For large images, we need to call CachedImage::changedInRect()
if this image needs a repaint. If the decoding queue is idle, we should close it.
(WebCore::BitmapImage::isLargeImageAsyncDecodingRequired): Deleted. Function was renamed.
(WebCore::BitmapImage::isAnimatedImageAsyncDecodingRequired): Deleted. Ditto.
* platform/graphics/BitmapImage.h:

* platform/graphics/ImageFrameCache.cpp:
(WebCore::ImageFrameCache::~ImageFrameCache): hasDecodingQueue() was renamed to hasAsyncDecodingQueue().
(WebCore::ImageFrameCache::decodingQueue): Change the QNS of the decoding thread to be WorkQueue::QOS::Default.
WorkQueue::QOS::UserInteractive causes the scrolling thread to preempted  which can make the scrolling choppy.
(WebCore::ImageFrameCache::startAsyncDecodingQueue): hasDecodingQueue() was renamed to hasAsyncDecodingQueue().
(WebCore::ImageFrameCache::requestFrameAsyncDecodingAtIndex): Ditto.
(WebCore::ImageFrameCache::isAsyncDecodingQueueIdle): A helper function to tell whether the decoding thread is idle.
(WebCore::ImageFrameCache::stopAsyncDecodingQueue): hasDecodingQueue() was renamed to hasAsyncDecodingQueue().
* platform/graphics/ImageFrameCache.h:
(WebCore::ImageFrameCache::hasAsyncDecodingQueue): Rename this function to be consistent with the rest of the functions.
(WebCore::ImageFrameCache::hasDecodingQueue): Deleted.

* platform/graphics/ImageSource.cpp:
(WebCore::ImageSource::shouldUseAsyncDecoding): Renaming a function. Change the heuristic for large images be
a little bigger than the heuristic for animated images.
(WebCore::ImageSource::isAsyncDecodingRequired): Deleted.
* platform/graphics/ImageSource.h:
(WebCore::ImageSource::hasAsyncDecodingQueue): hasDecodingQueue() was renamed to hasAsyncDecodingQueue().
(WebCore::ImageSource::isAsyncDecodingQueueIdle): A wrapper for ImageFrameCache::isAsyncDecodingQueueIdle().
(WebCore::ImageSource::hasDecodingQueue): Deleted.

* platform/graphics/cg/ImageDecoderCG.cpp:
(WebCore::ImageDecoder::createFrameImageAtIndex): CGImageSourceCreateThumbnailAtIndex() returns a CGImage with
the image native size regardless of the subsamplingLevel unless kCGImageSourceSubsampleFactor is passed. Here
we are trying to see which size is smaller: the image native size or the sizeForDrawing. If we want a CGImage
with the image native size, sizeForDrawing will not passed. So we need to get the image native size with the
default subsampling and then compare it with sizeForDrawing.

Source/WebKit2:

Add WK2 preferences for setting/getting LargeImageAsyncDecoding and
AnimatedImageAsyncDecoding.

* UIProcess/API/C/WKPreferences.cpp:
(WKPreferencesSetLargeImageAsyncDecodingEnabled):
(WKPreferencesGetLargeImageAsyncDecodingEnabled):
(WKPreferencesSetAnimatedImageAsyncDecodingEnabled):
(WKPreferencesGetAnimatedImageAsyncDecodingEnabled):
* UIProcess/API/C/WKPreferencesRefPrivate.h:

Tools:

Disable LargeImageAsyncDecoding for DRT/WTR.

* DumpRenderTree/mac/DumpRenderTree.mm:
(resetWebPreferencesToConsistentValues):
* WebKitTestRunner/TestController.cpp:
(WTR::TestController::resetPreferencesToConsistentValues):

Modified Paths

Diff

Modified: trunk/Source/WebCore/ChangeLog (213763 => 213764)


--- trunk/Source/WebCore/ChangeLog	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Source/WebCore/ChangeLog	2017-03-12 03:00:46 UTC (rev 213764)
@@ -1,3 +1,79 @@
+2017-03-11  Said Abou-Hallawa  <[email protected]>
+
+        Enable async image decoding for large images
+        https://bugs.webkit.org/show_bug.cgi?id=165039
+
+        Reviewed by Simon Fraser.
+
+        When BitmapImage::draw() is called for a large image, we are going to request async
+        image decoding for the native image instead of drawing it. If a lower resolution 
+        native image is available for this, it is going to be drawn. Otherwise nothing will
+        be drawn. In both cases, a repaint will be scheduled for the image observer. This
+        should improve the image first time paint and the scrolling scenarios. It also makes
+        the scrolling more responsive by removing the decoding step from the main thread. 
+        
+        For now we are going to disable the asynchronous image decoding for the webkit test
+        runner because drawing the image does not block the page rendering anymore. An image
+        can be repainted later when its frame is ready for painting. This can cause a test
+        to fail because the webkit test runner may capture an image for the page before painting
+        all the images. The asynchronous image decoding can to be explicitly enabled from
+        the test page. Once the specs of the image 'async' attribute and 'ready' event is
+        approved, this should be revisited. It is important to test what we ship and eventually
+        async image decoding should be enabled in the webkit test runner.
+
+        * loader/cache/CachedImage.h: Change the default of LargeImageAsyncDecoding and AnimatedImageAsyncDecoding
+        to be false. This change fixes a layout test which creates an CachedImage inside an ImageDocument. The
+        CachedImage in this case does not have a loader so getting the values of these options from the settings
+        which is false for the DRT/WTR did not happen.
+        * platform/graphics/BitmapImage.cpp:
+        (WebCore::BitmapImage::destroyDecodedData): ImageSource::hasDecodingQueue() is renamed to ImageSource::hasAsyncDecodingQueue().
+        (WebCore::BitmapImage::frameImageAtIndex): Use String::utf8().data() instead of String::characters8().
+        (WebCore::BitmapImage::draw): If drawing the current frame is called while it is being
+        decoded, draw the current native image if the current frame was decoded but for a 
+        different size  and and will not invoke decoding while painting. If the frame is being
+        decoded and there isn't a decoded frame, return without drawing but set a flag that
+        that this image needs a repaint.
+        (WebCore::BitmapImage::shouldUseAsyncDecodingForLargeImage): Renaming a function.
+        (WebCore::BitmapImage::shouldUseAsyncDecodingForAnimatedImage): Ditto.
+        (WebCore::BitmapImage::internalStartAnimation): Use String::utf8().data() instead of String::characters8().
+        (WebCore::BitmapImage::advanceAnimation): Ditto.
+        (WebCore::BitmapImage::internalAdvanceAnimation): Ditto.
+        (WebCore::BitmapImage::newFrameNativeImageAvailableAtIndex): Now this callback can be
+        called form the ImageFrameCache when finishing a frame of an animated image or the
+        frame of a large image. For large images, we need to call CachedImage::changedInRect()
+        if this image needs a repaint. If the decoding queue is idle, we should close it.
+        (WebCore::BitmapImage::isLargeImageAsyncDecodingRequired): Deleted. Function was renamed.
+        (WebCore::BitmapImage::isAnimatedImageAsyncDecodingRequired): Deleted. Ditto.
+        * platform/graphics/BitmapImage.h:
+
+        * platform/graphics/ImageFrameCache.cpp:
+        (WebCore::ImageFrameCache::~ImageFrameCache): hasDecodingQueue() was renamed to hasAsyncDecodingQueue().
+        (WebCore::ImageFrameCache::decodingQueue): Change the QNS of the decoding thread to be WorkQueue::QOS::Default.
+        WorkQueue::QOS::UserInteractive causes the scrolling thread to preempted  which can make the scrolling choppy.
+        (WebCore::ImageFrameCache::startAsyncDecodingQueue): hasDecodingQueue() was renamed to hasAsyncDecodingQueue().
+        (WebCore::ImageFrameCache::requestFrameAsyncDecodingAtIndex): Ditto.
+        (WebCore::ImageFrameCache::isAsyncDecodingQueueIdle): A helper function to tell whether the decoding thread is idle.
+        (WebCore::ImageFrameCache::stopAsyncDecodingQueue): hasDecodingQueue() was renamed to hasAsyncDecodingQueue().
+        * platform/graphics/ImageFrameCache.h:
+        (WebCore::ImageFrameCache::hasAsyncDecodingQueue): Rename this function to be consistent with the rest of the functions.
+        (WebCore::ImageFrameCache::hasDecodingQueue): Deleted.
+
+        * platform/graphics/ImageSource.cpp:
+        (WebCore::ImageSource::shouldUseAsyncDecoding): Renaming a function. Change the heuristic for large images be
+        a little bigger than the heuristic for animated images.
+        (WebCore::ImageSource::isAsyncDecodingRequired): Deleted.
+        * platform/graphics/ImageSource.h:
+        (WebCore::ImageSource::hasAsyncDecodingQueue): hasDecodingQueue() was renamed to hasAsyncDecodingQueue().
+        (WebCore::ImageSource::isAsyncDecodingQueueIdle): A wrapper for ImageFrameCache::isAsyncDecodingQueueIdle().
+        (WebCore::ImageSource::hasDecodingQueue): Deleted.
+        
+        * platform/graphics/cg/ImageDecoderCG.cpp:
+        (WebCore::ImageDecoder::createFrameImageAtIndex): CGImageSourceCreateThumbnailAtIndex() returns a CGImage with
+        the image native size regardless of the subsamplingLevel unless kCGImageSourceSubsampleFactor is passed. Here
+        we are trying to see which size is smaller: the image native size or the sizeForDrawing. If we want a CGImage
+        with the image native size, sizeForDrawing will not passed. So we need to get the image native size with the
+        default subsampling and then compare it with sizeForDrawing.
+
 2017-03-11  Jon Lee  <[email protected]>
 
         WebGPU prototype - Front-End

Modified: trunk/Source/WebCore/loader/cache/CachedImage.h (213763 => 213764)


--- trunk/Source/WebCore/loader/cache/CachedImage.h	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Source/WebCore/loader/cache/CachedImage.h	2017-03-12 03:00:46 UTC (rev 213764)
@@ -144,8 +144,8 @@
 #else
         bool m_allowSubsampling { false };
 #endif
-        bool m_allowLargeImageAsyncDecoding { true };
-        bool m_allowAnimatedImageAsyncDecoding { true };
+        bool m_allowLargeImageAsyncDecoding { false };
+        bool m_allowAnimatedImageAsyncDecoding { false };
         bool m_showDebugBackground { false };
     };

Modified: trunk/Source/WebCore/platform/graphics/BitmapImage.cpp (213763 => 213764)


--- trunk/Source/WebCore/platform/graphics/BitmapImage.cpp	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Source/WebCore/platform/graphics/BitmapImage.cpp	2017-03-12 03:00:46 UTC (rev 213764)
@@ -67,7 +67,7 @@
 {
     if (!destroyAll)
         m_source.destroyDecodedDataBeforeFrame(m_currentFrame);
-    else if (m_source.hasDecodingQueue())
+    else if (m_source.hasAsyncDecodingQueue())
         m_source.destroyAllDecodedDataExcludeFrame(m_currentFrame);
     else
         m_source.destroyAllDecodedData();
@@ -74,7 +74,7 @@
 
     // There's no need to throw away the decoder unless we're explicitly asked
     // to destroy all of the frames.
-    if (!destroyAll || m_source.hasDecodingQueue())
+    if (!destroyAll || m_source.hasAsyncDecodingQueue())
         m_source.clearFrameBufferCache(m_currentFrame);
     else
         m_source.clear(data());
@@ -103,7 +103,7 @@
 NativeImagePtr BitmapImage::frameImageAtIndex(size_t index, const std::optional<SubsamplingLevel>& subsamplingLevel, const std::optional<IntSize>& sizeForDrawing, const GraphicsContext* targetContext)
 {
     if (!frameHasValidNativeImageAtIndex(index, subsamplingLevel, sizeForDrawing)) {
-        LOG(Images, "BitmapImage::%s - %p - url: %s [subsamplingLevel was %d, resampling]", __FUNCTION__, this, sourceURL().characters8(), static_cast<int>(frameSubsamplingLevelAtIndex(index)));
+        LOG(Images, "BitmapImage::%s - %p - url: %s [subsamplingLevel was %d, resampling]", __FUNCTION__, this, sourceURL().utf8().data(), static_cast<int>(frameSubsamplingLevelAtIndex(index)));
         invalidatePlatformData();
     }
 
@@ -161,29 +161,53 @@
     if (destRect.isEmpty() || srcRect.isEmpty())
         return;
 
-    m_sizeForDrawing = enclosingIntRect(destRect).size();
-    StartAnimationResult result = internalStartAnimation();
+    float scale = subsamplingScale(context, destRect, srcRect);
+    m_currentSubsamplingLevel = allowSubsampling() ? m_source.subsamplingLevelForScale(scale) : SubsamplingLevel::Default;
+    m_sizeForDrawing = enclosingIntRect(context.getCTM().mapRect(destRect)).size();
 
-    Color color;
-    if (result == StartAnimationResult::DecodingActive && showDebugBackground())
-        color = Color::yellow;
-    else
-        color = singlePixelSolidColor();
+    LOG(Images, "BitmapImage::%s - %p - url: %s [subsamplingLevel = %d scale = %.4f]", __FUNCTION__, this, sourceURL().utf8().data(), static_cast<int>(m_currentSubsamplingLevel), scale);
 
-    if (color.isValid()) {
-        fillWithSolidColor(context, destRect, color, op);
+    StartAnimationResult result = internalStartAnimation();
+    if (result == StartAnimationResult::DecodingActive && showDebugBackground()) {
+        fillWithSolidColor(context, destRect, Color::yellow, op);
         return;
     }
 
-    float scale = subsamplingScale(context, destRect, srcRect);
-    m_currentSubsamplingLevel = allowSubsampling() ? m_source.subsamplingLevelForScale(scale) : SubsamplingLevel::Default;
-    LOG(Images, "BitmapImage::%s - %p - url: %s [m_currentFrame = %ld subsamplingLevel = %d scale = %.4f]", __FUNCTION__, this, sourceURL().characters8(), m_currentFrame, static_cast<int>(m_currentSubsamplingLevel), scale);
+    ASSERT_IMPLIES(result == StartAnimationResult::DecodingActive, m_source.frameHasValidNativeImageAtIndex(m_currentFrame, m_currentSubsamplingLevel, m_sizeForDrawing));
 
-    ASSERT_IMPLIES(result == StartAnimationResult::DecodingActive, m_source.frameHasValidNativeImageAtIndex(m_currentFrame, m_currentSubsamplingLevel, m_sizeForDrawing));
-    auto image = frameImageAtIndex(m_currentFrame, m_currentSubsamplingLevel, m_sizeForDrawing, &context);
+    NativeImagePtr image;
+    if (shouldUseAsyncDecodingForLargeImage()) {
+        if (m_source.frameHasValidNativeImageAtIndex(m_currentFrame, m_currentSubsamplingLevel, m_sizeForDrawing))
+            image = frameImageAtIndex(m_currentFrame, m_currentSubsamplingLevel, m_sizeForDrawing, &context);
+        else {
+            ASSERT(!canAnimate() && !m_currentFrame);
+            if (!frameIsBeingDecodedAtIndex(m_currentFrame, m_sizeForDrawing)) {
+                m_source.requestFrameAsyncDecodingAtIndex(0, m_currentSubsamplingLevel, m_sizeForDrawing);
+                LOG(Images, "BitmapImage::%s - %p - url: %s [requesting large async decoding]", __FUNCTION__, this, sourceURL().utf8().data());
+            }
+
+            if (!frameHasDecodedNativeImage(m_currentFrame)) {
+                if (showDebugBackground())
+                    fillWithSolidColor(context, destRect, Color::yellow, op);
+                return;
+            }
+
+            image = frameImageAtIndex(m_currentFrame);
+        }
+    } else {
+        ASSERT(!frameIsBeingDecodedAtIndex(m_currentFrame, m_sizeForDrawing));
+        image = frameImageAtIndex(m_currentFrame, m_currentSubsamplingLevel, { }, &context);
+    }
+
     if (!image) // If it's too early we won't have an image yet.
         return;
 
+    Color color = singlePixelSolidColor();
+    if (color.isValid()) {
+        fillWithSolidColor(context, destRect, color, op);
+        return;
+    }
+
     ImageOrientation orientation(description.imageOrientation());
     if (description.respectImageOrientation() == RespectImageOrientation)
         orientation = frameOrientationAtIndex(m_currentFrame);
@@ -238,14 +262,14 @@
     return shouldAnimate() && frameCount() > 1;
 }
 
-bool BitmapImage::isLargeImageAsyncDecodingRequired()
+bool BitmapImage::shouldUseAsyncDecodingForLargeImage()
 {
-    return !canAnimate() && allowLargeImageAsyncDecoding() && (isAsyncDecodingForcedForTesting() || m_source.isAsyncDecodingRequired());
+    return !canAnimate() && allowLargeImageAsyncDecoding() && (shouldUseAsyncDecodingForTesting() || m_source.shouldUseAsyncDecoding());
 }
 
-bool BitmapImage::isAnimatedImageAsyncDecodingRequired()
+bool BitmapImage::shouldUseAsyncDecodingForAnimatedImage()
 {
-    return canAnimate() && allowAnimatedImageAsyncDecoding() && (isAsyncDecodingForcedForTesting() || m_source.isAsyncDecodingRequired());
+    return canAnimate() && allowAnimatedImageAsyncDecoding() && (shouldUseAsyncDecodingForTesting() || m_source.shouldUseAsyncDecoding());
 }
 
 void BitmapImage::clearTimer()
@@ -267,11 +291,11 @@
 
     if (m_frameTimer)
         return StartAnimationResult::TimerActive;
-    
+
     // Don't start a new animation until we draw the frame that is currently being decoded.
     size_t nextFrame = (m_currentFrame + 1) % frameCount();
     if (frameIsBeingDecodedAtIndex(nextFrame, m_sizeForDrawing)) {
-        LOG(Images, "BitmapImage::%s - %p - url: %s [nextFrame = %ld is being decoded]", __FUNCTION__, this, sourceURL().characters8(), nextFrame);
+        LOG(Images, "BitmapImage::%s - %p - url: %s [nextFrame = %ld is being decoded]", __FUNCTION__, this, sourceURL().utf8().data(), nextFrame);
         return StartAnimationResult::DecodingActive;
     }
 
@@ -312,14 +336,14 @@
     // it will be decoded on a separate work queue. When decoding nextFrame finishes, we will be notified
     // through the callback newFrameNativeImageAvailableAtIndex(). Otherwise, advanceAnimation() will be called
     // when the timer fires and m_currentFrame will be advanced to nextFrame since it is not being decoded.
-    if (m_sizeForDrawing && isAnimatedImageAsyncDecodingRequired()) {
-        bool isAsyncDecode = m_source.requestFrameAsyncDecodingAtIndex(nextFrame, m_currentSubsamplingLevel, *m_sizeForDrawing);
+    if (shouldUseAsyncDecodingForAnimatedImage()) {
+        bool isAsyncDecode = m_source.requestFrameAsyncDecodingAtIndex(nextFrame, m_currentSubsamplingLevel, m_sizeForDrawing);
 
 #if !LOG_DISABLED
         if (isAsyncDecode)
-            LOG(Images, "BitmapImage::%s - %p - url: %s [requesting async decoding for nextFrame = %ld]", __FUNCTION__, this, sourceURL().characters8(), nextFrame);
+            LOG(Images, "BitmapImage::%s - %p - url: %s [requesting async decoding for nextFrame = %ld]", __FUNCTION__, this, sourceURL().utf8().data(), nextFrame);
         else
-            LOG(Images, "BitmapImage::%s - %p - url: %s [cachedFrameCount = %ld nextFrame = %ld]", __FUNCTION__, this, sourceURL().characters8(), ++m_cachedFrameCount, nextFrame);
+            LOG(Images, "BitmapImage::%s - %p - url: %s [cachedFrameCount = %ld nextFrame = %ld]", __FUNCTION__, this, sourceURL().utf8().data(), ++m_cachedFrameCount, nextFrame);
 #else
         UNUSED_PARAM(isAsyncDecode);
 #endif
@@ -338,7 +362,7 @@
 
     // Pretend as if decoding nextFrame has taken m_frameDecodingDurationForTesting from
     // the time this decoding was requested.
-    if (isAsyncDecodingForcedForTesting()) {
+    if (shouldUseAsyncDecodingForTesting()) {
         double time = monotonicallyIncreasingTime();
         // Start a timer with the remaining time from now till the m_desiredFrameDecodeTime.
         if (m_desiredFrameDecodeTimeForTesting > std::max(time, m_desiredFrameStartTime)) {
@@ -346,7 +370,7 @@
             return;
         }
     }
-    
+
     // Don't advance to nextFrame unless its decoding has finished or was not required.
     size_t nextFrame = (m_currentFrame + 1) % frameCount();
     if (!frameIsBeingDecodedAtIndex(nextFrame, m_sizeForDrawing))
@@ -355,7 +379,7 @@
         // Force repaint if showDebugBackground() is on.
         if (showDebugBackground())
             imageObserver()->changedInRect(this);
-        LOG(Images, "BitmapImage::%s - %p - url: %s [lateFrameCount = %ld nextFrame = %ld]", __FUNCTION__, this, sourceURL().characters8(), ++m_lateFrameCount, nextFrame);
+        LOG(Images, "BitmapImage::%s - %p - url: %s [lateFrameCount = %ld nextFrame = %ld]", __FUNCTION__, this, sourceURL().utf8().data(), ++m_lateFrameCount, nextFrame);
     }
 }
 
@@ -369,7 +393,7 @@
     if (imageObserver())
         imageObserver()->animationAdvanced(this);
 
-    LOG(Images, "BitmapImage::%s - %p - url: %s [m_currentFrame = %ld]", __FUNCTION__, this, sourceURL().characters8(), m_currentFrame);
+    LOG(Images, "BitmapImage::%s - %p - url: %s [m_currentFrame = %ld]", __FUNCTION__, this, sourceURL().utf8().data(), m_currentFrame);
 }
 
 void BitmapImage::stopAnimation()
@@ -395,13 +419,21 @@
 void BitmapImage::newFrameNativeImageAvailableAtIndex(size_t index)
 {
     UNUSED_PARAM(index);
-    ASSERT(index == (m_currentFrame + 1) % frameCount());
-
-    // Don't advance to nextFrame unless the timer was fired before its decoding finishes.
-    if (canAnimate() && !m_frameTimer)
-        internalAdvanceAnimation();
-    else
-        LOG(Images, "BitmapImage::%s - %p - url: %s [earlyFrameCount = %ld nextFrame = %ld]", __FUNCTION__, this, sourceURL().characters8(), ++m_earlyFrameCount, index);
+    if (canAnimate()) {
+        ASSERT(index == (m_currentFrame + 1) % frameCount());
+        
+        // Don't advance to nextFrame unless the timer was fired before its decoding finishes.
+        if (canAnimate() && !m_frameTimer)
+            internalAdvanceAnimation();
+        else
+            LOG(Images, "BitmapImage::%s - %p - url: %s [earlyFrameCount = %ld nextFrame = %ld]", __FUNCTION__, this, sourceURL().utf8().data(), ++m_earlyFrameCount, index);
+    } else {
+        ASSERT(index == m_currentFrame && !m_currentFrame);
+        imageObserver()->changedInRect(this, nullptr);
+        
+        if (m_source.isAsyncDecodingQueueIdle())
+            m_source.stopAsyncDecodingQueue();
+    }
 }
 
 void BitmapImage::dump(TextStream& ts) const

Modified: trunk/Source/WebCore/platform/graphics/BitmapImage.h (213763 => 213764)


--- trunk/Source/WebCore/platform/graphics/BitmapImage.h	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Source/WebCore/platform/graphics/BitmapImage.h	2017-03-12 03:00:46 UTC (rev 213764)
@@ -96,10 +96,10 @@
     bool currentFrameKnownToBeOpaque() const override { return !frameHasAlphaAtIndex(currentFrame()); }
     ImageOrientation orientationForCurrentFrame() const override { return frameOrientationAtIndex(currentFrame()); }
 
-    bool isAsyncDecodingForcedForTesting() const { return m_frameDecodingDurationForTesting > 0; }
+    bool shouldUseAsyncDecodingForTesting() const { return m_frameDecodingDurationForTesting > 0; }
     void setFrameDecodingDurationForTesting(float duration) { m_frameDecodingDurationForTesting = duration; }
-    bool isLargeImageAsyncDecodingRequired();
-    bool isAnimatedImageAsyncDecodingRequired();
+    bool shouldUseAsyncDecodingForLargeImage();
+    bool shouldUseAsyncDecodingForAnimatedImage();
 
     // Accessors for native image formats.
 #if USE(APPKIT)
@@ -202,7 +202,7 @@
 
     size_t m_currentFrame { 0 }; // The index of the current frame of animation.
     SubsamplingLevel m_currentSubsamplingLevel { SubsamplingLevel::Default };
-    std::optional<IntSize> m_sizeForDrawing;
+    IntSize m_sizeForDrawing;
     std::unique_ptr<Timer> m_frameTimer;
     RepetitionCount m_repetitionsComplete { RepetitionCountNone }; // How many repetitions we've finished.
     double m_desiredFrameStartTime { 0 }; // The system time at which we hope to see the next call to startAnimation().

Modified: trunk/Source/WebCore/platform/graphics/ImageFrameCache.cpp (213763 => 213764)


--- trunk/Source/WebCore/platform/graphics/ImageFrameCache.cpp	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Source/WebCore/platform/graphics/ImageFrameCache.cpp	2017-03-12 03:00:46 UTC (rev 213764)
@@ -67,7 +67,7 @@
 
 ImageFrameCache::~ImageFrameCache()
 {
-    ASSERT(!hasDecodingQueue());
+    ASSERT(!hasAsyncDecodingQueue());
 }
 
 void ImageFrameCache::destroyDecodedData(size_t frameCount, size_t excludeFrame)
@@ -256,7 +256,7 @@
 Ref<WorkQueue> ImageFrameCache::decodingQueue()
 {
     if (!m_decodingQueue)
-        m_decodingQueue = WorkQueue::create("org.webkit.ImageDecoder", WorkQueue::Type::Serial, WorkQueue::QOS::UserInteractive);
+        m_decodingQueue = WorkQueue::create("org.webkit.ImageDecoder", WorkQueue::Type::Serial, WorkQueue::QOS::Default);
     
     return *m_decodingQueue;
 }
@@ -263,7 +263,7 @@
 
 void ImageFrameCache::startAsyncDecodingQueue()
 {
-    if (hasDecodingQueue() || !isDecoderAvailable())
+    if (hasAsyncDecodingQueue() || !isDecoderAvailable())
         return;
 
     m_frameRequestQueue.open();
@@ -306,7 +306,7 @@
     if (frame.hasValidNativeImage(subsamplingLevel, sizeForDrawing))
         return false;
 
-    if (!hasDecodingQueue())
+    if (!hasAsyncDecodingQueue())
         startAsyncDecodingQueue();
     
     frame.enqueueSizeForDecoding(sizeForDrawing);
@@ -314,9 +314,18 @@
     return true;
 }
 
+bool ImageFrameCache::isAsyncDecodingQueueIdle() const
+{
+    for (const ImageFrame& frame : m_frames) {
+        if (frame.isBeingDecoded())
+            return false;
+    }
+    return true;
+}
+    
 void ImageFrameCache::stopAsyncDecodingQueue()
 {
-    if (!hasDecodingQueue())
+    if (!hasAsyncDecodingQueue())
         return;
     
     m_frameRequestQueue.close();

Modified: trunk/Source/WebCore/platform/graphics/ImageFrameCache.h (213763 => 213764)


--- trunk/Source/WebCore/platform/graphics/ImageFrameCache.h	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Source/WebCore/platform/graphics/ImageFrameCache.h	2017-03-12 03:00:46 UTC (rev 213764)
@@ -71,7 +71,8 @@
     void startAsyncDecodingQueue();
     bool requestFrameAsyncDecodingAtIndex(size_t, SubsamplingLevel, const IntSize&);
     void stopAsyncDecodingQueue();
-    bool hasDecodingQueue() { return m_decodingQueue; }
+    bool hasAsyncDecodingQueue() const { return m_decodingQueue; }
+    bool isAsyncDecodingQueueIdle() const;
 
     // Image metadata which is calculated either by the ImageDecoder or directly
     // from the NativeImage if this class was created for a memory image.

Modified: trunk/Source/WebCore/platform/graphics/ImageSource.cpp (213763 => 213764)


--- trunk/Source/WebCore/platform/graphics/ImageSource.cpp	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Source/WebCore/platform/graphics/ImageSource.cpp	2017-03-12 03:00:46 UTC (rev 213764)
@@ -148,10 +148,10 @@
     return isDecoderAvailable() ? m_decoder->isAllDataReceived() : m_frameCache->frameCount();
 }
 
-bool ImageSource::isAsyncDecodingRequired()
+bool ImageSource::shouldUseAsyncDecoding()
 {
     // FIXME: figure out the best heuristic for enabling async image decoding.
-    return size().area() * sizeof(RGBA32) >= 100 * KB;
+    return size().area() * sizeof(RGBA32) >= (frameCount() > 1 ? 100 * KB : 500 * KB);
 }
 
 SubsamplingLevel ImageSource::maximumSubsamplingLevel()

Modified: trunk/Source/WebCore/platform/graphics/ImageSource.h (213763 => 213764)


--- trunk/Source/WebCore/platform/graphics/ImageSource.h	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Source/WebCore/platform/graphics/ImageSource.h	2017-03-12 03:00:46 UTC (rev 213764)
@@ -68,9 +68,10 @@
     unsigned decodedSize() const { return m_frameCache->decodedSize(); }
     bool isAllDataReceived();
 
-    bool isAsyncDecodingRequired();
+    bool shouldUseAsyncDecoding();
     bool requestFrameAsyncDecodingAtIndex(size_t index, SubsamplingLevel subsamplingLevel, const IntSize& sizeForDrawing) { return m_frameCache->requestFrameAsyncDecodingAtIndex(index, subsamplingLevel, sizeForDrawing); }
-    bool hasDecodingQueue() const { return m_frameCache->hasDecodingQueue(); }
+    bool hasAsyncDecodingQueue() const { return m_frameCache->hasAsyncDecodingQueue(); }
+    bool isAsyncDecodingQueueIdle() const  { return m_frameCache->isAsyncDecodingQueueIdle(); }
     void stopAsyncDecodingQueue() { m_frameCache->stopAsyncDecodingQueue(); }
 
     // Image metadata which is calculated by the decoder or can deduced by the case of the memory NativeImage.

Modified: trunk/Source/WebCore/platform/graphics/cg/ImageDecoderCG.cpp (213763 => 213764)


--- trunk/Source/WebCore/platform/graphics/cg/ImageDecoderCG.cpp	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Source/WebCore/platform/graphics/cg/ImageDecoderCG.cpp	2017-03-12 03:00:46 UTC (rev 213764)
@@ -377,7 +377,13 @@
         options = imageSourceOptions(subsamplingLevel);
         image = adoptCF(CGImageSourceCreateImageAtIndex(m_nativeDecoder.get(), index, options.get()));
     } else {
-        IntSize size = frameSizeAtIndex(index, subsamplingLevel);
+        // CGImageSourceCreateThumbnailAtIndex() returns a CGImage with the image native size
+        // regardless of the subsamplingLevel unless kCGImageSourceSubsampleFactor is passed.
+        // Here we are trying to see which size is smaller: the image native size or the
+        // sizeForDrawing. If we want a CGImage with the image native size, sizeForDrawing will
+        // not passed. So we need to get the image native size with the default subsampling and
+        // then compare it with sizeForDrawing.
+        IntSize size = frameSizeAtIndex(index, SubsamplingLevel::Default);
 
         if (size.unclampedArea() < sizeForDrawing.value().unclampedArea()) {
             // Decode an image asynchronously for its native size.

Modified: trunk/Source/WebKit2/ChangeLog (213763 => 213764)


--- trunk/Source/WebKit2/ChangeLog	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Source/WebKit2/ChangeLog	2017-03-12 03:00:46 UTC (rev 213764)
@@ -1,3 +1,20 @@
+2017-03-11  Said Abou-Hallawa  <[email protected]>
+
+        Enable async image decoding for large images
+        https://bugs.webkit.org/show_bug.cgi?id=165039
+
+        Reviewed by Simon Fraser.
+
+        Add WK2 preferences for setting/getting LargeImageAsyncDecoding and
+        AnimatedImageAsyncDecoding.
+        
+        * UIProcess/API/C/WKPreferences.cpp:
+        (WKPreferencesSetLargeImageAsyncDecodingEnabled):
+        (WKPreferencesGetLargeImageAsyncDecodingEnabled):
+        (WKPreferencesSetAnimatedImageAsyncDecodingEnabled):
+        (WKPreferencesGetAnimatedImageAsyncDecodingEnabled):
+        * UIProcess/API/C/WKPreferencesRefPrivate.h:
+
 2017-03-11  Alex Christensen  <[email protected]>
 
         Rollout r213746

Modified: trunk/Source/WebKit2/UIProcess/API/C/WKPreferences.cpp (213763 => 213764)


--- trunk/Source/WebKit2/UIProcess/API/C/WKPreferences.cpp	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Source/WebKit2/UIProcess/API/C/WKPreferences.cpp	2017-03-12 03:00:46 UTC (rev 213764)
@@ -1691,6 +1691,26 @@
     return toImpl(preferencesRef)->linkPreloadEnabled();
 }
 
+void WKPreferencesSetLargeImageAsyncDecodingEnabled(WKPreferencesRef preferencesRef, bool flag)
+{
+    toImpl(preferencesRef)->setLargeImageAsyncDecodingEnabled(flag);
+}
+
+bool WKPreferencesGetLargeImageAsyncDecodingEnabled(WKPreferencesRef preferencesRef)
+{
+    return toImpl(preferencesRef)->largeImageAsyncDecodingEnabled();
+}
+
+void WKPreferencesSetAnimatedImageAsyncDecodingEnabled(WKPreferencesRef preferencesRef, bool flag)
+{
+    toImpl(preferencesRef)->setAnimatedImageAsyncDecodingEnabled(flag);
+}
+
+bool WKPreferencesGetAnimatedImageAsyncDecodingEnabled(WKPreferencesRef preferencesRef)
+{
+    return toImpl(preferencesRef)->animatedImageAsyncDecodingEnabled();
+}
+
 void WKPreferencesSetShouldSuppressKeyboardInputDuringProvisionalNavigation(WKPreferencesRef preferencesRef, bool flag)
 {
     toImpl(preferencesRef)->setShouldSuppressKeyboardInputDuringProvisionalNavigation(flag);

Modified: trunk/Source/WebKit2/UIProcess/API/C/WKPreferencesRefPrivate.h (213763 => 213764)


--- trunk/Source/WebKit2/UIProcess/API/C/WKPreferencesRefPrivate.h	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Source/WebKit2/UIProcess/API/C/WKPreferencesRefPrivate.h	2017-03-12 03:00:46 UTC (rev 213764)
@@ -465,6 +465,14 @@
 WK_EXPORT void WKPreferencesSetSubtleCryptoEnabled(WKPreferencesRef, bool flag);
 WK_EXPORT bool WKPreferencesGetSubtleCryptoEnabled(WKPreferencesRef);
 
+// Defaults to true.
+WK_EXPORT void WKPreferencesSetLargeImageAsyncDecodingEnabled(WKPreferencesRef preferencesRef, bool flag);
+WK_EXPORT bool WKPreferencesGetLargeImageAsyncDecodingEnabled(WKPreferencesRef preferencesRef);
+
+// Defaults to true.
+WK_EXPORT void WKPreferencesSetAnimatedImageAsyncDecodingEnabled(WKPreferencesRef preferencesRef, bool flag);
+WK_EXPORT bool WKPreferencesGetAnimatedImageAsyncDecodingEnabled(WKPreferencesRef preferencesRef);
+
 // Defaults to false
 WK_EXPORT void WKPreferencesSetShouldSuppressKeyboardInputDuringProvisionalNavigation(WKPreferencesRef, bool flag);
 WK_EXPORT bool WKPreferencesGetShouldSuppressKeyboardInputDuringProvisionalNavigation(WKPreferencesRef);

Modified: trunk/Tools/ChangeLog (213763 => 213764)


--- trunk/Tools/ChangeLog	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Tools/ChangeLog	2017-03-12 03:00:46 UTC (rev 213764)
@@ -1,3 +1,17 @@
+2017-03-11  Said Abou-Hallawa  <[email protected]>
+
+        Enable async image decoding for large images
+        https://bugs.webkit.org/show_bug.cgi?id=165039
+
+        Reviewed by Simon Fraser.
+
+        Disable LargeImageAsyncDecoding for DRT/WTR.
+
+        * DumpRenderTree/mac/DumpRenderTree.mm:
+        (resetWebPreferencesToConsistentValues):
+        * WebKitTestRunner/TestController.cpp:
+        (WTR::TestController::resetPreferencesToConsistentValues):
+
 2017-03-10  Alex Christensen  <[email protected]>
 
         Fix watch and tv builds after r213294

Modified: trunk/Tools/DumpRenderTree/mac/DumpRenderTree.mm (213763 => 213764)


--- trunk/Tools/DumpRenderTree/mac/DumpRenderTree.mm	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Tools/DumpRenderTree/mac/DumpRenderTree.mm	2017-03-12 03:00:46 UTC (rev 213764)
@@ -956,6 +956,8 @@
     [preferences setHiddenPageCSSAnimationSuspensionEnabled:NO];
     
     [preferences setMediaStreamEnabled:YES];
+    
+    [preferences setLargeImageAsyncDecodingEnabled:NO];
 
     [WebPreferences _clearNetworkLoaderSession];
     [WebPreferences _setCurrentNetworkLoaderSessionCookieAcceptPolicy:NSHTTPCookieAcceptPolicyOnlyFromMainDocumentDomain];

Modified: trunk/Tools/WebKitTestRunner/TestController.cpp (213763 => 213764)


--- trunk/Tools/WebKitTestRunner/TestController.cpp	2017-03-12 01:07:32 UTC (rev 213763)
+++ trunk/Tools/WebKitTestRunner/TestController.cpp	2017-03-12 03:00:46 UTC (rev 213764)
@@ -719,6 +719,8 @@
     WKCookieManagerDeleteAllCookies(WKContextGetCookieManager(m_context.get()));
 
     WKPreferencesSetMockCaptureDevicesEnabled(preferences, true);
+    
+    WKPreferencesSetLargeImageAsyncDecodingEnabled(preferences, false);
 
     platformResetPreferencesToConsistentValues();
 }

_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to