Title: [218375] trunk
Revision
218375
Author
jer.no...@apple.com
Date
2017-06-15 19:49:38 -0700 (Thu, 15 Jun 2017)

Log Message

[WebRTC] Removing a MediaStreamTrack from a MediaStream reports no recording to WebKit clients
https://bugs.webkit.org/show_bug.cgi?id=173398
<rdar://problem/32592961>

Reviewed by Eric Carlson.

Source/WebCore:

API Test: Tests/WebKit2/MediaStreamTrackDetached.mm

Move the definition of a MediaStream as a MediaProducer from the stream itself to its constituent
MediaStreamTracks. This ensures that, even if a MediaStreamTrack is removed from its stream, the
document (and thus the clients) are notified that media capture is still occurring.

Though MediaStream is no longer a MediaProducer, it still uses the MediaProducer's state concept
to determine when to fire events. However it's mediaState() implementation will be moved into
MediaStreamTrack, and will instead simply bitwise-or together each of it's track's mediaState().

The MediaStream notifies the document that its state has changed asynchronously, so do the same
for MediaStreamTrack (which reduces the number of calls to the client when changes all occur
during a single run loop).

Because the MediaStreamTrackPrivate may be started externally (not by the MediaStreamTrack directly),
add a new client method that notifies observers when the track has been started, and the
MediaStreamTrack will use this notification to update the document with it's new mediaState().

* Modules/mediastream/MediaStream.cpp:
(WebCore::MediaStream::MediaStream):
(WebCore::MediaStream::~MediaStream):
(WebCore::MediaStream::mediaState):
(WebCore::MediaStream::statusDidChange):
(WebCore::MediaStream::characteristicsChanged):
(WebCore::MediaStream::pageMutedStateDidChange): Deleted.
* Modules/mediastream/MediaStream.h:
* Modules/mediastream/MediaStreamTrack.cpp:
(WebCore::MediaStreamTrack::MediaStreamTrack):
(WebCore::MediaStreamTrack::~MediaStreamTrack):
(WebCore::MediaStreamTrack::pageMutedStateDidChange):
(WebCore::MediaStreamTrack::mediaState):
(WebCore::MediaStreamTrack::trackStarted):
(WebCore::MediaStreamTrack::configureTrackRendering):
(WebCore::MediaStreamTrack::stop):
(WebCore::MediaStreamTrack::document):
* Modules/mediastream/MediaStreamTrack.h:
(WebCore::MediaStreamTrack::source):
* platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:
* platform/mediastream/MediaStreamPrivate.cpp:
(WebCore::MediaStreamPrivate::trackStarted):
* platform/mediastream/MediaStreamPrivate.h:
* platform/mediastream/MediaStreamTrackPrivate.cpp:
(WebCore::MediaStreamTrackPrivate::sourceStarted):
* platform/mediastream/MediaStreamTrackPrivate.h:
* platform/mediastream/RealtimeMediaSource.cpp:
(WebCore::RealtimeMediaSource::start):
* platform/mediastream/RealtimeMediaSource.h:

Tools:

* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WebKit2/MediaStreamTrackDetached.mm: Added.
(-[MediaStreamTrackDetachedUIDelegate _webView:requestUserMediaAuthorizationForDevices:url:mainFrameURL:decisionHandler:]):
(-[MediaStreamTrackDetachedUIDelegate _webView:checkUserMediaPermissionForURL:mainFrameURL:frameIdentifier:decisionHandler:]):
(-[MediaStreamTrackDetachedUIDelegate _webView:mediaCaptureStateDidChange:]):
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WebKit2/mediastreamtrack-detached.html: Added.

Modified Paths

Added Paths

Diff

Modified: trunk/Source/WebCore/ChangeLog (218374 => 218375)


--- trunk/Source/WebCore/ChangeLog	2017-06-16 01:43:27 UTC (rev 218374)
+++ trunk/Source/WebCore/ChangeLog	2017-06-16 02:49:38 UTC (rev 218375)
@@ -1,3 +1,59 @@
+2017-06-15  Jer Noble  <jer.no...@apple.com>
+
+        [WebRTC] Removing a MediaStreamTrack from a MediaStream reports no recording to WebKit clients
+        https://bugs.webkit.org/show_bug.cgi?id=173398
+        <rdar://problem/32592961>
+
+        Reviewed by Eric Carlson.
+
+        API Test: Tests/WebKit2/MediaStreamTrackDetached.mm
+
+        Move the definition of a MediaStream as a MediaProducer from the stream itself to its constituent
+        MediaStreamTracks. This ensures that, even if a MediaStreamTrack is removed from its stream, the
+        document (and thus the clients) are notified that media capture is still occurring.
+
+        Though MediaStream is no longer a MediaProducer, it still uses the MediaProducer's state concept
+        to determine when to fire events. However it's mediaState() implementation will be moved into
+        MediaStreamTrack, and will instead simply bitwise-or together each of it's track's mediaState().
+
+        The MediaStream notifies the document that its state has changed asynchronously, so do the same
+        for MediaStreamTrack (which reduces the number of calls to the client when changes all occur
+        during a single run loop).
+
+        Because the MediaStreamTrackPrivate may be started externally (not by the MediaStreamTrack directly),
+        add a new client method that notifies observers when the track has been started, and the
+        MediaStreamTrack will use this notification to update the document with it's new mediaState().
+
+        * Modules/mediastream/MediaStream.cpp:
+        (WebCore::MediaStream::MediaStream):
+        (WebCore::MediaStream::~MediaStream):
+        (WebCore::MediaStream::mediaState):
+        (WebCore::MediaStream::statusDidChange):
+        (WebCore::MediaStream::characteristicsChanged):
+        (WebCore::MediaStream::pageMutedStateDidChange): Deleted.
+        * Modules/mediastream/MediaStream.h:
+        * Modules/mediastream/MediaStreamTrack.cpp:
+        (WebCore::MediaStreamTrack::MediaStreamTrack):
+        (WebCore::MediaStreamTrack::~MediaStreamTrack):
+        (WebCore::MediaStreamTrack::pageMutedStateDidChange):
+        (WebCore::MediaStreamTrack::mediaState):
+        (WebCore::MediaStreamTrack::trackStarted):
+        (WebCore::MediaStreamTrack::configureTrackRendering):
+        (WebCore::MediaStreamTrack::stop):
+        (WebCore::MediaStreamTrack::document):
+        * Modules/mediastream/MediaStreamTrack.h:
+        (WebCore::MediaStreamTrack::source):
+        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:
+        * platform/mediastream/MediaStreamPrivate.cpp:
+        (WebCore::MediaStreamPrivate::trackStarted):
+        * platform/mediastream/MediaStreamPrivate.h:
+        * platform/mediastream/MediaStreamTrackPrivate.cpp:
+        (WebCore::MediaStreamTrackPrivate::sourceStarted):
+        * platform/mediastream/MediaStreamTrackPrivate.h:
+        * platform/mediastream/RealtimeMediaSource.cpp:
+        (WebCore::RealtimeMediaSource::start):
+        * platform/mediastream/RealtimeMediaSource.h:
+
 2017-06-15  Simon Fraser  <simon.fra...@apple.com>
 
         Use WTFLogAlways for debug logging so that it shows up in device system logs

Modified: trunk/Source/WebCore/Modules/mediastream/MediaStream.cpp (218374 => 218375)


--- trunk/Source/WebCore/Modules/mediastream/MediaStream.cpp	2017-06-16 01:43:27 UTC (rev 218374)
+++ trunk/Source/WebCore/Modules/mediastream/MediaStream.cpp	2017-06-16 02:49:38 UTC (rev 218375)
@@ -90,7 +90,6 @@
     setIsActive(m_private->active());
     m_private->addObserver(*this);
     MediaStreamRegistry::shared().registerStream(*this);
-    document()->addAudioProducer(this);
     suspendIfNeeded();
 }
 
@@ -111,7 +110,6 @@
         track->addObserver(*this);
         m_trackSet.add(track->id(), WTFMove(track));
     }
-    document()->addAudioProducer(this);
     suspendIfNeeded();
 }
 
@@ -125,7 +123,6 @@
     for (auto& track : m_trackSet.values())
         track->removeObserver(*this);
     if (Document* document = this->document()) {
-        document->removeAudioProducer(this);
         if (m_isWaitingUntilMediaCanStart)
             document->removeMediaCanStartListener(this);
     }
@@ -322,53 +319,16 @@
     }
 }
 
-void MediaStream::pageMutedStateDidChange()
-{
-    if (!m_isActive)
-        return;
-
-    Document* document = this->document();
-    if (!document)
-        return;
-
-    m_private->setCaptureTracksMuted(document->page()->isMediaCaptureMuted());
-}
-
 MediaProducer::MediaStateFlags MediaStream::mediaState() const
 {
-    MediaStateFlags state = IsNotPlaying;
+    MediaProducer::MediaStateFlags state = MediaProducer::IsNotPlaying;
 
     if (!m_isActive || !document() || !document()->page())
         return state;
 
-    bool pageCaptureMuted = document()->page()->isMediaCaptureMuted();
-    for (const auto& track : m_trackSet.values()) {
-        if (!track->isCaptureTrack() || track->ended())
-            continue;
+    for (const auto& track : m_trackSet.values())
+        state |= track->mediaState();
 
-        if (track->source().type() == RealtimeMediaSource::Type::Audio) {
-            if (track->source().interrupted() && !pageCaptureMuted)
-                state |= HasInterruptedAudioCaptureDevice;
-            else if (track->muted())
-                state |= HasMutedAudioCaptureDevice;
-            else if (m_isProducingData && m_private->isProducingData()) {
-                state |= HasActiveAudioCaptureDevice;
-                ASSERT(!track->source().interrupted());
-                ASSERT(!track->muted());
-            }
-        } else {
-            if (track->source().interrupted() && !pageCaptureMuted)
-                state |= HasInterruptedVideoCaptureDevice;
-            else if (track->muted())
-                state |= HasMutedVideoCaptureDevice;
-            else if (m_isProducingData && m_private->isProducingData()) {
-                state |= HasActiveVideoCaptureDevice;
-                ASSERT(!track->source().interrupted());
-                ASSERT(!track->muted());
-            }
-        }
-    }
-
     return state;
 }
 
@@ -379,13 +339,12 @@
     if (Document* document = this->document()) {
         if (m_isActive)
             document->setHasActiveMediaStreamTrack();
-        document->updateIsPlayingMedia();
     }
 }
 
 void MediaStream::characteristicsChanged()
 {
-    MediaStateFlags state = mediaState();
+    auto state = mediaState();
     if (m_state != state) {
         m_state = state;
         statusDidChange();

Modified: trunk/Source/WebCore/Modules/mediastream/MediaStream.h (218374 => 218375)


--- trunk/Source/WebCore/Modules/mediastream/MediaStream.h	2017-06-16 01:43:27 UTC (rev 218374)
+++ trunk/Source/WebCore/Modules/mediastream/MediaStream.h	2017-06-16 02:49:38 UTC (rev 218375)
@@ -54,7 +54,6 @@
     , public ActiveDOMObject
     , public MediaStreamTrack::Observer
     , public MediaStreamPrivate::Observer
-    , private MediaProducer
     , private MediaCanStartListener
     , private PlatformMediaSessionClient
     , public RefCounted<MediaStream> {
@@ -136,9 +135,7 @@
     void didRemoveTrack(MediaStreamTrackPrivate&) final;
     void characteristicsChanged() final;
 
-    // MediaProducer
-    void pageMutedStateDidChange() final;
-    MediaProducer::MediaStateFlags mediaState() const final;
+    MediaProducer::MediaStateFlags mediaState() const;
 
     // MediaCanStartListener
     void mediaCanStart(Document&) final;
@@ -180,7 +177,7 @@
     Vector<Observer*> m_observers;
     std::unique_ptr<PlatformMediaSession> m_mediaSession;
 
-    MediaStateFlags m_state { IsNotPlaying };
+    MediaProducer::MediaStateFlags m_state { MediaProducer::IsNotPlaying };
 
     bool m_isActive { false };
     bool m_isProducingData { false };

Modified: trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp (218374 => 218375)


--- trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp	2017-06-16 01:43:27 UTC (rev 218374)
+++ trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp	2017-06-16 02:49:38 UTC (rev 218375)
@@ -30,6 +30,7 @@
 
 #if ENABLE(MEDIA_STREAM)
 
+#include "Document.h"
 #include "Event.h"
 #include "EventNames.h"
 #include "JSOverconstrainedError.h"
@@ -38,6 +39,7 @@
 #include "MediaStreamPrivate.h"
 #include "NotImplemented.h"
 #include "OverconstrainedError.h"
+#include "Page.h"
 #include "ScriptExecutionContext.h"
 #include <wtf/NeverDestroyed.h>
 
@@ -52,15 +54,22 @@
     : ActiveDOMObject(&context)
     , m_private(WTFMove(privateTrack))
     , m_weakPtrFactory(this)
+    , m_taskQueue(context)
 {
     suspendIfNeeded();
 
     m_private->addObserver(*this);
+
+    if (auto document = this->document())
+        document->addAudioProducer(this);
 }
 
 MediaStreamTrack::~MediaStreamTrack()
 {
     m_private->removeObserver(*this);
+
+    if (auto document = this->document())
+        document->removeAudioProducer(this);
 }
 
 const AtomicString& MediaStreamTrack::kind() const
@@ -128,6 +137,8 @@
 
     m_private->endTrack();
     m_ended = true;
+
+    configureTrackRendering();
 }
 
 MediaStreamTrack::TrackSettings MediaStreamTrack::getSettings() const
@@ -285,6 +296,53 @@
     m_observers.removeFirst(&observer);
 }
 
+void MediaStreamTrack::pageMutedStateDidChange()
+{
+    if (m_ended || !isCaptureTrack())
+        return;
+
+    Document* document = this->document();
+    if (!document || !document->page())
+        return;
+
+    m_private->setMuted(document->page()->isMediaCaptureMuted());
+}
+
+MediaProducer::MediaStateFlags MediaStreamTrack::mediaState() const
+{
+    if (m_ended || !isCaptureTrack())
+        return IsNotPlaying;
+
+    Document* document = this->document();
+    if (!document || !document->page())
+        return IsNotPlaying;
+
+    bool pageCaptureMuted = document->page()->isMediaCaptureMuted();
+
+    if (source().type() == RealtimeMediaSource::Type::Audio) {
+        if (source().interrupted() && !pageCaptureMuted)
+            return HasInterruptedAudioCaptureDevice;
+        if (muted())
+            return HasMutedAudioCaptureDevice;
+        if (m_private->isProducingData())
+            return HasActiveAudioCaptureDevice;
+    } else {
+        if (source().interrupted() && !pageCaptureMuted)
+            return HasInterruptedVideoCaptureDevice;
+        if (muted())
+            return HasMutedVideoCaptureDevice;
+        if (m_private->isProducingData())
+            return HasActiveVideoCaptureDevice;
+    }
+
+    return IsNotPlaying;
+}
+
+void MediaStreamTrack::trackStarted(MediaStreamTrackPrivate&)
+{
+    configureTrackRendering();
+}
+
 void MediaStreamTrack::trackEnded(MediaStreamTrackPrivate&)
 {
     // http://w3c.github.io/mediacapture-main/#life-cycle
@@ -332,6 +390,11 @@
 
 void MediaStreamTrack::configureTrackRendering()
 {
+    m_taskQueue.enqueueTask([this] {
+        if (auto document = this->document())
+            document->updateIsPlayingMedia();
+    });
+
     // 4.3.1
     // ... media from the source only flows when a MediaStreamTrack object is both unmuted and enabled
 }
@@ -339,6 +402,7 @@
 void MediaStreamTrack::stop()
 {
     stopTrack();
+    m_taskQueue.close();
 }
 
 const char* MediaStreamTrack::activeDOMObjectName() const
@@ -361,6 +425,11 @@
     return m_private->audioSourceProvider();
 }
 
+Document* MediaStreamTrack::document() const
+{
+    return downcast<Document>(scriptExecutionContext());
+}
+
 } // namespace WebCore
 
 #endif // ENABLE(MEDIA_STREAM)

Modified: trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.h (218374 => 218375)


--- trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.h	2017-06-16 01:43:27 UTC (rev 218374)
+++ trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.h	2017-06-16 02:49:38 UTC (rev 218375)
@@ -32,8 +32,10 @@
 #include "ActiveDOMObject.h"
 #include "DoubleRange.h"
 #include "EventTarget.h"
+#include "GenericTaskQueue.h"
 #include "JSDOMPromiseDeferred.h"
 #include "LongRange.h"
+#include "MediaProducer.h"
 #include "MediaStreamTrackPrivate.h"
 #include "MediaTrackConstraints.h"
 
@@ -40,10 +42,16 @@
 namespace WebCore {
 
 class AudioSourceProvider;
+class Document;
 
 struct MediaTrackConstraints;
 
-class MediaStreamTrack : public RefCounted<MediaStreamTrack>, public ActiveDOMObject, public EventTargetWithInlineData, private MediaStreamTrackPrivate::Observer {
+class MediaStreamTrack :
+    public RefCounted<MediaStreamTrack>,
+    public ActiveDOMObject,
+    public EventTargetWithInlineData,
+    private MediaProducer,
+    private MediaStreamTrackPrivate::Observer {
 public:
     class Observer {
     public:
@@ -110,11 +118,15 @@
     const MediaTrackConstraints& getConstraints() const { return m_constraints; }
     void applyConstraints(const std::optional<MediaTrackConstraints>&, DOMPromiseDeferred<void>&&);
 
-    RealtimeMediaSource& source() { return m_private->source(); }
+    RealtimeMediaSource& source() const { return m_private->source(); }
     MediaStreamTrackPrivate& privateTrack() { return m_private.get(); }
 
     AudioSourceProvider* audioSourceProvider();
 
+    // MediaProducer
+    void pageMutedStateDidChange() final;
+    MediaProducer::MediaStateFlags mediaState() const final;
+
     void addObserver(Observer&);
     void removeObserver(Observer&);
 
@@ -132,6 +144,8 @@
 
     void configureTrackRendering();
 
+    Document* document() const;
+
     // ActiveDOMObject API.
     void stop() final;
     const char* activeDOMObjectName() const final;
@@ -144,6 +158,7 @@
     ScriptExecutionContext* scriptExecutionContext() const final { return ActiveDOMObject::scriptExecutionContext(); }
 
     // MediaStreamTrackPrivate::Observer
+    void trackStarted(MediaStreamTrackPrivate&) final;
     void trackEnded(MediaStreamTrackPrivate&) final;
     void trackMutedChanged(MediaStreamTrackPrivate&) final;
     void trackSettingsChanged(MediaStreamTrackPrivate&) final;
@@ -157,6 +172,7 @@
     MediaTrackConstraints m_constraints;
     std::optional<DOMPromiseDeferred<void>> m_promise;
     WeakPtrFactory<MediaStreamTrack> m_weakPtrFactory;
+    GenericTaskQueue<ScriptExecutionContext> m_taskQueue;
 
     bool m_ended { false };
 };

Modified: trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h (218374 => 218375)


--- trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h	2017-06-16 01:43:27 UTC (rev 218374)
+++ trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h	2017-06-16 02:49:38 UTC (rev 218375)
@@ -201,6 +201,7 @@
     void didRemoveTrack(MediaStreamTrackPrivate&) override;
 
     // MediaStreamPrivateTrack::Observer
+    void trackStarted(MediaStreamTrackPrivate&) override { };
     void trackEnded(MediaStreamTrackPrivate&) override { };
     void trackMutedChanged(MediaStreamTrackPrivate&) override { };
     void trackSettingsChanged(MediaStreamTrackPrivate&) override { };

Modified: trunk/Source/WebCore/platform/mediastream/MediaStreamPrivate.cpp (218374 => 218375)


--- trunk/Source/WebCore/platform/mediastream/MediaStreamPrivate.cpp	2017-06-16 01:43:27 UTC (rev 218374)
+++ trunk/Source/WebCore/platform/mediastream/MediaStreamPrivate.cpp	2017-06-16 02:49:38 UTC (rev 218375)
@@ -281,6 +281,13 @@
     });
 }
 
+void MediaStreamPrivate::trackStarted(MediaStreamTrackPrivate&)
+{
+    scheduleDeferredTask([this] {
+        characteristicsChanged();
+    });
+}
+
 void MediaStreamPrivate::trackEnded(MediaStreamTrackPrivate&)
 {
     scheduleDeferredTask([this] {

Modified: trunk/Source/WebCore/platform/mediastream/MediaStreamPrivate.h (218374 => 218375)


--- trunk/Source/WebCore/platform/mediastream/MediaStreamPrivate.h	2017-06-16 01:43:27 UTC (rev 218374)
+++ trunk/Source/WebCore/platform/mediastream/MediaStreamPrivate.h	2017-06-16 02:49:38 UTC (rev 218375)
@@ -125,6 +125,7 @@
     MediaStreamPrivate(const MediaStreamTrackPrivateVector&, String&&);
 
     // MediaStreamTrackPrivate::Observer
+    void trackStarted(MediaStreamTrackPrivate&) override;
     void trackEnded(MediaStreamTrackPrivate&) override;
     void trackMutedChanged(MediaStreamTrackPrivate&) override;
     void trackSettingsChanged(MediaStreamTrackPrivate&) override;

Modified: trunk/Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp (218374 => 218375)


--- trunk/Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp	2017-06-16 01:43:27 UTC (rev 218374)
+++ trunk/Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp	2017-06-16 02:49:38 UTC (rev 218375)
@@ -151,6 +151,12 @@
     return m_source->audioSourceProvider();
 }
 
+void MediaStreamTrackPrivate::sourceStarted()
+{
+    for (auto& observer : m_observers)
+        observer->trackStarted(*this);
+}
+
 void MediaStreamTrackPrivate::sourceStopped()
 {
     if (m_isEnded)

Modified: trunk/Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h (218374 => 218375)


--- trunk/Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h	2017-06-16 01:43:27 UTC (rev 218374)
+++ trunk/Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h	2017-06-16 02:49:38 UTC (rev 218375)
@@ -43,6 +43,7 @@
     public:
         virtual ~Observer() { }
 
+        virtual void trackStarted(MediaStreamTrackPrivate&) { };
         virtual void trackEnded(MediaStreamTrackPrivate&) = 0;
         virtual void trackMutedChanged(MediaStreamTrackPrivate&) = 0;
         virtual void trackSettingsChanged(MediaStreamTrackPrivate&) = 0;
@@ -102,6 +103,7 @@
     MediaStreamTrackPrivate(Ref<RealtimeMediaSource>&&, String&& id);
 
     // RealtimeMediaSourceObserver
+    void sourceStarted() final;
     void sourceStopped() final;
     void sourceMutedChanged() final;
     void sourceEnabledChanged() final;

Modified: trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp (218374 => 218375)


--- trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp	2017-06-16 01:43:27 UTC (rev 218374)
+++ trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp	2017-06-16 02:49:38 UTC (rev 218375)
@@ -162,6 +162,9 @@
 
     m_isProducingData = true;
     startProducingData();
+
+    for (Observer& observer : m_observers)
+        observer.sourceStarted();
 }
 
 void RealtimeMediaSource::stop()

Modified: trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.h (218374 => 218375)


--- trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.h	2017-06-16 01:43:27 UTC (rev 218374)
+++ trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.h	2017-06-16 02:49:38 UTC (rev 218375)
@@ -70,6 +70,7 @@
         virtual ~Observer() { }
         
         // Source state changes.
+        virtual void sourceStarted() { }
         virtual void sourceStopped() { }
         virtual void sourceMutedChanged() { }
         virtual void sourceEnabledChanged() { }

Modified: trunk/Tools/ChangeLog (218374 => 218375)


--- trunk/Tools/ChangeLog	2017-06-16 01:43:27 UTC (rev 218374)
+++ trunk/Tools/ChangeLog	2017-06-16 02:49:38 UTC (rev 218375)
@@ -1,3 +1,19 @@
+2017-06-15  Jer Noble  <jer.no...@apple.com>
+
+        [WebRTC] Removing a MediaStreamTrack from a MediaStream reports no recording to WebKit clients
+        https://bugs.webkit.org/show_bug.cgi?id=173398
+        <rdar://problem/32592961>
+
+        Reviewed by Eric Carlson.
+
+        * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
+        * TestWebKitAPI/Tests/WebKit2/MediaStreamTrackDetached.mm: Added.
+        (-[MediaStreamTrackDetachedUIDelegate _webView:requestUserMediaAuthorizationForDevices:url:mainFrameURL:decisionHandler:]):
+        (-[MediaStreamTrackDetachedUIDelegate _webView:checkUserMediaPermissionForURL:mainFrameURL:frameIdentifier:decisionHandler:]):
+        (-[MediaStreamTrackDetachedUIDelegate _webView:mediaCaptureStateDidChange:]):
+        (TestWebKitAPI::TEST):
+        * TestWebKitAPI/Tests/WebKit2/mediastreamtrack-detached.html: Added.
+
 2017-06-15  Matt Lewis  <jlew...@apple.com>
 
         Unreviewed, rolling out r218365.

Modified: trunk/Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj (218374 => 218375)


--- trunk/Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj	2017-06-16 01:43:27 UTC (rev 218374)
+++ trunk/Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj	2017-06-16 02:49:38 UTC (rev 218375)
@@ -601,6 +601,8 @@
 		CDC8E4951BC6F10800594FEC /* video-with-audio.mp4 in Copy Resources */ = {isa = PBXBuildFile; fileRef = CDC8E48A1BC5C96200594FEC /* video-with-audio.mp4 */; };
 		CDC8E4961BC6F10800594FEC /* video-without-audio.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = CDC8E48B1BC5C96200594FEC /* video-without-audio.html */; };
 		CDC8E4971BC6F10800594FEC /* video-without-audio.mp4 in Copy Resources */ = {isa = PBXBuildFile; fileRef = CDC8E48C1BC5C96200594FEC /* video-without-audio.mp4 */; };
+		CDC9442E1EF1FC080059C3C4 /* MediaStreamTrackDetached.mm in Sources */ = {isa = PBXBuildFile; fileRef = CDC9442C1EF1FC080059C3C4 /* MediaStreamTrackDetached.mm */; };
+		CDC9442F1EF205D60059C3C4 /* mediastreamtrack-detached.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = CDC9442B1EF1FBD20059C3C4 /* mediastreamtrack-detached.html */; };
 		CDCFA7AA1E45183200C2433D /* SampleMap.cpp in Sources */ = {isa = PBXBuildFile; fileRef = CDCFA7A91E45122F00C2433D /* SampleMap.cpp */; };
 		CDE195B51CFE0B880053D256 /* FullscreenTopContentInset.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = CDE195B21CFE0ADE0053D256 /* FullscreenTopContentInset.html */; };
 		CE06DF9B1E1851F200E570C9 /* SecurityOrigin.cpp in Sources */ = {isa = PBXBuildFile; fileRef = CE06DF9A1E1851F200E570C9 /* SecurityOrigin.cpp */; };
@@ -709,6 +711,7 @@
 			dstPath = TestWebKitAPI.resources;
 			dstSubfolderSpec = 7;
 			files = (
+				CDC9442F1EF205D60059C3C4 /* mediastreamtrack-detached.html in Copy Resources */,
 				F46849C01EEF5EF300B937FE /* rich-and-plain-text.html in Copy Resources */,
 				93E2D2761ED7D53200FA76F6 /* offscreen-iframe-of-media-document.html in Copy Resources */,
 				F46A095A1ED8A6E600D4AA55 /* apple.gif in Copy Resources */,
@@ -1523,6 +1526,8 @@
 		CDC8E48A1BC5C96200594FEC /* video-with-audio.mp4 */ = {isa = PBXFileReference; lastKnownFileType = file; path = "video-with-audio.mp4"; sourceTree = "<group>"; };
 		CDC8E48B1BC5C96200594FEC /* video-without-audio.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "video-without-audio.html"; sourceTree = "<group>"; };
 		CDC8E48C1BC5C96200594FEC /* video-without-audio.mp4 */ = {isa = PBXFileReference; lastKnownFileType = file; path = "video-without-audio.mp4"; sourceTree = "<group>"; };
+		CDC9442B1EF1FBD20059C3C4 /* mediastreamtrack-detached.html */ = {isa = PBXFileReference; lastKnownFileType = text.html; path = "mediastreamtrack-detached.html"; sourceTree = "<group>"; };
+		CDC9442C1EF1FC080059C3C4 /* MediaStreamTrackDetached.mm */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.objcpp; path = MediaStreamTrackDetached.mm; sourceTree = "<group>"; };
 		CDCFA7A91E45122F00C2433D /* SampleMap.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SampleMap.cpp; sourceTree = "<group>"; };
 		CDE195B21CFE0ADE0053D256 /* FullscreenTopContentInset.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = FullscreenTopContentInset.html; sourceTree = "<group>"; };
 		CDE195B31CFE0ADE0053D256 /* FullscreenTopContentInset.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = FullscreenTopContentInset.mm; sourceTree = "<group>"; };
@@ -2163,6 +2168,7 @@
 				33DC8910141953A300747EF7 /* LoadCanceledNoServerRedirectCallback.cpp */,
 				33DC89131419579F00747EF7 /* LoadCanceledNoServerRedirectCallback_Bundle.cpp */,
 				8AA28C1916D2FA7B002FF4DB /* LoadPageOnCrash.cpp */,
+				CDC9442C1EF1FC080059C3C4 /* MediaStreamTrackDetached.mm */,
 				7A5623101AD5AF3E0096B920 /* MenuTypesForMouseEvents.cpp */,
 				51CB4AD71B3A079C00C1B1C6 /* ModalAlertsSPI.cpp */,
 				33BE5AF4137B5A6C00705813 /* MouseMoveAfterCrash.cpp */,
@@ -2347,6 +2353,7 @@
 				930AD401150698B30067970F /* lots-of-text.html */,
 				5797FE321EB15A8900B2F4A0 /* navigation-client-default-crypto.html */,
 				AD57AC1D1DA7463800FF1BDE /* many-iframes.html */,
+				CDC9442B1EF1FBD20059C3C4 /* mediastreamtrack-detached.html */,
 				51CD1C711B38D48400142CA5 /* modal-alerts-in-new-about-blank-window.html */,
 				7A1458FB1AD5C03500E06772 /* mouse-button-listener.html */,
 				33E79E05137B5FCE00E32D99 /* mouse-move-listener.html */,
@@ -2907,6 +2914,7 @@
 				51D1249B1E785425002B2820 /* CookieManager.cpp in Sources */,
 				7CCE7EAC1A411A3400447C4C /* Counters.cpp in Sources */,
 				7AEAD47F1E20116C00416EFE /* CrossPartitionFileSchemeAccess.mm in Sources */,
+				CDC9442E1EF1FC080059C3C4 /* MediaStreamTrackDetached.mm in Sources */,
 				7CCE7EDB1A411A9200447C4C /* CSSParser.cpp in Sources */,
 				7CCE7F291A411B1000447C4C /* CustomProtocolsInvalidScheme.mm in Sources */,
 				7CCE7F2A1A411B1000447C4C /* CustomProtocolsSyncXHRTest.mm in Sources */,

Added: trunk/Tools/TestWebKitAPI/Tests/WebKit2/MediaStreamTrackDetached.mm (0 => 218375)


--- trunk/Tools/TestWebKitAPI/Tests/WebKit2/MediaStreamTrackDetached.mm	                        (rev 0)
+++ trunk/Tools/TestWebKitAPI/Tests/WebKit2/MediaStreamTrackDetached.mm	2017-06-16 02:49:38 UTC (rev 218375)
@@ -0,0 +1,90 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#import "config.h"
+
+#if WK_API_ENABLED
+
+#if ENABLE(MEDIA_STREAM)
+
+#import "PlatformUtilities.h"
+#import "Test.h"
+#import "TestWKWebView.h"
+#import <WebKit/WKPreferencesPrivate.h>
+#import <WebKit/WKUIDelegatePrivate.h>
+#import <WebKit/WKWebViewConfiguration.h>
+#import <WebKit/_WKProcessPoolConfiguration.h>
+
+static bool hasRecievedCorrectCaptureState = false;
+
+@interface MediaStreamTrackDetachedUIDelegate : NSObject<WKUIDelegate>
+- (void)_webView:(WKWebView *)webView requestUserMediaAuthorizationForDevices:(_WKCaptureDevices)devices url:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL decisionHandler:(void (^)(BOOL authorized))decisionHandler;
+- (void)_webView:(WKWebView *)webView checkUserMediaPermissionForURL:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL frameIdentifier:(NSUInteger)frameIdentifier decisionHandler:(void (^)(NSString *salt, BOOL authorized))decisionHandler;
+- (void)_webView:(WKWebView *)webView mediaCaptureStateDidChange:(_WKMediaCaptureState)state;
+@end
+
+@implementation MediaStreamTrackDetachedUIDelegate
+- (void)_webView:(WKWebView *)webView requestUserMediaAuthorizationForDevices:(_WKCaptureDevices)devices url:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL decisionHandler:(void (^)(BOOL authorized))decisionHandler
+{
+    decisionHandler(YES);
+}
+
+- (void)_webView:(WKWebView *)webView checkUserMediaPermissionForURL:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL frameIdentifier:(NSUInteger)frameIdentifier decisionHandler:(void (^)(NSString *salt, BOOL authorized))decisionHandler
+{
+    decisionHandler(@"0x987654321", YES);
+}
+- (void)_webView:(WKWebView *)webView mediaCaptureStateDidChange:(_WKMediaCaptureState)state
+{
+    if (state == _WKMediaCaptureStateActiveMicrophone)
+        hasRecievedCorrectCaptureState = true;
+}
+@end
+
+namespace TestWebKitAPI {
+
+TEST(WebKit2, MediaStreamTrackDetached)
+{
+    auto configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
+    auto processPoolConfig = adoptNS([[_WKProcessPoolConfiguration alloc] init]);
+    auto preferences = [configuration preferences];
+    preferences._mediaCaptureRequiresSecureConnection = NO;
+    preferences._mediaDevicesEnabled = YES;
+    auto webView = adoptNS([[TestWKWebView alloc] initWithFrame:CGRectMake(0, 0, 320, 500) configuration:configuration.get() processPoolConfiguration:processPoolConfig.get()]);
+    auto delegate = adoptNS([[MediaStreamTrackDetachedUIDelegate alloc] init]);
+    webView.get().UIDelegate = delegate.get();
+
+    hasRecievedCorrectCaptureState = false;
+    [webView loadTestPageNamed:@"mediastreamtrack-detached"];
+
+    TestWebKitAPI::Util::run(&hasRecievedCorrectCaptureState);
+
+
+}
+
+} // namespace TestWebKitAPI
+
+#endif // ENABLE(MEDIA_STREAM)
+
+#endif // WK_API_ENABLED

Added: trunk/Tools/TestWebKitAPI/Tests/WebKit2/mediastreamtrack-detached.html (0 => 218375)


--- trunk/Tools/TestWebKitAPI/Tests/WebKit2/mediastreamtrack-detached.html	                        (rev 0)
+++ trunk/Tools/TestWebKitAPI/Tests/WebKit2/mediastreamtrack-detached.html	2017-06-16 02:49:38 UTC (rev 218375)
@@ -0,0 +1,23 @@
+<!DOCTYPE html>
+<html>
+<head>
+<script>
+var audioTrack;
+
+function go() {
+    var constraints = { audio: true, video: true};
+    navigator.mediaDevices.getUserMedia(constraints).then(gotUserMedia);
+}
+
+function gotUserMedia(stream) {
+    audioTrack = stream.getAudioTracks()[0];
+    stream.removeTrack(audioTrack);
+
+    stream.getTracks().forEach(track => { track.stop(); });
+}
+</script>
+</head>
+<body _onload_="go()">
+    <div>This tests the behavior of the media capture indicator when a live MediaStreamTrack is removed from a MediaStream, and all the other tracks are stopped. The page should request access to the camera and microphone, and then display a audio-capture icon. If the page displays a camera icon or no capture icon at all, the test has failed.</div>
+</body>
+</html>
\ No newline at end of file
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to