Diff
Modified: trunk/ChangeLog (228616 => 228617)
--- trunk/ChangeLog 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/ChangeLog 2018-02-19 08:58:14 UTC (rev 228617)
@@ -1,3 +1,14 @@
+2018-02-19 Philippe Normand <[email protected]>
+
+ [GStreamer] Playbin3 support
+ https://bugs.webkit.org/show_bug.cgi?id=182530
+
+ Reviewed by Xabier Rodriguez-Calvar.
+
+ * Source/cmake/GStreamerDefinitions.cmake: New
+ USE(GSTREAMER_PLAYBIN3) feature. This should be enabled only for
+ very recent versions of GStreamer (1.14 at least) and is optional, for now.
+
2018-02-18 Philippe Normand <[email protected]>
[GTK][WPE] Fullscreen video is broken
Modified: trunk/Source/WebCore/ChangeLog (228616 => 228617)
--- trunk/Source/WebCore/ChangeLog 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/ChangeLog 2018-02-19 08:58:14 UTC (rev 228617)
@@ -1,3 +1,101 @@
+2018-02-19 Philippe Normand <[email protected]>
+
+ [GStreamer] Playbin3 support
+ https://bugs.webkit.org/show_bug.cgi?id=182530
+
+ Reviewed by Xabier Rodriguez-Calvar.
+
+ This patch introduces opt-in support for the playbin3 GStreamer
+ element. This new playback element is still considered
+ experimental but it can still be very useful for media assets
+ containing multiple tracks of the same type. In such scenario
+ audio/video decoders would be created only for the selected tracks
+ whereas playbin2 creates decoder for each track.
+
+ * platform/graphics/gstreamer/AudioTrackPrivateGStreamer.cpp: Take
+ a weak pointer of the media player instead of playbin and call the
+ player when enabling tracks. Also use the GstStream API instead of
+ inspecting the pad when playbin3 is used.
+ (WebCore::AudioTrackPrivateGStreamer::AudioTrackPrivateGStreamer):
+ (WebCore::AudioTrackPrivateGStreamer::disconnect):
+ (WebCore::AudioTrackPrivateGStreamer::markAsActive):
+ (WebCore::AudioTrackPrivateGStreamer::setEnabled):
+ * platform/graphics/gstreamer/AudioTrackPrivateGStreamer.h:
+ * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: GstStream and
+ GstStreamCollection smart pointer support.
+ (WTF::adoptGRef):
+ (WTF::refGPtr<GstStream>):
+ (WTF::derefGPtr<GstStream>):
+ (WTF::refGPtr<GstStreamCollection>):
+ (WTF::derefGPtr<GstStreamCollection>):
+ * platform/graphics/gstreamer/GRefPtrGStreamer.h:
+ * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp:
+ Use the GstStream API instead of inspecting the pad when playbin3
+ is used.
+ (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer):
+ (WebCore::InbandTextTrackPrivateGStreamer::disconnect):
+ * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h:
+ (WebCore::InbandTextTrackPrivateGStreamer::create):
+ * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
+ (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer):
+ m_{audio,video,text}Tracks are now hashmaps, which is a bit more
+ convenient to use than Vectors.
+ (WebCore::MediaPlayerPrivateGStreamer::updateTracks): Update our
+ tracks implementations depending on the streams stored in the
+ collection.
+ (WebCore::MediaPlayerPrivateGStreamer::enableTrack): Activate the
+ given track. This method is called by the TrackPrivate
+ implementations.
+ (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfVideo):
+ Refactored legacy (playbin2) support for track hashmap storage.
+ (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfAudio): Ditto.
+ (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Ditto.
+ (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Use track hashmap storage.
+ (WebCore::MediaPlayerPrivateGStreamer::handleMessage): React on
+ playbin3 GstStream messages, store the collection provided and
+ activate the given selected streams accordingly.
+ (WebCore::MediaPlayerPrivateGStreamer::purgeInvalidAudioTracks): Invalidate tracks no longer needed.
+ (WebCore::MediaPlayerPrivateGStreamer::purgeInvalidVideoTracks): Ditto.
+ (WebCore::MediaPlayerPrivateGStreamer::purgeInvalidTextTracks): Ditto.
+ (WebCore::MediaPlayerPrivateGStreamer::sourceSetupCallback):
+ Refactoring, use source-setup signal instead of the source
+ property which is not supported in playbin3.
+ (WebCore::MediaPlayerPrivateGStreamer::uriDecodeBinElementAddedCallback):
+ Promoted logging messages, GST_TRACE should be used for very
+ frequent and less interesting messages.
+ (WebCore::MediaPlayerPrivateGStreamer::downloadBufferFileCreatedCallback): Ditto.
+ (WebCore::MediaPlayerPrivateGStreamer::sourceSetup): Called by the source-setup signal handler.
+ (WebCore::MediaPlayerPrivateGStreamer::setDownloadBuffering): Debugging message added.
+ (WebCore::MediaPlayerPrivateGStreamer::setPreload): Ditto.
+ (WebCore::MediaPlayerPrivateGStreamer::createGSTPlayBin): Create a
+ playbin3 element if USE(GSTREAMER_PLAYBIN3) was enabled and
+ connect to playbin2 signals otherwise.
+ * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
+ (WebCore::MediaPlayerPrivateGStreamer::createWeakPtr): Promoted to public.
+ * platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h:
+ New StreamCollectionChanged notification type, used the sub-class.
+ * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp:
+ (WebCore::TrackPrivateBaseGStreamer::TrackPrivateBaseGStreamer): Support for GstStream API.
+ (WebCore::TrackPrivateBaseGStreamer::disconnect): Clear GstStream too.
+ (WebCore::TrackPrivateBaseGStreamer::tagsChanged): Get tags from GstStream.
+ (WebCore::TrackPrivateBaseGStreamer::notifyTrackOfActiveChanged): No need to check m_pad twice.
+ (WebCore::TrackPrivateBaseGStreamer::notifyTrackOfTagsChanged): Ditto.
+ * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h:
+ * platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp: Take
+ a weak pointer of the media player instead of playbin and call the
+ player when enabling tracks. Also use the GstStream API instead of
+ inspecting the pad when playbin3 is used.
+ (WebCore::VideoTrackPrivateGStreamer::VideoTrackPrivateGStreamer):
+ (WebCore::VideoTrackPrivateGStreamer::disconnect):
+ (WebCore::VideoTrackPrivateGStreamer::markAsActive):
+ (WebCore::VideoTrackPrivateGStreamer::setSelected):
+ * platform/graphics/gstreamer/VideoTrackPrivateGStreamer.h:
+ * platform/graphics/gstreamer/mse/AppendPipeline.cpp: Pass player pointer to track constructor.
+ (WebCore::AppendPipeline::connectDemuxerSrcPadToAppsink):
+ * platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp: Source-setup signal handler.
+ (WebCore::MediaPlayerPrivateGStreamerMSE::sourceSetup):
+ * platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h:
+
2018-02-19 Fujii Hironori <[email protected]>
REGRESSION(r219298): RELEASE_ASSERT(!m_owningPointerForClose) fails in WebCore::IDBServer::UniqueIDBDatabase::scheduleShutdownForClose
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/AudioTrackPrivateGStreamer.cpp (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/AudioTrackPrivateGStreamer.cpp 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/AudioTrackPrivateGStreamer.cpp 2018-02-19 08:58:14 UTC (rev 228617)
@@ -29,13 +29,14 @@
#include "AudioTrackPrivateGStreamer.h"
+#include "MediaPlayerPrivateGStreamer.h"
#include <glib-object.h>
namespace WebCore {
-AudioTrackPrivateGStreamer::AudioTrackPrivateGStreamer(GRefPtr<GstElement> playbin, gint index, GRefPtr<GstPad> pad)
+AudioTrackPrivateGStreamer::AudioTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstPad> pad)
: TrackPrivateBaseGStreamer(this, index, pad)
- , m_playbin(playbin)
+ , m_player(player)
{
// FIXME: Get a real ID from the tkhd atom.
m_id = "A" + String::number(index);
@@ -42,12 +43,27 @@
notifyTrackOfActiveChanged();
}
+#if USE(GSTREAMER_PLAYBIN3)
+AudioTrackPrivateGStreamer::AudioTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstStream> stream)
+ : TrackPrivateBaseGStreamer(this, index, stream)
+ , m_player(player)
+{
+ m_id = gst_stream_get_stream_id(stream.get());
+ notifyTrackOfActiveChanged();
+}
+#endif
+
void AudioTrackPrivateGStreamer::disconnect()
{
- m_playbin.clear();
+ m_player = nullptr;
TrackPrivateBaseGStreamer::disconnect();
}
+void AudioTrackPrivateGStreamer::markAsActive()
+{
+ AudioTrackPrivate::setEnabled(true);
+}
+
void AudioTrackPrivateGStreamer::setEnabled(bool enabled)
{
if (enabled == this->enabled())
@@ -54,8 +70,8 @@
return;
AudioTrackPrivate::setEnabled(enabled);
- if (enabled && m_playbin)
- g_object_set(m_playbin.get(), "current-audio", m_index, nullptr);
+ if (enabled && m_player)
+ m_player->enableTrack(TrackPrivateBaseGStreamer::TrackType::Audio, m_index);
}
} // namespace WebCore
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/AudioTrackPrivateGStreamer.h (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/AudioTrackPrivateGStreamer.h 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/AudioTrackPrivateGStreamer.h 2018-02-19 08:58:14 UTC (rev 228617)
@@ -23,8 +23,7 @@
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
-#ifndef AudioTrackPrivateGStreamer_h
-#define AudioTrackPrivateGStreamer_h
+#pragma once
#if ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
@@ -31,19 +30,30 @@
#include "AudioTrackPrivate.h"
#include "GRefPtrGStreamer.h"
#include "TrackPrivateBaseGStreamer.h"
+#include <gst/gst.h>
+#include <wtf/WeakPtr.h>
namespace WebCore {
+class MediaPlayerPrivateGStreamer;
class AudioTrackPrivateGStreamer final : public AudioTrackPrivate, public TrackPrivateBaseGStreamer {
public:
- static RefPtr<AudioTrackPrivateGStreamer> create(GRefPtr<GstElement> playbin, gint index, GRefPtr<GstPad> pad)
+ static RefPtr<AudioTrackPrivateGStreamer> create(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstPad> pad)
{
- return adoptRef(*new AudioTrackPrivateGStreamer(playbin, index, pad));
+ return adoptRef(*new AudioTrackPrivateGStreamer(player, index, pad));
}
+#if USE(GSTREAMER_PLAYBIN3)
+ static RefPtr<AudioTrackPrivateGStreamer> create(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstStream> stream)
+ {
+ return adoptRef(*new AudioTrackPrivateGStreamer(player, index, stream));
+ }
+#endif
+
void disconnect() override;
void setEnabled(bool) override;
+ void markAsActive();
void setActive(bool enabled) override { setEnabled(enabled); }
int trackIndex() const override { return m_index; }
@@ -53,14 +63,15 @@
AtomicString language() const override { return m_language; }
private:
- AudioTrackPrivateGStreamer(GRefPtr<GstElement> playbin, gint index, GRefPtr<GstPad>);
+ AudioTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer>, gint index, GRefPtr<GstPad>);
+#if USE(GSTREAMER_PLAYBIN3)
+ AudioTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer>, gint index, GRefPtr<GstStream>);
+#endif
AtomicString m_id;
- GRefPtr<GstElement> m_playbin;
+ WeakPtr<MediaPlayerPrivateGStreamer> m_player;
};
} // namespace WebCore
#endif // ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
-
-#endif // AudioTrackPrivateGStreamer_h
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/GRefPtrGStreamer.cpp (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/GRefPtrGStreamer.cpp 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/GRefPtrGStreamer.cpp 2018-02-19 08:58:14 UTC (rev 228617)
@@ -353,6 +353,48 @@
gst_query_unref(ptr);
}
+#if USE(GSTREAMER_PLAYBIN3)
+template <> GRefPtr<GstStream> adoptGRef(GstStream* ptr)
+{
+ ASSERT(!ptr || !g_object_is_floating(ptr));
+ return GRefPtr<GstStream>(ptr, GRefPtrAdopt);
+}
+
+template <> GstStream* refGPtr<GstStream>(GstStream* ptr)
+{
+ if (ptr)
+ gst_object_ref_sink(GST_OBJECT_CAST(ptr));
+
+ return ptr;
+}
+
+template <> void derefGPtr<GstStream>(GstStream* ptr)
+{
+ if (ptr)
+ gst_object_unref(ptr);
+}
+
+template <> GRefPtr<GstStreamCollection> adoptGRef(GstStreamCollection* ptr)
+{
+ ASSERT(!ptr || !g_object_is_floating(ptr));
+ return GRefPtr<GstStreamCollection>(ptr, GRefPtrAdopt);
+}
+
+template <> GstStreamCollection* refGPtr<GstStreamCollection>(GstStreamCollection* ptr)
+{
+ if (ptr)
+ gst_object_ref_sink(GST_OBJECT_CAST(ptr));
+
+ return ptr;
+}
+
+template <> void derefGPtr<GstStreamCollection>(GstStreamCollection* ptr)
+{
+ if (ptr)
+ gst_object_unref(ptr);
+}
+#endif
+
template <> GRefPtr<WebKitVideoSink> adoptGRef(WebKitVideoSink* ptr)
{
ASSERT(!ptr || !g_object_is_floating(ptr));
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/GRefPtrGStreamer.h (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/GRefPtrGStreamer.h 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/GRefPtrGStreamer.h 2018-02-19 08:58:14 UTC (rev 228617)
@@ -17,8 +17,8 @@
* Boston, MA 02110-1301, USA.
*/
-#ifndef GRefPtrGStreamer_h
-#define GRefPtrGStreamer_h
+#pragma once
+
#if USE(GSTREAMER)
#include <wtf/glib/GRefPtr.h>
@@ -43,6 +43,11 @@
typedef struct _WebKitVideoSink WebKitVideoSink;
typedef struct _WebKitWebSrc WebKitWebSrc;
+#if USE(GSTREAMER_PLAYBIN3)
+typedef struct _GstStream GstStream;
+typedef struct _GstStreamCollection GstStreamCollection;
+#endif
+
#if USE(GSTREAMER_GL)
typedef struct _GstGLDisplay GstGLDisplay;
typedef struct _GstGLContext GstGLContext;
@@ -127,6 +132,16 @@
template<> WebKitWebSrc* refGPtr<WebKitWebSrc>(WebKitWebSrc* ptr);
template<> void derefGPtr<WebKitWebSrc>(WebKitWebSrc* ptr);
+#if USE(GSTREAMER_PLAYBIN3)
+template<> GRefPtr<GstStream> adoptGRef(GstStream*);
+template<> GstStream* refGPtr<GstStream>(GstStream*);
+template<> void derefGPtr<GstStream>(GstStream*);
+
+template<> GRefPtr<GstStreamCollection> adoptGRef(GstStreamCollection*);
+template<> GstStreamCollection* refGPtr<GstStreamCollection>(GstStreamCollection*);
+template<> void derefGPtr<GstStreamCollection>(GstStreamCollection*);
+#endif
+
#if USE(GSTREAMER_GL)
template<> GRefPtr<GstGLDisplay> adoptGRef(GstGLDisplay* ptr);
template<> GstGLDisplay* refGPtr<GstGLDisplay>(GstGLDisplay* ptr);
@@ -140,5 +155,3 @@
} // namespace WTF
#endif // USE(GSTREAMER)
-
-#endif // GRefPtrGStreamer_h
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp 2018-02-19 08:58:14 UTC (rev 228617)
@@ -40,7 +40,8 @@
namespace WebCore {
InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer(gint index, GRefPtr<GstPad> pad)
- : InbandTextTrackPrivate(WebVTT), TrackPrivateBaseGStreamer(this, index, pad)
+ : InbandTextTrackPrivate(WebVTT)
+ , TrackPrivateBaseGStreamer(this, index, pad)
{
m_eventProbe = gst_pad_add_probe(m_pad.get(), GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM, [] (GstPad*, GstPadProbeInfo* info, gpointer userData) -> GstPadProbeReturn {
auto* track = static_cast<InbandTextTrackPrivateGStreamer*>(userData);
@@ -57,13 +58,21 @@
notifyTrackOfStreamChanged();
}
+#if USE(GSTREAMER_PLAYBIN3)
+InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer(gint index, GRefPtr<GstStream> stream)
+ : InbandTextTrackPrivate(WebVTT)
+ , TrackPrivateBaseGStreamer(this, index, stream)
+{
+ m_streamId = gst_stream_get_stream_id(stream.get());
+ GST_INFO("Track %d got stream start for stream %s.", m_index, m_streamId.utf8().data());
+}
+#endif
+
void InbandTextTrackPrivateGStreamer::disconnect()
{
- if (!m_pad)
- return;
+ if (m_pad)
+ gst_pad_remove_probe(m_pad.get(), m_eventProbe);
- gst_pad_remove_probe(m_pad.get(), m_eventProbe);
-
TrackPrivateBaseGStreamer::disconnect();
}
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h 2018-02-19 08:58:14 UTC (rev 228617)
@@ -23,8 +23,7 @@
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
-#ifndef InbandTextTrackPrivateGStreamer_h
-#define InbandTextTrackPrivateGStreamer_h
+#pragma once
#if ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
@@ -31,12 +30,12 @@
#include "GRefPtrGStreamer.h"
#include "InbandTextTrackPrivate.h"
#include "TrackPrivateBaseGStreamer.h"
+#include <gst/gst.h>
#include <wtf/Lock.h>
namespace WebCore {
class MediaPlayerPrivateGStreamer;
-typedef struct _GstSample GstSample;
class InbandTextTrackPrivateGStreamer : public InbandTextTrackPrivate, public TrackPrivateBaseGStreamer {
public:
@@ -45,6 +44,13 @@
return adoptRef(*new InbandTextTrackPrivateGStreamer(index, pad));
}
+#if USE(GSTREAMER_PLAYBIN3)
+ static Ref<InbandTextTrackPrivateGStreamer> create(gint index, GRefPtr<GstStream> stream)
+ {
+ return adoptRef(*new InbandTextTrackPrivateGStreamer(index, stream));
+ }
+#endif
+
void disconnect() override;
AtomicString label() const override { return m_label; }
@@ -57,6 +63,9 @@
private:
InbandTextTrackPrivateGStreamer(gint index, GRefPtr<GstPad>);
+#if USE(GSTREAMER_PLAYBIN3)
+ InbandTextTrackPrivateGStreamer(gint index, GRefPtr<GstStream>);
+#endif
void streamChanged();
@@ -64,7 +73,7 @@
void notifyTrackOfStreamChanged();
gulong m_eventProbe;
- Vector<GRefPtr<GstSample> > m_pendingSamples;
+ Vector<GRefPtr<GstSample>> m_pendingSamples;
String m_streamId;
Lock m_sampleMutex;
};
@@ -72,5 +81,3 @@
} // namespace WebCore
#endif // ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
-
-#endif // InbandTextTrackPrivateGStreamer_h
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp 2018-02-19 08:58:14 UTC (rev 228617)
@@ -166,14 +166,14 @@
MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer()
{
#if ENABLE(VIDEO_TRACK)
- for (size_t i = 0; i < m_audioTracks.size(); ++i)
- m_audioTracks[i]->disconnect();
+ for (auto& track : m_audioTracks.values())
+ track->disconnect();
- for (size_t i = 0; i < m_textTracks.size(); ++i)
- m_textTracks[i]->disconnect();
+ for (auto& track : m_textTracks.values())
+ track->disconnect();
- for (size_t i = 0; i < m_videoTracks.size(); ++i)
- m_videoTracks[i]->disconnect();
+ for (auto& track : m_videoTracks.values())
+ track->disconnect();
#endif
if (m_fillTimer.isActive())
m_fillTimer.stop();
@@ -612,6 +612,158 @@
return m_seeking;
}
+#if USE(GSTREAMER_PLAYBIN3)
+void MediaPlayerPrivateGStreamer::updateTracks()
+{
+ ASSERT(!m_isLegacyPlaybin);
+
+ bool useMediaSource = isMediaSource();
+ unsigned length = gst_stream_collection_get_size(m_streamCollection.get());
+ Vector<String> validAudioStreams;
+ Vector<String> validVideoStreams;
+ Vector<String> validTextStreams;
+ for (unsigned i = 0; i < length; i++) {
+ GRefPtr<GstStream> stream = gst_stream_collection_get_stream(m_streamCollection.get(), i);
+ String streamId(gst_stream_get_stream_id(stream.get()));
+ GstStreamType type = gst_stream_get_stream_type(stream.get());
+ GST_DEBUG("Inspecting %s track with ID %s", gst_stream_type_get_name(type), streamId.utf8().data());
+ if (type & GST_STREAM_TYPE_AUDIO) {
+ validAudioStreams.append(streamId);
+#if ENABLE(VIDEO_TRACK)
+ if (!useMediaSource) {
+ unsigned localIndex = i - validVideoStreams.size() - validTextStreams.size();
+ if (localIndex < m_audioTracks.size()) {
+ if (m_audioTracks.contains(streamId))
+ continue;
+ }
+
+ RefPtr<AudioTrackPrivateGStreamer> track = AudioTrackPrivateGStreamer::create(createWeakPtr(), i, stream);
+ m_audioTracks.add(track->id(), track);
+ m_player->addAudioTrack(*track);
+ }
+#endif
+ } else if (type & GST_STREAM_TYPE_VIDEO) {
+ validVideoStreams.append(streamId);
+#if ENABLE(VIDEO_TRACK)
+ if (!useMediaSource) {
+ unsigned localIndex = i - validAudioStreams.size() - validTextStreams.size();
+ if (localIndex < m_videoTracks.size()) {
+ if (m_videoTracks.contains(streamId))
+ continue;
+ }
+
+ RefPtr<VideoTrackPrivateGStreamer> track = VideoTrackPrivateGStreamer::create(createWeakPtr(), i, stream);
+ m_videoTracks.add(track->id(), track);
+ m_player->addVideoTrack(*track);
+ }
+#endif
+ } else if (type & GST_STREAM_TYPE_TEXT) {
+ validTextStreams.append(streamId);
+#if ENABLE(VIDEO_TRACK)
+ if (!useMediaSource) {
+ unsigned localIndex = i - validVideoStreams.size() - validAudioStreams.size();
+ if (localIndex < m_textTracks.size()) {
+ if (m_textTracks.contains(streamId))
+ continue;
+ }
+
+ RefPtr<InbandTextTrackPrivateGStreamer> track = InbandTextTrackPrivateGStreamer::create(localIndex, stream);
+ m_textTracks.add(streamId, track);
+ m_player->addTextTrack(*track);
+ }
+#endif
+ } else
+ GST_WARNING("Unknown track type found for stream %s", streamId.utf8().data());
+ }
+
+ GST_INFO("Media has %u video tracks, %u audio tracks and %u text tracks", validVideoStreams.size(), validAudioStreams.size(), validTextStreams.size());
+
+ bool oldHasAudio = m_hasAudio;
+ bool oldHasVideo = m_hasVideo;
+ m_hasAudio = !validAudioStreams.isEmpty();
+ m_hasVideo = !validVideoStreams.isEmpty();
+ if ((oldHasVideo != m_hasVideo) || (oldHasAudio != m_hasAudio))
+ m_player->characteristicChanged();
+
+ if (m_hasVideo)
+ m_player->sizeChanged();
+
+ if (useMediaSource) {
+ GST_DEBUG("Tracks managed by source element. Bailing out now.");
+ m_player->client().mediaPlayerEngineUpdated(m_player);
+ return;
+ }
+
+#if ENABLE(VIDEO_TRACK)
+ purgeInvalidAudioTracks(validAudioStreams);
+ purgeInvalidVideoTracks(validVideoStreams);
+ purgeInvalidTextTracks(validTextStreams);
+#endif
+
+ m_player->client().mediaPlayerEngineUpdated(m_player);
+}
+#endif
+
+void MediaPlayerPrivateGStreamer::enableTrack(TrackPrivateBaseGStreamer::TrackType trackType, unsigned index)
+{
+ const char* propertyName;
+ const char* trackTypeAsString;
+ GList* selectedStreams = nullptr;
+
+ switch (trackType) {
+ case TrackPrivateBaseGStreamer::TrackType::Audio:
+ propertyName = "current-audio";
+ trackTypeAsString = "audio";
+ if (!m_currentTextStreamId.isEmpty())
+ selectedStreams = g_list_append(selectedStreams, g_strdup(m_currentTextStreamId.utf8().data()));
+ if (!m_currentVideoStreamId.isEmpty())
+ selectedStreams = g_list_append(selectedStreams, g_strdup(m_currentVideoStreamId.utf8().data()));
+ break;
+ case TrackPrivateBaseGStreamer::TrackType::Video:
+ propertyName = "current-video";
+ trackTypeAsString = "video";
+ if (!m_currentAudioStreamId.isEmpty())
+ selectedStreams = g_list_append(selectedStreams, g_strdup(m_currentAudioStreamId.utf8().data()));
+ if (!m_currentTextStreamId.isEmpty())
+ selectedStreams = g_list_append(selectedStreams, g_strdup(m_currentTextStreamId.utf8().data()));
+ break;
+ case TrackPrivateBaseGStreamer::TrackType::Text:
+ propertyName = "current-text";
+ trackTypeAsString = "text";
+ if (!m_currentAudioStreamId.isEmpty())
+ selectedStreams = g_list_append(selectedStreams, g_strdup(m_currentAudioStreamId.utf8().data()));
+ if (!m_currentVideoStreamId.isEmpty())
+ selectedStreams = g_list_append(selectedStreams, g_strdup(m_currentVideoStreamId.utf8().data()));
+ break;
+ case TrackPrivateBaseGStreamer::TrackType::Unknown:
+ default:
+ ASSERT_NOT_REACHED();
+ }
+
+ GST_INFO("Enabling %s track with index: %lu", trackTypeAsString, index);
+ // FIXME: Remove isMediaSource() test below when fixing https://bugs.webkit.org/show_bug.cgi?id=182531
+ if (m_isLegacyPlaybin || isMediaSource()) {
+ GstElement* element = isMediaSource() ? m_source.get() : m_pipeline.get();
+ g_object_set(element, propertyName, index, nullptr);
+ }
+#if USE(GSTREAMER_PLAYBIN3)
+ else {
+ GstStream* stream = gst_stream_collection_get_stream(m_streamCollection.get(), index);
+ if (stream) {
+ String streamId = gst_stream_get_stream_id(stream);
+ selectedStreams = g_list_append(selectedStreams, g_strdup(streamId.utf8().data()));
+ } else
+ GST_WARNING("%s stream %lu not found", trackTypeAsString, index);
+
+ // TODO: MSE GstStream API support: https://bugs.webkit.org/show_bug.cgi?id=182531
+ gst_element_send_event(m_pipeline.get(), gst_event_new_select_streams(selectedStreams));
+ }
+#endif
+
+ if (selectedStreams)
+ g_list_free_full(selectedStreams, reinterpret_cast<GDestroyNotify>(g_free));
+}
+
void MediaPlayerPrivateGStreamer::videoChangedCallback(MediaPlayerPrivateGStreamer* player)
{
player->m_notifier->notify(MainThreadNotification::VideoChanged, [player] { player->notifyPlayerOfVideo(); });
@@ -622,6 +774,8 @@
if (UNLIKELY(!m_pipeline || !m_source))
return;
+ ASSERT(m_isLegacyPlaybin || isMediaSource());
+
gint numTracks = 0;
bool useMediaSource = isMediaSource();
GstElement* element = useMediaSource ? m_source.get() : m_pipeline.get();
@@ -644,29 +798,30 @@
}
#if ENABLE(VIDEO_TRACK)
+ Vector<String> validVideoStreams;
for (gint i = 0; i < numTracks; ++i) {
GRefPtr<GstPad> pad;
g_signal_emit_by_name(m_pipeline.get(), "get-video-pad", i, &pad.outPtr(), nullptr);
ASSERT(pad);
+ String streamId = "V" + String::number(i);
+ validVideoStreams.append(streamId);
if (i < static_cast<gint>(m_videoTracks.size())) {
- RefPtr<VideoTrackPrivateGStreamer> existingTrack = m_videoTracks[i];
- existingTrack->setIndex(i);
- if (existingTrack->pad() == pad)
- continue;
+ RefPtr<VideoTrackPrivateGStreamer> existingTrack = m_videoTracks.get(streamId);
+ if (existingTrack) {
+ existingTrack->setIndex(i);
+ if (existingTrack->pad() == pad)
+ continue;
+ }
}
- RefPtr<VideoTrackPrivateGStreamer> track = VideoTrackPrivateGStreamer::create(m_pipeline, i, pad);
- m_videoTracks.append(track);
+ RefPtr<VideoTrackPrivateGStreamer> track = VideoTrackPrivateGStreamer::create(createWeakPtr(), i, pad);
+ ASSERT(streamId == track->id());
+ m_videoTracks.add(streamId, track);
m_player->addVideoTrack(*track);
}
- while (static_cast<gint>(m_videoTracks.size()) > numTracks) {
- RefPtr<VideoTrackPrivateGStreamer> track = m_videoTracks.last();
- track->disconnect();
- m_videoTracks.removeLast();
- m_player->removeVideoTrack(*track);
- }
+ purgeInvalidVideoTracks(validVideoStreams);
#endif
m_player->client().mediaPlayerEngineUpdated(m_player);
@@ -693,6 +848,8 @@
if (UNLIKELY(!m_pipeline || !m_source))
return;
+ ASSERT(m_isLegacyPlaybin || isMediaSource());
+
gint numTracks = 0;
bool useMediaSource = isMediaSource();
GstElement* element = useMediaSource ? m_source.get() : m_pipeline.get();
@@ -711,29 +868,30 @@
}
#if ENABLE(VIDEO_TRACK)
+ Vector<String> validAudioStreams;
for (gint i = 0; i < numTracks; ++i) {
GRefPtr<GstPad> pad;
g_signal_emit_by_name(m_pipeline.get(), "get-audio-pad", i, &pad.outPtr(), nullptr);
ASSERT(pad);
+ String streamId = "A" + String::number(i);
+ validAudioStreams.append(streamId);
if (i < static_cast<gint>(m_audioTracks.size())) {
- RefPtr<AudioTrackPrivateGStreamer> existingTrack = m_audioTracks[i];
- existingTrack->setIndex(i);
- if (existingTrack->pad() == pad)
- continue;
+ RefPtr<AudioTrackPrivateGStreamer> existingTrack = m_audioTracks.get(streamId);
+ if (existingTrack) {
+ existingTrack->setIndex(i);
+ if (existingTrack->pad() == pad)
+ continue;
+ }
}
- RefPtr<AudioTrackPrivateGStreamer> track = AudioTrackPrivateGStreamer::create(m_pipeline, i, pad);
- m_audioTracks.insert(i, track);
+ RefPtr<AudioTrackPrivateGStreamer> track = AudioTrackPrivateGStreamer::create(createWeakPtr(), i, pad);
+ ASSERT(streamId == track->id());
+ m_audioTracks.add(streamId, track);
m_player->addAudioTrack(*track);
}
- while (static_cast<gint>(m_audioTracks.size()) > numTracks) {
- RefPtr<AudioTrackPrivateGStreamer> track = m_audioTracks.last();
- track->disconnect();
- m_audioTracks.removeLast();
- m_player->removeAudioTrack(*track);
- }
+ purgeInvalidAudioTracks(validAudioStreams);
#endif
m_player->client().mediaPlayerEngineUpdated(m_player);
@@ -750,6 +908,8 @@
if (UNLIKELY(!m_pipeline || !m_source))
return;
+ ASSERT(m_isLegacyPlaybin || isMediaSource());
+
gint numTracks = 0;
bool useMediaSource = isMediaSource();
GstElement* element = useMediaSource ? m_source.get() : m_pipeline.get();
@@ -760,29 +920,35 @@
return;
}
+ Vector<String> validTextStreams;
for (gint i = 0; i < numTracks; ++i) {
GRefPtr<GstPad> pad;
g_signal_emit_by_name(m_pipeline.get(), "get-text-pad", i, &pad.outPtr(), nullptr);
ASSERT(pad);
+ GRefPtr<GstEvent> event = adoptGRef(gst_pad_get_sticky_event(pad.get(), GST_EVENT_STREAM_START, 0));
+ if (!event)
+ continue;
+
+ const char* streamId;
+ gst_event_parse_stream_start(event.get(), &streamId);
+
+ validTextStreams.append(streamId);
if (i < static_cast<gint>(m_textTracks.size())) {
- RefPtr<InbandTextTrackPrivateGStreamer> existingTrack = m_textTracks[i];
- existingTrack->setIndex(i);
- if (existingTrack->pad() == pad)
- continue;
+ RefPtr<InbandTextTrackPrivateGStreamer> existingTrack = m_textTracks.get(streamId);
+ if (existingTrack) {
+ existingTrack->setIndex(i);
+ if (existingTrack->pad() == pad)
+ continue;
+ }
}
RefPtr<InbandTextTrackPrivateGStreamer> track = InbandTextTrackPrivateGStreamer::create(i, pad);
- m_textTracks.insert(i, track);
+ m_textTracks.add(streamId, track);
m_player->addTextTrack(*track);
}
- while (static_cast<gint>(m_textTracks.size()) > numTracks) {
- RefPtr<InbandTextTrackPrivateGStreamer> track = m_textTracks.last();
- track->disconnect();
- m_textTracks.removeLast();
- m_player->removeTextTrack(*track);
- }
+ purgeInvalidTextTracks(validTextStreams);
}
GstFlowReturn MediaPlayerPrivateGStreamer::newTextSampleCallback(MediaPlayerPrivateGStreamer* player)
@@ -808,8 +974,8 @@
const gchar* id;
gst_event_parse_stream_start(streamStartEvent.get(), &id);
for (size_t i = 0; i < m_textTracks.size(); ++i) {
- RefPtr<InbandTextTrackPrivateGStreamer> track = m_textTracks[i];
- if (track->streamId() == id) {
+ RefPtr<InbandTextTrackPrivateGStreamer> track = m_textTracks.get(id);
+ if (track) {
track->handleSample(sample);
found = true;
break;
@@ -1097,6 +1263,61 @@
gst_tag_list_unref(tags);
break;
}
+#if USE(GSTREAMER_PLAYBIN3)
+ case GST_MESSAGE_STREAM_COLLECTION: {
+ GRefPtr<GstStreamCollection> collection;
+ gst_message_parse_stream_collection(message, &collection.outPtr());
+
+ if (collection) {
+ m_streamCollection.swap(collection);
+ m_notifier->notify(MainThreadNotification::StreamCollectionChanged, [this] {
+ this->updateTracks();
+ });
+ }
+ break;
+ }
+ case GST_MESSAGE_STREAMS_SELECTED: {
+ GRefPtr<GstStreamCollection> collection;
+ gst_message_parse_streams_selected(message, &collection.outPtr());
+
+ if (!collection)
+ break;
+
+ m_streamCollection.swap(collection);
+ m_currentAudioStreamId = "";
+ m_currentVideoStreamId = "";
+ m_currentTextStreamId = "";
+
+ unsigned length = gst_message_streams_selected_get_size(message);
+ for (unsigned i = 0; i < length; i++) {
+ GRefPtr<GstStream> stream = adoptGRef(gst_message_streams_selected_get_stream(message, i));
+ if (!stream)
+ continue;
+ GstStreamType type = gst_stream_get_stream_type(stream.get());
+ String streamId(gst_stream_get_stream_id(stream.get()));
+
+ GST_DEBUG("Selecting %s track with ID: %s", gst_stream_type_get_name(type), streamId.utf8().data());
+ // Playbin3 can send more than one selected stream of the same type
+ // but there's no priority or ordering system in place, so we assume
+ // the selected stream is the last one as reported by playbin3.
+ if (type & GST_STREAM_TYPE_AUDIO) {
+ m_currentAudioStreamId = streamId;
+ auto track = m_audioTracks.get(m_currentAudioStreamId);
+ ASSERT(track);
+ track->markAsActive();
+ } else if (type & GST_STREAM_TYPE_VIDEO) {
+ m_currentVideoStreamId = streamId;
+ auto track = m_videoTracks.get(m_currentVideoStreamId);
+ ASSERT(track);
+ track->markAsActive();
+ } else if (type & GST_STREAM_TYPE_TEXT)
+ m_currentTextStreamId = streamId;
+ else
+ GST_WARNING("Unknown stream type with stream-id %s", streamId);
+ }
+ break;
+ }
+#endif
default:
GST_DEBUG("Unhandled GStreamer message type: %s", GST_MESSAGE_TYPE_NAME(message));
break;
@@ -1213,6 +1434,42 @@
for (GList* i = gst_toc_entry_get_sub_entries(entry); i; i = i->next)
processTableOfContentsEntry(static_cast<GstTocEntry*>(i->data));
}
+
+void MediaPlayerPrivateGStreamer::purgeInvalidAudioTracks(Vector<String> validTrackIds)
+{
+ for (auto audioTrackId : m_audioTracks.keys()) {
+ if (validTrackIds.contains(audioTrackId))
+ continue;
+ auto track = m_audioTracks.get(audioTrackId);
+ track->disconnect();
+ m_player->removeAudioTrack(*track);
+ m_audioTracks.remove(audioTrackId);
+ }
+}
+
+void MediaPlayerPrivateGStreamer::purgeInvalidVideoTracks(Vector<String> validTrackIds)
+{
+ for (auto videoTrackId : m_videoTracks.keys()) {
+ if (validTrackIds.contains(videoTrackId))
+ continue;
+ auto track = m_videoTracks.get(videoTrackId);
+ track->disconnect();
+ m_player->removeVideoTrack(*track);
+ m_videoTracks.remove(videoTrackId);
+ }
+}
+
+void MediaPlayerPrivateGStreamer::purgeInvalidTextTracks(Vector<String> validTrackIds)
+{
+ for (auto textTrackId : m_textTracks.keys()) {
+ if (validTrackIds.contains(textTrackId))
+ continue;
+ auto track = m_textTracks.get(textTrackId);
+ track->disconnect();
+ m_player->removeTextTrack(*track);
+ m_textTracks.remove(textTrackId);
+ }
+}
#endif
static int findHLSQueue(const GValue* item)
@@ -1400,9 +1657,9 @@
return m_totalBytes;
}
-void MediaPlayerPrivateGStreamer::sourceChangedCallback(MediaPlayerPrivateGStreamer* player)
+void MediaPlayerPrivateGStreamer::sourceSetupCallback(MediaPlayerPrivateGStreamer* player, GstElement* sourceElement)
{
- player->sourceChanged();
+ player->sourceSetup(sourceElement);
}
void MediaPlayerPrivateGStreamer::uriDecodeBinElementAddedCallback(GstBin* bin, GstElement* element, MediaPlayerPrivateGStreamer* player)
@@ -1419,7 +1676,7 @@
GUniquePtr<char> newDownloadTemplate(g_build_filename(G_DIR_SEPARATOR_S, "var", "tmp", "WebKit-Media-XXXXXX", nullptr));
g_object_set(element, "temp-template", newDownloadTemplate.get(), nullptr);
- GST_TRACE("Reconfigured file download template from '%s' to '%s'", oldDownloadTemplate.get(), newDownloadTemplate.get());
+ GST_DEBUG("Reconfigured file download template from '%s' to '%s'", oldDownloadTemplate.get(), newDownloadTemplate.get());
player->purgeOldDownloadFiles(oldDownloadTemplate.get());
}
@@ -1439,7 +1696,7 @@
return;
}
- GST_TRACE("Unlinked media temporary file %s after creation", downloadFile.get());
+ GST_DEBUG("Unlinked media temporary file %s after creation", downloadFile.get());
}
void MediaPlayerPrivateGStreamer::purgeOldDownloadFiles(const char* downloadFileTemplate)
@@ -1461,13 +1718,14 @@
}
}
-void MediaPlayerPrivateGStreamer::sourceChanged()
+void MediaPlayerPrivateGStreamer::sourceSetup(GstElement* sourceElement)
{
+ GST_DEBUG("Source element set-up for %s", GST_ELEMENT_NAME(sourceElement));
+
if (WEBKIT_IS_WEB_SRC(m_source.get()) && GST_OBJECT_PARENT(m_source.get()))
g_signal_handlers_disconnect_by_func(GST_ELEMENT_PARENT(m_source.get()), reinterpret_cast<gpointer>(uriDecodeBinElementAddedCallback), this);
- m_source.clear();
- g_object_get(m_pipeline.get(), "source", &m_source.outPtr(), nullptr);
+ m_source = sourceElement;
if (WEBKIT_IS_WEB_SRC(m_source.get())) {
webKitWebSrcSetMediaPlayer(WEBKIT_WEB_SRC(m_source.get()), m_player);
@@ -2019,8 +2277,10 @@
unsigned flagDownload = getGstPlayFlag("download");
// We don't want to stop downloading if we already started it.
- if (flags & flagDownload && m_readyState > MediaPlayer::HaveNothing && !m_resetPipeline)
+ if (flags & flagDownload && m_readyState > MediaPlayer::HaveNothing && !m_resetPipeline) {
+ GST_DEBUG("Download already started, not starting again");
return;
+ }
bool shouldDownload = !isLiveStream() && m_preload == MediaPlayer::Auto;
if (shouldDownload) {
@@ -2036,6 +2296,7 @@
void MediaPlayerPrivateGStreamer::setPreload(MediaPlayer::Preload preload)
{
+ GST_DEBUG("Setting preload to %s", convertEnumerationToString(preload).utf8().data());
if (preload == MediaPlayer::Auto && isLiveStream())
return;
@@ -2140,9 +2401,17 @@
// gst_element_factory_make() returns a floating reference so
// we should not adopt.
+#if USE(GSTREAMER_PLAYBIN3)
+ m_isLegacyPlaybin = false;
+ setPipeline(gst_element_factory_make("playbin3", "play"));
+#else
+ m_isLegacyPlaybin = true;
setPipeline(gst_element_factory_make("playbin", "play"));
+#endif
setStreamVolumeElement(GST_STREAM_VOLUME(m_pipeline.get()));
+ GST_INFO("Using legacy playbin element: %s", boolForPrinting(m_isLegacyPlaybin));
+
// Let also other listeners subscribe to (application) messages in this bus.
GRefPtr<GstBus> bus = adoptGRef(gst_pipeline_get_bus(GST_PIPELINE(m_pipeline.get())));
gst_bus_add_signal_watch_full(bus.get(), RunLoopSourcePriority::RunLoopDispatcher);
@@ -2150,11 +2419,15 @@
g_object_set(m_pipeline.get(), "mute", m_player->muted(), nullptr);
- g_signal_connect_swapped(m_pipeline.get(), "notify::source", G_CALLBACK(sourceChangedCallback), this);
- g_signal_connect_swapped(m_pipeline.get(), "video-changed", G_CALLBACK(videoChangedCallback), this);
- g_signal_connect_swapped(m_pipeline.get(), "audio-changed", G_CALLBACK(audioChangedCallback), this);
+ g_signal_connect_swapped(m_pipeline.get(), "source-setup", G_CALLBACK(sourceSetupCallback), this);
+ if (m_isLegacyPlaybin) {
+ g_signal_connect_swapped(m_pipeline.get(), "video-changed", G_CALLBACK(videoChangedCallback), this);
+ g_signal_connect_swapped(m_pipeline.get(), "audio-changed", G_CALLBACK(audioChangedCallback), this);
+ }
+
#if ENABLE(VIDEO_TRACK)
- g_signal_connect_swapped(m_pipeline.get(), "text-changed", G_CALLBACK(textChangedCallback), this);
+ if (m_isLegacyPlaybin)
+ g_signal_connect_swapped(m_pipeline.get(), "text-changed", G_CALLBACK(textChangedCallback), this);
GstElement* textCombiner = webkitTextCombinerNew();
ASSERT(textCombiner);
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h 2018-02-19 08:58:14 UTC (rev 228617)
@@ -22,8 +22,8 @@
* Boston, MA 02110-1301, USA.
*/
-#ifndef MediaPlayerPrivateGStreamer_h
-#define MediaPlayerPrivateGStreamer_h
+#pragma once
+
#if ENABLE(VIDEO) && USE(GSTREAMER)
#include "GRefPtrGStreamer.h"
@@ -36,13 +36,11 @@
#include <wtf/RunLoop.h>
#include <wtf/WeakPtr.h>
-#if ENABLE(VIDEO_TRACK) && USE(GSTREAMER_MPEGTS)
+#if ENABLE(VIDEO_TRACK)
+#include "TrackPrivateBaseGStreamer.h"
#include <wtf/text/AtomicStringHash.h>
#endif
-typedef struct _GstBuffer GstBuffer;
-typedef struct _GstMessage GstMessage;
-typedef struct _GstElement GstElement;
typedef struct _GstMpegtsSection GstMpegtsSection;
namespace WebCore {
@@ -67,6 +65,8 @@
explicit MediaPlayerPrivateGStreamer(MediaPlayer*);
virtual ~MediaPlayerPrivateGStreamer();
+ WeakPtr<MediaPlayerPrivateGStreamer> createWeakPtr() { return m_weakPtrFactory.createWeakPtr(*this); }
+
static void registerMediaEngine(MediaEngineRegistrar);
void handleMessage(GstMessage*);
void handlePluginInstallerResult(GstInstallPluginsReturn);
@@ -116,7 +116,7 @@
virtual void durationChanged();
void loadingFailed(MediaPlayer::NetworkState);
- virtual void sourceChanged();
+ virtual void sourceSetup(GstElement*);
GstElement* audioSink() const override;
virtual void configurePlaySink() { }
@@ -131,6 +131,8 @@
bool isLiveStream() const override { return m_isStreaming; }
+ void enableTrack(TrackPrivateBaseGStreamer::TrackType, unsigned index);
+
private:
static void getSupportedTypes(HashSet<String, ASCIICaseInsensitiveHash>&);
static MediaPlayer::SupportsType supportsType(const MediaEngineSupportParameters&);
@@ -137,8 +139,6 @@
static bool isAvailable();
- WeakPtr<MediaPlayerPrivateGStreamer> createWeakPtr() { return m_weakPtrFactory.createWeakPtr(*this); }
-
GstElement* createAudioSink() override;
MediaTime playbackPosition() const;
@@ -153,12 +153,17 @@
virtual void setDownloadBuffering();
void processBufferingStats(GstMessage*);
-#if ENABLE(VIDEO_TRACK) && USE(GSTREAMER_MPEGTS)
+#if ENABLE(VIDEO_TRACK)
+#if USE(GSTREAMER_MPEGTS)
void processMpegTsSection(GstMpegtsSection*);
#endif
-#if ENABLE(VIDEO_TRACK)
+
void processTableOfContents(GstMessage*);
void processTableOfContentsEntry(GstTocEntry*);
+
+ void purgeInvalidAudioTracks(Vector<String> validTrackIds);
+ void purgeInvalidVideoTracks(Vector<String> validTrackIds);
+ void purgeInvalidTextTracks(Vector<String> validTrackIds);
#endif
virtual bool doSeek(const MediaTime& position, float rate, GstSeekFlags seekType);
virtual void updatePlaybackRate();
@@ -173,6 +178,10 @@
void setPlaybinURL(const URL& urlString);
+#if USE(GSTREAMER_PLAYBIN3)
+ void updateTracks();
+#endif
+
protected:
void cacheDuration();
@@ -214,7 +223,7 @@
static void setAudioStreamPropertiesCallback(MediaPlayerPrivateGStreamer*, GObject*);
- static void sourceChangedCallback(MediaPlayerPrivateGStreamer*);
+ static void sourceSetupCallback(MediaPlayerPrivateGStreamer*, GstElement*);
static void videoChangedCallback(MediaPlayerPrivateGStreamer*);
static void videoSinkCapsChangedCallback(MediaPlayerPrivateGStreamer*);
static void audioChangedCallback(MediaPlayerPrivateGStreamer*);
@@ -247,6 +256,13 @@
mutable unsigned long long m_totalBytes;
URL m_url;
bool m_preservesPitch;
+ bool m_isLegacyPlaybin;
+#if USE(GSTREAMER_PLAYBIN3)
+ GRefPtr<GstStreamCollection> m_streamCollection;
+#endif
+ String m_currentAudioStreamId;
+ String m_currentVideoStreamId;
+ String m_currentTextStreamId;
#if ENABLE(WEB_AUDIO)
std::unique_ptr<AudioSourceProviderGStreamer> m_audioSourceProvider;
#endif
@@ -254,17 +270,16 @@
GRefPtr<GstElement> m_downloadBuffer;
RefPtr<MediaPlayerRequestInstallMissingPluginsCallback> m_missingPluginsCallback;
#if ENABLE(VIDEO_TRACK)
- Vector<RefPtr<AudioTrackPrivateGStreamer>> m_audioTracks;
- Vector<RefPtr<InbandTextTrackPrivateGStreamer>> m_textTracks;
- Vector<RefPtr<VideoTrackPrivateGStreamer>> m_videoTracks;
+ HashMap<AtomicString, RefPtr<AudioTrackPrivateGStreamer>> m_audioTracks;
+ HashMap<AtomicString, RefPtr<InbandTextTrackPrivateGStreamer>> m_textTracks;
+ HashMap<AtomicString, RefPtr<VideoTrackPrivateGStreamer>> m_videoTracks;
RefPtr<InbandMetadataTextTrackPrivateGStreamer> m_chaptersTrack;
-#endif
-#if ENABLE(VIDEO_TRACK) && USE(GSTREAMER_MPEGTS)
+#if USE(GSTREAMER_MPEGTS)
HashMap<AtomicString, RefPtr<InbandMetadataTextTrackPrivateGStreamer>> m_metadataTracks;
#endif
+#endif
virtual bool isMediaSource() const { return false; }
};
}
#endif // USE(GSTREAMER)
-#endif
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h 2018-02-19 08:58:14 UTC (rev 228617)
@@ -209,7 +209,10 @@
#if ENABLE(VIDEO_TRACK)
TextChanged = 1 << 5,
#endif
- SizeChanged = 1 << 6
+ SizeChanged = 1 << 6,
+#if ENABLE(GSTREAMER_PLAYBIN3)
+ StreamCollectionChanged = 1 << 7
+#endif
};
WeakPtrFactory<MediaPlayerPrivateGStreamerBase> m_weakPtrFactory;
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp 2018-02-19 08:58:14 UTC (rev 228617)
@@ -54,11 +54,24 @@
g_signal_connect_swapped(m_pad.get(), "notify::active", G_CALLBACK(activeChangedCallback), this);
g_signal_connect_swapped(m_pad.get(), "notify::tags", G_CALLBACK(tagsChangedCallback), this);
- // We can't call notifyTrackOfTagsChanged() directly, because we need tagsChanged()
- // to setup m_tags.
+ // We can't call notifyTrackOfTagsChanged() directly, because we need tagsChanged() to setup m_tags.
tagsChanged();
}
+#if USE(GSTREAMER_PLAYBIN3)
+TrackPrivateBaseGStreamer::TrackPrivateBaseGStreamer(TrackPrivateBase* owner, gint index, GRefPtr<GstStream> stream)
+ : m_notifier(MainThreadNotifier<MainThreadNotification>::create())
+ , m_index(index)
+ , m_stream(stream)
+ , m_owner(owner)
+{
+ ASSERT(m_stream);
+
+ // We can't call notifyTrackOfTagsChanged() directly, because we need tagsChanged() to setup m_tags.
+ tagsChanged();
+}
+#endif
+
TrackPrivateBaseGStreamer::~TrackPrivateBaseGStreamer()
{
disconnect();
@@ -67,14 +80,20 @@
void TrackPrivateBaseGStreamer::disconnect()
{
+ m_tags.clear();
+
+#if USE(GSTREAMER_PLAYBIN3)
+ if (m_stream)
+ m_stream.clear();
+#endif
+
+ m_notifier->cancelPendingNotifications();
+
if (!m_pad)
return;
- m_notifier->cancelPendingNotifications();
g_signal_handlers_disconnect_matched(m_pad.get(), G_SIGNAL_MATCH_DATA, 0, 0, nullptr, nullptr, this);
-
m_pad.clear();
- m_tags.clear();
}
void TrackPrivateBaseGStreamer::activeChangedCallback(TrackPrivateBaseGStreamer* track)
@@ -90,11 +109,20 @@
void TrackPrivateBaseGStreamer::tagsChanged()
{
GRefPtr<GstTagList> tags;
- if (g_object_class_find_property(G_OBJECT_GET_CLASS(m_pad.get()), "tags"))
- g_object_get(m_pad.get(), "tags", &tags.outPtr(), nullptr);
+ if (m_pad) {
+ if (g_object_class_find_property(G_OBJECT_GET_CLASS(m_pad.get()), "tags"))
+ g_object_get(m_pad.get(), "tags", &tags.outPtr(), nullptr);
+ else
+ tags = adoptGRef(gst_tag_list_new_empty());
+ }
+#if USE(GSTREAMER_PLAYBIN3)
+ else if (m_stream)
+ tags = adoptGRef(gst_stream_get_tags(m_stream.get()));
+#endif
else
tags = adoptGRef(gst_tag_list_new_empty());
+ GST_DEBUG("Inspecting track at index %d with tags: %" GST_PTR_FORMAT, m_index, tags.get());
{
LockHolder lock(m_tagMutex);
m_tags.swap(tags);
@@ -109,7 +137,7 @@
return;
gboolean active = false;
- if (m_pad && g_object_class_find_property(G_OBJECT_GET_CLASS(m_pad.get()), "active"))
+ if (g_object_class_find_property(G_OBJECT_GET_CLASS(m_pad.get()), "active"))
g_object_get(m_pad.get(), "active", &active, nullptr);
setActive(active);
@@ -143,9 +171,6 @@
void TrackPrivateBaseGStreamer::notifyTrackOfTagsChanged()
{
- if (!m_pad)
- return;
-
TrackPrivateBaseClient* client = m_owner->client();
if (!client)
return;
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h 2018-02-19 08:58:14 UTC (rev 228617)
@@ -23,13 +23,13 @@
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
-#ifndef TrackPrivateBaseGStreamer_h
-#define TrackPrivateBaseGStreamer_h
+#pragma once
#if ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
#include "GRefPtrGStreamer.h"
#include "MainThreadNotifier.h"
+#include <gst/gst.h>
#include <wtf/Lock.h>
#include <wtf/text/WTFString.h>
@@ -41,6 +41,13 @@
public:
virtual ~TrackPrivateBaseGStreamer();
+ enum TrackType {
+ Audio,
+ Video,
+ Text,
+ Unknown
+ };
+
GstPad* pad() const { return m_pad.get(); }
virtual void disconnect();
@@ -51,7 +58,9 @@
protected:
TrackPrivateBaseGStreamer(TrackPrivateBase* owner, gint index, GRefPtr<GstPad>);
-
+#if USE(GSTREAMER_PLAYBIN3)
+ TrackPrivateBaseGStreamer(TrackPrivateBase* owner, gint index, GRefPtr<GstStream>);
+#endif
void notifyTrackOfActiveChanged();
void notifyTrackOfTagsChanged();
@@ -67,6 +76,9 @@
AtomicString m_label;
AtomicString m_language;
GRefPtr<GstPad> m_pad;
+#if USE(GSTREAMER_PLAYBIN3)
+ GRefPtr<GstStream> m_stream;
+#endif
private:
bool getLanguageCode(GstTagList* tags, AtomicString& value);
@@ -87,5 +99,3 @@
} // namespace WebCore
#endif // ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
-
-#endif // TrackPrivateBaseGStreamer_h
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp 2018-02-19 08:58:14 UTC (rev 228617)
@@ -29,13 +29,14 @@
#include "VideoTrackPrivateGStreamer.h"
+#include "MediaPlayerPrivateGStreamer.h"
#include <glib-object.h>
namespace WebCore {
-VideoTrackPrivateGStreamer::VideoTrackPrivateGStreamer(GRefPtr<GstElement> playbin, gint index, GRefPtr<GstPad> pad)
+VideoTrackPrivateGStreamer::VideoTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstPad> pad)
: TrackPrivateBaseGStreamer(this, index, pad)
- , m_playbin(playbin)
+ , m_player(player)
{
// FIXME: Get a real ID from the tkhd atom.
m_id = "V" + String::number(index);
@@ -42,12 +43,27 @@
notifyTrackOfActiveChanged();
}
+#if USE(GSTREAMER_PLAYBIN3)
+VideoTrackPrivateGStreamer::VideoTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstStream> stream)
+ : TrackPrivateBaseGStreamer(this, index, stream)
+ , m_player(player)
+{
+ m_id = gst_stream_get_stream_id(stream.get());
+ notifyTrackOfActiveChanged();
+}
+#endif
+
void VideoTrackPrivateGStreamer::disconnect()
{
- m_playbin.clear();
+ m_player = nullptr;
TrackPrivateBaseGStreamer::disconnect();
}
+void VideoTrackPrivateGStreamer::markAsActive()
+{
+ VideoTrackPrivate::setSelected(true);
+}
+
void VideoTrackPrivateGStreamer::setSelected(bool selected)
{
if (selected == this->selected())
@@ -54,8 +70,8 @@
return;
VideoTrackPrivate::setSelected(selected);
- if (selected && m_playbin)
- g_object_set(m_playbin.get(), "current-video", m_index, nullptr);
+ if (selected && m_player)
+ m_player->enableTrack(TrackPrivateBaseGStreamer::TrackType::Video, m_index);
}
} // namespace WebCore
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.h (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.h 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.h 2018-02-19 08:58:14 UTC (rev 228617)
@@ -23,8 +23,7 @@
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
-#ifndef VideoTrackPrivateGStreamer_h
-#define VideoTrackPrivateGStreamer_h
+#pragma once
#if ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
@@ -32,17 +31,28 @@
#include "TrackPrivateBaseGStreamer.h"
#include "VideoTrackPrivate.h"
+#include <gst/gst.h>
+#include <wtf/WeakPtr.h>
+
namespace WebCore {
+class MediaPlayerPrivateGStreamer;
class VideoTrackPrivateGStreamer final : public VideoTrackPrivate, public TrackPrivateBaseGStreamer {
public:
- static Ref<VideoTrackPrivateGStreamer> create(GRefPtr<GstElement> playbin, gint index, GRefPtr<GstPad> pad)
+ static Ref<VideoTrackPrivateGStreamer> create(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstPad> pad)
{
- return adoptRef(*new VideoTrackPrivateGStreamer(playbin, index, pad));
+ return adoptRef(*new VideoTrackPrivateGStreamer(player, index, pad));
}
+#if USE(GSTREAMER_PLAYBIN3)
+ static Ref<VideoTrackPrivateGStreamer> create(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstStream> stream)
+ {
+ return adoptRef(*new VideoTrackPrivateGStreamer(player, index, stream));
+ }
+#endif
void disconnect() override;
+ void markAsActive();
void setSelected(bool) override;
void setActive(bool enabled) override { setSelected(enabled); }
@@ -53,14 +63,14 @@
AtomicString language() const override { return m_language; }
private:
- VideoTrackPrivateGStreamer(GRefPtr<GstElement> playbin, gint index, GRefPtr<GstPad>);
-
+ VideoTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer>, gint index, GRefPtr<GstPad>);
+#if USE(GSTREAMER_PLAYBIN3)
+ VideoTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer>, gint index, GRefPtr<GstStream>);
+#endif
AtomicString m_id;
- GRefPtr<GstElement> m_playbin;
+ WeakPtr<MediaPlayerPrivateGStreamer> m_player;
};
} // namespace WebCore
#endif // ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
-
-#endif // VideoTrackPrivateGStreamer_h
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp 2018-02-19 08:58:14 UTC (rev 228617)
@@ -1073,11 +1073,11 @@
switch (m_streamType) {
case WebCore::MediaSourceStreamTypeGStreamer::Audio:
if (m_playerPrivate)
- m_track = WebCore::AudioTrackPrivateGStreamer::create(m_playerPrivate->pipeline(), id(), sinkSinkPad.get());
+ m_track = WebCore::AudioTrackPrivateGStreamer::create(m_playerPrivate->createWeakPtr(), id(), sinkSinkPad.get());
break;
case WebCore::MediaSourceStreamTypeGStreamer::Video:
if (m_playerPrivate)
- m_track = WebCore::VideoTrackPrivateGStreamer::create(m_playerPrivate->pipeline(), id(), sinkSinkPad.get());
+ m_track = WebCore::VideoTrackPrivateGStreamer::create(m_playerPrivate->createWeakPtr(), id(), sinkSinkPad.get());
break;
case WebCore::MediaSourceStreamTypeGStreamer::Text:
m_track = WebCore::InbandTextTrackPrivateGStreamer::create(id(), sinkSinkPad.get());
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp 2018-02-19 08:58:14 UTC (rev 228617)
@@ -486,10 +486,9 @@
return m_mediaSource ? m_mediaSource->buffered() : std::make_unique<PlatformTimeRanges>();
}
-void MediaPlayerPrivateGStreamerMSE::sourceChanged()
+void MediaPlayerPrivateGStreamerMSE::sourceSetup(GstElement* sourceElement)
{
- m_source = nullptr;
- g_object_get(m_pipeline.get(), "source", &m_source.outPtr(), nullptr);
+ m_source = sourceElement;
ASSERT(WEBKIT_IS_MEDIA_SRC(m_source.get()));
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h (228616 => 228617)
--- trunk/Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h 2018-02-19 08:58:14 UTC (rev 228617)
@@ -71,7 +71,7 @@
std::unique_ptr<PlatformTimeRanges> buffered() const override;
MediaTime maxMediaTimeSeekable() const override;
- void sourceChanged() override;
+ void sourceSetup(GstElement*) override;
void setReadyState(MediaPlayer::ReadyState);
void waitForSeekCompleted();
Modified: trunk/Source/cmake/GStreamerDefinitions.cmake (228616 => 228617)
--- trunk/Source/cmake/GStreamerDefinitions.cmake 2018-02-19 08:51:26 UTC (rev 228616)
+++ trunk/Source/cmake/GStreamerDefinitions.cmake 2018-02-19 08:58:14 UTC (rev 228617)
@@ -5,3 +5,4 @@
WEBKIT_OPTION_DEFINE(USE_GSTREAMER_GL "Whether to enable support for GStreamer GL" PRIVATE ON)
WEBKIT_OPTION_DEFINE(USE_GSTREAMER_MPEGTS "Whether to enable support for MPEG-TS" PRIVATE OFF)
+WEBKIT_OPTION_DEFINE(USE_GSTREAMER_PLAYBIN3 "Whether to enable support for GStreamer's playbin3 element" PRIVATE OFF)
