Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 37fb0a0ead92c3e0470ee9bbf6fdedb9ea54f528
https://github.com/WebKit/WebKit/commit/37fb0a0ead92c3e0470ee9bbf6fdedb9ea54f528
Author: Vivienne Watermeier <vwaterme...@igalia.com>
Date: 2025-08-06 (Wed, 06 Aug 2025)
Changed paths:
M LayoutTests/TestExpectations
A
LayoutTests/media/media-source/content/counting-h264-mp4ttsimple-manifest.json
A LayoutTests/media/media-source/content/counting-h264-mp4ttsimple.mp4
A
LayoutTests/media/media-source/content/test-inband-cea608-track-manifest.json
A LayoutTests/media/media-source/content/test-inband-cea608-track.mp4
A
LayoutTests/media/media-source/media-source-inband-cea608-track-expected.txt
A LayoutTests/media/media-source/media-source-inband-cea608-track.html
A LayoutTests/media/media-source/media-source-mp4ttsimple-expected.txt
A LayoutTests/media/media-source/media-source-mp4ttsimple.html
M LayoutTests/platform/glib/TestExpectations
M Source/WebCore/platform/graphics/gstreamer/GStreamerRegistryScanner.cpp
M Source/WebCore/platform/graphics/gstreamer/GStreamerRegistryScanner.h
M
Source/WebCore/platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp
M Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp
M Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.h
M
Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.cpp
Log Message:
-----------
[GStreamer][MSE] Extend text track support to MP4/3GPP timed text and CEA-608
https://bugs.webkit.org/show_bug.cgi?id=295634
Reviewed by Philippe Normand and Alicia Boya Garcia.
Adds support for MPEG-4/3GPP timed text (known in MP4 as tx3g), as well as
experimental support for CEA-608 captions. Both formats are supported
by converting to WebVTT, using the optional parser/encoder elements in the
append pipeline.
tx3g support is straightforward - since qtdemux pushes raw text buffers,
we just need to plug webvttenc.
CEA-608 support rests on two elements: ccconverter and cea608tott.
As 608 captions are essentially a constant stream of byte pairs, translating
them into WebVTT is not quite so simple.
Specifically, cea608tott currently swallows the very last cue of the stream,
as without an EOS event it can never know at what point that caption is
"complete".
For experimental support, I consider this "good enough for now".
Considerations for solving this in the future:
- append pipeline does not expect EOS events in normal operation, so while it
would
work to push the final cue, it will introduce much bigger issues
- instead, we could modify cea608tott itself to introduce a custom event, which
would
have the same effect on the element, without breaking the append pipeline
- the other issue is with timing of this event:
- pushing it in response to endOfStream(), append pipeline would end up
receiving a
buffer outside of an append, which it can't handle
- pushing it at each end-of-append will work, but will have the side effect
of cutting any caption that stretches across multiple source buffers
Note also that currently, we expect the 608 captions to exist in the MP4 as a
separate track, just like other subtitle formats would, though it is more common
to see it inserted into the video stream. To extract those, we would need to
additionally
plug the corresponding *ccextractor element after the video decoder.
That would require extra effort to handle this "hidden" track properly in
AppendPipeline.
* LayoutTests/TestExpectations:
* LayoutTests/platform/glib/TestExpectations: Enable the new tests only for GTK
+ WPE ports.
*
LayoutTests/media/media-source/content/counting-h264-mp4ttsimple-manifest.json:
Added.
* LayoutTests/media/media-source/content/counting-h264-mp4ttsimple.mp4: Added.
* LayoutTests/media/media-source/media-source-mp4ttsimple-expected.txt: Added.
* LayoutTests/media/media-source/media-source-mp4ttsimple.html: Added.
Same as media-source-vttsimple.
*
LayoutTests/media/media-source/content/test-inband-cea608-track-manifest.json:
Added.
* LayoutTests/media/media-source/content/test-inband-cea608-track.mp4: Added.
* LayoutTests/media/media-source/media-source-inband-cea608-track-expected.txt:
Added.
* LayoutTests/media/media-source/media-source-inband-cea608-track.html: Added.
Tests conversion to WebVTT. Note cea608tott only pushes 29 cues, instead of
the expected 30 (see above).
* Source/WebCore/platform/graphics/gstreamer/GStreamerRegistryScanner.cpp:
Indicate support for tx3g and x-cea-608, if webvttenc and cea608tott
respectively are available.
(WebCore::GStreamerRegistryScanner::ElementFactories::ElementFactories):
(WebCore::GStreamerRegistryScanner::ElementFactories::~ElementFactories):
(WebCore::GStreamerRegistryScanner::ElementFactories::elementFactoryTypeToString):
(WebCore::GStreamerRegistryScanner::ElementFactories::factory const):
(WebCore::GStreamerRegistryScanner::ElementFactories::hasElementForCaps const):
(WebCore::GStreamerRegistryScanner::initializeDecoders):
* Source/WebCore/platform/graphics/gstreamer/GStreamerRegistryScanner.h: Add
ElementFactories::Type::CaptionEncoder
*
Source/WebCore/platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp:
Fix some logging.
(WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer):
(WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample):
* Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp:
(WebCore::AppendPipeline::appsrcEndOfAppendCheckerProbe): Write pipeline dump
at end-of-append.
(WebCore::AppendPipeline::parseDemuxerSrcPadCaps): Handle
closedcaption/x-cea-608 as StreamType::Text
(WebCore::AppendPipeline::appsinkCapsChanged): Small fix for some "incompatible
caps" errors I encountered.
(WebCore::createOptionalParserForFormat): Plug ccconverter for
closedcaption/x-cea-608
(WebCore::createOptionalEncoderForFormat): Plug cea608tott for
closedcaption/x-cea-608
(WebCore::AppendPipeline::Track::emplaceOptionalElementsForFormat): Combine
both emplaceOptional... methods into one.
(WebCore::AppendPipeline::Track::emplaceOptionalParserForFormat): Deleted.
(WebCore::AppendPipeline::Track::emplaceOptionalEncoderForFormat): Deleted.
(WebCore::AppendPipeline::Track::initializeElements):
(WebCore::AppendPipeline::recycleTrackForPad):
* Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.h:
*
Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.cpp:
(WebCore::SourceBufferPrivateGStreamer::enqueueSample): Log contents of text
samples.
(WebCore::SourceBufferPrivateGStreamer::isReadyForMoreSamples): Fix some
logging.
Canonical link: https://commits.webkit.org/298284@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes
