Diff
Modified: trunk/LayoutTests/ChangeLog (119906 => 119907)
--- trunk/LayoutTests/ChangeLog 2012-06-09 16:53:17 UTC (rev 119906)
+++ trunk/LayoutTests/ChangeLog 2012-06-09 17:01:44 UTC (rev 119907)
@@ -1,3 +1,17 @@
+2012-06-09 Victor Carbune <[email protected]>
+
+ Basic support for timestamps within a TextTrackCue
+ https://bugs.webkit.org/show_bug.cgi?id=88187
+
+ Implemented support for timestamps within a TextTrackCue.
+ This enables rendering functionality for Karaoke and Paint-on captions.
+
+ Reviewed by Eric Carlson.
+
+ * media/track/captions-webvtt/captions-inner-timestamps.vtt: Added.
+ * media/track/track-cue-rendering-inner-timestamps-expected.txt: Added.
+ * media/track/track-cue-rendering-inner-timestamps.html: Added.
+
2012-06-08 Vsevolod Vlasov <[email protected]>
IndexedDB: Inspector should handle null, string, and array keyPaths
Added: trunk/LayoutTests/media/track/captions-webvtt/captions-inner-timestamps.vtt (0 => 119907)
--- trunk/LayoutTests/media/track/captions-webvtt/captions-inner-timestamps.vtt (rev 0)
+++ trunk/LayoutTests/media/track/captions-webvtt/captions-inner-timestamps.vtt 2012-06-09 17:01:44 UTC (rev 119907)
@@ -0,0 +1,13 @@
+WEBVTT
+
+1
+00:00:00.000 --> 00:00:00.950
+<00:00:00.200>Chocolate <00:00:00.450>Rain <00:00:00.700>Karaoke
+
+2
+00:00:01.000 --> 00:00:03.600
+<00:00:01.200>Some <00:00:01.450>stay <00:00:01.700>dry <00:00:02.200>and <00:00:02.450>others <00:00:02.950>feel <00:00:03.200>the <00:00:03.450>pain
+
+3
+00:00:03.700 --> 00:00:04.250
+<00:00:03.750>Chocolate <00:00:04.000>Rain
Added: trunk/LayoutTests/media/track/track-cue-rendering-inner-timestamps-expected.txt (0 => 119907)
--- trunk/LayoutTests/media/track/track-cue-rendering-inner-timestamps-expected.txt (rev 0)
+++ trunk/LayoutTests/media/track/track-cue-rendering-inner-timestamps-expected.txt 2012-06-09 17:01:44 UTC (rev 119907)
@@ -0,0 +1,157 @@
+Test that TextTrack's cues are rendered correctly when they contain inner timestamps.
+EVENT(canplaythrough)
+
+** First cue (empty) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 0
+Past:
+Future: Chocolate Rain Karaoke
+Complete: <00:00:00.200>Chocolate <00:00:00.450>Rain <00:00:00.700>Karaoke
+
+** First cue (one word) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 0.25
+Past: Chocolate
+Future: Rain Karaoke
+Complete: <00:00:00.200>Chocolate <00:00:00.450>Rain <00:00:00.700>Karaoke
+
+** First cue (two words) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 0.5
+Past: Chocolate Rain
+Future: Karaoke
+Complete: <00:00:00.200>Chocolate <00:00:00.450>Rain <00:00:00.700>Karaoke
+
+** First cue (three words) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 0.75
+Past: Chocolate Rain Karaoke
+Future:
+Complete: <00:00:00.200>Chocolate <00:00:00.450>Rain <00:00:00.700>Karaoke
+
+** Second cue (empty) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 1
+Past:
+Future: Some stay dry and others feel the pain
+Complete: <00:00:01.200>Some <00:00:01.450>stay <00:00:01.700>dry <00:00:02.200>and <00:00:02.450>others <00:00:02.950>feel <00:00:03.200>the <00:00:03.450>pain
+
+** Second cue (one word) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 1.25
+Past: Some
+Future: stay dry and others feel the pain
+Complete: <00:00:01.200>Some <00:00:01.450>stay <00:00:01.700>dry <00:00:02.200>and <00:00:02.450>others <00:00:02.950>feel <00:00:03.200>the <00:00:03.450>pain
+
+** Second cue (two words) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 1.5
+Past: Some stay
+Future: dry and others feel the pain
+Complete: <00:00:01.200>Some <00:00:01.450>stay <00:00:01.700>dry <00:00:02.200>and <00:00:02.450>others <00:00:02.950>feel <00:00:03.200>the <00:00:03.450>pain
+
+** Second cue (three words) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 1.75
+Past: Some stay dry
+Future: and others feel the pain
+Complete: <00:00:01.200>Some <00:00:01.450>stay <00:00:01.700>dry <00:00:02.200>and <00:00:02.450>others <00:00:02.950>feel <00:00:03.200>the <00:00:03.450>pain
+
+** Second cue (same as before) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 2
+Past: Some stay dry
+Future: and others feel the pain
+Complete: <00:00:01.200>Some <00:00:01.450>stay <00:00:01.700>dry <00:00:02.200>and <00:00:02.450>others <00:00:02.950>feel <00:00:03.200>the <00:00:03.450>pain
+
+** Second cue (four words) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 2.25
+Past: Some stay dry and
+Future: others feel the pain
+Complete: <00:00:01.200>Some <00:00:01.450>stay <00:00:01.700>dry <00:00:02.200>and <00:00:02.450>others <00:00:02.950>feel <00:00:03.200>the <00:00:03.450>pain
+
+** Second cue (five words) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 2.5
+Past: Some stay dry and others
+Future: feel the pain
+Complete: <00:00:01.200>Some <00:00:01.450>stay <00:00:01.700>dry <00:00:02.200>and <00:00:02.450>others <00:00:02.950>feel <00:00:03.200>the <00:00:03.450>pain
+
+** Second cue (same as before) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 2.75
+Past: Some stay dry and others
+Future: feel the pain
+Complete: <00:00:01.200>Some <00:00:01.450>stay <00:00:01.700>dry <00:00:02.200>and <00:00:02.450>others <00:00:02.950>feel <00:00:03.200>the <00:00:03.450>pain
+
+** Second cue (six words) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 3
+Past: Some stay dry and others feel
+Future: the pain
+Complete: <00:00:01.200>Some <00:00:01.450>stay <00:00:01.700>dry <00:00:02.200>and <00:00:02.450>others <00:00:02.950>feel <00:00:03.200>the <00:00:03.450>pain
+
+** Second cue (seven words) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 3.25
+Past: Some stay dry and others feel the
+Future: pain
+Complete: <00:00:01.200>Some <00:00:01.450>stay <00:00:01.700>dry <00:00:02.200>and <00:00:02.450>others <00:00:02.950>feel <00:00:03.200>the <00:00:03.450>pain
+
+** Second cue (eight words) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 3.5
+Past: Some stay dry and others feel the pain
+Future:
+Complete: <00:00:01.200>Some <00:00:01.450>stay <00:00:01.700>dry <00:00:02.200>and <00:00:02.450>others <00:00:02.950>feel <00:00:03.200>the <00:00:03.450>pain
+
+** Third cue (one word) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 3.75
+Past: Chocolate
+Future: Rain
+Complete: <00:00:03.750>Chocolate <00:00:04.000>Rain
+
+** Third cue (two words) **
+EVENT(seeked)
+
+* Cue 0 *
+Current time: 4
+Past: Chocolate Rain
+Future:
+Complete: <00:00:03.750>Chocolate <00:00:04.000>Rain
+END OF TEST
+
Added: trunk/LayoutTests/media/track/track-cue-rendering-inner-timestamps.html (0 => 119907)
--- trunk/LayoutTests/media/track/track-cue-rendering-inner-timestamps.html (rev 0)
+++ trunk/LayoutTests/media/track/track-cue-rendering-inner-timestamps.html 2012-06-09 17:01:44 UTC (rev 119907)
@@ -0,0 +1,102 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+
+ <script src=""
+ <script src=""
+ <script src=""
+
+ <script>
+
+ var testTrack;
+ var cueDisplayElement;
+ var cueDisplayContainer;
+ var seekedCount = 0;
+
+ var testCaseDescription = [
+ "First cue (empty)",
+ "First cue (one word)",
+ "First cue (two words)",
+ "First cue (three words)",
+ "Second cue (empty)",
+ "Second cue (one word)",
+ "Second cue (two words)",
+ "Second cue (three words)",
+ "Second cue (same as before)",
+ "Second cue (four words)",
+ "Second cue (five words)",
+ "Second cue (same as before)",
+ "Second cue (six words)",
+ "Second cue (seven words)",
+ "Second cue (eight words)",
+ "Third cue (one word)",
+ "Third cue (two words)"
+ ];
+
+ // Each test case is found by incrementing with 0.250 the current time.
+ var testCaseSeekTime = [
+ ];
+
+ var currentTestCase = 0;
+
+ function seeked()
+ {
+ for (var i = 0; i < testTrack.track.activeCues.length; ++i) {
+ cueDisplayElement = textTrackDisplayElement(video, 'display', i);
+
+ consoleWrite("");
+ consoleWrite("* Cue " + i + " *");
+
+ consoleWrite("Current time: " + video.currentTime);
+ consoleWrite("Past: " + cueDisplayElement.childNodes[0].innerHTML);
+ consoleWrite("Future: " + cueDisplayElement.childNodes[1].innerHTML);
+
+ consoleWrite("Complete: " + testTrack.track.activeCues[i].text);
+ }
+
+ runNextTestCase();
+ };
+
+ function runNextTestCase()
+ {
+ if (currentTestCase == testCaseDescription.length) {
+ endTest();
+ return;
+ }
+
+ consoleWrite("");
+ consoleWrite("** " + testCaseDescription[currentTestCase] + " **");
+
+ video.currentTime = testCaseSeekTime[currentTestCase];
+ waitForEvent('seeked', seeked, false, true);
+
+ currentTestCase++;
+ }
+
+ function loaded()
+ {
+ consoleWrite("Test that TextTrack's cues are rendered correctly when they contain inner timestamps.");
+
+ findMediaElement();
+ testTrack = document.querySelector('track');
+ video.src = "" '../content/test');
+
+ waitForEvent('canplaythrough', function() {
+ cueDisplayContainer = textTrackDisplayElement(video);
+
+ for (i = 0.0; i < 4.500; i += 0.250)
+ testCaseSeekTime.push(i);
+
+ runNextTestCase();
+ });
+ }
+
+ </script>
+ </head>
+ <body _onload_="loaded()">
+ <video controls>
+ <track src="" kind="captions" default>
+ </video>
+ </body>
+</html>
Modified: trunk/Source/WebCore/ChangeLog (119906 => 119907)
--- trunk/Source/WebCore/ChangeLog 2012-06-09 16:53:17 UTC (rev 119906)
+++ trunk/Source/WebCore/ChangeLog 2012-06-09 17:01:44 UTC (rev 119907)
@@ -1,3 +1,44 @@
+2012-06-09 Victor Carbune <[email protected]>
+
+ Basic support for timestamps within a TextTrackCue
+ https://bugs.webkit.org/show_bug.cgi?id=88187
+
+ Implemented support for timestamps within a TextTrackCue.
+ This enables rendering functionality for Karaoke and Paint-on captions.
+
+ Reviewed by Eric Carlson.
+
+ Test: media/track/track-cue-rendering-inner-timestamps.html
+
+ * css/mediaControls.css: Remove the background shadow pseudo-id and
+ added two others, which can be further used for styling past / future
+ contents of the cue.
+ (video::-webkit-media-text-track-past-nodes): Used to style elements
+ within the cue that are before the current movie time.
+ (video::-webkit-media-text-track-future-nodes): Used to style elements
+ within the cue that are after the current movie time.
+ * html/HTMLMediaElement.cpp:
+ (WebCore::HTMLMediaElement::updateActiveTextTrackCues): Regardless of whether
+ the active set changed or not, the current cues need to be informed of the current
+ movie time.
+ * html/track/TextTrackCue.cpp:
+ (WebCore::TextTrackCue::TextTrackCue): Initiliazed past and future containers.
+ (WebCore::TextTrackCue::getCueAsHTML): Checked whether the current cue text has
+ inner timestamps or not, in order to render faster if no timestamps are there.
+ (WebCore::TextTrackCue::updateDisplayTree): Added method that keeps the past and future
+ containers up to date, given a current movie time.
+ (WebCore):
+ (WebCore::TextTrackCue::getDisplayTree): Replaced the cue background container with the
+ past cues container. The *internal* display tree of the cue now has two containers,
+ representing past and future elements.
+ * html/track/TextTrackCue.h: Added several variables to support the new functionality.
+ (TextTrackCue):
+ * html/track/WebVTTParser.cpp:
+ (WebCore::WebVTTParser::constructTreeFromToken): Updated representation for this patch.
+ * html/track/WebVTTParser.h:
+ (WebVTTParser): Made public the collectTimeStamp method, as this is required for
+ parsing timestamps within a cue as well.
+
2012-06-09 Vsevolod Vlasov <[email protected]>
Web Inspector: Open links in Sources panel by default and fallback to Resources and Network panel otherwise.
Modified: trunk/Source/WebCore/css/mediaControls.css (119906 => 119907)
--- trunk/Source/WebCore/css/mediaControls.css 2012-06-09 16:53:17 UTC (rev 119906)
+++ trunk/Source/WebCore/css/mediaControls.css 2012-06-09 17:01:44 UTC (rev 119907)
@@ -225,13 +225,27 @@
-webkit-box-flex: 1;
}
-video::-webkit-media-text-track-background {
+video::-webkit-media-text-track-past-nodes {
display: inline;
+
background-color: rgba(0, 0, 0, 0.8);
padding: 2px 2px;
+
white-space: pre-wrap;
}
+video::-webkit-media-text-track-future-nodes {
+ visibility: hidden;
+ display: inline;
+
+ background-color: rgba(0, 0, 0, 0.8);
+
+ margin-left: -2px;
+ padding: 2px 2px 2px 0px;
+
+ white-space: pre-wrap;
+}
+
video::-webkit-media-text-track-display {
position: absolute;
color: rgba(255, 255, 255, 1);
Modified: trunk/Source/WebCore/html/HTMLMediaElement.cpp (119906 => 119907)
--- trunk/Source/WebCore/html/HTMLMediaElement.cpp 2012-06-09 16:53:17 UTC (rev 119906)
+++ trunk/Source/WebCore/html/HTMLMediaElement.cpp 2012-06-09 17:01:44 UTC (rev 119907)
@@ -1101,7 +1101,9 @@
if (!currentCues.contains(previousCues[i]) && previousCues[i].data()->isActive())
activeSetChanged = true;
- for (size_t i = 0; !activeSetChanged && i < currentCuesSize; ++i) {
+ for (size_t i = 0; i < currentCuesSize; ++i) {
+ currentCues[i].data()->updateDisplayTree(movieTime);
+
if (!currentCues[i].data()->isActive())
activeSetChanged = true;
}
Modified: trunk/Source/WebCore/html/track/TextTrackCue.cpp (119906 => 119907)
--- trunk/Source/WebCore/html/track/TextTrackCue.cpp 2012-06-09 16:53:17 UTC (rev 119906)
+++ trunk/Source/WebCore/html/track/TextTrackCue.cpp 2012-06-09 17:01:44 UTC (rev 119907)
@@ -40,6 +40,7 @@
#include "DocumentFragment.h"
#include "Event.h"
#include "HTMLDivElement.h"
+#include "HTMLMediaElement.h"
#include "Text.h"
#include "TextTrack.h"
#include "TextTrackCueList.h"
@@ -99,10 +100,14 @@
, m_cueIndex(invalidCueIndex)
, m_writingDirection(Horizontal)
, m_cueAlignment(Middle)
+ , m_documentFragment(0)
, m_scriptExecutionContext(context)
, m_isActive(false)
, m_pauseOnExit(pauseOnExit)
, m_snapToLines(true)
+ , m_hasInnerTimestamps(false)
+ , m_pastDocumentNodes(HTMLDivElement::create(static_cast<Document*>(context)))
+ , m_futureDocumentNodes(HTMLDivElement::create(static_cast<Document*>(context)))
, m_displayTreeShouldChange(true)
, m_displayTree(HTMLDivElement::create(static_cast<Document*>(context)))
, m_displayXPosition(undefinedPosition)
@@ -372,9 +377,16 @@
RefPtr<DocumentFragment> clonedFragment;
Document* document;
- if (!m_documentFragment)
+ if (!m_documentFragment) {
+ m_hasInnerTimestamps = false;
m_documentFragment = WebVTTParser::create(0, m_scriptExecutionContext)->createDocumentFragmentFromCueText(m_content);
+ for (Node *child = m_documentFragment->firstChild(); !m_hasInnerTimestamps && child; child = child->nextSibling()) {
+ if (child->nodeName() == "timestamp")
+ m_hasInnerTimestamps = true;
+ }
+ }
+
document = static_cast<Document*>(m_scriptExecutionContext);
clonedFragment = DocumentFragment::create(document);
@@ -528,9 +540,53 @@
// FIXME(Bug 79916): CSS top and left properties need to be applied.
}
+void TextTrackCue::updateDisplayTree(float movieTime)
+{
+ // The display tree may contain WebVTT timestamp objects representing
+ // timestamps (processing instructions), along with displayable nodes.
+ DEFINE_STATIC_LOCAL(const String, timestampTag, ("timestamp"));
+
+ DEFINE_STATIC_LOCAL(const AtomicString, trackPastNodesShadowPseudoId, ("-webkit-media-text-track-past-nodes"));
+ DEFINE_STATIC_LOCAL(const AtomicString, trackFutureNodesShadowPseudoId, ("-webkit-media-text-track-future-nodes"));
+
+ bool isPastNode = true;
+
+ // Clear the contents of the two sets.
+ m_futureDocumentNodes->removeChildren();
+ m_futureDocumentNodes->setShadowPseudoId(trackFutureNodesShadowPseudoId);
+
+ m_pastDocumentNodes->removeChildren();
+ m_pastDocumentNodes->setShadowPseudoId(trackPastNodesShadowPseudoId);
+
+ // Update the two sets containing past and future WebVTT objects.
+ RefPtr<DocumentFragment> referenceTree = getCueAsHTML();
+
+ if (!m_hasInnerTimestamps) {
+ m_pastDocumentNodes->appendChild(referenceTree);
+ return;
+ }
+
+ for (Node *child = referenceTree->firstChild(); child; child = child->nextSibling()) {
+ if (child->nodeName() == timestampTag) {
+ unsigned int position = 0;
+ String timestamp = child->nodeValue();
+
+ double timestampTime = WebVTTParser::create(0, m_scriptExecutionContext)->collectTimeStamp(timestamp, &position);
+ ASSERT(timestampTime != -1);
+
+ if (timestampTime > movieTime)
+ isPastNode = false;
+ }
+
+ if (isPastNode)
+ m_pastDocumentNodes->appendChild(child->cloneNode(true), ASSERT_NO_EXCEPTION, false);
+ else
+ m_futureDocumentNodes->appendChild(child->cloneNode(true), ASSERT_NO_EXCEPTION, false);
+ }
+}
+
PassRefPtr<HTMLDivElement> TextTrackCue::getDisplayTree()
{
- DEFINE_STATIC_LOCAL(const AtomicString, trackBackgroundShadowPseudoId, ("-webkit-media-text-track-background"));
DEFINE_STATIC_LOCAL(const AtomicString, trackDisplayBoxShadowPseudoId, ("-webkit-media-text-track-display"));
if (!m_displayTreeShouldChange)
@@ -542,6 +598,7 @@
// 10.11. Apply the terms of the CSS specifications to nodes within the
// following constraints, thus obtaining a set of CSS boxes positioned
// relative to an initial containing block:
+ m_displayTree->setShadowPseudoId(trackDisplayBoxShadowPseudoId, ASSERT_NO_EXCEPTION);
m_displayTree->removeChildren();
// The document tree is the tree of WebVTT Node Objects rooted at nodes.
@@ -549,14 +606,11 @@
// The children of the nodes must be wrapped in an anonymous box whose
// 'display' property has the value 'inline'. This is the WebVTT cue
// background box.
- RefPtr<HTMLDivElement> cueBackgroundBox = HTMLDivElement::create(static_cast<Document*>(m_scriptExecutionContext));
- cueBackgroundBox->setShadowPseudoId(trackBackgroundShadowPseudoId);
- cueBackgroundBox->appendChild(getCueAsHTML(), ASSERT_NO_EXCEPTION, true);
+ // Note: This is contained by default in m_pastDocumentNodes.
+ m_displayTree->appendChild(m_pastDocumentNodes, ASSERT_NO_EXCEPTION, true);
+ m_displayTree->appendChild(m_futureDocumentNodes, ASSERT_NO_EXCEPTION, true);
- m_displayTree->setShadowPseudoId(trackDisplayBoxShadowPseudoId, ASSERT_NO_EXCEPTION);
- m_displayTree->appendChild(cueBackgroundBox, ASSERT_NO_EXCEPTION, true);
-
// FIXME(BUG 79916): Runs of children of WebVTT Ruby Objects that are not
// WebVTT Ruby Text Objects must be wrapped in anonymous boxes whose
// 'display' property has the value 'ruby-base'.
@@ -601,6 +655,9 @@
CSSValuePre);
}
+ if (m_hasInnerTimestamps)
+ updateDisplayTree(track()->mediaElement()->currentTime());
+
m_displayTreeShouldChange = false;
// 10.15. Let cue's text track cue display state have the CSS boxes in
Modified: trunk/Source/WebCore/html/track/TextTrackCue.h (119906 => 119907)
--- trunk/Source/WebCore/html/track/TextTrackCue.h 2012-06-09 16:53:17 UTC (rev 119906)
+++ trunk/Source/WebCore/html/track/TextTrackCue.h 2012-06-09 17:01:44 UTC (rev 119907)
@@ -102,6 +102,7 @@
void setIsActive(bool);
PassRefPtr<HTMLDivElement> getDisplayTree();
+ void updateDisplayTree(float);
void removeDisplayTree();
virtual const AtomicString& interfaceName() const;
@@ -167,6 +168,10 @@
bool m_pauseOnExit;
bool m_snapToLines;
+ bool m_hasInnerTimestamps;
+ RefPtr<HTMLDivElement> m_pastDocumentNodes;
+ RefPtr<HTMLDivElement> m_futureDocumentNodes;
+
bool m_displayTreeShouldChange;
RefPtr<HTMLDivElement> m_displayTree;
Modified: trunk/Source/WebCore/html/track/WebVTTParser.h (119906 => 119907)
--- trunk/Source/WebCore/html/track/WebVTTParser.h 2012-06-09 16:53:17 UTC (rev 119906)
+++ trunk/Source/WebCore/html/track/WebVTTParser.h 2012-06-09 17:01:44 UTC (rev 119907)
@@ -94,6 +94,7 @@
void getNewCues(Vector<RefPtr<TextTrackCue> >&);
PassRefPtr<DocumentFragment> createDocumentFragmentFromCueText(const String&);
+ double collectTimeStamp(const String&, unsigned*);
protected:
WebVTTParser(WebVTTParserClient*, ScriptExecutionContext*);
@@ -110,7 +111,7 @@
void createNewCue();
void resetCueValues();
- double collectTimeStamp(const String&, unsigned*);
+
void skipWhiteSpace(const String&, unsigned*);
static void skipLineTerminator(const char* data, unsigned length, unsigned*);
static String collectNextLine(const char* data, unsigned length, unsigned*);
