Re: [whatwg] JavaScript function for closing tags

[Silvia Pfeiffer]

We could specify that WebVTT cues of type metadata should contain valid
JSON - that would make sense to me.

Cues of type captions or subtitles stupid get parsed dune by the addCue()
function of the texttrack API - but not all browsers implement this yet.
Would be worth registering bugs on browsers.

Cheers,
Silvia.

Best Regards,
Silvia.

On 18 Oct. 2017 2:51 am, "Michael A. Peters"  wrote:

> On 10/16/2017 10:08 AM, Roger Hågensen wrote:
>
>> On 2017-10-14 10:13, Michael A. Peters wrote:
>>
>>> I use TextTrack API but it's documention does not specify that it
>>> closes open tags within a cue, in fact I'm fairly certain it doesn't
>>> because some people use it for json and other related none tag related
>>> content.
>>>
>> Looking at https://www.html5rocks.com/en/tutorials/track/basics/
>> it seems JSON can be used, no idea if content type is different or not
>> for that.
>>
>> Some errors using the tracks in XML were solved by the innerHTML trick
>>> where I create a separate html document, append the cue, and then grab
>>> the innerHTML but that doesn't always work to close tags when html
>>> entities are part of the cue string.
>>>
>>
>> Mixing XML and HTML is not a good idea. Would it not be easier to have
>> the server send out proper XML instead of hTML? Valid XML is also valid
>> HTML (the reverse is not always true).
>>
>
> I agree, but what I was using an html document for - when using JS
> innerHTML it has closing tags so the only issue would be tags that html
> itself does not close (e.g. br) but those are not applicable with a WebVTT
> cue - which is only suppose to support a very small number of tags, all
> which have closing tags.
>
> The problem is WebVTT does not require tags be closed in a cue, e.g.
>
> 04:05.000 --> 04:07.250
> This is a cue.
>
> That's allowed in WebVTT
>
> I convert c.foo into
>
> This is a cue.
>
> and when I add that to the html document and use innerHTML it then has the
> closing  on it.
>
> While it seems to work with some html entities, it breaks with others like
> 
>
> So for now I have to just make sure all my WebVTT are closed and not use
> the hack that adds closing tags - but since WebVTT cues do not have to have
> closing tags, but the cues need to work in XML documents, a built-in parser
> in JS that can add missing closing tags I think would be a good thing.
>
>
> And if XML and HTML is giving you issues then use JSON instead.
>> I did not see JSON mentioned in the W3C spec though.
>>
>
> I think the JSON in WebVTT cues is not spec but some are using it.
>
> Basically the textrack API seems to allow almost any string, it really has
> to as WebVTT is not static and the spec changes. I wouldn't mind JSON being
> added to WebVTT as it would be a handy way to encode metadata about the
> media but that's another topic.
>
> A built in JS HTML parser may also be of benefit in preventing code
> injection, e.g. stripping out tags from a WebVTT cue that a website does
> not allow.
>
> The TextTrack API doesn't filter out things like script or other tags that
> aren't part of WebVTT which means any site that allows users to upload
> WebVTT files is creating a potential code injection vulnerability.
>
> Server-side code should filter it on upload, but it would be nice to
> *someday* be able to pass a string through a native JS filter much the same
> way we can with htmltidy server-side and remove all but white-listed tags
> and attributes and get back a cleaned string with all tags closed.
>
> It looks like Google has a library that does that but it isn't intended
> for client-side JS and may not be fast enough for things like phones to
> process time-sensitive cues (I don't know).
>
> I might be wrong but it looked like the google library I found was
> intended for server-side Node.js use.
>
>

Re: [whatwg] JavaScript function for closing tags

[Michael A. Peters]

On 10/16/2017 10:08 AM, Roger Hågensen wrote:

On 2017-10-14 10:13, Michael A. Peters wrote:

I use TextTrack API but it's documention does not specify that it
closes open tags within a cue, in fact I'm fairly certain it doesn't
because some people use it for json and other related none tag related
content.

Looking at https://www.html5rocks.com/en/tutorials/track/basics/
it seems JSON can be used, no idea if content type is different or not
for that.


Some errors using the tracks in XML were solved by the innerHTML trick
where I create a separate html document, append the cue, and then grab
the innerHTML but that doesn't always work to close tags when html
entities are part of the cue string.


Mixing XML and HTML is not a good idea. Would it not be easier to have
the server send out proper XML instead of hTML? Valid XML is also valid
HTML (the reverse is not always true).


I agree, but what I was using an html document for - when using JS 
innerHTML it has closing tags so the only issue would be tags that html 
itself does not close (e.g. br) but those are not applicable with a 
WebVTT cue - which is only suppose to support a very small number of 
tags, all which have closing tags.


The problem is WebVTT does not require tags be closed in a cue, e.g.

04:05.000 --> 04:07.250
This is a cue.

That's allowed in WebVTT

I convert c.foo into

This is a cue.

and when I add that to the html document and use innerHTML it then has 
the closing  on it.


While it seems to work with some html entities, it breaks with others 
like 


So for now I have to just make sure all my WebVTT are closed and not use 
the hack that adds closing tags - but since WebVTT cues do not have to 
have closing tags, but the cues need to work in XML documents, a 
built-in parser in JS that can add missing closing tags I think would be 
a good thing.




And if XML and HTML is giving you issues then use JSON instead.
I did not see JSON mentioned in the W3C spec though.


I think the JSON in WebVTT cues is not spec but some are using it.

Basically the textrack API seems to allow almost any string, it really 
has to as WebVTT is not static and the spec changes. I wouldn't mind 
JSON being added to WebVTT as it would be a handy way to encode metadata 
about the media but that's another topic.


A built in JS HTML parser may also be of benefit in preventing code 
injection, e.g. stripping out tags from a WebVTT cue that a website does 
not allow.


The TextTrack API doesn't filter out things like script or other tags 
that aren't part of WebVTT which means any site that allows users to 
upload WebVTT files is creating a potential code injection vulnerability.


Server-side code should filter it on upload, but it would be nice to 
*someday* be able to pass a string through a native JS filter much the 
same way we can with htmltidy server-side and remove all but 
white-listed tags and attributes and get back a cleaned string with all 
tags closed.


It looks like Google has a library that does that but it isn't intended 
for client-side JS and may not be fast enough for things like phones to 
process time-sensitive cues (I don't know).


I might be wrong but it looked like the google library I found was 
intended for server-side Node.js use.

Re: [whatwg] JavaScript function for closing tags

[Roger Hågensen]

On 2017-10-14 10:13, Michael A. Peters wrote:
I use TextTrack API but it's documention does not specify that it closes 
open tags within a cue, in fact I'm fairly certain it doesn't because 
some people use it for json and other related none tag related content.

Looking at https://www.html5rocks.com/en/tutorials/track/basics/
it seems JSON can be used, no idea if content type is different or not 
for that.


Some errors using the tracks in XML were solved by the innerHTML trick 
where I create a separate html document, append the cue, and then grab 
the innerHTML but that doesn't always work to close tags when html 
entities are part of the cue string.


Mixing XML and HTML is not a good idea. Would it not be easier to have 
the server send out proper XML instead of hTML? Valid XML is also valid 
HTML (the reverse is not always true).

And if XML and HTML is giving you issues then use JSON instead.
I did not see JSON mentioned in the W3C spec though.


There does not seem to be a JavaScript API for closing open tags.

This is problematic when dealing with WebVTT which does not require 
tags be

closed.

Where it is the biggest problem is when the document is being served as
XML+XHTML


If a XML document is being served with unclosed tags then it's not valid 
XML, so it's no wonder if that then causes issues.



--
Unless specified otherwise, anything I write publicly is considered 
Public Domain (CC0). My opinions are my own unless specified otherwise.

Roger Hågensen,
Freelancer, Norway.

Re: [whatwg] JavaScript function for closing tags

[Michael A. Peters]

I use TextTrack API but it's documention does not specify that it closes 
open tags within a cue, in fact I'm fairly certain it doesn't because 
some people use it for json and other related none tag related content.


Some errors using the tracks in XML were solved by the innerHTML trick 
where I create a separate html document, append the cue, and then grab 
the innerHTML but that doesn't always work to close tags when html 
entities are part of the cue string.


I use JS regex to turn  into  and  
into  and just jQuery .append() to place 
the cue in a div for captions/subtitles since browser standard html5 
audio players have zero support for captions by themselves, and the 
jQuery .append() is what needs closing tags in XML or it understandably 
completely fails.


Also using it for chapters which don't appear to be supported in either 
browser standard audio or video players.


On 10/14/2017 12:46 AM, Silvia Pfeiffer wrote:

Hi Michael,

It seems to me that the TextTrack API is made for this use case.
Why does it not work for you?

Cheers,
Silvia.


On Sat, Oct 14, 2017 at 4:36 PM, Michael A. Peters
 wrote:

There does not seem to be a JavaScript API for closing open tags.

This is problematic when dealing with WebVTT which does not require tags be
closed.

Where it is the biggest problem is when the document is being served as
XML+XHTML

I tried the following hack which seemed to be working:

cleandoc = document.implementation.createHTMLDocument("FuBar");
cleanbody = document.createElementNS("http://www.w3.org/1999/xhtml;,
"body");
cleandoc.documentElement.appendChild(cleanbody);


Then I could do the following when with a WebVTT cue:

cleanbody.innerHTML = string;
return (cleanbody.innerHTML);

That *mostly* works but seems to sometimes fail when string contains
entities, such as 

What happens is it returns an empty string.

Given that WebVTT is part of HTML5 and browser native html5 audio players
don't support caption tracks forcing us to write our own implementations if
we want captions with audio, it sure would be nice if there was a pure
JavaScript way to just add closing tags to a string because there is never a
guarantee valid WebVTT cue has closed tags which are required for XHTML sent
as XML.

Seems to me that a JS native function to add missing closing tags would have
more application than just WebVTT cues.

I looked for a jQuery filter that does it, but could not find one.

It also could be of benefit in emulating document.write() as many of
Google's tools *still* require document.write() despite the issues with
document.write() and XML having been known for 15+ years now.

Any chance of getting a parser into JavaScript that at least would be
capable of closing open tags in a string passed to it?

Re: [whatwg] JavaScript function for closing tags

[Silvia Pfeiffer]

Hi Michael,

It seems to me that the TextTrack API is made for this use case.
Why does it not work for you?

Cheers,
Silvia.


On Sat, Oct 14, 2017 at 4:36 PM, Michael A. Peters
 wrote:
> There does not seem to be a JavaScript API for closing open tags.
>
> This is problematic when dealing with WebVTT which does not require tags be
> closed.
>
> Where it is the biggest problem is when the document is being served as
> XML+XHTML
>
> I tried the following hack which seemed to be working:
>
> cleandoc = document.implementation.createHTMLDocument("FuBar");
> cleanbody = document.createElementNS("http://www.w3.org/1999/xhtml;,
> "body");
> cleandoc.documentElement.appendChild(cleanbody);
>
>
> Then I could do the following when with a WebVTT cue:
>
> cleanbody.innerHTML = string;
> return (cleanbody.innerHTML);
>
> That *mostly* works but seems to sometimes fail when string contains
> entities, such as 
>
> What happens is it returns an empty string.
>
> Given that WebVTT is part of HTML5 and browser native html5 audio players
> don't support caption tracks forcing us to write our own implementations if
> we want captions with audio, it sure would be nice if there was a pure
> JavaScript way to just add closing tags to a string because there is never a
> guarantee valid WebVTT cue has closed tags which are required for XHTML sent
> as XML.
>
> Seems to me that a JS native function to add missing closing tags would have
> more application than just WebVTT cues.
>
> I looked for a jQuery filter that does it, but could not find one.
>
> It also could be of benefit in emulating document.write() as many of
> Google's tools *still* require document.write() despite the issues with
> document.write() and XML having been known for 15+ years now.
>
> Any chance of getting a parser into JavaScript that at least would be
> capable of closing open tags in a string passed to it?

[whatwg] JavaScript function for closing tags

[Michael A. Peters]

There does not seem to be a JavaScript API for closing open tags.

This is problematic when dealing with WebVTT which does not require tags 
be closed.


Where it is the biggest problem is when the document is being served as 
XML+XHTML


I tried the following hack which seemed to be working:

cleandoc = document.implementation.createHTMLDocument("FuBar");
cleanbody = document.createElementNS("http://www.w3.org/1999/xhtml;, 
"body");

cleandoc.documentElement.appendChild(cleanbody);


Then I could do the following when with a WebVTT cue:

cleanbody.innerHTML = string;
return (cleanbody.innerHTML);

That *mostly* works but seems to sometimes fail when string contains 
entities, such as 


What happens is it returns an empty string.

Given that WebVTT is part of HTML5 and browser native html5 audio 
players don't support caption tracks forcing us to write our own 
implementations if we want captions with audio, it sure would be nice if 
there was a pure JavaScript way to just add closing tags to a string 
because there is never a guarantee valid WebVTT cue has closed tags 
which are required for XHTML sent as XML.


Seems to me that a JS native function to add missing closing tags would 
have more application than just WebVTT cues.


I looked for a jQuery filter that does it, but could not find one.

It also could be of benefit in emulating document.write() as many of 
Google's tools *still* require document.write() despite the issues with 
document.write() and XML having been known for 15+ years now.


Any chance of getting a parser into JavaScript that at least would be 
capable of closing open tags in a string passed to it?