Re: Should web specifications try to describe object lifetimes? [was Intent to implement: AudioWorklet]

[Karl Tomlinson]

On Fri, 4 May 2018 14:32:20 -0400, Boris Zbarsky wrote:

> On 5/4/18 3:34 AM, Karl Tomlinson wrote:
>> Not sure I understand your question, but the observable behavior
>> described by this section, or specifically "Before an AudioNode is
>> deleted, it will disconnect itself from any other AudioNodes"
>
> Right.  So that observably influences the sound produced or some
> such, sounds like?

That's correct.  The connections influence the behavior of the
"other AudioNodes".  Depending on how the rest of the graph is
assembled, the behavior change may or may not be heard.  There are
some graph designs where the behavior change will be heard.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to implement: AudioWorklet

[Nils Ohlmeier]

Looks like the existing WebAudio issue 
https://github.com/WebAudio/web-audio-api/issues/1471 

is where this should get resolved (for people who haven’t switched over there 
yet).

Best regards
  Nils Ohlmeier

> On May 2, 2018, at 09:05, hongc...@chromium.org wrote:
> 
> It's great to see the intent for the AudioWorklet, but also I can see the GC 
> observability is still being discussed here.
> 
> Karl, can you open a new spec issue if you think this needs another look from 
> AudioWG and TAG?
> 
> -Hongchan
> 
> 
> On Wednesday, May 2, 2018 at 8:17:30 AM UTC-7, Boris Zbarsky wrote:
>> On 5/2/18 5:21 AM, Karl Tomlinson wrote:
>>> [[AudioNode Lifetime]] https://github.com/WebAudio/web-audio-api/issues/1471
>> 
>> I've read through that thread, but I'm still a little unclear on where
>> thing stand.  With the latest proposal, can there be observable
>> situations where the output sound depends on GC timing?
>> 
>> If so, that doesn't sound acceptable.  It also sounds to me like Alex
>> Russell in
>> https://github.com/WebAudio/web-audio-api/issues/1471#issuecomment-362604396
>> didn't believe there was an observable sound difference possible.  If
>> one is possible, we should communicate this to him very clearly
>> 
>> -Boris
> 
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform



signature.asc
Description: Message signed with OpenPGP
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Should web specifications try to describe object lifetimes? [was Intent to implement: AudioWorklet]

[Boris Zbarsky]

On 5/4/18 3:34 AM, Karl Tomlinson wrote:

Not sure I understand your question, but the observable behavior
described by this section, or specifically "Before an AudioNode is
deleted, it will disconnect itself from any other AudioNodes"


Right.  So that observably influences the sound produced or some such, 
sounds like?



Yes, the source of the observable GC problem is very much in the
spec now, and tracked by the [[AudioNode Lifetime]] issue.


Understood.

-Boris
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Should web specifications try to describe object lifetimes? [was Intent to implement: AudioWorklet]

[Karl Tomlinson]

Boris Zbarsky writes:

> So if there is in fact a problem still remaining, convincing Alex
> that it's there is pretty valuable.  Having him convinced it
> doesn't exist is actively harmful as far as getting it fixed goes.

Thank you.  Alex sounds like a very valuable person to get on side.

We may still have some time on our hands, but I'm not sure how long
to pursue the WG about this before we aim to get Alex involved.

Neither Chrome 66 nor current Chromium [[processor source code]]
support AudioWorkletNodes with lifetimes based on input
connections in a way that is compatible with the spec and so
Chrome or the spec will change at some point.

It would be nice, however, to resolve these issues before Chrome
ships such changes.

Hongchan has assigned himself to the [[AudioNode Lifetime]] spec
bug, and so I'm happy for him to have a go, if he is planning to
fix it.

> If there is no difference in observable behavior, what would a
> normative requirement mean?

Good question.  I will find that useful at some point.

> Karl Tomlinson wrote:
>> If the whole normative AudioNode lifetime section were dropped
>> then this would clearly be an implementation issue rather than a
>> spec issue.
>
> Assuming there is no observable behavior involved (other than
> memory usage), right?

Not sure I understand your question, but the observable behavior
described by this section, or specifically "Before an AudioNode is
deleted, it will disconnect itself from any other AudioNodes", is
the source of observable behavior on GC.  Removing this section
would remove the core problem.

The rest of the section describes conditions that would keep an
AudioNode alive, and describes when AudioNodes are no longer
required.  These portions are not _necessary_, as nothing
elsewhere indicates that AudioNodes should ever be removed, but
there seem to be differing opinions on the importance of these
clauses.  (Compare e.g. [[PermissionStatus listeners]],
[[WebSocket listeners]])

>> The history here is:
>
> Thank you for the summary.  That was helpful.  One thing I'm not
> 100% clear on at this point: are there spec issues that can lead
> to observable GC, or are we just talking about implementation bugs
> that make GC observable now?

Yes, the source of the observable GC problem is very much in the
spec now, and tracked by the [[AudioNode Lifetime]] issue.

Once that is resolved, I'm hoping that a spec change to support
something like [[silence tracking]] can make AudioWorklet suitable
for custom AudioNodes.

[[processor source code]]
https://chromium.googlesource.com/chromium/src/+blame/9d4a34cadf1b7d85f4ad0b709374aaaf59325a8d/third_party/WebKit/Source/modules/webaudio/AudioWorkletNode.cpp#110

[[PermissionStatus listeners]]
https://github.com/w3c/permissions/issues/170

[[WebSocket listeners]]
https://html.spec.whatwg.org/#garbage-collection-3

[[AudioNode Lifetime]]
https://github.com/WebAudio/web-audio-api/issues/1471

[[silence tracking]]
https://github.com/WebAudio/web-audio-api/issues/1453
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Should web specifications try to describe object lifetimes? [was Intent to implement: AudioWorklet]

[Boris Zbarsky]

On 5/3/18 6:21 PM, Karl Tomlinson wrote:

I didn't understand why he was highlighting "order of object
tear-down", nor why he was implying that only "VERY fine-grained"
knowledge was a problem.


Alex, depending on whether he's speaking with his TAG hat or Google hat 
on, is either trying to figure out whether there is a problem with the 
spec or trying to justify Google shipping a particular behavior.  I'm 
not quite sure which hat he's wearing here.



He does seem to have misunderstood that there is a problem to fix.
But I assume the way forward here is to help the WG find a
solution that works.  What is the advantage of explaining the
situation to Alex?


As a TAG member, he can work with the TAG to explain to the WG that 
their current spec is not acceptable, assuming he's convinced on that.


With his Google hat on, he can make similar arguments internally about 
Chrome's implementation.


So if there is in fact a problem still remaining, convincing Alex that 
it's there is pretty valuable.  Having him convinced it doesn't exist is 
actively harmful as far as getting it fixed goes.



Should specifications instead just focus on observable behavior,
and leave it to implementations to optimize and to reclaim
resources that are no longer required?


Generally speaking, yes.  That said, specifications should be really 
clear on which resources are no longer required, if possible.



Are you aware of any guidance I could reference, if advocating
this?  (The WG has been very keen to make this normative.)


If there is no difference in observable behavior, what would a normative 
requirement mean?  And if there _is_ a difference in observable 
behavior, then of course the actual observable behavior that 
implementors need to implement needs to be specced normatively; this 
doesn't require saying anything about GC.



Perhaps it would be best to just have an informative section
explaining design decisions made for the sake of making resource
reclamation possible, such as lack of graph introspection.


This would definitely be useful.


Perhaps
it would also be helpful to have some informative reminders to
implementations re what GC must not affect.


Yes, that would be helpful.


If the whole normative AudioNode lifetime section were dropped
then this would clearly be an implementation issue rather than a
spec issue.


Assuming there is no observable behavior involved (other than memory 
usage), right?



The history here is:


Thank you for the summary.  That was helpful.  One thing I'm not 100% 
clear on at this point: are there spec issues that can lead to 
observable GC, or are we just talking about implementation bugs that 
make GC observable now?


-Boris
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Should web specifications try to describe object lifetimes? [was Intent to implement: AudioWorklet]

[Robert O'Callahan]

I read the threads you referenced and the latest spec, and I think you're
absolutely right about everything :-).

On Fri, May 4, 2018 at 10:21 AM, Karl Tomlinson  wrote:

> Thank you for taking a look, Boris.  I'm quite unclear how any of
> the changes proposed in the [[March F2F resolution]] comment would
> resolve the issue, and so I expect not.
>
> [[March F2F resolution]]
> https://github.com/WebAudio/web-audio-api/issues/1471#
> issuecomment-376223916


I wonder what that resolution entails since the spec changes haven't
happened yet.

However, it sounds like it's heading in the wrong direction. Rather than
trying to fix the description of AudioNode lifetimes, the entire section
should be deleted.

He does seem to have misunderstood that there is a problem to fix.
> But I assume the way forward here is to help the WG find a
> solution that works.  What is the advantage of explaining the
> situation to Alex?
>

AIUI If you can't convince someone on the WG that there is problem to fix,
but you can convince someone on the TAG, then they can require the WG to
fix it.

I'm happy to query the F2F resolution, but I wonder which is the
> best way to resolve this.
>
> Is having web specifications try to describe object lifetimes
> helpful, or is it just over-prescribing?
>

I think it's very dangerous. If those descriptions are normative then it's
easy to accidentally make GC observable. If they're not normative then they
confuse implementors and users into accidentally treating them as normative.

Should specifications instead just focus on observable behavior,
> and leave it to implementations to optimize and to reclaim
> resources that are no longer required?
>

Yes. People need to understand that GC is purely an optimization.

Perhaps it would be best to just have an informative section
> explaining design decisions made for the sake of making resource
> reclamation possible, such as lack of graph introspection.  Perhaps
> it would also be helpful to have some informative reminders to
> implementations re what GC must not affect.
>

Yes.


> If the whole normative AudioNode lifetime section were dropped
> then this would clearly be an implementation issue rather than a
> spec issue.
>

Yes!

Maybe I can help by writing a blog post or something...

Rob
-- 
Su ot deraeppa sah dna Rehtaf eht htiw saw hcihw, efil lanrete eht uoy ot
mialcorp ew dna, ti ot yfitset dna ti nees evah ew; deraeppa efil eht. Efil
fo Drow eht gninrecnoc mialcorp ew siht - dehcuot evah sdnah ruo dna ta
dekool evah ew hcihw, seye ruo htiw nees evah ew hcihw, draeh evah ew
hcihw, gninnigeb eht morf saw hcihw taht.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Should web specifications try to describe object lifetimes? [was Intent to implement: AudioWorklet]

[L. David Baron]

On Friday 2018-05-04 10:21 +1200, Karl Tomlinson wrote:
> Is having web specifications try to describe object lifetimes
> helpful, or is it just over-prescribing?
> 
> Should specifications instead just focus on observable behavior,
> and leave it to implementations to optimize and to reclaim
> resources that are no longer required?
> 
> Are you aware of any guidance I could reference, if advocating
> this?  (The WG has been very keen to make this normative.)

Without digging too far into this specific case, I can attempt to
offer some answers to the general questions:

There is design guidance here, in a document published by the W3C
Technical Architecture Group:
https://w3ctag.github.io/design-principles/#js-gc
that garbage collection should not be observable.

Ensuring that GC isn't observable sometimes does require normative
descriptions of minimum object lifetimes, such as in
https://github.com/w3c/permissions/issues/170 , but generally
normative descriptions of maximum object lifetimes shouldn't be
needed.

That said, in general it's useful for specifications to give
informative explanations of the design decisions that led to the
current specification (since that helps future maintenance of the
specification, and also likely helps implementors contribute to that
maintenance).

It's also generally useful for specifications to give informative
implementation advice when there are things worth pointing out that
an implementor might not realize immediately.

I'd like to specifications have both sorts of informative sections
substantially more often than they do today.

> Perhaps it would be best to just have an informative section
> explaining design decisions made for the sake of making resource
> reclamation possible, such as lack of graph introspection.  Perhaps
> it would also be helpful to have some informative reminders to
> implementations re what GC must not affect.

These both sound useful, as I mentioned above.

-David

-- 
턞   L. David Baron http://dbaron.org/   턂
턢   Mozilla  https://www.mozilla.org/   턂
 Before I built a wall I'd ask to know
 What I was walling in or walling out,
 And to whom I was like to give offense.
   - Robert Frost, Mending Wall (1914)


signature.asc
Description: PGP signature
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Should web specifications try to describe object lifetimes? [was Intent to implement: AudioWorklet]

[Karl Tomlinson]

Boris Zbarsky writes:

> On 5/2/18 5:21 AM, Karl Tomlinson wrote:
>> [[AudioNode Lifetime]] https://github.com/WebAudio/web-audio-api/issues/1471
>
> I've read through that thread, but I'm still a little unclear on
> where thing stand.  With the latest proposal, can there be
> observable situations where the output sound depends on GC timing?

Thank you for taking a look, Boris.  I'm quite unclear how any of
the changes proposed in the [[March F2F resolution]] comment would
resolve the issue, and so I expect not.

[[March F2F resolution]]
https://github.com/WebAudio/web-audio-api/issues/1471#issuecomment-376223916

> If so, that doesn't sound acceptable.  It also sounds to me like
> Alex Russell in
> https://github.com/WebAudio/web-audio-api/issues/1471#issuecomment-362604396
> didn't believe there was an observable sound difference possible.
> If one is possible, we should communicate this to him very
> clearly

The spec says that when a node is deleted, it will disconnect
itself from other nodes.  This disconnection definitely can lead
to observable sound differences, if the node can be deleted.  The
spec is unclear on whether a node with observable connections can
be deleted, but there is certainly reason to interpret it in this
way and members of the working group were doing so.  If anything,
the March F2F resolution seems to reinforce this view.

I was also quite unclear as to the point of Alex's comment.
I didn't understand why he was highlighting "order of object
tear-down", nor why he was implying that only "VERY fine-grained"
knowledge was a problem.

I'm not clear as to exactly what he was referring with
"@joeberkovitz's change to cause the spec to not discuss GC", but
changing the spec to not discuss GC would certainly resolve the
issue.  My take-away was that Alex was advocating no behavior
changes on GC.

He does seem to have misunderstood that there is a problem to fix.
But I assume the way forward here is to help the WG find a
solution that works.  What is the advantage of explaining the
situation to Alex?

I'm happy to query the F2F resolution, but I wonder which is the
best way to resolve this.

Is having web specifications try to describe object lifetimes
helpful, or is it just over-prescribing?

Should specifications instead just focus on observable behavior,
and leave it to implementations to optimize and to reclaim
resources that are no longer required?

Are you aware of any guidance I could reference, if advocating
this?  (The WG has been very keen to make this normative.)

Perhaps it would be best to just have an informative section
explaining design decisions made for the sake of making resource
reclamation possible, such as lack of graph introspection.  Perhaps
it would also be helpful to have some informative reminders to
implementations re what GC must not affect.

If the whole normative AudioNode lifetime section were dropped
then this would clearly be an implementation issue rather than a
spec issue.

The history here is:

  Initially the #lifetime-AudioNode section was only informative,
  providing some hints to the implementation re situations when GC
  must not be performed.  These were not a complete list.
  At the same time, implementations had bugs that made GC
  observable, but this was not usually noticed in general Web
  Audio usage.  These implementation bugs were fixable.

  An API/model was designed to support [[dynamic AudioWorkletNode
  lifetime]].  While not directly saying so at the time AFAIK,
  this API/model essentially depends on the implementation bugs to
  work as intended.  This was stated much [[later]].

  WG pressure in response to the [[dynamic AudioWorkletNode
  lifetime]] led to the #lifetime-AudioNode section being made
  [[normative]].  Subsequent additions were made to make it more
  comprehensive.  This made the existing implementation bugs very
  much spec bugs.

  The bugs are now very noticeable in typical AudioWorkletNode
  usage.

[[dynamic AudioWorkletNode lifetime]]
https://github.com/WebAudio/web-audio-api/pull/959#issuecomment-260979231

[[later]]
https://github.com/WebAudio/web-audio-api/issues/1453#issuecomment-349340594

[[normative]]
https://github.com/WebAudio/web-audio-api/pull/1161#issuecomment-284499700
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to implement: AudioWorklet

[hongchan]

It's great to see the intent for the AudioWorklet, but also I can see the GC 
observability is still being discussed here.

Karl, can you open a new spec issue if you think this needs another look from 
AudioWG and TAG?

-Hongchan


On Wednesday, May 2, 2018 at 8:17:30 AM UTC-7, Boris Zbarsky wrote:
> On 5/2/18 5:21 AM, Karl Tomlinson wrote:
> > [[AudioNode Lifetime]] https://github.com/WebAudio/web-audio-api/issues/1471
> 
> I've read through that thread, but I'm still a little unclear on where 
> thing stand.  With the latest proposal, can there be observable 
> situations where the output sound depends on GC timing?
> 
> If so, that doesn't sound acceptable.  It also sounds to me like Alex 
> Russell in 
> https://github.com/WebAudio/web-audio-api/issues/1471#issuecomment-362604396 
> didn't believe there was an observable sound difference possible.  If 
> one is possible, we should communicate this to him very clearly
> 
> -Boris

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to implement: AudioWorklet

[Boris Zbarsky]

On 5/2/18 5:21 AM, Karl Tomlinson wrote:

[[AudioNode Lifetime]] https://github.com/WebAudio/web-audio-api/issues/1471


I've read through that thread, but I'm still a little unclear on where 
thing stand.  With the latest proposal, can there be observable 
situations where the output sound depends on GC timing?


If so, that doesn't sound acceptable.  It also sounds to me like Alex 
Russell in 
https://github.com/WebAudio/web-audio-api/issues/1471#issuecomment-362604396 
didn't believe there was an observable sound difference possible.  If 
one is possible, we should communicate this to him very clearly


-Boris
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to implement: AudioWorklet

[Xidorn Quan]

On Wed, May 2, 2018, at 11:34 PM, Tom Ritter wrote:
> On Wed, May 2, 2018 at 5:11 AM, Robert O'Callahan  
> wrote:
> > On Wed, May 2, 2018 at 9:21 PM, Karl Tomlinson  wrote:
> >
> >> It seems that Chrome works around this by choosing to garbage
> >> collect input nodes even when their presence is specified to
> >> require (observable) AudioWorkletProcessor.process() calls.
> >> This garbage collection is performed in a way that causes the
> >> process() calls to be halted (which stops sound production), and
> >> so the AudioWorkletProcessor can subsequently also be garbage
> >> collected if there are no rooted references, as usual.
> >>
> >> Having the behavior of AudioWorkletProcess depend on whether or
> >> not the client maintains references to input nodes is not
> >> something I'd like to implement.  It would be comparable to an
> >> audio element stopping playback at a time when an implementation
> >> chooses to perform garbage collection after the client has removed
> >> its last reference.  It is contrary to [[TAG design principles]].
> >> The Chrome approach seems to be based on a different understanding
> >> of [[AudioNode Lifetime]].
> >>
> >
> > Making GC timing observable is a big, big problem. I hope you escalate this
> > aggressively with the Chrome team.
> 
> Hey Rob,
> 
> It's not difficult to force a GC in javascript, so I'm curious why
> making the duration of a GC observable is a problem?  Do you think you
> could unpack it for me?

The duration of GC being observable is not a problem. The problem is the 
timing, i.e. when GC is triggered. The main point is that, any accessible 
object should not have observable behavior difference regarding whether GC is 
triggered.

Having that observable is a problem because different implementations trigger 
GC at different time, and even a single implementation may choose to trigger GC 
at different time when they try to adopt some new optimization.

If an accessible object have different behavior depending on whether GC has 
been triggered, web content may start depending on the timing which is 
currently used by some implementation. That can later become a huge webcompat 
nightmare for everyone.

- Xidorn
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to implement: AudioWorklet

[Tom Ritter]

On Wed, May 2, 2018 at 5:11 AM, Robert O'Callahan  wrote:
> On Wed, May 2, 2018 at 9:21 PM, Karl Tomlinson  wrote:
>
>> It seems that Chrome works around this by choosing to garbage
>> collect input nodes even when their presence is specified to
>> require (observable) AudioWorkletProcessor.process() calls.
>> This garbage collection is performed in a way that causes the
>> process() calls to be halted (which stops sound production), and
>> so the AudioWorkletProcessor can subsequently also be garbage
>> collected if there are no rooted references, as usual.
>>
>> Having the behavior of AudioWorkletProcess depend on whether or
>> not the client maintains references to input nodes is not
>> something I'd like to implement.  It would be comparable to an
>> audio element stopping playback at a time when an implementation
>> chooses to perform garbage collection after the client has removed
>> its last reference.  It is contrary to [[TAG design principles]].
>> The Chrome approach seems to be based on a different understanding
>> of [[AudioNode Lifetime]].
>>
>
> Making GC timing observable is a big, big problem. I hope you escalate this
> aggressively with the Chrome team.

Hey Rob,

It's not difficult to force a GC in javascript, so I'm curious why
making the duration of a GC observable is a problem?  Do you think you
could unpack it for me?

-tom
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to implement: AudioWorklet

[Robert O'Callahan]

On Wed, May 2, 2018 at 9:21 PM, Karl Tomlinson  wrote:

> It seems that Chrome works around this by choosing to garbage
> collect input nodes even when their presence is specified to
> require (observable) AudioWorkletProcessor.process() calls.
> This garbage collection is performed in a way that causes the
> process() calls to be halted (which stops sound production), and
> so the AudioWorkletProcessor can subsequently also be garbage
> collected if there are no rooted references, as usual.
>
> Having the behavior of AudioWorkletProcess depend on whether or
> not the client maintains references to input nodes is not
> something I'd like to implement.  It would be comparable to an
> audio element stopping playback at a time when an implementation
> chooses to perform garbage collection after the client has removed
> its last reference.  It is contrary to [[TAG design principles]].
> The Chrome approach seems to be based on a different understanding
> of [[AudioNode Lifetime]].
>

Making GC timing observable is a big, big problem. I hope you escalate this
aggressively with the Chrome team.

Rob
-- 
Su ot deraeppa sah dna Rehtaf eht htiw saw hcihw, efil lanrete eht uoy ot
mialcorp ew dna, ti ot yfitset dna ti nees evah ew; deraeppa efil eht. Efil
fo Drow eht gninrecnoc mialcorp ew siht - dehcuot evah sdnah ruo dna ta
dekool evah ew hcihw, seye ruo htiw nees evah ew hcihw, draeh evah ew
hcihw, gninnigeb eht morf saw hcihw taht.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Intent to implement: AudioWorklet

[Karl Tomlinson]

=Summary/benefits:

"The AudioWorklet object allows developers to supply scripts
 (such as JavaScript or WebAssembly code) to process audio on the
 rendering thread, supporting custom AudioNodes." [[Concepts]]

Allowing scripts to process audio on the rendering thread is
important for low latency general client-side generation or
processing of audio, free from problems of main thread delays.
This is what game developers, for example, have wanted for some
time.  Other parts of the Web Audio API may have been presented in
the past as solutions, but they were not a good fit in general.

A MessageChannel permits, for example, decoding on a web worker
thread and delivery on an audio thread, with no main thread
influence.

Custom AudioNodes would be important as a way for developers to
extend the Web Audio API.  Hopefully we may no longer even need to
add more special-purpose nodes to the API surface.  See, however,
Usability/interoperability concerns below.

Some work has already landed in Gecko, but I'm not aware of a
previous explicit intent to implement.

=Bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=1062849

=Link to standard:
https://webaudio.github.io/web-audio-api/#audioworklet

=Platform coverage:
Desktop + Android.

=Estimated or target release:
When ready,
which may require resolution of GC/interoperability concerns below.

=Preference behind which this will be implemented: 
dom.audioWorklet.enabled and dom.worklet.enabled

=Is this feature enabled by default in sandboxed iframes?
No.
=If not, is there a proposed sandbox flag to enable it?
Yes, "allow-scripts".

=DevTools bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=1458445

=Do other browser engines implement this? 
Blink: since Chrome 66, Opera 53.
https://www.chromestatus.com/feature/4588498229133312
Edge: bug assigned.
https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/15812544/
Webkit: no indication.
https://bugs.webkit.org/show_bug.cgi?id=182506

=web-platform-tests:
https://github.com/w3c/web-platform-tests/tree/master/webaudio/the-audio-api/the-audioworklet-interface

=Secure contexts:
Yes.

=Usability/interoperability concerns:

AudioNodes are often typically set up, scheduled, and forgotten.
Once they have finished what they have been scheduled to do and
upstream nodes have also finished, associated resources can be
reclaimed, but some effects on downstream nodes remain.

AudioWorkletNode, as currently specified, OTOH, is different.
There is provision through an [[active source]] flag for an
AudioWorkletProcessor to indicate that if there are no further
inputs, then it no longer needs to perform further processing.
However, the client needs to disconnect the inputs when finished.
If the input nodes are forgotten (as is typical), then processing
continues repeatedly and indefinitely (unless the whole
AudioContext is stopped).  The need for clients to keep track of
whether inputs to these nodes have finished makes
AudioWorkletNodes with inputs second class nodes in practice.

A solution based on silent input rather than connection count was
proposed in https://github.com/WebAudio/web-audio-api/issues/1453
but this appears to have been rejected.

It seems that Chrome works around this by choosing to garbage
collect input nodes even when their presence is specified to
require (observable) AudioWorkletProcessor.process() calls.
This garbage collection is performed in a way that causes the
process() calls to be halted (which stops sound production), and
so the AudioWorkletProcessor can subsequently also be garbage
collected if there are no rooted references, as usual.

Having the behavior of AudioWorkletProcess depend on whether or
not the client maintains references to input nodes is not
something I'd like to implement.  It would be comparable to an
audio element stopping playback at a time when an implementation
chooses to perform garbage collection after the client has removed
its last reference.  It is contrary to [[TAG design principles]].
The Chrome approach seems to be based on a different understanding
of [[AudioNode Lifetime]].

Because Chrome reclaims CPU and memory resources even when the
client does not disconnect inputs from AudioWorkletNode, authors
are likely to forget to track input nodes, in which case their
applications will have performance problems in implementations
with deterministic behavior.

=Security/privacy concerns:

The audio thread runs with reasonably precise timing, providing a
clock edge.

The additional surface from AudioWorklet is that script can run on
the audio thread, at the precise time, and send messages to other
threads, rather than merely having browser-messages from the audio
thread to the main thread.

This may be mitigated to some extent by slightly reduced precision
from [[AudioIPC]] and perhaps by using tail dispatch for messages
from the audio thread.

[[Concepts]] https://webaudio.github.io/web-audio-api/#AudioWorklet-concepts
[[active source]]