Re: Should web specifications try to describe object lifetimes? [was Intent to implement: AudioWorklet]

[Karl Tomlinson]

On Fri, 4 May 2018 14:32:20 -0400, Boris Zbarsky wrote:

> On 5/4/18 3:34 AM, Karl Tomlinson wrote:
>> Not sure I understand your question, but the observable behavior
>> described by this section, or specifically "Before an AudioNode is
>> deleted, it will disconnect itself from any other AudioNodes"
>
> Right.  So that observably influences the sound produced or some
> such, sounds like?

That's correct.  The connections influence the behavior of the
"other AudioNodes".  Depending on how the rest of the graph is
assembled, the behavior change may or may not be heard.  There are
some graph designs where the behavior change will be heard.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Should web specifications try to describe object lifetimes? [was Intent to implement: AudioWorklet]

[Boris Zbarsky]

On 5/4/18 3:34 AM, Karl Tomlinson wrote:

Not sure I understand your question, but the observable behavior
described by this section, or specifically "Before an AudioNode is
deleted, it will disconnect itself from any other AudioNodes"


Right.  So that observably influences the sound produced or some such, 
sounds like?



Yes, the source of the observable GC problem is very much in the
spec now, and tracked by the [[AudioNode Lifetime]] issue.


Understood.

-Boris
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Should web specifications try to describe object lifetimes? [was Intent to implement: AudioWorklet]

[Karl Tomlinson]

Boris Zbarsky writes:

> So if there is in fact a problem still remaining, convincing Alex
> that it's there is pretty valuable.  Having him convinced it
> doesn't exist is actively harmful as far as getting it fixed goes.

Thank you.  Alex sounds like a very valuable person to get on side.

We may still have some time on our hands, but I'm not sure how long
to pursue the WG about this before we aim to get Alex involved.

Neither Chrome 66 nor current Chromium [[processor source code]]
support AudioWorkletNodes with lifetimes based on input
connections in a way that is compatible with the spec and so
Chrome or the spec will change at some point.

It would be nice, however, to resolve these issues before Chrome
ships such changes.

Hongchan has assigned himself to the [[AudioNode Lifetime]] spec
bug, and so I'm happy for him to have a go, if he is planning to
fix it.

> If there is no difference in observable behavior, what would a
> normative requirement mean?

Good question.  I will find that useful at some point.

> Karl Tomlinson wrote:
>> If the whole normative AudioNode lifetime section were dropped
>> then this would clearly be an implementation issue rather than a
>> spec issue.
>
> Assuming there is no observable behavior involved (other than
> memory usage), right?

Not sure I understand your question, but the observable behavior
described by this section, or specifically "Before an AudioNode is
deleted, it will disconnect itself from any other AudioNodes", is
the source of observable behavior on GC.  Removing this section
would remove the core problem.

The rest of the section describes conditions that would keep an
AudioNode alive, and describes when AudioNodes are no longer
required.  These portions are not _necessary_, as nothing
elsewhere indicates that AudioNodes should ever be removed, but
there seem to be differing opinions on the importance of these
clauses.  (Compare e.g. [[PermissionStatus listeners]],
[[WebSocket listeners]])

>> The history here is:
>
> Thank you for the summary.  That was helpful.  One thing I'm not
> 100% clear on at this point: are there spec issues that can lead
> to observable GC, or are we just talking about implementation bugs
> that make GC observable now?

Yes, the source of the observable GC problem is very much in the
spec now, and tracked by the [[AudioNode Lifetime]] issue.

Once that is resolved, I'm hoping that a spec change to support
something like [[silence tracking]] can make AudioWorklet suitable
for custom AudioNodes.

[[processor source code]]
https://chromium.googlesource.com/chromium/src/+blame/9d4a34cadf1b7d85f4ad0b709374aaaf59325a8d/third_party/WebKit/Source/modules/webaudio/AudioWorkletNode.cpp#110

[[PermissionStatus listeners]]
https://github.com/w3c/permissions/issues/170

[[WebSocket listeners]]
https://html.spec.whatwg.org/#garbage-collection-3

[[AudioNode Lifetime]]
https://github.com/WebAudio/web-audio-api/issues/1471

[[silence tracking]]
https://github.com/WebAudio/web-audio-api/issues/1453
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Should web specifications try to describe object lifetimes? [was Intent to implement: AudioWorklet]

[Boris Zbarsky]

On 5/3/18 6:21 PM, Karl Tomlinson wrote:

I didn't understand why he was highlighting "order of object
tear-down", nor why he was implying that only "VERY fine-grained"
knowledge was a problem.


Alex, depending on whether he's speaking with his TAG hat or Google hat 
on, is either trying to figure out whether there is a problem with the 
spec or trying to justify Google shipping a particular behavior.  I'm 
not quite sure which hat he's wearing here.



He does seem to have misunderstood that there is a problem to fix.
But I assume the way forward here is to help the WG find a
solution that works.  What is the advantage of explaining the
situation to Alex?


As a TAG member, he can work with the TAG to explain to the WG that 
their current spec is not acceptable, assuming he's convinced on that.


With his Google hat on, he can make similar arguments internally about 
Chrome's implementation.


So if there is in fact a problem still remaining, convincing Alex that 
it's there is pretty valuable.  Having him convinced it doesn't exist is 
actively harmful as far as getting it fixed goes.



Should specifications instead just focus on observable behavior,
and leave it to implementations to optimize and to reclaim
resources that are no longer required?


Generally speaking, yes.  That said, specifications should be really 
clear on which resources are no longer required, if possible.



Are you aware of any guidance I could reference, if advocating
this?  (The WG has been very keen to make this normative.)


If there is no difference in observable behavior, what would a normative 
requirement mean?  And if there _is_ a difference in observable 
behavior, then of course the actual observable behavior that 
implementors need to implement needs to be specced normatively; this 
doesn't require saying anything about GC.



Perhaps it would be best to just have an informative section
explaining design decisions made for the sake of making resource
reclamation possible, such as lack of graph introspection.


This would definitely be useful.


Perhaps
it would also be helpful to have some informative reminders to
implementations re what GC must not affect.


Yes, that would be helpful.


If the whole normative AudioNode lifetime section were dropped
then this would clearly be an implementation issue rather than a
spec issue.


Assuming there is no observable behavior involved (other than memory 
usage), right?



The history here is:


Thank you for the summary.  That was helpful.  One thing I'm not 100% 
clear on at this point: are there spec issues that can lead to 
observable GC, or are we just talking about implementation bugs that 
make GC observable now?


-Boris
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Should web specifications try to describe object lifetimes? [was Intent to implement: AudioWorklet]

[Robert O'Callahan]

I read the threads you referenced and the latest spec, and I think you're
absolutely right about everything :-).

On Fri, May 4, 2018 at 10:21 AM, Karl Tomlinson  wrote:

> Thank you for taking a look, Boris.  I'm quite unclear how any of
> the changes proposed in the [[March F2F resolution]] comment would
> resolve the issue, and so I expect not.
>
> [[March F2F resolution]]
> https://github.com/WebAudio/web-audio-api/issues/1471#
> issuecomment-376223916


I wonder what that resolution entails since the spec changes haven't
happened yet.

However, it sounds like it's heading in the wrong direction. Rather than
trying to fix the description of AudioNode lifetimes, the entire section
should be deleted.

He does seem to have misunderstood that there is a problem to fix.
> But I assume the way forward here is to help the WG find a
> solution that works.  What is the advantage of explaining the
> situation to Alex?
>

AIUI If you can't convince someone on the WG that there is problem to fix,
but you can convince someone on the TAG, then they can require the WG to
fix it.

I'm happy to query the F2F resolution, but I wonder which is the
> best way to resolve this.
>
> Is having web specifications try to describe object lifetimes
> helpful, or is it just over-prescribing?
>

I think it's very dangerous. If those descriptions are normative then it's
easy to accidentally make GC observable. If they're not normative then they
confuse implementors and users into accidentally treating them as normative.

Should specifications instead just focus on observable behavior,
> and leave it to implementations to optimize and to reclaim
> resources that are no longer required?
>

Yes. People need to understand that GC is purely an optimization.

Perhaps it would be best to just have an informative section
> explaining design decisions made for the sake of making resource
> reclamation possible, such as lack of graph introspection.  Perhaps
> it would also be helpful to have some informative reminders to
> implementations re what GC must not affect.
>

Yes.


> If the whole normative AudioNode lifetime section were dropped
> then this would clearly be an implementation issue rather than a
> spec issue.
>

Yes!

Maybe I can help by writing a blog post or something...

Rob
-- 
Su ot deraeppa sah dna Rehtaf eht htiw saw hcihw, efil lanrete eht uoy ot
mialcorp ew dna, ti ot yfitset dna ti nees evah ew; deraeppa efil eht. Efil
fo Drow eht gninrecnoc mialcorp ew siht - dehcuot evah sdnah ruo dna ta
dekool evah ew hcihw, seye ruo htiw nees evah ew hcihw, draeh evah ew
hcihw, gninnigeb eht morf saw hcihw taht.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Should web specifications try to describe object lifetimes? [was Intent to implement: AudioWorklet]

[L. David Baron]

On Friday 2018-05-04 10:21 +1200, Karl Tomlinson wrote:
> Is having web specifications try to describe object lifetimes
> helpful, or is it just over-prescribing?
> 
> Should specifications instead just focus on observable behavior,
> and leave it to implementations to optimize and to reclaim
> resources that are no longer required?
> 
> Are you aware of any guidance I could reference, if advocating
> this?  (The WG has been very keen to make this normative.)

Without digging too far into this specific case, I can attempt to
offer some answers to the general questions:

There is design guidance here, in a document published by the W3C
Technical Architecture Group:
https://w3ctag.github.io/design-principles/#js-gc
that garbage collection should not be observable.

Ensuring that GC isn't observable sometimes does require normative
descriptions of minimum object lifetimes, such as in
https://github.com/w3c/permissions/issues/170 , but generally
normative descriptions of maximum object lifetimes shouldn't be
needed.

That said, in general it's useful for specifications to give
informative explanations of the design decisions that led to the
current specification (since that helps future maintenance of the
specification, and also likely helps implementors contribute to that
maintenance).

It's also generally useful for specifications to give informative
implementation advice when there are things worth pointing out that
an implementor might not realize immediately.

I'd like to specifications have both sorts of informative sections
substantially more often than they do today.

> Perhaps it would be best to just have an informative section
> explaining design decisions made for the sake of making resource
> reclamation possible, such as lack of graph introspection.  Perhaps
> it would also be helpful to have some informative reminders to
> implementations re what GC must not affect.

These both sound useful, as I mentioned above.

-David

-- 
턞   L. David Baron http://dbaron.org/   턂
턢   Mozilla  https://www.mozilla.org/   턂
 Before I built a wall I'd ask to know
 What I was walling in or walling out,
 And to whom I was like to give offense.
   - Robert Frost, Mending Wall (1914)


signature.asc
Description: PGP signature
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Should web specifications try to describe object lifetimes? [was Intent to implement: AudioWorklet]

[Karl Tomlinson]

Boris Zbarsky writes:

> On 5/2/18 5:21 AM, Karl Tomlinson wrote:
>> [[AudioNode Lifetime]] https://github.com/WebAudio/web-audio-api/issues/1471
>
> I've read through that thread, but I'm still a little unclear on
> where thing stand.  With the latest proposal, can there be
> observable situations where the output sound depends on GC timing?

Thank you for taking a look, Boris.  I'm quite unclear how any of
the changes proposed in the [[March F2F resolution]] comment would
resolve the issue, and so I expect not.

[[March F2F resolution]]
https://github.com/WebAudio/web-audio-api/issues/1471#issuecomment-376223916

> If so, that doesn't sound acceptable.  It also sounds to me like
> Alex Russell in
> https://github.com/WebAudio/web-audio-api/issues/1471#issuecomment-362604396
> didn't believe there was an observable sound difference possible.
> If one is possible, we should communicate this to him very
> clearly

The spec says that when a node is deleted, it will disconnect
itself from other nodes.  This disconnection definitely can lead
to observable sound differences, if the node can be deleted.  The
spec is unclear on whether a node with observable connections can
be deleted, but there is certainly reason to interpret it in this
way and members of the working group were doing so.  If anything,
the March F2F resolution seems to reinforce this view.

I was also quite unclear as to the point of Alex's comment.
I didn't understand why he was highlighting "order of object
tear-down", nor why he was implying that only "VERY fine-grained"
knowledge was a problem.

I'm not clear as to exactly what he was referring with
"@joeberkovitz's change to cause the spec to not discuss GC", but
changing the spec to not discuss GC would certainly resolve the
issue.  My take-away was that Alex was advocating no behavior
changes on GC.

He does seem to have misunderstood that there is a problem to fix.
But I assume the way forward here is to help the WG find a
solution that works.  What is the advantage of explaining the
situation to Alex?

I'm happy to query the F2F resolution, but I wonder which is the
best way to resolve this.

Is having web specifications try to describe object lifetimes
helpful, or is it just over-prescribing?

Should specifications instead just focus on observable behavior,
and leave it to implementations to optimize and to reclaim
resources that are no longer required?

Are you aware of any guidance I could reference, if advocating
this?  (The WG has been very keen to make this normative.)

Perhaps it would be best to just have an informative section
explaining design decisions made for the sake of making resource
reclamation possible, such as lack of graph introspection.  Perhaps
it would also be helpful to have some informative reminders to
implementations re what GC must not affect.

If the whole normative AudioNode lifetime section were dropped
then this would clearly be an implementation issue rather than a
spec issue.

The history here is:

  Initially the #lifetime-AudioNode section was only informative,
  providing some hints to the implementation re situations when GC
  must not be performed.  These were not a complete list.
  At the same time, implementations had bugs that made GC
  observable, but this was not usually noticed in general Web
  Audio usage.  These implementation bugs were fixable.

  An API/model was designed to support [[dynamic AudioWorkletNode
  lifetime]].  While not directly saying so at the time AFAIK,
  this API/model essentially depends on the implementation bugs to
  work as intended.  This was stated much [[later]].

  WG pressure in response to the [[dynamic AudioWorkletNode
  lifetime]] led to the #lifetime-AudioNode section being made
  [[normative]].  Subsequent additions were made to make it more
  comprehensive.  This made the existing implementation bugs very
  much spec bugs.

  The bugs are now very noticeable in typical AudioWorkletNode
  usage.

[[dynamic AudioWorkletNode lifetime]]
https://github.com/WebAudio/web-audio-api/pull/959#issuecomment-260979231

[[later]]
https://github.com/WebAudio/web-audio-api/issues/1453#issuecomment-349340594

[[normative]]
https://github.com/WebAudio/web-audio-api/pull/1161#issuecomment-284499700
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform