Hi Dirk Thanks for bringing up this discussion. I find it educational.
On Tue, May 2, 2017 at 10:27 PM, Dirk Hogan <dirk.ho...@forgerock.com> wrote: > Hi Neil- > OK - now we are getting somewhere. > > You need to dig a bit deeper to see what establishes a happens-before > relationship. For within a single thread, it is the Program Order: the > underlying virtual machine cannot allow the visibility of memory effects > which contradict the sequence of instructions defined in your .java file, > even though these commands will be re-ordered by the compiler, and > pipelined and executed concurrently by machine cores. > > Between threads, it is the Synchronization Order, which is NOT Program > Order. Between threads, the memory effects of your .java file writes/reads > do NOT have to be honored. So if you say 'But Felix writes the reference > before publishing the service', and thus concluding that the reference will > always be present in the service, you are correct, but ONLY from the > perspective of the thread which wrote the reference and published the > service. For ALL other threads, all bets are off, unless there is a > Synchronization Action (volatile reference read/write, monitor lock/unlock > on the same monitor) which dictates a Synchronization Order of the > visibility of memory effects between DISTINCT threads. I read some of your references and according to my understanding you are disregarding the transitive nature of different orderings here. Let me explain: - assume we have a component C with a field f (annotated with @Reference) - assume C is a delayed component providing a service - assume the component is satisfied and its corresponding ServiceFactory sf is registered with the service-registry - assume no other thread has requested the service before - thread T1 requests C from the service registry - thread T2 requests C from the service registry (assume T1 wins the race) - T2: waits for a CountDownLatch (that's how it's implemented in Felix) - T1: SCR creates an instance of C called c - T1: SCR injects @Reference fields, i.e. it sets c.f - T1: SCR calls c.activate() - T1: c is published by setting it in a volatile field of the ServiceFactory: sf.componentInstance - T1: "releases" the CountDownLatch allowing T2 to proceed - T2: reads c from sf.componentInstance - T2: reads c.f All actions that happen in T1 are governed by Program Order. Setting sf.componentInstance is visible across threads due to Synchronization Order (it is a volatile field). Because actions in T1 are bound by PO, the field sf.componentInstance is set AFTER c.f is set. T2 can only get a reference to c AFTER sf.componentInstance is set. Therefore, transitively, T2 can only get a reference to c AFTER c.f is set. This is in line with my understanding of the following quote from https://shipilev.net/blog/2014/jmm-pragmatics/, at the end of the section "Happens-Before: Publication": "Release can be thought of as the operation which releases all the preceding state updates into the wild, and acquire is the paired operation which receives those updates. So, after a successful acquire, we can see all the updates preceding the paired release." Regards Julian > > The definition of Synchronization Actions, how they combine to create a > Synchronization Order, and how this, in turn, allows a happens-before > relationship to be established, is a formal set of rules defined in the > JMM. There is no ambiguity here. A read of section 17.4 will illuminate as > much. It is very dense - that is why I turned to the other references I > provided in the links above. They are not a quick/easy read, but are > necessary to coherently reason about memory visibility across threads. > > Thanks > > Dirk > > On Tue, May 2, 2017 at 1:10 PM, Neil Bartlett <njbartl...@gmail.com> wrote: > >> I think I get what Dirk is saying, and although I disagree with some of >> his analysis there is arguably a gap in the DS specification that makes >> this unclear. I hope somebody can point at the vital sentence I have >> missed, but here is my analysis: >> >> The JLS defines happens-before thus: "Two actions can be ordered by a >> happens-before relationship. If one action happens-before another, then the >> first is visible to and ordered before the second.” ( >> https://docs.oracle.com/javase/specs/jls/se7/html/jls-17.html#jls-17.4.5 < >> https://docs.oracle.com/javase/specs/jls/se7/html/jls-17.html#jls-17.4.5 >> >). >> >> Define two actions, A1 and A2. A1 is the setting of an @Reference field in >> a component. A2 is reading the value of that field. It is Dirk’s contention >> that no provable happens-before relationship edge exists between A1 and A2, >> and therefore it’s possible for A2 to access the value of the @Reference >> field that obtained before A1. Let’s see. >> >> A1 is an action that is taken by SCR on an object that it creates (the >> component instance). A2 can only occur in the following circumstances: >> >> 1. SCR itself reads the field. >> SCR is in full control of its own code and I am assuming a correct SCR >> implementation. >> >> 2. The activate method of the component uses the value of the field. >> Section 112.3.8.1 of OSGi Compendium R6 states "there is a happens-before >> relationship between setting the field and activating the component >> instance”. Also section 112.3.6.1 states “The bind method is called and/or >> the field is set before the component instance is activated”, which is less >> clear about the happens-before but I think the intent is there, and it also >> clarifies that this applies to bind methods as well as injected fields. >> Therefore action A1 happens-before A2 in this case. >> >> Note that the spec does not tell the implementation HOW to achieve this >> happens-before edge. As Dirk points out, if it is done with a synchronized >> block then it would have to be on the same monitor. For example, the >> implementation can achieve this by synchronizing on the component instance >> itself when binding fields and calling the activate method. >> >> 3. The activate method starts a thread which uses the value of the field >> (I think this maps to Dirk’s HTTP-serving example). >> From point 2 above, we know that A1 happens-before the activate method is >> invoked. JLS 17.4.5 states that “a call to start() on a thread >> happens-before any actions in the started thread”. JLS also states that >> happens-before is transitive, i.e. if X happens-before Y and Y >> happens-before Z then X happens-before Z. >> >> Therefore A1 happens-before A2 in this case. >> >> 4. Service method(s) on the component use the value of the field. >> The analysis here diverges for immediate vs delayed components. >> >> For immediate components, Compendium section 112.5.3 states: "If an >> immediate component configuration is satisfied and specifies a service, SCR >> must register the component configuration as a service in the service >> registry and then activate the component configuration”. This does NOT >> state that the registration of the service must happen after the binding of >> the reference fields. So it seems an implementation is free to hand the >> service object to a consumer before the static references are bound. This >> surprised me. >> >> For delayed components, Compendium section 112.5.4 states: "When the >> service is requested … SCR must create and activate a [unique] component >> configuration”. It does NOT explicitly state that the static references >> must be bound before the service object is handed to the caller of >> getService(). Again this surprised me. >> >> >> In summary, it’s clear to me that the *intention* of the specification is >> for static references to be bound before the service becomes accessible, >> and that component writers should not have to declare those fields >> volatile, but this intention is not adequately spelled out. The best >> statement I could find is in section 112.3.6.1: "A component instance >> never sees any of the dynamics of the static reference”, which is very >> vague. >> >> Again, I may well have missed something or made an error in my analysis. I >> look forward to be being corrected. >> >> >> Neil >> >> >> > On 2 May 2017, at 19:56, David Jencks <david.a.jen...@gmail.com> wrote: >> > >> > I don’t understand your example or I don’t understand how your example >> relates to an osgi or ds problem. >> > >> > I think you are proposing that the DS component is an http server and >> starts threads etc. If so, it’s that components responsibility to make >> sure its’ internal operations are thread safe, and this has little to do >> with osgi or ds. >> > >> > If you are asking about osgi, for instance if the component exposes >> HttpServlet and is registered with the whiteboard http service and receives >> calls from it, then the whiteboard had to call getService to get the DS >> component instance. In this case I think that there are plenty of >> synchronization barriers in the service registry and in e.g. >> SingleComponentManager.getService to assure that the view a request >> thread has of the component is the same as the view the activation thread >> had. >> > >> > I could be wrong about whether there are actually enough synchronization >> barriers. Have you looked at the code to see? >> > >> > thanks >> > david jencks >> > >> >> On May 2, 2017, at 10:45 AM, Dirk Hogan <dirk.ho...@forgerock.com> >> wrote: >> >> >> >> Hi Raymond- >> >> Assume Felix does it's thing: sets all @References, activates all >> >> components, publishes all services. It has resolved all dependencies, >> and >> >> set references accordingly. >> >> >> >> Thus, in the example above, myOSGiComponent has its @Reference to >> >> SomeOtherOSGiService resolved - Felix has set this reference. At the >> >> conclusion of the activate method, myOSGiComponent has exposed some >> >> functionality via http, even successfully dereferenced myOSGiComponent, >> and >> >> now requests start to come it, serviced by the http-server's >> thread-pool. >> >> Part of servicing this request involves dereferencing (reading) the >> >> @Reference myServiceReference, which, I agree has been set (written) by >> a >> >> Felix thread. >> >> >> >> The point of the Java Memory Model is that memory state, including >> >> references, read and written across threads, do not have to have >> >> determinate state, unless there is a Synchronization Action which >> ensures >> >> that the state written by thread A is visible to any other thread - a >> >> volatile reference provides such a Synchronization Action. So code in >> >> myOSGiComponent activate method can successfully dereference the >> >> SomeOtherOSGiService reference, because the same thread which set the >> >> reference is calling activate. But this does NOT mean that another >> thread, >> >> reading the SAME reference, will see a properly initialized reference. >> >> >> >> Again, this should be self-evident, and the various rules of the JMM >> model >> >> are defined in section 17.4 of the Java Language Specification. If it is >> >> not clear, then an understanding if the JMM is required. This >> understanding >> >> can be gained by reading the JLS, reading the links above, the chapter >> >> in *Concurrency >> >> in Practice* on the JMM, or http://gee.cs.oswego.edu/dl/cpj/jmm.html >> >> provides a good intro. >> >> >> >> The bottom line: memory state, including references, shared among >> multiple >> >> threads can have indeterminate state when written/read, unless explicit >> >> steps, defined in the JMM, are taken to ensure this visibility and thus >> >> coherent state. This is not an opinion. >> >> >> >> Thanks >> >> >> >> Dirk >> >> >> >> >> >> >> >> On Tue, May 2, 2017 at 10:23 AM, Raymond Auge <raymond.a...@liferay.com >> > >> >> wrote: >> >> >> >>> On Tue, May 2, 2017 at 1:18 PM, Raymond Auge <raymond.a...@liferay.com >> > >> >>> wrote: >> >>> >> >>>> >> >>>> >> >>>> On Tue, May 2, 2017 at 1:14 PM, Raymond Auge < >> raymond.a...@liferay.com> >> >>>> wrote: >> >>>> >> >>>>> >> >>>>> On Tue, May 2, 2017 at 11:46 AM, Dirk Hogan < >> dirk.ho...@forgerock.com> >> >>>>> wrote: >> >>>>> >> >>>>>> But no similar guarantee applies when another thread hits the >> >>>>>> component, >> >>>>>> >> >>>>> >> >>>>> I believe the discussion boils down to why you believe the above is >> >>>>> possible? >> >>>>> >> >>>>> If SCR has not yet made the component available to other threads >> (which, >> >>>>> to my knowledge, it only does by registering it as a service to the >> >>> service >> >>>>> registry) how can _another_ thread access the component at all? >> >>>>> >> >>>>> I don't even see a way for components within the same DS bundle to >> >>>>> interact with each other other than via the service registry because >> >>>>> @Reference only works through the registry. >> >>>>> >> >>>> >> >>>> Correction, components could interact with each other via the >> >>>> ComponentContext.. but I guess we'd have to know if that's part of >> your >> >>> use >> >>>> case. >> >>>> >> >>> >> >>> Actually, no, it's not even possible via the ComponentContext. It only >> >>> allows access to Services. >> >>> >> >>>> >> >>>> - Ray >> >>>> >> >>>> >> >>>>> >> >>>>> Please, can you explain in more detail what case might allow for the >> >>>>> above? Because I believe the assumption is that it's not currently >> >>>>> possible. Certainly there could be a bug and I'm certain everyone >> would >> >>>>> want to see that fixed. >> >>>>> >> >>>>> -- >> >>>>> *Raymond Augé* <http://www.liferay.com/web/raymond.auge/profile> >> >>>>> (@rotty3000) >> >>>>> Senior Software Architect *Liferay, Inc.* <http://www.liferay.com> >> >>>>> (@Liferay) >> >>>>> Board Member & EEG Co-Chair, OSGi Alliance <http://osgi.org> >> >>>>> (@OSGiAlliance) >> >>>>> >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> *Raymond Augé* <http://www.liferay.com/web/raymond.auge/profile> >> >>>> (@rotty3000) >> >>>> Senior Software Architect *Liferay, Inc.* <http://www.liferay.com> >> >>>> (@Liferay) >> >>>> Board Member & EEG Co-Chair, OSGi Alliance <http://osgi.org> >> >>>> (@OSGiAlliance) >> >>>> >> >>> >> >>> >> >>> >> >>> -- >> >>> *Raymond Augé* <http://www.liferay.com/web/raymond.auge/profile> >> >>> (@rotty3000) >> >>> Senior Software Architect *Liferay, Inc.* <http://www.liferay.com> >> >>> (@Liferay) >> >>> Board Member & EEG Co-Chair, OSGi Alliance <http://osgi.org> >> >>> (@OSGiAlliance) >> >>> >> > >> >>
