Re: Endpoint registry and stopping/starting nodes
...snip seems quite fragile, In what way? if it wouldn't be more robust to do the earlier suggestion of just having the sca binding look up the endpoint from the endpoint registry on each invocation. No I don't think so. In distributed environments you will *always* have to deal with the error case. See fallacies 1 and 2 [1]. If you were to rely solely on the registry and the registry tells you that the endpoint hasn't change is this true or has the registry just not caught up with real changes out there. Simon [1] http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Computing
Re: Endpoint registry and stopping/starting nodes
On Fri, Aug 28, 2009 at 8:59 AM, Simon Lawssimonsl...@googlemail.com wrote: ...snip seems quite fragile, In what way? if it wouldn't be more robust to do the earlier suggestion of just having the sca binding look up the endpoint from the endpoint registry on each invocation. No I don't think so. In distributed environments you will *always* have to deal with the error case. See fallacies 1 and 2 [1]. If you were to rely solely on the registry and the registry tells you that the endpoint hasn't change is this true or has the registry just not caught up with real changes out there. Simon [1] http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Computing Exactly. Whos to say the registry will have caught up by the 2nd attempt? Should there be several retries? A configurable delay between retries? Attempting to deal with it adds significant complexity whereas just doing the very fast local registry lookup on each request is simple and likely will have the same result. ...ant
Re: Endpoint registry and stopping/starting nodes
On Fri, Aug 28, 2009 at 9:39 AM, Simon Lawssimonsl...@googlemail.com wrote: is simple and likely will have the same result. The same result being that you need a strategy for dealing with errors. Simon I meant just fail by passing the error back up to the client instead of trying to do anything clever. It doesn't seem so unreasonable to have requests fail that happen to coincide with while a node is being restarted. ...ant
Re: Endpoint registry and stopping/starting nodes
On Fri, Aug 28, 2009 at 9:43 AM, ant elderantel...@apache.org wrote: On Fri, Aug 28, 2009 at 9:39 AM, Simon Lawssimonsl...@googlemail.com wrote: is simple and likely will have the same result. The same result being that you need a strategy for dealing with errors. Simon I meant just fail by passing the error back up to the client instead of trying to do anything clever. It doesn't seem so unreasonable to have requests fail that happen to coincide with while a node is being restarted. ...ant Actually, couldn't we do something slightly more clever than the current retry fix in JDKInvocationHandler by checking that reference target endpoint is still the same as the one in the registry before the invoke and if not then doing the wire rebuild before the invoke instead of waiting till after the request fails? ...ant
Re: Endpoint registry and stopping/starting nodes
personally I'd rather... Catch the error look to see if the registry has a new endpoint if it has use it if it hasn't throw the error However this is not a big piece of code so there is plenty of scope for us to experiment. I would go ahead and do whatever you prefer for the time being and at least we will have something to play with. Simon
Re: Endpoint registry and stopping/starting nodes
On Fri, Aug 28, 2009 at 10:06 AM, Simon Lawssimonsl...@googlemail.com wrote: personally I'd rather... Catch the error look to see if the registry has a new endpoint if it has use it if it hasn't throw the error However this is not a big piece of code so there is plenty of scope for us to experiment. I would go ahead and do whatever you prefer for the time being and at least we will have something to play with. Simon Ok done in r808838 which adds a way to check if the wire taget endpoint reference is out of date comapred to the endpoint registry and if so does a wire rebuild. ...ant
Re: Endpoint registry and stopping/starting nodes
2009/8/27 ant elder ant.el...@gmail.com: I've added a testcase in r808333 for using multiple nodes and having a service node stopped and replaced with a new node running the service, this should work but doesn't as the client node keeps using the references for the original node. I'm wondering how to fix this, it seems like the runtime should be looking up the endpoint from the registry on each invocation. It's exactly what happens from 1.1 up to know and it's a penalty kick for performance. At that time i modified the node to avoid it, caching most recently used endpoints. Avoiding resolution every time. Doing resolution every time at lower level could mean open a socket towards the domain, so handling tcp, processing soap, and so on. It worked well in my workpool-demo apps because the working set of endpoint it was pretty the same, it may not work in the general case. Cheers, Giorgio. -- Quiero ser el rayo de sol que cada día te despierta para hacerte respirar y vivir en me. Favola -Moda.
Re: Endpoint registry and stopping/starting nodes
On Thu, Aug 27, 2009 at 10:36 AM, Giorgio Zoppigiorgio.zo...@gmail.com wrote: 2009/8/27 ant elder ant.el...@gmail.com: I've added a testcase in r808333 for using multiple nodes and having a service node stopped and replaced with a new node running the service, this should work but doesn't as the client node keeps using the references for the original node. I'm wondering how to fix this, it seems like the runtime should be looking up the endpoint from the registry on each invocation. It's exactly what happens from 1.1 up to know and it's a penalty kick for performance. At that time i modified the node to avoid it, caching most recently used endpoints. Avoiding resolution every time. Doing resolution every time at lower level could mean open a socket towards the domain, so handling tcp, processing soap, and so on. It worked well in my workpool-demo apps because the working set of endpoint it was pretty the same, it may not work in the general case. Cheers, Giorgio. I was thinking it would be something like the EndpointRegistry impl managing its local copy of the endpoints it knows about and the sca binding invoker on each invocation looking up the endpoint from the registry. That way the binding lookup is still a quick local call and its only the EndpointRegistry impl that has to handle keeping its local cache up to date. Does that sound ok? ...ant
Re: Endpoint registry and stopping/starting nodes
Way back in the mists of time what the default binding used to do was trap any target not found type exceptions and re-resolve the endpoint in that case. Worked OK, easy to do and doesn't lead to a resolution on each call. If you catch the exception, set the EndpointReference status back to EndpointReference.WIRED_TARGET_NOT_FOUND and then call EndpointReferenceBuilderImpl.runtimeBuild(). If the result of that is an endpoint reference in EndpointReference.WIRED_TARGET_FOUND_AND_MATCHED status then retry the call. If not then throw a runtime exception. That should do the trick. Simon
Re: Endpoint registry and stopping/starting nodes
On Thu, Aug 27, 2009 at 11:01 AM, Simon Lawssimonsl...@googlemail.com wrote: Way back in the mists of time what the default binding used to do was trap any target not found type exceptions and re-resolve the endpoint in that case. Worked OK, easy to do and doesn't lead to a resolution on each call. If you catch the exception, set the EndpointReference status back to EndpointReference.WIRED_TARGET_NOT_FOUND and then call EndpointReferenceBuilderImpl.runtimeBuild(). If the result of that is an endpoint reference in EndpointReference.WIRED_TARGET_FOUND_AND_MATCHED status then retry the call. If not then throw a runtime exception. That should do the trick. Simon I should have been a bit clearer about where this needs to happen. Firstly down in the binding invoker you'll need to catch the protocol specific exception and then throw something (an endpoint reference no longer valid exception) back up to be caught in the RuntimeWire.invoke(). From there you can reset the invocation chains using initInvocationChains(). This in turn will call the EndpointReferenceBuilderImpl.runtimeBuild() for you. Simon
Re: Endpoint registry and stopping/starting nodes
Nothing is throwing any target not found type exceptions. And It's currently an AxisFault. To make this make sense in this special binding.sca case you'll have to reinstate the binding.sca invoker [1] to trap the exception and wrap it with something that the RuntimeWire can spot. Simon [1] http://svn.apache.org/repos/asf/tuscany/branches/sca-java-1.x/modules/binding-sca-axis2/src/main/java/org/apache/tuscany/sca/binding/sca/axis2/impl/Axis2SCABindingInvoker.java
Re: Endpoint registry and stopping/starting nodes
On Thu, Aug 27, 2009 at 11:19 AM, Simon Lawssimonsl...@googlemail.com wrote: Nothing is throwing any target not found type exceptions. And It's currently an AxisFault. To make this make sense in this special binding.sca case you'll have to reinstate the binding.sca invoker [1] to trap the exception and wrap it with something that the RuntimeWire can spot. Simon In the testcase this is all local with two nodes within the single jvm so using the local sca binding not the WS based one so there's no AxisFault. ...ant
Re: Endpoint registry and stopping/starting nodes
On Thu, Aug 27, 2009 at 11:29 AM, ant elderantel...@apache.org wrote: On Thu, Aug 27, 2009 at 11:19 AM, Simon Lawssimonsl...@googlemail.com wrote: Nothing is throwing any target not found type exceptions. And It's currently an AxisFault. To make this make sense in this special binding.sca case you'll have to reinstate the binding.sca invoker [1] to trap the exception and wrap it with something that the RuntimeWire can spot. Simon In the testcase this is all local with two nodes within the single jvm so using the local sca binding not the WS based one so there's no AxisFault. ...ant To make this a bit clearer, this is the stack trace of the reference invocation just before it fails: Thread [main] (Suspended (breakpoint at line 42 in StatelessScopeContainer)) StatelessScopeContainerKEY.getWrapper(KEY) line: 42 JavaImplementationInvoker.invoke(Message) line: 100 PassByValueInterceptor.invoke(Message) line: 61 SCABindingInvoker.invoke(Message) line: 61 PassByValueInterceptor.invoke(Message) line: 61 JDKInvocationHandler.invoke(InvocationChain, Object[], RuntimeWire, EndpointReference) line: 277 JDKInvocationHandler.invoke(Object, Method, Object[]) line: 142 $Proxy10.sayHello(String) line: not available HelloworldImpl.sayHello(String) line: 40 NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method] NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39 DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25 Method.invoke(Object, Object...) line: 597 JavaImplementationInvoker.invoke(Message) line: 136 PassByValueInterceptor.invoke(Message) line: 61 SCABindingInvoker.invoke(Message) line: 61 PassByValueInterceptor.invoke(Message) line: 61 JDKInvocationHandler.invoke(InvocationChain, Object[], RuntimeWire, EndpointReference) line: 277 JDKInvocationHandler.invoke(Object, Method, Object[]) line: 142 $Proxy10.sayHello(String) line: not available StopStartNodesTestCaseFIXME.testCalculator() line: 76 That scope container at the top is actually in a STOPPED state but nothing in the getWrapper call path checks that so eventually it gets an NPE. We could (should) change that so it notices the scope container is stopped and throws an exception. The SCABindingInvoker.invoke call could then catch that exception and do something, I guess thats where you're suggesting inserting the initInvocationChains call. If the service node has been moved from being local to be remote we actually need to change the SCABindingInvoker impl from being the local sca binding to be the remote one. I wonder if we need a new generic SCA binding impl that can handle both local and remote invocations based on whether the endpoint is local or remote. ...ant
Re: Endpoint registry and stopping/starting nodes
On Thu, Aug 27, 2009 at 12:01 PM, ant elderantel...@apache.org wrote: On Thu, Aug 27, 2009 at 11:29 AM, ant elderantel...@apache.org wrote: On Thu, Aug 27, 2009 at 11:19 AM, Simon Lawssimonsl...@googlemail.com wrote: Nothing is throwing any target not found type exceptions. And It's currently an AxisFault. To make this make sense in this special binding.sca case you'll have to reinstate the binding.sca invoker [1] to trap the exception and wrap it with something that the RuntimeWire can spot. Simon In the testcase this is all local with two nodes within the single jvm so using the local sca binding not the WS based one so there's no AxisFault. ...ant To make this a bit clearer, this is the stack trace of the reference invocation just before it fails: Thread [main] (Suspended (breakpoint at line 42 in StatelessScopeContainer)) StatelessScopeContainerKEY.getWrapper(KEY) line: 42 JavaImplementationInvoker.invoke(Message) line: 100 PassByValueInterceptor.invoke(Message) line: 61 SCABindingInvoker.invoke(Message) line: 61 PassByValueInterceptor.invoke(Message) line: 61 JDKInvocationHandler.invoke(InvocationChain, Object[], RuntimeWire, EndpointReference) line: 277 JDKInvocationHandler.invoke(Object, Method, Object[]) line: 142 $Proxy10.sayHello(String) line: not available HelloworldImpl.sayHello(String) line: 40 NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method] NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39 DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25 Method.invoke(Object, Object...) line: 597 JavaImplementationInvoker.invoke(Message) line: 136 PassByValueInterceptor.invoke(Message) line: 61 SCABindingInvoker.invoke(Message) line: 61 PassByValueInterceptor.invoke(Message) line: 61 JDKInvocationHandler.invoke(InvocationChain, Object[], RuntimeWire, EndpointReference) line: 277 JDKInvocationHandler.invoke(Object, Method, Object[]) line: 142 $Proxy10.sayHello(String) line: not available StopStartNodesTestCaseFIXME.testCalculator() line: 76 That scope container at the top is actually in a STOPPED state but nothing in the getWrapper call path checks that so eventually it gets an NPE. We could (should) change that so it notices the scope container is stopped and throws an exception. The SCABindingInvoker.invoke call could then catch that exception and do something, I guess thats where you're suggesting inserting the initInvocationChains call. If the service node has been moved from being local to be remote we actually need to change the SCABindingInvoker impl from being the local sca binding to be the remote one. I wonder if we need a new generic SCA binding impl that can handle both local and remote invocations based on whether the endpoint is local or remote. ...ant Sounds right to me. Catch the exception in the SCABindingInvoker.invoke and turn it into something the runtime wire understands. If you rebuild the chains it should rebuild the binding invoker etc. for you based on the configuration of the endpoint reference. Simon
Re: Endpoint registry and stopping/starting nodes
On Thu, Aug 27, 2009 at 12:40 PM, Simon Lawssimonsl...@googlemail.com wrote: On Thu, Aug 27, 2009 at 12:01 PM, ant elderantel...@apache.org wrote: On Thu, Aug 27, 2009 at 11:29 AM, ant elderantel...@apache.org wrote: On Thu, Aug 27, 2009 at 11:19 AM, Simon Lawssimonsl...@googlemail.com wrote: Nothing is throwing any target not found type exceptions. And It's currently an AxisFault. To make this make sense in this special binding.sca case you'll have to reinstate the binding.sca invoker [1] to trap the exception and wrap it with something that the RuntimeWire can spot. Simon In the testcase this is all local with two nodes within the single jvm so using the local sca binding not the WS based one so there's no AxisFault. ...ant To make this a bit clearer, this is the stack trace of the reference invocation just before it fails: Thread [main] (Suspended (breakpoint at line 42 in StatelessScopeContainer)) StatelessScopeContainerKEY.getWrapper(KEY) line: 42 JavaImplementationInvoker.invoke(Message) line: 100 PassByValueInterceptor.invoke(Message) line: 61 SCABindingInvoker.invoke(Message) line: 61 PassByValueInterceptor.invoke(Message) line: 61 JDKInvocationHandler.invoke(InvocationChain, Object[], RuntimeWire, EndpointReference) line: 277 JDKInvocationHandler.invoke(Object, Method, Object[]) line: 142 $Proxy10.sayHello(String) line: not available HelloworldImpl.sayHello(String) line: 40 NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method] NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39 DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25 Method.invoke(Object, Object...) line: 597 JavaImplementationInvoker.invoke(Message) line: 136 PassByValueInterceptor.invoke(Message) line: 61 SCABindingInvoker.invoke(Message) line: 61 PassByValueInterceptor.invoke(Message) line: 61 JDKInvocationHandler.invoke(InvocationChain, Object[], RuntimeWire, EndpointReference) line: 277 JDKInvocationHandler.invoke(Object, Method, Object[]) line: 142 $Proxy10.sayHello(String) line: not available StopStartNodesTestCaseFIXME.testCalculator() line: 76 That scope container at the top is actually in a STOPPED state but nothing in the getWrapper call path checks that so eventually it gets an NPE. We could (should) change that so it notices the scope container is stopped and throws an exception. The SCABindingInvoker.invoke call could then catch that exception and do something, I guess thats where you're suggesting inserting the initInvocationChains call. If the service node has been moved from being local to be remote we actually need to change the SCABindingInvoker impl from being the local sca binding to be the remote one. I wonder if we need a new generic SCA binding impl that can handle both local and remote invocations based on whether the endpoint is local or remote. ...ant Sounds right to me. Catch the exception in the SCABindingInvoker.invoke and turn it into something the runtime wire understands. If you rebuild the chains it should rebuild the binding invoker etc. for you based on the configuration of the endpoint reference. Simon Ok I've committed some changes to get that working for the StopStartNodesTestCase. Its limited to that specific scenario and seems quite fragile, it wont be working for different scopes, different wiring, callbacks, non-java implementations etc, and I expect its causing wire rebuilds when it catches exceptions for other errors not related to the service node being restarted. I'm wondering if it wouldn't be more robust to do the earlier suggestion of just having the sca binding look up the endpoint from the endpoint registry on each invocation. ...ant
