Hi all, Since there were concerns as to the maintenance and upkeep of third party libraries as well as the usage of mDNS in many network set-ups, we have not gone ahead with the mDNS approach.
All product analytics distributions now ship with a predefined port offset of 1, and the product runtimes have also been pre-configured to look for the analytics distributions taking this offset into account. Now, in single-node setups, one can simply download both the runtime and analytics distributions for testing purposes without the need for any additional configuration (apart from enabling statistics publication on the runtime). Thanks, On 6 July 2016 at 11:25, Afkham Azeez <[email protected]> wrote: > I am not in favor of implementing demoware which will not work in a > production setup. The cumulative maintenance overhead over several years > could be massive. From experience we know what a pain such features can be. > > On Wed, Jul 6, 2016 at 11:20 AM, Anjana Fernando <[email protected]> wrote: > >> Hi everyone, >> >> We've identified some concerns while doing the implementation. Basically >> as Gokul mentioned, he has completed the implementation of this, and when >> we are approving a 3'rd party library related to this, Azeez brought up >> some concerns on the usefulness of this feature. Basically, the aim of this >> was to use it mainly in a demo scenario to set it up quickly. If the demo >> scenario is a single machine running multiple instances of the servers, we >> can get the zero configuration target by hard coding the hosts and the >> proper port offsetted ports by default. So then the other scenario is a >> situation when multiple machines are in place in a physical network, but >> then the argument is, most probably this will be a not so demo scenario, >> and will anyway require other configurations like registry mounting etc.. >> and also considering most network infrastructure disable multicasting by >> default, this may not work at all. >> >> While doing the implementation, we identified some of these concerns, and >> had some chats with Srinath and all, but we decided to finish the >> implementation and see how it will work out anyway. But now, I guess we >> should make the final decision on whether we should actually put it in or >> not. >> >> Cheers, >> Anjana. >> >> On Mon, Jul 4, 2016 at 3:52 PM, Gokul Balakrishnan <[email protected]> >> wrote: >> >>> Hi all, >>> >>> I have completed implementation of this component, and the relevant PR >>> may be found at [1]. >>> >>> If I explain the mechanism a bit, an mDNS service is registered for the >>> Thrift server (i.e. DAS, CEP or product-analytics distributions), provided >>> that a system flag "receiverDiscoveryEnabled" is set. The service will be >>> automatically maintained per each network interface found on the server >>> node (since as explained above, we cannot infer the network interface(s) >>> reachable from outside the node/LAN). >>> >>> A Thrift client (in this case, the EventStreamService OSGi service which >>> is our recommended way of publishing data to DAS/CEP/product-analytics), if >>> started with the same flag, is then able to discover the above service, and >>> it will resolve the first available Thrift endpoint through multicast DNS. >>> Then, its local configuration will be overwritten with the discovered >>> Thrift endpoint (with a log printed stating this fact). The client is then >>> able to send events to the Thrift server with zero configuration from the >>> user's part. >>> >>> Questions/comments welcome. >>> >>> Thanks, >>> >>> [1] https://github.com/wso2/carbon-analytics-common/pull/270 >>> >>> On 30 June 2016 at 16:49, Gokul Balakrishnan <[email protected]> wrote: >>> >>>> Sure Srinath. However, prior to committing, there are a few concerns >>>> with the approach which I think should be considered: most importantly, how >>>> we should resolve the hostname/IP of the Thrift server. Currently, the >>>> Thrift server is set to listen to all interfaces for incoming connections >>>> (0.0.0.0), whether valid or not. This is to overcome the earlier issues >>>> we've had where the Thrift server would bind to the first Network interface >>>> it found. >>>> >>>> On the server side, we will need to initialize the service discovery to >>>> a specific network interface which is also capable of accepting Thrift >>>> connections. The issue is that it's not possible to always guarantee that >>>> this interface is reachable from outside the local machine or LAN. For >>>> instance, the IP that we've bound to the multicast service could be from a >>>> virtual network interface. Normally, within a single node, this will not >>>> pose an issue, but overcoming this issue in a multi-node environment will >>>> mean maintaining a configuration which points to the correct IP which could >>>> accept incoming Thrift connections from outside the node (since we can't >>>> infer this information from the Thrift server, as it's already bound to ALL >>>> interfaces). This of course will negate our goal of zero configuration. >>>> >>>> The alternative is to let the analytics (server) node provide the list >>>> of ALL its network interfaces to the client and the client then going >>>> through them one by one to try and create a socket connection, to see if >>>> one or more them is accepting Thrift connections. This is not desirable due >>>> to obvious reasons. >>>> >>>> How shall we proceed? >>>> >>>> Thanks, >>>> >>>> On 30 June 2016 at 08:52, Srinath Perera <[email protected]> wrote: >>>> >>>>> Thanks, sound good. Please do ASAP. We are at beta, so too late even >>>>> now. >>>>> >>>>> --Srinath >>>>> >>>>> On Thu, Jun 30, 2016 at 8:09 AM, Gokul Balakrishnan <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Srinath, >>>>>> >>>>>> As per our offline chat earlier, the initial plan is to locate >>>>>> the Thrift endpoint through mDNS service discovery, considering the host >>>>>> and port first. >>>>>> >>>>>> I have used the JmDNS library pointed by Nirmal to do a PoC on this >>>>>> scenario, and I've also already incorporated the logic into the >>>>>> databridge >>>>>> Thrift server to enable service registration through a system property >>>>>> the >>>>>> users could set (-DenableDiscovery). The corresponding client code goes >>>>>> into the publisher OSGi service initialisation. This too is controllable >>>>>> by >>>>>> the same system property the user could set on the Thrift client (which >>>>>> will be the product talking to DAS/CEP). >>>>>> >>>>>> I'm doing some testing on the entire scenario, and once completed, >>>>>> I'll commit the changes into the relevant repos and send an update to >>>>>> this >>>>>> thread. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> >>>>>> On Thursday, 30 June 2016, Srinath Perera <[email protected]> wrote: >>>>>> >>>>>>> Resending as it hits a filter rule. >>>>>>> >>>>>>> Gokul, please give an update on this? >>>>>>> >>>>>>> --Srinath >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Gokul Balakrishnan >>>>>> Senior Software Engineer, >>>>>> WSO2, Inc. http://wso2.com >>>>>> M +94 77 5935 789 | +44 7563 570502 >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> ============================ >>>>> Srinath Perera, Ph.D. >>>>> http://people.apache.org/~hemapani/ >>>>> http://srinathsview.blogspot.com/ >>>>> >>>> >>>> >>>> >>>> -- >>>> Gokul Balakrishnan >>>> Senior Software Engineer, >>>> WSO2, Inc. http://wso2.com >>>> M +94 77 5935 789 | +44 7563 570502 >>>> >>>> >>> >>> >>> -- >>> Gokul Balakrishnan >>> Senior Software Engineer, >>> WSO2, Inc. http://wso2.com >>> M +94 77 5935 789 | +44 7563 570502 >>> >>> >> >> >> -- >> *Anjana Fernando* >> Senior Technical Lead >> WSO2 Inc. | http://wso2.com >> lean . enterprise . middleware >> > > > > -- > *Afkham Azeez* > Director of Architecture; WSO2, Inc.; http://wso2.com > Member; Apache Software Foundation; http://www.apache.org/ > * <http://www.apache.org/>* > *email: **[email protected]* <[email protected]> > * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * > *http://blog.afkham.org* <http://blog.afkham.org> > *twitter: **http://twitter.com/afkham_azeez* > <http://twitter.com/afkham_azeez> > *linked-in: **http://lk.linkedin.com/in/afkhamazeez > <http://lk.linkedin.com/in/afkhamazeez>* > > *Lean . Enterprise . Middleware* > -- Gokul Balakrishnan Senior Software Engineer, WSO2, Inc. http://wso2.com M +94 77 5935 789 | +44 7563 570502
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
