Hi Tom, I have submitted the final proposal. Thank you for the guidance provided so far. I'm eagerly looking forward to complete my project for GSoC 2017.
Thank you! Kind Regards, *Imesha Sudasingha* Undergraduate of Department of Computer Science and Engineering, University of Moratuwa, Sri Lanka. <https://lk.linkedin.com/in/imeshasudasingha> <https://github.com/IMS94> <http://stackoverflow.com/users/4012073/imesha-sudasingha> <https://twitter.com/Imesha94> On 1 April 2017 at 13:20, Imesha Sudasingha <[email protected]> wrote: > Hi Tom, > > I have updated my proposal a bit and started working on the implementation > as well. What I have done so far can be found at [1]. Please have a look > and let me know your think of it if you find some free time. > > Thank you! > > [1] https://github.com/IMS94/oodt > > Kind Regards, > *Imesha Sudasingha* > Undergraduate of Department of Computer Science and Engineering, > University of Moratuwa, > Sri Lanka. > > <https://lk.linkedin.com/in/imeshasudasingha> <https://github.com/IMS94> > <http://stackoverflow.com/users/4012073/imesha-sudasingha> > <https://twitter.com/Imesha94> > > On 29 March 2017 at 15:08, Imesha Sudasingha <[email protected]> > wrote: > >> Hi Tom, >> >> I was thinking on how to inherit the configuration for components at >> zookeeper level. What do you think of something like this: >> >> "We ask the user to manually configure only one node/component. The >> configuration manager that I'm making will take care of that and will store >> the configuration in Zookeeper then. Whenever new nodes/components are >> being deployed, they automatically fetch configuration from zookeeper >> through the configuration manager that I'm making." >> >> This way, the user has to configure only one component manually which >> will be used as the reference for other components that are coming up. Your >> ideas on this will be highly appreciated. >> >> Thank you! >> >> >> Kind Regards, >> *Imesha Sudasingha* >> Undergraduate of Department of Computer Science and Engineering, >> University of Moratuwa, >> Sri Lanka. >> >> <https://lk.linkedin.com/in/imeshasudasingha> <https://github.com/IMS94> >> <http://stackoverflow.com/users/4012073/imesha-sudasingha> >> <https://twitter.com/Imesha94> >> >> On 28 March 2017 at 08:46, Imesha Sudasingha <[email protected]> >> wrote: >> >>> Thanks a lot Tom! >>> >>> Kind Regards, >>> *Imesha Sudasingha* >>> Undergraduate of Department of Computer Science and Engineering, >>> University of Moratuwa, >>> Sri Lanka. >>> >>> <https://lk.linkedin.com/in/imeshasudasingha> >>> <https://github.com/IMS94> >>> <http://stackoverflow.com/users/4012073/imesha-sudasingha> >>> <https://twitter.com/Imesha94> >>> >>> On 28 March 2017 at 08:40, Tom Barber <[email protected]> wrote: >>> >>>> Thanks Imesha >>>> >>>> I shall check it tomorrow and get back to you ASAP. >>>> >>>> Tom >>>> >>>> On Tue, Mar 28, 2017 at 2:33 AM, Imesha Sudasingha < >>>> [email protected]> >>>> wrote: >>>> >>>> > Hi Tom, >>>> > >>>> > As you suggested, I have created a copy of my proposal at [1] and >>>> shared >>>> > with comment permissions. It will be a great help if you can review it >>>> > sooner and provide a kind feedback. >>>> > >>>> > Thank you! >>>> > >>>> > [1] >>>> > https://docs.google.com/document/d/1bnFEVViCwMACHJRB1OiYZ- >>>> > GAywCNJRxrUJPpHttpgpE/edit?usp=sharing >>>> > >>>> > Kind Regards, >>>> > *Imesha Sudasingha* >>>> > Undergraduate of Department of Computer Science and Engineering, >>>> > University of Moratuwa, >>>> > Sri Lanka. >>>> > >>>> > <https://lk.linkedin.com/in/imeshasudasingha> < >>>> https://github.com/IMS94> >>>> > <http://stackoverflow.com/users/4012073/imesha-sudasingha> >>>> > <https://twitter.com/Imesha94> >>>> > >>>> > On 28 March 2017 at 01:13, Tom Barber <[email protected]> >>>> wrote: >>>> > >>>> > > Hi Imesha >>>> > > >>>> > > I'm trying to figure out if/how I can get access to it on the >>>> dashboard. >>>> > In >>>> > > the mean time if you want to pastebin it, chuck it in a word doc, or >>>> > > something completely different I'd be happy to take a look. >>>> > > >>>> > > Tom >>>> > > >>>> > > On Sun, Mar 26, 2017 at 3:09 PM, Imesha Sudasingha < >>>> > > [email protected]> >>>> > > wrote: >>>> > > >>>> > > > Hi Tom, >>>> > > > >>>> > > > Did you get a chance to look at my proposal? >>>> > > > >>>> > > > Kind Regards, >>>> > > > *Imesha Sudasingha* >>>> > > > Undergraduate of Department of Computer Science and Engineering, >>>> > > > University of Moratuwa, >>>> > > > Sri Lanka. >>>> > > > >>>> > > > <https://lk.linkedin.com/in/imeshasudasingha> < >>>> > https://github.com/IMS94 >>>> > > > >>>> > > > <http://stackoverflow.com/users/4012073/imesha-sudasingha> >>>> > > > <https://twitter.com/Imesha94> >>>> > > > >>>> > > > On 23 March 2017 at 10:40, Imesha Sudasingha < >>>> [email protected]> >>>> > > > wrote: >>>> > > > >>>> > > > > Hi Tom, >>>> > > > > >>>> > > > > I have written a draft proposal and shared it with the Apache >>>> > Software >>>> > > > > Foundation through the GSoC dashboard. If you get a chance, can >>>> you >>>> > > > please >>>> > > > > review it and let me know what are the improvements required to >>>> be >>>> > > made. >>>> > > > > >>>> > > > > Thanks in advance! >>>> > > > > >>>> > > > > Kind Regards, >>>> > > > > *Imesha Sudasingha* >>>> > > > > Undergraduate of Department of Computer Science and >>>> Engineering, >>>> > > > > University of Moratuwa, >>>> > > > > Sri Lanka. >>>> > > > > >>>> > > > > <https://lk.linkedin.com/in/imeshasudasingha> < >>>> > > https://github.com/IMS94 >>>> > > > > >>>> > > > > <http://stackoverflow.com/users/4012073/imesha-sudasingha> >>>> > > > > <https://twitter.com/Imesha94> >>>> > > > > >>>> > > > > On 22 March 2017 at 14:36, Imesha Sudasingha < >>>> > [email protected]> >>>> > > > > wrote: >>>> > > > > >>>> > > > >> Hi Tom, >>>> > > > >> >>>> > > > >> I think I had missed your point previously. I was referring to >>>> just >>>> > > > >> storing the configuration of different components in different >>>> > ZNodes >>>> > > > and >>>> > > > >> querying them later. Then I understood that we also need to >>>> have >>>> > > > >> inheritance at zookeeper level in order to make it easy to >>>> configure >>>> > > > OODT >>>> > > > >> clusters as you have mentioned in the issue [1]. To configure >>>> > clusters >>>> > > > of >>>> > > > >> OODT components with minimum configuration, I will have to >>>> review my >>>> > > > design >>>> > > > >> again and make a lot of improvements. For example, I have to >>>> think >>>> > > > about a >>>> > > > >> way to store configuration in zookeeper with minimum >>>> duplications >>>> > and >>>> > > > give >>>> > > > >> components enough flexibility to initialize with no or minimum >>>> > manual >>>> > > > >> configuration. >>>> > > > >> >>>> > > > >> Since I have to submit the GSoC proposal before 3rd of April, I >>>> > think >>>> > > > >> that I won't have enough time to review the design and make the >>>> > > > necessary >>>> > > > >> improvements before that. Will that negatively affect the >>>> > possibility >>>> > > > of me >>>> > > > >> being able to work on this project for GSoC 2017? Please let >>>> me know >>>> > > you >>>> > > > >> opinion. >>>> > > > >> >>>> > > > >> Thank you! >>>> > > > >> >>>> > > > >> [1] https://issues.apache.org/jira/browse/OODT-945 >>>> > > > >> >>>> > > > >> Kind Regards, >>>> > > > >> *Imesha Sudasingha* >>>> > > > >> Undergraduate of Department of Computer Science and >>>> Engineering, >>>> > > > >> University of Moratuwa, >>>> > > > >> Sri Lanka. >>>> > > > >> >>>> > > > >> <https://lk.linkedin.com/in/imeshasudasingha> < >>>> > > > https://github.com/IMS94> >>>> > > > >> <http://stackoverflow.com/users/4012073/imesha-sudasingha> >>>> > > > >> <https://twitter.com/Imesha94> >>>> > > > >> >>>> > > > >> On 21 March 2017 at 10:52, Imesha Sudasingha < >>>> > [email protected] >>>> > > > >>>> > > > >> wrote: >>>> > > > >> >>>> > > > >>> Hi Tom, >>>> > > > >>> >>>> > > > >>> Sorry for the late reply. Zookeeper is just providing an ZNode >>>> > > > >>> structure, which can be considered as a tree where a node is >>>> > accessed >>>> > > > as >>>> > > > >>> directories. For example, in [1] the file manager properties >>>> of the >>>> > > > >>> instance with name "*node1*" is stored at the ZPath (Path to >>>> that >>>> > > > >>> ZNode) "*/oodt/node1/file-mgr/filemgr.properties*". >>>> > > > >>> >>>> > > > >>> Therefore, my idea is that we can make such directories >>>> (ZPaths) >>>> > per >>>> > > > >>> each instance in our cluster (ex: */oodt/node1 for instance >>>> > "node1"*, >>>> > > > */oodt/node2 >>>> > > > >>> for instance "node2"* and so on). Then inside each such path >>>> > > (assigned >>>> > > > >>> per node), we will have separate ZNodes for each component. >>>> For >>>> > > > example, in >>>> > > > >>> */oodt/node1* we will have an ZNode /oodt/node1/file-mgr which >>>> > stores >>>> > > > >>> the configurations of file manager of node1. Similarly, an >>>> ZNode >>>> > > > >>> /oodt/node1/res-mgr will store the configuration of the >>>> resource >>>> > > > manager of >>>> > > > >>> node1. >>>> > > > >>> >>>> > > > >>> Hope you got the idea. Since zookeeper is not some kind of a >>>> graph >>>> > > > >>> database, I couldn't think of a method to keep inheritance >>>> among >>>> > > > >>> configuration at Zookeeper's side. However, the design I >>>> suggest >>>> > will >>>> > > > store >>>> > > > >>> the configuration independently and will allow the >>>> users/developers >>>> > > to >>>> > > > >>> query those configuration. Therefore in my opinion, we should >>>> let >>>> > the >>>> > > > >>> developer to compare the configuration and make decisions. >>>> This >>>> > way, >>>> > > > we can >>>> > > > >>> allow this distributed configuration manager to manage >>>> > configuration >>>> > > > of any >>>> > > > >>> current component or any upcoming component making it more >>>> > > extend-able. >>>> > > > >>> >>>> > > > >>> Can you let me know what do you think of it? >>>> > > > >>> >>>> > > > >>> Thank you! >>>> > > > >>> >>>> > > > >>> [1] https://drive.google.com/open? >>>> id=0B415Bnipaf1lNThKYXZMN3pFemc >>>> > > > >>> >>>> > > > >>> Kind Regards, >>>> > > > >>> *Imesha Sudasingha* >>>> > > > >>> Undergraduate of Department of Computer Science and >>>> Engineering, >>>> > > > >>> University of Moratuwa, >>>> > > > >>> Sri Lanka. >>>> > > > >>> >>>> > > > >>> <https://lk.linkedin.com/in/imeshasudasingha> >>>> > > > >>> <https://github.com/IMS94> >>>> > > > >>> <http://stackoverflow.com/users/4012073/imesha-sudasingha> >>>> > > > >>> <https://twitter.com/Imesha94> >>>> > > > >>> >>>> > > > >>> On 20 March 2017 at 05:02, Tom Barber < >>>> [email protected]> >>>> > > wrote: >>>> > > > >>> >>>> > > > >>>> Hi Imesha >>>> > > > >>>> >>>> > > > >>>> Sorry for the delay, I've been travelling and only just got >>>> myself >>>> > > > >>>> through >>>> > > > >>>> my backlog. >>>> > > > >>>> >>>> > > > >>>> The overall design looks good. I was also thinking it would >>>> make >>>> > > sense >>>> > > > >>>> if >>>> > > > >>>> possible to have some form of inheritance in the config. So, >>>> for >>>> > > > >>>> example, >>>> > > > >>>> if you have 5 file managers, and they all share the same >>>> config >>>> > > > except a >>>> > > > >>>> base url or whatever, it would make sense to have a single >>>> set of >>>> > > > >>>> configs >>>> > > > >>>> and then individual overrides for configs that change. (My >>>> > knowledge >>>> > > > of >>>> > > > >>>> Zookeeper isn't great, I'm assuming something along these >>>> lines is >>>> > > > >>>> possible). >>>> > > > >>>> >>>> > > > >>>> It would also make sense, I think to register this stuff >>>> with the >>>> > > > >>>> resource >>>> > > > >>>> manager, but I'm not sure how that integration would work >>>> quite >>>> > yet, >>>> > > > so >>>> > > > >>>> we'll need to figure that one out. >>>> > > > >>>> >>>> > > > >>>> Let me know if you have any questions about the above. >>>> > > > >>>> >>>> > > > >>>> Tom >>>> > > > >>>> >>>> > > > >>>> >>>> > > > >>>> On Sat, Mar 18, 2017 at 12:39 PM, Imesha Sudasingha < >>>> > > > >>>> [email protected] >>>> > > > >>>> > wrote: >>>> > > > >>>> >>>> > > > >>>> > Hi Tom, >>>> > > > >>>> > >>>> > > > >>>> > What do you think of the design? Do you agree on >>>> implementing >>>> > this >>>> > > > >>>> > configuration management section in a separate module? >>>> > > > >>>> > >>>> > > > >>>> > I was thinking on starting on the implementation as a >>>> proof of >>>> > > > >>>> concept. Can >>>> > > > >>>> > you give me your kind feedback before I start on that? >>>> > > > >>>> > >>>> > > > >>>> > Thank you! >>>> > > > >>>> > >>>> > > > >>>> > Kind Regards, >>>> > > > >>>> > *Imesha Sudasingha* >>>> > > > >>>> > Undergraduate of Department of Computer Science and >>>> > Engineering, >>>> > > > >>>> > University of Moratuwa, >>>> > > > >>>> > Sri Lanka. >>>> > > > >>>> > >>>> > > > >>>> > <https://lk.linkedin.com/in/imeshasudasingha> < >>>> > > > >>>> https://github.com/IMS94> >>>> > > > >>>> > <http://stackoverflow.com/users/4012073/imesha-sudasingha> >>>> > > > >>>> > <https://twitter.com/Imesha94> >>>> > > > >>>> > >>>> > > > >>>> > On 16 March 2017 at 17:51, Imesha Sudasingha < >>>> > > > [email protected] >>>> > > > >>>> > >>>> > > > >>>> > wrote: >>>> > > > >>>> > >>>> > > > >>>> > > Hi Tom, >>>> > > > >>>> > > >>>> > > > >>>> > > Thanks for the quick reply. I came up with a small >>>> design [1]. >>>> > > > >>>> > > >>>> > > > >>>> > > What I suggest is a separate module for configuration >>>> > > management, >>>> > > > >>>> which >>>> > > > >>>> > > can be extended in future and used by many components as >>>> well. >>>> > > > >>>> > > >>>> > > > >>>> > > As shown in the design [1], File Manager (or any >>>> component >>>> > that >>>> > > is >>>> > > > >>>> > willing >>>> > > > >>>> > > to have this extended functionality of distributed >>>> > configuration >>>> > > > >>>> > > management) will have an instance of >>>> *ConfigurationManager* >>>> > > which >>>> > > > >>>> is an >>>> > > > >>>> > > interface defined to load configuration (specifically, >>>> > > .properties >>>> > > > >>>> files) >>>> > > > >>>> > > and get configurations of each running instance of the >>>> same >>>> > > > >>>> component (If >>>> > > > >>>> > > distributed configuration management is enabled, this >>>> will >>>> > > return >>>> > > > >>>> the >>>> > > > >>>> > > configuration of all the other instances). >>>> > > > >>>> > > >>>> > > > >>>> > > There will be two implementing classes of that interface, >>>> > > > >>>> > > *StandaloneConfigurationManager* and >>>> > > > *DistributedConfigurationManag >>>> > > > >>>> er.* >>>> > > > >>>> > As >>>> > > > >>>> > > the name implies, standalone configuration manager is to >>>> be >>>> > used >>>> > > > if >>>> > > > >>>> the >>>> > > > >>>> > > users do not intend to used distributed configuration >>>> > > management. >>>> > > > >>>> Also >>>> > > > >>>> > this >>>> > > > >>>> > > will be same as the current mechanism of configuration >>>> > > management. >>>> > > > >>>> Then >>>> > > > >>>> > the >>>> > > > >>>> > > distributed configuration manager will be the >>>> implementation >>>> > > using >>>> > > > >>>> > > Zookeeper. >>>> > > > >>>> > > >>>> > > > >>>> > > At zookeeper level, we will need to have a proper naming >>>> > > mechanism >>>> > > > >>>> and a >>>> > > > >>>> > > structure to store configuration. For example,* >>>> > > > >>>> > > /${nodeName}/${componentName}/{$configFileName}* will >>>> be the >>>> > > > place >>>> > > > >>>> where >>>> > > > >>>> > > the configuration file contents will be stored. >>>> > *getFile(String >>>> > > > >>>> fileName) >>>> > > > >>>> > > and getFiles(String directiryName)* of >>>> *NodeConfiguration* >>>> > class >>>> > > > >>>> will >>>> > > > >>>> > > return the content of the configuration files and >>>> > configuration >>>> > > > >>>> > directories >>>> > > > >>>> > > respectively. I have demonstrated the ZNode structre I'm >>>> > > expecting >>>> > > > >>>> to >>>> > > > >>>> > > design in [2] >>>> > > > >>>> > > >>>> > > > >>>> > > Please note that this is just a very high level overview >>>> of my >>>> > > > >>>> design and >>>> > > > >>>> > > the real difficulty of implementation comes when testing >>>> this >>>> > > and >>>> > > > >>>> > > addressing the edge cases of zookeeper. Also, I have only >>>> > added >>>> > > > few >>>> > > > >>>> > methods >>>> > > > >>>> > > that came to my mind to the *ConfigurationManager* >>>> interface. >>>> > If >>>> > > > >>>> you find >>>> > > > >>>> > > any fault in this design or have a better design in mind >>>> > please >>>> > > > let >>>> > > > >>>> me >>>> > > > >>>> > know >>>> > > > >>>> > > so that I can optimize this design and do my best to >>>> come up >>>> > > with >>>> > > > >>>> the >>>> > > > >>>> > best >>>> > > > >>>> > > possible solution. >>>> > > > >>>> > > >>>> > > > >>>> > > Thank you! >>>> > > > >>>> > > >>>> > > > >>>> > > [1] https://drive.google.com/open?id= >>>> > > 0B415Bnipaf1ldnV1dWt4VVVLam8 >>>> > > > >>>> > > [2] https://drive.google.com/open?id= >>>> > > 0B415Bnipaf1lNThKYXZMN3pFemc >>>> > > > >>>> > > >>>> > > > >>>> > > Kind Regards, >>>> > > > >>>> > > *Imesha Sudasingha* >>>> > > > >>>> > > Undergraduate of Department of Computer Science and >>>> > > Engineering, >>>> > > > >>>> > > University of Moratuwa, >>>> > > > >>>> > > Sri Lanka. >>>> > > > >>>> > > >>>> > > > >>>> > > <https://lk.linkedin.com/in/imeshasudasingha> < >>>> > > > >>>> https://github.com/IMS94 >>>> > > > >>>> > > >>>> > > > >>>> > > <http://stackoverflow.com/user >>>> s/4012073/imesha-sudasingha> >>>> > > > >>>> > > <https://twitter.com/Imesha94> >>>> > > > >>>> > > >>>> > > > >>>> > > On 16 March 2017 at 16:55, Tom Barber < >>>> > [email protected]> >>>> > > > >>>> wrote: >>>> > > > >>>> > > >>>> > > > >>>> > >> Hi Imesha, >>>> > > > >>>> > >> >>>> > > > >>>> > >> Basically, just allowing the dynamic registration of >>>> file >>>> > > > managers >>>> > > > >>>> so >>>> > > > >>>> > that >>>> > > > >>>> > >> new ones can come and go. >>>> > > > >>>> > >> >>>> > > > >>>> > >> Tom >>>> > > > >>>> > >> >>>> > > > >>>> > >> On Thu, Mar 16, 2017 at 10:00 AM, Imesha Sudasingha < >>>> > > > >>>> > >> [email protected] >>>> > > > >>>> > >> > wrote: >>>> > > > >>>> > >> >>>> > > > >>>> > >> > Hi Tom, >>>> > > > >>>> > >> > >>>> > > > >>>> > >> > Need a small clarification. In the last reply, you >>>> > mentioned >>>> > > > >>>> that file >>>> > > > >>>> > >> > managers may be designed to be *transient. *What do >>>> you >>>> > mean >>>> > > by >>>> > > > >>>> that? >>>> > > > >>>> > A >>>> > > > >>>> > >> > quick response would really help me as I'm trying to >>>> come >>>> > up >>>> > > > >>>> with a >>>> > > > >>>> > >> design. >>>> > > > >>>> > >> > >>>> > > > >>>> > >> > Thank you! >>>> > > > >>>> > >> > >>>> > > > >>>> > >> > *Imesha Sudasingha* >>>> > > > >>>> > >> > Undergraduate of Department of Computer Science and >>>> > > > Engineering, >>>> > > > >>>> > >> > University of Moratuwa, >>>> > > > >>>> > >> > Sri Lanka. >>>> > > > >>>> > >> > >>>> > > > >>>> > >> > <https://lk.linkedin.com/in/imeshasudasingha> < >>>> > > > >>>> > >> https://github.com/IMS94> >>>> > > > >>>> > >> > <http://stackoverflow.com/user >>>> s/4012073/imesha-sudasingha> >>>> > > > >>>> > >> > <https://twitter.com/Imesha94> >>>> > > > >>>> > >> > >>>> > > > >>>> > >> > On 13 March 2017 at 22:03, Imesha Sudasingha < >>>> > > > >>>> [email protected] >>>> > > > >>>> > > >>>> > > > >>>> > >> > wrote: >>>> > > > >>>> > >> > >>>> > > > >>>> > >> > > Hi Tom, >>>> > > > >>>> > >> > > >>>> > > > >>>> > >> > > Thanks for the clarification. It really helped a >>>> lot to >>>> > > fill >>>> > > > >>>> most of >>>> > > > >>>> > >> the >>>> > > > >>>> > >> > > gaps I had unanswered. I have a draft design in my >>>> mind >>>> > and >>>> > > > >>>> will >>>> > > > >>>> > post >>>> > > > >>>> > >> for >>>> > > > >>>> > >> > > your review with details soon. >>>> > > > >>>> > >> > > >>>> > > > >>>> > >> > > As for my current understanding, my task is to >>>> implement >>>> > a >>>> > > > >>>> mechanism >>>> > > > >>>> > >> to >>>> > > > >>>> > >> > > switch/chose configuration mechanism (between local >>>> files >>>> > > and >>>> > > > >>>> > zookeer >>>> > > > >>>> > >> for >>>> > > > >>>> > >> > > the moment) and then allow APIs to query those >>>> mechanisms >>>> > > (in >>>> > > > >>>> the >>>> > > > >>>> > >> > zookeeper >>>> > > > >>>> > >> > > scenario) for future use (For File Managers to know >>>> about >>>> > > > other >>>> > > > >>>> > >> instances >>>> > > > >>>> > >> > > and make decisions accordingly). That include >>>> > > > synchronization, >>>> > > > >>>> > >> fetching >>>> > > > >>>> > >> > > configuration from zookeeper if an instance go down >>>> and >>>> > > come >>>> > > > >>>> again >>>> > > > >>>> > >> and so >>>> > > > >>>> > >> > > on. Am I correct? >>>> > > > >>>> > >> > > >>>> > > > >>>> > >> > > I will try to come up with the design I mentioned, >>>> so >>>> > that >>>> > > > >>>> both of >>>> > > > >>>> > us >>>> > > > >>>> > >> can >>>> > > > >>>> > >> > > discuss more about the improvements and requirements >>>> > soon. >>>> > > > >>>> > >> > > >>>> > > > >>>> > >> > > Thank you for the support! >>>> > > > >>>> > >> > > >>>> > > > >>>> > >> > > Kind Regards, >>>> > > > >>>> > >> > > *Imesha Sudasingha* >>>> > > > >>>> > >> > > Undergraduate of Department of Computer Science and >>>> > > > >>>> Engineering, >>>> > > > >>>> > >> > > University of Moratuwa, >>>> > > > >>>> > >> > > Sri Lanka. >>>> > > > >>>> > >> > > >>>> > > > >>>> > >> > > <https://lk.linkedin.com/in/imeshasudasingha> < >>>> > > > >>>> > >> https://github.com/IMS94 >>>> > > > >>>> > >> > > >>>> > > > >>>> > >> > > <http://stackoverflow.com/users/4012073/imesha- >>>> > sudasingha> >>>> > > > >>>> > >> > > <https://twitter.com/Imesha94> >>>> > > > >>>> > >> > > >>>> > > > >>>> > >> > > On 13 March 2017 at 20:19, Tom Barber < >>>> > > > [email protected] >>>> > > > >>>> > >>>> > > > >>>> > >> wrote: >>>> > > > >>>> > >> > > >>>> > > > >>>> > >> > >> Hi Imesha >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> The best way to demonstrate this is to show you a >>>> > > reference >>>> > > > >>>> build >>>> > > > >>>> > for >>>> > > > >>>> > >> > >> OODT.. >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> https://dl.dropboxusercontent. >>>> > > > com/u/8503756/oodt-example.tgz >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> If you can grab that archive and extract it, >>>> you'll see >>>> > > that >>>> > > > >>>> is how >>>> > > > >>>> > >> > >> currently OODT is deployed. >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> If you take the file manager as the reference >>>> module >>>> > > inside >>>> > > > >>>> > >> oodt/filemgr >>>> > > > >>>> > >> > >> you'll see >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> etc/filemgr.properties >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> and >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> policy/oodt/* >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> These are the configuration locations of a single >>>> file >>>> > > > >>>> manager. >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> The problem I want to solve is this: >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> a) If I run > 1 file manager they are completely >>>> > separate >>>> > > > and >>>> > > > >>>> they >>>> > > > >>>> > >> don't >>>> > > > >>>> > >> > >> know of each other. So using Zookeeper will allow >>>> > multiple >>>> > > > >>>> file >>>> > > > >>>> > >> managers >>>> > > > >>>> > >> > >> to >>>> > > > >>>> > >> > >> register against ZK and learn of each other. >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> b) Backing them onto ZK will allow file managers >>>> to come >>>> > > and >>>> > > > >>>> go on >>>> > > > >>>> > >> the >>>> > > > >>>> > >> > >> fly, >>>> > > > >>>> > >> > >> providing scaling capabilities or maybe fm's that >>>> are >>>> > > > >>>> designed to >>>> > > > >>>> > be >>>> > > > >>>> > >> > >> transient. >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> Just for clarity, the ZK configuration will be an >>>> > optional >>>> > > > >>>> > >> requirement, >>>> > > > >>>> > >> > so >>>> > > > >>>> > >> > >> we need to extend the existing configuration >>>> mechanism, >>>> > to >>>> > > > >>>> support >>>> > > > >>>> > >> both >>>> > > > >>>> > >> > >> the >>>> > > > >>>> > >> > >> current config and the ZK config instead of ZK >>>> being >>>> > > > >>>> mandatory. >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> Tom >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> On Mon, Mar 13, 2017 at 3:20 AM, Imesha Sudasingha >>>> < >>>> > > > >>>> > >> > >> [email protected]> >>>> > > > >>>> > >> > >> wrote: >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > >> > Hi Tom, >>>> > > > >>>> > >> > >> > >>>> > > > >>>> > >> > >> > Related to the conversation on JIRA [1] ; >>>> > > > >>>> > >> > >> > >>>> > > > >>>> > >> > >> > I went through the CAS-File manager module and I >>>> now >>>> > > > >>>> understand >>>> > > > >>>> > >> how to >>>> > > > >>>> > >> > >> > address ".properties" files. Since you mentioned >>>> in >>>> > JIRA >>>> > > > >>>> [2] that >>>> > > > >>>> > >> > there >>>> > > > >>>> > >> > >> are >>>> > > > >>>> > >> > >> > occasions where we use *spring configuration >>>> files >>>> > *for >>>> > > > >>>> > >> configuration >>>> > > > >>>> > >> > >> > purposes, I want to have a look at that type of >>>> > > > >>>> configuration >>>> > > > >>>> > too. >>>> > > > >>>> > >> Can >>>> > > > >>>> > >> > >> you >>>> > > > >>>> > >> > >> > give me an example module where spring >>>> configuration >>>> > > files >>>> > > > >>>> are >>>> > > > >>>> > >> used to >>>> > > > >>>> > >> > >> load >>>> > > > >>>> > >> > >> > configuration? >>>> > > > >>>> > >> > >> > >>>> > > > >>>> > >> > >> > Also, will the content of a configuration file >>>> exceed >>>> > > 1MB >>>> > > > >>>> > (Maximum >>>> > > > >>>> > >> > data >>>> > > > >>>> > >> > >> > size allowed in an ZNode) in any case? >>>> > > > >>>> > >> > >> > >>>> > > > >>>> > >> > >> > Thank you! >>>> > > > >>>> > >> > >> > >>>> > > > >>>> > >> > >> > [1] >>>> > > > >>>> > >> > >> > https://issues.apache.org/jira >>>> > > > >>>> /browse/OODT-945?focusedCommen >>>> > > > >>>> > >> > >> tId=15906204& >>>> > > > >>>> > >> > >> > page=com.atlassian.jira.plugin >>>> .system.issuetabpanels: >>>> > > > >>>> > >> > >> > comment-tabpanel#comment-15906204 >>>> > > > >>>> > >> > >> > >>>> > > > >>>> > >> > >> > [2] >>>> > > > >>>> > >> > >> > https://issues.apache.org/jira >>>> > > > >>>> /browse/OODT-945?focusedCommen >>>> > > > >>>> > >> > >> tId=15906524& >>>> > > > >>>> > >> > >> > page=com.atlassian.jira.plugin >>>> .system.issuetabpanels: >>>> > > > >>>> > >> > >> > comment-tabpanel#comment-15906524 >>>> > > > >>>> > >> > >> > >>>> > > > >>>> > >> > >> > Kind Regards, >>>> > > > >>>> > >> > >> > *Imesha Sudasingha* >>>> > > > >>>> > >> > >> > Undergraduate of Department of Computer Science >>>> and >>>> > > > >>>> Engineering, >>>> > > > >>>> > >> > >> > University of Moratuwa, >>>> > > > >>>> > >> > >> > Sri Lanka. >>>> > > > >>>> > >> > >> > >>>> > > > >>>> > >> > >> > <https://lk.linkedin.com/in/imeshasudasingha> < >>>> > > > >>>> > >> > >> https://github.com/IMS94> >>>> > > > >>>> > >> > >> > <http://stackoverflow.com/users/4012073/imesha- >>>> > > sudasingha >>>> > > > > >>>> > > > >>>> > >> > >> > <https://twitter.com/Imesha94> >>>> > > > >>>> > >> > >> > >>>> > > > >>>> > >> > >> >>>> > > > >>>> > >> > > >>>> > > > >>>> > >> > > >>>> > > > >>>> > >> > >>>> > > > >>>> > >> >>>> > > > >>>> > > >>>> > > > >>>> > > >>>> > > > >>>> > >>>> > > > >>>> >>>> > > > >>> >>>> > > > >>> >>>> > > > >> >>>> > > > > >>>> > > > >>>> > > >>>> > >>>> >>> >>> >> >
