Hi Tom, I have updated my proposal a bit and started working on the implementation as well. What I have done so far can be found at [1]. Please have a look and let me know your think of it if you find some free time.
Thank you! [1] https://github.com/IMS94/oodt Kind Regards, *Imesha Sudasingha* Undergraduate of Department of Computer Science and Engineering, University of Moratuwa, Sri Lanka. <https://lk.linkedin.com/in/imeshasudasingha> <https://github.com/IMS94> <http://stackoverflow.com/users/4012073/imesha-sudasingha> <https://twitter.com/Imesha94> On 29 March 2017 at 15:08, Imesha Sudasingha <[email protected]> wrote: > Hi Tom, > > I was thinking on how to inherit the configuration for components at > zookeeper level. What do you think of something like this: > > "We ask the user to manually configure only one node/component. The > configuration manager that I'm making will take care of that and will store > the configuration in Zookeeper then. Whenever new nodes/components are > being deployed, they automatically fetch configuration from zookeeper > through the configuration manager that I'm making." > > This way, the user has to configure only one component manually which will > be used as the reference for other components that are coming up. Your > ideas on this will be highly appreciated. > > Thank you! > > > Kind Regards, > *Imesha Sudasingha* > Undergraduate of Department of Computer Science and Engineering, > University of Moratuwa, > Sri Lanka. > > <https://lk.linkedin.com/in/imeshasudasingha> <https://github.com/IMS94> > <http://stackoverflow.com/users/4012073/imesha-sudasingha> > <https://twitter.com/Imesha94> > > On 28 March 2017 at 08:46, Imesha Sudasingha <[email protected]> > wrote: > >> Thanks a lot Tom! >> >> Kind Regards, >> *Imesha Sudasingha* >> Undergraduate of Department of Computer Science and Engineering, >> University of Moratuwa, >> Sri Lanka. >> >> <https://lk.linkedin.com/in/imeshasudasingha> <https://github.com/IMS94> >> <http://stackoverflow.com/users/4012073/imesha-sudasingha> >> <https://twitter.com/Imesha94> >> >> On 28 March 2017 at 08:40, Tom Barber <[email protected]> wrote: >> >>> Thanks Imesha >>> >>> I shall check it tomorrow and get back to you ASAP. >>> >>> Tom >>> >>> On Tue, Mar 28, 2017 at 2:33 AM, Imesha Sudasingha < >>> [email protected]> >>> wrote: >>> >>> > Hi Tom, >>> > >>> > As you suggested, I have created a copy of my proposal at [1] and >>> shared >>> > with comment permissions. It will be a great help if you can review it >>> > sooner and provide a kind feedback. >>> > >>> > Thank you! >>> > >>> > [1] >>> > https://docs.google.com/document/d/1bnFEVViCwMACHJRB1OiYZ- >>> > GAywCNJRxrUJPpHttpgpE/edit?usp=sharing >>> > >>> > Kind Regards, >>> > *Imesha Sudasingha* >>> > Undergraduate of Department of Computer Science and Engineering, >>> > University of Moratuwa, >>> > Sri Lanka. >>> > >>> > <https://lk.linkedin.com/in/imeshasudasingha> < >>> https://github.com/IMS94> >>> > <http://stackoverflow.com/users/4012073/imesha-sudasingha> >>> > <https://twitter.com/Imesha94> >>> > >>> > On 28 March 2017 at 01:13, Tom Barber <[email protected]> wrote: >>> > >>> > > Hi Imesha >>> > > >>> > > I'm trying to figure out if/how I can get access to it on the >>> dashboard. >>> > In >>> > > the mean time if you want to pastebin it, chuck it in a word doc, or >>> > > something completely different I'd be happy to take a look. >>> > > >>> > > Tom >>> > > >>> > > On Sun, Mar 26, 2017 at 3:09 PM, Imesha Sudasingha < >>> > > [email protected]> >>> > > wrote: >>> > > >>> > > > Hi Tom, >>> > > > >>> > > > Did you get a chance to look at my proposal? >>> > > > >>> > > > Kind Regards, >>> > > > *Imesha Sudasingha* >>> > > > Undergraduate of Department of Computer Science and Engineering, >>> > > > University of Moratuwa, >>> > > > Sri Lanka. >>> > > > >>> > > > <https://lk.linkedin.com/in/imeshasudasingha> < >>> > https://github.com/IMS94 >>> > > > >>> > > > <http://stackoverflow.com/users/4012073/imesha-sudasingha> >>> > > > <https://twitter.com/Imesha94> >>> > > > >>> > > > On 23 March 2017 at 10:40, Imesha Sudasingha < >>> [email protected]> >>> > > > wrote: >>> > > > >>> > > > > Hi Tom, >>> > > > > >>> > > > > I have written a draft proposal and shared it with the Apache >>> > Software >>> > > > > Foundation through the GSoC dashboard. If you get a chance, can >>> you >>> > > > please >>> > > > > review it and let me know what are the improvements required to >>> be >>> > > made. >>> > > > > >>> > > > > Thanks in advance! >>> > > > > >>> > > > > Kind Regards, >>> > > > > *Imesha Sudasingha* >>> > > > > Undergraduate of Department of Computer Science and Engineering, >>> > > > > University of Moratuwa, >>> > > > > Sri Lanka. >>> > > > > >>> > > > > <https://lk.linkedin.com/in/imeshasudasingha> < >>> > > https://github.com/IMS94 >>> > > > > >>> > > > > <http://stackoverflow.com/users/4012073/imesha-sudasingha> >>> > > > > <https://twitter.com/Imesha94> >>> > > > > >>> > > > > On 22 March 2017 at 14:36, Imesha Sudasingha < >>> > [email protected]> >>> > > > > wrote: >>> > > > > >>> > > > >> Hi Tom, >>> > > > >> >>> > > > >> I think I had missed your point previously. I was referring to >>> just >>> > > > >> storing the configuration of different components in different >>> > ZNodes >>> > > > and >>> > > > >> querying them later. Then I understood that we also need to have >>> > > > >> inheritance at zookeeper level in order to make it easy to >>> configure >>> > > > OODT >>> > > > >> clusters as you have mentioned in the issue [1]. To configure >>> > clusters >>> > > > of >>> > > > >> OODT components with minimum configuration, I will have to >>> review my >>> > > > design >>> > > > >> again and make a lot of improvements. For example, I have to >>> think >>> > > > about a >>> > > > >> way to store configuration in zookeeper with minimum >>> duplications >>> > and >>> > > > give >>> > > > >> components enough flexibility to initialize with no or minimum >>> > manual >>> > > > >> configuration. >>> > > > >> >>> > > > >> Since I have to submit the GSoC proposal before 3rd of April, I >>> > think >>> > > > >> that I won't have enough time to review the design and make the >>> > > > necessary >>> > > > >> improvements before that. Will that negatively affect the >>> > possibility >>> > > > of me >>> > > > >> being able to work on this project for GSoC 2017? Please let me >>> know >>> > > you >>> > > > >> opinion. >>> > > > >> >>> > > > >> Thank you! >>> > > > >> >>> > > > >> [1] https://issues.apache.org/jira/browse/OODT-945 >>> > > > >> >>> > > > >> Kind Regards, >>> > > > >> *Imesha Sudasingha* >>> > > > >> Undergraduate of Department of Computer Science and >>> Engineering, >>> > > > >> University of Moratuwa, >>> > > > >> Sri Lanka. >>> > > > >> >>> > > > >> <https://lk.linkedin.com/in/imeshasudasingha> < >>> > > > https://github.com/IMS94> >>> > > > >> <http://stackoverflow.com/users/4012073/imesha-sudasingha> >>> > > > >> <https://twitter.com/Imesha94> >>> > > > >> >>> > > > >> On 21 March 2017 at 10:52, Imesha Sudasingha < >>> > [email protected] >>> > > > >>> > > > >> wrote: >>> > > > >> >>> > > > >>> Hi Tom, >>> > > > >>> >>> > > > >>> Sorry for the late reply. Zookeeper is just providing an ZNode >>> > > > >>> structure, which can be considered as a tree where a node is >>> > accessed >>> > > > as >>> > > > >>> directories. For example, in [1] the file manager properties >>> of the >>> > > > >>> instance with name "*node1*" is stored at the ZPath (Path to >>> that >>> > > > >>> ZNode) "*/oodt/node1/file-mgr/filemgr.properties*". >>> > > > >>> >>> > > > >>> Therefore, my idea is that we can make such directories >>> (ZPaths) >>> > per >>> > > > >>> each instance in our cluster (ex: */oodt/node1 for instance >>> > "node1"*, >>> > > > */oodt/node2 >>> > > > >>> for instance "node2"* and so on). Then inside each such path >>> > > (assigned >>> > > > >>> per node), we will have separate ZNodes for each component. For >>> > > > example, in >>> > > > >>> */oodt/node1* we will have an ZNode /oodt/node1/file-mgr which >>> > stores >>> > > > >>> the configurations of file manager of node1. Similarly, an >>> ZNode >>> > > > >>> /oodt/node1/res-mgr will store the configuration of the >>> resource >>> > > > manager of >>> > > > >>> node1. >>> > > > >>> >>> > > > >>> Hope you got the idea. Since zookeeper is not some kind of a >>> graph >>> > > > >>> database, I couldn't think of a method to keep inheritance >>> among >>> > > > >>> configuration at Zookeeper's side. However, the design I >>> suggest >>> > will >>> > > > store >>> > > > >>> the configuration independently and will allow the >>> users/developers >>> > > to >>> > > > >>> query those configuration. Therefore in my opinion, we should >>> let >>> > the >>> > > > >>> developer to compare the configuration and make decisions. This >>> > way, >>> > > > we can >>> > > > >>> allow this distributed configuration manager to manage >>> > configuration >>> > > > of any >>> > > > >>> current component or any upcoming component making it more >>> > > extend-able. >>> > > > >>> >>> > > > >>> Can you let me know what do you think of it? >>> > > > >>> >>> > > > >>> Thank you! >>> > > > >>> >>> > > > >>> [1] https://drive.google.com/open? >>> id=0B415Bnipaf1lNThKYXZMN3pFemc >>> > > > >>> >>> > > > >>> Kind Regards, >>> > > > >>> *Imesha Sudasingha* >>> > > > >>> Undergraduate of Department of Computer Science and >>> Engineering, >>> > > > >>> University of Moratuwa, >>> > > > >>> Sri Lanka. >>> > > > >>> >>> > > > >>> <https://lk.linkedin.com/in/imeshasudasingha> >>> > > > >>> <https://github.com/IMS94> >>> > > > >>> <http://stackoverflow.com/users/4012073/imesha-sudasingha> >>> > > > >>> <https://twitter.com/Imesha94> >>> > > > >>> >>> > > > >>> On 20 March 2017 at 05:02, Tom Barber <[email protected] >>> > >>> > > wrote: >>> > > > >>> >>> > > > >>>> Hi Imesha >>> > > > >>>> >>> > > > >>>> Sorry for the delay, I've been travelling and only just got >>> myself >>> > > > >>>> through >>> > > > >>>> my backlog. >>> > > > >>>> >>> > > > >>>> The overall design looks good. I was also thinking it would >>> make >>> > > sense >>> > > > >>>> if >>> > > > >>>> possible to have some form of inheritance in the config. So, >>> for >>> > > > >>>> example, >>> > > > >>>> if you have 5 file managers, and they all share the same >>> config >>> > > > except a >>> > > > >>>> base url or whatever, it would make sense to have a single >>> set of >>> > > > >>>> configs >>> > > > >>>> and then individual overrides for configs that change. (My >>> > knowledge >>> > > > of >>> > > > >>>> Zookeeper isn't great, I'm assuming something along these >>> lines is >>> > > > >>>> possible). >>> > > > >>>> >>> > > > >>>> It would also make sense, I think to register this stuff with >>> the >>> > > > >>>> resource >>> > > > >>>> manager, but I'm not sure how that integration would work >>> quite >>> > yet, >>> > > > so >>> > > > >>>> we'll need to figure that one out. >>> > > > >>>> >>> > > > >>>> Let me know if you have any questions about the above. >>> > > > >>>> >>> > > > >>>> Tom >>> > > > >>>> >>> > > > >>>> >>> > > > >>>> On Sat, Mar 18, 2017 at 12:39 PM, Imesha Sudasingha < >>> > > > >>>> [email protected] >>> > > > >>>> > wrote: >>> > > > >>>> >>> > > > >>>> > Hi Tom, >>> > > > >>>> > >>> > > > >>>> > What do you think of the design? Do you agree on >>> implementing >>> > this >>> > > > >>>> > configuration management section in a separate module? >>> > > > >>>> > >>> > > > >>>> > I was thinking on starting on the implementation as a proof >>> of >>> > > > >>>> concept. Can >>> > > > >>>> > you give me your kind feedback before I start on that? >>> > > > >>>> > >>> > > > >>>> > Thank you! >>> > > > >>>> > >>> > > > >>>> > Kind Regards, >>> > > > >>>> > *Imesha Sudasingha* >>> > > > >>>> > Undergraduate of Department of Computer Science and >>> > Engineering, >>> > > > >>>> > University of Moratuwa, >>> > > > >>>> > Sri Lanka. >>> > > > >>>> > >>> > > > >>>> > <https://lk.linkedin.com/in/imeshasudasingha> < >>> > > > >>>> https://github.com/IMS94> >>> > > > >>>> > <http://stackoverflow.com/users/4012073/imesha-sudasingha> >>> > > > >>>> > <https://twitter.com/Imesha94> >>> > > > >>>> > >>> > > > >>>> > On 16 March 2017 at 17:51, Imesha Sudasingha < >>> > > > [email protected] >>> > > > >>>> > >>> > > > >>>> > wrote: >>> > > > >>>> > >>> > > > >>>> > > Hi Tom, >>> > > > >>>> > > >>> > > > >>>> > > Thanks for the quick reply. I came up with a small design >>> [1]. >>> > > > >>>> > > >>> > > > >>>> > > What I suggest is a separate module for configuration >>> > > management, >>> > > > >>>> which >>> > > > >>>> > > can be extended in future and used by many components as >>> well. >>> > > > >>>> > > >>> > > > >>>> > > As shown in the design [1], File Manager (or any component >>> > that >>> > > is >>> > > > >>>> > willing >>> > > > >>>> > > to have this extended functionality of distributed >>> > configuration >>> > > > >>>> > > management) will have an instance of >>> *ConfigurationManager* >>> > > which >>> > > > >>>> is an >>> > > > >>>> > > interface defined to load configuration (specifically, >>> > > .properties >>> > > > >>>> files) >>> > > > >>>> > > and get configurations of each running instance of the >>> same >>> > > > >>>> component (If >>> > > > >>>> > > distributed configuration management is enabled, this will >>> > > return >>> > > > >>>> the >>> > > > >>>> > > configuration of all the other instances). >>> > > > >>>> > > >>> > > > >>>> > > There will be two implementing classes of that interface, >>> > > > >>>> > > *StandaloneConfigurationManager* and >>> > > > *DistributedConfigurationManag >>> > > > >>>> er.* >>> > > > >>>> > As >>> > > > >>>> > > the name implies, standalone configuration manager is to >>> be >>> > used >>> > > > if >>> > > > >>>> the >>> > > > >>>> > > users do not intend to used distributed configuration >>> > > management. >>> > > > >>>> Also >>> > > > >>>> > this >>> > > > >>>> > > will be same as the current mechanism of configuration >>> > > management. >>> > > > >>>> Then >>> > > > >>>> > the >>> > > > >>>> > > distributed configuration manager will be the >>> implementation >>> > > using >>> > > > >>>> > > Zookeeper. >>> > > > >>>> > > >>> > > > >>>> > > At zookeeper level, we will need to have a proper naming >>> > > mechanism >>> > > > >>>> and a >>> > > > >>>> > > structure to store configuration. For example,* >>> > > > >>>> > > /${nodeName}/${componentName}/{$configFileName}* will be >>> the >>> > > > place >>> > > > >>>> where >>> > > > >>>> > > the configuration file contents will be stored. >>> > *getFile(String >>> > > > >>>> fileName) >>> > > > >>>> > > and getFiles(String directiryName)* of *NodeConfiguration* >>> > class >>> > > > >>>> will >>> > > > >>>> > > return the content of the configuration files and >>> > configuration >>> > > > >>>> > directories >>> > > > >>>> > > respectively. I have demonstrated the ZNode structre I'm >>> > > expecting >>> > > > >>>> to >>> > > > >>>> > > design in [2] >>> > > > >>>> > > >>> > > > >>>> > > Please note that this is just a very high level overview >>> of my >>> > > > >>>> design and >>> > > > >>>> > > the real difficulty of implementation comes when testing >>> this >>> > > and >>> > > > >>>> > > addressing the edge cases of zookeeper. Also, I have only >>> > added >>> > > > few >>> > > > >>>> > methods >>> > > > >>>> > > that came to my mind to the *ConfigurationManager* >>> interface. >>> > If >>> > > > >>>> you find >>> > > > >>>> > > any fault in this design or have a better design in mind >>> > please >>> > > > let >>> > > > >>>> me >>> > > > >>>> > know >>> > > > >>>> > > so that I can optimize this design and do my best to come >>> up >>> > > with >>> > > > >>>> the >>> > > > >>>> > best >>> > > > >>>> > > possible solution. >>> > > > >>>> > > >>> > > > >>>> > > Thank you! >>> > > > >>>> > > >>> > > > >>>> > > [1] https://drive.google.com/open?id= >>> > > 0B415Bnipaf1ldnV1dWt4VVVLam8 >>> > > > >>>> > > [2] https://drive.google.com/open?id= >>> > > 0B415Bnipaf1lNThKYXZMN3pFemc >>> > > > >>>> > > >>> > > > >>>> > > Kind Regards, >>> > > > >>>> > > *Imesha Sudasingha* >>> > > > >>>> > > Undergraduate of Department of Computer Science and >>> > > Engineering, >>> > > > >>>> > > University of Moratuwa, >>> > > > >>>> > > Sri Lanka. >>> > > > >>>> > > >>> > > > >>>> > > <https://lk.linkedin.com/in/imeshasudasingha> < >>> > > > >>>> https://github.com/IMS94 >>> > > > >>>> > > >>> > > > >>>> > > <http://stackoverflow.com/users/4012073/imesha-sudasingha >>> > >>> > > > >>>> > > <https://twitter.com/Imesha94> >>> > > > >>>> > > >>> > > > >>>> > > On 16 March 2017 at 16:55, Tom Barber < >>> > [email protected]> >>> > > > >>>> wrote: >>> > > > >>>> > > >>> > > > >>>> > >> Hi Imesha, >>> > > > >>>> > >> >>> > > > >>>> > >> Basically, just allowing the dynamic registration of file >>> > > > managers >>> > > > >>>> so >>> > > > >>>> > that >>> > > > >>>> > >> new ones can come and go. >>> > > > >>>> > >> >>> > > > >>>> > >> Tom >>> > > > >>>> > >> >>> > > > >>>> > >> On Thu, Mar 16, 2017 at 10:00 AM, Imesha Sudasingha < >>> > > > >>>> > >> [email protected] >>> > > > >>>> > >> > wrote: >>> > > > >>>> > >> >>> > > > >>>> > >> > Hi Tom, >>> > > > >>>> > >> > >>> > > > >>>> > >> > Need a small clarification. In the last reply, you >>> > mentioned >>> > > > >>>> that file >>> > > > >>>> > >> > managers may be designed to be *transient. *What do you >>> > mean >>> > > by >>> > > > >>>> that? >>> > > > >>>> > A >>> > > > >>>> > >> > quick response would really help me as I'm trying to >>> come >>> > up >>> > > > >>>> with a >>> > > > >>>> > >> design. >>> > > > >>>> > >> > >>> > > > >>>> > >> > Thank you! >>> > > > >>>> > >> > >>> > > > >>>> > >> > *Imesha Sudasingha* >>> > > > >>>> > >> > Undergraduate of Department of Computer Science and >>> > > > Engineering, >>> > > > >>>> > >> > University of Moratuwa, >>> > > > >>>> > >> > Sri Lanka. >>> > > > >>>> > >> > >>> > > > >>>> > >> > <https://lk.linkedin.com/in/imeshasudasingha> < >>> > > > >>>> > >> https://github.com/IMS94> >>> > > > >>>> > >> > <http://stackoverflow.com/user >>> s/4012073/imesha-sudasingha> >>> > > > >>>> > >> > <https://twitter.com/Imesha94> >>> > > > >>>> > >> > >>> > > > >>>> > >> > On 13 March 2017 at 22:03, Imesha Sudasingha < >>> > > > >>>> [email protected] >>> > > > >>>> > > >>> > > > >>>> > >> > wrote: >>> > > > >>>> > >> > >>> > > > >>>> > >> > > Hi Tom, >>> > > > >>>> > >> > > >>> > > > >>>> > >> > > Thanks for the clarification. It really helped a lot >>> to >>> > > fill >>> > > > >>>> most of >>> > > > >>>> > >> the >>> > > > >>>> > >> > > gaps I had unanswered. I have a draft design in my >>> mind >>> > and >>> > > > >>>> will >>> > > > >>>> > post >>> > > > >>>> > >> for >>> > > > >>>> > >> > > your review with details soon. >>> > > > >>>> > >> > > >>> > > > >>>> > >> > > As for my current understanding, my task is to >>> implement >>> > a >>> > > > >>>> mechanism >>> > > > >>>> > >> to >>> > > > >>>> > >> > > switch/chose configuration mechanism (between local >>> files >>> > > and >>> > > > >>>> > zookeer >>> > > > >>>> > >> for >>> > > > >>>> > >> > > the moment) and then allow APIs to query those >>> mechanisms >>> > > (in >>> > > > >>>> the >>> > > > >>>> > >> > zookeeper >>> > > > >>>> > >> > > scenario) for future use (For File Managers to know >>> about >>> > > > other >>> > > > >>>> > >> instances >>> > > > >>>> > >> > > and make decisions accordingly). That include >>> > > > synchronization, >>> > > > >>>> > >> fetching >>> > > > >>>> > >> > > configuration from zookeeper if an instance go down >>> and >>> > > come >>> > > > >>>> again >>> > > > >>>> > >> and so >>> > > > >>>> > >> > > on. Am I correct? >>> > > > >>>> > >> > > >>> > > > >>>> > >> > > I will try to come up with the design I mentioned, so >>> > that >>> > > > >>>> both of >>> > > > >>>> > us >>> > > > >>>> > >> can >>> > > > >>>> > >> > > discuss more about the improvements and requirements >>> > soon. >>> > > > >>>> > >> > > >>> > > > >>>> > >> > > Thank you for the support! >>> > > > >>>> > >> > > >>> > > > >>>> > >> > > Kind Regards, >>> > > > >>>> > >> > > *Imesha Sudasingha* >>> > > > >>>> > >> > > Undergraduate of Department of Computer Science and >>> > > > >>>> Engineering, >>> > > > >>>> > >> > > University of Moratuwa, >>> > > > >>>> > >> > > Sri Lanka. >>> > > > >>>> > >> > > >>> > > > >>>> > >> > > <https://lk.linkedin.com/in/imeshasudasingha> < >>> > > > >>>> > >> https://github.com/IMS94 >>> > > > >>>> > >> > > >>> > > > >>>> > >> > > <http://stackoverflow.com/users/4012073/imesha- >>> > sudasingha> >>> > > > >>>> > >> > > <https://twitter.com/Imesha94> >>> > > > >>>> > >> > > >>> > > > >>>> > >> > > On 13 March 2017 at 20:19, Tom Barber < >>> > > > [email protected] >>> > > > >>>> > >>> > > > >>>> > >> wrote: >>> > > > >>>> > >> > > >>> > > > >>>> > >> > >> Hi Imesha >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> The best way to demonstrate this is to show you a >>> > > reference >>> > > > >>>> build >>> > > > >>>> > for >>> > > > >>>> > >> > >> OODT.. >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> https://dl.dropboxusercontent. >>> > > > com/u/8503756/oodt-example.tgz >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> If you can grab that archive and extract it, you'll >>> see >>> > > that >>> > > > >>>> is how >>> > > > >>>> > >> > >> currently OODT is deployed. >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> If you take the file manager as the reference module >>> > > inside >>> > > > >>>> > >> oodt/filemgr >>> > > > >>>> > >> > >> you'll see >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> etc/filemgr.properties >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> and >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> policy/oodt/* >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> These are the configuration locations of a single >>> file >>> > > > >>>> manager. >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> The problem I want to solve is this: >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> a) If I run > 1 file manager they are completely >>> > separate >>> > > > and >>> > > > >>>> they >>> > > > >>>> > >> don't >>> > > > >>>> > >> > >> know of each other. So using Zookeeper will allow >>> > multiple >>> > > > >>>> file >>> > > > >>>> > >> managers >>> > > > >>>> > >> > >> to >>> > > > >>>> > >> > >> register against ZK and learn of each other. >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> b) Backing them onto ZK will allow file managers to >>> come >>> > > and >>> > > > >>>> go on >>> > > > >>>> > >> the >>> > > > >>>> > >> > >> fly, >>> > > > >>>> > >> > >> providing scaling capabilities or maybe fm's that >>> are >>> > > > >>>> designed to >>> > > > >>>> > be >>> > > > >>>> > >> > >> transient. >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> Just for clarity, the ZK configuration will be an >>> > optional >>> > > > >>>> > >> requirement, >>> > > > >>>> > >> > so >>> > > > >>>> > >> > >> we need to extend the existing configuration >>> mechanism, >>> > to >>> > > > >>>> support >>> > > > >>>> > >> both >>> > > > >>>> > >> > >> the >>> > > > >>>> > >> > >> current config and the ZK config instead of ZK >>> being >>> > > > >>>> mandatory. >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> Tom >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> On Mon, Mar 13, 2017 at 3:20 AM, Imesha Sudasingha < >>> > > > >>>> > >> > >> [email protected]> >>> > > > >>>> > >> > >> wrote: >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > >> > Hi Tom, >>> > > > >>>> > >> > >> > >>> > > > >>>> > >> > >> > Related to the conversation on JIRA [1] ; >>> > > > >>>> > >> > >> > >>> > > > >>>> > >> > >> > I went through the CAS-File manager module and I >>> now >>> > > > >>>> understand >>> > > > >>>> > >> how to >>> > > > >>>> > >> > >> > address ".properties" files. Since you mentioned >>> in >>> > JIRA >>> > > > >>>> [2] that >>> > > > >>>> > >> > there >>> > > > >>>> > >> > >> are >>> > > > >>>> > >> > >> > occasions where we use *spring configuration files >>> > *for >>> > > > >>>> > >> configuration >>> > > > >>>> > >> > >> > purposes, I want to have a look at that type of >>> > > > >>>> configuration >>> > > > >>>> > too. >>> > > > >>>> > >> Can >>> > > > >>>> > >> > >> you >>> > > > >>>> > >> > >> > give me an example module where spring >>> configuration >>> > > files >>> > > > >>>> are >>> > > > >>>> > >> used to >>> > > > >>>> > >> > >> load >>> > > > >>>> > >> > >> > configuration? >>> > > > >>>> > >> > >> > >>> > > > >>>> > >> > >> > Also, will the content of a configuration file >>> exceed >>> > > 1MB >>> > > > >>>> > (Maximum >>> > > > >>>> > >> > data >>> > > > >>>> > >> > >> > size allowed in an ZNode) in any case? >>> > > > >>>> > >> > >> > >>> > > > >>>> > >> > >> > Thank you! >>> > > > >>>> > >> > >> > >>> > > > >>>> > >> > >> > [1] >>> > > > >>>> > >> > >> > https://issues.apache.org/jira >>> > > > >>>> /browse/OODT-945?focusedCommen >>> > > > >>>> > >> > >> tId=15906204& >>> > > > >>>> > >> > >> > page=com.atlassian.jira.plugin >>> .system.issuetabpanels: >>> > > > >>>> > >> > >> > comment-tabpanel#comment-15906204 >>> > > > >>>> > >> > >> > >>> > > > >>>> > >> > >> > [2] >>> > > > >>>> > >> > >> > https://issues.apache.org/jira >>> > > > >>>> /browse/OODT-945?focusedCommen >>> > > > >>>> > >> > >> tId=15906524& >>> > > > >>>> > >> > >> > page=com.atlassian.jira.plugin >>> .system.issuetabpanels: >>> > > > >>>> > >> > >> > comment-tabpanel#comment-15906524 >>> > > > >>>> > >> > >> > >>> > > > >>>> > >> > >> > Kind Regards, >>> > > > >>>> > >> > >> > *Imesha Sudasingha* >>> > > > >>>> > >> > >> > Undergraduate of Department of Computer Science >>> and >>> > > > >>>> Engineering, >>> > > > >>>> > >> > >> > University of Moratuwa, >>> > > > >>>> > >> > >> > Sri Lanka. >>> > > > >>>> > >> > >> > >>> > > > >>>> > >> > >> > <https://lk.linkedin.com/in/imeshasudasingha> < >>> > > > >>>> > >> > >> https://github.com/IMS94> >>> > > > >>>> > >> > >> > <http://stackoverflow.com/users/4012073/imesha- >>> > > sudasingha >>> > > > > >>> > > > >>>> > >> > >> > <https://twitter.com/Imesha94> >>> > > > >>>> > >> > >> > >>> > > > >>>> > >> > >> >>> > > > >>>> > >> > > >>> > > > >>>> > >> > > >>> > > > >>>> > >> > >>> > > > >>>> > >> >>> > > > >>>> > > >>> > > > >>>> > > >>> > > > >>>> > >>> > > > >>>> >>> > > > >>> >>> > > > >>> >>> > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> >> >> >
