Thanks for the suggestions and in our initial design we had the same breakdown as you suggest.We intend to use G-Reg as the meta data layer. I had the understanding that we can plug the file system to g-reg as the data layer and store data into that through registry API instead to the database.
As i have understood the data layer should be separately handled (As a different component) and collaborate with G-Reg as the metadata layer. On Thu, Jan 6, 2011 at 10:10 AM, Samisa Abeysinghe <[email protected]> wrote: > +1 for using file system! > > > On Wed, Jan 5, 2011 at 5:43 PM, Senaka Fernando <[email protected]> wrote: > >> >> >> On Wed, Jan 5, 2011 at 4:03 PM, Sanjiva Weerawarana <[email protected]>wrote: >> >>> +1 .. I think we need to design the file storage component we're going to >>> do for filebus properly and use that here too to store stuff bigger than >>> some size on the filesystem and use the filebus features to manage/scale >>> that side. >> >> >> +1. >> >> Thanks, >> Senaka. >> >>> >>> Sanjiva. >>> >>> >>> On Wed, Jan 5, 2011 at 3:52 PM, Paul Fremantle <[email protected]> wrote: >>> >>>> I can't think it is a good idea to store Gb size data in our Registry. I >>>> would think you would be better storing the files in a filesystem and using >>>> the Registry to handle the metadata. >>>> >>>> Paul >>>> >>>> >>>> On 3 January 2011 08:48, Senaka Fernando <[email protected]> wrote: >>>> >>>>> >>>>> >>>>> On Mon, Jan 3, 2011 at 11:46 AM, Kishanthan Thangarajah < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Senaka, >>>>>> I tried several times increasing the memory size, but i cant upload a >>>>>> file with size of 700mb to registry. Always getting the java memory(heap >>>>>> size) error. >>>>>> My ram is 3GB, >>>>>> There are several parameters i find in wrapper.conf. >>>>>> -Xms256m >>>>>> -Xmx512m >>>>>> -XX:MaxPermSize=128m >>>>>> >>>>>> # Initial Java Heap Size (in MB) >>>>>> wrapper.java.initmemory=256 >>>>>> # Maximum Java Heap Size (in MB) >>>>>> wrapper.java.maxmemory=512 >>>>>> >>>>>> what i did is that i incresed the maxmemory to 1024 and MaxPermSize to >>>>>> 512m. >>>>>> Do i need to change anything else? >>>>>> >>>>> >>>>> No, AFAIU, you've done it correctly, but you'll need more memory. For >>>>> example, a PermSize of 128M can handle 100MB, for 700MB, I believe that >>>>> you'll need a PermSize of around 900MB. You will also need to increase the >>>>> other memory parameters accordingly. >>>>> >>>>> Thanks, >>>>> Senaka. >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> On Thu, Dec 30, 2010 at 10:27 PM, shelan Perera >>>>>> <[email protected]>wrote: >>>>>> >>>>>>> Hi Senaka, >>>>>>> >>>>>>> File size may vary few Giga bytes (10-100 with realistic limits). >>>>>>> Actually it is an attempt to create a big storage aggregating few >>>>>>> G-Regs through a peer to peer network.I need to have a persistent >>>>>>> storage >>>>>>> attached to G-Reg ( Such as a file system). Would G-Reg be efficient in >>>>>>> handling or have the feasibility. >>>>>>> >>>>>>> >>>>>>> On Thu, Dec 30, 2010 at 10:01 PM, Senaka Fernando >>>>>>> <[email protected]>wrote: >>>>>>> >>>>>>>> Hi Shelan, >>>>>>>> >>>>>>>> When you say large chunks how big are these? >>>>>>>> >>>>>>>> The maximum file upload size varies with the maximum amount of >>>>>>>> memory available. Also, you need to edit a parameter on the carbon.xml >>>>>>>> file >>>>>>>> to increase the default file upload size of 100MB, and the >>>>>>>> wso2server.sh (or >>>>>>>> wrapper.conf) to increase the default allocation of memory (which is >>>>>>>> 512MB >>>>>>>> with 128MB permanent generation space). >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Senaka. >>>>>>>> >>>>>>>> On Thu, Dec 30, 2010 at 9:12 PM, shelan Perera >>>>>>>> <[email protected]>wrote: >>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> I have an requirement to upload large chunks of data to G-Reg. >>>>>>>>> What is the maximum file upload size G-Reg can handle if I upload >>>>>>>>> remotely to a server.? >>>>>>>>> >>>>>>>>> I intend to use WSO2 G-Reg as a meta data management layer at the >>>>>>>>> primary level but would like to know is it capable of handling large >>>>>>>>> data >>>>>>>>> chunks to upload through G-Reg directly. >>>>>>>>> Could you please enlighten me on the suitability of G-Reg for the >>>>>>>>> above requirement. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Shelan Perera >>>>>>>>> >>>>>>>>> Blog : http://www.shelanlk.com >>>>>>>>> Twitter:shelan >>>>>>>>> skype :shelan.perera >>>>>>>>> gtalk :shelanrc >>>>>>>>> >>>>>>>>> I am the master of my fate: >>>>>>>>> I am the captain of my soul. >>>>>>>>> *invictus* >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Carbon-dev mailing list >>>>>>>>> [email protected] >>>>>>>>> https://wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Senaka Fernando* >>>>>>>> Associate Technical Lead & Product Manager - WSO2 G-Reg; >>>>>>>> WSO2, Inc.; http://wso2.com* >>>>>>>> Member; Apache Software Foundation; http://apache.org >>>>>>>> >>>>>>>> E-mail: senaka AT wso2.com >>>>>>>> **P: +1 408 754 7388; ext: 51736*; *M: +94 77 322 1818 >>>>>>>> Linked-In: http://www.linkedin.com/in/senakafernando >>>>>>>> >>>>>>>> *Lean . Enterprise . Middleware >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Carbon-dev mailing list >>>>>>>> [email protected] >>>>>>>> https://wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Shelan Perera >>>>>>> >>>>>>> Blog : http://www.shelan.info >>>>>>> Twitter:shelan >>>>>>> skype :shelan.perera >>>>>>> gtalk :shelanrc >>>>>>> >>>>>>> I am the master of my fate: >>>>>>> I am the captain of my soul. >>>>>>> *invictus* >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Carbon-dev mailing list >>>>>>> [email protected] >>>>>>> https://wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Kishanthan.T >>>>>> University of Moratuwa. >>>>>> SriLanka. >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Senaka Fernando* >>>>> Associate Technical Lead & Product Manager - WSO2 G-Reg; >>>>> WSO2, Inc.; http://wso2.com* >>>>> Member; Apache Software Foundation; http://apache.org >>>>> >>>>> E-mail: senaka AT wso2.com >>>>> **P: +1 408 754 7388; ext: 51736*; *M: +94 77 322 1818 >>>>> Linked-In: http://www.linkedin.com/in/senakafernando >>>>> >>>>> *Lean . Enterprise . Middleware >>>>> >>>>> >>>>> _______________________________________________ >>>>> Carbon-dev mailing list >>>>> [email protected] >>>>> https://wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>>>> >>>>> >>>> >>>> >>>> -- >>>> Paul Fremantle >>>> CTO and Co-Founder, WSO2 >>>> OASIS WS-RX TC Co-chair, VP, Apache Synapse >>>> >>>> Office: +44 844 484 8143 >>>> Cell: +44 798 447 4618 >>>> >>>> blog: http://pzf.fremantle.org >>>> twitter.com/pzfreo >>>> [email protected] >>>> >>>> wso2.com Lean Enterprise Middleware >>>> >>>> Disclaimer: This communication may contain privileged or other >>>> confidential information and is intended exclusively for the addressee/s. >>>> If >>>> you are not the intended recipient/s, or believe that you may have received >>>> this communication in error, please reply to the sender indicating that >>>> fact >>>> and delete the copy you received and in addition, you should not print, >>>> copy, retransmit, disseminate, or otherwise use the information contained >>>> in >>>> this communication. Internet communications cannot be guaranteed to be >>>> timely, secure, error or virus-free. The sender does not accept liability >>>> for any errors or omissions. >>>> >>>> _______________________________________________ >>>> Carbon-dev mailing list >>>> [email protected] >>>> https://wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>>> >>>> >>> >>> >>> -- >>> Sanjiva Weerawarana, Ph.D. >>> Founder, Chairman & CEO; WSO2, Inc.; http://wso2.com/ >>> email: [email protected]; phone: +94 11 763 9614; cell: +94 77 787 6880 | >>> +1 650 265 8311 >>> blog: http://sanjiva.weerawarana.org/ >>> >>> Lean . Enterprise . Middleware >>> >>> _______________________________________________ >>> Carbon-dev mailing list >>> [email protected] >>> https://wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>> >>> >> >> >> -- >> *Senaka Fernando* >> Associate Technical Lead & Product Manager - WSO2 G-Reg; >> WSO2, Inc.; http://wso2.com* >> Member; Apache Software Foundation; http://apache.org >> >> E-mail: senaka AT wso2.com >> **P: +1 408 754 7388; ext: 51736*; *M: +94 77 322 1818 >> Linked-In: http://www.linkedin.com/in/senakafernando >> >> *Lean . Enterprise . Middleware >> >> >> _______________________________________________ >> Carbon-dev mailing list >> [email protected] >> https://wso2.org/cgi-bin/mailman/listinfo/carbon-dev >> >> Thanks, > Samisa... > > Samisa Abeysinghe > VP Engineering > WSO2 Inc. > http://wso2.com > http://wso2.org > > > > > _______________________________________________ > Carbon-dev mailing list > [email protected] > https://wso2.org/cgi-bin/mailman/listinfo/carbon-dev > > -- Shelan Perera Blog : http://www.shelan.info Twitter:shelan skype :shelan.perera gtalk :shelanrc I am the master of my fate: I am the captain of my soul. *invictus*
_______________________________________________ Carbon-dev mailing list [email protected] https://wso2.org/cgi-bin/mailman/listinfo/carbon-dev
