Thanks for the quick reply. Well yea 1 billion is an exaggerated number I agree :). Lets bring it drastically down to 1million pages. The use case I have is a public facing website similar to wikipedia (with a different business model). Each page would have 5 to 7 images and 5 links to youtube videos. It should also be able to show all the pages written or edited by a particular user in his account setting (or in the left nav when he is logged in). System crash and Performance is my big concern. I believe if I setup HDFS and setup a cluster for the pages then I am safe for the system crash. But If all the pages has to be loaded on memory then it is only vertical scalable and may not provide a high availability system (Imagining a use case when the system goes down and an another system is spawning up ,meanwhile, all the edits being done at that moment would be lost and users will be logged out).
On 2018/03/22 09:10:08, "lgilardon...@gmail.com" <lgilardon...@gmail.com> wrote: > That's a good point Jürgen. > > This however raised two question in my mind. > First one is that *could* be a main point for future work (it would mean > to build a persistent intralinks repo to be buid and maintained > incrementally on pages change - which is surely a major overhaul but > does not look like rocket science). > Second one however - as a consequence - is .. when this would became a > real issue? 1k pages (believe not - we are running an intranet > that order of magnitude size and does not looks a real issue)? 10k? 100k? > Last consequential answer .. letting apart rebuilding wikipedia (which > it is not a real case i guess - and anyway as from > https://en.wikipedia.org/wiki/Wikipedia:Size_comparisons > it is far from that number) I hardly can image a 1B pages wiki. > @dagarwal82 may we know which kind of wiki (i.e. which kind of content) > are you thinking about? > > On 3/22/2018 8:26 AM, Jürgen Weber wrote: > > guess not, on startup JSPWiki loads all pages into memory to parse > > intra-page links. > > > > 2018-03-22 5:24 GMT+01:00 dagarwa...@gmail.com <dagarwa...@gmail.com>: > >> JspWiki uses filesystem instead of a conventional database. Would you > >> suggest using jspwiki if it were to cater to 1billion page ? (Provided, a > >> solid backup mechanism ). Would it scale that much with both performance > >> and data-integrity perspective ? > > >
