Re: Can jspwiki be used for large deployment

[Juan Pablo Santos Rodríguez]

Hi,

a little late to chime in, but did wanted to note a couple things:

- there's a contributed jdbc provider ([#1], referenced at [#2]), so it is
possible to store your wiki pages on a database

- if I'm not mistaken, all pages aren't stored on memory, references, both
ingoing and outgoing, to them are ([#3]). Those references are Strings
(page names), so references should be on the order of hundreds of bytes, as
opposed to whole pages. These references are used to perform searches like
top ten lists, unreferenced pages, etc. which otherwise would be expensive
to make. There's also the lucene index which IIRC is backed by a file, and
ehcache if you're have caching turned on, which you can customize too. I've
never tried to see where the upper limit on pages would be, so I'm not
really sure how many pages JSPWiki can handle on an standard installation.

Out of curiosity, I've done an mvn tomcat7:run-war with latest JSPWiki and
the default set of pages and the amount of memory it takes is ~60MB, on a
JDK 1.8.0_121, no memory parametrization at all. Would anyone mind sharing
the amount of pages vs JDK - memory settings vs memory consumed on JSPWiki
instances?

Last, @V.Fedorov, ELWiki looks really interesting! is there any link we
could use to refer to it at
https://jspwiki-wiki.apache.org/Wiki.jsp?page=JSPWikiApplications ?


best regards,
juan pablo


#1]: https://github.com/djemerson01/jspwiki-jdbcprovider
[#2]: https://jspwiki-wiki.apache.org/Wiki.jsp?page=ContributedProviders
[#3]:
https://github.com/apache/jspwiki/blob/master/jspwiki-war/src/main/java/org/apache/wiki/ReferenceManager.java

On Thu, Mar 22, 2018 at 7:16 PM, V. Fedorov  wrote:

> Believe it or not, but the JSPwiki code is already ported on components
> - OSGi of a bandles, and...:
>  - the layer of visualization has produced with use of Eclipse RAP;
>  - data are submitted by EMF model;
>  - the persistence of data is carried out by means of CDO.
>
> Screenshot of the functioning wiki - see https://yadi.sk/i/UezpmVo63Tf4
> 2g
> (in the picture: at the left - IDE; on the right - the browser with
> ElWiki).
>
> There is a one change in syntax of a wiki-marking - for references
> between pages it is required to specify the page identifier.
> For example: [Sample page | @218]
> Because of it the name wiki is: ElWiki (Enhanced links wiki).
> The data model of these pages is organized hierarchically, references
> between them are based on identifiers of pages.
> So existence of pages with identical names, for example in different
> branches of wiki is possible.
>
> Used data model is presented here - https://yadi.sk/i/jaljk1m23Tf3zw
>
> In the presented realization there is no need to load all pages, for
> processing of communications between them.
>
> Work on the project is continued. (changes has been started and
> continuing by me)
>
> PS: In general, I use JSPwiki for logging of my practices, notes from
> books, articles.
> Though in my option - the pages of JSPwiki are stored in the database,
> not on a disk,
>  but in general, loading all pages in memory, impossibility to set the
> any name of page, and etc. -- doesn't suit me now.
>
> On Чт, 2018-03-22 at 08:26 +0100, Jürgen Weber wrote:
> > guess not, on startup JSPWiki loads all pages into memory to parse
> > intra-page links.
> >
> > 2018-03-22 5:24 GMT+01:00 dagarwa...@gmail.com 
> > :
> > >
> > > JspWiki uses filesystem instead of a conventional database. Would
> > > you suggest using jspwiki if it were to cater to 1billion page ?
> > > (Provided, a solid backup mechanism ). Would it scale that much
> > > with both performance and data-integrity perspective ?
>

Re: Can jspwiki be used for large deployment

[dagarwal82]

Thanks for the quick reply. Well yea 1 billion is an exaggerated number I agree 
:). Lets bring it drastically down to 1million pages. The use case I have is a 
public facing website similar to wikipedia (with a different business model). 
Each page would have 5 to 7 images and 5 links to youtube videos. It should 
also be able to show all the pages written or edited by a particular user in 
his account setting (or in the left nav when he is logged in). System crash and 
Performance is my big concern. I believe if I setup HDFS and setup a cluster 
for the pages then I am safe for the system crash. But If all the pages has to 
be loaded on memory then it is only vertical scalable and may not provide a 
high availability system (Imagining a use case when the system goes down and an 
another system is spawning up ,meanwhile, all the edits being done at that 
moment would be lost and users will be logged out).

On 2018/03/22 09:10:08, "lgilardon...@gmail.com"  
wrote: 
> That's a good point Jürgen.
> 
> This however raised two question in my mind.
> First one is that *could* be a main point for future work (it would mean 
> to build a persistent intralinks repo to be buid and maintained
> incrementally on pages change - which is surely a major overhaul but 
> does not look like rocket science).
> Second one however - as a consequence - is .. when this would became a 
> real issue? 1k pages (believe not - we are running an intranet
> that order of magnitude size and does not looks a real issue)? 10k? 100k?
> Last consequential answer .. letting apart rebuilding wikipedia (which 
> it is not a real case i guess - and anyway as from 
> https://en.wikipedia.org/wiki/Wikipedia:Size_comparisons
> it is far from that number) I hardly can image a 1B pages wiki.
> @dagarwal82 may we know which kind of wiki (i.e. which kind of content) 
> are you thinking about?
> 
> On 3/22/2018 8:26 AM, Jürgen Weber wrote:
> > guess not, on startup JSPWiki loads all pages into memory to parse
> > intra-page links.
> >
> > 2018-03-22 5:24 GMT+01:00 dagarwa...@gmail.com :
> >> JspWiki uses filesystem instead of a conventional database. Would you 
> >> suggest using jspwiki if it were to cater to 1billion page ? (Provided, a 
> >> solid backup mechanism ). Would it scale that much with both performance 
> >> and data-integrity perspective ?
> 
> 
> 

Re: Can jspwiki be used for large deployment

[lgilardon...@gmail.com]

That's a good point Jürgen.

This however raised two question in my mind.
First one is that *could* be a main point for future work (it would mean 
to build a persistent intralinks repo to be buid and maintained
incrementally on pages change - which is surely a major overhaul but 
does not look like rocket science).
Second one however - as a consequence - is .. when this would became a 
real issue? 1k pages (believe not - we are running an intranet

that order of magnitude size and does not looks a real issue)? 10k? 100k?
Last consequential answer .. letting apart rebuilding wikipedia (which 
it is not a real case i guess - and anyway as from 
https://en.wikipedia.org/wiki/Wikipedia:Size_comparisons

it is far from that number) I hardly can image a 1B pages wiki.
@dagarwal82 may we know which kind of wiki (i.e. which kind of content) 
are you thinking about?


On 3/22/2018 8:26 AM, Jürgen Weber wrote:

guess not, on startup JSPWiki loads all pages into memory to parse
intra-page links.

2018-03-22 5:24 GMT+01:00 dagarwa...@gmail.com :

JspWiki uses filesystem instead of a conventional database. Would you suggest 
using jspwiki if it were to cater to 1billion page ? (Provided, a solid backup 
mechanism ). Would it scale that much with both performance and data-integrity 
perspective ?

Re: Can jspwiki be used for large deployment

[Jürgen Weber]

guess not, on startup JSPWiki loads all pages into memory to parse
intra-page links.

2018-03-22 5:24 GMT+01:00 dagarwa...@gmail.com :
> JspWiki uses filesystem instead of a conventional database. Would you suggest 
> using jspwiki if it were to cater to 1billion page ? (Provided, a solid 
> backup mechanism ). Would it scale that much with both performance and 
> data-integrity perspective ?

Can jspwiki be used for large deployment , say 1billion pages

[dagarwal82]

Hey Guys,

I was looking around a good apache java wiki engine and I came across jspwiki. 
It looks beautiful. However, I noticed it is using filesystem not conventional 
databases. So, I was wondering if it is really suitable for a large deployment 
where number of pages may go beyond a billion. I understand it would require 
large servers and backup strategy. But is jspwiki suitable for this much of 
data in terms of performance and data-integrity.