About 10M hits a day. No java app hosted on a single machine would survive for 5 minutes with our load.
----- Original Message ---- > From: Adam Heath <[email protected]> > To: Joe Schaefer <[email protected]> > Cc: Jacques Le Roux <[email protected]>; [email protected]; >[email protected] > Sent: Tue, September 21, 2010 5:48:26 PM > Subject: Re: ofbiz wiki(confluence) > > On 09/21/2010 04:41 PM, Joe Schaefer wrote: > > Sounds interesting, but for us we require static exports. > > Since you're using flat files that might not be all that > > hard for you to implement. > > Why? What kind of load do you have? We(brainfood) have survied > slashdotting, without resorting to fancy frontends like varnish. It's > been written to be nonblocking(no synchronized keywords), use > weak/soft references, and not create sessions until absolutely nescessary. > > > Confluence as a CMS has an interesting future ahead of it > > at the ASF. Right now we have a hard dependency on the > > auto-export plugin, whose support characteristics prevent > > us from running the latest versions of confluence. If > > the situation doesn't change over the next few months, > > we'll likely just phase out the CMS aspects of confluence > > and replace it with something that natively supports > > static exports. > > We would like to support static exports too, and it might even be > possible, with little effort. However, it's just not been nescessary > for us, as we've never had a problem with any kind of load whatsoever. > > > ----- Original Message ---- > >> From: Adam Heath<[email protected]> > >> To: Jacques Le Roux<[email protected]> > >> Cc: Joe Schaefer<[email protected]>; [email protected]; > >> [email protected] > >> Sent: Tue, September 21, 2010 5:34:10 PM > >> Subject: Re: ofbiz wiki(confluence) > >> > >> On 09/21/2010 03:53 PM, Jacques Le Roux wrote: > >>> Thanks Joe, > >>> > >>> I quickly tried through Subclipse and got an error. > >>> I guess now Adam has a better idea of what I was talking about. > >>> I mean maybe Webslinger could be used, just my 2 cts... > >> > >> I will attempt to describe webslinger for those who haven't ever heard > >> of >it > >> before. > >> > >> The major features(bullet points) of webslinger-core are: > >> > >> * Content data stored as raw files. This is to allow normal programs, >like > >> grep, find, vim, dreamweaver, photoshop, git, svn work without >modifications. > >> > >> * Makes use of commons-vfs, and a custom set of layered filesystems. > >> > >> * One layered filesystem is called 'flat'. Arbitrary attributes > >> (FileObject.getContent().getAttribute(name)) are stored as path/to/file@, > >> >into > >> separate files. Again, this allows for easy integration with other >systems. > >> > >> * Another layered filesystem is called 'wsvfs'. This is an overlay/cow >type > >> filesystem, where multiple real filesystems are combined on the fly, >giving > >> merged directory listings, with support for up-copy and whiteout. Any >point of > >> the tree can 'overlay' any other part of the tree, altho this feature >isn't > >> normally nescessary. > >> > >> * Automatic extension resolution. This allows for pretty urls that > >> don't >have > >> extensions, and allow the implementation on the server to be changed as > >> nescessary. End-users have problems with extensions, so that is hidden. > >> > >> * Any 'path' can be configured to do it's own sub-path management. This >allows > >> for nice urls like /shop/product/$productId/detail and > >> /shop/cart/add/$productId and /Login/Path/To/Protected/Page. These urls >then > >> show up nicely in hit reports. They are also easier for end-users to >remember. > >> > >> * Automatic attribute inheritance. Extensions are used to find the >mime-type > >> of a file. Or the mime-type can be set directly on the file. Then, any > >> attribute files set in /WEB-INF/DefaultMimeAttributes/$mime/$type are >inherited > >> for the resource in question. This allows mapping all ${page}.cf to > >> application/x-server-side-confluence, creating an attribute called 'type' > >> >with > >> a value of 'confluence-page'. More on the types in a bit. > >> > >> * Every resource has a type, and a handler. Standard types are jsp, cgi, > >> binary. Base types are event(bsf-based), code, template. Type can also >be > >> servlet, or, even more advanced(but not ready to be released) is > >> 'vaadin' >as a > >> type. > >> > >> * Several languages are integrated: template: freemarker/velocity/text, > >> bsf+code: groovy/janino(java)/jython/rhino/bsh/quercus(php). > >> > >> * Macros called by a template language can be implemented in *any* >webslinger > >> resource(any type, any language). Each integrated template type has >proxies > >> implemented that allow it to call back into webslinger macros. velocity-> > >> #Merge("/path/to/file", "/template/to/wrap/it/with"), freemarker-> ><@Merge > >> path="/path/to-file" template0="/template/to/wrap/it/with"/>. Support for > >> macros with content bodies is fully supported as well. > >> > >> * Support for one-type 'wrapper' of a text output, and then different >page > >> styles. Partial-ajax page updates can then skip this, and do smart >updates of > >> regions of the browser. > >> > >> The above list is an non-inclusive list of features in webslinger-core. >It's > >> really generic, and not tied to any particular implementation. > >> > >> The other major thing different about it, is that webslinger is *itself* > >> >a > >> servlet container, just like catalina or glashfish. However, what sets > >> it >apart > >>from all others, is that it doesn't run standalone; instead, it is >installed > >> into a parent container. It then fakes/wraps everything, to support > >> it's >fancy > >> stuff. It supports running standard servlets, but then get backed by > >> commons-vfs, with overlay support, etc. This implementation isn't >perfect, and > >> really needs to be improved upon. > >> > >> I've been working on a demo for the ofbiz community to play with. > >> However, >the > >> existing embedded site in the repository was rather small, so I wrote an > >> importer to pull stuff from cwiki, which is what then started this >thread. > >> > >> ps: the license on all our code is asl 2.0 > >> > >>> > >>> Jacques > >>> > >>> From: "Joe Schaefer"<[email protected]> > >>>> The url is here > >>>> https://svn.apache.org/repos/infra/infrastructure/trunk/projects/cms > >>>> and is publicly readable. > >>>> > >>>> > >>>> > >>>> ----- Original Message ---- > >>>>> From: Adam Heath<[email protected]> > >>>>> To: Jacques Le Roux<[email protected]> > >>>>> Cc: [email protected]; [email protected] > >>>>> Sent: Tue, September 21, 2010 3:42:35 PM > >>>>> Subject: Re: ofbiz wiki(confluence) > >>>>> > >>>>> On 09/21/2010 02:07 PM, Jacques Le Roux wrote: > >>>>>> From: "Adam Heath"<[email protected]> > >>>>> >> On 09/21/2010 11:53 AM, Jacques Le Roux wrote: > >>>>> >>> From: "Adam Heath"<[email protected]> > >>>>> >>>> So, I need some admin help with cwiki.apache.org, or at least > >>>>> advice. > >>>>>>>>> I've got a script that uses xmlrpc to confluence, and fetches all > >>>>>>>>> previous page(+versions), comments, attachments(+versions), tracks > >>>>> >>>> renames, usernames, and commit messages. I then take all this > >>>>> data and > >>>>>>>>> convert it into a long series of git commits, with the files > >>>>> layed out > >>>>>>>>> in a proper webslinger design. The author of each git commit is the > >>>>>>>>> person who changed the page, added a comment, or uploaded a new > >>>>>>>>> attachment. > >>>>>>> > >>>>>>> This webslinger layout is still in flux, as is my script. The basic > >>>>>>> logic works, however, by fetching all meta data, storing most of >the > >>>>>>> bulk of that in a temporary cache folder(only for the duration of >the > >>>>>>> script), then sorting each item by date, and replaying the set of > >>>>>>> changes one by one. > >>>>>>> > >>>>>>> It's optimized by storing the 'lastFoo' stuff for each > >>>>>>> page/comment/attachment/(title->pageId mapping) as needed, so that > >>>>>>> >it > >>>>>>> can detect newer versions, etc, and not have to do anything. A > >>>>> refresh > >>>>>>> after a full download against the OFBIZ space takes 2 minutes, with > >>>>>>> nothing new to fetch. > >>>>>>> > >>>>>>>>> The issue I am having is the confluence installed on cwiki is >old. > >>>>>>>>> Newer versions support returning the > >>>>> PageHistorySummary.versionComment > >>>>> >>>> thru the rpc; currently, I have to fall back and do a screen > >>>>> scrape of > >>>>>>>>> the viewpreviousversions.action page. > >>>>>>> > >>>>> >> CONFDEV docs definately list a versionComment field on > >>>>> >> PageHistorySummary, that is not exposed in 3.2.0 installed on >cwiki. > >>>>>>> > >>>>>>>>> Where should I ask for help on this, getting this new api > >>>>> implemented? > >>>>>>>> > >>>>>>>> infra team: [email protected] > >>>>> >>> > >>>>>>>> I put them in copy > >>>>> >> > >>>>>>> Thanks. I'm putting more information in this email; I've left > >>>>>>> d...@ofbiz on the cc for this email, as others might be interested > >>>>>>> in > >>>>>>> what I have discovered. > >>>>>>> > >>>>>>>>> I also have suggestions to make the api more lightweight, when > >>>>> doing > >>>>>>>>> incremental updates(which my system supports). > >>>>>>> > >>>>>>> Here are the suggestions: > >>>>>>> > >>>>>>> I can fetch all attachments for a page. But the attachment data > >>>>>>> returned doesn't include the current version as a field. I have to > >>>>>>> split the download url(which is sub-optimal; it includes the current > >>>>>>> version as a parameter). It might be nice to have an > >>>>> AttachmentSummary > >>>>>>> type record. > >>>>>>> > >>>>>>> What if uploads an attachment, then a new version of the attachment, > >>>>>>> then changes the page, then deletes the attachment? How could I >fetch > >>>>>>> that information? I don't see a way to fetch all attachments for all > >>>>>>> time against a particular history. This is also a problem for deleted > >>>>>>> pages, comments, and labels(probably others). > >>>>>>> > >>>>>>> Comments in confluence support editting. Is this history stored, >and > >>>>>>> if so, can I get access to it? > >>>>> >> > >>>>>>> Are labels versioned? > >>>>> >> > >>>>>>> Children of pages are versioned, only because pages themselves are > >>>>>>> versioned, which includes the value of the parentId at the time the > >>>>>>> page was changed. However, the frontend doesn't let you see older > >>>>>>> children, when looking at previous versions. > >>>>> >> > >>>>>>> It'd be nice if when calling getPageHistory, I could request a subset > >>>>>>> of the list, instead of *all* page versions. If a page has 271 > >>>>>>> versions, and I have already fetched them, and the current page has a > >>>>>>> version of 274, then I only really need to fetch 3 >PageHistorySummary > >>>>>>> records(to get the versionComment from newer versions of >confluence). > >>>>>>> > >>>>> >> BlogEntrySummary doesn't include version, but BlogEntry does. >And I > >>>>>>> can't fetch old versions of blogs. > >>>>> >> > >>>>>>> > >>>>> >>>> > >>>>>>>>> As a side note, there is a severe lack of version comments. This > >>>>>>>>> script ends up producing 3117 commits. Some of those are page > >>>>> >>>> renames/comments/attachments, which don't have a commit >message. > >>>>> Most > >>>>>>>>> are page commits. There are only 70 change messages. It'd be > >>>>> nice if > >>>>>>>>> people would comment when they change a page, but I don't see a > >>>>> way to > >>>>>>>>> enforce that. > >>>>>>>>> > >>>>> >>> > >>>>>> > >>>>>> Adam, > >>>>> > > >>>>>> There is currently a beginning effort to create a CMS for apache.org > >>>>> > (infrastructure/trunk/projects/cms) is yours related to this >effort? > >>>>> > >>>>> No, it's not. Based on how much time I've spent already(started my > >>>>> imoprter last friday), and how familiar I am with ofbiz, it'd > >>>>> probably take me a 2 months to get mostly feature compatible with > >>>>> confluence(that's for a single person working in his spare time). > >>>>> > >>>>>> Jacques > >>>>>> PS: Not sure how to access to infrastructure/trunk/projects/cms/README > >>>>>> with the rights I have > >>>>> > >>>>> You mean it's not public? > >>>>> > >>>> > >>>> > >>>> > >>> > >> > >> > > > > > > > >
