Hi, On 19 June 2015 at 11:13, Bertrand Delacretaz <[email protected]> wrote:
> Hi Ian, > > On Thu, Jun 18, 2015 at 3:40 PM, Ian Boston <[email protected]> wrote: > > ...The BG Servlet output stream currently writes a new jcr:data node > every > > time flush is called. Would it be possible with the current repository > > implementation to make that flush append to the end of a jcr:data node > that > > it creates 1 jcr:data for all the content.... > > I don't know if Oak supports such an append in a way that allows other > clients to see the stream before it's closed. > > If it does, the nodestream package where this is implemented [1] is > not exported so changes shouldn't have any impact on external modules > as long as functionality is preserved. Test coverage of that package > is pretty good, though some tests make assumptions about what happens > under the hood and should be changed then. > > > IIUC the reason is to allow anything that is waiting for the output to > get > > updates on flush, but shouldn't this be a concern of the repository to > > implement an output stream that can be appended to and once appended read > > from by another process, rather than explicitly implementing a block > > strategy in the application. > > I agree in principle, but the bgservlets module is an old thing that's > not really needed anymore, we might as well deprecate it as there are > other ways to run things in the background in Sling today. > > Are you using bgservlets for a new project, or do you just need it due > to existing code? If the latter, porting that code to use Sling jobs > might be an option. > Its existing code that is triggered by a Job and calls flush every line output to ensure a UI is updated. The flush has been removed, but users will probably complain that nothing appears to be happening without the flush. I have suggested that the existing code avoid the BGServlet already using a service called from a Job, but I am almost certain other things use it. We observed 28K nodes created per invocation to store 300K of output, which obviously consumed nearly all of the repository (MongoMK) write capacity, as each flush also saves. I didn't look into how much disk space was consumed by the the 28K nodes, but it was probably 100 times. Best Regards Ian > > -Bertrand > > [1] > https://svn.apache.org/repos/asf/sling/trunk/contrib/extensions/bgservlets/engine/src/main/java/org/apache/sling/bgservlets/impl/nodestream >
