Re: thought I had fixe it

Mon, 24 Oct 2011 10:23:24 -0700 

Hi there,
I think you may have the solution--I am indeed putting in a very heavy weight object into the continuation. Let me go through this code and get back to you with questions if I may. 

Paul

On 10/24/2011 12:55 PM, Leszek Gawron wrote:

Hello,


@Jasha: advising a user to limit the amount of data usually does not 
work. :)



On 2011-10-24 18:29, Paul Joseph wrote:

Actually the real reason is that the user is using a query that required
crafting to return 25K results.  He is seeking to show the system "does
not scale."

Actually I am using in my XML template, a cocoon flowscript repeater
widget that does paging.  But this is well after the Java logic has
built the large array of results (the array results and the vector
secureResults below)


I suspect what might be the problem:


1. Are your continuations session bound? They are not by default - 
even though you set a short session expiry time most of the data is 
still held by continuations which are being expired with totally 
different mechanism. Container bound continuations are useful in only 
one case: you build a session-less site. For "session" exprience 
ALWAYS use session-bound-continuations.



2. Your flowscript continuation probably holds the ENTIRE rowset and 
will hold it until it expires (and all children expire). You are not 
supposed to put heavyweight object into your continuation.


Your solutions are:


a. use a generator instead of flowscript. Implementing own generator 
is actually dead easy. Style your data with XSLT.



b. make flowscript "forget" the data after the report has been 
rendered. You probably regenerate your rowset with each request anyway.


This one is actually also easy. This is an example of my production code:


/* TEMPLATE
    var config = {
        defaultOrderBy: "name",
        defaultDirection: "asc",
        valueListProvider: cocoon.getComponent( "valueListBeanName" ),
        filterModelName: "filterModel",
        viewName: "viewName",
        errorRedirect: "/",
        rowSelectionReturns: false,
        bizData: bizData,
        selections: selections,
        formHandler: function ( action, bizData ) {
        },
        rowHandler: function( id, action, bizData ) {
        }
    }
*/
function sortedFilteredView( config ) {
    var defaultMaxResults = new Integer( 100 );

    var modelName = ( config.filterModelName != undefined ) ? 
config.filterModelName : config.viewName;

    var form = new Form( "cocoon:/form-def/" + modelName );

    //form.createBinding("cocoon:/form-bind/" + 
config.filterModelName );


    form.locale = determineLocale();
    var model = form.getModel();


    var bizData = ( config.bizData != undefined ) ? config.bizData : 
new java.util.HashMap();

    model.orderBy = config.defaultOrderBy;

    model.direction = ( config.defaultDirection != undefined ) ? 
config.defaultDirection : "asc";
    model.maxResults = ( config.maxResults != undefined ) ? 
config.maxResults : defaultMaxResults;


    if ( config.formInitializer != undefined ) {
        config.formInitializer( model );
    }

    model.skipResults = new Integer( 0 );
    while ( true ) {

        model.pageNo = new java.lang.Integer( model.skipResults / 
model.maxResults + 1 );
        var filterContext = 
Packages.org.apache.cocoon.forms.util.ContainerWidgetAsMap( form.form );
        var items = config.valueListProvider.generateResults( 
filterContext, bizData );



        var totalItemCount = config.valueListProvider.countEntries( 
filterContext, bizData );
        var pageCount = new java.lang.Integer( java.lang.Math.ceil( 
totalItemCount / model.maxResults ) );

        var firstPage = ( model.skipResults == 0 );

        var lastPage = ( model.skipResults + model.maxResults >= 
totalItemCount );


        form.showForm(     "form/" + config.viewName,
                        {     items    : items,
                            orderBy  : model.orderBy,
                            direction: model.direction,
                            maxResults: model.maxResults,
                            pageNumber: model.pageNo,
                            pageCount: pageCount,
                            firstPage: firstPage,
                            lastPage : lastPage,
                            viewConfig: config,

                            startIndex: ((model.pageNo - 1) * 
model.maxResults + 1),

                            bizData  : bizData,
                            selections: config.selections },
                            function() {
                                delete items;
                            }
                         );

        if ( form.submitId == "finish" ) { //search
            model.skipResults = new Integer( 0 );
        } else if ( form.submitId == "cancel" ) {
            return null;
        } else if ( form.submitId == "next" ) {

            model.skipResults = new Integer( model.skipResults + 
model.maxResults );

        } else if ( form.submitId == "prev" ) {

            model.skipResults = new Integer( model.skipResults - 
model.maxResults );

            if ( model.skipResults < 0 )
                model.skipResults = new Integer( 0 );
        } else if ( form.submitId == "first" ) {
            model.skipResults = new Integer( 0 );
        } else if ( form.submitId == "last" ) {

            model.skipResults = new java.lang.Integer( ( 
java.lang.Math.ceil( totalItemCount / model.maxResults ) - 1 ) * 
model.maxResults );

        } else if ( form.submitId == "changePage" ) {
            if ( model.pageNo != null ) {

                var skipCount = ( model.pageNo  - 1 ) * 
model.maxResults;

                    if ( skipCount < totalItemCount )
                        model.skipResults = new Integer( skipCount );
                    else

                        model.skipResults = new java.lang.Integer( ( 
java.lang.Math.ceil( totalItemCount / model.maxResults ) - 1 ) * 
model.maxResults );

                    if ( skipCount < 0 )
                        model.skipResults = new Integer( 0 );
            }
        } else if ( form.submitId == "changeMaxResults" ) {
            model.skipResults = new Integer( 0 );
        } else if ( form.submitId != null ) {

            if ( form.submitId == "selectRow" && 
config.rowSelectionReturns == true )

                return model.rowId;
            if ( config.rowHandler != undefined && model.rowId != null )

                config.rowHandler( model.rowId, form.submitId, 
bizData );
            if ( config.formHandler != undefined && model.rowId == 
null )

                config.formHandler( form.submitId, bizData );
        }
        if ( model.maxResults <= 0 )
            model.maxResults = defaultMaxResults;
        model.rowId = null;
    }
}



The code is responsible for showing a pageable sortable filterable 
rowset.


The most important part is :


var items = config.valueListProvider.generateResults( filterContext, 
bizData );


This object is VERY heavy (it might any row count you might like).

The next important thing is after rendering this object is deleted:


        form.showForm(     "form/" + config.viewName,
                        {     items    : items,
                            orderBy  : model.orderBy,
                            direction: model.direction,
                            maxResults: model.maxResults,
                            pageNumber: model.pageNo,
                            pageCount: pageCount,
                            firstPage: firstPage,
                            lastPage : lastPage,
                            viewConfig: config,

                            startIndex: ((model.pageNo - 1) * 
model.maxResults + 1),

                            bizData  : bizData,
                            selections: config.selections },
                            function() {
                                delete items;
                            }
                         );



There is a parameter for cocoon.sendPageAndWait which is not widely 
known: a cleanup function.


http://cocoon.apache.org/2.1/userdocs/flow/api.html#cocoon

So your code has to look something like that:

var report = generateYourHorriblyBigReport();

cocoon.sendPageAndWait( "myreport.html",
                        { report: report },
                        function() {
                           delete report;
                        }
                       );




I advice you to use some kind of profiler. Money spent on YourProfiler 
might be one of best spent dollars. You can:


 - make a memory snapshot
 - do your request
 - make a subsequent memory snapshot

 - compare both snapshots: there should be NO significant memory 
consumption difference.


HTH

    lg



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]























					Reply via email to