On Fri, Jun 17, 2011 at 09:44, <gst...@apache.org> wrote: > Author: gstein > Date: Fri Jun 17 05:44:02 2011 > New Revision: 1136777 > > URL: http://svn.apache.org/viewvc?rev=1136777&view=rev > Log: > Continued work on the pausing mechanic in the XML parser of ra_serf. > > The in-memory blocks no longer use a bucket allocator, since that is > generally internal to serf's bucket types. Not a big deal, as we can just > use a simple linked-list of reusable blocks. > > This adds code for writing content to the "pending" content, and begins > the code for injecting that content into the XML parser when ready. > > The update editor now contains logic for pausing and un-pausing the parser > as certain conditions are met. It also makes the appropriate call to > process the pending content. > > However: no logic is changed in the network reading portion, or the normal > parser injection. In essence, the "pause" state is ignored and the > "pending" content is never accumulated. Further development and testing is > needed. This revision is a continued preview of the direction, for > feedback and code review. > [..]
> > Modified: subversion/trunk/subversion/libsvn_ra_serf/util.c > URL: > http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_ra_serf/util.c?rev=1136777&r1=1136776&r2=1136777&view=diff > ============================================================================== [...] > static svn_error_t * > +write_to_pending(svn_ra_serf__xml_parser_t *ctx, > + const char *data, > + apr_size_t len, > + apr_pool_t *scratch_pool) > +{ > + struct pending_buffer_t *pb; > + > + /* The caller should not have provided us more than we can store into > + a single memory block. */ > + SVN_ERR_ASSERT(len <= PARSE_CHUNK_SIZE); > + > + if (ctx->pending == NULL) > + ctx->pending = apr_pcalloc(ctx->pool, sizeof(*ctx->pending)); > + > + /* We do not (yet) have a spill file, but the amount stored in memory > + has grown too large. Create the file and place the pending data into > + the temporary file. */ > + if (ctx->pending->spill == NULL > + && ctx->pending->memory_size > SPILL_SIZE) > + { > + SVN_ERR(svn_io_open_unique_file3(&ctx->pending->spill, > + NULL /* temp_path */, > + NULL /* dirpath */, > + svn_io_file_del_on_pool_cleanup, > + ctx->pool, scratch_pool)); > + } > + > + /* Once a spill file has been constructed, then we need to put all > + arriving data into the file. We will no longer attempt to hold it > + in memory. */ > + if (ctx->pending->spill != NULL) > + { > + /* NOTE: we assume the file position is at the END. The caller should > + ensure this, so that we will append. */ > + SVN_ERR(svn_io_file_write_full(ctx->pending->spill, data, len, > + NULL, scratch_pool)); I think you need 'return' or 'else' here, otherwise data will be stored in file _AND_ in memory. > + } > + > + /* We're still within bounds of holding the pending information in > + memory. Get a buffer (already available, or alloc it), copy the data > + there, and link it into our pending data. */ > + if (ctx->pending->avail != NULL) > + { > + pb = ctx->pending->avail; > + ctx->pending->avail = pb->next; > + } > + else > + { > + pb = apr_palloc(ctx->pool, sizeof(*pb)); > + } > + /* NOTE: *pb is uninitialized. All fields must be stored. */ > + > + pb->size = len; > + memcpy(pb->data, data, len); > + pb->next = NULL; > + > + /* Start a list of buffers, or append to the end of the linked list > + of buffers. */ > + if (ctx->pending->tail == NULL) > + { > + ctx->pending->head = pb; > + ctx->pending->tail = pb; > + } > + else > + { > + ctx->pending->tail->next = pb; > + ctx->pending->tail = pb; > + } > + > + /* We need to record how much is buffered in memory. Once we reach > + SPILL_SIZE (or thereabouts, it doesn't have to be precise), then > + we'll switch to putting the content into a file. */ > + ctx->pending->memory_size += len; > + > + return SVN_NO_ERROR; > +} > + > + [...] -- Ivan Zhakov
