On Mon, 2010-08-02 at 20:00 -0700, Mike McCune wrote: > I spent some time today tracking down the exception that a few people > were seeing: > > InvalidDocument: document too large - BSON documents are limited to 4 MB > > I wrote a unit test to create a repo and add 5,000 packages to the > repository and easily hit the above error: > > File "/home/mmccune/devel/pulp/test/unit/test_api.py", line 615, in > test_sync_large_repo > self.rapi.update(repo) > File "/home/mmccune/devel/pulp/test/unit/../../src/pulp/auditing.py", > line 166, in _audit > _record_event() > File "/home/mmccune/devel/pulp/test/unit/../../src/pulp/auditing.py", > line 141, in _record_event > _objdb.insert(event, safe=False, check_keys=False) > File "build/bdist.linux-x86_64/egg/pymongo/collection.py", line 245, > in insert > message.insert(self.__full_name, docs, check_keys, safe), safe) > InvalidDocument: document too large - BSON documents are limited to 4 MB > > Note the trace is contained within the auditing insert() call. > > Turns out the Event class has both a 'params' field and a field called > 'action' but the action field also contained a copy of the params so for > a large Repo document you get an Event document that is 2-3X the size of > the actual parameter being audited. > > Jason is working to trim this down but the short story is, we aren't > hitting the 4MB limit because of the structure of our actual Repo > document, we are hitting it because of the way we were auditing the > parameters of a method call. > > Jason also in his latest update took out the repo from the param logging > in RepoApi.update() > > @audit(params=[]) > def update(self, repo_data): > > Also, to make people feel more confortable, with the fix to the audit() > call I was able to create a repo with 20,000 packages without error > whereas before it would bail with a repo with 5,000. > >
I believe (read 'hope') that the fix is in... -- Jason L Connor Senior Software Engineer Systems Management and Cloud Enablement Red Hat, Inc. +1.919.890.8331 Freenode: linear
signature.asc
Description: This is a digitally signed message part
_______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
