Hi Paul, The interesting thing about this issue is that it's a big metadata problem with one of the RPMs in the repo, but it's not an issue with the size of the files inside the RPMs or the number of RPMs in the repo. This is why "bigger" repos work as expected, but this the packagecloud.io repo doesn't. See this comment [0] for more details on this.
Would you be interested in filing a bug against package.io asking them to not put so many files in a single RPM? Currently the private-chef-11.2.2-1.el6.x86_64.rpm RPM has 79,292 files in it. If they spread those files over more RPMs, the metadata per RPM would be reduced and this issue would go away. I think that is the best thing to do. [0]: https://bugzilla.redhat.com/show_bug.cgi?id=1167950#c1 -Brian ----- Original Message ----- > From: "Paul Gonin" <[email protected]> > To: "Brian Bouterse" <[email protected]> > Cc: "Michael Hrivnak" <[email protected]>, [email protected] > Sent: Wednesday, November 26, 2014 10:17:14 AM > Subject: Re: [Pulp-list] Errors in pulp.log : BSON document too large > > Hi Brian, > > Thanks for the feedback. > I understand it is a specifi issue with packagecloud.io > I am suprised to have a metadata file size issue with this repo, there > are only 111 rpm packages and I have been synchronizing already some > bigger repos without a problem :) > > Regards > Paul > > > On Tue, Nov 25, 2014 at 6:10 PM, Brian Bouterse <[email protected]> wrote: > > Hi Paul, > > > > Mongo does have a 16MB document limit [0], and this repo must have metadata > > that is a little larger than that. I was able to reproduce this bug, and I > > documented it here [1]. I did find some old BSON related Pulp bugs [2] > > [3], but those all seemed to be resolved so [1] is the only open BSON > > related bug I know of. > > > > [0]: http://docs.mongodb.org/manual/reference/limits/#bson-documents > > [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1167950 > > [2]: https://bugzilla.redhat.com/show_bug.cgi?id=748812 > > [3]: https://bugzilla.redhat.com/show_bug.cgi?id=620495 > > > > -Brian > > > > > > ----- Original Message ----- > >> From: "Paul Gonin" <[email protected]> > >> To: "Michael Hrivnak" <[email protected]>, [email protected] > >> Sent: Monday, November 24, 2014 4:32:01 PM > >> Subject: Re: [Pulp-list] Errors in pulp.log : BSON document too large > >> > >> Hi Michael, > >> > >> I ran manually the sync and it completed successfully from the admin > >> client point of view: > >> $ pulp-admin rpm repo sync run --repo-id=asis-chef-rhel6 > >> +----------------------------------------------------------------------+ > >> Synchronizing Repository [asis-chef-rhel6] > >> +----------------------------------------------------------------------+ > >> > >> This command may be exited by pressing ctrl+c without affecting the actual > >> operation on the server. > >> > >> Downloading metadata... > >> [\] > >> ... completed > >> > >> Downloading repository content... > >> [== ] 5% > >> RPMs: 1/5 items > >> Delta RPMs: 0/0 items > >> > >> ... completed > >> > >> Downloading distribution files... > >> [==================================================] 100% > >> Distributions: 0/0 items > >> ... completed > >> > >> Importing errata... > >> [-] > >> ... completed > >> > >> Importing package groups/categories... > >> [-] > >> ... completed > >> > >> Publishing packages... > >> [==== ] 9% > >> Packages: 5/54 items > >> Publishing distributions... > >> [==================================================] 100% > >> Distributions: 0/0 items > >> [==================================================] 100% > >> Packages: 54/54 items > >> ... completed > >> > >> Generating metadata > >> [|] > >> ... completed > >> > >> Publishing repository over HTTP > >> [-] > >> ... completed > >> > >> no error in ~/.pulp/admin.log > >> > >> In pulp.log : > >> 2014-11-24 21:55:26,888 pulp.plugins.conduits.mixins:ERROR: Content > >> unit association failed [Unit [key={'name': 'private-chef', > >> 'checksum': '2d068c98c36c4eb548773efc616fec5c36bd2bdd', 'epoch': '0', > >> 'version': '11.2.3', 'release': '1.el6', 'arch': 'x86_64', > >> 'checksumtype': 'sha'}] [type=rpm] [id=None]] > >> 2014-11-24 21:55:26,890 nectar.downloaders.base:ERROR: BSON document > >> too large (16967489 bytes) - the connected server supports BSON > >> document sizes up to 16777216 bytes. > >> 2014-11-24 21:56:36,058 pulp.plugins.conduits.mixins:ERROR: Content > >> unit association failed [Unit [key={'name': 'private-chef', > >> 'checksum': 'd5d8bda7a3c260d42bbc79adb318cc05c794ae55', 'epoch': '0', > >> 'version': '11.2.5', 'release': '1.el6', 'arch': 'x86_64', > >> 'checksumtype': 'sha'}] [type=rpm] [id=None]] > >> 2014-11-24 21:56:36,059 nectar.downloaders.base:ERROR: BSON document > >> too large (16968021 bytes) - the connected server supports BSON > >> document sizes up to 16777216 bytes. > >> 2014-11-24 21:56:45,599 pulp.plugins.conduits.mixins:ERROR: Content > >> unit association failed [Unit [key={'name': 'private-chef', > >> 'checksum': '8ffc55a8eaaf5ce31d9a37aa257bbf6ffdd0aa34', 'epoch': '0', > >> 'version': '11.2.2', 'release': '1.el6', 'arch': 'x86_64', > >> 'checksumtype': 'sha'}] [type=rpm] [id=None]] > >> 2014-11-24 21:56:45,600 nectar.downloaders.base:ERROR: BSON document > >> too large (16959923 bytes) - the connected server supports BSON > >> document sizes up to 16777216 bytes. > >> 2014-11-24 21:57:03,051 pulp.plugins.conduits.mixins:ERROR: Content > >> unit association failed [Unit [key={'name': 'private-chef', > >> 'checksum': 'a0ef74235e3826dbf21ebea40070832d8430d988', 'epoch': '0', > >> 'version': '11.2.4', 'release': '1.el6', 'arch': 'x86_64', > >> 'checksumtype': 'sha'}] [type=rpm] [id=None]] > >> 2014-11-24 21:57:03,052 nectar.downloaders.base:ERROR: BSON document > >> too large (16992417 bytes) - the connected server supports BSON > >> document sizes up to 16777216 bytes. > >> > >> In nectar.log > >> 2014-11-24 21:50:52,988 > >> requests.packages.urllib3.connectionpool:DEBUG: "GET > >> https://packagecloud-repositories.s3.amazonaws.com/141/315/el/6/package_files/904.rpm?AWSAccessKeyId=AKIAI44QGWC7C5WEV4XA&Signature=PuNQ0kK68ejp4jkxhVgmWlgpHHY%3D&Expires=1416862551 > >> HTTP/1.1" 200 576836991 > >> 2014-11-24 21:50:53,197 > >> requests.packages.urllib3.connectionpool:DEBUG: Setting read timeout > >> to <object object at 0x7fe99d14c100> > >> 2014-11-24 21:50:53,310 > >> requests.packages.urllib3.connectionpool:DEBUG: Setting read timeout > >> to <object object at 0x7fe99d14c100> > >> 2014-11-24 21:50:53,430 > >> requests.packages.urllib3.connectionpool:DEBUG: "GET > >> https://packagecloud-repositories.s3.amazonaws.com/141/315/el/6/package_files/1290.rpm?AWSAccessKeyId=AKIAI44QGWC7C5WEV4XA&Signature=AUyaEOnkxhQ7nRAHeGDuxsFLZJE%3D&Expires=1416862553 > >> HTTP/1.1" 200 577050972 > >> 2014-11-24 21:50:53,550 > >> requests.packages.urllib3.connectionpool:DEBUG: "GET > >> https://packagecloud.io/chef/stable/el/6/x86_64/private-chef-11.2.5-1.el6.x86_64.rpm > >> HTTP/1.1" 302 0 > >> 2014-11-24 21:50:53,552 requests.packages.urllib3.connectionpool:INFO: > >> Starting new HTTPS connection (1): > >> packagecloud-repositories.s3.amazonaws.com > >> 2014-11-24 21:50:53,758 > >> requests.packages.urllib3.connectionpool:DEBUG: Setting read timeout > >> to <object object at 0x7fe99d14c100> > >> 2014-11-24 21:50:53,998 > >> requests.packages.urllib3.connectionpool:DEBUG: "GET > >> https://packagecloud-repositories.s3.amazonaws.com/141/315/el/6/package_files/1366.rpm?AWSAccessKeyId=AKIAI44QGWC7C5WEV4XA&Signature=bgbKpTUhe3bNNWJHTgLJDsTcJZ0%3D&Expires=1416862552 > >> HTTP/1.1" 200 578311972 > >> 2014-11-24 21:50:54,248 > >> requests.packages.urllib3.connectionpool:DEBUG: Setting read timeout > >> to <object object at 0x7fe99d14c100> > >> 2014-11-24 21:50:54,478 > >> requests.packages.urllib3.connectionpool:DEBUG: "GET > >> https://packagecloud-repositories.s3.amazonaws.com/141/315/el/6/package_files/1448.rpm?AWSAccessKeyId=AKIAI44QGWC7C5WEV4XA&Signature=vGIQTaz8OYV8cIrUCQY4dsxvQ6U%3D&Expires=1416862553 > >> HTTP/1.1" 200 577030942 > >> 2014-11-24 21:55:26,890 nectar.downloaders.base:ERROR: BSON document > >> too large (16967489 bytes) - the connected server supports BSON > >> document sizes up to 16777216 bytes. > >> Traceback (most recent call last): > >> File "/usr/lib/python2.6/site-packages/nectar/downloaders/base.py", > >> line 110, in _fire_event_to_listener > >> event_listener_callback(*args, **kwargs) > >> File > >> > >> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/listener.py", > >> line 97, in download_succeeded > >> self.sync_conduit.save_unit(unit) > >> File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", > >> line 485, in save_unit > >> unit.id = content_manager.add_content_unit(unit.type_id, None, > >> pulp_unit) > >> File > >> "/usr/lib/python2.6/site-packages/pulp/server/managers/content/cud.py", > >> line 47, in add_content_unit > >> collection.insert(unit_doc, safe=True) > >> File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", > >> line 114, in retry > >> return method(*args, **kwargs) > >> File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", > >> line 357, in insert > >> continue_on_error, self.__uuid_subtype), safe) > >> File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", > >> line 143, in _with_end_request > >> return method(*args, **kwargs) > >> File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", > >> line 910, in _send_message > >> (request_id, data) = self.__check_bson_size(message) > >> File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", > >> line 881, in __check_bson_size > >> (max_doc_size, self.__max_bson_size)) > >> ImporterConduitException: BSON document too large (16967489 bytes) - > >> the connected server supports BSON document sizes up to 16777216 > >> bytes. > >> 2014-11-24 21:56:36,059 nectar.downloaders.base:ERROR: BSON document > >> too large (16968021 bytes) - the connected server supports BSON > >> document sizes up to 16777216 bytes. > >> Traceback (most recent call last): > >> File "/usr/lib/python2.6/site-packages/nectar/downloaders/base.py", > >> line 110, in _fire_event_to_listener > >> event_listener_callback(*args, **kwargs) > >> File > >> > >> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/listener.py", > >> line 97, in download_succeeded > >> self.sync_conduit.save_unit(unit) > >> File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", > >> line 485, in save_unit > >> unit.id = content_manager.add_content_unit(unit.type_id, None, > >> pulp_unit) > >> File > >> "/usr/lib/python2.6/site-packages/pulp/server/managers/content/cud.py", > >> line 47, in add_content_unit > >> collection.insert(unit_doc, safe=True) > >> File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", > >> line 114, in retry > >> return method(*args, **kwargs) > >> File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", > >> line 357, in insert > >> continue_on_error, self.__uuid_subtype), safe) > >> File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", > >> line 143, in _with_end_request > >> return method(*args, **kwargs) > >> File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", > >> line 910, in _send_message > >> (request_id, data) = self.__check_bson_size(message) > >> File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", > >> line 881, in __check_bson_size > >> (max_doc_size, self.__max_bson_size)) > >> ImporterConduitException: BSON document too large (16968021 bytes) - > >> the connected server supports BSON document sizes up to 16777216 > >> bytes. > >> 2014-11-24 21:56:45,600 nectar.downloaders.base:ERROR: BSON document > >> too large (16959923 bytes) - the connected server supports BSON > >> document sizes up to 16777216 bytes. > >> Traceback (most recent call last): > >> File "/usr/lib/python2.6/site-packages/nectar/downloaders/base.py", > >> line 110, in _fire_event_to_listener > >> event_listener_callback(*args, **kwargs) > >> File > >> > >> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/listener.py", > >> line 97, in download_succeeded > >> self.sync_conduit.save_unit(unit) > >> File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", > >> line 485, in save_unit > >> unit.id = content_manager.add_content_unit(unit.type_id, None, > >> pulp_unit) > >> File > >> "/usr/lib/python2.6/site-packages/pulp/server/managers/content/cud.py", > >> line 47, in add_content_unit > >> collection.insert(unit_doc, safe=True) > >> File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", > >> line 114, in retry > >> return method(*args, **kwargs) > >> File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", > >> line 357, in insert > >> continue_on_error, self.__uuid_subtype), safe) > >> File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", > >> line 143, in _with_end_request > >> return method(*args, **kwargs) > >> File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", > >> line 910, in _send_message > >> (request_id, data) = self.__check_bson_size(message) > >> File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", > >> line 881, in __check_bson_size > >> (max_doc_size, self.__max_bson_size)) > >> ImporterConduitException: BSON document too large (16959923 bytes) - > >> the connected server supports BSON document sizes up to 16777216 > >> bytes. > >> 2014-11-24 21:57:03,052 nectar.downloaders.base:ERROR: BSON document > >> too large (16992417 bytes) - the connected server supports BSON > >> document sizes up to 16777216 bytes. > >> Traceback (most recent call last): > >> File "/usr/lib/python2.6/site-packages/nectar/downloaders/base.py", > >> line 110, in _fire_event_to_listener > >> event_listener_callback(*args, **kwargs) > >> File > >> > >> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/listener.py", > >> line 97, in download_succeeded > >> self.sync_conduit.save_unit(unit) > >> File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", > >> line 485, in save_unit > >> unit.id = content_manager.add_content_unit(unit.type_id, None, > >> pulp_unit) > >> File > >> "/usr/lib/python2.6/site-packages/pulp/server/managers/content/cud.py", > >> line 47, in add_content_unit > >> collection.insert(unit_doc, safe=True) > >> File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", > >> line 114, in retry > >> return method(*args, **kwargs) > >> File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", > >> line 357, in insert > >> continue_on_error, self.__uuid_subtype), safe) > >> File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", > >> line 143, in _with_end_request > >> return method(*args, **kwargs) > >> File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", > >> line 910, in _send_message > >> (request_id, data) = self.__check_bson_size(message) > >> File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", > >> line 881, in __check_bson_size > >> (max_doc_size, self.__max_bson_size)) > >> ImporterConduitException: BSON document too large (16992417 bytes) - > >> the connected server supports BSON document sizes up to 16777216 > >> bytes. > >> 2014-11-24 21:57:03,062 nectar.downloaders.threaded:DEBUG: starting > >> workers > >> > >> > >> mongodb.log empty > >> > >> Thanks ! > >> Regards > >> Paul > >> > >> On Mon, Nov 24, 2014 at 7:44 PM, Michael Hrivnak <[email protected]> > >> wrote: > >> > Paul, > >> > > >> > Is there a full stack trace in the log that you can share? > >> > > >> > Thanks, > >> > Michael > >> > > >> > ----- Original Message ----- > >> > From: "Paul Gonin" <[email protected]> > >> > To: [email protected] > >> > Sent: Monday, November 24, 2014 5:36:30 AM > >> > Subject: [Pulp-list] Errors in pulp.log : BSON document too large > >> > > >> > Hi ! > >> > > >> > A few days ago I started synchronizing chef repo from packagecloud.io > >> > https://packagecloud.io/chef/stable/el/6/x86_64 > >> > > >> > I scheduled the sync to be ran every night at 2am. > >> > > >> > Since then I have the following MongoDB related error in pulp.log > >> > which seems related to MongoDB document size limit to 16MB > >> > > >> > 2014-11-22 02:36:56,824 nectar.downloaders.base:ERROR: BSON document > >> > too large (16968021 bytes) - the connected server supports BSON > >> > document sizes up to 16777216 bytes. > >> > > >> > Any idea how to address it ? > >> > I looked in Bugzilla if it is known issue but could not anything > >> > related to BSON. > >> > > >> > Thanks > >> > Paul > >> > > >> > _______________________________________________ > >> > Pulp-list mailing list > >> > [email protected] > >> > https://www.redhat.com/mailman/listinfo/pulp-list > >> > >> _______________________________________________ > >> Pulp-list mailing list > >> [email protected] > >> https://www.redhat.com/mailman/listinfo/pulp-list > >> > _______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
