Jira (PUP-2561) PMT may deadlock when packing or unpacking large tarballs

[Ethan Brown (JIRA)]

Title: Message Title

Ethan Brown commented on an issue   Re: PMT may deadlock when packing or unpacking large tarballs So I can reproduce both issues here on OSX 10.9. When trying to install from a very large tarball, gzip will die partway through on the old pre-merge SHA (d72f9e8) with an error something like this. Note in this case, I grabbed an existing tarball, extracted it, copied in a bunch of large dummy files, then repacked with `tar -czvf` to create a sufficiently large (in this case a few hundred MB should do it) tarball. Debug: Executing 'gzip -dc /Users/Iristyle/source/puppet/joshcooper-powershell-0.0.6.tar.gz' Error: Invalid argument - /var/folders/1b/rzv96bhd5h9bwtdnz7g0xmlm0000gn/T/puppet20140514-90098-wo95tw Error: Try 'puppet help module install' for usage When bumping to the latest version of the code, the module is successfully installed, as it will use unix pipes with gzip and tar gzip -dc foo.tar.gz | tar xof - As for the building aspect, I can also reproduce on OSX 10.9 by cloning a repository that is large enough to exceed the pipe buffer when built. In this instance, output will be something like the following with the pre-merge code before it deadlocks Debug: Executing 'tar cf - foo-bar-0.0.1' Debug: Executing 'gzip -c' In the fixed code, the output is more like: Debug: Executing 'tar cf - foo-bar-0.0.1 | gzip -c > foo-bar-0.0.1.tar.gz' Add Comment   Puppet / PUP-2561 PMT may deadlock when packing or unpacking large tarballs Due to changes that were pulled into the 3.6.0 release candidate, some large tarballs may cause the IO pipeline to fill up prematurely, which blocks all subsequent writes. Since reads are waiting for writes to finish, this causes module installation and upgrades to hang. This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.