Re: [WiX-users] RemoveFolderEx really slow on large folders
Yes, what it is getting hung up on is enumerating a hundred thousand files one by one and adding them up to a giant string that is passed on to the MSI engine. I looked at the RemoveFolderEx C++ implementation and it confirmed all my suspicions, so I went ahead and took it out and created my own managed custom action instead, which does the whole thing in one line: Directory.Delete(path, true) :-) Of course I added proper error handling and session return codes etc. but now it's super fast and doing exactly what I needed it to do. I'll probably do a blog post on it, I'll update this thread with the link when it's up. // Sascha On Thu, May 8, 2014 at 12:06 PM, Nick Ramirez nickra...@hotmail.com wrote: Is it getting hung up on something? When you uninstall with logging, does the log show anything happening that takes a long time around where it calls RemoveFiles? -- View this message in context: http://windows-installer-xml-wix-toolset.687559.n2.nabble.com/RemoveFolderEx-really-slow-on-large-folders-tp7594451p7594615.html Sent from the wix-users mailing list archive at Nabble.com. -- Is your legacy SCM system holding you back? Join Perforce May 7 to find out: • 3 signs your SCM is hindering your productivity • Requirements for releasing software faster • Expert tips and advice for migrating your SCM now http://p.sf.net/sfu/perforce ___ WiX-users mailing list WiX-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wix-users -- Is your legacy SCM system holding you back? Join Perforce May 7 to find out: #149; 3 signs your SCM is hindering your productivity #149; Requirements for releasing software faster #149; Expert tips and advice for migrating your SCM now http://p.sf.net/sfu/perforce ___ WiX-users mailing list WiX-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wix-users
Re: [WiX-users] RemoveFolderEx really slow on large folders
In general, the table-driven approach that RemoveFolderEx follows is superior. First, it handles locked files appropriately. Second, it does the deletes at the same time and sequence as other deletes are being performed by the installer. Third, it is much safer. Yes, if all the files and subfolders have no locks or special attributes on them, you can easily delete a folder and its children with one call. But having personally experienced what happens when a buggy call to such code is made (created by an associate no longer with the team), I can tell you that the speed with which my test machine had its C:\Program Files tree deleted was truly impressive. As was my anger and the day it took to re-image the machine. -- John Merryweather Cooper Build Install Engineer – ESA Jack Henry Associates, Inc.® Shawnee Mission, KS 66227 Office: 913-341-3434 x791011 jocoo...@jackhenry.com www.jackhenry.com -Original Message- From: Sascha Sertel [mailto:sascha.ser...@gmail.com] Sent: Friday, May 9, 2014 12:11 PM To: General discussion about the WiX toolset. Subject: Re: [WiX-users] RemoveFolderEx really slow on large folders Yes, what it is getting hung up on is enumerating a hundred thousand files one by one and adding them up to a giant string that is passed on to the MSI engine. I looked at the RemoveFolderEx C++ implementation and it confirmed all my suspicions, so I went ahead and took it out and created my own managed custom action instead, which does the whole thing in one line: Directory.Delete(path, true) :-) Of course I added proper error handling and session return codes etc. but now it's super fast and doing exactly what I needed it to do. I'll probably do a blog post on it, I'll update this thread with the link when it's up. // Sascha On Thu, May 8, 2014 at 12:06 PM, Nick Ramirez nickra...@hotmail.com wrote: Is it getting hung up on something? When you uninstall with logging, does the log show anything happening that takes a long time around where it calls RemoveFiles? -- View this message in context: http://windows-installer-xml-wix-toolset.687559.n2.nabble.com/RemoveFo lderEx-really-slow-on-large-folders-tp7594451p7594615.html Sent from the wix-users mailing list archive at Nabble.com. -- Is your legacy SCM system holding you back? Join Perforce May 7 to find out: • 3 signs your SCM is hindering your productivity • Requirements for releasing software faster • Expert tips and advice for migrating your SCM now http://p.sf.net/sfu/perforce ___ WiX-users mailing list WiX-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wix-users -- Is your legacy SCM system holding you back? Join Perforce May 7 to find out: #149; 3 signs your SCM is hindering your productivity #149; Requirements for releasing software faster #149; Expert tips and advice for migrating your SCM now http://p.sf.net/sfu/perforce ___ WiX-users mailing list WiX-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wix-users NOTICE: This electronic mail message and any files transmitted with it are intended exclusively for the individual or entity to which it is addressed. The message, together with any attachment, may contain confidential and/or privileged information. Any unauthorized review, use, printing, saving, copying, disclosure or distribution is strictly prohibited. If you have received this message in error, please immediately advise the sender by reply email and delete all copies. -- Is your legacy SCM system holding you back? Join Perforce May 7 to find out: #149; 3 signs your SCM is hindering your productivity #149; Requirements for releasing software faster #149; Expert tips and advice for migrating your SCM now http://p.sf.net/sfu/perforce ___ WiX-users mailing list WiX-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wix-users
Re: [WiX-users] RemoveFolderEx really slow on large folders
How does rollback work with your CA? From: Sascha Sertel [mailto:sascha.ser...@gmail.com] Yes, what it is getting hung up on is enumerating a hundred thousand files one by one and adding them up to a giant string that is passed on to the MSI engine. I looked at the RemoveFolderEx C++ implementation and it confirmed all my suspicions, so I went ahead and took it out and created my own managed custom action instead, which does the whole thing in one line: Directory.Delete(path, true) :-) Of course I added proper error handling and session return codes etc. but now it's super fast and doing exactly what I needed it to do. I'll probably do a blog post on it, I'll update this thread with the link when it's up. -- Is your legacy SCM system holding you back? Join Perforce May 7 to find out: #149; 3 signs your SCM is hindering your productivity #149; Requirements for releasing software faster #149; Expert tips and advice for migrating your SCM now http://p.sf.net/sfu/perforce ___ WiX-users mailing list WiX-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wix-users
Re: [WiX-users] RemoveFolderEx really slow on large folders
@John: I completely understand why the util:RemoveFolderEx action works the way it works, and for many applications that is the right way to go, and that's why I had used it to begin with. Unfortunately for this particular app it was simply unacceptable to be stuck for 45 minutes during uninstall or even upgrade (where none of the files are being deleted but the action still enumerates them all) because of the unusually large number of files we're dealing with, so I opted for the custom action in favor of execution time but at the cost of the tight integration with the MSI process. I was also hugely simplifying things in my previous reply, I am handling several locked file and other error cases and I won't just error out as soon as I hit the first snag. @Jeremy: It doesn't. Taking a copy of all the files before deleting them to be able to roll back a failed uninstall was not really an option here, and the app does also not depend on these files, they are mostly used for caching purposes. If that whole folder gets wiped out it can be easily recreated by the app. So not being able to roll back this delete operation was an acceptable compromise. // Sascha On Fri, May 9, 2014 at 10:31 AM, Jeremy Farrell jeremy.farr...@oracle.comwrote: How does rollback work with your CA? From: Sascha Sertel [mailto:sascha.ser...@gmail.com] Yes, what it is getting hung up on is enumerating a hundred thousand files one by one and adding them up to a giant string that is passed on to the MSI engine. I looked at the RemoveFolderEx C++ implementation and it confirmed all my suspicions, so I went ahead and took it out and created my own managed custom action instead, which does the whole thing in one line: Directory.Delete(path, true) :-) Of course I added proper error handling and session return codes etc. but now it's super fast and doing exactly what I needed it to do. I'll probably do a blog post on it, I'll update this thread with the link when it's up. -- Is your legacy SCM system holding you back? Join Perforce May 7 to find out: #149; 3 signs your SCM is hindering your productivity #149; Requirements for releasing software faster #149; Expert tips and advice for migrating your SCM now http://p.sf.net/sfu/perforce ___ WiX-users mailing list WiX-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wix-users -- Is your legacy SCM system holding you back? Join Perforce May 7 to find out: #149; 3 signs your SCM is hindering your productivity #149; Requirements for releasing software faster #149; Expert tips and advice for migrating your SCM now http://p.sf.net/sfu/perforce ___ WiX-users mailing list WiX-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wix-users
Re: [WiX-users] RemoveFolderEx really slow on large folders
Is it getting hung up on something? When you uninstall with logging, does the log show anything happening that takes a long time around where it calls RemoveFiles? -- View this message in context: http://windows-installer-xml-wix-toolset.687559.n2.nabble.com/RemoveFolderEx-really-slow-on-large-folders-tp7594451p7594615.html Sent from the wix-users mailing list archive at Nabble.com. -- Is your legacy SCM system holding you back? Join Perforce May 7 to find out: #149; 3 signs your SCM is hindering your productivity #149; Requirements for releasing software faster #149; Expert tips and advice for migrating your SCM now http://p.sf.net/sfu/perforce ___ WiX-users mailing list WiX-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wix-users
[WiX-users] RemoveFolderEx really slow on large folders
Hi there, I'm using the util:RemoveFolderEx custom action to remove a folder during uninstall that contains temporary files that are no longer needed (think of it as a browser cache for example). Due to the nature of the application the number of files in the folder can be in the thousands. I noticed that in that case running the RemoveFolderEx action can take a long time, like 10 minutes for a folder that contains 25,000 files, and up to 45 minutes on a folder with 120,000 files in it. The number of subdirectories in the folder I am deleting is small, but I'm guessing that the action adds every single file to a list of files and folders to be removed by MSI, and deleting on a file by file basis instead of the entire folder at once makes it take so long? Or is it taking a copy of all the files in case uninstall fails and it needs to rollback? Unless there is a way to make RemoveFolderEx be faster, what other options do I have? Write my own custom action? I am using my own managed bootstrapper already, so adding a managed custom action wouldn't be too big of a deal I guess. Any ideas/suggestions welcome (other than why am I creating so many temporary files, I can't change that at this time). // Sascha -- Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available. Simple to use. Nothing to install. Get started now for free. http://p.sf.net/sfu/SauceLabs ___ WiX-users mailing list WiX-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wix-users
