Have you considered chunking the problem up and using compress v revZip library? I ran into something similar when I wanted to keep memory down for encrypting/decrypting multi-gig log files…got a lot of advice about just using shell—but for the same sorts of reasons as you I wanted to do things within lc. I used a standard 1MB chunk size so just added a visual working on '1 of n progress bar’. That might be more effort than it’s worth to you.
Might be simpler to create 1 or more archives using the revZip library vs rolling your own. You’d need to stat the files to determine the sizes and # of archives and limit each to < 2GB. the detailed files (synonymous with the long files) Mark > On Oct 11, 2022, at 4:14 PM, Paul Dupuis via use-livecode > <use-livecode@lists.runrev.com <mailto:use-livecode@lists.runrev.com>> wrote: > > Thank you for the history. > > Yes, using other ZIP tools via shell is a possibility, but one I would rather > not do. While some researchers who use our app are very computer savvy, most > are not. Having a LiveCode ZIP in a window with a nice UI that uses > terminology our data analysis tool already uses to let them 'package' their > data works well for usage and adoption. > > We're already doing invisible shell work-arounds for a number of engine bugs, > with using ffmpg to replace Export Snapshot (see > https://quality.livecode.com/show_bug.cgi?id=23694 > <https://quality.livecode.com/show_bug.cgi?id=23694>) the latest. I would > just rather not add another 'work-around'. I was actually spending time today > cleaning out an old work-around from when a minimized stack set one of its > coordinates to -32000 as a way of indicating it was minimized (before the > iconic property existed) and work-around fro when a stack was minimized it > would get a 'resizeStack' message with a new width and height of 0,0. None of > these apply any more but we don't always have time or awareness to remove > work-arounds when they are fixed, which can sometimes be years after the > work-around was put into place. > > Anyway, thank you for the suggested approach. It is actually really goo to > know someone else has already done it. > > > On 10/11/2022 4:34 PM, Pi Digital via use-livecode wrote: >> Hi Paul >> >> I got affected by that and other win revzip issues too. I ended up parsing >> them through a shell script and using windows built in unzipper. Of course >> the downfall with that is a lack of feedback on progress which, for large >> files like this, requires a great deal of time to process and feedback is >> almost a necessity. I made the shell visible just so that the customer was >> aware of its progress. It was ugly in terms of UIX but it got us out of a >> short term hole. I hope you can get a more long term solution. >> >> I’m sure you’re capable of building the shell script but let us know if not. >> >> All the best >> Sean >> >> >> >>> On 11 Oct 2022, at 20:37, Paul Dupuis via use-livecode >>> <use-livecode@lists.runrev.com <mailto:use-livecode@lists.runrev.com>> >>> wrote: >>> >>> We just got bit by a 4+ year old bug (See >>> https://quality.livecode.com/show_bug.cgi?id=20859 >>> <https://quality.livecode.com/show_bug.cgi?id=20859>) where the revZIP >>> library on macOS can create and read ZIP archives over 2GB, BUT the Windows >>> library can not. >>> >>> We distribute our desktop research application as "cross-platform" for >>> macOS and Windows and just had a researcher on macOS build a "package" (our >>> term for a custom ZIP archive our LiveCode app can make of the researcher's >>> data files) over 2GB and sent to their Window colleague who can NOT open it. >>> >>> Yes, we walked the Windows researcher through unzipping the archive with >>> WinZIP and accessing all the data files manually, but so much for the >>> "elegance" of our app being cross-platform due to using LiveCode. >>> >>> Now, do we add code to check the size and prohibit macOS users from >>> creating "packages" over 2GB so if they are sent to a Windows researcher, >>> they can be opened OR do we just warn the Windows researcher if the archive >>> is over 2GB that they will not be able to open it? Neither is a good option. >>> >>> This just makes us look bad. >>> >>> Paul Dupuis >>> Researchware >>> >>> _______________________________________________ >>> use-livecode mailing list >>> use-livecode@lists.runrev.com <mailto:use-livecode@lists.runrev.com> >>> Please visit this url to subscribe, unsubscribe and manage your >>> subscription preferences: >>> http://lists.runrev.com/mailman/listinfo/use-livecode >>> <http://lists.runrev.com/mailman/listinfo/use-livecode> >> _______________________________________________ >> use-livecode mailing list >> use-livecode@lists.runrev.com <mailto:use-livecode@lists.runrev.com> >> Please visit this url to subscribe, unsubscribe and manage your subscription >> preferences: >> http://lists.runrev.com/mailman/listinfo/use-livecode >> <http://lists.runrev.com/mailman/listinfo/use-livecode> > > > _______________________________________________ > use-livecode mailing list > use-livecode@lists.runrev.com <mailto:use-livecode@lists.runrev.com> > Please visit this url to subscribe, unsubscribe and manage your subscription > preferences: > http://lists.runrev.com/mailman/listinfo/use-livecode > <http://lists.runrev.com/mailman/listinfo/use-livecode> _______________________________________________ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
