Re: Retrieving the EXIF date/time from 250k images
On Aug 18, 2022, at 7:47 AM, Mike Abdullah wrote: > > It’s not a very good fit, but when you say a “GCD concurrent queue”, you’d > need to be more specific. There are several configs possible. Do you mean a > global queue, or one you made yourself? If you made it yourself, how did you > configure it? Like Alex, for me this was 10-12 years ago so I don’t remember much about exactly how I tried to use GCD. It spun up too many threads, I re-read the docs and assumed it was spinning up new threads based on file IO blocking, and killing itself. I didn’t see a handy way to limit GCD concurrent queueing, and NSOperationQueue had maxConcurrentOperationCount documented. NSOperationQueue was also more idiomatic Objective-C and easier to use/understand in an Obj-C program. > The tricky problem is that GCD aims to optimise CPU usage. Reading image > metadata from disk is almost certainly not CPU bound; it’s going to be > limited by disk speed and the file system instead. GCD will have a tendency > to see tasks blocked on doing this, not occupying the CPU, and so spin up > another task in the hope that one _will_ then use the CPU, eventually > grinding to a halt as a huge number of threads contend for access to the file > system. In my case the app reads hundreds images from disk or network file share, performs operations (scale/rotate/crop/etc), rendering on CPU (better accuracy for printing) and writing to disk or network share. Perfect fit for NSOperationQueue with maxConcurrentOperationCount, so I was grateful someone at Apple had already done the hard work and made concurrent programming much easier for people with tasks like mine. The OP is just grabbing image metadata but would almost certainly run into the same problem using GCD's dispatch_async so they’ll also be much better off using NSOperationQueue with maxConcurrentOperationCount. Jim Crate > On 17 Aug 2022, at 20:32, James Crate via Cocoa-dev > wrote: >> >> I have an app that does some image processing, and when I tried to use GCD >> it created several hundred threads which didn’t work very well. >> NSOperationQueue allows you to set the max concurrent operations, and the >> batch exporting process fully utilizes all logical cores on the CPU. >> >> opsQueue.maxConcurrentOperationCount = >> NSProcessInfo.processInfo.processorCount; >> >> Maybe I was using GCD wrong, or maybe reading, processing, and writing >> several hundred images is not a good fit for GCD concurrent queue? In any >> case NSOperationQueue is easy to use and works well. >> >> Jim Crate ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Retrieving the EXIF date/time from 250k images
> On Aug 18, 2022, at 6:47 AM, Mike Abdullah via Cocoa-dev > wrote: > > It’s not a very good fit, but when you say a “GCD concurrent queue”, you’d > need to be more specific. There are several configs possible. Do you mean a > global queue, or one you made yourself? If you made it yourself, how did you > configure it? > That was ~14 years ago. I’m just sharing what I remember. In my case, we wanted the threads to read and then perform actions on the images. Of course, reading images from media will be I/O bound from the device they’re being read from. Cheers, Alex Zavatone ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Retrieving the EXIF date/time from 250k images
It’s not a very good fit, but when you say a “GCD concurrent queue”, you’d need to be more specific. There are several configs possible. Do you mean a global queue, or one you made yourself? If you made it yourself, how did you configure it? The tricky problem is that GCD aims to optimise CPU usage. Reading image metadata from disk is almost certainly not CPU bound; it’s going to be limited by disk speed and the file system instead. GCD will have a tendency to see tasks blocked on doing this, not occupying the CPU, and so spin up another task in the hope that one _will_ then use the CPU, eventually grinding to a halt as a huge number of threads contend for access to the file system. Some kind of limit is wise. You may also wish to look at dispatch_apply(), which does work in parallel, but via a serial API. It’s a concurrent version of a for loop. Even better, for what you’re asking for, there’s a very good chance that Spotlight has the info you want indexed and on-hand. You can look into whether that is available via an API as a fast path to take when possible. Mike. > On 17 Aug 2022, at 20:32, James Crate via Cocoa-dev > wrote: > > I have an app that does some image processing, and when I tried to use GCD it > created several hundred threads which didn’t work very well. NSOperationQueue > allows you to set the max concurrent operations, and the batch exporting > process fully utilizes all logical cores on the CPU. > > opsQueue.maxConcurrentOperationCount = > NSProcessInfo.processInfo.processorCount; > > Maybe I was using GCD wrong, or maybe reading, processing, and writing > several hundred images is not a good fit for GCD concurrent queue? In any > case NSOperationQueue is easy to use and works well. > > Jim Crate ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
