> Anything computationally expensive that's naturally suitable for > parallelization. Not all problems are easy to parallelize. Dividing the work > and marshaling the results can end up being more trouble than it's worth. Then > again, when it's good...it's great.
Well, a few ideas...keeping in mind that this is all actually a bit harder to implement. Why? Because if you split one task into several parts but still need a single result, you need to figure out correctly how to divide the task, how to send the instructions to process each part, how you return the final results, and how you stitch them together and use them. The payoff can be huge, in the right situation, but the startup costs are higher than some other uses of CALL PROCESS/CALL FORM. Anyway, a few examples: * Long searches/calculations: These can be divided up by some kind of condition (ID ranges, date ranges, record numbers, etc.) and then the final results can be pushed to a controller for assembly and final disposition. * Fuzzy matching: The problem with the better fuzzy match algorithms is that they tend to be too slow to run in real time. Perhaps that can be changed. * Burst mode: Say you've got a surge of SMS messages (or whatever) to send out. I think that you can use 4DIC from within a worker. (I couldn't see this after a quick look at the docs - but I think I remember Thomas saying that in his presentation?) Fine. Start a bunch of named workers and push the tasks out to them. This could be a big win. * Analysis: Lots and lots of interesting information may be buried in your systems' data...but it's not information until you extract and refine it. Who knows what sorts of interesting results you'll get from scanning for interesting trends? If you can use idle cores, why not? I'm sure other people have nice, specific examples... On Sat, Jan 7, 2017 at 1:27 PM, David Adams <[email protected]> wrote: > Will the initials DPA serve? I don't have insights into the 4D internals. > I've always approached 4D as a black box and only believe what I can > measure. That's served me very well down the years. > > Below are some thoughts on CALL WORKER and CALL FORM. For what it's worth, > I consider these the most important new commands in the 4D language since, > well, I can't even remember. They are potentially revolutionary for 4D > developers and are very, very welcome indeed. They're what have convinced > me to look seriously at 4D again and to move to V15/V16. I'll also again > encourage everyone to check out Thomas Mauls *excellent* presentation on > the subject. Look for it here and scroll ahead to around minute 36+: > > https://4dmethod.com/ > > > I’m wondering if you’ve got some more cool v16 multiprocess, preemptive > CALL WORKER demos and > > ideas to share with the 4D community. It’s a new area and we need ideas > on how to best utilize it. > > Use it as a non-persistent, first-in-first-out message/task queue. That's > what they've given us. The cores business is a side-effect, or at least I'd > encourage you to think of them that way. EVERYONE can benefit from CALL > WORKER/CALL PROCESS, but not everyone has a problem that immediately > benefits from multiple cores. If you do, so much the better. It would be a > real shame if people didn't look at these commands because they think that > they're for problems they don't have. While they do solve some problems > that most people don't have, they solve more problems that all of us have. > Get in there early and experiment! (I'm working on getting myself numbers > organized so that I can actually try the 4D implementation. It looks sweet > to me.) > > Here's what to use it for - just a couple of ideas off the top of my head: > > Anything computationally expensive that's naturally suitable for > parallelization. > Not all problems are easy to parallelize. Dividing the work and marshaling > the results can end up being more trouble than it's worth. Then again, when > it's good...it's great. > > IP messaging > Finally, we've got a native message queue. At last. These commands should > have been introduced with New process and then we could have skipped ~20 > years of people messing around with 4D's Rohypnol commands. > Message-oriented architectures are, well, the wave of the present. It > promotes loose coupling and better reusability. In my coding religion, it's > one of the pillars of the creed. Think of everything as messages. > > Moving towards a native MVC-like GUI control system > It's becoming more plausible to do something MVC-like in 4D now with > objects, form variables, and CALL FORM. Important. This is a huge subject, > but something that everyone should try and get their heads around. Right > away. > > Logging! > If you write a custom log file, it's a total pain to have to manage > writes from multiple processes. Don't. Everything calls into CALL PROCESS > and only that process has write access to the log. This is how servers do > logs, and now so can we. (In the past, you could do the same with NTK's IPC > tools. This is how to write a Web server log, as an example.) Going out to > the file system is slow and lock/unlocks are unexpectedly expensive. Having > lived through a switchover of this description some years back, I'll say > that the speed gains can be *massive.* > > Note: As far as I can see, the 4D message queues die with the server or at > least their contents are not preserved. That kind of makes sense as they > operate in a context that would be hard to restore. So, you might need to > address that if you need a 100% perfect log. Skipping many thoughts on > solutions here...that's a big thread of its own. But a good one...there are > some pretty understandable and interesting design choices to make there. > > Unique access > Logging is a specific example of managing a shared resource, but > generalize it in your mind to other tasks where you are fighting for access > to a shared or unique resource. Some of them will do quite well in a worker > process. *Reducing contention is a big win.* It can also make your code a > whole lot simpler and more reliable. > > Cores? > Instead of focusing on cores, focus on messaging. I'll argue that it's > very likely that most developers do not have a wide range of problems that > will meaningfully benefit from multiple cores. Many should have a few > problems where it's a *massive* benefit, but probably not day to day. 4D > Server/4D themselves absolutely, absolutely do. Us? Not so much. I'm happy > to be wrong about this, but I've been working with message and task queues > since I wrote one at 4D, Inc. in the 1980's and they're part of my toolkit > for any project. That's not the same thing as CALL FORM/CALL WORKER, but > you end up with a lot of the same questions on a design level. In all of > that time, I haven't ended up finding a huge range of tasks that customers > had that could really be split up in real time for massive benefit. Keep in > mind that if more cores are what you need, a multi-machine system gives you > that in a big fat hurry. The CALL WORKER/CALL FORM system is for a single > machine, but what if you have stack of machines sitting around? Those are > extra cores. So, anyone that's ever written a batch processor or > nightly/cron-type system, think about those tasks too. > > P.S. The cool kids want to push tasks out to the GPU. Standard cores are > so 2014! > ********************************************************************** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:[email protected] **********************************************************************
