I also really enjoyed watching the video. It is a subject that I was really
excited about at the Summit and I appreciate the 4DMethod user group making
this possible.
In the video, Thomas talked about a new (to 4D) paradigm where all UI related
code could run on the main thread (the Application process) and background
processes (workers) could be used for potentially long-running code that
shouldn't block the main thread. This is exactly what many other environments
do and would be accomplished by opening all windows using DIALOG(*) from within
the Application process context and then communicating between threads using
CALL FORM (I really wish this was named CALL WINDOW) and CALL WORKER. I've been
mulling this over and am excited about some possibilities, but also see several
pitfalls that I don't know how to overcome with the current 4D language (at
least not without slow workarounds). I'm hoping to receive some guidance from
4D on these thoughts as well as spark discussion from everyone about these new
features.
(To be clear, Thomas did not recommend converting existing applications to this
approach. In fact, he recommended _not_ doing so. I have the luxury of having
just started a new project, so I'm really interested in how this approach might
work in real life.)
- There has been worry about not being able to access interprocess variables
from a pre-emptive worker process. If all the windows are in the same process,
I would expect that quite a few interprocess variables could be converted to
process variables. These variables could be passed to a worker process when it
needed to do some work so it had the most recent values. For example, some of
us keep a list of currently open windows for building a "Windows" menu.
Probably most of us keep this in an interprocess array right now and protect
access to it using a semaphore. If all windows (and menus) were in the same
process, this could become a process array and would not need to be protected
by a semaphore anymore.
Having said that, I still think there is a need for workers to be able to
occasionally access shared resources. I hope these features evolve so that
shared resources can be accessed by protecting with a semaphore or some other
locking mechanism. I understand that semaphores are somewhat expensive and, if
used a lot, would negate the usefulness of pre-emptive processes since the
underlying threads would be blocking so often. But I think they are still
useful when used judiciously and are really quite fast when needed (just a
small portion of a millisecond in most cases).
- The documentation says the Application process is automatically a worker
process. I thought this was interesting because it means that if all an
application's windows are in this process, we can use CALL FORM to execute code
within a window's context, but we can also use CALL WORKER to execute code in
that process, but not tied to a specific window. This "smells" important to me,
but I can't quite put my finger on why. Something to keep in mind, though.
- I think this paradigm is interesting for progress indicators. I like the idea
of showing progress right in the window with a spinner or thermometer. My
current way to show progress is to open a progress window in another process
since it is a bit tricky to update the UI of a window during the run loop. But
if a worker process is doing all the work, the progress indicator could be
shown in the window that started the work and the worker process could use CALL
FORM periodically to tell it how much work was done. However, I see two
problems I'm not sure how to overcome:
1. How can the user cancel some work that is being done? If I send a
message using CALL WORKER to cancel the work, I don't think it gets the message
until the work is done. Is it possible for a worker process to periodically
check for new "messages" while in the middle of work? Or would the window
itself have to chop up the work in some manner and only send the worker a bit
at a time? That seems much slower as the two processes would have to have back
and forth communication for each “block” of work.
2. By shifting the work to a worker process, the UI on the window stays
active. In some cases this is desirable, but in others you may not want the
user to be able to click or type anything in that window until the work is
done. Something to consider.
- In my current code, I tend to let almost every window have its own process.
One reason for this is that the window has full control over selections and
current records in every table. Even when the widows are array and variable
based, this is very helpful. It seems like things could get very complicated if
you had several windows open in the same process. For example, what if two of
the windows were list windows for the same table and a third table was editing
a record in that table? I suppose I could use named selections, but they seem
limited. For example, I don't know how to programmatically ask a named
selection to sort itself a certain way without making it the current selection
and then pushing the current selection back into the named selection. Is this
an expensive operation? I don't know as I've not tested it. Intuitively it
seems like it would be. It is certainly expensive code wise in the sense that
I'd have to remember to never accidentally use the current selection except to
do temporary work for a named selection.
Without an object oriented language, this makes me wish for a C_SELECTION type
variable where queries, sorts, etc. could all work directly on this "variable".
In any case, this issue looks like a deal breaker for this paradigm to me.
Hopefully I'm missing something?
- Somewhat related to the last point, I can imagine that arrays would often
need to be passed to and from a worker. But arrays can't be passed as
parameters and I understand that it would be unsafe to pass pointers to a
pre-emptive process. So we are left with packing the arrays into a C_Object or
Blob, sending that, and then unpacking into arrays. And often the reverse
process on the way back. Is there a better way to handle this?
- I imagine most of us have some kind of inter-process messaging code right
now. Mine makes it simple for a process or window to subscribe to certain
messages and then other processes can simply send a certain type of message and
any process that is subscribed will receive the message. I like this way of
doing it because the "sending" process doesn't have to know which, if any,
processes are subscribed. Of course, this is all done using inter-process
variables. I was thinking of how this type of messaging could be built on top
of these new features without interprocess variables. Perhaps this would work:
Have a worker process whose only job is to keep track of all other processes
and what they are subscribed to. Whenever a process subscribes (or
unsubscribes) to a message, they do so by sending this worker the needed
information. The worker tracks all of this in process (or just local?)
variables. Whenever a process needs to send a message, it actually sends it to
this worker which then figures out which other processes it needs to relay the
message to. This worker could also ensure that the same message wasn't sent
twice in a row when that option is important.
This gives me a potential pattern for how to convert some other pieces of code
that rely on inter-process variables to the new paradigm.
I’m really excited about these new features, but unsure how to tackle some real
world use cases. Again, I realize that many will only use these features on an
as-needed basis because it doesn’t make sense to try to convert an existing app
to them. But I want to understand the new paradigms in the context of beginning
a brand new app using best practices.
Thanks for any thoughts!
--
Cannon Smith
Synergy Farm Solutions Inc.
Hill Spring, AB Canada
403-626-3236
<[email protected]>
<www.synergyfarmsolutions.com>
**********************************************************************
4D Internet Users Group (4D iNUG)
FAQ: http://lists.4d.com/faqnug.html
Archive: http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub: mailto:[email protected]
**********************************************************************
