Re: NB: Pre-emptive mode hassle
Right, about the custom code scanner/pre-processor/compiler macro idea, I forgot to mention that you could use such a tool to remedy problems in 4D. For background, some of this takes a syntax in your code that enables you to identify the following: -- Required parameters -- Optional parameters -- Parameters that are of magic types, such as strings that are actually method names. There are a ton of ways to implement that, I've settled on one I'm happy enough with...but it involves work no matter what. (4D's compiler declaration system isn't very expressive.) Anyway, with that in place and a custom code scanner, here are some very handy things that you can do: * Count parameters and see if optional parameters are missing or if there are too many parameters in place. (The compiler doesn't do this reliably and has no concept of an optional parameter.) * Interpret parameters of magical types. For example, check that a call to ON ERR CALL is taking a proper method name, or that a call to CALL WORKER is taking a preemptively safe method. (This second one is harder to do, but doable.) * Strip out calls to TRACE as they (mysteriously) are thread-unsafe. Why? Why are they even in the compiled output? It makes no sense, they can't run anyway. And more...I'm sure everyone can come up with a few ideas for what you can do by rewriting code just before compile and build. ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: NB: Pre-emptive mode hassle
> Also trying to justify reducing the size of the process variable table. > Supposedly having 1 - 10 object variables was far superior to 100 process > variables. Who the hell cares if your process variable table is 1MB in size > when everyone has over 4,000MB to work with? And don’t try to say “it will > save time starting new processes”. Really? By how much real world time? By > 4ms? Can any human notice a decrease in time by that much? That whole business with the process variable table was and is nonsense. It makes no measurable difference to process startup and never has. > And if you really think about it, a C_OBJECT variable most likely takes up > more space in memory than a C_TEXT or C_LONGINT or C_BOOLEAN. You know, with > all the extra overhead that a C_OBJECT has to manage to provide the fast > lookup and access to properties. Don't know, can't know, 4D refuses to say. Absolutely refuses. "Because it might change." ? Such an absurd and unconvincing position to take. Here is where I *don't* talk about the Postgres documentation other than to say "how it's done." > I do a lot of UI work with 4D. I love doing UI work. Windows and dialog boxes > that users deal with all day every day. So 4D is great for a lot of the work > that I do. Right on! I wasn't saying 4D is necessarily bad for that - just that it's not my favorite thing to do. I actually really like 4D's form editor and think that they should have patented the field entry UI. That's genius. The Properly List's buggy text handling has been a pain for 20 years or sobut, overall, I like multiple pages and other stuff in there. > Isn’t it interesting that 4D became so useful and popular because it provided > a great way to build very nice user interfaces to database solutions. I never > realized you don’t care about UI stuff and actively avoid it. It explains a > lot. You are a code monster and backend database monster. If 4D is not > advancing the language and the database you think it is sitting still. Too bad > they can’t seem to advance both with equal speed. Yeah, I'm happiest in a pure code situation, for sure. I very much think about code as a series of modules with well-defined interactions. UI tends to gum that up...plus there are those slow, erratic, inefficient peripherals attached - people. Hey don't get me wrong! I'm a total people personwell, apart from the people part. > And by the way, your 4D Method presentation example database had a very nice > UI. Beautiful. So you do have talent or good taste there, in my opinion. Dude, you need to raise your standards, but thanks. All I've ever really done is steal the 4D graphic design charter from the 1990s, grab the FatCow icon set, and make sure that things tend to be aligned. I did just pick up "Information Dashboard Design" by Stephen Few, which I'm pretty excited about. But that's about the visual display of statistical data and is for my work with Domo, Loggly/SumoLogic not 4D. ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
(Offish) RE: NB: Pre-emptive mode hassle
> Nobody likes when someone says “what’s wrong with you? You are a weirdo for > not > thinking of this.” > > you are going to catch a lot more flies with honey than with vinegar. Give it > a > try you are going to find it really does work. > Who do you want to talk to or listen to? A “nice guy” or an “angry man > shaking his > fist”. I like that. :-) Regards Lutz -- Lutz Epperlein -- Agendo Gesellschaft für politische Planung mbH Köpenicker Str. 9 10997 Berlin http://www.agendo.de/ -- ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: NB: Pre-emptive mode hassle
On Wed, Oct 11, 2017 at 3:00 PM, Tim Nevels wrote: > [snip] > > > I sure can. Remember when C_OBJECT variables were introduced? There were > several developers here that talked about completely overhauling all their > code to eliminate process and interprocess variables and replace all of it > with object variables. Code was working fine. They just wanted to use this > cool new variable type. > [snip] > > I do a lot of UI work with 4D. I love doing UI work. Windows and dialog > boxes that users deal with all day every day. So 4D is great for a lot of > the work that I do. > [snip] > +1 _ Steve Simpson Cimarron Software ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: NB: Pre-emptive mode hassle
On Oct 11, 2017, at 10:40 AM, David Adams wrote: > If you don't mind and are on the forums, would you mind putting in a > feature request? And, for what it's worth, it would be super weird if they > *hadn't* thought of this. The languages they write in have all kinds of > compiler pre-processing instrucxtions and compiler macros. Nobody likes when someone says “what’s wrong with you? You are a weirdo for not thinking of this.” Who knows the minds of 4D Engineers? I’m just trying to be nice and give them the benefit of the doubt. (I know that’s not your style, but it’s mine.) David, you are going to catch a lot more flies with honey than with vinegar. Give it a try you are going to find it really does work. Who do you want to talk to or listen to? A “nice guy” or an “angry man shaking his fist”. >> I would suggest something like this: >> //%C- to turn off compiling code >> SET TEXT TO PASTEBOARD($stuff_t) >> //%C+ to turn on compiling code >> That’s how you do it with range checking using //%R- and //%R+. > > Nice! I was thinking of how you turn off warnings (look at > Compiler_Suppress_Warnings in our source if you don't know that one), but > your way is vastly better. It's actually a really exciting feature request > the way you have it, submit it! Done. > Compiler directive to turn off compiling code http://forums.4d.com/Post/FR/21136869/1/21136931#21136931 > > Sent on : Today - 11:27 AM > I would like a compiler directive like we have for range checking that you > can use to turn off compiling of code so that you can mix thread safe and non > thread safe code in a method. > > Example: > > //%C- to turn off compiling code > SET TEXT TO PASTEBOARD($stuff_t) > //%C+ to turn on compiling code > > That way when the compiler checks to see if a method is allowed to run as > preemptive it can skip over everything bracketed by "//%C-" and "//%C+". > > Good use is for debugging code that you only need when running interpreted. > But there are other good uses for a feature like this. >> Too many developers go crazy when 4D adds some new features and get it in >> their head they MUST >> rewrite everything to use the new features. That’s not perspective. I think >> “nice, another tool >> in the toolbox that I can use for something new when I get the chance." > > Honestly, I can't remember anyone I've met like that. Plenty of people dive > in to check out a new featureabout 6-12 months after they're > introduced. I doubt a lot of people are trying to use preemptive mode. Or, > if they are, they're not saying anything here. I sure can. Remember when C_OBJECT variables were introduced? There were several developers here that talked about completely overhauling all their code to eliminate process and interprocess variables and replace all of it with object variables. Code was working fine. They just wanted to use this cool new variable type. Also trying to justify reducing the size of the process variable table. Supposedly having 1 - 10 object variables was far superior to 100 process variables. Who the hell cares if your process variable table is 1MB in size when everyone has over 4,000MB to work with? And don’t try to say “it will save time starting new processes”. Really? By how much real world time? By 4ms? Can any human notice a decrease in time by that much? And if you really think about it, a C_OBJECT variable most likely takes up more space in memory than a C_TEXT or C_LONGINT or C_BOOLEAN. You know, with all the extra overhead that a C_OBJECT has to manage to provide the fast lookup and access to properties. There is no such thing as a free lunch. It may look on the surface to have savings, but under the hood — and we don’t know how it is implemented under the hood — they is most likely a price for using this new variable type. >> I'd love to hear from other people that are using preemptive processes >> regarding things they found that are good, bad, surprising, exciting, and >> me'h. >> >> CALL FORM. CALL FORM. CALL FORM. That’s the most useful new command I’ve >> found. > > Yes, it solves a lot of problems that I don't have ;-) I'm just not into > complicated 4D GUI work by choice. I do a lot of UI work with 4D. I love doing UI work. Windows and dialog boxes that users deal with all day every day. So 4D is great for a lot of the work that I do. Isn’t it interesting that 4D became so useful and popular because it provided a great way to build very nice user interfaces to database solutions. I never realized you don’t care about UI stuff and actively avoid it. It explains a lot. You are a code monster and backend database monster. If 4D is not advancing the language and the database you think it is sitting still. Too bad they can’t seem to advance both with equal speed. And by the way, your 4D Method presentation example database had a very nice UI. Beautiful. So you do have talent or good taste there, in my opinion. Tim
Re: NB: Pre-emptive mode hassle
> This would be super useful!. Great feature that would make it much easier to adapt methods so that they would run in > preemptive mode. The 4D engineers probably never even thought about doing something like this. If you don't mind and are on the forums, would you mind putting in a feature request? And, for what it's worth, it would be super weird if they *hadn't* thought of this. The languages they write in have all kinds of compiler pre-processing instrucxtions and compiler macros. > I would suggest something like this: > //%C- to turn off compiling code > SET TEXT TO PASTEBOARD($stuff_t) > //%C+ to turn on compiling code > That’s how you do it with range checking using //%R- and //%R+. Nice! I was thinking of how you turn off warnings (look at Compiler_Suppress_Warnings in our source if you don't know that one), but your way is vastly better. It's actually a really exciting feature request the way you have it, submit it! > We have to remember to bring this up at 4D Summit next year in the Q session so it is publicly directed at Laurent. You > get him to publicly say “yeah, that is a good idea, we’ll do it”. That’s the best way to cut through the red tape and get a > new feature implemented. Hmm. Yes. That sounds like an efficient way to manage software development. Cough-cough. Hopeully, the feature request system has some value. (Time will tell.) > That is exactly the advice JPR gave at the 4D World Tour. If it ain’t broke, don’t rewrite it! > Use it for new code or if you have to redo things to get a performance boost. Fair point, but my point is a bit different: It is suprisingly hard to reuse *any* code. Error handlers, utility methods, tons of stuff that we all have around. And not for obvious reasons like showing UI. So then what? You've got two different versions of your code to make one preemptively safe? When the only difference is, say, the TRACE command. Duplicating routines and making the drift apart is a bad sort of code decay. That's the hardship, not the idea of writing fresh code for fresh problems. > Too many developers go crazy when 4D adds some new features and get it in their head they MUST > rewrite everything to use the new features. That’s not perspective. I think “nice, another tool > in the toolbox that I can use for something new when I get the chance." Honestly, I can't remember anyone I've met like that. Plenty of people dive in to check out a new featureabout 6-12 months after they're introduced. I doubt a lot of people are trying to use preemptive mode. Or, if they are, they're not saying anything here. > I'd love to hear from other people that are using preemptive processes > regarding things they found that are good, bad, surprising, exciting, and > me'h. > CALL FORM. CALL FORM. CALL FORM. That’s the most useful new command I’ve found. Yes, it solves a lot of problems that I don't have ;-) I'm just not into complicated 4D GUI work by choice. If I were doing that sort of thing, I'd use CALL FORM for sure. (EXECUTE METHOD IN WINDOW.) The old Outside Call/CALL PROCESS system was dangerous garbage. And don't get me started on GET/SET PROCESS VARIABLE. Rob's IPC channels in NTK work great. The design is better than CALL FORM and would be familiar to anyone that's used conventially message queues. (You send messages, you don't push code into another thread and expand it.) But plug-ins aren't thread-safe. ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: NB: Pre-emptive mode hassle
On Oct 10, 2017, at 1:16 PM, David Adams wrote: > * TRACE is not thread-safe. It's also meaningless compiled. And yet, it > blocks you. Why? Probably the 4D engineers never thought to exclude it. It should be excluded since it has no impact when running compiled. > * SET TEXT TO PASTEBOARD is not thread-safe. I only use it for debug stuff, > it would be nice to have an option to have a compiler declaration syntax to > say something like: > > // %i C:123 > ... or whateverlike the range check compiler comments. The idea being > you can say "just compile this out, don't include it when you see it" and > then the command number. (Made up number above.) This would be super useful!. Great feature that would make it much easier to adapt methods so that they would run in preemptive mode. The 4D engineers probably never even thought about doing something like this. I would suggest something like this: //%C- to turn off compiling code SET TEXT TO PASTEBOARD($stuff_t) //%C+ to turn on compiling code That’s how you do it with range checking using //%R- and //%R+. And for all you byte hounds that strive to make everything as small as possible you could bracket all your debug code with this to reduce your compiled size. We have to remember to bring this up at 4D Summit next year in the Q session so it is publicly directed at Laurent. You get him to publicly say “yeah, that is a good idea, we’ll do it”. That’s the best way to cut through the red tape and get a new feature implemented. > But it's going to be easier to write > fresh code than rewrite or reuse old code in a lot of real-world databases, That is exactly the advice JPR gave at the 4D World Tour. If it ain’t broke, don’t rewrite it! Use it for new code or if you have to redo things to get a performance boost. Too many developers go crazy when 4D adds some new features and get it in their head they MUST rewrite everything to use the new features. That’s not perspective. I think “nice, another tool in the toolbox that I can use for something new when I get the chance." > I'd love to hear from other people that are using preemptive processes > regarding things they found that are good, bad, surprising, exciting, and > me'h. CALL FORM. CALL FORM. CALL FORM. That’s the most useful new command I’ve found. And it works from any type of process. So super useful for interprocess communication. No more CALL PROCESS(-1) or using IP variables. Just CALL FORM($winRef_l;”methodName”;param1;param2;…). So clean and nice. Tim Tim Nevels Innovative Solutions 785-749-3444 timnev...@mac.com ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: NB: Pre-emptive mode hassle
Thanks Tim! More findings: * I don't get it. I really am unclear how exactly things are supposed to be set up. It's confusing, at least to me. I've tried diagrams and writing it out...but I'm still not clear. * I want to use the 'indifferent' execution mode everywhere I can, but that's not working out. * You cannot set execution mode in batch mode the way you can with the invisible, etc. attributes. * Dang, it's seriously time-consuming and hard to nail it down. Hopefully, the tools will improve a lot in future versions. I have some optimism that they will. * TRACE is not thread-safe. It's also meaningless compiled. And yet, it blocks you. Why? * SET TEXT TO PASTEBOARD is not thread-safe. I only use it for debug stuff, it would be nice to have an option to have a compiler declaration syntax to say something like: // %i C:123 ... or whateverlike the range check compiler comments. The idea being you can say "just compile this out, don't include it when you see it" and then the command number. (Made up number above.) * If you're building a component and you want the shared methods to be accessible to preemptive methods in the host, it looks like you *must* declare the component method as 'can be run in a preemptive', etc. The 'indifferent' option doesn't seem to work. That could be documented, I have no idea. And, as a reminder, no IP variables. For existing pseudo-constants, use real custom constants instead: Managing Custom Constants with Code http://kb.4d.com/assetid=77806 All thanks and praise to Cannon! With all of the above said, my biggest advice is *wait*. This stuff is going to have to get better. (The compiler in particular.) If you have some tasks where preemptive threads are going to be a big help (workers or regular processes), fine, go for it. But it's going to be easier to write fresh code than rewrite or reuse old code in a lot of real-world databases, so try and start with something this is either 1) super simple or 2) mostly based on new code. I'd love to hear from other people that are using preemptive processes regarding things they found that are good, bad, surprising, exciting, and me'h. ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: NB: Pre-emptive mode hassle
On Oct 10, 2017, at 11:12 AM, David Adams wrote: > Typical, my "summary" is longer than the original. You are a true trail blazer, David. And you know what they say… trail blazers are the ones that take the arrows. And you are definitely bloody from all your preemptive process work. Possible new title: Bloody, angry man shaking his fist. :) And you make a good point that this is version 1 of the preemptive stuff. A lot of missing features, bugs and work arounds needed. But I bet by 4D v18 all these preemptive issues will get worked out and it will then things will be much easier. And keep those summary long. They usually contain a lot of valuable information. Tim Tim Nevels Innovative Solutions 785-749-3444 timnev...@mac.com ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: NB: Pre-emptive mode hassle
Right, I just scanned a big code base for methods passed to CALL WORKER (some my code, some not) and found that 5 out of 7 won't run in preemptive mode. Oh crap. ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: NB: Pre-emptive mode hassle
Typical, my "summary" is longer than the original. ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: NB: Pre-emptive mode hassle
To summarize the bit above: -- If you pass non-preemptive code to a pre-emptive worker, it won't work. -- That's not a bug, that's a limitation. -- 4D doesn't automatically handle this situation well, but they've said that they'll make ON ERR CALL work, which seems like a reasonable response. -- The Compiler should check methods passed to workers. Maybe it will, maybe it won't. I think that it should (obviously), but I don't get to decide. -- For now, what you need to do is remember to *manually* check code passed to the second parameter of CALL WORKER. It's on us. -- The easiest way that I've found to take this step is as follows: * Create a method marked to run in preemptive mode. You don't run it, it's just a place to expose code to the compiler. * Use Find in Design to locate all CALL WORKER calls. * Copy and paste in whatever methods you pass in the second parameter. These are what get run through EXECUTE by the worker, once received. * Use the compiler to check syntax. Now you should see errors on the methods that are going to fail. A bonus from doing things this way is that you get an error the *specific* methods that are causing trouble. For those that don't know, when you compile 4D scans the visible call tree for each method that _might_ be called from within a preemptive process. However, it only reports specific errors at the top level. It's pretty frustrating trying to figure out what the problem is (don't forget triggers). The docs say that this is the way the Compiler works, so it's not technically a bug (unexpected behavior.) It is, however, a giant, time-sucking hole that makes getting code into preemptive mode harder than it needs to be. Seems like a good feature request - the compiler gathers the necessary information during its scan and then throws it away and gives us a report that says "Something seems not okay?" Anyway, that's the state of the art today. P.S. Because of the silent failures in preemptive mode, your code may not be executing at all. Or some of it might not be. Orit might be causing problems. There's just no way to know without looking very, very carefully. It's early days for preemptive mode, so I'd suggest you consider any work with it experimental. ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
NB: Pre-emptive mode hassle
I've finally tracked down a weird problem. I have some code that I'm (still) trying to run in a worker. I've got the system set up so that I can launch in cooperative or preemptive mode based on a setting. This way, if there's a problem with preemptive mode, we can tweak a config setting and try again in cooperative mode. This is all working in tests. But my overall system was failing to deliver in preemptive mode. The preemptive worker launches just fine...by my subsequent calls to the work fail, but only in preemptive (multi-thread/multi-core) mode. I've got an ON ERR CALL installed that's thread-safe...but it never fires. I think this is part of a bug I reported months ago regarding passing non-thread safe code to a thread-safe worker. The compiler tries to stop us from *compiling* such code, but the compiled mode runtime interpreter does not. You can pass anything into a worker, even unsafe code. The exact results at that point areundefined. I think that Thomas Maul said on the forums that getting ON ERR working reliably in this situation would be done. (I'm in R3, maybe it has been done.) Anyway, the warning is this: You have to take extra steps to vet your code because *the compiler does not inspect the statements that you submit through CALL WORKER.* I think it's recently started testing that ON ERR CALL methods exist in one situation, but that's the first and only time I'm aware of that 4D sees arguments to EXECUTE (under whatever name) as anything bug raw strings. So, lesson relearned. Anyway, what you need to do is look through your code and see any that might be Unclean. The compiler is said to be able to dump out this attribute for all methods. The other way is to create a method that is marked to run in preemptive mode and then put calls to your other methods in there. In this case, the compiler will flag any that are unsafe for you. Kind of a hassle, but there it is. ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
