Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2017-02-22 Thread Robert ListMail via 4D_Tech
James, when 4D is keeping the Postgres DB updated what mechanism is used? On the same machine, LAN or using a SOAP call that goes over the internet? Thanks, R Sent from my iPhone > On Feb 22, 2017, at 10:23 AM, James Crate via 4D_Tech <4d_tech@lists.4d.com> > wrote: > > 4D pushes data to Pos

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2017-02-22 Thread James Crate via 4D_Tech
It’s not very sophisticated. 4D pushes data to PostgreSQL, or retrieves data from PostgreSQL, based on either user-triggered actions or periodic task execution. At no point does the Rails/PostgreSQL system access 4D. The fact that 4D uses SQL to access the PostgreSQL database has specific disad

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2017-02-21 Thread Robert ListMail via 4D_Tech
Jim, that sound sophisticated. :). So, when Rails is handling a web request in what ways can Rails get to 4D data? Does all data need to be in PostgresSQL database Rails knows about? Thanks, Robert Sent from my iPhone > On Feb 21, 2017, at 9:33 AM, James Crate via 4D_Tech <4d_tech@lists.4d.co

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2017-02-21 Thread James Crate via 4D_Tech
>> On Nov 3, 2016, at 8:11 PM, James Crate wrote: >> >> Ruby/Rails apps - processing large numbers of files; performing slow fuzzy >> search on slices of data; > On Feb 20, 2017, at 7:13 PM, Robert ListMail via 4D_Tech > <4d_tech@lists.4d.com> wrote: > > Jim, are you using Ruby/Rails with 4D

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2017-02-20 Thread Robert ListMail via 4D_Tech
Jim, are you using Ruby/Rails with 4D in any way…? Thanks, Robert > On Nov 3, 2016, at 8:11 PM, James Crate wrote: > > Ruby/Rails apps - processing large numbers of files; performing slow fuzzy > search on slices of data; *

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-05 Thread Keisuke Miyako
Hello, I'm not sure if that is a fair comparison. if a variable is declared as integer in one method and real in another, the compiler will throw an error because the type must be fixed in order to generate compiled code. it doesn't matter if the offending code is behind conditions, that contrad

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-04 Thread Peter Bozek
On Fri, Nov 4, 2016 at 10:32 PM, Tony Ringsmuth wrote: > My real objection is not the IP variable thing: as stated in other > emails, it’s excluding entire methods, based on any content, and any > sub-content just makes this a terrible implementation in my view. > Tony, You are right in the se

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-04 Thread Tony Ringsmuth
Hey Chip Thanks for the reply. My real objection is not the IP variable thing: as stated in other emails, it’s excluding entire methods, based on any content, and any sub-content just makes this a terrible implementation in my view. Think of it like this: There are some commands that we canno

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-04 Thread Tony Ringsmuth
Hey Keith >Sorry Tony, I also dislike the idea of IP variables being copied into new > preemptive processes. It seems pretty un-popular with the community. I’m at peace with that. My big beef (as stated before) is that excluding entire methods, based on any content, and any sub-content jus

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-04 Thread Cannon Smith
I think that’s about right, except I think you’d have to instantiate it at the same time it was declared. Maybe something like this: ARRAY TEXT($atTemp;2) $atTemp{1}:=“value 1” $atTemp{2}:=“value 2” ARRAY TEXT(><“ here instead of “<>” to let 4D know it is a static variable, for lack of a bette

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-04 Thread Chip Scheide
create a local array/object declare the static array Copy Array (local;static) haven't worked with objects - but same Idea I assume On Tue, 1 Nov 2016 18:23:56 -0600, Cannon Smith wrote: > > I’ve been thinking about how a static variable could be instantiated. > Think of an array, for example.

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-04 Thread Chip Scheide
my 2 cents If i am to understand correctly, Tony, you and I use IP variables in a similar manner - we use them (primarily) as 'Konstants'. I think a possible solution to at least this part of the IP issue is: - some mechanism that allows us to either easily (as part of 4D not a plugin/compo

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-04 Thread Peter Jakobsson
On 4 Nov 2016, at 13:54, Keith White wrote: > The main issue we'll face with attempting to get our code to be thread safe > will be IP Arrays. Our IP arrays are static (in that they are populated on > server startup) and without them performance would be very badly degraded I am in exactly th

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-04 Thread Keith White
Hi Given the current restrictions, we're currently not in a position to contemplate Preemptive. Of course we'll be working on our code to move towards it. But what I just don't get is why 4D thinks that putting compiler _errors_ in for this feature instead of just warnings, is a great idea.

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-03 Thread David Adams
Fuzzy matching...now I really won't be able to sleep tonight. I was pretty obsessive for some time. There are a bunch of good papers about using n-gram tables (2, 3, or 4-grams) in SQL databases to perform highly optimized comparisons. It takes that little bit of extra setup, but then you can get s

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-03 Thread David Adams
I'm not sure how well it works with PostgreSQL, but for Levenshtein, you can screen out strings that are longer/shorter than your starting string by greater than your target threshold. So, if you have a 7 character string and want an edit distance of 2, your candidate pool can be limited to strings

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-03 Thread James Crate
On Nov 3, 2016, at 10:26 PM, David Adams wrote: > >> It is slow. However, being able to utilize all 16 logical cores on the > dual-processor xserve takes it from being “too slow” >> to “ok with a warning that this is slow”. Even still, it takes 30 seconds > or so to find possible duplicates in a

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-03 Thread David Adams
> It is slow. However, being able to utilize all 16 logical cores on the dual-processor xserve takes it from being “too slow” > to “ok with a warning that this is slow”. Even still, it takes 30 seconds or so to find possible duplicates in a set of 1000 names. What algorithm(s) are you using and ag

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-03 Thread James Crate
On Nov 3, 2016, at 9:30 PM, David Adams wrote: > >> processing large numbers of files; performing slow fuzzy search on slices > of data; > > You had me at fuzzy ;-) There are so many fantastic fuzzy algorithms that > are too slow to run in real-time or on normal gear. Any speed improvement > wou

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-03 Thread David Adams
> processing large numbers of files; performing slow fuzzy search on slices of data; You had me at fuzzy ;-) There are so many fantastic fuzzy algorithms that are too slow to run in real-time or on normal gear. Any speed improvement would be welcome. I've pitched 4D about integrating various fuzzy

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-03 Thread James Crate
On Nov 1, 2016, at 5:49 PM, David Adams wrote: > > I'm not going to answer either of Tony's questions as I don't feel > qualified to say one way or another. I'll admit that I've paid almost no > attention to the preemptive/cooperative subject in upcoming versions of 4D. > Can those of you who hav

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-02 Thread David Adams
> Does that appear “complex” to you? Not at all. It also doesn't look like it uses any of the commands that are forbidden in preemptive mode. > In this case it would be difficult to summarize the data in all the possible query options users have.. > It could be done but you’d have to store so many

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Tony Ringsmuth
David, Tony wrote: > Currently, 4D is trying to get around range-checking type overhead within Preemptive > processes by doing their checking at compile time. David Adams wrote: Where did you hear this? I didn't see any mention of this in the docs you mentioned and don't s

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Tim Nevels
On Nov 1, 2016, at 7:23 PM, David Adams wrote: > Who knew? Great trick. A good alternative (but more complicated) is to use > a memory-based message queue and send all of the log data to a single > process that has the file open. This is a perfect use of CALL WORKER. Yes, I agree. > I gave up o

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Neil Dennis
So if you are going to write it twice (once for preemptive and once for not) separate the methods and you are there…. Method1 $Find_i:=find in array(<>aPerson_ID;$PersonID) $TheName:=<>aPerson_Name{$Find_i} Method1Preemptive Query([person];[Person]id=$PersonID)

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread David Adams
Memory-based caches are great and it is frustrating to have to resort to building them scratch. How much speed difference will it make? I have no idea, and it depends on what you're caching, where you're caching & how much it costs to load the cache. I was recently working with PHP (on its own, not

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread John DeSoi
> On Nov 1, 2016, at 2:03 PM, Tony Ringsmuth wrote: > > QUESTION #1 > In light of the current restrictions, how likely are you to leverage > the use of Preemptive Processing in the near future? > NOT LIKELY / SOMEWHAT LIKELY / VERY LIKELY Not likely because a method and all depen

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Tony Ringsmuth
Neil > If it works for preemptive then it should work in all cases. Two cases means > double maintenance, and double testing Currently, I have a lot of places that get cached values out of IP variables that I load on startup. I want my cooperative process to be able to continue to take advant

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread David Adams
> Currently, 4D is trying to get around range-checking type overhead within Preemptive > processes by doing their checking at compile time. Where did you hear this? I didn't see any mention of this in the docs you mentioned and don't see how what the compiler scans for replaces runtime range check

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Neil Dennis
> If($IsPreemptive) > $GetSomeValue:=vMyValue > ELSE > $GetSomeValue:=<>MyIpVar > END IF So if you have to visit the code to make it work both ways why not just simply use $GetSomeValue:=vMyValue If it works for preemptive then it should work in all cases. Two cases means double maintenance

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Tony Ringsmuth
OK: I can see that my idea of Interprocess variables getting a separate instance in Preemptive processes is not popular with several of you. Consider then, a middle ground: Currently, 4D is trying to get around range-checking type overhead within Preemptive processes by doing their checking at

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Cannon Smith
Keep in mind that a static variable, once created, could not be changed by _any_ process. If it could be changed by nonpreemptive processes, then it isn’t much different than a normal IP variable in the sense that even reading it would require an OS level lock of some type. But I still think the

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Tim Nevels
On Nov 1, 2016, at 6:00 PM, Cannon Smith wrote: > The other solution would be to have 4D introduce the idea of a static > variable. So if I declared an interprocess variable the normal way, it would > not be allowed in preemptive processes. But if it was declared as static, > then it is only re

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread David Adams
On Wed, Nov 2, 2016 at 10:00 AM, Tim Nevels wrote: On Nov 1, 2016, at 4:49 PM, David Adams wrote: > There's a note about docrefs being shared in cooperative threads and not > shared in preemptive threads. I'm not at all clear what that means. I would > expect you would pass paths anyway...isn't t

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Tim Nevels
On Nov 1, 2016, at 4:49 PM, David Adams wrote: > There's a note about docrefs being shared in cooperative threads and not > shared in preemptive threads. I'm not at all clear what that means. I would > expect you would pass paths anyway...isn't that what everyone does now > anyway? I think this m

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Cannon Smith
I’m sorry, Tony, but I’m not a fan of the idea of IP variables behaving differently in different contexts. I haven’t actually used preemptive processes yet, but I’ve been studying them as I plan to use them in a few different contexts in a new project I’m on once I get to that point. I’ve been

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread David Adams
I'm not going to answer either of Tony's questions as I don't feel qualified to say one way or another. I'll admit that I've paid almost no attention to the preemptive/cooperative subject in upcoming versions of 4D. Can those of you who have tried it all out and gotten some experience provide a bit

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Peter Bozek
On Tue, Nov 1, 2016 at 10:22 PM, Tony Ringsmuth wrote: > > Are you saying that will not be allowed and thus the StandardTriggerCode > method cannot run preemptively? > Yes, If you have ANY trigger in the whole database that references ANY > restricted item (IP vars, Plugins, Various 4D commands o

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Tony Ringsmuth
Tim, Here’s the section where you can read up on SAVE RECORD and pointers: http://livedoc.4d.com/4D-Language-Reference-16/Processes/Preemptive-4D-processes.300-3036340.en.html Read the section under triggers. > Are you saying that will not be allowed and thus the StandardTriggerCode > method ca

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Peter Bozek
On Tue, Nov 1, 2016 at 9:36 PM, Tony Ringsmuth wrote: > As 4D is implementing it; your are correct. > But the goal should be to Provide a path for > - The best possible performance gain > - With the least required amount of re-development > I think they may be succeeding on #1, but totally faili

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Tim Nevels
On Nov 1, 2016, at 3:35 PM, Tony Ringsmuth wrote: > CURRENT RESTRICTIONS to using preemptive processes (up through the upcoming > release of v16): > 1: Any method that uses any restricted items cannot be called preemptively. > Restricted items include: > A: Interprocess variables >

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Tony Ringsmuth
Jody, Regarding #3: Pointers in SAVE RECORD… This ½ of my biggest frustration: The other half is that if a method contains any restricted item, then the whole method cannot be used. These are what kill it for me. This “All or nothing” of a method (and any sub-dependency) is the killer. It e

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Tony Ringsmuth
Dennis, You didn’t answer question #1. >Dennis said: Before getting too cavalier on requesting a change in the >implementation... make sure you understand why the restrictions are in place. Tony’s response: yes: I understand the reason for the restriction; and I agree, the speed loss for preem

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Tony Ringsmuth
Hey Peter, thanks for your responses. My replies below: You didn’t answer my questions #1 or #2. >(Peter said) 1. Existing code will not be usable in preemptive processes > anyway, developers will have to significantly change or - best - write the code from scratch. (Tony’s reply)

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread G-Mail
Tony: > QUESTION #1 VERY LIKELY (We will) With the understanding of some frustration we would have, and understanding of what may be their reasons, we would use them because of the advantages of them being preemptive. The item I do not like is item #3. How can I code without pointers ;-) T

RE: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Dennis, Neil
Before getting too cavalier on requesting a change in the implementation... make sure you understand why the restrictions are in place. In most places the implementation would be too slow to offset the benefit (e.g. putting semaphores around inter-process variables), in some cases to protect int

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Peter Bozek
On Tue, Nov 1, 2016 at 8:03 PM, Tony Ringsmuth wrote: > I appreciate your feedback in this > Thanks! > Tony Ringsmuth > Tony, I will try to list some arguments why I think 4D implemented preemptive processes the way they did: 1. Existing code will not be usable in preemptive processes anyway, d

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Spencer Hinsdale
Hi Tony, I love your preemptive enthusiasm! But I also want you to update to watchOS 3 so you have the Breath App at hand. IMO 4D has always, always done a terrific job of migrating us, be it to unicode, or out of subtables, or to that new OS. Do you remember Windows NT? Microsoft totally ab

Re: Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Kenneth Geiger
Hi Tony, Question 1: Not Likely Question 2: Likely Regards, Ken Geiger On Tue, Nov 1, 2016 at 1:03 PM, Tony Ringsmuth wrote: > Greetings > > I’m on a quest to persuade 4D to adopt a more useful implementation of > preemptive processing than they are currently adopting. > Please answer questio

Survey on 4D’s strategy for implementing Preemptive Processing / worker processes

2016-11-01 Thread Tony Ringsmuth
Greetings I’m on a quest to persuade 4D to adopt a more useful implementation of preemptive processing than they are currently adopting. Please answer questions #1 and #2 below if you can. CURRENT RESTRICTIONS to using preemptive processes (up through the upcoming release of v16): 1: Any metho