Randy: Good summary. This code is slightly more efficient on the transfer because it packs multiple records into a given BLOB but after reading your posting, this issue could be compounded by lack of server processing power.
"Just gotta make sure the client "syncs" often." At times, some users dump thousands of records so perhaps the server's getting overwhelmed. Given that the computing resources used by the server to *de*code all those BLOB's will be in the ballpark of the sum of the resources that all of the standalones used to *en*code all of that data, if the standalones don't "synch early, synch often", that's going to put a significant load on the server machine. The more complex the encode/decode, the worse things are for the server. And it the server's underpowered, large intermittent synch sessions will exacerbate the situation. Maybe one way to look at this is that this is an inverse "distributed processing" situation - a single server has to reverse all of the processing that's been done by N client workstations. -- Douglas von Roeder 949-336-2902 On Tue, Apr 25, 2017 at 8:40 AM, Randy Engle via 4D_Tech < 4d_tech@lists.4d.com> wrote: > Hi Douglas, > > I've been using Web Services (SOAP) for quite some time with our > Synchronization module. > I only pack the fields that have changed (or all if new record) > Send an array of field numbers, and text array of string(values) > Pack it all into a blob. > Send 1 record per web service call. > Seems fast enough for our purposes. 5-10+ records per second, depending > upon record size and network. > Just gotta make sure the client "syncs" often. > > Randy Engle > XC2 Software LLC > > -----Original Message----- > From: 4D_Tech [mailto:4d_tech-boun...@lists.4d.com] On Behalf Of Douglas > von Roeder via 4D_Tech > Sent: Monday, April 24, 2017 6:26 PM > To: 4D iNug Technical <4d_tech@lists.4d.com> > Cc: Douglas von Roeder <dvonroe...@gmail.com> > Subject: Experience with FTSY Sync Code//Speed up Sync Code > > Anyone here have experience with Brad Weber's "FTSY Sync" code? > > The code in question was written almost 20 years ago to synchronize > records between standalones and a client server system, and I know that is > was used by a couple of companies inclduing Husqvarna in North Carolina. > > One aspect of the code that's challenging is that the V11+ code (the "new" > code) could no longer use 4D Open so the design was changed to pack field > data into BLOB's. The BLOB's contain metadata for every field including the > field number, the field type, the data length, etc. > > When the synch records are unpacked, the metadata is used to move > sequentially through the BLOB, converting each byte range back to its > native 4D type using BLOB to text, BLOB to real, BLOB to longint, etc. > > My suspicion is that this method of encoding/decoding is contributing to > poor performance* for updating records and I'm hoping that someone has > resolved this issue. > > > > The underlying question is how much faster/slower would it be to > encode/decode data using an alternative method? > > A much more simple alternative is to use a "field ID" (String(Table > number;"000")+ the ID (a string)" as the tag/property name and use OB > Get/Set(property;data;field type) to deal with the data. > > This approach would eliminate a significant amount of code, no question, > but what would be the impact on performance? > > Comments, thoughts, and questions appreciated. > > > *this is a V13 system so I can't use Pause index > > > > ********************************************************************** > 4D Internet Users Group (4D iNUG) > FAQ: http://lists.4d.com/faqnug.html > Archive: http://lists.4d.com/archives.html > Options: http://lists.4d.com/mailman/options/4d_tech > Unsub: mailto:4d_tech-unsubscr...@lists.4d.com > ********************************************************************** > ********************************************************************** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **********************************************************************
