Re: read larger data streams over sockets - broken?

2017-12-06 Thread Phil Davis via use-livecode 

On 12/6/17 6:12 PM, Sannyasin Brahmanathaswami via use-livecode wrote:

May not mean anything for your use case, but I have built an in-house revision 
control system for InDesign documents… it's been in use for several years after 
the death of Adobe VersionCue (which was a night mare anyway) our team loves 
it: super simple, never fails, we never lose data, ever.

I use shell commands for everything.  not sure why you need sockets if you are 
just copy files around…from our Macs to the Synology server in the next room…


Right, a NAS device can work very well as it obviously does in your 
situation. Unfortunately this a point of sale (and more) business system 
where we want to control how much access the users have to the data, 
especially regarding things like customer credit card info. So we don't 
want to have the server mounted on each user's desktop (as would be 
required with a NAS) where they can just go in and poke around.


But thanks for your offer!

Phil



my long stack script just manages different

put format "mv \"someFile\", \"newFileName\"  # into cmdRename
and
cp # similar strings

then

get shell cmdRename

and the OS does everything..

If you want to see the stack you are welcome. It's being used (or was) by some 
publication team in Singapore for the same work.

On 12/6/17, 1:52 PM, "use-livecode on behalf of Phil Davis via use-livecode" 
 wrote:

 Is anyone else having problems with moving multi-MB data streams across
 sockets in LC 8?
 
 I support a LAN-based system that relies on sockets to move data between

 server and clients. The server's response to a single client request
 will often consist of several hundred KB to several MB of data. Until LC
 8 the successful approach has been:

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


--
Phil Davis


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: read larger data streams over sockets - broken?

2017-12-06 Thread Monte Goulding via use-livecode 
Just a guess it sounds like some kind of encoding issue but if you can provide 
demo client and server stacks that demonstrate the issue that will help us 
track down the issue.

Also let us know the previous version it was working on. 

Cheers

Monte
> On 7 Dec 2017, at 10:51 am, Phil Davis via use-livecode 
>  wrote:
> 
> Is anyone else having problems with moving multi-MB data streams across 
> sockets in LC 8?
> 
> I support a LAN-based system that relies on sockets to move data between 
> server and clients. The server's response to a single client request will 
> often consist of several hundred KB to several MB of data. Until LC 8 the 
> successful approach has been:
> 
> - client sends request to server
> 
> - server returns data to client
> 
> - client reads the returned data:
> 
>   read from socket tMySocket for 1 line -- contains the length of the
>   data that follows
>   put it into tDataLength
>   repeat while length(tData) < tDataLength
>read from socket tMySocket for tDataLength
>wait 1 millisecond with messages
>put it after tData
>   end repeat
> 
> This approach no longer works in LC 8. There are 2 new issues:
> - large time delays
> - wrong final data size
> 
> Here are some actual results from a client that was trying to receive 151k of 
> data - not that big!
> 
>   tDataLength = 151954
> 
>   read #1 got 0 chars in 15.25521 secs
>   read #2 got 131761 chars in 15.001723 secs
>   read #3 got 131761 chars in 0.001813 secs
>   actual received data length = 263522
>   total elapsed time = 30.259262 secs
> 
> I changed the client code to read 'until EOF' and got similar outcomes:
> 
>   tDataLength = 151954
> 
>   read until EOF #1 got 0 chars in 15.254117 secs
>   read until EOF #2 got 131761 chars in 15.004341 secs
>   read until EOF #3 got 131761 chars in 0.001733 secs
>   actual received data length = 263522
>   total elapsed time = 30.260464 secs
> 
> Either 'read' structure works fine as long as the data length doesn't exceed 
> roughly 128k (that's the max in my world at least - YMMV). I'm not sure if 
> the 128k size constraint is coming from LC or from my network (a standard 
> gigabit-Ethernet LAN with router, switches, computers) or from some other 
> source.
> 
> I know this is a bug and plan to file a bug report, but I wanted to see if 
> anyone else has experienced something similar.
> 
> Thanks -
> Phil Davis
> 
> -- 
> Phil Davis
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: read larger data streams over sockets - broken?

2017-12-06 Thread Sannyasin Brahmanathaswami via use-livecode 
May not mean anything for your use case, but I have built an in-house revision 
control system for InDesign documents… it's been in use for several years after 
the death of Adobe VersionCue (which was a night mare anyway) our team loves 
it: super simple, never fails, we never lose data, ever. 

I use shell commands for everything.  not sure why you need sockets if you are 
just copy files around…from our Macs to the Synology server in the next room… 

my long stack script just manages different  

put format "mv \"someFile\", \"newFileName\"  # into cmdRename
and
cp # similar strings

then 

get shell cmdRename

and the OS does everything..

If you want to see the stack you are welcome. It's being used (or was) by some 
publication team in Singapore for the same work.

On 12/6/17, 1:52 PM, "use-livecode on behalf of Phil Davis via use-livecode" 
 wrote:

Is anyone else having problems with moving multi-MB data streams across 
sockets in LC 8?

I support a LAN-based system that relies on sockets to move data between 
server and clients. The server's response to a single client request 
will often consist of several hundred KB to several MB of data. Until LC 
8 the successful approach has been:

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

read larger data streams over sockets - broken?

2017-12-06 Thread Phil Davis via use-livecode 
Is anyone else having problems with moving multi-MB data streams across 
sockets in LC 8?


I support a LAN-based system that relies on sockets to move data between 
server and clients. The server's response to a single client request 
will often consist of several hundred KB to several MB of data. Until LC 
8 the successful approach has been:


- client sends request to server

- server returns data to client

- client reads the returned data:

   read from socket tMySocket for 1 line -- contains the length of the
   data that follows
   put it into tDataLength
   repeat while length(tData) < tDataLength
    read from socket tMySocket for tDataLength
    wait 1 millisecond with messages
    put it after tData
   end repeat

This approach no longer works in LC 8. There are 2 new issues:
    - large time delays
    - wrong final data size

Here are some actual results from a client that was trying to receive 
151k of data - not that big!


   tDataLength = 151954

   read #1 got 0 chars in 15.25521 secs
   read #2 got 131761 chars in 15.001723 secs
   read #3 got 131761 chars in 0.001813 secs
   actual received data length = 263522
   total elapsed time = 30.259262 secs

I changed the client code to read 'until EOF' and got similar outcomes:

   tDataLength = 151954

   read until EOF #1 got 0 chars in 15.254117 secs
   read until EOF #2 got 131761 chars in 15.004341 secs
   read until EOF #3 got 131761 chars in 0.001733 secs
   actual received data length = 263522
   total elapsed time = 30.260464 secs

Either 'read' structure works fine as long as the data length doesn't 
exceed roughly 128k (that's the max in my world at least - YMMV). I'm 
not sure if the 128k size constraint is coming from LC or from my 
network (a standard gigabit-Ethernet LAN with router, switches, 
computers) or from some other source.


I know this is a bug and plan to file a bug report, but I wanted to see 
if anyone else has experienced something similar.


Thanks -
Phil Davis

--
Phil Davis

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode