Re: Autosaving stacks corrupted on network drives

[Richard Gaskin via use-livecode]

Brian Milby wrote:

> The stack file isn’t a package.  If using an old format, then it may
> have a resource fork.

Anything in the resource for was put there by the developer.  Having 
been born on Unix. MetaCard (the LC engine's original name) provided 
functions for using resources on Mac, but never relied on them in its 
engine or IDE.


As for Dropbox and other cloud storage services:

Even without multiple simultaneously users modifying the file, this is a 
pervasive problem not at all limited to LiveCode stack files.  Indeed, 
LC's saves are pretty sparse and rewrite the entire file, as opposed to 
more complex paging systems like HyperCard and SuperCard used.


One of the more complex paging data formats we LC folks use frequently 
is SQLite,


Tons of questions about that all over the Web:


One software vendor has gone so far as to explicitly advise their 
customers not to work on SQLite files directly in cloud-synced folders, 
recommending copying them there for backups only:


   Caution

   We strongly recommend that you do not work with a project inside
   the synchronized storage area, meaning that you never open a project
   file from right within the cloud folder (e.g. your Dropbox folder).
   Instead, we recommend to copy the project to a location that is not
   synchronized e.g. to your desktop and copy it back to your cloud
   service when you are finished working on it.

   Especially when working on a Mac computer we cannot guarantee that
   project files are damaged by your cloud service starting a
   synchronisation process while the MAXQDA project is opened, thus
   possibly corrupting the project.



This is reinforced by a "superuser" in the Dropbox forums:

   "First, it's suggested that you not use database files within
Dropbox. Conflicted copies are a likely result (as someone else
posted about recently as well).
 




On the SQLite list someone wrote:

  "If Dropbox decides to take a copy while SQLite is in the middle of
   processing a transaction, you will temporarily have a copy of the
   database with a partially-processed transaction on all the computers
   which have access to that shared area.

  "Dropbox doesn’t understand that the database file and the journal
   file go together, even if they’re in the same folder.  And in order
   to stop one user from hogging its servers there’s sometimes a short
   delay between when it updates its copy of one file and when it
   updates its copy of another file.  So it’s possible for one computer
   which has a copy of the database to have a newer database file than
   its journal file, or vice versa."




A user in a discussion on this problem in the Xojo forums noted succinctly:
"It does not need a Data corruption, it just needs 1 Network related 
delay of your sync and you loose Data."




Since LC writes stack files whole when saving, I'm not sure why this 
would be as much of a problem as it appears to be with paged formats.


I wonder if LC's pre-save step of making a backup copy of a file somehow 
throws off Dropbox's sync so that it's tardy in then syncing the actual 
new stack file.  Perhaps the extra network traffic may be enough to 
throw things off, per the Xojo user's point.



PS: FWIW with my own use syncing my LC Plugins folder and work files, 
I've seen far fewer "conflict files" since I switched to Nextcloud after 
having used Dropbox and Google Drive.  Maybe it's just luck, or maybe 
it's that most of the time I'm syncing over the local network (my 
Nextcloud box sits three feet from my main workstation).  Or maybe 
Nextcloud's use of OS-provided file-changed flags is just a bit more 
tidy than whatever monitoring method Dropbox and others use.


--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 
 ambassa...@fourthworld.comhttp://www.FourthWorld.com

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Autosaving stacks corrupted on network drives

[Bob Sneidar via use-livecode]

There may be a way to shell the permissions, but it would require sudo on a Mac 
and UAC on Windows. There is a locked flag in both OS I believe... but you 
would then have to script saves. An alternative way to semaphore this process 
is to have a text file that you increment a number for anytime changes are 
saved, then check to see if the prior number matches the current number and act 
accordingly. 

But all this really lends itself to the adoption of Trevor's Levure Framework, 
doesn't it? If the only changes being made are script changes, then set the 
permission of the UI stack so that only one person or a group of persons can 
open it for editing, and leave the script only behaviors editable by all. 

Someone can still overwrite someone elses changes though. I was at a SoCal 
users group some time back, and one of the attendees showed off a very nice 
commercial app, and they created their own system where files were checked out, 
including UI files. Otherwise GIT can be used to control access and versioning. 

Bob S


> On Oct 25, 2017, at 11:03 , Ralph DiMola via use-livecode 
>  wrote:
> 
> True, The stack is not open after it's read into memory. I can see why/why
> not this is done from a LC point of view. If the stack is being read in from
> a web server it's not possible to lock it. If the file is local/network then
> the stack could remain open and locked for write. Other processes could then
> open/read it but not save it. This looks like it's been around for a long
> time. Could locking stack(s) break existing projects? I vote to lock for
> write any open stacks.
> 
> Ralph DiMola
> IT Director
> Evergreen Information Services
> rdim...@evergreeninfo.net


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

RE: Autosaving stacks corrupted on network drives

[Ralph DiMola via use-livecode]

True, The stack is not open after it's read into memory. I can see why/why
not this is done from a LC point of view. If the stack is being read in from
a web server it's not possible to lock it. If the file is local/network then
the stack could remain open and locked for write. Other processes could then
open/read it but not save it. This looks like it's been around for a long
time. Could locking stack(s) break existing projects? I vote to lock for
write any open stacks.

Ralph DiMola
IT Director
Evergreen Information Services
rdim...@evergreeninfo.net


-Original Message-
From: use-livecode [mailto:use-livecode-boun...@lists.runrev.com] On Behalf
Of Bob Sneidar via use-livecode
Sent: Wednesday, October 25, 2017 1:13 PM
To: How to use LiveCode
Cc: Bob Sneidar
Subject: Re: Autosaving stacks corrupted on network drives

Okay I think I see what is going on here. A Livecode stack, though it
appears to be open in the IDE, is NOT open from the OS point of view. To
test this, create a new livecode stack, save it, then attempt to delete it
in the OS while it's still open in the IDE. If my theory is correct, you
should have no problem doing so. 

I did this and sure enough, I was able to put the file in the trash, AND
THEN EMPTY IT! But when I saved again in the IDE, it recreated the file. 

Bob S


> On Oct 25, 2017, at 09:26 , Mike Kerner via use-livecode
<use-livecode@lists.runrev.com> wrote:
> 
> Now if what you're suggesting is that the server OS should manage the 
> access process so that you cannot have two simultaneous accesses, I 
> would agree, but my experience has been that it doesn't always work that
way.
> 
> On Wed, Oct 25, 2017 at 12:19 PM, Mike Kerner 
> <mikeker...@roadrunner.com>
> wrote:
> 
>> I'm saying that it is possible to access the same file from the same 
>> share from multiple machines and leave the file in an indeterminate 
>> or corrupted state, afterwards, especially if both machines accessing 
>> the share are trying to save changes to said file.  The purpose of 
>> the semaphore would be to stop the simultaneous access of the file from
multiple machines.
>> 
>> 
>> On Wed, Oct 25, 2017 at 11:54 AM, Bob Sneidar via use-livecode < 
>> use-livecode@lists.runrev.com> wrote:
>> 
>>> I thought the OS takes care of that. Are we saying that when saving 
>>> to a network share this process of creating the semaphore file does not
happen?
>>> 
>>> Bob S
>>> 
>>> 
>>>> On Oct 24, 2017, at 16:01 , Mike Kerner via use-livecode <
>>> use-livecode@lists.runrev.com> wrote:
>>>> 
>>>> Possible solutions:
>>>> 1) Do what LibreOffice does.  Create an invisible semaphore file 
>>>> that
>>> the
>>>> stack checks for on open.  If it exists, the open is rejected and 
>>>> the
>>> stack
>>>> immediately closes.  This will keep secondary and simultaneous 
>>>> users
>>> from
>>>> getting their grubby paws into the stack before the save/sync is
>>> complete.
>>>> As part of this solution, I would suggest any quit/closeStack event 
>>>> has
>>> a
>>>> built in delay to confirm that the sync/save has completed before
>>> removing
>>>> the semaphore.  This is still tricky as you are at the mercy of the 
>>>> sync/save tool, however, if you're clever, you can use the 
>>>> service's
>>> api to
>>>> check, separately, if the sync/save is complete before you let LC 
>>>> finish closing down.
>>>> 2) Split the data out into a separate data file or better into a
>>> database
>>>> (because most databases use transactions, with greatly minimizes 
>>>> the probability of corruption).
>>> 
>>> 
>>> ___
>>> use-livecode mailing list
>>> use-livecode@lists.runrev.com
>>> Please visit this url to subscribe, unsubscribe and manage your 
>>> subscription preferences:
>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>> 
>> 
>> 
>> 
>> --
>> On the first day, God created the heavens and the Earth On the second 
>> day, God created the oceans.
>> On the third day, God put the animals on hold for a few hours,
>>   and did a little diving.
>> And God said, "This is good."
>> 
> 
> 
> 
> --
> On the first day, God created the heavens and the Earth On the second 
> day, God created the oceans.
> On the third day, God put the animals on hold for a few hours,
>   and did a little diving.
> And God said

Re: Autosaving stacks corrupted on network drives

[Brian Milby via use-livecode]

The stack file isn’t a package.  If using an old format, then it may have a
resource fork.  The stack is read into memory and the file closed (not sure
how resource fork stuff is handled though).

On Wed, Oct 25, 2017 at 12:16 PM Mike Kerner via use-livecode <
use-livecode@lists.runrev.com> wrote:

> I was thinking about the mechanics of this.  A stack is really a
> folder/bundle, at least on macos/ios, so I wonder if parts of the stack
> bundle are open and parts are not, which could really make things
> problematic...
>
> On Wed, Oct 25, 2017 at 1:12 PM, Bob Sneidar via use-livecode <
> use-livecode@lists.runrev.com> wrote:
>
> > Okay I think I see what is going on here. A Livecode stack, though it
> > appears to be open in the IDE, is NOT open from the OS point of view. To
> > test this, create a new livecode stack, save it, then attempt to delete
> it
> > in the OS while it's still open in the IDE. If my theory is correct, you
> > should have no problem doing so.
> >
> > I did this and sure enough, I was able to put the file in the trash, AND
> > THEN EMPTY IT! But when I saved again in the IDE, it recreated the file.
> >
> > Bob S
> >
> >
> > > On Oct 25, 2017, at 09:26 , Mike Kerner via use-livecode <
> > use-livecode@lists.runrev.com> wrote:
> > >
> > > Now if what you're suggesting is that the server OS should manage the
> > > access process so that you cannot have two simultaneous accesses, I
> would
> > > agree, but my experience has been that it doesn't always work that way.
> > >
> > > On Wed, Oct 25, 2017 at 12:19 PM, Mike Kerner <
> mikeker...@roadrunner.com
> > >
> > > wrote:
> > >
> > >> I'm saying that it is possible to access the same file from the same
> > share
> > >> from multiple machines and leave the file in an indeterminate or
> > corrupted
> > >> state, afterwards, especially if both machines accessing the share are
> > >> trying to save changes to said file.  The purpose of the semaphore
> > would be
> > >> to stop the simultaneous access of the file from multiple machines.
> > >>
> > >>
> > >> On Wed, Oct 25, 2017 at 11:54 AM, Bob Sneidar via use-livecode <
> > >> use-livecode@lists.runrev.com> wrote:
> > >>
> > >>> I thought the OS takes care of that. Are we saying that when saving
> to
> > a
> > >>> network share this process of creating the semaphore file does not
> > happen?
> > >>>
> > >>> Bob S
> > >>>
> > >>>
> >  On Oct 24, 2017, at 16:01 , Mike Kerner via use-livecode <
> > >>> use-livecode@lists.runrev.com> wrote:
> > 
> >  Possible solutions:
> >  1) Do what LibreOffice does.  Create an invisible semaphore file
> that
> > >>> the
> >  stack checks for on open.  If it exists, the open is rejected and
> the
> > >>> stack
> >  immediately closes.  This will keep secondary and simultaneous users
> > >>> from
> >  getting their grubby paws into the stack before the save/sync is
> > >>> complete.
> >  As part of this solution, I would suggest any quit/closeStack event
> > has
> > >>> a
> >  built in delay to confirm that the sync/save has completed before
> > >>> removing
> >  the semaphore.  This is still tricky as you are at the mercy of the
> >  sync/save tool, however, if you're clever, you can use the service's
> > >>> api to
> >  check, separately, if the sync/save is complete before you let LC
> > finish
> >  closing down.
> >  2) Split the data out into a separate data file or better into a
> > >>> database
> >  (because most databases use transactions, with greatly minimizes the
> >  probability of corruption).
> > >>>
> > >>>
> > >>> ___
> > >>> use-livecode mailing list
> > >>> use-livecode@lists.runrev.com
> > >>> Please visit this url to subscribe, unsubscribe and manage your
> > >>> subscription preferences:
> > >>> http://lists.runrev.com/mailman/listinfo/use-livecode
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> On the first day, God created the heavens and the Earth
> > >> On the second day, God created the oceans.
> > >> On the third day, God put the animals on hold for a few hours,
> > >>   and did a little diving.
> > >> And God said, "This is good."
> > >>
> > >
> > >
> > >
> > > --
> > > On the first day, God created the heavens and the Earth
> > > On the second day, God created the oceans.
> > > On the third day, God put the animals on hold for a few hours,
> > >   and did a little diving.
> > > And God said, "This is good."
> > > ___
> > > use-livecode mailing list
> > > use-livecode@lists.runrev.com
> > > Please visit this url to subscribe, unsubscribe and manage your
> > subscription preferences:
> > > http://lists.runrev.com/mailman/listinfo/use-livecode
> >
> >
> > ___
> > use-livecode mailing list
> > use-livecode@lists.runrev.com
> > Please visit this url to subscribe, unsubscribe and manage your
> > subscription preferences:
> > 

Re: Autosaving stacks corrupted on network drives

[Mike Kerner via use-livecode]

I was thinking about the mechanics of this.  A stack is really a
folder/bundle, at least on macos/ios, so I wonder if parts of the stack
bundle are open and parts are not, which could really make things
problematic...

On Wed, Oct 25, 2017 at 1:12 PM, Bob Sneidar via use-livecode <
use-livecode@lists.runrev.com> wrote:

> Okay I think I see what is going on here. A Livecode stack, though it
> appears to be open in the IDE, is NOT open from the OS point of view. To
> test this, create a new livecode stack, save it, then attempt to delete it
> in the OS while it's still open in the IDE. If my theory is correct, you
> should have no problem doing so.
>
> I did this and sure enough, I was able to put the file in the trash, AND
> THEN EMPTY IT! But when I saved again in the IDE, it recreated the file.
>
> Bob S
>
>
> > On Oct 25, 2017, at 09:26 , Mike Kerner via use-livecode <
> use-livecode@lists.runrev.com> wrote:
> >
> > Now if what you're suggesting is that the server OS should manage the
> > access process so that you cannot have two simultaneous accesses, I would
> > agree, but my experience has been that it doesn't always work that way.
> >
> > On Wed, Oct 25, 2017 at 12:19 PM, Mike Kerner  >
> > wrote:
> >
> >> I'm saying that it is possible to access the same file from the same
> share
> >> from multiple machines and leave the file in an indeterminate or
> corrupted
> >> state, afterwards, especially if both machines accessing the share are
> >> trying to save changes to said file.  The purpose of the semaphore
> would be
> >> to stop the simultaneous access of the file from multiple machines.
> >>
> >>
> >> On Wed, Oct 25, 2017 at 11:54 AM, Bob Sneidar via use-livecode <
> >> use-livecode@lists.runrev.com> wrote:
> >>
> >>> I thought the OS takes care of that. Are we saying that when saving to
> a
> >>> network share this process of creating the semaphore file does not
> happen?
> >>>
> >>> Bob S
> >>>
> >>>
>  On Oct 24, 2017, at 16:01 , Mike Kerner via use-livecode <
> >>> use-livecode@lists.runrev.com> wrote:
> 
>  Possible solutions:
>  1) Do what LibreOffice does.  Create an invisible semaphore file that
> >>> the
>  stack checks for on open.  If it exists, the open is rejected and the
> >>> stack
>  immediately closes.  This will keep secondary and simultaneous users
> >>> from
>  getting their grubby paws into the stack before the save/sync is
> >>> complete.
>  As part of this solution, I would suggest any quit/closeStack event
> has
> >>> a
>  built in delay to confirm that the sync/save has completed before
> >>> removing
>  the semaphore.  This is still tricky as you are at the mercy of the
>  sync/save tool, however, if you're clever, you can use the service's
> >>> api to
>  check, separately, if the sync/save is complete before you let LC
> finish
>  closing down.
>  2) Split the data out into a separate data file or better into a
> >>> database
>  (because most databases use transactions, with greatly minimizes the
>  probability of corruption).
> >>>
> >>>
> >>> ___
> >>> use-livecode mailing list
> >>> use-livecode@lists.runrev.com
> >>> Please visit this url to subscribe, unsubscribe and manage your
> >>> subscription preferences:
> >>> http://lists.runrev.com/mailman/listinfo/use-livecode
> >>>
> >>
> >>
> >>
> >> --
> >> On the first day, God created the heavens and the Earth
> >> On the second day, God created the oceans.
> >> On the third day, God put the animals on hold for a few hours,
> >>   and did a little diving.
> >> And God said, "This is good."
> >>
> >
> >
> >
> > --
> > On the first day, God created the heavens and the Earth
> > On the second day, God created the oceans.
> > On the third day, God put the animals on hold for a few hours,
> >   and did a little diving.
> > And God said, "This is good."
> > ___
> > use-livecode mailing list
> > use-livecode@lists.runrev.com
> > Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> > http://lists.runrev.com/mailman/listinfo/use-livecode
>
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>



-- 
On the first day, God created the heavens and the Earth
On the second day, God created the oceans.
On the third day, God put the animals on hold for a few hours,
   and did a little diving.
And God said, "This is good."
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Autosaving stacks corrupted on network drives

[Bob Sneidar via use-livecode]

Okay I think I see what is going on here. A Livecode stack, though it appears 
to be open in the IDE, is NOT open from the OS point of view. To test this, 
create a new livecode stack, save it, then attempt to delete it in the OS while 
it's still open in the IDE. If my theory is correct, you should have no problem 
doing so. 

I did this and sure enough, I was able to put the file in the trash, AND THEN 
EMPTY IT! But when I saved again in the IDE, it recreated the file. 

Bob S


> On Oct 25, 2017, at 09:26 , Mike Kerner via use-livecode 
>  wrote:
> 
> Now if what you're suggesting is that the server OS should manage the
> access process so that you cannot have two simultaneous accesses, I would
> agree, but my experience has been that it doesn't always work that way.
> 
> On Wed, Oct 25, 2017 at 12:19 PM, Mike Kerner 
> wrote:
> 
>> I'm saying that it is possible to access the same file from the same share
>> from multiple machines and leave the file in an indeterminate or corrupted
>> state, afterwards, especially if both machines accessing the share are
>> trying to save changes to said file.  The purpose of the semaphore would be
>> to stop the simultaneous access of the file from multiple machines.
>> 
>> 
>> On Wed, Oct 25, 2017 at 11:54 AM, Bob Sneidar via use-livecode <
>> use-livecode@lists.runrev.com> wrote:
>> 
>>> I thought the OS takes care of that. Are we saying that when saving to a
>>> network share this process of creating the semaphore file does not happen?
>>> 
>>> Bob S
>>> 
>>> 
 On Oct 24, 2017, at 16:01 , Mike Kerner via use-livecode <
>>> use-livecode@lists.runrev.com> wrote:
 
 Possible solutions:
 1) Do what LibreOffice does.  Create an invisible semaphore file that
>>> the
 stack checks for on open.  If it exists, the open is rejected and the
>>> stack
 immediately closes.  This will keep secondary and simultaneous users
>>> from
 getting their grubby paws into the stack before the save/sync is
>>> complete.
 As part of this solution, I would suggest any quit/closeStack event has
>>> a
 built in delay to confirm that the sync/save has completed before
>>> removing
 the semaphore.  This is still tricky as you are at the mercy of the
 sync/save tool, however, if you're clever, you can use the service's
>>> api to
 check, separately, if the sync/save is complete before you let LC finish
 closing down.
 2) Split the data out into a separate data file or better into a
>>> database
 (because most databases use transactions, with greatly minimizes the
 probability of corruption).
>>> 
>>> 
>>> ___
>>> use-livecode mailing list
>>> use-livecode@lists.runrev.com
>>> Please visit this url to subscribe, unsubscribe and manage your
>>> subscription preferences:
>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>> 
>> 
>> 
>> 
>> --
>> On the first day, God created the heavens and the Earth
>> On the second day, God created the oceans.
>> On the third day, God put the animals on hold for a few hours,
>>   and did a little diving.
>> And God said, "This is good."
>> 
> 
> 
> 
> -- 
> On the first day, God created the heavens and the Earth
> On the second day, God created the oceans.
> On the third day, God put the animals on hold for a few hours,
>   and did a little diving.
> And God said, "This is good."
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Autosaving stacks corrupted on network drives

[Mike Kerner via use-livecode]

Now if what you're suggesting is that the server OS should manage the
access process so that you cannot have two simultaneous accesses, I would
agree, but my experience has been that it doesn't always work that way.

On Wed, Oct 25, 2017 at 12:19 PM, Mike Kerner 
wrote:

> I'm saying that it is possible to access the same file from the same share
> from multiple machines and leave the file in an indeterminate or corrupted
> state, afterwards, especially if both machines accessing the share are
> trying to save changes to said file.  The purpose of the semaphore would be
> to stop the simultaneous access of the file from multiple machines.
>
>
> On Wed, Oct 25, 2017 at 11:54 AM, Bob Sneidar via use-livecode <
> use-livecode@lists.runrev.com> wrote:
>
>> I thought the OS takes care of that. Are we saying that when saving to a
>> network share this process of creating the semaphore file does not happen?
>>
>> Bob S
>>
>>
>> > On Oct 24, 2017, at 16:01 , Mike Kerner via use-livecode <
>> use-livecode@lists.runrev.com> wrote:
>> >
>> > Possible solutions:
>> > 1) Do what LibreOffice does.  Create an invisible semaphore file that
>> the
>> > stack checks for on open.  If it exists, the open is rejected and the
>> stack
>> > immediately closes.  This will keep secondary and simultaneous users
>> from
>> > getting their grubby paws into the stack before the save/sync is
>> complete.
>> > As part of this solution, I would suggest any quit/closeStack event has
>> a
>> > built in delay to confirm that the sync/save has completed before
>> removing
>> > the semaphore.  This is still tricky as you are at the mercy of the
>> > sync/save tool, however, if you're clever, you can use the service's
>> api to
>> > check, separately, if the sync/save is complete before you let LC finish
>> > closing down.
>> > 2) Split the data out into a separate data file or better into a
>> database
>> > (because most databases use transactions, with greatly minimizes the
>> > probability of corruption).
>>
>>
>> ___
>> use-livecode mailing list
>> use-livecode@lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>
>
>
>
> --
> On the first day, God created the heavens and the Earth
> On the second day, God created the oceans.
> On the third day, God put the animals on hold for a few hours,
>and did a little diving.
> And God said, "This is good."
>



-- 
On the first day, God created the heavens and the Earth
On the second day, God created the oceans.
On the third day, God put the animals on hold for a few hours,
   and did a little diving.
And God said, "This is good."
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Autosaving stacks corrupted on network drives

[Mike Kerner via use-livecode]

I'm saying that it is possible to access the same file from the same share
from multiple machines and leave the file in an indeterminate or corrupted
state, afterwards, especially if both machines accessing the share are
trying to save changes to said file.  The purpose of the semaphore would be
to stop the simultaneous access of the file from multiple machines.


On Wed, Oct 25, 2017 at 11:54 AM, Bob Sneidar via use-livecode <
use-livecode@lists.runrev.com> wrote:

> I thought the OS takes care of that. Are we saying that when saving to a
> network share this process of creating the semaphore file does not happen?
>
> Bob S
>
>
> > On Oct 24, 2017, at 16:01 , Mike Kerner via use-livecode <
> use-livecode@lists.runrev.com> wrote:
> >
> > Possible solutions:
> > 1) Do what LibreOffice does.  Create an invisible semaphore file that the
> > stack checks for on open.  If it exists, the open is rejected and the
> stack
> > immediately closes.  This will keep secondary and simultaneous users from
> > getting their grubby paws into the stack before the save/sync is
> complete.
> > As part of this solution, I would suggest any quit/closeStack event has a
> > built in delay to confirm that the sync/save has completed before
> removing
> > the semaphore.  This is still tricky as you are at the mercy of the
> > sync/save tool, however, if you're clever, you can use the service's api
> to
> > check, separately, if the sync/save is complete before you let LC finish
> > closing down.
> > 2) Split the data out into a separate data file or better into a database
> > (because most databases use transactions, with greatly minimizes the
> > probability of corruption).
>
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>



-- 
On the first day, God created the heavens and the Earth
On the second day, God created the oceans.
On the third day, God put the animals on hold for a few hours,
   and did a little diving.
And God said, "This is good."
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Autosaving stacks corrupted on network drives

[Bob Sneidar via use-livecode]

I thought the OS takes care of that. Are we saying that when saving to a 
network share this process of creating the semaphore file does not happen? 

Bob S


> On Oct 24, 2017, at 16:01 , Mike Kerner via use-livecode 
>  wrote:
> 
> Possible solutions:
> 1) Do what LibreOffice does.  Create an invisible semaphore file that the
> stack checks for on open.  If it exists, the open is rejected and the stack
> immediately closes.  This will keep secondary and simultaneous users from
> getting their grubby paws into the stack before the save/sync is complete.
> As part of this solution, I would suggest any quit/closeStack event has a
> built in delay to confirm that the sync/save has completed before removing
> the semaphore.  This is still tricky as you are at the mercy of the
> sync/save tool, however, if you're clever, you can use the service's api to
> check, separately, if the sync/save is complete before you let LC finish
> closing down.
> 2) Split the data out into a separate data file or better into a database
> (because most databases use transactions, with greatly minimizes the
> probability of corruption).


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Autosaving stacks corrupted on network drives

[Mike Kerner via use-livecode]

Yes.  Transactions are the individual operations, i.e. an audit trail.
Then if there is corruption in the db, it is able to "play back" the
transactions to recreate it.

On Wed, Oct 25, 2017 at 10:31 AM, tbodine via use-livecode <
use-livecode@lists.runrev.com> wrote:

> Thanks, Mark and Trevor, for those ideas.
>
> >>Mark wrote:
> Split the data out into a separate data file or better into a database
> (because most databases use transactions, with greatly minimizes the
> probability of corruption).
>
> Does "databases use transactions" mean DBs don't rewrite the entire file
> when there is a Save operation, but only write the changes into the DB
> file?
> (Hence, faster saves?)
>
> Tom B.
>
>
>
>
> --
> Sent from: http://runtime-revolution.278305.n4.nabble.com/
> Revolution-User-f278306.html
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>



-- 
On the first day, God created the heavens and the Earth
On the second day, God created the oceans.
On the third day, God put the animals on hold for a few hours,
   and did a little diving.
And God said, "This is good."
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Autosaving stacks corrupted on network drives

[tbodine via use-livecode]

Thanks, Mark and Trevor, for those ideas.

>>Mark wrote:
Split the data out into a separate data file or better into a database 
(because most databases use transactions, with greatly minimizes the 
probability of corruption).  

Does "databases use transactions" mean DBs don't rewrite the entire file
when there is a Save operation, but only write the changes into the DB file?
(Hence, faster saves?)

Tom B.




--
Sent from: 
http://runtime-revolution.278305.n4.nabble.com/Revolution-User-f278306.html

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Autosaving stacks corrupted on network drives

[Trevor DeVore via use-livecode]

On Tue, Oct 24, 2017 at 6:03 PM Mike Kerner via use-livecode <
use-livecode@lists.runrev.com> wrote:

> Yes, this especially happens with simultaneous sync tools such as google
> drive (now backup and sync), microsoft ,oneDrive box, and dropbox.  There
> are a variety of things that play into this annoying behavior, but the most
> common one is timing and latency/connectivity.  It really gets to be bad if
> you  quit/save from one machine, and then fire it up from another before
> the sync/save is complete.


I’ve seen similar problems as well. Another option would be to save the
file to the temp folder and, if that operation is successful, delete the
original file and move the file from the temp folder to the correct
location.

Trevor DeVore
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Autosaving stacks corrupted on network drives

[Mike Kerner via use-livecode]

Yes, this especially happens with simultaneous sync tools such as google
drive (now backup and sync), microsoft ,oneDrive box, and dropbox.  There
are a variety of things that play into this annoying behavior, but the most
common one is timing and latency/connectivity.  It really gets to be bad if
you  quit/save from one machine, and then fire it up from another before
the sync/save is complete.

Possible solutions:
1) Do what LibreOffice does.  Create an invisible semaphore file that the
stack checks for on open.  If it exists, the open is rejected and the stack
immediately closes.  This will keep secondary and simultaneous users from
getting their grubby paws into the stack before the save/sync is complete.
As part of this solution, I would suggest any quit/closeStack event has a
built in delay to confirm that the sync/save has completed before removing
the semaphore.  This is still tricky as you are at the mercy of the
sync/save tool, however, if you're clever, you can use the service's api to
check, separately, if the sync/save is complete before you let LC finish
closing down.
2) Split the data out into a separate data file or better into a database
(because most databases use transactions, with greatly minimizes the
probability of corruption).

On Tue, Oct 24, 2017 at 3:55 PM, tbodine via use-livecode <
use-livecode@lists.runrev.com> wrote:

> Hi all.
>
> Looking for your insights on an issue. Here's the scenario..
>
> I have a standalone Win/Mac app that autosaves user data to a stack file as
> the user moves from card to card. Typically, files reach sizes of 100-200k.
> Normally, this works great. But a couple of times a year, some user will
> manage to get corrupted data causing his stack files to be unusable.
>
> I've opened some of these corrupt stack files in a text editor and found
> the
> data loss shows the Save operation was interrupted. There's no garble. Just
> an abrupt end to the data and the file.
>
> I've found a common link in these cases -- either the app is on a network
> drive or the stack file is on a network drive.
>
> My theory is that network save operations are slower than saving on a local
> drive, and somehow this contributes to the data loss. But how?
>
> All theories (except conspiracy) are welcome!
>
> Thanks,
> Tom B.
>
>
>
>
>
>
> --
> Sent from: http://runtime-revolution.278305.n4.nabble.com/Revolution-
> User-f278306.html
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>



-- 
On the first day, God created the heavens and the Earth
On the second day, God created the oceans.
On the third day, God put the animals on hold for a few hours,
   and did a little diving.
And God said, "This is good."
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode