Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Mattias Gaertner]

On Sun, 4 Oct 2015 21:06:57 +0200
Mattias Gaertner  wrote:

> On Sun, 4 Oct 2015 19:17:14 +0100
> Graeme Geldenhuys  wrote:
> 
> > On 2015-10-04 19:14, Graeme Geldenhuys wrote:
> > > I can't remember what it is called, but I can find the name if
> > > I search for it
> > 
> > I think it was called "SELinux" - Security-Enhanced Linux.
> 
> If it would be a security barrier, it would never work - not sometimes.

After some research it turns out it is something trivial.

"cp -i" checks if the file is in use and if yes it does not overwrite
it, even though you answered "yes".
You can overwrite the file with "cp -f".
Or you can remove the file and then copy.


Mattias
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Flávio Etrusco]

On Mon, Oct 5, 2015 at 1:58 PM, Jürgen Hestermann
 wrote:
>
>
> Am 2015-10-04 um 19:11 schrieb mic...@gmail.com:
>>>
>>> But that makes no sense to me. If the OS is designed to potentially
>>> remove
>>> running programs (or parts of it) from memory it cannot allow overwriting
>>> the file on disk in any case. When it does so, it cannot remove parts
>>> from
>>> memory because it may not be available on disk anymore. So it must
>>> disallow
>>> the disk change from the beginning, otherwise it may not be able to
>>> remove
>>> anything.
>>>
>>
>> http://askubuntu.com/questions/44339/how-does-updating-running-application-binaries-during-an-upgrade-work
>>
>>
>> http://unix.stackexchange.com/questions/49299/what-is-linux-doing-differently-that-allows-me-to-remove-replace-files-where-win
>>
>>
>> http://superuser.com/questions/251129/moving-a-file-while-its-in-use-how-does-it-work
>>
>
> In these links I didn't find any explanation that tell me if (and if yes,
> why) it *sometimes* does not work.

AFAIK (and the articles seem to confirm that) no "part is replaced"
and no parts are (necessarily) held in memory: a new file is created
on disk and the old file is kept on disk, tough not mapped the file
name, until all processes that hold "pointers" to the old file are
finished.

IIRC NTFS works like that too and Windows actively creates locks to
files in use to avoid deletion.

-Flávio
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Jürgen Hestermann]

Am 2015-10-04 um 19:11 schrieb mic...@gmail.com:

But that makes no sense to me. If the OS is designed to potentially remove
running programs (or parts of it) from memory it cannot allow overwriting
the file on disk in any case. When it does so, it cannot remove parts from
memory because it may not be available on disk anymore. So it must disallow
the disk change from the beginning, otherwise it may not be able to remove
anything.


http://askubuntu.com/questions/44339/how-does-updating-running-application-binaries-during-an-upgrade-work

http://unix.stackexchange.com/questions/49299/what-is-linux-doing-differently-that-allows-me-to-remove-replace-files-where-win

http://superuser.com/questions/251129/moving-a-file-while-its-in-use-how-does-it-work



In these links I didn't find any explanation that tell me if (and if yes, why) 
it *sometimes* does not work.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Mark Morgan Lloyd]

Bo Berglund wrote:

On Sun, 04 Oct 2015 00:36:09 +0200, Bo Berglund
 wrote:


But if I open a terminal from the menu in the VNC window it shows it
is running as root...

I have to check with the RPi forum on this VNC problem.


I have now found out the reason for the VNC shell running as root and
I have managed to get it half way solved...

It had to do with how tightvncserver was initialized following
installation and how the vncboot script was written for starting vnc
server on boot. The tutorial I used for freepascal and lazarus on pi
had a section on vnc access too so I used that when installing rather
than the official documentation tutorial on the pi website.

In any case I have now found it and tried to modify the installation
and vncboot script so that the VNC shell runs as pi. And this sems to
have succeeded.
But in the process ALL of the start menu programs in the VNC shell
disappeared! I now only have Run.., which does nothing, and Shutdown.
Plus the file manager and terminal on separate start icons. These two
show that the shell now runs as user pi.

But I have no idea how to get to Lazarus anymore since it disappeared
from the Menu. And I am not that used to Linux that I can fix this.
Sigh
Maybe I have to scrap the SDcard and start over (another day lost).


This is not the place to discuss this but hopefully Management will 
allow a single message through, after that we will need to adjourn to 
fpc-other.


There are three ways of running VNC. The first- and most common- is to 
start it up to look at an already-running session for remote support 
etc., which is probably what you've done.


The other two start it up listening to a TCP port, one way is to use 
inetd.conf and the other is to use inittab /but/ this latter doesn't 
work on Jessie or later (there might be an equivalent way to plumb it 
into systemd but I've not managed yet). Either of these ways will start 
up a new copy of the system's default display manager (lightdm or 
whatever), and at that point you can log in as an arbitrary user with 
your usual desktop... it is probably this that you're missing.


Otherwise, AS A NUMBER OF PEOPLE INCLUDING MYSELF HAVE TOLD YOU 
REPEATEDLY, you can quite simply run Lazarus- and I mean Lazarus, 
complete with its GUI stuff- over SSH. This is by far the simplest way, 
although as I've said elsewhere there can be a performance impact.


If you want to continue this, please start a thread in either fpc-other 
or lazarus-other asking about running Lazarus remotely.


--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Graeme Geldenhuys]

On 2015-10-04 13:16, Mattias Gaertner wrote:
> Replacing a running program by simply copying can result under Linux in
> 'text file busy'.

Then consider me luckily. :) In my 15 years of using Linux and 4 years
of FreeBSD I have never seen that error.

As an example, while developing I always run my programs for a console
window. I often rebuild the program, while the previous one was still
running. With Linux and FreeBSD that never gives an error, but under
Windows it will say it couldn't link the executable.


Regards,
  - Graeme -

-- 
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Michael Van Canneyt]

On Sun, 4 Oct 2015, Graeme Geldenhuys wrote:


On 2015-10-04 13:16, Mattias Gaertner wrote:

Replacing a running program by simply copying can result under Linux in
'text file busy'.


Then consider me luckily. :) In my 15 years of using Linux and 4 years
of FreeBSD I have never seen that error.


I see it nearly every day, when I recompile a fastcgi process that is still in 
use by Apache.

But I was also surprised to see it. I'm still not sure what the exact cause is, 
because it works sometimes, and sometimes not.


Michael.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Mark Morgan Lloyd]

waldo kitty wrote:

On 10/03/2015 02:30 PM, Bo Berglund wrote:

The program will probably be started by cron every minute to check if
it has anything to do and if so execute its data collection task and
quit. Otherwise just quit, but here maybe also check for an update on
the web?
But how could one exchange a running program on the pi from within
itself?


off the top of my head:

since you're running from cron and exiting, this could be pretty simple...

1. at start up, check if self is named known temp name. if yes, copy 
self to normal name, spawn normal name and die.
2. at start up pull md5 checksum file from server and compare with own 
md5 checksum.


Check size first in case somebody has inadvertently put up a binary with 
a couple of dozen Mb debug info attached :-)


One way round this is to use (something like) Subversion at both ends of 
the link, since this implements a fairly efficient check and is 
generally OK for binaries.



3. if md5 sums are different, download new version to known temp name.
4. at end of execution, see if known temp name exists. if it does, spawn 
it and die.


maybe something like that... of course, somewhere in there you'll still 
do your required processing... the end of execution will wait for 
everything to be done before doing step 4...


one might also use a special command line parameter to tell the known 
temp name to copy itself over to the normal name in step 1.


i)   Does the program need to detect changes while it's running?

ii)  What provision is made for ensuring that a new version of the 
program comes from the designated builder?


iii) Assuming that the program's not being rebuilt from source on the 
machine it's to run on, does the binary name include the name of the 
architecture etc.?


I've looked a bit at (i) for graphical programs. It can be done fairly 
reliably using dynamically-linked libraries (.so files on unix) but not 
IME quite reliably enough for remote sites. And while we're in the UK, 
we have kit scattered around from Sweden to Turkey.


Code signing is not really implemented on Linux, so (ii) is a problem.

Considering (iii), Lazarus can be told to put the architecture etc. in 
the name of a generated binary, but going further than that- and in 
particular specifying subarchitectures like the HF on the RPi- is a 
problem hardly helped by the fact that there's no provision for this in 
the ELF header.


--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Graeme Geldenhuys]

On 2015-10-03 19:30, Bo Berglund wrote:
> But how could one exchange a running program on the pi from within
> itself?

On non-Windows platforms it is much easier, because you can normally
replace a program on file, while it is running in memory. Windows
doesn't allow that. Either way the solution is quite simple for most
platforms.

tiOPF's 'tiopf_apps' repository includes one such solution. In summary,
the program checks for updates at startup - using HTTP, a database or
whatever. If a update exists, launch the update program and quit. The
update program will then fetch the update, and unpack the archive, then
launch the original program and quit itself.

For our desktop applications will added a message prompt to ask the
end-user if the want to update, but for a automated system you obviously
don't want that part. This solution has worked for us in production site
for a good few years already.

Regards,
  - Graeme -

-- 
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Bo Berglund]

On Sun, 04 Oct 2015 00:36:09 +0200, Bo Berglund
 wrote:

>But if I open a terminal from the menu in the VNC window it shows it
>is running as root...
>
>I have to check with the RPi forum on this VNC problem.

I have now found out the reason for the VNC shell running as root and
I have managed to get it half way solved...

It had to do with how tightvncserver was initialized following
installation and how the vncboot script was written for starting vnc
server on boot. The tutorial I used for freepascal and lazarus on pi
had a section on vnc access too so I used that when installing rather
than the official documentation tutorial on the pi website.

In any case I have now found it and tried to modify the installation
and vncboot script so that the VNC shell runs as pi. And this sems to
have succeeded.
But in the process ALL of the start menu programs in the VNC shell
disappeared! I now only have Run.., which does nothing, and Shutdown.
Plus the file manager and terminal on separate start icons. These two
show that the shell now runs as user pi.

But I have no idea how to get to Lazarus anymore since it disappeared
from the Menu. And I am not that used to Linux that I can fix this.
Sigh
Maybe I have to scrap the SDcard and start over (another day lost).

-- 
Bo Berglund
Developer in Sweden

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Mattias Gaertner]

On Sun, 4 Oct 2015 12:37:29 +0100
Graeme Geldenhuys  wrote:

> On 2015-10-03 19:30, Bo Berglund wrote:
> > But how could one exchange a running program on the pi from within
> > itself?
> 
> On non-Windows platforms it is much easier, because you can normally
> replace a program on file, while it is running in memory. Windows
> doesn't allow that. Either way the solution is quite simple for most
> platforms.

That is not entirely true.

Replacing a running program by simply copying can result under Linux in
'text file busy'.

You can rename a running executable, allowing to copy a new
version. This works on Windows and Linux.

Mattias
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Jürgen Hestermann]

Am 2015-10-04 um 15:41 schrieb Mark Morgan Lloyd:

Michael Van Canneyt wrote:

On Sun, 4 Oct 2015, Graeme Geldenhuys wrote:


On 2015-10-04 13:16, Mattias Gaertner wrote:

Replacing a running program by simply copying can result under Linux in
'text file busy'.


Then consider me luckily. :) In my 15 years of using Linux and 4 years
of FreeBSD I have never seen that error.


I see it nearly every day, when I recompile a fastcgi process that is still in 
use by Apache.

But I was also surprised to see it. I'm still not sure what the exact cause is, 
because it works sometimes, and sometimes not.


It generally happens if the OS (almost any OS) decides that constrained 
resources means that a segment can usefully be removed from memory, and further 
decides that it's a read-only segment so rather than writing it to swap it 
will- if it needs it again- reread it from the original binary.



But that makes no sense to me. If the OS is designed to potentially remove 
running programs (or parts of it) from memory it cannot allow overwriting the 
file on disk in any case. When it does so, it cannot remove parts from memory 
because it may not be available on disk anymore. So it must disallow the disk 
change from the beginning, otherwise it may not be able to remove anything.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Mark Morgan Lloyd]

Jürgen Hestermann wrote:

Am 2015-10-04 um 15:41 schrieb Mark Morgan Lloyd:

Michael Van Canneyt wrote:

On Sun, 4 Oct 2015, Graeme Geldenhuys wrote:


On 2015-10-04 13:16, Mattias Gaertner wrote:
Replacing a running program by simply copying can result under 
Linux in

'text file busy'.


Then consider me luckily. :) In my 15 years of using Linux and 4 years
of FreeBSD I have never seen that error.


I see it nearly every day, when I recompile a fastcgi process that is 
still in use by Apache.


But I was also surprised to see it. I'm still not sure what the exact 
cause is, because it works sometimes, and sometimes not.


It generally happens if the OS (almost any OS) decides that 
constrained resources means that a segment can usefully be removed 
from memory, and further decides that it's a read-only segment so 
rather than writing it to swap it will- if it needs it again- reread 
it from the original binary.




But that makes no sense to me. If the OS is designed to potentially 
remove running programs (or parts of it) from memory it cannot allow 
overwriting the file on disk in any case. When it does so, it cannot 


If it hasn't yet removed a read-only segment then it doesn't have to 
hold the binary open. If it wants to remove a read-only segment and 
finds that the binary has already been removed or changed then it can 
write it to swap.


remove parts from memory because it may not be available on disk 
anymore. So it must disallow the disk change from the beginning, 
otherwise it may not be able to remove anything.


If the OS has initially read the entire program into memory then it can 
close the file.


This is why people who periodically reinvent the idea of forcing the OS 
to discard initialisation code aren't being quite as bright as they 
think, since in general there isn't a flag which says "this will never, 
under any circumstances whatsoever, be reexecuted".


This is an enormous can of worms, particularly if the OS (or Apache 
etc.) is inclined to share segments in memory if a program is reexecuted.


--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Mark Morgan Lloyd]

Michael Van Canneyt wrote:

On Sun, 4 Oct 2015, Graeme Geldenhuys wrote:


On 2015-10-04 13:16, Mattias Gaertner wrote:

Replacing a running program by simply copying can result under Linux in
'text file busy'.


Then consider me luckily. :) In my 15 years of using Linux and 4 years
of FreeBSD I have never seen that error.


I see it nearly every day, when I recompile a fastcgi process that is 
still in use by Apache.


But I was also surprised to see it. I'm still not sure what the exact 
cause is, because it works sometimes, and sometimes not.


It generally happens if the OS (almost any OS) decides that constrained 
resources means that a segment can usefully be removed from memory, and 
further decides that it's a read-only segment so rather than writing it 
to swap it will- if it needs it again- reread it from the original binary.


--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[waldo kitty]

On 10/03/2015 05:40 PM, Bo Berglund wrote:

On Sat, 3 Oct 2015 15:04:31 -0400, waldo kitty
 wrote:


On 10/03/2015 02:30 PM, Bo Berglund wrote:

The program will probably be started by cron every minute to check if
it has anything to do and if so execute its data collection task and
quit. Otherwise just quit, but here maybe also check for an update on
the web?
But how could one exchange a running program on the pi from within
itself?


off the top of my head:

since you're running from cron and exiting, this could be pretty simple...

1. at start up, check if self is named known temp name. if yes, copy self to
normal name, spawn normal name and die.
2. at start up pull md5 checksum file from server and compare with own md5 
checksum.
3. if md5 sums are different, download new version to known temp name.


Would this not compare equal for the exact same version? All others
including old and newer would fail and thus trigger an update.


if the md5 on the server is the same as the md5 of the currently running 
program, there should not be an update made...



BTW, how does one get the md5 value of ones own executable file?


there's a command line tool known as md5sum... it is also possible to calculate 
the same in your own code... what i tried to propose is that when you create the 
new binary, you run md5sum on it and place both on the server... you would also 
have a md5 file beside the current binary... then you only need to pull the md5 
file from the server and compare it with the local one...



4. at end of execution, see if known temp name exists. if it does, spawn it and 
die.

maybe something like that... of course, somewhere in there you'll still do your
required processing... the end of execution will wait for everything to be done
before doing step 4...

one might also use a special command line parameter to tell the known temp name
to copy itself over to the normal name in step 1.


Interesting ideas, will have to think about it.


as i noted at the beginning, this was off the top of my head and likely how i 
would start if i were to do something similar... it may look and act completely 
different in the end as each set of bugs and problems is worked out ;)



I have not programmed in Linux before so I have no idea if programs
for Linux carry a version resource that can be checked for the version
number. Is this the case or is that only a Windows thingie?


the version resource can be done, sure... but how can you check that when 
querying the server other than remembering to manually update it each time? 
yeah, the md5 file needs to be done and uploaded so it is almost the same type 
of thing...



If not then the MD5 checksum seems to be a good way...


it is actually pretty common... i do a lot of work with IDS/IPS stuff and we use 
md5 files on the server with each set of rules archives... we pull the 30someodd 
byte md5 file and compare it with the one we have locally before considering to 
download the XXsomeodd Meg files...



I just tested on Lazarus inside the Pi2B and it was simple to do a MD5
sum of one's own executable:

strMD5 := MD5Print(MD5File(Application.Exename));

The only problem is that it seems to take quite some time to do
(several seconds) is that normal?


depending on the size of the binary, it could... so if you save that to a file 
to put with that binary on the server, then you could pull the md5 file and 
compare it with the output of the above when checking if there's an update... is 
several seconds execution too long? if so, then write it to a local md5 file to 
compare with the server's md5...


--
 NOTE: No off-list assistance is given without prior approval.
   *Please keep mailing list traffic on the list* unless
   private contact is specifically requested and granted.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[micsch]

> But that makes no sense to me. If the OS is designed to potentially remove
> running programs (or parts of it) from memory it cannot allow overwriting
> the file on disk in any case. When it does so, it cannot remove parts from
> memory because it may not be available on disk anymore. So it must disallow
> the disk change from the beginning, otherwise it may not be able to remove
> anything.
>

http://askubuntu.com/questions/44339/how-does-updating-running-application-binaries-during-an-upgrade-work

http://unix.stackexchange.com/questions/49299/what-is-linux-doing-differently-that-allows-me-to-remove-replace-files-where-win

http://superuser.com/questions/251129/moving-a-file-while-its-in-use-how-does-it-work
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Graeme Geldenhuys]

On 2015-10-04 13:57, Michael Van Canneyt wrote:
> But I was also surprised to see it. I'm still not sure what the exact cause 
> is, 
> because it works sometimes, and sometimes not.

That is strange, because I can recompile a running FastCGI app (Wisa
project) and get no error in my Linux VM - obviously I need to restart
Apache to get the new executable to take affect, but Linux never
complains about overwriting it, even though it is in use.

Do you have something like Ubuntu or SUSE's "enhanced security" enabled?
I don't. I can't remember what it is called, but I can find the name if
I search for it. Maybe that has an affect on this?

Regards,
  - Graeme -

-- 
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Mattias Gaertner]

On Sun, 4 Oct 2015 19:17:14 +0100
Graeme Geldenhuys  wrote:

> On 2015-10-04 19:14, Graeme Geldenhuys wrote:
> > I can't remember what it is called, but I can find the name if
> > I search for it
> 
> I think it was called "SELinux" - Security-Enhanced Linux.

If it would be a security barrier, it would never work - not sometimes.

Mattias
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Graeme Geldenhuys]

On 2015-10-04 19:14, Graeme Geldenhuys wrote:
> I can't remember what it is called, but I can find the name if
> I search for it

I think it was called "SELinux" - Security-Enhanced Linux.

Regards,
  - Graeme -

-- 
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Bo Berglund]

On Sun, 4 Oct 2015 12:37:29 +0100, Graeme Geldenhuys
 wrote:

>On 2015-10-03 19:30, Bo Berglund wrote:
>> But how could one exchange a running program on the pi from within
>> itself?
>
>On non-Windows platforms it is much easier, because you can normally
>replace a program on file, while it is running in memory. Windows
>doesn't allow that. Either way the solution is quite simple for most
>platforms.
>
>tiOPF's 'tiopf_apps' repository includes one such solution. In summary,
>the program checks for updates at startup - using HTTP, a database or
>whatever. If a update exists, launch the update program and quit. The
>update program will then fetch the update, and unpack the archive, then
>launch the original program and quit itself.
>
>For our desktop applications will added a message prompt to ask the
>end-user if the want to update, but for a automated system you obviously
>don't want that part. This solution has worked for us in production site
>for a good few years already.
>

Thanks for this info!
What I had in mind was to let the main program check the update state
when it it is run by cron (but only once a day, say first start after
midnight).
THe check would be done just before it exits (if it has nothing to do
or if it has finished a measurement run).
If it finds an update it will download it and apparently will be able
to replace its own executable before it exits. That is simplifying
things a lot compared to Windows.
Or it launches an updater program (separate application) and then
exits. Then the updater must be able to perform the update before the
next time cron calls a run. Could be tricky. There is a race
condition. But that would be there also if the main application
handles the update because we cannot say if cron starts it again while
it is busy measuring. Maybe I would have to think about this race a
bit, a lock file maybe

Anyway the next time it will be the new version running.
Of course no user dialog, just silently update if needed.


-- 
Bo Berglund
Developer in Sweden

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Graeme Geldenhuys]

On 2015-10-04 23:23, Bo Berglund wrote:
> handles the update because we cannot say if cron starts it again while
> it is busy measuring. Maybe I would have to think about this race a
> bit, a lock file maybe

That is easy to solve too... something I have done in practice too. Use
the SimpleIPC unit included with FPC. You can use that to check if
another instance of the application is already running. If it finds
another instance, then simply exit immediately.

SimpleIPC can be used for many things, this is just one example.

Regards,
  - Graeme -

-- 
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Ralf Quint]

On 10/3/2015 11:30 AM, Bo Berglund wrote:

I am wondering if there is some method to autoupdate an fpc program
running on a RaspberryPi? It really should not matter where the fpc
program runs but I figured the operating system has some influence, so
let's limit this to Linux.

What I would like is to endow my data collection control program on a
Pi with the ability to check for updates on the web and then update
itself. But I have no clue as to the best way to do this.

The program will probably be started by cron every minute to check if
it has anything to do and if so execute its data collection task and
quit. Otherwise just quit, but here maybe also check for an update on
the web?
But how could one exchange a running program on the pi from within
itself?

Any common methods available somewhere?


Well, there is a Lazarus component for that... ;-)

http://wiki.freepascal.org/LazAutoUpdater

Ralf

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Bo Berglund]

On Sat, 3 Oct 2015 13:58:27 -0700, Ralf Quint
 wrote:

>On 10/3/2015 11:30 AM, Bo Berglund wrote:
>> I am wondering if there is some method to autoupdate an fpc program
>> running on a RaspberryPi? It really should not matter where the fpc
>> program runs but I figured the operating system has some influence, so
>> let's limit this to Linux.
>>
>> What I would like is to endow my data collection control program on a
>> Pi with the ability to check for updates on the web and then update
>> itself. But I have no clue as to the best way to do this.
>>
>> The program will probably be started by cron every minute to check if
>> it has anything to do and if so execute its data collection task and
>> quit. Otherwise just quit, but here maybe also check for an update on
>> the web?
>> But how could one exchange a running program on the pi from within
>> itself?
>>
>> Any common methods available somewhere?
>>
>Well, there is a Lazarus component for that... ;-)
>
>http://wiki.freepascal.org/LazAutoUpdater
>

Yes, I found that but it seems to be a visual component that will
interact with a user.
My program is a command line application started by cron and running
on a headless Pi2B installed inside a box in the wilderness. Only
communicates via a mobile broadband modem/router...
I need a non-visual one.
And I did not intend to host my program on Sourceforge either...


-- 
Bo Berglund
Developer in Sweden

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

[fpc-pascal] Methods for autoupdating fpc programs?

[Bo Berglund]

I am wondering if there is some method to autoupdate an fpc program
running on a RaspberryPi? It really should not matter where the fpc
program runs but I figured the operating system has some influence, so
let's limit this to Linux.

What I would like is to endow my data collection control program on a
Pi with the ability to check for updates on the web and then update
itself. But I have no clue as to the best way to do this.

The program will probably be started by cron every minute to check if
it has anything to do and if so execute its data collection task and
quit. Otherwise just quit, but here maybe also check for an update on
the web?
But how could one exchange a running program on the pi from within
itself?

Any common methods available somewhere?

-- 
Bo Berglund
Developer in Sweden

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Ralf Quint]

On 10/3/2015 2:15 PM, Bo Berglund wrote:
Yes, I found that but it seems to be a visual component that will 
interact with a user. My program is a command line application started 
by cron and running on a headless Pi2B installed inside a box in the 
wilderness. Only communicates via a mobile broadband modem/router... I 
need a non-visual one. And I did not intend to host my program on 
Sourceforge either... 


Well, then run a shell script in cron instead of the executable direct. 
And before starting the executable in that script, check for the 
existence of a new version that your application has downloaded, and if 
existent, copy it over the existing one and remove the download...
That's a basic method that will work for pretty much any application, 
not only on written in FPC...


Ralf


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[waldo kitty]

On 10/03/2015 02:30 PM, Bo Berglund wrote:

The program will probably be started by cron every minute to check if
it has anything to do and if so execute its data collection task and
quit. Otherwise just quit, but here maybe also check for an update on
the web?
But how could one exchange a running program on the pi from within
itself?


off the top of my head:

since you're running from cron and exiting, this could be pretty simple...

1. at start up, check if self is named known temp name. if yes, copy self to 
normal name, spawn normal name and die.

2. at start up pull md5 checksum file from server and compare with own md5 
checksum.
3. if md5 sums are different, download new version to known temp name.
4. at end of execution, see if known temp name exists. if it does, spawn it and 
die.

maybe something like that... of course, somewhere in there you'll still do your 
required processing... the end of execution will wait for everything to be done 
before doing step 4...


one might also use a special command line parameter to tell the known temp name 
to copy itself over to the normal name in step 1.


--
 NOTE: No off-list assistance is given without prior approval.
   *Please keep mailing list traffic on the list* unless
   private contact is specifically requested and granted.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Sven Barth]

Am 04.10.2015 00:02 schrieb "Jonas Maebe" :
>
> Bo Berglund wrote:
>>
>> So this little program is24.5  Mbytes in size!
>>
>> Amazing and no wonder the MD5 calculation takes such a long time!
>
>
> Program size and speed are generally not directly correlated (think of
e.g. inlining, which increases program size and which can also increase
speed at the same time).

In this case it is, because he generates a checksum of the running binary ;)

Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Bo Berglund]

On Sat, 03 Oct 2015 23:40:01 +0200, Bo Berglund
 wrote:

>I just tested on Lazarus inside the Pi2B and it was simple to do a MD5
>sum of one's own executable:
>
>strMD5 := MD5Print(MD5File(Application.Exename));
>
>The only problem is that it seems to take quite some time to do
>(several seconds) is that normal?

OUCH!
My test application created in Lazarus contains a single small form
with two edit boxes and two buttons.
THe code behind each button is 3-4 lines each invoking MD5 functions.

I was intrigued by the location of the executable shown in the
messages box:
/root/tmp/project1
So I decided to go there and have a look at the file itself.
This is what I found (had to do sudo su to be allowed to view this
location):
-rwxr-xr-x 1 root root 24493881 Oct  3 21:35 project1

So this little program is 24.5 Mbytes in size!
Amazing and no wonder the MD5 calculation takes such a long time!
Why is FPC producing such an incredibly big program file?

And why is Lazarus running as root? I would have believed it was using
the Pi user rather than root? (Maybe this is a Lazarus question rather
than FPC).

-- 
Bo Berglund
Developer in Sweden

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Bo Berglund]

On Sat, 3 Oct 2015 15:04:31 -0400, waldo kitty
 wrote:

>On 10/03/2015 02:30 PM, Bo Berglund wrote:
>> The program will probably be started by cron every minute to check if
>> it has anything to do and if so execute its data collection task and
>> quit. Otherwise just quit, but here maybe also check for an update on
>> the web?
>> But how could one exchange a running program on the pi from within
>> itself?
>
>off the top of my head:
>
>since you're running from cron and exiting, this could be pretty simple...
>
>1. at start up, check if self is named known temp name. if yes, copy self to 
>normal name, spawn normal name and die.
>2. at start up pull md5 checksum file from server and compare with own md5 
>checksum.
>3. if md5 sums are different, download new version to known temp name.

Would this not compare equal for the exact same version? All others
including old and newer would fail and thus trigger an update.
BTW, how does one get the md5 value of ones own executable file?

>4. at end of execution, see if known temp name exists. if it does, spawn it 
>and die.
>
>maybe something like that... of course, somewhere in there you'll still do 
>your 
>required processing... the end of execution will wait for everything to be 
>done 
>before doing step 4...
>
>one might also use a special command line parameter to tell the known temp 
>name 
>to copy itself over to the normal name in step 1.

Interesting ideas, will have to think about it.

I have not programmed in Linux before so I have no idea if programs
for Linux carry a version resource that can be checked for the version
number. Is this the case or is that only a Windows thingie?

If not then the MD5 checksum seems to be a good way...

I just tested on Lazarus inside the Pi2B and it was simple to do a MD5
sum of one's own executable:

strMD5 := MD5Print(MD5File(Application.Exename));

The only problem is that it seems to take quite some time to do
(several seconds) is that normal?

-- 
Bo Berglund
Developer in Sweden

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Jonas Maebe]

Bo Berglund wrote:

So this little program is24.5  Mbytes in size!
Amazing and no wonder the MD5 calculation takes such a long time!


Program size and speed are generally not directly correlated (think of 
e.g. inlining, which increases program size and which can also increase 
speed at the same time).



Why is FPC producing such an incredibly big program file?


a) you have debug info enabled
b) you don't have dead code stripping (a.k.a. "smart linking") enabled

This also suggests you are creating a debug build, which additionally 
means you probably don't have any optimisations enabled.



And why is Lazarus running as root?


Because you presumably started it as root. If you start an application 
as a normal user, it cannot get root permissions (unless your lazarus 
binary is marked as setuid root, but that's unlikely).



Jonas
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Sven Barth]

Am 03.10.2015 23:56 schrieb "Bo Berglund" :
> So this little program is 24.5 Mbytes in size!
> Amazing and no wonder the MD5 calculation takes such a long time!
> Why is FPC producing such an incredibly big program file?

That's the debug information. If you disable the generation of the debug
information or tell the compiler to use an external file for that (both
options are available in the project options), then the size should be
around 1.5 MB (or was it 3?). However the size is only growing very slowly
with each added form/control/code as the bulk of it stems from the LCL (so
a pure console application as you plan is much smaller anyway).

Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Methods for autoupdating fpc programs?

[Bo Berglund]

On Sun, 04 Oct 2015 00:02:01 +0200, Jonas Maebe
 wrote:

>> And why is Lazarus running as root?
>
>Because you presumably started it as root. If you start an application 
>as a normal user, it cannot get root permissions (unless your lazarus 
>binary is marked as setuid root, but that's unlikely).

If that is the case then my entire VNC session must be running as root
rather than user pi...

I have no idea how this happens since I only connect on the IP:5900
socket and the desktop is displayed on which I start Lazarus from the
Menu. There is no user login required where the active user is set.

But if I open a terminal from the menu in the VNC window it shows it
is running as root...

I have to check with the RPi forum on this VNC problem.

-- 
Bo Berglund
Developer in Sweden

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal