Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-17 Thread Alan McKinnon
On 17/12/2014 09:45, Helmut Jarausch wrote:
 On 12/17/2014 06:48:55 AM, meino.cra...@gmx.de wrote:
 
 Is it possible, to do ONE call to emerge, which asks (according
 to option -a, if set ) and given a yes first fetches ALL necessary
 files and data and compiles then everything?

 
 With some manual operations:
 
 Capture the output of emerge -vp 
 Edit it to call
 ebuild full patch to package.ebuild fetch
 for each package.
 
 Then issue emerge without -p


Way too complicated.

emerge -pvuND world

check list, ensure everything is OK, etc etc. Then

emerge -vunDf world
emerge -vuND world

-f only fetches, it does not compile. When it completes, switch the pc
off. The last emerge compiles and everything has been fetched (well
usually is has been fetched).


What the OP is trying to do is not really possible. Portage is designed
with an assumption in mind: the host is always connected to the internet
and can fetch whatever it needs to fetch whenever it needs to fetch it
based on what is in the ebuilds. 100% off-line operation is not part of
the spec, so the OP is always going to have to deal with occasional
emerge failures due to the host being offline

-- 
Alan McKinnon
alan.mckin...@gmail.com




Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-17 Thread meino . cramer
Alan McKinnon alan.mckin...@gmail.com [14-12-17 09:24]:
 On 17/12/2014 09:45, Helmut Jarausch wrote:
  On 12/17/2014 06:48:55 AM, meino.cra...@gmx.de wrote:
  
  Is it possible, to do ONE call to emerge, which asks (according
  to option -a, if set ) and given a yes first fetches ALL necessary
  files and data and compiles then everything?
 
  
  With some manual operations:
  
  Capture the output of emerge -vp 
  Edit it to call
  ebuild full patch to package.ebuild fetch
  for each package.
  
  Then issue emerge without -p
 
 
 Way too complicated.
 
 emerge -pvuND world
 
 check list, ensure everything is OK, etc etc. Then
 
 emerge -vunDf world
 emerge -vuND world
 
 -f only fetches, it does not compile. When it completes, switch the pc
 off. The last emerge compiles and everything has been fetched (well
 usually is has been fetched).
 
 
 What the OP is trying to do is not really possible. Portage is designed
 with an assumption in mind: the host is always connected to the internet
 and can fetch whatever it needs to fetch whenever it needs to fetch it
 based on what is in the ebuilds. 100% off-line operation is not part of
 the spec, so the OP is always going to have to deal with occasional
 emerge failures due to the host being offline
 
 -- 
 Alan McKinnon
 alan.mckin...@gmail.com
 
 

Hi,

thanks for your replies...

...I currently do two emerges, one with -f the second
without.
And as merntioned in my initiao mail, I dont want it, since it implies
two Calculating dependencies which is once too often...

This was the initial reason for asking...

Best regards,
Meino




Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-17 Thread Alan McKinnon
On 17/12/2014 10:41, meino.cra...@gmx.de wrote:
 Alan McKinnon alan.mckin...@gmail.com [14-12-17 09:24]:
 On 17/12/2014 09:45, Helmut Jarausch wrote:
 On 12/17/2014 06:48:55 AM, meino.cra...@gmx.de wrote:
 
 Is it possible, to do ONE call to emerge, which asks (according
 to option -a, if set ) and given a yes first fetches ALL necessary
 files and data and compiles then everything?


 With some manual operations:

 Capture the output of emerge -vp 
 Edit it to call
 ebuild full patch to package.ebuild fetch
 for each package.

 Then issue emerge without -p


 Way too complicated.

 emerge -pvuND world

 check list, ensure everything is OK, etc etc. Then

 emerge -vunDf world
 emerge -vuND world

 -f only fetches, it does not compile. When it completes, switch the pc
 off. The last emerge compiles and everything has been fetched (well
 usually is has been fetched).


 What the OP is trying to do is not really possible. Portage is designed
 with an assumption in mind: the host is always connected to the internet
 and can fetch whatever it needs to fetch whenever it needs to fetch it
 based on what is in the ebuilds. 100% off-line operation is not part of
 the spec, so the OP is always going to have to deal with occasional
 emerge failures due to the host being offline

 -- 
 Alan McKinnon
 alan.mckin...@gmail.com


 
 Hi,
 
 thanks for your replies...
 
 ...I currently do two emerges, one with -f the second
 without.
 And as merntioned in my initiao mail, I dont want it, since it implies
 two Calculating dependencies which is once too often...
 
 This was the initial reason for asking...



I know, but it's your only option



-- 
Alan McKinnon
alan.mckin...@gmail.com




Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-17 Thread Neil Bothwick
On Wed, 17 Dec 2014 07:53:53 +0100, meino.cra...@gmx.de wrote:

   Note that says parallel-fetch not build.  From the man page:
  
   parallel-fetch:  Fetch in the background while compiling. Run `tail
   -f /var/log/emerge-fetch.log` in a terminal to view parallel-fetch
   progress.  
  
  Ahh, I think I see what you are saying.  You want it to fetch and NOT
  compile until the fetch is finished.  I'm not sure if there is a way
  to do that or not.  Since it should be able to compile and fetch at
  the same time, why not try it that way and see how well it works?
 
 Yes, thats it: First download all stuff THEN start compiling.

Why? The downloads will happen at the same rate but you'll have a head
start on the compiling. The only disadvantage i can see is that you will
not have a notification of when the download finishes, but you could work
around that by having another script check emerge-fetch.log and send a
shutdown to the PC when there is no further output.
 
 Would --jobs=0 help here? This would say No packages are build
 simultanously...I check that!

No. --jobs controls package building, nothing to do with downloading.
parallel-fetch in the closest to what you want as it grabs all the
downloads as soon as possible.


-- 
Neil Bothwick

And on the seventh day God said :wq and then make


pgpfItNVFk2FI.pgp
Description: OpenPGP digital signature


Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-17 Thread meino . cramer
Neil Bothwick n...@digimed.co.uk [14-12-17 10:40]:
 On Wed, 17 Dec 2014 07:53:53 +0100, meino.cra...@gmx.de wrote:
 
Note that says parallel-fetch not build.  From the man page:
   
parallel-fetch:  Fetch in the background while compiling. Run `tail
-f /var/log/emerge-fetch.log` in a terminal to view parallel-fetch
progress.  
   
   Ahh, I think I see what you are saying.  You want it to fetch and NOT
   compile until the fetch is finished.  I'm not sure if there is a way
   to do that or not.  Since it should be able to compile and fetch at
   the same time, why not try it that way and see how well it works?
  
  Yes, thats it: First download all stuff THEN start compiling.
 
 Why? The downloads will happen at the same rate but you'll have a head
 start on the compiling. The only disadvantage i can see is that you will
 not have a notification of when the download finishes, but you could work
 around that by having another script check emerge-fetch.log and send a
 shutdown to the PC when there is no further output.
  
  Would --jobs=0 help here? This would say No packages are build
  simultanously...I check that!
 
 No. --jobs controls package building, nothing to do with downloading.
 parallel-fetch in the closest to what you want as it grabs all the
 downloads as soon as possible.
 
 
 -- 
 Neil Bothwick
 
 And on the seventh day God said :wq and then make

Hi Neil,

how can I (or the script) distinguish between an internet
connection, which is heavily slowed down (no data), blocked or an currently
not responding server and the end of all needed downloads?

How can the script check for the last needed file has been downloaded
successfully ?

Best regards,
Meino





Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-17 Thread Dale
meino.cra...@gmx.de wrote:
 Neil Bothwick n...@digimed.co.uk [14-12-17 10:40]:
 On Wed, 17 Dec 2014 07:53:53 +0100, meino.cra...@gmx.de wrote:

 Note that says parallel-fetch not build.  From the man page:

 parallel-fetch:  Fetch in the background while compiling. Run `tail
 -f /var/log/emerge-fetch.log` in a terminal to view parallel-fetch
 progress.  
 Ahh, I think I see what you are saying.  You want it to fetch and NOT
 compile until the fetch is finished.  I'm not sure if there is a way
 to do that or not.  Since it should be able to compile and fetch at
 the same time, why not try it that way and see how well it works?
 Yes, thats it: First download all stuff THEN start compiling.
 Why? The downloads will happen at the same rate but you'll have a head
 start on the compiling. The only disadvantage i can see is that you will
 not have a notification of when the download finishes, but you could work
 around that by having another script check emerge-fetch.log and send a
 shutdown to the PC when there is no further output.
 Would --jobs=0 help here? This would say No packages are build
 simultanously...I check that!
 No. --jobs controls package building, nothing to do with downloading.
 parallel-fetch in the closest to what you want as it grabs all the
 downloads as soon as possible.


 -- 
 Neil Bothwick

 And on the seventh day God said :wq and then make
 Hi Neil,

 how can I (or the script) distinguish between an internet
 connection, which is heavily slowed down (no data), blocked or an currently
 not responding server and the end of all needed downloads?

 How can the script check for the last needed file has been downloaded
 successfully ?

 Best regards,
 Meino



If I understand you correctly, emerge can run into the same issue.  If
for example it needs to download a tarball or patch and the server that
has it is not available, then emerge will skip that, download the rest
and then stop fetching.  So, either way, you can end up with things not
downloaded.  At least with the fetch option, it does all this at the
beginning of the process instead of when it gets to the package it wants
to emerge. 

The only way I see for this to work and not have to compute twice, set
the fetch option, start the emerge and then monitor the fetch log.  When
it is done with the fetch part, then you can disconnect the internet
connection and it should continue compiling. 

Other than that, I don't know of a way to do what you want.  It is 4AM
here so that may cloud up things a bit here.  :/

Dale

:-)  :-) 




Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-17 Thread Neil Bothwick
On Wed, 17 Dec 2014 10:52:44 +0100, meino.cra...@gmx.de wrote:

   Yes, thats it: First download all stuff THEN start compiling.  
  
  Why? The downloads will happen at the same rate but you'll have a head
  start on the compiling. The only disadvantage i can see is that you
  will not have a notification of when the download finishes, but you
  could work around that by having another script check
  emerge-fetch.log and send a shutdown to the PC when there is no
  further output.  
   
   Would --jobs=0 help here? This would say No packages are build
   simultanously...I check that!  
  
  No. --jobs controls package building, nothing to do with downloading.
  parallel-fetch in the closest to what you want as it grabs all the
  downloads as soon as possible.

 how can I (or the script) distinguish between an internet
 connection, which is heavily slowed down (no data), blocked or an
 currently not responding server and the end of all needed downloads?

pgrep wget will tell you if a download is still in progress. It seems
reasonable to assume that if there is no further output to the log and
wget is no longer running, portage is no longer downloading files.

Or you could get clever and set FETCH_COMMAND to a script that fetches the file 
and
then notifies of completion.

 How can the script check for the last needed file has been downloaded
 successfully ?

It can't, any more than portage does. Whether the download phases exists
successfully or unsuccessfully your Internet connection is no longer
being used, so you may as well shut down the PC.

You are trying to use portage in a way that was not intended. That
involves compromises, some work or both.

Another alternative would be to use a USB to ethernet adaptor on the
embedded board and connect it directory to your router.


-- 
Neil Bothwick

The word 'Windows' is a word out of an old dialect of the Apaches.
It means: 'White man staring through glass-screen onto an hourglass...')


pgpJiiv0J62Xk.pgp
Description: OpenPGP digital signature


Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-17 Thread Matti Nykyri
 On Dec 17, 2014, at 14:13, Neil Bothwick n...@digimed.co.uk wrote:
 
 On Wed, 17 Dec 2014 10:52:44 +0100, meino.cra...@gmx.de wrote:
 
 Yes, thats it: First download all stuff THEN start compiling.  

If I were you, I would setup your pc to do cross-compiling of your arietta's 
packages and build them into binpkg's. This could be all stored on the pc and 
accessed via nfs for example. Then the first dependency calculation would be 
done on the pc to build the packages and the second on arietta using only 
binary packages.

You should keep /etc/portage, /var/lib/portage and /usr/portage on the PC and 
not modifiable from the arietta. This way you only need to install the run time 
dependencies to the aritte. And install from bin pkg is really fast. 

 Another alternative would be to use a USB to ethernet adaptor on the
 embedded board and connect it directory to your router.

This also sounds good. Or setup server which has the usb and is always on.

-- 
-Matti


Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-17 Thread meino . cramer
Matti Nykyri matti.nyk...@iki.fi [14-12-17 15:00]:
  On Dec 17, 2014, at 14:13, Neil Bothwick n...@digimed.co.uk wrote:
  
  On Wed, 17 Dec 2014 10:52:44 +0100, meino.cra...@gmx.de wrote:
  
  Yes, thats it: First download all stuff THEN start compiling.  
 
 If I were you, I would setup your pc to do cross-compiling of your arietta's 
 packages and build them into binpkg's. This could be all stored on the pc and 
 accessed via nfs for example. Then the first dependency calculation would be 
 done on the pc to build the packages and the second on arietta using only 
 binary packages.
 
 You should keep /etc/portage, /var/lib/portage and /usr/portage on the PC and 
 not modifiable from the arietta. This way you only need to install the run 
 time dependencies to the aritte. And install from bin pkg is really fast. 
 
  Another alternative would be to use a USB to ethernet adaptor on the
  embedded board and connect it directory to your router.
 
 This also sounds good. Or setup server which has the usb and is always on.
 
 -- 
 -Matti

Hi Matti,

thanks for your reply! :)

crosscompiling is a pain. I tried several ways to do that (distcc was
among them) and it fails too often, for two reasons: Often the sources
are not prepared to be crosscompiled an include headers of my PC
(64bit) into the build of my ARM boards (32bit). Second reason: If the
crosscompilation needs meta-tools like moc for qt it fails too. The
time to fiddle out that mess is nothing what I have... ;)

Ethernet over USB:
1.) For each update I have to rearrange my setup here then. Back and
forth. Back and forth...
2.) The DSL modem is running longer than needed. I dont like the idea
to have my internet connection running over such a long time
unattended.

The problem must be solved in software.

Best regards,
Meino





Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-17 Thread Poison BL.
On Wed, Dec 17, 2014 at 9:31 AM,  meino.cra...@gmx.de wrote:
 Matti Nykyri matti.nyk...@iki.fi [14-12-17 15:00]:
  On Dec 17, 2014, at 14:13, Neil Bothwick n...@digimed.co.uk wrote:
 
  On Wed, 17 Dec 2014 10:52:44 +0100, meino.cra...@gmx.de wrote:
 
  Yes, thats it: First download all stuff THEN start compiling.

 If I were you, I would setup your pc to do cross-compiling of your arietta's 
 packages and build them into binpkg's. This could be all stored on the pc 
 and accessed via nfs for example. Then the first dependency calculation 
 would be done on the pc to build the packages and the second on arietta 
 using only binary packages.

 You should keep /etc/portage, /var/lib/portage and /usr/portage on the PC 
 and not modifiable from the arietta. This way you only need to install the 
 run time dependencies to the aritte. And install from bin pkg is really fast.

  Another alternative would be to use a USB to ethernet adaptor on the
  embedded board and connect it directory to your router.

 This also sounds good. Or setup server which has the usb and is always on.

 --
 -Matti

 Hi Matti,

 thanks for your reply! :)

 crosscompiling is a pain. I tried several ways to do that (distcc was
 among them) and it fails too often, for two reasons: Often the sources
 are not prepared to be crosscompiled an include headers of my PC
 (64bit) into the build of my ARM boards (32bit). Second reason: If the
 crosscompilation needs meta-tools like moc for qt it fails too. The
 time to fiddle out that mess is nothing what I have... ;)

 Ethernet over USB:
 1.) For each update I have to rearrange my setup here then. Back and
 forth. Back and forth...
 2.) The DSL modem is running longer than needed. I dont like the idea
 to have my internet connection running over such a long time
 unattended.

 The problem must be solved in software.

 Best regards,
 Meino

The more common fix when dealing with that range of hardware is to
build the packages on a more powerful system, then transfer them as
binary packages. Doing so for arm board's a touch less trivial, but
doable. This also solves the problem of fetching the same source
packages repeatedly, if you share Distfiles between the build
environments. I set up similar some time back based on these
instructions:

https://www.gentoo.org/proj/en/base/embedded/handbook/?part=1chap=5

for a RasPI I was playing with and it worked pretty well.

-- 
Poison [BLX]
Joshua M. Murphy



Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-17 Thread Frank Steinmetzger
On Wed, Dec 17, 2014 at 09:41:06AM +0100, meino.cra...@gmx.de wrote:

   With some manual operations:
  
   Capture the output of emerge -vp 
   Edit it to call
   ebuild full patch to package.ebuild fetch
   for each package.
  
   Then issue emerge without -p
 
 
  Way too complicated.
 
  emerge -pvuND world
 
  check list, ensure everything is OK, etc etc. Then
 
  emerge -vunDf world
  emerge -vuND world
  […]
 

 Hi,

 thanks for your replies...

 ...I currently do two emerges, one with -f the second
 without.
 And as merntioned in my initiao mail, I dont want it, since it implies
 two Calculating dependencies which is once too often...

 This was the initial reason for asking...

Well, when I run into this (my old netbook needs over 15 minutes for the
dependency calculations of a deep world update), I do -avuD world. Once the
output is shown, portage will wait for me to answer 'yes'.

Then I go to another terminal where I run emerge -f with the -O option,
because now I know which packages to fetch, thus eliminating the need for a
second dependency calculation. If the list of packages is small, I just type
the names by hand. If the list is larger, I usually prepare a list in vim
that I can paste to the emerge command.

Say you have output from emerge like so (my situation right now):

...
[ebuild U  ] media-fonts/dejavu-2.34 [2.33] USE=X -fontforge 4.597 KiB
[ebuild U  ] dev-libs/tinyxml2-2.2.0:0/2 [1.0.9_p20121123:0/0] 
USE=-static-libs {-test} 445 KiB
[ebuild  rR] kde-base/libkexiv2-4.14.3:4/4.14  USE=xmp (-aqua) -debug 0 
KiB
...

Copy-paste that into a file 'emerge.out'. If you run screen/tmux, that’s
easily done even on a mouse-less terminal. Then use the following
sed-oneliner on it which I just conjured up:

sed -e '/ 0 KiB$/d' -e 's/.\{17\}/=/' -e 's/ .*//'

It first removes all lines ending in  0 KiB (meaning no download
necessary) and then removes everything around the package name and version,
converting

[ebuild U  ] media-fonts/dejavu-2.34 [2.33] USE=X -fontforge 4.597 KiB

into

=media-fonts/dejavu-2.34

Finally, feed that to emerge -fO:

emerge -fO $(sed -e '/ 0 KiB$/d' -e 's/.\{17\}/=/' -e 's/ .*//'  emerge.out)

-- 
Gruß | Greetings | Qapla’
Please do not share anything from, with or about me with any social network.

Arrogant is he who, on his birthday, sends his parents a good wishes card.


signature.asc
Description: Digital signature


[gentoo-user] question/feature request: First fetch, then compile...

2014-12-16 Thread meino . cramer
Hi,

On my embedded systems (beaglebone black, 2 x Arietta G25) I installed
Gentoo (of course!:).
Since these systems and especially the Ariettas are not as fast as a
PC the greater update, which additionally includes C++ sources to
compile takes time (read: hours) to finish. Often I run this over
night.

Since my PC do the forwarding of requests to the internet, it has to
run the whole time also.

To circumvent this I access the embedded systems via abduco/dvtm, so
I can log out while the process keeps running.

The current (shorted decription) workflow is

eix-sync
emerge ... -f (fetching all items, so the connection to the internet is
   no longer needed)
emerge ...(starting the compilation, logout and shutdown the PC)


This includes Calculating dependencies twice of the same set of
data, which also takes a longer time. This is -- technically -- not
needed.

Is it possible, to do ONE call to emerge, which asks (according
to option -a, if set ) and given a yes first fetches ALL necessary
files and data and compiles then everything?

This would save one Calculating dependencies and also reduces writes
to the flash memory.

Is it currently possible somehow and if not: I would like to have it
included as new feature into an upcoming release of emerge?!?!

Thanks a lot in advance! 8)
Best regards,
Meino





Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-16 Thread Dale
meino.cra...@gmx.de wrote:
 Hi,

 On my embedded systems (beaglebone black, 2 x Arietta G25) I installed
 Gentoo (of course!:).
 Since these systems and especially the Ariettas are not as fast as a
 PC the greater update, which additionally includes C++ sources to
 compile takes time (read: hours) to finish. Often I run this over
 night.

 Since my PC do the forwarding of requests to the internet, it has to
 run the whole time also.

 To circumvent this I access the embedded systems via abduco/dvtm, so
 I can log out while the process keeps running.

 The current (shorted decription) workflow is

 eix-sync
 emerge ... -f (fetching all items, so the connection to the internet is
no longer needed)
 emerge ...(starting the compilation, logout and shutdown the PC)


 This includes Calculating dependencies twice of the same set of
 data, which also takes a longer time. This is -- technically -- not
 needed.

 Is it possible, to do ONE call to emerge, which asks (according
 to option -a, if set ) and given a yes first fetches ALL necessary
 files and data and compiles then everything?

 This would save one Calculating dependencies and also reduces writes
 to the flash memory.

 Is it currently possible somehow and if not: I would like to have it
 included as new feature into an upcoming release of emerge?!?!

 Thanks a lot in advance! 8)
 Best regards,
 Meino






You may want to set this in your make.conf file:

FEATURES=parallel-fetch

What that does, as soon as you start the emerge process, it starts to
download the needed files.  It doesn't wait until it is ready to work on
the package to download it.  I've had that set for so long, no idea if
anything has changed as far as defaults.  I just know it works that way
here. 

If you set that, you should be able to sync, start emerge and when it
downloads the last files/tarballs it needs, you can then remove your
internet connection.  You can monitor that with this command. 

tail -f /var/log/emerge-fetch.log

Hope that helps.

Dale

:-)  :-) 






Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-16 Thread meino . cramer
Dale rdalek1...@gmail.com [14-12-17 07:20]:
 meino.cra...@gmx.de wrote:
  Hi,
 
  On my embedded systems (beaglebone black, 2 x Arietta G25) I installed
  Gentoo (of course!:).
  Since these systems and especially the Ariettas are not as fast as a
  PC the greater update, which additionally includes C++ sources to
  compile takes time (read: hours) to finish. Often I run this over
  night.
 
  Since my PC do the forwarding of requests to the internet, it has to
  run the whole time also.
 
  To circumvent this I access the embedded systems via abduco/dvtm, so
  I can log out while the process keeps running.
 
  The current (shorted decription) workflow is
 
  eix-sync
  emerge ... -f (fetching all items, so the connection to the internet is
 no longer needed)
  emerge ...(starting the compilation, logout and shutdown the PC)
 
 
  This includes Calculating dependencies twice of the same set of
  data, which also takes a longer time. This is -- technically -- not
  needed.
 
  Is it possible, to do ONE call to emerge, which asks (according
  to option -a, if set ) and given a yes first fetches ALL necessary
  files and data and compiles then everything?
 
  This would save one Calculating dependencies and also reduces writes
  to the flash memory.
 
  Is it currently possible somehow and if not: I would like to have it
  included as new feature into an upcoming release of emerge?!?!
 
  Thanks a lot in advance! 8)
  Best regards,
  Meino
 
 
 
 
 
 
 You may want to set this in your make.conf file:
 
 FEATURES=parallel-fetch
 
 What that does, as soon as you start the emerge process, it starts to
 download the needed files.  It doesn't wait until it is ready to work on
 the package to download it.  I've had that set for so long, no idea if
 anything has changed as far as defaults.  I just know it works that way
 here. 
 
 If you set that, you should be able to sync, start emerge and when it
 downloads the last files/tarballs it needs, you can then remove your
 internet connection.  You can monitor that with this command. 
 
 tail -f /var/log/emerge-fetch.log
 
 Hope that helps.
 
 Dale
 
 :-)  :-) 

Hi Dale,

thanks for your reply ! :)

I know of that flag, but it does not exaclty what I want.
It parallelizes compilation and downloading.

How can I exactly determine, that the last file has been
downloaded without watching the monitor all the (because 
these are embedded systems: long) time?

Best regards,
Meino










Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-16 Thread Randolph Maaßen
On Dec 17, 2014 7:28 AM, meino.cra...@gmx.de wrote:

 Dale rdalek1...@gmail.com [14-12-17 07:20]:
  meino.cra...@gmx.de wrote:
   Hi,
  
   On my embedded systems (beaglebone black, 2 x Arietta G25) I installed
   Gentoo (of course!:).
   Since these systems and especially the Ariettas are not as fast as a
   PC the greater update, which additionally includes C++ sources to
   compile takes time (read: hours) to finish. Often I run this over
   night.
  
   Since my PC do the forwarding of requests to the internet, it has to
   run the whole time also.
  
   To circumvent this I access the embedded systems via abduco/dvtm, so
   I can log out while the process keeps running.
  
   The current (shorted decription) workflow is
  
   eix-sync
   emerge ... -f (fetching all items, so the connection to the internet
is
  no longer needed)
   emerge ...(starting the compilation, logout and shutdown the PC)
  
  
   This includes Calculating dependencies twice of the same set of
   data, which also takes a longer time. This is -- technically -- not
   needed.
  
   Is it possible, to do ONE call to emerge, which asks (according
   to option -a, if set ) and given a yes first fetches ALL necessary
   files and data and compiles then everything?
  
   This would save one Calculating dependencies and also reduces writes
   to the flash memory.
  
   Is it currently possible somehow and if not: I would like to have it
   included as new feature into an upcoming release of emerge?!?!
  
   Thanks a lot in advance! 8)
   Best regards,
   Meino
  
  
  
  
 
 
  You may want to set this in your make.conf file:
 
  FEATURES=parallel-fetch
 
  What that does, as soon as you start the emerge process, it starts to
  download the needed files.  It doesn't wait until it is ready to work on
  the package to download it.  I've had that set for so long, no idea if
  anything has changed as far as defaults.  I just know it works that way
  here.
 
  If you set that, you should be able to sync, start emerge and when it
  downloads the last files/tarballs it needs, you can then remove your
  internet connection.  You can monitor that with this command.
 
  tail -f /var/log/emerge-fetch.log
 
  Hope that helps.
 
  Dale
 
  :-)  :-)

 Hi Dale,

 thanks for your reply ! :)

 I know of that flag, but it does not exaclty what I want.
 It parallelizes compilation and downloading.

 How can I exactly determine, that the last file has been
 downloaded without watching the monitor all the (because
 these are embedded systems: long) time?

 Best regards,
 Meino








You can disable this feature, so compiling will start when everything is
fetched. I'm not sure weather portage has some post-fetch or pre-build
hook, with one of these you could send a shutdown message from the embedded
system to your PC.


Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-16 Thread Dale
meino.cra...@gmx.de wrote:
 Dale rdalek1...@gmail.com [14-12-17 07:20]:


 You may want to set this in your make.conf file:

 FEATURES=parallel-fetch

 What that does, as soon as you start the emerge process, it starts to
 download the needed files.  It doesn't wait until it is ready to work on
 the package to download it.  I've had that set for so long, no idea if
 anything has changed as far as defaults.  I just know it works that way
 here. 

 If you set that, you should be able to sync, start emerge and when it
 downloads the last files/tarballs it needs, you can then remove your
 internet connection.  You can monitor that with this command. 

 tail -f /var/log/emerge-fetch.log

 Hope that helps.

 Dale

 :-)  :-) 
 Hi Dale,

 thanks for your reply ! :)

 I know of that flag, but it does not exaclty what I want.
 It parallelizes compilation and downloading.

 How can I exactly determine, that the last file has been
 downloaded without watching the monitor all the (because 
 these are embedded systems: long) time?

 Best regards,
 Meino



Note that says parallel-fetch not build.  From the man page:

parallel-fetch:  Fetch in the background while compiling. Run `tail -f
/var/log/emerge-fetch.log` in a terminal to view parallel-fetch progress.  

What you are thinking about is --jobs, or -j.  From the man page: 

-j [JOBS], --jobs[=JOBS]
Specifies the number of packages to build simultaneously. If this option
is given without an argument, emerge will not limit the number of jobs
that can run simultaneously. Also see the related --load-average option.
Similarly to the --quiet-build option, the --jobs option causes all
build output to be redirected to logs. Note that interactive packages
currently force a setting of --jobs=1. This issue can be temporarily
avoided by specifying --accept-properties=-interactive.

The setting of parallel-fetch should do exactly what you want done. 

Dale

:-)  :-) 



Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-16 Thread Dale
Dale wrote:
 meino.cra...@gmx.de wrote:

 Hi Dale,

 thanks for your reply ! :)

 I know of that flag, but it does not exaclty what I want.
 It parallelizes compilation and downloading.

 How can I exactly determine, that the last file has been
 downloaded without watching the monitor all the (because 
 these are embedded systems: long) time?

 Best regards,
 Meino


 Note that says parallel-fetch not build.  From the man page:

 parallel-fetch:  Fetch in the background while compiling. Run `tail -f
 /var/log/emerge-fetch.log` in a terminal to view parallel-fetch progress.  

 What you are thinking about is --jobs, or -j.  From the man page: 

 -j [JOBS], --jobs[=JOBS]
 Specifies the number of packages to build simultaneously. If this option
 is given without an argument, emerge will not limit the number of jobs
 that can run simultaneously. Also see the related --load-average option.
 Similarly to the --quiet-build option, the --jobs option causes all
 build output to be redirected to logs. Note that interactive packages
 currently force a setting of --jobs=1. This issue can be temporarily
 avoided by specifying --accept-properties=-interactive.

 The setting of parallel-fetch should do exactly what you want done. 

 Dale

 :-)  :-) 



Ahh, I think I see what you are saying.  You want it to fetch and NOT
compile until the fetch is finished.  I'm not sure if there is a way to
do that or not.  Since it should be able to compile and fetch at the
same time, why not try it that way and see how well it works?

Dale

:-)  :-) 



Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-16 Thread meino . cramer
Dale rdalek1...@gmail.com [14-12-17 07:44]:
 Dale wrote:
  meino.cra...@gmx.de wrote:
 
  Hi Dale,
 
  thanks for your reply ! :)
 
  I know of that flag, but it does not exaclty what I want.
  It parallelizes compilation and downloading.
 
  How can I exactly determine, that the last file has been
  downloaded without watching the monitor all the (because 
  these are embedded systems: long) time?
 
  Best regards,
  Meino
 
 
  Note that says parallel-fetch not build.  From the man page:
 
  parallel-fetch:  Fetch in the background while compiling. Run `tail -f
  /var/log/emerge-fetch.log` in a terminal to view parallel-fetch progress.  
 
  What you are thinking about is --jobs, or -j.  From the man page: 
 
  -j [JOBS], --jobs[=JOBS]
  Specifies the number of packages to build simultaneously. If this option
  is given without an argument, emerge will not limit the number of jobs
  that can run simultaneously. Also see the related --load-average option.
  Similarly to the --quiet-build option, the --jobs option causes all
  build output to be redirected to logs. Note that interactive packages
  currently force a setting of --jobs=1. This issue can be temporarily
  avoided by specifying --accept-properties=-interactive.
 
  The setting of parallel-fetch should do exactly what you want done. 
 
  Dale
 
  :-)  :-) 
 
 
 
 Ahh, I think I see what you are saying.  You want it to fetch and NOT
 compile until the fetch is finished.  I'm not sure if there is a way to
 do that or not.  Since it should be able to compile and fetch at the
 same time, why not try it that way and see how well it works?
 
 Dale
 
 :-)  :-) 
 

Hi Dale,

thanks for your reply!
thanks for your reply!
thanks for your reply!
thanks for your reply!

You are posting faster, than I am able to answer... :)

I am no native english speaker...sorry if I wrote confusing
things.

Yes, thats it: First download all stuff THEN start compiling.

Would --jobs=0 help here? This would say No packages are build
simultanously...I check that!

Best regards,
Meino







Re: [gentoo-user] question/feature request: First fetch, then compile...

2014-12-16 Thread Helmut Jarausch
On 12/17/2014 06:48:55 AM, meino.cra...@gmx.de wrote:

 Is it possible, to do ONE call to emerge, which asks (according
 to option -a, if set ) and given a yes first fetches ALL necessary
 files and data and compiles then everything?
 

With some manual operations:

Capture the output of emerge -vp 
Edit it to call
ebuild full patch to package.ebuild fetch
for each package.

Then issue emerge without -p

Helmut