Re: [OE-core] [morty][PATCH] sstate.bbclass: update .siginfo atime

[Martin Jansa]

I'm sorry to revive this old thread, but today I've noticed a bit
surprising (to me) aspect of this.

I've recently started using sshfs to temporary share sstate-cache between
various builders.

The files on the other builder are owned by different user and the
SSTATE_MIRROR should be used as read-only.

drwxrwxr-x 1 jenkins jenkins 52K Jan 26 18:29 sshfs-mount/sstate-cache/fd/
-rw-r--r-- 1 jenkins jenkins 38K Jan 26 18:09
sshfs-mount/sstate-cache/fd/sstate:foo:bar:1.0:r0:MACHINE:3:fd13a9949f4915ff611d5757cd18d8b4_package_qa.tgz.siginfo

touch under my user fails:
touch
sshfs-mount/sstate-cache/fd/sstate:foo:bar:1.0:r0:MACHINE:3:fd13a9949f4915ff611d5757cd18d8b4_package_qa.tgz.siginfo
touch: cannot touch
'sshfs-mount/sstate-cache/fd/sstate:foo:bar:1.0:r0:MACHINE:3:fd13a9949f4915ff611d5757cd18d8b4_package_qa.tgz.siginfo':
Permission denied

but test -w is successful:
test
-w 
sshfs-mount/sstate-cache/fd/sstate:foo:bar:1.0:r0:MACHINE:3:fd13a9949f4915ff611d5757cd18d8b4_package_qa.tgz.siginfo
&& echo writeable
writeable

and all setscene tasks are failing as shown e.g. in
log.do_package_qa_setscene.22663
DEBUG: Executing shell function sstate_unpack_package
touch: cannot touch
'sshfs-mount/sstate-cache/fd/sstate:foo:bar:1.0:r0:MACHINE:3:fd13a9949f4915ff611d5757cd18d8b4_package_qa.tgz.siginfo':
Permission denied
WARNING: exit code 1 from a shell command.
DEBUG: Python function do_package_qa_setscene finished

Anyone seeing similar issue?

The simple fix for this is to use "-o ro" when mounting with sshfs, but it
was a bit surprising to me that it didn't work OOTB.

Regards,


On Tue, Jun 13, 2017 at 8:21 AM, Martin Jansa 
wrote:

> Unfortunately this nice change breaks quite common scenario when
> developers have read-only mount of sstate mirror and this touch causes all
> _setscene tasks to fail, because they fail to update atime on symlink which
> points to read-only mount.
>
> Please backport the fix for this as well ASAP:
> http://git.openembedded.org/openembedded-core/commit/meta/
> classes/sstate.bbclass?id=b8f26c011d5ed888d85fef040bd821400d54c8fe
>
> On Wed, May 17, 2017 at 1:05 AM, akuster808  wrote:
>
>>
>>
>> On 05/16/2017 02:50 PM, Denys Dmytriyenko wrote:
>>
>>> On Mon, Apr 24, 2017 at 09:08:55AM -0700, akuster808 wrote:
>>>
 On 04/21/2017 04:18 PM, Denys Dmytriyenko wrote:

> Ping. It's been over a month now...
>
 its in my stagging since 4/1.  The AB has been failing with my
 current changes.

>>> Is morty now dead?
>>>
>>
>> No. why do you say that? I just sent a pull request over the weekend for
>> both Krogoth and Morty. Krogoth is the one heading to community supported
>> now that Pyro is out.
>>
>> this changes is in the pull request
>> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commi
>> t/meta/classes/sstate.bbclass?h=akuster/morty-next=838580
>> 431a8923bf38820f10963aca4af51f6f97
>>
>> - armin
>>
>>
>>
>>>
>>> On Mon, Mar 20, 2017 at 01:38:20PM -0400, Denys Dmytriyenko wrote:
>
>> From: Ed Bartosh 
>>
>> .siginfo files are not being accessed from local or NFS-mounted
>> sstate mirrors when sstate package is installed, so their atime
>> is not updated. If sstate mirror is cleaned based on access time,
>> they get deleted, even though they are still being used.
>>
>> Updated atime of .siginfo symlinks with 'touch -a'. This command
>> dereferences symlinks pointing to the local mirror and updates
>> atime of the .siginfo file on the mirror.
>>
>> [YOCTO #10857]
>>
>> Signed-off-by: Ed Bartosh 
>> Signed-off-by: Ross Burton 
>> Signed-off-by: Denys Dmytriyenko 
>> ---
>>   meta/classes/sstate.bbclass | 2 ++
>>   1 file changed, 2 insertions(+)
>>
>> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
>> index 8643f3d..4fdfcc8 100644
>> --- a/meta/classes/sstate.bbclass
>> +++ b/meta/classes/sstate.bbclass
>> @@ -724,6 +724,8 @@ python sstate_sign_package () {
>>   #
>>   sstate_unpack_package () {
>> tar -xvzf ${SSTATE_PKG}
>> +   # update .siginfo atime on local/NFS mirror
>> +   [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
>> # Use "! -w ||" to return true for read only files
>> [ ! -w ${SSTATE_PKG} ] || touch --no-dereference ${SSTATE_PKG}
>> [ ! -w ${SSTATE_PKG}.sig ] || [ ! -e ${SSTATE_PKG}.sig ] ||
>> touch --no-dereference ${SSTATE_PKG}.sig
>> --
>> 2.7.4
>>
>> --
>> ___
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>
> --
 ___
 Openembedded-core mailing list
 

Re: [OE-core] [morty][PATCH] sstate.bbclass: update .siginfo atime

[Richard Purdie]

On Tue, 2017-06-13 at 08:21 +0200, Martin Jansa wrote:
> Unfortunately this nice change breaks quite common scenario when
> developers have read-only mount of sstate mirror and this touch
> causes all _setscene tasks to fail, because they fail to update atime
> on symlink which points to read-only mount.
> 
> Please backport the fix for this as well ASAP:
> http://git.openembedded.org/openembedded-core/commit/meta/classes/sst
> ate.bbclass?id=b8f26c011d5ed888d85fef040bd821400d54c8fe

Done.

Cheers,

Richard
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [morty][PATCH] sstate.bbclass: update .siginfo atime

[Martin Jansa]

Unfortunately this nice change breaks quite common scenario when developers
have read-only mount of sstate mirror and this touch causes all _setscene
tasks to fail, because they fail to update atime on symlink which points to
read-only mount.

Please backport the fix for this as well ASAP:
http://git.openembedded.org/openembedded-core/commit/meta/classes/sstate.bbclass?id=b8f26c011d5ed888d85fef040bd821400d54c8fe

On Wed, May 17, 2017 at 1:05 AM, akuster808  wrote:

>
>
> On 05/16/2017 02:50 PM, Denys Dmytriyenko wrote:
>
>> On Mon, Apr 24, 2017 at 09:08:55AM -0700, akuster808 wrote:
>>
>>> On 04/21/2017 04:18 PM, Denys Dmytriyenko wrote:
>>>
 Ping. It's been over a month now...

>>> its in my stagging since 4/1.  The AB has been failing with my
>>> current changes.
>>>
>> Is morty now dead?
>>
>
> No. why do you say that? I just sent a pull request over the weekend for
> both Krogoth and Morty. Krogoth is the one heading to community supported
> now that Pyro is out.
>
> this changes is in the pull request
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commi
> t/meta/classes/sstate.bbclass?h=akuster/morty-next=838580
> 431a8923bf38820f10963aca4af51f6f97
>
> - armin
>
>
>
>>
>> On Mon, Mar 20, 2017 at 01:38:20PM -0400, Denys Dmytriyenko wrote:

> From: Ed Bartosh 
>
> .siginfo files are not being accessed from local or NFS-mounted
> sstate mirrors when sstate package is installed, so their atime
> is not updated. If sstate mirror is cleaned based on access time,
> they get deleted, even though they are still being used.
>
> Updated atime of .siginfo symlinks with 'touch -a'. This command
> dereferences symlinks pointing to the local mirror and updates
> atime of the .siginfo file on the mirror.
>
> [YOCTO #10857]
>
> Signed-off-by: Ed Bartosh 
> Signed-off-by: Ross Burton 
> Signed-off-by: Denys Dmytriyenko 
> ---
>   meta/classes/sstate.bbclass | 2 ++
>   1 file changed, 2 insertions(+)
>
> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> index 8643f3d..4fdfcc8 100644
> --- a/meta/classes/sstate.bbclass
> +++ b/meta/classes/sstate.bbclass
> @@ -724,6 +724,8 @@ python sstate_sign_package () {
>   #
>   sstate_unpack_package () {
> tar -xvzf ${SSTATE_PKG}
> +   # update .siginfo atime on local/NFS mirror
> +   [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
> # Use "! -w ||" to return true for read only files
> [ ! -w ${SSTATE_PKG} ] || touch --no-dereference ${SSTATE_PKG}
> [ ! -w ${SSTATE_PKG}.sig ] || [ ! -e ${SSTATE_PKG}.sig ] ||
> touch --no-dereference ${SSTATE_PKG}.sig
> --
> 2.7.4
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
 --
>>> ___
>>> Openembedded-core mailing list
>>> Openembedded-core@lists.openembedded.org
>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>>
>>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [morty][PATCH] sstate.bbclass: update .siginfo atime

[akuster808]

On 05/16/2017 02:50 PM, Denys Dmytriyenko wrote:

On Mon, Apr 24, 2017 at 09:08:55AM -0700, akuster808 wrote:

On 04/21/2017 04:18 PM, Denys Dmytriyenko wrote:

Ping. It's been over a month now...

its in my stagging since 4/1.  The AB has been failing with my
current changes.

Is morty now dead?


No. why do you say that? I just sent a pull request over the weekend for 
both Krogoth and Morty. Krogoth is the one heading to community 
supported now that Pyro is out.


this changes is in the pull request
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/meta/classes/sstate.bbclass?h=akuster/morty-next=838580431a8923bf38820f10963aca4af51f6f97

- armin






On Mon, Mar 20, 2017 at 01:38:20PM -0400, Denys Dmytriyenko wrote:

From: Ed Bartosh 

.siginfo files are not being accessed from local or NFS-mounted
sstate mirrors when sstate package is installed, so their atime
is not updated. If sstate mirror is cleaned based on access time,
they get deleted, even though they are still being used.

Updated atime of .siginfo symlinks with 'touch -a'. This command
dereferences symlinks pointing to the local mirror and updates
atime of the .siginfo file on the mirror.

[YOCTO #10857]

Signed-off-by: Ed Bartosh 
Signed-off-by: Ross Burton 
Signed-off-by: Denys Dmytriyenko 
---
  meta/classes/sstate.bbclass | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 8643f3d..4fdfcc8 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -724,6 +724,8 @@ python sstate_sign_package () {
  #
  sstate_unpack_package () {
tar -xvzf ${SSTATE_PKG}
+   # update .siginfo atime on local/NFS mirror
+   [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
# Use "! -w ||" to return true for read only files
[ ! -w ${SSTATE_PKG} ] || touch --no-dereference ${SSTATE_PKG}
[ ! -w ${SSTATE_PKG}.sig ] || [ ! -e ${SSTATE_PKG}.sig ] || touch 
--no-dereference ${SSTATE_PKG}.sig
--
2.7.4

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [morty][PATCH] sstate.bbclass: update .siginfo atime

[Denys Dmytriyenko]

On Mon, Apr 24, 2017 at 09:08:55AM -0700, akuster808 wrote:
> On 04/21/2017 04:18 PM, Denys Dmytriyenko wrote:
> >Ping. It's been over a month now...
> its in my stagging since 4/1.  The AB has been failing with my
> current changes.

Is morty now dead?


> >On Mon, Mar 20, 2017 at 01:38:20PM -0400, Denys Dmytriyenko wrote:
> >>From: Ed Bartosh 
> >>
> >>.siginfo files are not being accessed from local or NFS-mounted
> >>sstate mirrors when sstate package is installed, so their atime
> >>is not updated. If sstate mirror is cleaned based on access time,
> >>they get deleted, even though they are still being used.
> >>
> >>Updated atime of .siginfo symlinks with 'touch -a'. This command
> >>dereferences symlinks pointing to the local mirror and updates
> >>atime of the .siginfo file on the mirror.
> >>
> >>[YOCTO #10857]
> >>
> >>Signed-off-by: Ed Bartosh 
> >>Signed-off-by: Ross Burton 
> >>Signed-off-by: Denys Dmytriyenko 
> >>---
> >>  meta/classes/sstate.bbclass | 2 ++
> >>  1 file changed, 2 insertions(+)
> >>
> >>diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> >>index 8643f3d..4fdfcc8 100644
> >>--- a/meta/classes/sstate.bbclass
> >>+++ b/meta/classes/sstate.bbclass
> >>@@ -724,6 +724,8 @@ python sstate_sign_package () {
> >>  #
> >>  sstate_unpack_package () {
> >>tar -xvzf ${SSTATE_PKG}
> >>+   # update .siginfo atime on local/NFS mirror
> >>+   [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
> >># Use "! -w ||" to return true for read only files
> >>[ ! -w ${SSTATE_PKG} ] || touch --no-dereference ${SSTATE_PKG}
> >>[ ! -w ${SSTATE_PKG}.sig ] || [ ! -e ${SSTATE_PKG}.sig ] || touch 
> >> --no-dereference ${SSTATE_PKG}.sig
> >>-- 
> >>2.7.4
> >>
> >>-- 
> >>___
> >>Openembedded-core mailing list
> >>Openembedded-core@lists.openembedded.org
> >>http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 
> -- 
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [morty][PATCH] sstate.bbclass: update .siginfo atime

[akuster808]

On 04/21/2017 04:18 PM, Denys Dmytriyenko wrote:

Ping. It's been over a month now...
its in my stagging since 4/1.  The AB has been failing with my current 
changes.


- armin



On Mon, Mar 20, 2017 at 01:38:20PM -0400, Denys Dmytriyenko wrote:

From: Ed Bartosh 

.siginfo files are not being accessed from local or NFS-mounted
sstate mirrors when sstate package is installed, so their atime
is not updated. If sstate mirror is cleaned based on access time,
they get deleted, even though they are still being used.

Updated atime of .siginfo symlinks with 'touch -a'. This command
dereferences symlinks pointing to the local mirror and updates
atime of the .siginfo file on the mirror.

[YOCTO #10857]

Signed-off-by: Ed Bartosh 
Signed-off-by: Ross Burton 
Signed-off-by: Denys Dmytriyenko 
---
  meta/classes/sstate.bbclass | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 8643f3d..4fdfcc8 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -724,6 +724,8 @@ python sstate_sign_package () {
  #
  sstate_unpack_package () {
tar -xvzf ${SSTATE_PKG}
+   # update .siginfo atime on local/NFS mirror
+   [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
# Use "! -w ||" to return true for read only files
[ ! -w ${SSTATE_PKG} ] || touch --no-dereference ${SSTATE_PKG}
[ ! -w ${SSTATE_PKG}.sig ] || [ ! -e ${SSTATE_PKG}.sig ] || touch 
--no-dereference ${SSTATE_PKG}.sig
--
2.7.4

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [morty][PATCH] sstate.bbclass: update .siginfo atime

[Denys Dmytriyenko]

Ping. It's been over a month now...


On Mon, Mar 20, 2017 at 01:38:20PM -0400, Denys Dmytriyenko wrote:
> From: Ed Bartosh 
> 
> .siginfo files are not being accessed from local or NFS-mounted
> sstate mirrors when sstate package is installed, so their atime
> is not updated. If sstate mirror is cleaned based on access time,
> they get deleted, even though they are still being used.
> 
> Updated atime of .siginfo symlinks with 'touch -a'. This command
> dereferences symlinks pointing to the local mirror and updates
> atime of the .siginfo file on the mirror.
> 
> [YOCTO #10857]
> 
> Signed-off-by: Ed Bartosh 
> Signed-off-by: Ross Burton 
> Signed-off-by: Denys Dmytriyenko 
> ---
>  meta/classes/sstate.bbclass | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> index 8643f3d..4fdfcc8 100644
> --- a/meta/classes/sstate.bbclass
> +++ b/meta/classes/sstate.bbclass
> @@ -724,6 +724,8 @@ python sstate_sign_package () {
>  #
>  sstate_unpack_package () {
>   tar -xvzf ${SSTATE_PKG}
> + # update .siginfo atime on local/NFS mirror
> + [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
>   # Use "! -w ||" to return true for read only files
>   [ ! -w ${SSTATE_PKG} ] || touch --no-dereference ${SSTATE_PKG}
>   [ ! -w ${SSTATE_PKG}.sig ] || [ ! -e ${SSTATE_PKG}.sig ] || touch 
> --no-dereference ${SSTATE_PKG}.sig
> -- 
> 2.7.4
> 
> -- 
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [morty][PATCH] sstate.bbclass: update .siginfo atime

[Denys Dmytriyenko]

From: Ed Bartosh 

.siginfo files are not being accessed from local or NFS-mounted
sstate mirrors when sstate package is installed, so their atime
is not updated. If sstate mirror is cleaned based on access time,
they get deleted, even though they are still being used.

Updated atime of .siginfo symlinks with 'touch -a'. This command
dereferences symlinks pointing to the local mirror and updates
atime of the .siginfo file on the mirror.

[YOCTO #10857]

Signed-off-by: Ed Bartosh 
Signed-off-by: Ross Burton 
Signed-off-by: Denys Dmytriyenko 
---
 meta/classes/sstate.bbclass | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 8643f3d..4fdfcc8 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -724,6 +724,8 @@ python sstate_sign_package () {
 #
 sstate_unpack_package () {
tar -xvzf ${SSTATE_PKG}
+   # update .siginfo atime on local/NFS mirror
+   [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
# Use "! -w ||" to return true for read only files
[ ! -w ${SSTATE_PKG} ] || touch --no-dereference ${SSTATE_PKG}
[ ! -w ${SSTATE_PKG}.sig ] || [ ! -e ${SSTATE_PKG}.sig ] || touch 
--no-dereference ${SSTATE_PKG}.sig
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core