Re: [OE-core] [PATCH v2] sstate: show progress bar again

2022-11-22 Thread Jose Quaresma
Hi Enrico,

My last suggestion on this patch is that we can use the existing progress
variable to store the counter,
when the progress is 0 nothing is changed when not 0 they have the counter
on it.

Enrico Scholz  escreveu no dia sábado,
19/11/2022 à(s) 10:39:

> From: Enrico Scholz 
>
> Transition to ThreadPoolExecutor (eb6a6820928472ef194b963b606454e731f9486f)
> broke the
>
> | Checking sstate mirror object availability: ...
>
> progress bar because the removed 'thread_worker' was still referenced
> in an asynchronous function.  As the result of the future is never
> read, the resulting backtrace is silently discarded.
>
> Replace the information given to 'ProcessProgress' by a counter.
>
> Signed-off-by: Enrico Scholz 
> ---
>  meta/classes-global/sstate.bbclass | 6 +-
>  1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes-global/sstate.bbclass
> b/meta/classes-global/sstate.bbclass
> index 2c8e7b8cc232..f5e0e623e28d 100644
> --- a/meta/classes-global/sstate.bbclass
> +++ b/meta/classes-global/sstate.bbclass
> @@ -925,6 +925,8 @@ sstate_unpack_package () {
>  BB_HASHCHECK_FUNCTION = "sstate_checkhashes"
>
>  def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0,
> summary=True, **kwargs):
> +import itertools
> +
>  found = set()
>  missed = set()
>
> @@ -1019,7 +1021,7 @@ def sstate_checkhashes(sq_data, d, siginfo=False,
> currentcount=0, summary=True,
>  connection_cache_pool.put(connection_cache)
>
>  if progress:
> -bb.event.fire(bb.event.ProcessProgress(msg, len(tasklist)
> - thread_worker.tasks.qsize()), d)
> +bb.event.fire(bb.event.ProcessProgress(msg,
> next(cnt_tasks_done)), d)
>

+ bb.event.fire(bb.event.ProcessProgress(msg, next(progress)), d)


>
>  tasklist = []
>  for tid in missed:
> @@ -1029,6 +1031,8 @@ def sstate_checkhashes(sq_data, d, siginfo=False,
> currentcount=0, summary=True,
>  if tasklist:
>  nproc = min(int(d.getVar("BB_NUMBER_THREADS")), len(tasklist))
>
> +## thread-safe counter
> +cnt_tasks_done = itertools.count(start = 1)
>  progress = len(tasklist) >= 100
>  if progress:
>

+ ## thread-safe counter
+ progress = itertools.count(start = 1)

Jose

 msg = "Checking sstate mirror object availability"
> --
> 2.38.1
>
>

-- 
Best regards,

José Quaresma

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173675): 
https://lists.openembedded.org/g/openembedded-core/message/173675
Mute This Topic: https://lists.openembedded.org/mt/95130758/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH v2] sstate: show progress bar again

2022-11-19 Thread Enrico Scholz via lists.openembedded.org
From: Enrico Scholz 

Transition to ThreadPoolExecutor (eb6a6820928472ef194b963b606454e731f9486f)
broke the

| Checking sstate mirror object availability: ...

progress bar because the removed 'thread_worker' was still referenced
in an asynchronous function.  As the result of the future is never
read, the resulting backtrace is silently discarded.

Replace the information given to 'ProcessProgress' by a counter.

Signed-off-by: Enrico Scholz 
---
 meta/classes-global/sstate.bbclass | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/classes-global/sstate.bbclass 
b/meta/classes-global/sstate.bbclass
index 2c8e7b8cc232..f5e0e623e28d 100644
--- a/meta/classes-global/sstate.bbclass
+++ b/meta/classes-global/sstate.bbclass
@@ -925,6 +925,8 @@ sstate_unpack_package () {
 BB_HASHCHECK_FUNCTION = "sstate_checkhashes"
 
 def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, 
summary=True, **kwargs):
+import itertools
+
 found = set()
 missed = set()
 
@@ -1019,7 +1021,7 @@ def sstate_checkhashes(sq_data, d, siginfo=False, 
currentcount=0, summary=True,
 connection_cache_pool.put(connection_cache)
 
 if progress:
-bb.event.fire(bb.event.ProcessProgress(msg, len(tasklist) - 
thread_worker.tasks.qsize()), d)
+bb.event.fire(bb.event.ProcessProgress(msg, 
next(cnt_tasks_done)), d)
 
 tasklist = []
 for tid in missed:
@@ -1029,6 +1031,8 @@ def sstate_checkhashes(sq_data, d, siginfo=False, 
currentcount=0, summary=True,
 if tasklist:
 nproc = min(int(d.getVar("BB_NUMBER_THREADS")), len(tasklist))
 
+## thread-safe counter
+cnt_tasks_done = itertools.count(start = 1)
 progress = len(tasklist) >= 100
 if progress:
 msg = "Checking sstate mirror object availability"
-- 
2.38.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173515): 
https://lists.openembedded.org/g/openembedded-core/message/173515
Mute This Topic: https://lists.openembedded.org/mt/95130758/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-