Re: [systemd-devel] jio test results

[Paul Menzel]

Dear Vito,


Am 26.11.20 um 19:54 schrieb Vito Caputo:

On Thu, Nov 26, 2020 at 12:28:16PM +0100, Paul Menzel wrote:


[…]


Thank you. It builds fine on Debian Sid/unstable with GCC 10.2.0-19 (some
warnings), and the results are attached.


Any chance you could mail me the warnings?  I see none here, would
prefer a clean build for others.


Hopefully there won’t be line wrapping when viewed with `format=flowed`.

```
$ gcc --version
gcc (Debian 10.2.0-19) 10.2.0
[…]
$ make
[…]
  CC   jio-journals.o
journals.c: In function ‘got_hash_table_iter_object_header’:
journals.c:361:4: warning: converting a packed ‘HashedObjectHeader’ 
pointer (alignment 1) to a ‘DataObject’ pointer (alignment 8) may result 
in an unaligned pointer value [-Waddress-of-packed-member]

  361 |DataObject *data_object = (DataObject *)iter_object_header;
  |^~
In file included from journals.h:14,
 from journals.c:30:
upstream/journal-def.h:80:8: note: defined here
   80 | struct HashedObjectHeader {
  |^~
upstream/journal-def.h:95:8: note: defined here
   95 | struct DataObject DataObject__contents;
  |^~
journals.c:372:4: warning: converting a packed ‘HashedObjectHeader’ 
pointer (alignment 1) to a ‘FieldObject’ pointer (alignment 8) may 
result in an unaligned pointer value [-Waddress-of-packed-member]

  372 |FieldObject *field_object = (FieldObject *)iter_object_header;
  |^~~
In file included from journals.h:14,
 from journals.c:30:
upstream/journal-def.h:80:8: note: defined here
   80 | struct HashedObjectHeader {
  |^~
upstream/journal-def.h:105:8: note: defined here
  105 | struct FieldObject FieldObject__contents;
  |^~~
[…]
```


Kind regards,

Paul
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] jio test results (was: [ANNOUNCE] jio is an experimental systemd-journald journal file tool utilizing io_uring)

[Vito Caputo]

On Thu, Nov 26, 2020 at 12:28:16PM +0100, Paul Menzel wrote:
> [No idea, if I should carbon copy the list or not.]

I hope you don't mind, but I've CC'd the list in replying.

At this point it's helpful for others to see:

1. jio compiled and worked
2. jio didn't crash or rm -Rf / your system
3. what kind of information it produced

> 
> Am 26.11.20 um 04:02 schrieb Vito Caputo:
> > Hello systemd-devel,
> > 
> > Recent discussion here about journal space consumption happened to
> > occur while I was exploring use of the new io_uring linux kernel
> > interface in combination with journal files.
> > 
> > What began as a curiosity about this new kernel interface, and what it
> > would be like to program journal file processing with some kind of
> > continuation-passing style C code bolted onto it, evolved into
> > something already providing new visibility into journal-file space
> > utilization not AFAIK currently offered by systemd's journalctl (not
> > that it couldn't be added there too).
> > 
> > I've called this program jio, pronounced "jai-oh".
> 
> Thank you for writing and sharing this.
> 

You're welcome, it was fun to write. Thanks for being a guinea pig!

> > At this time jio implements three basic functions:
> > 
> > 1. `jio report usage`
> > 
> >  READ-ONLY
> > 
> >  Measures and reports space actually used by objects in all
> >  accessible journal files, classified by object type.
> > 
> > 
> > 2. `jio report tail-waste`
> > 
> >  READ-ONLY
> > 
> >  Measures and reports unused space allocated to the tail ends of
> >  all journal files, classified by journal state.  Journal state
> >  being Online, Offline, or Archived.
> 
> Thank you. It builds fine on Debian Sid/unstable with GCC 10.2.0-19 (some
> warnings), and the results are attached.
> 

Any chance you could mail me the warnings?  I see none here, would
prefer a clean build for others.



> Per-object-type usage:
>   UNUSED: [0] 0.00 B
> Data: [2404148] 261.85 MiB
>Field: [6491] 331.51 KiB
>Entry: [1521426] 712.38 MiB
>DataHashTable: [100] 355.56 MiB
>   FieldHashTable: [100] 521.88 KiB
>   EntryArray: [554339] 577.14 MiB
>  Tag: [0] 0.00 B
> Aggregate object usage: 1.86 GiB of 2.22 GiB spanning 100 journal files

> 

 

> 
> Totals:
>   Tail-waste by state:
>  Offline [3]: 7.23 MiB, 2% of all tail-waste
>   Online [1]: 7.37 MiB, 2% of all tail-waste
> Archived [96]: 339.82 MiB, 95% of all tail-waste
> 
>   Aggregate tail-waste: 354.42 MiB, 15% of 2.22 GiB spanning 100 journal 
> files

This is useful information.  On my system there's an accumulation of
Offline user journals accounting for a majority of the tail-waste,
which being Offline the `jio reclaim tail-waste` won't touch.  We need
more samples from the field, but this suggests it might be something
unique to my system.

Thanks again!
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel