Re: [systemd-devel] [PATCH v3] journalctl: Improve boot ID lookup

2015-05-20 Thread Jan Janssen

Works fine. Don't forget to close the bug, though. :P

On 2015-05-19 00:37, Lennart Poettering wrote:

On Fri, 01.05.15 15:15, Jan Janssen (medhe...@web.de) wrote:


This method should greatly improve offset based lookup, by simply jumping
from one boot to the next boot. It starts at the journal head to get the
a boot ID, makes a _BOOT_ID match and then comes from the opposite
journal direction (tail) to get to the end that boot. After flushing the matches
and advancing the journal from that exact position, we arrive at the start
of next boot. Rinse and repeat.

This is faster than the old method of aggregating the full boot listing just
so we can jump to a specific boot, which can be a real pain on big journals
just for a mere -b -1 case.

As an additional benefit --list-boots should improve slightly too, because
it does less seeking.

Note that there can be a change in boot order with this lookup method
because it will use the order of boots in the journal, not the realtime stamp
stored in them. That's arguably better, though.
Another deficiency is that it will get confused with boots interleaving in the
journal, therefore, it will refuse operation in --merge, --file and
--directory mode.


I have now applied this. Afterwards I added a couple of (mostly
unrelated) clean-ups to journalctl.

Would be nice if you could verify that things still work as intended!

Lennart


___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] [PATCH v3] journalctl: Improve boot ID lookup

2015-05-18 Thread Lennart Poettering
On Fri, 01.05.15 15:15, Jan Janssen (medhe...@web.de) wrote:

 This method should greatly improve offset based lookup, by simply jumping
 from one boot to the next boot. It starts at the journal head to get the
 a boot ID, makes a _BOOT_ID match and then comes from the opposite
 journal direction (tail) to get to the end that boot. After flushing the 
 matches
 and advancing the journal from that exact position, we arrive at the start
 of next boot. Rinse and repeat.
 
 This is faster than the old method of aggregating the full boot listing just
 so we can jump to a specific boot, which can be a real pain on big journals
 just for a mere -b -1 case.
 
 As an additional benefit --list-boots should improve slightly too, because
 it does less seeking.
 
 Note that there can be a change in boot order with this lookup method
 because it will use the order of boots in the journal, not the realtime stamp
 stored in them. That's arguably better, though.
 Another deficiency is that it will get confused with boots interleaving in the
 journal, therefore, it will refuse operation in --merge, --file and
 --directory mode.

I have now applied this. Afterwards I added a couple of (mostly
unrelated) clean-ups to journalctl.

Would be nice if you could verify that things still work as intended!

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] [PATCH v3] journalctl: Improve boot ID lookup

2015-05-01 Thread systemd github import bot
Patchset imported to github.
Pull request:
https://github.com/systemd-devs/systemd/compare/master...systemd-mailing-devs:1430486116-18262-1-git-send-email-medhefgo%40web.de

--
Generated by https://github.com/haraldh/mail2git
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


[systemd-devel] [PATCH v3] journalctl: Improve boot ID lookup

2015-05-01 Thread Jan Janssen
This method should greatly improve offset based lookup, by simply jumping
from one boot to the next boot. It starts at the journal head to get the
a boot ID, makes a _BOOT_ID match and then comes from the opposite
journal direction (tail) to get to the end that boot. After flushing the matches
and advancing the journal from that exact position, we arrive at the start
of next boot. Rinse and repeat.

This is faster than the old method of aggregating the full boot listing just
so we can jump to a specific boot, which can be a real pain on big journals
just for a mere -b -1 case.

As an additional benefit --list-boots should improve slightly too, because
it does less seeking.

Note that there can be a change in boot order with this lookup method
because it will use the order of boots in the journal, not the realtime stamp
stored in them. That's arguably better, though.
Another deficiency is that it will get confused with boots interleaving in the
journal, therefore, it will refuse operation in --merge, --file and --directory 
mode.

https://bugs.freedesktop.org/show_bug.cgi?id=72601
---
 src/journal/journalctl.c | 275 ++-
 1 file changed, 174 insertions(+), 101 deletions(-)

diff --git a/src/journal/journalctl.c b/src/journal/journalctl.c
index 666aa20..c059b77 100644
--- a/src/journal/journalctl.c
+++ b/src/journal/journalctl.c
@@ -129,6 +129,7 @@ typedef struct boot_id_t {
 sd_id128_t id;
 uint64_t first;
 uint64_t last;
+LIST_FIELDS(struct boot_id_t, boot_list);
 } boot_id_t;
 
 static void pager_open_if_enabled(void) {
@@ -733,6 +734,11 @@ static int parse_argv(int argc, char *argv[]) {
 return -EINVAL;
 }
 
+if ((arg_boot || arg_action == ACTION_LIST_BOOTS)  (arg_file || 
arg_directory || arg_merge)) {
+log_error(Using --boot or --list-boots with --file, 
--directory or --merge is not supported.);
+return -EINVAL;
+}
+
 return 1;
 }
 
@@ -852,111 +858,203 @@ static int add_matches(sd_journal *j, char **args) {
 return 0;
 }
 
-static int boot_id_cmp(const void *a, const void *b) {
-uint64_t _a, _b;
+static int discover_next_boot(sd_journal *j,
+  boot_id_t **boot,
+  bool advance_older,
+  bool read_realtime) {
+int r;
+char match[9+32+1] = _BOOT_ID=;
+_cleanup_free_ boot_id_t *next_boot = NULL;
 
-_a = ((const boot_id_t *)a)-first;
-_b = ((const boot_id_t *)b)-first;
+assert(j);
+assert(boot);
 
-return _a  _b ? -1 : (_a  _b ? 1 : 0);
-}
+/* We expect the journal to be on the last position of a boot
+ * (in relation to the direction we are going), so that the next
+ * invocation of sd_journal_next/previous will be from a different
+ * boot. We then collect any information we desire and then jump
+ * to the last location of the new boot by using a _BOOT_ID match
+ * coming from the other journal direction. */
 
-static int get_boots(sd_journal *j,
- boot_id_t **boots,
- unsigned int *count,
- boot_id_t *query_ref_boot) {
-int r;
-const void *data;
-size_t length, allocated = 0;
+/* Make sure we aren't restricted by any _BOOT_ID matches, so that
+ * we can actually advance to a *different* boot. */
+sd_journal_flush_matches(j);
 
-assert(j);
-assert(boots);
-assert(count);
+if (advance_older)
+r = sd_journal_previous(j);
+else
+r = sd_journal_next(j);
+if (r  0)
+return r;
+else if (r == 0)
+return 0; /* End of journal, yay. */
+
+next_boot = new0(boot_id_t, 1);
+if (!next_boot)
+return log_oom();
 
-r = sd_journal_query_unique(j, _BOOT_ID);
+r = sd_journal_get_monotonic_usec(j, NULL, next_boot-id);
 if (r  0)
 return r;
 
-*count = 0;
-SD_JOURNAL_FOREACH_UNIQUE(j, data, length) {
-boot_id_t *id;
+if (read_realtime) {
+r = sd_journal_get_realtime_usec(j, next_boot-first);
+if (r  0)
+return r;
+}
 
-assert(startswith(data, _BOOT_ID=));
+/* Now seek to the last occurrence of this boot ID. */
+sd_id128_to_string(next_boot-id, match + 9);
+r = sd_journal_add_match(j, match, sizeof(match) - 1);
+if (r  0)
+return r;
 
-if (!GREEDY_REALLOC(*boots, allocated, *count + 1))
-return log_oom();
+if (advance_older)
+r = sd_journal_seek_head(j);
+else
+r = sd_journal_seek_tail(j);
+if (r  0)
+