Re: [OPEN-ILS-GENERAL] metabib.browse_entry unused cleanup?

[Bill Erickson]

Hi Josh,

I don't have a yearly checklist.  I'll check the browse entries if it seems
like it's been a while or if I just did a full re-ingest.  It's low
priority.

Most of the cleanup these days is nightly, and it's the usual suspects,
circ/holds aging, auditor data, and action trigger data are the biggies.
Incidentally, the initial copy auditor and action/trigger cleanup resulted
in deleting about 1/3 of the database (by disk usage).

I still want to implement a form of bib record deletion that removes most
of the ingested data.  (I could say goodbye to 44M unused rows in
metabib.real_full_rec, for example).  That will require discussion, though,
since not everyone will want to get rid of everything (e.g. you may want to
report on deleted dibs).  This has come up in IRC a few times as well.  LP
pending.

-b



On Tue, Apr 3, 2018 at 8:44 AM, Josh Stompro <stomp...@exchange.larl.org>
wrote:

> Thanks Bill, is there any chance you have your checklist/script for what
> you cleanup shared anywhere?  I would love to see what else you target when
> you clean things up once a year.
>
>
>
> Josh Stompro - LARL IT Director
>
>
>
> *From:* Open-ils-general [mailto:open-ils-general-
> boun...@list.georgialibraries.org] *On Behalf Of *Bill Erickson
> *Sent:* Monday, April 02, 2018 3:31 PM
> *To:* Evergreen Discussion Group <open-ils-general@list.
> georgialibraries.org>
> *Subject:* Re: [OPEN-ILS-GENERAL] metabib.browse_entry unused cleanup?
>
>
>
> Hi Josh,
>
>
>
> I clean these up about once a year.  It's never been a problem, though.  I
> just like cleaning things.
>
>
>
> -b
>
>
>
> On Mon, Apr 2, 2018 at 4:10 PM, Josh Stompro <stomp...@exchange.larl.org>
> wrote:
>
> Hello, I’ve been changing around some indexing rules and applied the fix
> to the full stop/rda relator code normalization issue and now just noticed
> that all the no longer used browse entries are sticking around in
> metabib.browse_entry.
>
>
>
> There are 175K entries out of 950K that are no longer used according to
> metabib.browse_entry_def_map and metabib.browse_entry_simple_heading_map.
>
>
>
> Has anyone found this to be an issue and removed those unused entries?  I
> don’t think this is causing any issues, it just seems strange to hold on to
> that data.  Any typo that was ever entered in a browse field and corrected
> is recorded there.
>
>
>
> Thanks
>
> Josh
>
>
>
> Lake Agassiz Regional Library - Moorhead MN larl.org
>
> Josh Stompro | Office 218.233.3757 EXT-139 <(218)%20233-3757>
>
> LARL IT Director | Cell 218.790.2110 <(218)%20790-2110>
>
>
>
>
>

Re: [OPEN-ILS-GENERAL] metabib.browse_entry unused cleanup?

[Josh Stompro]

Thanks Bill, is there any chance you have your checklist/script for what you 
cleanup shared anywhere?  I would love to see what else you target when you 
clean things up once a year.

Josh Stompro - LARL IT Director

From: Open-ils-general 
[mailto:open-ils-general-boun...@list.georgialibraries.org] On Behalf Of Bill 
Erickson
Sent: Monday, April 02, 2018 3:31 PM
To: Evergreen Discussion Group <open-ils-general@list.georgialibraries.org>
Subject: Re: [OPEN-ILS-GENERAL] metabib.browse_entry unused cleanup?

Hi Josh,

I clean these up about once a year.  It's never been a problem, though.  I just 
like cleaning things.

-b

On Mon, Apr 2, 2018 at 4:10 PM, Josh Stompro 
<stomp...@exchange.larl.org<mailto:stomp...@exchange.larl.org>> wrote:
Hello, I’ve been changing around some indexing rules and applied the fix to the 
full stop/rda relator code normalization issue and now just noticed that all 
the no longer used browse entries are sticking around in metabib.browse_entry.

There are 175K entries out of 950K that are no longer used according to 
metabib.browse_entry_def_map and metabib.browse_entry_simple_heading_map.

Has anyone found this to be an issue and removed those unused entries?  I don’t 
think this is causing any issues, it just seems strange to hold on to that 
data.  Any typo that was ever entered in a browse field and corrected is 
recorded there.

Thanks
Josh

Lake Agassiz Regional Library - Moorhead MN larl.org<http://larl.org>
Josh Stompro | Office 218.233.3757 EXT-139<tel:(218)%20233-3757>
LARL IT Director | Cell 218.790.2110<tel:(218)%20790-2110>

Re: [OPEN-ILS-GENERAL] metabib.browse_entry unused cleanup?

[Mike Rylander]

Josh,

It's not causing any issues in any instance that EOLI oversees.  It's
a unique string listing based on a B+Tree index, so until you get past
several quintillion entries you should be fine.  The one benefit of
removing orphaned entries would be for the off chance that the terms
in the string are collectively entirely unique and you've disabled the
visibility test for autosuggest, where you might get a browse
suggestion that would lead to a dead end.

If you do clear them out, make sure you vacuum and analyze the table
(/not/ vacuum full, mind) afterwords.

HTH,

--
Mike Rylander
 | President
 | Equinox Open Library Initiative
 | phone:  1-877-OPEN-ILS (673-6457)
 | email:  mi...@equinoxinitiative.org
 | web:  http://equinoxinitiative.org


On Mon, Apr 2, 2018 at 4:10 PM, Josh Stompro  wrote:
> Hello, I’ve been changing around some indexing rules and applied the fix to
> the full stop/rda relator code normalization issue and now just noticed that
> all the no longer used browse entries are sticking around in
> metabib.browse_entry.
>
>
>
> There are 175K entries out of 950K that are no longer used according to
> metabib.browse_entry_def_map and metabib.browse_entry_simple_heading_map.
>
>
>
> Has anyone found this to be an issue and removed those unused entries?  I
> don’t think this is causing any issues, it just seems strange to hold on to
> that data.  Any typo that was ever entered in a browse field and corrected
> is recorded there.
>
>
>
> Thanks
>
> Josh
>
>
>
> Lake Agassiz Regional Library - Moorhead MN larl.org
>
> Josh Stompro | Office 218.233.3757 EXT-139
>
> LARL IT Director | Cell 218.790.2110
>
>

Re: [OPEN-ILS-GENERAL] metabib.browse_entry unused cleanup?

[Bill Erickson]

Hi Josh,

I clean these up about once a year.  It's never been a problem, though.  I
just like cleaning things.

-b

On Mon, Apr 2, 2018 at 4:10 PM, Josh Stompro 
wrote:

> Hello, I’ve been changing around some indexing rules and applied the fix
> to the full stop/rda relator code normalization issue and now just noticed
> that all the no longer used browse entries are sticking around in
> metabib.browse_entry.
>
>
>
> There are 175K entries out of 950K that are no longer used according to
> metabib.browse_entry_def_map and metabib.browse_entry_simple_heading_map.
>
>
>
> Has anyone found this to be an issue and removed those unused entries?  I
> don’t think this is causing any issues, it just seems strange to hold on to
> that data.  Any typo that was ever entered in a browse field and corrected
> is recorded there.
>
>
>
> Thanks
>
> Josh
>
>
>
> Lake Agassiz Regional Library - Moorhead MN larl.org
>
> Josh Stompro | Office 218.233.3757 EXT-139 <(218)%20233-3757>
>
> LARL IT Director | Cell 218.790.2110 <(218)%20790-2110>
>
>
>

[OPEN-ILS-GENERAL] metabib.browse_entry unused cleanup?

[Josh Stompro]

Hello, I've been changing around some indexing rules and applied the fix to the 
full stop/rda relator code normalization issue and now just noticed that all 
the no longer used browse entries are sticking around in metabib.browse_entry.

There are 175K entries out of 950K that are no longer used according to 
metabib.browse_entry_def_map and metabib.browse_entry_simple_heading_map.

Has anyone found this to be an issue and removed those unused entries?  I don't 
think this is causing any issues, it just seems strange to hold on to that 
data.  Any typo that was ever entered in a browse field and corrected is 
recorded there.

Thanks
Josh

Lake Agassiz Regional Library - Moorhead MN larl.org
Josh Stompro | Office 218.233.3757 EXT-139
LARL IT Director | Cell 218.790.2110