[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

David Cook  changed:

   What|Removed |Added

 Status|Failed QA   |RESOLVED
 Resolution|--- |DUPLICATE

--- Comment #62 from David Cook  ---


*** This bug has been marked as a duplicate of bug 13706 ***

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Frank Hansen  changed:

   What|Removed |Added

 CC||frank.han...@ub.lu.se

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Agnes Rivers-Moore  changed:

   What|Removed |Added

 CC||a...@hanover.ca

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Katrin Fischer  changed:

   What|Removed |Added

 CC||katrin.fisc...@bsz-bw.de
 Status|Needs Signoff   |Failed QA

--- Comment #61 from Katrin Fischer  ---
Patches apply, but there is are some problems noted by the qa script:

 FAIL   misc/migration_tools/dedup_records.pl
   OK critic
   FAIL   forbidden patterns
forbidden pattern: merge marker (===) (line 419)
   OK git manipulation
   OK pod
   OK spelling
   FAIL   valid
when is experimental

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Marc Véron  changed:

   What|Removed |Added

 CC||ve...@veron.ch

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Marjorie Barry-Vila  changed:

   What|Removed |Added

 CC||marjorie.barry-v...@ccsr.qc
   ||.ca

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419
Bug 7419 depends on bug 11700, which changed state.

Bug 11700 Summary: C4::AuthoritiesMarc::merge enhancements/fixes
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11700

   What|Removed |Added

 Status|In Discussion   |RESOLVED
 Resolution|--- |DUPLICATE

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Nicole C. Engard  changed:

   What|Removed |Added

 CC|neng...@gmail.com   |

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Josef Moravec  changed:

   What|Removed |Added

 CC||josef.mora...@gmail.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #60 from Julian Maurice  ---
Created attachment 42972
  -->
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=42972=edit
Bug 7419: Allow wildcards in tagspec

Options --match and --check can now contains tagspec like "2.."

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice  changed:

   What|Removed |Added

 Status|Failed QA   |Needs Signoff

--- Comment #59 from Julian Maurice  ---
I cannot reproduce the error, so I presume I had a bad Zebra configuration.
Switching back to Needs Signoff

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #25088|0   |1
is obsolete||

--- Comment #53 from Julian Maurice julian.maur...@biblibre.com ---
Comment on attachment 25088
  -- http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=25088
Bug 7419: New script dedup_authorities.pl

Moved the dedup_authorities.pl patch to a new bug (bug 13706)

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #20783|0   |1
is obsolete||

--- Comment #55 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 35887
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=35887action=edit
Bug 7419: General-purpose record deduplicator

Rebased on master

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #18713|0   |1
is obsolete||

--- Comment #54 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 35886
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=35886action=edit
Bug 7419: correct LCCN/see from indexing

Rebased on master

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #20784|0   |1
is obsolete||

--- Comment #56 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 35888
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=35888action=edit
Bug 7419: Add more missing authority indexes

Rebased on master

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #20785|0   |1
is obsolete||

--- Comment #57 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 35889
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=35889action=edit
Bug 7419: quiet warnings in deduplicator

Rebased on master

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

 Status|Needs Signoff   |Failed QA

--- Comment #58 from Julian Maurice julian.maur...@biblibre.com ---
I always have the same error while running the script:

oAuth error: Unsupported Used attribute (114) Any Bib-1

Examples:

perl misc/migration_tools/dedup_records.pl -t -v -a -r --match=he/200a
Query: @or @attr 7=1 @attr 4=109 @attr 1=Local-Number 0  @attr 1=Any  @attr 4=1
 @attr 5=100 @attr 6=3 X (where X is the content of 200$a)

perl misc/migration_tools/dedup_records.pl -t -v -a -r --match=at/152b
Query: @or @attr 7=1 @attr 4=109 @attr 1=Local-Number 0  @attr 1=Any  @attr 4=1
 @attr 5=100 @attr 6=3 X (where X is 152$b)

Also, the documentation says we could use --match=he/2..a but it doesn't seem
to work (no zebra query is launched)

Changing status to Failed QA

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #25087|0   |1
is obsolete||

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #52 from Julian Maurice julian.maur...@biblibre.com ---
The patch that move indexation subs out of rebuild_zebra.pl was moved to a new
bug (Bug 13705)

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #51 from Jared Camins-Esakov jcam...@cpbibliography.com ---
(In reply to M. de Rooy from comment #50)
 When I am testing dedup-records.pl with Match/100abc on PERSO_NAME, the
 script finds two duplicates Surname, C.-J. and Surname, C-J.
 It does not find: Surname, Fred and Surname, F. (Note: Could be a match, but
 maybe not.)
 
 Without diving deep into C4/Matcher.pm myself, could you give some hints how
 to get the script to report such ones?

You would need to write your own matcher that identified Surname, Fred and
Surname, F. as duplicates. If you have a 4xx field for Surname, F. in the new
record you could use Previous-heading-see-from/100abcd.

 On the two patch sets in this report: I would suggest to move the two
 patches of Julian to another report. This is confusing. Also note that I
 would not recommend to refactor rebuild_zebra on a report about authorities;
 you'd better split them up.

I agree. The two deduplicators are not mutually exclusive in my opinion, so
perhaps it would make sense for me to create a separate bug for Julian's
patches with a subject something like Authority deduplicator for UNIMARC?

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

M. de Rooy m.de.r...@rijksmuseum.nl changed:

   What|Removed |Added

 CC||m.de.r...@rijksmuseum.nl

--- Comment #50 from M. de Rooy m.de.r...@rijksmuseum.nl ---
When I am testing dedup-records.pl with Match/100abc on PERSO_NAME, the script
finds two duplicates Surname, C.-J. and Surname, C-J.
It does not find: Surname, Fred and Surname, F. (Note: Could be a match, but
maybe not.)

Without diving deep into C4/Matcher.pm myself, could you give some hints how to
get the script to report such ones?

On the two patch sets in this report: I would suggest to move the two patches
of Julian to another report. This is confusing. Also note that I would not
recommend to refactor rebuild_zebra on a report about authorities; you'd better
split them up.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #25089|0   |1
is obsolete||

--- Comment #49 from Julian Maurice julian.maur...@biblibre.com ---
Comment on attachment 25089
  -- http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=25089
Fix C4::Context::Zconn()

Obsolete patch on Zconn (new bug 11701)

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #17790|0   |1
is obsolete||

--- Comment #46 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 25087
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=25087action=edit
Bug 7419: Move indexation subs in C4::ZebraIndex

Rebased on master

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

 CC||julian.maur...@biblibre.com
 Depends on||11700

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #22907|0   |1
is obsolete||

--- Comment #48 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 25089
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=25089action=edit
Fix C4::Context::Zconn()

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #17791|0   |1
is obsolete||
  Attachment #17792|0   |1
is obsolete||
  Attachment #22150|0   |1
is obsolete||

--- Comment #47 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 25088
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=25088action=edit
Bug 7419: New script dedup_authorities.pl

Rebased on master
Removed all changes to C4::AuthoritiesMarc::merge (see bug 11700)

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #44 from stephane.dela...@biblibre.com ---
(In reply to Jared Camins-Esakov from comment #43)
 (In reply to stephane.delaune from comment #42)
  Hi, I test this to dedup authorities. I begin with the 4 Jared patches (
  attachment.cgi?id= 18713 20783 20784 20785 ) : this dev is more complete but
  didn't work to dedup my authorities.
 
 What results did you see?

I had launched with the command line:
~/src/misc/migration_tools/dedup_records.pl -t -r -v -a -l authtypecode='SNG'
-m 'at/152b##he/2..axyz'
I just noticed a lot of error messages: oAuth error: Unsupported Use attribute
(114) Any Bib-1
However, I did not notice anything more and I'm not able to reproduce the same
test conditions

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #45 from stephane.dela...@biblibre.com ---
Created attachment 22907
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22907action=edit
Zconn fix, works with Julian Maurice patchs

Zconn fix, works with Julian Maurice patchs

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

stephane.dela...@biblibre.com changed:

   What|Removed |Added

 CC||stephane.delaune@biblibre.c
   ||om

--- Comment #41 from stephane.dela...@biblibre.com ---
Created attachment 22150
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22150action=edit
works with Julian Maurice patchs

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #42 from stephane.dela...@biblibre.com ---
Hi, I test this to dedup authorities. I begin with the 4 Jared patches (
attachment.cgi?id= 18713 20783 20784 20785 ) : this dev is more complete but
didn't work to dedup my authorities.

So I tested The 3 Julian's patches + 1 mine ( attachment.cgi?id= 17790 17791
17792 22150 ) : this dev is less complete (take only authorities) but works to
dedup my authorities

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #43 from Jared Camins-Esakov jcam...@cpbibliography.com ---
(In reply to stephane.delaune from comment #42)
 Hi, I test this to dedup authorities. I begin with the 4 Jared patches (
 attachment.cgi?id= 18713 20783 20784 20785 ) : this dev is more complete but
 didn't work to dedup my authorities.

What results did you see?

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

I'm just a bot git...@bugs.koha-community.org changed:

   What|Removed |Added

 Status|Needs Signoff   |Patch doesn't apply
 CC||git...@bugs.koha-community.
   ||org

--- Comment #39 from I'm just a bot git...@bugs.koha-community.org ---
Applying: Bug 7284: Authority matching improvements
Using index info to reconstruct a base tree...
MC4/AuthoritiesMarc.pm
MC4/Biblio.pm
MC4/Heading.pm
MC4/Heading/MARC21.pm
MC4/Search.pm
Macqui/addorderiso2709.pl
Macqui/neworderempty.pl
Mauthorities/auth_finder.pl
Mcataloguing/addbiblio.pl
Metc/zebradb/authorities/etc/bib1.att
Metc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml
Metc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl
Aetc/zebradb/marc_defs/marc21/authorities/koha-indexdefs-to-zebra.xsl
Metc/zebradb/marc_defs/unimarc/authorities/record.abs
Minstaller/data/mysql/sysprefs.sql
Minstaller/data/mysql/updatedatabase.pl
Mkoha-tmpl/intranet-tmpl/prog/en/includes/auth-finder-search.inc
Mkoha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/authorities.pref
Mmisc/link_bibs_to_authorities.pl
At/Heading_MARC21.t
At/db_dependent/lib/KohaTest/AuthoritiesMarc.pm
Falling back to patching base and 3-way merge...
CONFLICT (modify/delete): t/db_dependent/lib/KohaTest/AuthoritiesMarc.pm
deleted in HEAD and modified in Bug 7284: Authority matching improvements.
Version Bug 7284: Authority matching improvements of
t/db_dependent/lib/KohaTest/AuthoritiesMarc.pm left in tree.
Auto-merging t/db_dependent/Linker_FirstMatch.t
CONFLICT (add/add): Merge conflict in t/db_dependent/Linker_FirstMatch.t
Auto-merging t/Barcodes_annual.t
CONFLICT (content): Merge conflict in t/Barcodes_annual.t
Auto-merging misc/link_bibs_to_authorities.pl
CONFLICT (content): Merge conflict in misc/link_bibs_to_authorities.pl
Auto-merging
koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/authorities.pref
CONFLICT (content): Merge conflict in
koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/authorities.pref
Auto-merging koha-tmpl/intranet-tmpl/prog/en/includes/auth-finder-search.inc
CONFLICT (content): Merge conflict in
koha-tmpl/intranet-tmpl/prog/en/includes/auth-finder-search.inc
Auto-merging installer/data/mysql/updatedatabase.pl
CONFLICT (content): Merge conflict in installer/data/mysql/updatedatabase.pl
Auto-merging installer/data/mysql/sysprefs.sql
CONFLICT (content): Merge conflict in installer/data/mysql/sysprefs.sql
Auto-merging C4/Search.pm
CONFLICT (content): Merge conflict in C4/Search.pm
Auto-merging C4/Linker/Default.pm
CONFLICT (add/add): Merge conflict in C4/Linker/Default.pm
Auto-merging C4/Heading/UNIMARC.pm
CONFLICT (add/add): Merge conflict in C4/Heading/UNIMARC.pm
Auto-merging C4/Heading.pm
CONFLICT (content): Merge conflict in C4/Heading.pm
Auto-merging C4/Biblio.pm
CONFLICT (content): Merge conflict in C4/Biblio.pm
Auto-merging C4/AuthoritiesMarc.pm
CONFLICT (content): Merge conflict in C4/AuthoritiesMarc.pm
Patch failed at 0001 Bug 7284: Authority matching improvements
The copy of the patch that failed is found in:
   /home/christopher/git/koha/.git/rebase-apply/patch
When you have resolved this problem, run git am --continue.
If you prefer to skip this patch, run git am --skip instead.
To restore the original branch and stop patching, run git am --abort.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Chris Cormack ch...@bigballofwax.co.nz changed:

   What|Removed |Added

 Status|Patch doesn't apply |Needs Signoff
 CC||ch...@bigballofwax.co.nz

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #40 from Jared Camins-Esakov jcam...@cpbibliography.com ---
Please note that the final four patches comprise a counter-patch to the first
three, which are not MARC21-compatible.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Chris Cormack ch...@bigballofwax.co.nz changed:

   What|Removed |Added

Bot Control|--- |Ignore this bug

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Jared Camins-Esakov jcam...@cpbibliography.com changed:

   What|Removed |Added

 QA Contact|koha.sek...@gmail.com   |testo...@bugs.koha-communit
   ||y.org

--- Comment #38 from Jared Camins-Esakov jcam...@cpbibliography.com ---
Note that the new test plan does require you to be using QueryParser (and an
updated etc/searchengine/queryparser.yaml).

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #36 from Jared Camins-Esakov jcam...@cpbibliography.com ---
Created attachment 20784
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=20784action=edit
Bug 7419: Add more missing authority indexes

Koha's CCL configuration did not allow for most of the indexes that
were configured for authorities. This patch adds missing authority
indexes to QP.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Jared Camins-Esakov jcam...@cpbibliography.com changed:

   What|Removed |Added

  Attachment #18714|0   |1
is obsolete||

--- Comment #35 from Jared Camins-Esakov jcam...@cpbibliography.com ---
Created attachment 20783
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=20783action=edit
Bug 7419: General-purpose record deduplicator

This patch adds a script for deduplicating records. It is most useful
for authority records but by design could be easily extended for use
with bibliographic records, if someone had a good use case.

See the follow-up for an updated test plan.

Complete POD documentation:

SYNOPSIS
 dedup_records.pl --match=1 -a

 dedup_records.pl --match=LC-card-number/010a --select=date \
   --limit=authid  367123592 -a

 dedup_records.pl --match=Match/100abcdefghijklmnopqrstuvwxyz \
   --select=source=DLC --select=date \
   --limit=authtypecode='PERSO_NAME' -a

DESCRIPTION
   This script will identify duplicate records, and either suggest
   that you merge them (in the case of bibliographic records) or
   automatically merge them for you (in the case of authority records).

OPTIONS
   --help  Prints this help

   -v|--verbose
   Print verbose log information (warning: very verbose!).

   -t|--test
   Do not actually make any changes to the database, just
   report what changes would be made.

   -r|--report
   Print a report of what happened during the run.

   -l|--limit=S
   Only process those records that match the user-specified
   WHERE clause (the WHERE is implied and should not be
   included on the command line).

   -a|--authorities
   Check for duplicate authorities rather that duplicate
   bibliographic records.

   -s|--select=s
   Repeatable. Specify how to identify which record to
   prefer. See the section on SELECTORS below.

   -m|--match=s
   Specifies the matching rule to use. This can be the
   numeric ID of a matching rule that you have already
   configured (preferred), or you can specify a matching
   rule on the command-line in the following format:

  
index1/tag1subfield1[##index2/tag2subfield2[##...]]

   Examples:

   at/152b##he-main/2..a##he/2..bxyzt##ident/009@
   authtype/152b##he-main,ext/2..a##he,ext/2..bxyz
  
sn,ne,st-numeric/001##authtype/152b##he-main,ext/2..a##he,ext/2..bxyz

   -c|--check=s
   Only relevant when you are using a matching rule
   specified on the command line. Specifies sanity checks to
   use to ensure that the records are really duplicate. The
   format is tag1subfields1[,tag2subfields2[,...]]

   Examples:

   200abxyz will check subfields a,b,x,y,z of 200 fields
   009@,152b will check 009 data and 152$b subfields

SELECTORS
   This script supports a number of selectors for choosing which
   record is better.

   score   Prefer the record which is the best match based on the
   specified matching rule. This will probably only be
   useful in cases where the matching rule will not match
   the source record, since the source record will
   automatically be given a score of 2 * the matching rule
   threshold if it wasn't picked up by the matcher.

   datePrefer the record which is newer based on the 005 field.

   source=ABC
   MARC21 only. Prefer records which come from ABC based on
   the 003 field.

   usage   Authorities only. Prefer the record used in the most
   bibliographic records.

   ppn UNIMARC only. Prefer records which have a PPN in the 009
   field.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #37 from Jared Camins-Esakov jcam...@cpbibliography.com ---
Created attachment 20785
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=20785action=edit
Bug 7419: quiet warnings in deduplicator

Quiet some warnings from the deduplicator script.

Revised test plan:
1) Apply patches.
2) Import the sample records (sampleauths.mrc) file into Koha and
   make sure that they get indexed (by waiting until rebuild_zebra.pl
   runs automatically or by running rebuild_zebra.pl -a -z [-x] manually)
3) Deduplicate based on LCCN (replace {FIRSTAUTH} with the authid of the
   first imported record; on my system that number is 367123668):

misc/migration_tools/dedup_records.pl -t -v -a \
 -l authid = {FIRSTAUTH} -r -m lc-card/010a -s date

4) Check that you have 10 records that were chosen (you may have more,
   if you added more authorities after importing the sample data, but you
   should have at least the ten).
5) Deduplicate based on geographic heading, preferring Library of Congress
   authorities to local authorities:

misc/migration_tools/dedup_records.pl -t -v -a \
 -l authid = {FIRSTAUTH} -r -m geographic/151a -s source=DLC -s date

6) Check that you have 2 duplicate records replaced from amongst the new
   records.
7) Sign off.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #34 from Tomás Cohen Arazi tomasco...@gmail.com ---
(In reply to Mason James from comment #33)
 hi Tomas, 
 could you update the ticket with info about your testing of this bug
 
 Jared mentioned you found some problems?

The sample data and test plan didn't match, there was a problem with 155$a vs.
150$a (the second proposed test) and some warnings that needed to be addressed.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Mason James m...@kohaaloha.com changed:

   What|Removed |Added

 CC||m...@kohaaloha.com

--- Comment #33 from Mason James m...@kohaaloha.com ---
hi Tomas, 
could you update the ticket with info about your testing of this bug

Jared mentioned you found some problems?

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Jared Camins-Esakov jcam...@cpbibliography.com changed:

   What|Removed |Added

   Assignee|julian.maur...@biblibre.com |jcam...@cpbibliography.com

--- Comment #32 from Jared Camins-Esakov jcam...@cpbibliography.com ---
Several people have asked about a test plan, so I am repeating my test plan
from comment 30, since that comment is so long it can be hard to find:
To test using the sample records attached to the bug (MARC21 only):
1) Apply patches.
2) Import the sample records (sampleauths.mrc) file into Koha and
   make sure that they get indexed (by waiting until rebuild_zebra.pl
   runs automatically or by running rebuild_zebra.pl -a -z [-x] manually)
3) Deduplicate based on LCCN (replace {FIRSTAUTH} with the authid of the
   first imported record; on my system that number is 367123668; this is
   not actually necessary, but if you have a lot of authority records,
   the process could take quite a while otherwise):

misc/migration_tools/dedup_records.pl -t -v -a \
 -l authid = {FIRSTAUTH} -r -m lc-card/010a -s date

4) Check that you have 21 duplicate records replaced from amongst the
   new records (you will quite possibly have more than 21 duplicate
   records reported, depending on whether you have any of these
   authorities already, but you should have at least the 21).
5) Deduplicate based on genre heading, preferring Library of Congress
   authorities to local authorities:

misc/migration_tools/dedup_records.pl -t -v -a \
 -l authid = {FIRSTAUTH} -r -m he/155a -s source=DLC -s date

6) Check that you have 3 duplicate records replaced from amongst the new
   records.
7) Sign off.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #28 from Jared Camins-Esakov jcam...@cpbibliography.com ---
Created attachment 18712
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=18712action=edit
Sample duplicate MARC21 authorities

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #29 from Jared Camins-Esakov jcam...@cpbibliography.com ---
Created attachment 18713
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=18713action=edit
Bug 7419: correct LCCN/see from indexing

This patch adds indexes for previous headings and LCCN for MARC21
authority records. See the following patch for a test plan to confirm
that the indexing works correctly.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #30 from Jared Camins-Esakov jcam...@cpbibliography.com ---
Created attachment 18714
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=18714action=edit
Bug 7419: General-purpose record deduplicator

This patch adds a script for deduplicating records. It is most useful
for authority records but by design could be easily extended for use
with bibliographic records, if someone had a good use case.

To test using the sample records attached to the bug (MARC21 only):
1) Apply patches.
2) Import the sample records (sampleauths.mrc) file into Koha and
   make sure that they get indexed (by waiting until rebuild_zebra.pl
   runs automatically or by running rebuild_zebra.pl -a -z [-x] manually)
3) Deduplicate based on LCCN (replace {FIRSTAUTH} with the authid of the
   first imported record; on my system that number is 367123668; this is
   not actually necessary, but if you have a lot of authority records,
   the process could take quite a while otherwise):

misc/migration_tools/dedup_records.pl -t -v -a \
 -l authid = {FIRSTAUTH} -r -m lc-card/010a -s date

4) Check that you have 21 duplicate records replaced from amongst the
   new records (you will quite possibly have more than 21 duplicate
   records reported, depending on whether you have any of these
   authorities already, but you should have at least the 21).
5) Deduplicate based on genre heading, preferring Library of Congress
   authorities to local authorities:

misc/migration_tools/dedup_records.pl -t -v -a \
 -l authid = {FIRSTAUTH} -r -m he/155a -s source=DLC -s date

6) Check that you have 3 duplicate records replaced from amongst the new
   records.
7) Sign off.

Complete POD documentation:

SYNOPSIS
 dedup_records.pl --match=1 -a

 dedup_records.pl --match=LC-card-number/010a --select=date \
   --limit=authid  367123592 -a

 dedup_records.pl --match=Match/100abcdefghijklmnopqrstuvwxyz \
   --select=source=DLC --select=date \
   --limit=authtypecode='PERSO_NAME' -a

DESCRIPTION
   This script will identify duplicate records, and either suggest
   that you merge them (in the case of bibliographic records) or
   automatically merge them for you (in the case of authority records).

OPTIONS
   --help  Prints this help

   -v|--verbose
   Print verbose log information (warning: very verbose!).

   -t|--test
   Do not actually make any changes to the database, just
   report what changes would be made.

   -r|--report
   Print a report of what happened during the run.

   -l|--limit=S
   Only process those records that match the user-specified
   WHERE clause (the WHERE is implied and should not be
   included on the command line).

   -a|--authorities
   Check for duplicate authorities rather that duplicate
   bibliographic records.

   -s|--select=s
   Repeatable. Specify how to identify which record to
   prefer. See the section on SELECTORS below.

   -m|--match=s
   Specifies the matching rule to use. This can be the
   numeric ID of a matching rule that you have already
   configured (preferred), or you can specify a matching
   rule on the command-line in the following format:

  
index1/tag1subfield1[##index2/tag2subfield2[##...]]

   Examples:

   at/152b##he-main/2..a##he/2..bxyzt##ident/009@
   authtype/152b##he-main,ext/2..a##he,ext/2..bxyz
  
sn,ne,st-numeric/001##authtype/152b##he-main,ext/2..a##he,ext/2..bxyz

   -c|--check=s
   Only relevant when you are using a matching rule
   specified on the command line. Specifies sanity checks to
   use to ensure that the records are really duplicate. The
   format is tag1subfields1[,tag2subfields2[,...]]

   Examples:

   200abxyz will check subfields a,b,x,y,z of 200 fields
   009@,152b will check 009 data and 152$b subfields

SELECTORS
   This script supports a number of selectors for choosing which
   record is better.

   score   Prefer the record which is the best match based on the
   specified matching rule. This will probably only be
   useful in cases where the matching rule will not match
   the source record, since the source record will
   automatically be given a score of 2 * the matching rule
   threshold if it wasn't picked up by the matcher.

   datePrefer the record which is newer based on the 005 field.

   source=ABC
   MARC21 only. Prefer records which come from ABC based on
   the 003 field.

   usage   Authorities only. Prefer 

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #31 from Jared Camins-Esakov jcam...@cpbibliography.com ---
The final two patches comprise my counter-patch, which will handle both MARC21
and UNIMARC data (though the sample data provided on this bug is MARC21-only).

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #27 from Jared Camins-Esakov jcam...@cpbibliography.com ---
I tested this script and found that it did not work at all with MARC21 because
of the way it was retrieving authid. I am working on a counter-patch which I
hope to upload tomorrow which incorporates the matching options from your
script but should work properly with both UNIMARC and MARC21.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Koha Team AMU koha.aixmarsei...@gmail.com changed:

   What|Removed |Added

 CC||koha.aixmarsei...@gmail.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

David Cook dc...@prosentient.com.au changed:

   What|Removed |Added

 CC||dc...@prosentient.com.au

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #13629|0   |1
is obsolete||

--- Comment #24 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 17790
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=17790action=edit
Bug 7419: Move indexation subs in C4::ZebraIndex

Rebased on master

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #13630|0   |1
is obsolete||

--- Comment #25 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 17791
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=17791action=edit
Bug 7419: New script dedup_authorities.pl

Rebased on master

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

 Status|Needs Signoff   |Failed QA

--- Comment #23 from Julian Maurice julian.maur...@biblibre.com ---
Just retested it with Leïla and found an error

merging 25894 into 1198 failed :Can't call method fields on an undefined
value at /home/koha/src/C4/AuthoritiesMarc.pm line 1392.

Patch needs some work.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Bernardo Gonzalez Kriegel bgkrie...@gmail.com changed:

   What|Removed |Added

 CC||bgkrie...@gmail.com

--- Comment #22 from Bernardo Gonzalez Kriegel bgkrie...@gmail.com ---
Patch still applies.
Minor POD errors on script.
Missing license statement on script.

The problem is, I can't figure how to test it. And I think this could be a very
valuable tool.

If you could you add some test cases, or an accompanying .txt that explains
better the syntax it would be easier to test and use.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Tomás Cohen Arazi tomasco...@gmail.com changed:

   What|Removed |Added

 CC||tomasco...@gmail.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

 Status|Patch doesn't apply |Needs Signoff

--- Comment #21 from Julian Maurice julian.maur...@biblibre.com ---
Patches rebased on master

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #13073|0   |1
is obsolete||

--- Comment #19 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 13629
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=13629action=edit
Bug 7419: Move indexation subs in C4::ZebraIndex

Move indexation-related subs from rebuild_zebra.pl to new module
C4::ZebraIndex and create two new subs IndexRecord and DeleteRecordIndex

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #13074|0   |1
is obsolete||

--- Comment #20 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 13630
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=13630action=edit
Bug 7419: New script dedup_authorities.pl

This script allow to deduplicate authorities automatically.

It works this way:
 1) authorities are fetched from the database. You can limit fetched
 results by authtypecode, or directly by specifying WHERE clause
 2) for each authority:
  2.1) build a Zebra query, depending on --match parameter(s)
  2.2) run the query, retrieve the results
  2.3) for each result check if result is really a duplicate. check is
  customizable with option --check.
  2.4) among real duplicates, choose the one we want to keep (use
  --choose-method option).
  2.5) use C4::Authorities::merge to merge authorities
 3) index updated biblios and authorities, and remove from index
 removed authorities

Use --help for more informations on options

Script is in misc/migration_tools/

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #18 from Christophe Croullebois 
christophe.croulleb...@biblibre.com ---
Julian, it doesn't apply :

Applying: Bug 7419: Move indexation subs in C4::ZebraIndex
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Auto-merging misc/migration_tools/rebuild_zebra.pl
CONFLICT (content): Merge conflict in misc/migration_tools/rebuild_zebra.pl
Failed to merge in the changes.
Patch failed at 0001 Bug 7419: Move indexation subs in C4::ZebraIndex

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Christophe Croullebois christophe.croulleb...@biblibre.com changed:

   What|Removed |Added

 Status|Needs Signoff   |Patch doesn't apply
 CC||christophe.croullebois@bibl
   ||ibre.com

--- Comment #17 from Christophe Croullebois 
christophe.croulleb...@biblibre.com ---
Hello Julian,
if we don't use --verbose we have errors = Use of uninitialized value $verbose
in numeric ge (=) at ./dedup_authorities.pl line 126.
the same for lines 143,145,151.
For line 126 : if ($verbose = 2) should be : if ($verbose AND $verbose = 2)
thx

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #11487|0   |1
is obsolete||

--- Comment #14 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 13073
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=13073action=edit
Bug 7419: Move indexation subs in C4::ZebraIndex

rebased on master

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #11488|0   |1
is obsolete||

--- Comment #15 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 13074
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=13074action=edit
Bug 7419: New script dedup_authorities.pl

rebased on master

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

 Status|Patch doesn't apply |Needs Signoff

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #16 from Julian Maurice julian.maur...@biblibre.com ---
Pascale, you should be able to apply these patches now.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Pascale Nalon pascale.na...@gmail.com changed:

   What|Removed |Added

 Status|Needs Signoff   |Patch doesn't apply
 CC||pascale.na...@gmail.com

--- Comment #13 from Pascale Nalon pascale.na...@gmail.com ---
I tried to test this patch with the sandbox system, but the system send back
these errors :
The sandbox you've requested is now ready. The patches attached to bugzilla
7419 have been applied, and the result is
h1Something went wrong !/h1Applying: Bug 7419: Move indexation subs in
C4::ZebraIndexbr/
Using index info to reconstruct a base tree...br/
Falling back to patching base and 3-way merge...br/
Auto-merging misc/migration_tools/rebuild_zebra.plbr/
CONFLICT (content): Merge conflict in
misc/migration_tools/rebuild_zebra.plbr/
Failed to merge in the changes.br/
Patch failed at 0001 Bug 7419: Move indexation subs in C4::ZebraIndexbr/
When you have resolved this problem run git am --resolved.br/
If you would prefer to skip this patch, instead run git am --skip.br/
To restore the original branch and stop patching run git am --abort.br/
Bug 7419 - Add authority deduplication scriptbr/
br/
Bug 7419: Move indexation subs in C4::ZebraIndexbr/
Apply? [yn] br/
Patch left in
/tmp/Bug-7419-Move-indexation-subs-in-C4ZebraIndex-fNv1EB.patchbr/

Thanks for fixing it !

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #11412|0   |1
is obsolete||

--- Comment #9 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 11486
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=11486action=edit
Bug 7419: Move indexation subs in C4::ZebraIndex

Move indexation-related subs from rebuild_zebra.pl to new module
C4::ZebraIndex and create two new subs IndexRecord and DeleteRecordIndex

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #11486|0   |1
is obsolete||

--- Comment #10 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 11487
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=11487action=edit
Bug 7419: Move indexation subs in C4::ZebraIndex

Move indexation-related subs from rebuild_zebra.pl to new module
C4::ZebraIndex and create two new subs IndexRecord and DeleteRecordIndex

--
Same patch as above, but with the right commit message.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #11413|0   |1
is obsolete||

--- Comment #11 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 11488
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=11488action=edit
Bug 7419: New script dedup_authorities.pl

It works this way:
 1) authorities are fetched from the database. You can limit fetched
 results by authtypecode, or directly by specifying WHERE clause
 2) for each authority:
  2.1) build a Zebra query, depending on --match parameter(s)
  2.2) run the query, retrieve the results
  2.3) for each result check if result is really a duplicate. check is
  customizable with option --check.
  2.4) among real duplicates, choose the one we want to keep (use
  --choose-method option).
  2.5) use C4::Authorities::merge to merge authorities
 3) index updated biblios and authorities, and remove from index
 removed authorities

Use --help for more informations on options

Script is in misc/migration_tools/

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #12 from Julian Maurice julian.maur...@biblibre.com ---
Indexation subs are now in a new module C4::ZebraIndex, instead of C4::Search
(better to not overload C4::Search).

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #11161|0   |1
is obsolete||

--- Comment #7 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 11412
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=11412action=edit
Bug 7419: Move indexation subs in C4::Search

Rebased on master.

Add support for --skip-index option of rebuild_zebra.pl

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #8 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 11413
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=11413action=edit
Bug 7419: New script dedup_authorities.pl

Rebased on master

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

  Attachment #11162|0   |1
is obsolete||

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Julian Maurice julian.maur...@biblibre.com changed:

   What|Removed |Added

 Status|ASSIGNED|Needs Signoff
   Assignee|jcam...@cpbibliography.com  |julian.maur...@biblibre.com

--- Comment #5 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 11161
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=11161action=edit
Bug 7419: Move indexation subs in C4::Search

Move indexation-related subs from rebuild_zebra.pl to C4::Search and
create two new subs IndexRecord and DeleteRecordIndex

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #6 from Julian Maurice julian.maur...@biblibre.com ---
Created attachment 11162
  --
http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=11162action=edit
Bug 7419: New script dedup_authorities.pl

This script allow to deduplicate authorities automatically.

It works this way:
 1) authorities are fetched from the database. You can limit fetched
 results by authtypecode, or directly by specifying WHERE clause
 2) for each authority:
  2.1) build a Zebra query, depending on --match parameter(s)
  2.2) run the query, retrieve the results
  2.3) for each result check if result is really a duplicate. check is
  customizable with option --check.
  2.4) among real duplicates, choose the one we want to keep (use
  --choose-method option).
  2.5) use C4::Authorities::merge to merge authorities
 3) index updated biblios and authorities, and remove from index
 removed authorities

Use --help for more informations on options

Script is in misc/migration_tools/

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #4 from claire.hernan...@biblibre.com 
claire.hernan...@biblibre.com ---
Jared, I am talking about the initial Henri Damien script that evolved with
more options and bug fix. You will find the last version (can change again a
little but not a lot) here
http://git.biblibre.com/?p=koha;a=shortlog;h=refs/heads/ft/dedup_authorities

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

--- Comment #3 from Jared Camins-Esakov jcam...@cpbibliography.com ---
Claire,

(In reply to comment #2)
 Hello Jared. We did a big work on a similar script. I put this ticket on top
 of rebase tasks. We try to provide a lot of options, few more tests and in
 few weeks you could test it.

I actually borrowed from a deduplication script that hdl showed me, though I
think it may be that you have a newer version. The script I have works for
MARC21, and offers very basic options for UNIMARC, but I'd love to integrate my
changes into your script (or vice versa) to provide more options.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

claire.hernan...@biblibre.com claire.hernan...@biblibre.com changed:

   What|Removed |Added

 CC||claire.hernandez@biblibre.c
   ||om

--- Comment #2 from claire.hernan...@biblibre.com 
claire.hernan...@biblibre.com ---
Hello Jared. We did a big work on a similar script. I put this ticket on top of
rebase tasks. We try to provide a lot of options, few more tests and in few
weeks you could test it.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Jared Camins-Esakov jcam...@cpbibliography.com changed:

   What|Removed |Added

 Blocks||8212

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Bug 7419 depends on bug 7284, which changed state.

Bug 7284 Summary: Authority matching algorithm improvements
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7284

   What|Removed |Added

 Status|Pushed to Stable|RESOLVED
 Resolution|--- |FIXED

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Nicole C. Engard neng...@gmail.com changed:

   What|Removed |Added

 CC||neng...@gmail.com

-- 
Configure bugmail: 
http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 7419] Add authority deduplication script

[bugzilla-daemon]

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7419

Stefano Bargioni bargi...@pusc.it changed:

   What|Removed |Added

 CC||bargi...@pusc.it

--- Comment #1 from Stefano Bargioni bargi...@pusc.it 2012-01-13 08:21:42 UTC 
---
Authority deduplication is similar to authority unlink/link. Consider two terms
like
Bargioni, Stefano, 1959- linked to N1 biblios
and
Bargioni, Stefano linked to N2 biblios.
You need to unlink N2 biblios from the second term, i.e. link them to the first
one.
Deduplication can be automatic, while the step of choosing term-from and
term-to for unlink/link biblios must be manual.

-- 
Configure bugmail: 
http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/