[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Katrin Fischer  changed:

   What|Removed |Added

 Blocks||28447


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28447
[Bug 28447] Refine your search Holding libraries location code omits dot and
shows * instead of full name
-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Victor Grousset/tuxayo  changed:

   What|Removed |Added

 CC||vic...@tuxayo.net

--- Comment #28 from Victor Grousset/tuxayo  ---
Not backported to oldoldstable (19.05.x). Feel free to ask if it's needed.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Aleisha Amohia  changed:

   What|Removed |Added

 CC||alei...@catalyst.net.nz
 Version(s)|20.11.00, 20.05.04  |20.11.00, 20.05.04,
released in||19.11.10
 Status|Pushed to stable|Pushed to oldstable

--- Comment #27 from Aleisha Amohia  ---
backported to 19.11.x for 19.11.10

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Lucas Gass  changed:

   What|Removed |Added

 Version(s)|20.11.00|20.11.00, 20.05.04
released in||
 CC||lu...@bywatersolutions.com
 Status|Pushed to master|Pushed to stable

--- Comment #26 from Lucas Gass  ---
backported to 20.05.x for 20.05.04

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

--- Comment #25 from Jonathan Druart  
---
Pushed to master for 20.11, thanks to everybody involved!

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Jonathan Druart  changed:

   What|Removed |Added

 Version(s)||20.11.00
released in||
 Status|Passed QA   |Pushed to master

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

--- Comment #24 from Jonathan Druart  
---
Created attachment 109368
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=109368=edit
Bug 17661: Trivial simplification using += operator

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Katrin Fischer  changed:

   What|Removed |Added

 Attachment #109242|0   |1
is obsolete||

--- Comment #23 from Katrin Fischer  ---
Created attachment 109260
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=109260=edit
Bug 17661: (follow-up) Update regex to support Unicode characters

Rather than limiting initials to [A-Z] we should test for a broad
range of uppercase letters.

The ES/Zebra changes are slightly different because of Perl vs Java regex
conventions. POerl may support either, but I found 'Uppercase' to be a bit more
explicit

More info here:
https://perldoc.perl.org/perlunicode.html

TO test:
Same plan as before but use Ж. as the ending initial
Confirm the period is preserved and other punctuation removed

Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Katrin Fischer  changed:

   What|Removed |Added

 Attachment #109117|0   |1
is obsolete||

--- Comment #22 from Katrin Fischer  ---
Created attachment 109259
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=109259=edit
Bug 17661 - Ending punctuation causes duplicate facets

The current code for facets doesn't pull strip ending punctuation from facets
This causes duplicate facets for terms that should be combined

Sometimes series can have different punctuation depending on the field they are
in
Author initials punctuation should be preserved

To test:
1 - Do search and pull up some records
2 - Edit some of the records to have authors like:
Date, C.J.
Date, C.j.
Date, C.J .
3 - Edit the records to have some series statments like:
830 $aDate, C.J. ;$v5
830 $aDate, C.J. ; $v5
830 $aDate, C.J.; $v5
4 - Add some 490s to the record with first indicator 1 and series like:
You wouldn't want to--
You wouldn't want to
You wouldn't want to..
5 - Search again and note you have 3 facets each for author and series
6 - Apply patch
7 - Repeat
8 - Now you get 2 facets for author, period not removed when following Upper
case immediately, is otherwise
9 - Now you should have a single series facet
10 - Switch search engine to ES (index before applying patch)
11 - Note facets are separate again
12 - Reset mappings and reindex
   perl misc/search_tools/rebuild_elasticsearch -v -r
13 - Repeat search, facets combined as above

Signed-off-by: Sarah Cornell 

Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Katrin Fischer  changed:

   What|Removed |Added

   Patch complexity|--- |Small patch
 Status|Signed Off  |Passed QA

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

--- Comment #21 from Nick Clemens  ---
Created attachment 109242
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=109242=edit
Bug 17661: (follow-up) Update regex to support Unicode characters

Rather than limiting initials to [A-Z] we should test for a broad
range of uppercase letters.

The ES/Zebra changes are slightly different because of Perl vs Java regex
conventions. POerl may support either, but I found 'Uppercase' to be a bit more
explicit

More info here:
https://perldoc.perl.org/perlunicode.html

TO test:
Same plan as before but use Ж. as the ending initial
Confirm the period is preserved and other punctuation removed

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Nick Clemens  changed:

   What|Removed |Added

 Status|Failed QA   |Signed Off

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

--- Comment #20 from David Cook  ---
One downside of post-processing facet results is that your facet list length
will vary depending on its contents... 

It could be more reliable to handle this issue at index-time rather than
search-time. 

Since we're using DOM indexing with Zebra, we have a lot of power at index-time
(beyond the use of CHR and ICU mechanisms) which could handle whitespace and
punctuation for facet purposes. 

That being said... post-processing is probably easiest to have the most
consistent behaviour across 3 different facet providers...

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

--- Comment #19 from David Cook  ---
Ok still using CHR indexing but using the unnormalized 0 register...

Previous results used "elements zebra::facet::author:p"

These results use "elements zebra::facet::au:0"

http://www.indexdata.com/zebra/;>
  
OCarroll, Philip,
Berryman, Faye
Dale, Rae,
Powell, Jonathon,
Dale, Rae
Shapiro, Lawrence E
Reynolds, Kate E.,
Anderson, Vicki
Betts, Dion E.
Brennan, Tom
Caramagna, Joe
Foxx, Richard M
Gutstein, Steven E
Medlen, Joan E. Guthrie.
Renton, N. E. (Nicholas Edwin), 1931
Reynolds, Kate E.
Rosemann, Bill
Schaefer, Charles E
Snell, Martha E.
Attwood, Tony
  


I see that duplication now:
Dale, Rae,
Dale, Rae

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

--- Comment #18 from David Cook  ---
Ah, that example was using CHR indexing.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

--- Comment #17 from David Cook  ---
Yet, if I look at http://localhost:8081/cgi-bin/koha/catalogue/search.pl?q=test
I'm not seeing data that looks like that facet data I showed before (which was
from a different catalogue).

I'll dig a bit more...

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

David Cook  changed:

   What|Removed |Added

 CC||dc...@prosentient.com.au

--- Comment #16 from David Cook  ---
This is interesting.

I'm not surprised that "_get_facets_data_from_record" works in an undesirable
way, but I'm surprised that this would be necessary for
"_get_facet_from_result_set" using Zebra facets, since Zebra returns the data
in a normalized format.

For example:

Z> elements zebra::facet::author:p
Z> show 1
Sent presentRequest (1+1).
Records: 1
Record type: XML
http://www.indexdata.com/zebra/;>
  
berryman faye
dale rae
dale rae 1945
fitzroy programs
o carroll philip
o carroll philip 1945
powell jonathon
powell jonathon illustrator
reynolds kate e
shapiro lawrence e
carter emily
caramagna joe
schaefer charles e
digeronimo theresa foy
hughes edward e
kazdin alan e
reeve christine e
renton n e nicholas edwin 1931
snell martha e
attwood tony
  

nextResultSetPosition = 2
Elapsed: 0.030227

Now in this case there are some apparent "duplicates" where the author's date
is included in some records but not others. I would argue that is an authority
data issue though, since "John Smith", "John Smith 1945", and "John Smith 1995"
are all different authors. 

I don't know how Elasticsearch handles its facets, so I can't comment there.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

--- Comment #15 from Katrin Fischer  ---
I found some information here, that is promising:

https://perldoc.perl.org/perlunicode.html
For instance, \p{Uppercase} matches any single character with the Unicode
"Uppercase" property, while \p{L} matches any character with a General_Category
of "L" (letter) property 

So maybe we could just switch out [A-Z] for the \p{Uppercase}.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Katrin Fischer  changed:

   What|Removed |Added

 Status|Signed Off  |Failed QA

--- Comment #14 from Katrin Fischer  ---

This is a really great improvement and works really well with our sample data.

The only thing that I've been wondering about: seeing the A-Z as part of the
regex: What will happen with non-latin and letters with diacritics? 

At the moment Ж. (Cyrillic) will lose the '.' and not be recognized as an
initial. Is there a way we could make this work maybe? Could we check for the
characters before being a space followed by a single character maybe?

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Andrew Fuerste-Henry  changed:

   What|Removed |Added

 Attachment #108742|0   |1
is obsolete||

--- Comment #13 from Andrew Fuerste-Henry  ---
Created attachment 109117
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=109117=edit
Bug 17661 - Ending punctuation causes duplicate facets

The current code for facets doesn't pull strip ending punctuation from facets
This causes duplicate facets for terms that should be combined

Sometimes series can have different punctuation depending on the field they are
in
Author initials punctuation should be preserved

To test:
1 - Do search and pull up some records
2 - Edit some of the records to have authors like:
Date, C.J.
Date, C.j.
Date, C.J .
3 - Edit the records to have some series statments like:
830 $aDate, C.J. ;$v5
830 $aDate, C.J. ; $v5
830 $aDate, C.J.; $v5
4 - Add some 490s to the record with first indicator 1 and series like:
You wouldn't want to--
You wouldn't want to
You wouldn't want to..
5 - Search again and note you have 3 facets each for author and series
6 - Apply patch
7 - Repeat
8 - Now you get 2 facets for author, period not removed when following Upper
case immediately, is otherwise
9 - Now you should have a single series facet
10 - Switch search engine to ES (index before applying patch)
11 - Note facets are separate again
12 - Reset mappings and reindex
   perl misc/search_tools/rebuild_elasticsearch -v -r
13 - Repeat search, facets combined as above

Signed-off-by: Sarah Cornell 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Andrew Fuerste-Henry  changed:

   What|Removed |Added

 Status|Needs Signoff   |Signed Off

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Nick Clemens  changed:

   What|Removed |Added

 Attachment #106637|0   |1
is obsolete||

--- Comment #12 from Nick Clemens  ---
Created attachment 108742
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=108742=edit
Bug 17661 - Ending punctuation causes duplicate facets

The current code for facets doesn't pull strip ending punctuation from facets
This causes duplicate facets for terms that should be combined

Sometimes series can have different punctuation depending on the field they are
in
Author initials punctuation should be preserved

To test:
1 - Do search and pull up some records
2 - Edit some of the records to have authors like:
Date, C.J.
Date, C.j.
Date, C.J .
3 - Edit the records to have some series statments like:
830 $aDate, C.J. ;$v5
830 $aDate, C.J. ; $v5
830 $aDate, C.J.; $v5
4 - Add some 490s to the record with first indicator 1 and series like:
You wouldn't want to--
You wouldn't want to
You wouldn't want to..
5 - Search again and note you have 3 facets each for author and series
6 - Apply patch
7 - Repeat
8 - Now you get 2 facets for author, period not removed when following Upper
case immediately, is otherwise
9 - Now you should have a single series facet
10 - Switch search engine to ES (index before applying patch)
11 - Note facets are separate again
12 - Reset mappings and reindex
   perl misc/search_tools/rebuild_elasticsearch -v -r
13 - Repeat search, facets combined as above

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

--- Comment #11 from Nick Clemens  ---
(In reply to Katrin Fischer from comment #9)
> 1) Your regex always removes the first occurence, but we need the last.
> 2) . is not removed as it's not escaped in the regex (I think).
> 2) In case of "Date, C. J." we don't want the last, when it's preceded by a
> capital letter (very likely to be an initial).

1 - I had two different regexes, one was wrong
2 - period does not need to be escaped
3 - Done

I added eliminating spaces on either side of punctuation as well

Tested and played here to find cases:
https://regex101.com/

This is the base right now:
\s*(?https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Nick Clemens  changed:

   What|Removed |Added

 Status|Failed QA   |Needs Signoff

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Nick Clemens  changed:

   What|Removed |Added

  Attachment #68897|0   |1
is obsolete||

--- Comment #10 from Nick Clemens  ---
Created attachment 106637
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106637=edit
Bug 17661 - Ending punctuation causes duplicate facets

To test:
1 - Do search and pull up some records
2 - Edit some of the records to have authors like:
Date, C.J.
Date, C.j.
Date, C.J .
3 - Edit the records to have some series statments like:
830 $aDate, C.J. ;$v5
830 $aDate, C.J. ; $v5
830 $aDate, C.J.; $v5
4 - Search again and note you have 3 facets each for authr and series
5 - Apply patch
6 - Repeat
7 - Now you get 2 facets for author, period not removed when following Upper
case immediately, is otherwise
8 - Now you should have a single series facet
9 - Switch search engine to ES (index before applying patch)
10 - Note facets are separate again
11 - Reset mappings and reindex
   perl misc/search_tools/rebuild_elasticsearch -v -r
12 - Repeat search, facets combined as above

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Ray Delahunty  changed:

   What|Removed |Added

 CC||r.delahu...@arts.ac.uk

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Katrin Fischer  changed:

   What|Removed |Added

 Status|Needs Signoff   |Failed QA

--- Comment #9 from Katrin Fischer  ---
Hi Nick,

I thouhgt it would be very nice to revive this bug and looked at your WIP
patch. I think you were on the right track there, but I'd like to suggest some
changes to the regex.

You got:
$data =~ s/[.\-,]$//; 

I've tested with authors and made some changes on my data:

Without patch:

Aristophanes,
Aristophanes.
Beckett, Samuel,
Celan, Paul.
Date, C. J. 

With patch:

Aristophanes.
Beckett Samuel,
Celan Paul.
Date C. J. 

1) Your regex always removes the first occurence, but we need the last.
2) . is not removed as it's not escaped in the regex (I think).
2) In case of "Date, C. J." we don't want the last, when it's preceded by a
capital letter (very likely to be an initial).

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Katrin Fischer  changed:

   What|Removed |Added

 Status|NEW |Needs Signoff
 CC||katrin.fisc...@bsz-bw.de

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Niamh  changed:

   What|Removed |Added

 CC||niamh.walker-headon@it-tall
   ||aght.ie

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

krab...@nrhtx.com changed:

   What|Removed |Added

 CC||krab...@nrhtx.com

--- Comment #8 from krab...@nrhtx.com ---
My library would also be glad to see a resolution for this bug. The quality of
our OPAC is part of the customer service we offer to our patrons. Misleading
duplication in the OPAC because of how the 490 is read negatively impacts the
patron experience rather than helping us make our OPAC as effective as
possible. Thank you!

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Katrin Fischer  changed:

   What|Removed |Added

 CC||pasi.kalli...@koha-suomi.fi

--- Comment #7 from Katrin Fischer  ---
*** Bug 13231 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Heather  changed:

   What|Removed |Added

 CC||heather_hernan...@nps.gov

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Lisette Scheer  changed:

   What|Removed |Added

 CC||lisettesla...@gmail.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Michal Denar  changed:

   What|Removed |Added

 CC||blac...@gmail.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

--- Comment #6 from Marjorie Barry-Vila  ---
+1

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Jess Tobin  changed:

   What|Removed |Added

 CC||lara...@gmail.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Tami Wilkerson  changed:

   What|Removed |Added

 CC||twilker...@critfc.org

--- Comment #5 from Tami Wilkerson  ---
(In reply to Sarah Cornell from comment #4)
> Correcting this display problem would really improve user experience. 
> 
> Currently users need to discover by trial and error that the two entries
> "Brunkus, Denise," and Brunkus, Denise." run the same search.  Our
> librarians assume they run different searches, and it makes them distrust
> the catalog.  
> 
> This affects all facets that come from MARC: authors, series, places, topics.

I completely agree with Sarah. The duplication pushes other authors, series,
places, topics, etc. further down the list and users have to click "Show Me
More" to see them.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

--- Comment #4 from Sarah Cornell  ---
Correcting this display problem would really improve user experience. 

Currently users need to discover by trial and error that the two entries
"Brunkus, Denise," and Brunkus, Denise." run the same search.  Our librarians
assume they run different searches, and it makes them distrust the catalog.  

This affects all facets that come from MARC: authors, series, places, topics.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Sarah Cornell  changed:

   What|Removed |Added

 CC||sbcornell@cityofportsmouth.
   ||com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Patrick Robitaille  changed:

   What|Removed |Added

 CC||patrick.robitaille@collecto
   ||.ca

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 17661] Differences in field ending (whitespace, punctuation) cause duplicate facets

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17661

Nick Clemens  changed:

   What|Removed |Added

Summary|Differences in whitespace   |Differences in field ending
   |cause duplicate facets  |(whitespace, punctuation)
   ||cause duplicate facets

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/