Hie,
Here is my patch for facets performance in C4::Search.
Regards,
--
Fridolyn SOMERS
ICT engineer
PROGILONE - Lyon - France
[email protected]
---------- Forwarded message ----------
From: Chris Cormack <[email protected]>
Date: Wed, Nov 24, 2010 at 8:32 PM
Subject: Re: [Koha-devel] Facets performance
To: Fridolyn SOMERS <[email protected]>
Cc: [email protected]
* Fridolyn SOMERS ([email protected]) wrote:
> Little up.
> Any feedback is welcome.
Hi Fridolyn
Can you please send your patch to the
[email protected] mailing list.
Then my scripts will pick it up and I can apply it easily :)
Chris
>
> On Fri, Oct 22, 2010 at 12:47 AM, Chris Cormack <
[email protected]>
> wrote:
>
> 2010/10/21 Fridolyn SOMERS <[email protected]>:
> > Hie,
>
> Hi Fridolyn
> >
> > I have posted a proposed patch for Bug 3154 :
> > http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=3154
> >
> > It's about the fact that facets computation is limited to the
records
> in
> > search results page.
> > I think I've found a good way to improve the facets extraction
> performance.
> >
> > Any comment or modification is welcome.
> >
> This looks really promising, I probably won't get a chance to try it
> out until after Kohacon.
> But i'm looking forward to giving it some testing
>
> Thank you
> Chris
>
> --
> Fridolyn SOMERS
> ICT engineer
> PROGILONE - Lyon - France
> [email protected]
> _______________________________________________
> Koha-devel mailing list
> [email protected]
> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : http://www.koha-community.org/
> git : http://git.koha-community.org/
> bugs : http://bugs.koha-community.org/
--
Chris Cormack
Catalyst IT Ltd.
+64 4 803 2238
PO Box 11-053, Manners St, Wellington 6142, New Zealand
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
iEYEARECAAYFAkztaDEACgkQZgbcHEvgMLOl8gCeMe12GFTJ9Cs1S1Y1380G3nis
404AoIzjCDGgB39exQqsSziu3zep5/Oo
=zAnn
-----END PGP SIGNATURE-----
Index: C4/Search.pm
===================================================================
--- C4/Search.pm
+++ C4/Search.pm
@@ -418,7 +420,6 @@
for ( my $j = $offset ; $j < $times ; $j++ ) {
my $records_hash;
my $record;
- my $facet_record;
## Check if it's an index scan
if ($scan) {
@@ -451,33 +452,58 @@
# warn "RECORD $j:".$record;
$results_hash->{'RECORDS'}[$j] = $record;
-
- # Fill the facets while we're looping, but only for the biblioserver
- $facet_record = MARC::Record->new_from_usmarc($record)
- if $servers[ $i - 1 ] =~ /biblioserver/;
-
- #warn $servers[$i-1]."\n".$record; #.$facet_record->title();
- if ($facet_record) {
- for ( my $k = 0 ; $k <= @$facets ; $k++ ) {
- ($facets->[$k]) or next;
- my @fields = map {$facet_record->field($_)} @{$facets->[$k]->{'tags'}} ;
- for my $field (@fields) {
- my @subfields = $field->subfields();
- for my $subfield (@subfields) {
- my ( $code, $data ) = @$subfield;
- ($code eq $facets->[$k]->{'subfield'}) or next;
- $facets_counter->{ $facets->[$k]->{'link_value'} }->{$data}++;
- }
- }
- $facets_info->{ $facets->[$k]->{'link_value'} }->{'label_value'} =
- $facets->[$k]->{'label_value'};
- $facets_info->{ $facets->[$k]->{'link_value'} }->{'expanded'} =
- $facets->[$k]->{'expanded'};
- }
- }
}
+
}
$results_hashref->{ $servers[ $i - 1 ] } = $results_hash;
+
+ # Fill the facets while we're looping, but only for the biblioserver and not for a scan
+ if ( !$scan && $servers[ $i - 1 ] =~ /biblioserver/ ) {
+
+ my $jmax = $size;
+ if ( $jmax > 500 ) {
+ $jmax = 500; # limit to 500 first records
+ }
+
+ for ( my $k = 0 ; $k <= @$facets ; $k++ ) {
+ ($facets->[$k]) or next;
+ my @fcodes = @{$facets->[$k]->{'tags'}};
+ my $sfcode = $facets->[$k]->{'subfield'};
+
+ for ( my $j = 0 ; $j < $jmax ; $j++ ) {
+ my $render_record = $results[ $i - 1 ]->record($j)->render();
+ my @used_datas = ();
+
+ foreach my $fcode (@fcodes) {
+
+ # avoid first line
+ my $field_pattern = '\n'.$fcode.' ([^\n]+)';
+ my @field_tokens = ( $render_record =~ /$field_pattern/g ) ;
+
+ foreach my $field_token (@field_tokens) {
+ my $subfield_pattern = '\$'.$sfcode.' ([^\$]+)';
+ my @subfield_values = ( $field_token =~ /$subfield_pattern/g );
+
+ foreach my $subfield_value (@subfield_values) {
+
+ my $data = $subfield_value;
+ $data =~ s/^\s+//; # trim left
+ $data =~ s/\s+$//; # trim right
+
+ unless ( $data ~~ @used_datas ) {
+ $facets_counter->{ $facets->[$k]->{'link_value'} }->{$data}++;
+ push @used_datas, $data;
+ }
+ } # subfields
+ } # fields
+ } # field codes
+ } # records
+
+ $facets_info->{ $facets->[$k]->{'link_value'} }->{'label_value'} = $facets->[$k]->{'label_value'};
+ $facets_info->{ $facets->[$k]->{'link_value'} }->{'expanded'} = $facets->[$k]->{'expanded'};
+ } # facets
+ }
+ # End PROGILONE
}
# warn "connection ", $i-1, ": $size hits";
_______________________________________________
Koha-patches mailing list
[email protected]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-patches
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
