[dspace-tech] Re: Need help with Dspace 5.5 XMLUI Search returning too many results

[wltantzen]

Sorry, you will need to restart tomcat as well for solr to catch the 
changes in your schema file!

On Friday, December 2, 2016 at 9:57:09 AM UTC-6, wlta...@gmail.com wrote:
>
> Tom,
>
> Try this:
>
> In dspace/solr/search/conf/schema.xml, look for the  and 
>  elements where type="text".  Change those elements to 
> type="text_ws" and reindex your collection.
>
> The "text" fieldType splits tokens on numerics and case change.  The 
> "text_ws" splits exclusively on whitespace for exact matching of words, 
> which sounds like what you need.
>
> Let me know how that works!
> Cheers,
> Bill
>
> On Wednesday, November 30, 2016 at 1:48:17 PM UTC-6, Tom Avino wrote:
>>
>> Since we upgraded to Dspace 5.5 and XMLUI, the search results return 
>> almost all records.  It is not doing an exact search.  Many of our searches 
>> have mixed numbers and alpha characters.  The results return as is it 
>> breaks up the search.  For example, If I search for AH234M22, it searches 
>> for anything with AH, 234, M, or 22.  If We put double quotes, it will do 
>> an exact search.  That is not feasible doe to the vast number of users on 
>> the system.  It is frustrating, and I need to resolve this.
>>
>> Any help would be appreciated, please.
>>
>> Tom Avino
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

[dspace-tech] Re: Need help with Dspace 5.5 XMLUI Search returning too many results

[wltantzen]

Tom,

Try this:

In dspace/solr/search/conf/schema.xml, look for the  and 
 elements where type="text".  Change those elements to 
type="text_ws" and reindex your collection.

The "text" fieldType splits tokens on numerics and case change.  The 
"text_ws" splits exclusively on whitespace for exact matching of words, 
which sounds like what you need.

Let me know how that works!
Cheers,
Bill

On Wednesday, November 30, 2016 at 1:48:17 PM UTC-6, Tom Avino wrote:
>
> Since we upgraded to Dspace 5.5 and XMLUI, the search results return 
> almost all records.  It is not doing an exact search.  Many of our searches 
> have mixed numbers and alpha characters.  The results return as is it 
> breaks up the search.  For example, If I search for AH234M22, it searches 
> for anything with AH, 234, M, or 22.  If We put double quotes, it will do 
> an exact search.  That is not feasible doe to the vast number of users on 
> the system.  It is frustrating, and I need to resolve this.
>
> Any help would be appreciated, please.
>
> Tom Avino
>

-- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

Re: [dspace-tech] Select Collection step performance very slow in DSpace 6.x

[wltantzen]

No, this is postgresql -- latest and greatest version

On Friday, December 2, 2016 at 5:39:18 AM UTC-6, Adan Roman Ruiz wrote:
>
> Joan, 
>
> are you using oracle? there is some index issues in dspace 6.0
>
> regards
> Adán Román Ruiz
> ARVO Consultores
>
> We are experiencing the something similar but in all our repository 
> (except the home page), almost all pages in DSpace 6x takes between 6s to 
> 60seconds, we have tried to increase the memory of the tomcat, changed the 
> SO (Centos7 to Ubuntu), upgrading tomcat (8.0.5 to 8.5), giving more cpus, 
> changing data to ssd hard drive... all without a signigicant improvements 
> of performance.
>
> Our configs are:
>
> Vcpus : 8 (GenuineIntel  2099.998MHz)
>
> Tomcat (8.5.8) config:  -Xmx4072M -Xms512M -XX:PermSize=256M 
> -XX:MaxPermSize=512m
>
> Items: arroung 120k
>
> Now we have in production DSpace 3_x and this situation is preventing us 
> from upgrading also.
>
> Bests
>
> Joan 
> El 01/12/16 a les 22:15, wlta...@gmail.com  ha escrit:
>
> In DSpace 6x, it takes an intolerable 10 minutes (!!!) to populate the 
> Select Collection drop-down in my instance when logging in as a regular 
> user as opposed to an administrator.  
>
> I can add hardware details if anybody is interested, but I'm pretty sure 
> it's not the problem -- this is a beefy machine that hummed along nicely 
> with version 5.x...
>
> We have a little over 1000 collections in our repository, and we use 
> shibboleth for authentication so findAuthorizedPerformanceOptimize is set 
> to it's default of false.
>
> Some rudimentary profiling suggests that the isAdmin() call in authorize() 
> is the bottleneck.  In a recent test (pretty typical), the findAuthorized() 
> function takes 704,040 ms to return.  The time spent in the isAdmin() call 
> amounts to 590,691 ms, or about 83% of the time.  Another bottleneck within 
> this bottleneck is the recursive call to isAdmin(parent collection) -- 
> about 400,000 ms.  Clearly there is a lot of waste in this call because 
> many many collections will be tested over and over.  In our case, we have a 
> single community that is the ancestor of every community/collection, and 
> isAdmin() winds up being called against it 1012 times -- same as the number 
> of collections in the repository -- and 1011 of them are redundant.
>
> As a further test, I have short-circuited isAdmin() to return false 
> immediately.  The results in the rendering of the drop-down in about 94,000 
> msec.  Still way too long, but a drastic improvement.
>
> When logging in as an administrator, isAdmin() returns rather quickly and 
> the list is populated much quicker as well.  But for the everyday user, the 
> performance is a big red light preventing us from upgrading.
>
> Sorry, this is simply stating the problem with no solution.  But has 
> anybody else with a similarly sized repository experienced this slow down?
>
> Regards,
> Bill
>
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "DSpace Technical Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to dspace-tech...@googlegroups.com .
> To post to this group, send email to dspac...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/dspace-tech.
> For more options, visit https://groups.google.com/d/optout.
>
>
> -- 
> .
>
> Joan Caparrós
> Tècnic de Projectes
> Consorci de Serveis Universitaris de Catalunya (CSUC)
>
> Gran Capità, 2 (Edifici Nexus)•08034 Barcelona
> T.93 551 6204•F.93 205 6979•joan.c...@csuc.cat www.csuc.cat 
> •Twitter @CSUC_info•Facebook•Linkedin
> Subscriu-te al butlletí; (www.csuc.cat/butlleti)
> . 
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "DSpace Technical Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to dspace-tech...@googlegroups.com .
> To post to this group, send email to dspac...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/dspace-tech.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> [image: Avast logo] 
> 
>  
>
> El software de antivirus Avast ha analizado este correo electrónico en 
> busca de virus. 
> www.avast.com 
> 
>  
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at 

[dspace-tech] Select Collection step performance very slow in DSpace 6.x

[wltantzen]

In DSpace 6x, it takes an intolerable 10 minutes (!!!) to populate the 
Select Collection drop-down in my instance when logging in as a regular 
user as opposed to an administrator. 

I can add hardware details if anybody is interested, but I'm pretty sure 
it's not the problem -- this is a beefy machine that hummed along nicely 
with version 5.x...

We have a little over 1000 collections in our repository, and we use 
shibboleth for authentication so findAuthorizedPerformanceOptimize is set 
to it's default of false.

Some rudimentary profiling suggests that the isAdmin() call in authorize() 
is the bottleneck.  In a recent test (pretty typical), the findAuthorized() 
function takes 704,040 ms to return.  The time spent in the isAdmin() call 
amounts to 590,691 ms, or about 83% of the time.  Another bottleneck within 
this bottleneck is the recursive call to isAdmin(parent collection) -- 
about 400,000 ms.  Clearly there is a lot of waste in this call because 
many many collections will be tested over and over.  In our case, we have a 
single community that is the ancestor of every community/collection, and 
isAdmin() winds up being called against it 1012 times -- same as the number 
of collections in the repository -- and 1011 of them are redundant.

As a further test, I have short-circuited isAdmin() to return false 
immediately.  The results in the rendering of the drop-down in about 94,000 
msec.  Still way too long, but a drastic improvement.

When logging in as an administrator, isAdmin() returns rather quickly and 
the list is populated much quicker as well.  But for the everyday user, the 
performance is a big red light preventing us from upgrading.

Sorry, this is simply stating the problem with no solution.  But has 
anybody else with a similarly sized repository experienced this slow down?

Regards,
Bill


-- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.