Re: [dspace-tech] SOLR Configuration (Searching)
Tom, Thank you for the update. I submitted a DSpace ticket to consider making this change part of the default DSpace distribution. https://jira.duraspace.org/browse/DS-3691 Terry On Fri, Sep 8, 2017 at 8:53 AM, Tom Avinowrote: > Terry, > > I wanted to get back to you on this since you were so helpful with my SOLR > search. > What I was doing wrong was, after I modified the schema.xml file, I > ran dspace index-db-browse. I was told by Tim Donahue, that I need to run > index-discovery with the -b option. > So, everything is working great. I am using your recommended textgen and > am grateful for your help. > > Thanks, > > Tom Avino > > On Wednesday, August 30, 2017 at 10:43:07 AM UTC-4, Terry Brady wrote: >> >> Tom, >> >> Here is my modified schema.xml for DSpace 5x: https://gist.github.com/te >> rrywbrady/097164329b2574b9d6bd4bb8433ca37b >> >> I will be curious if this resolves issue for you. If so, we might want >> to propose this as an enhancement. >> >> Terry >> >> On Wed, Aug 30, 2017 at 5:43 AM, Tom Avino wrote: >> >>> Terry, >>> >>> Where and how did you change to textgen? I guess that is what I cannot >>> find, I see it in schema.xml, but not sure where to change it. >>> >>> Thanks >>> >>> On Tuesday, August 29, 2017 at 6:23:19 PM UTC-4, Terry Brady wrote: Unfortunately, I am not aware of any documentation on how to configure this file for DSpace. We found that the term stemming returned far too many false hits, so I changed several of our fields from "text" to "textgen" and the results were improved. https://github.com/DSpace/DSpace/blob/master/dspace/solr/ search/conf/schema.xml#L278-L279 On Tue, Aug 29, 2017 at 1:41 PM, Tom Avino wrote: > Since we switched from Dspace 3.3 JSPUI to XMLUI, our search is not > working the same. If our users type a specific item CA01234, it will not > find this even though it is in one of the metadata fields. I am not sure > how SOLR works. I have looked what is in schema.xml. I am unclear which > fieldtype is used when a search is done. It looks like each fieldtype can > have different parameters. There is "text_ws", "text", "textTight", etc. > Where in Dspace do you pick which one to use during a search??? > > Example > positionIncrementGap="100"> > > > ignoreCase="true" > words="stopwords.txt" > enablePositionIncrements="true" > /> > preserveOriginal="1" *splitOnNumerics="0" generateWordParts="1" > generateNumberParts="1" catenateWords="1" catenateNumbers="1" > catenateAll="0" splitOnCaseChange="0"*/> > > > Tom Avino > > -- > You received this message because you are subscribed to the Google > Groups "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to dspace-tech...@googlegroups.com. > To post to this group, send email to dspac...@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- Terry Brady Applications Programmer Analyst Georgetown University Library Information Technology http://georgetown-university-libraries.github.io/ 425-298-5498 (Seattle, WA) >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "DSpace Technical Support" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to dspace-tech...@googlegroups.com. >>> To post to this group, send email to dspac...@googlegroups.com. >>> Visit this group at https://groups.google.com/group/dspace-tech. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Terry Brady >> Applications Programmer Analyst >> Georgetown University Library Information Technology >> http://georgetown-university-libraries.github.io/ >> 425-298-5498 <(425)%20298-5498> (Seattle, WA) >> > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- Terry Brady Applications Programmer Analyst Georgetown University Library Information Technology https://github.com/terrywbrady/info 425-298-5498 (Seattle, WA) -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop
Re: [dspace-tech] SOLR Configuration (Searching)
Terry, I wanted to get back to you on this since you were so helpful with my SOLR search. What I was doing wrong was, after I modified the schema.xml file, I ran dspace index-db-browse. I was told by Tim Donahue, that I need to run index-discovery with the -b option. So, everything is working great. I am using your recommended textgen and am grateful for your help. Thanks, Tom Avino On Wednesday, August 30, 2017 at 10:43:07 AM UTC-4, Terry Brady wrote: > > Tom, > > Here is my modified schema.xml for DSpace 5x: > https://gist.github.com/terrywbrady/097164329b2574b9d6bd4bb8433ca37b > > I will be curious if this resolves issue for you. If so, we might want to > propose this as an enhancement. > > Terry > > On Wed, Aug 30, 2017 at 5:43 AM, Tom Avino> wrote: > >> Terry, >> >> Where and how did you change to textgen? I guess that is what I cannot >> find, I see it in schema.xml, but not sure where to change it. >> >> Thanks >> >> On Tuesday, August 29, 2017 at 6:23:19 PM UTC-4, Terry Brady wrote: >>> >>> Unfortunately, I am not aware of any documentation on how to configure >>> this file for DSpace. >>> >>> We found that the term stemming returned far too many false hits, so I >>> changed several of our fields from "text" to "textgen" and the results were >>> improved. >>> >>> >>> https://github.com/DSpace/DSpace/blob/master/dspace/solr/search/conf/schema.xml#L278-L279 >>> >>> On Tue, Aug 29, 2017 at 1:41 PM, Tom Avino wrote: >>> Since we switched from Dspace 3.3 JSPUI to XMLUI, our search is not working the same. If our users type a specific item CA01234, it will not find this even though it is in one of the metadata fields. I am not sure how SOLR works. I have looked what is in schema.xml. I am unclear which fieldtype is used when a search is done. It looks like each fieldtype can have different parameters. There is "text_ws", "text", "textTight", etc. Where in Dspace do you pick which one to use during a search??? Example >>> positionIncrementGap="100"> >>> ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" /> >>> preserveOriginal="1" *splitOnNumerics="0" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="0"*/> Tom Avino -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com. To post to this group, send email to dspac...@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> -- >>> Terry Brady >>> Applications Programmer Analyst >>> Georgetown University Library Information Technology >>> http://georgetown-university-libraries.github.io/ >>> 425-298-5498 (Seattle, WA) >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "DSpace Technical Support" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to dspace-tech...@googlegroups.com . >> To post to this group, send email to dspac...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/dspace-tech. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Terry Brady > Applications Programmer Analyst > Georgetown University Library Information Technology > http://georgetown-university-libraries.github.io/ > 425-298-5498 (Seattle, WA) > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] SOLR Configuration (Searching)
Terry, I was looking at the wrong schema.xml. I understand now. This is from yours and it makes sense now. Thank you again, and I will let you know if this works for me. I think it will On Wednesday, August 30, 2017 at 10:43:07 AM UTC-4, Terry Brady wrote: > > Tom, > > Here is my modified schema.xml for DSpace 5x: > https://gist.github.com/terrywbrady/097164329b2574b9d6bd4bb8433ca37b > > I will be curious if this resolves issue for you. If so, we might want to > propose this as an enhancement. > > Terry > > On Wed, Aug 30, 2017 at 5:43 AM, Tom Avino> wrote: > >> Terry, >> >> Where and how did you change to textgen? I guess that is what I cannot >> find, I see it in schema.xml, but not sure where to change it. >> >> Thanks >> >> On Tuesday, August 29, 2017 at 6:23:19 PM UTC-4, Terry Brady wrote: >>> >>> Unfortunately, I am not aware of any documentation on how to configure >>> this file for DSpace. >>> >>> We found that the term stemming returned far too many false hits, so I >>> changed several of our fields from "text" to "textgen" and the results were >>> improved. >>> >>> >>> https://github.com/DSpace/DSpace/blob/master/dspace/solr/search/conf/schema.xml#L278-L279 >>> >>> On Tue, Aug 29, 2017 at 1:41 PM, Tom Avino wrote: >>> Since we switched from Dspace 3.3 JSPUI to XMLUI, our search is not working the same. If our users type a specific item CA01234, it will not find this even though it is in one of the metadata fields. I am not sure how SOLR works. I have looked what is in schema.xml. I am unclear which fieldtype is used when a search is done. It looks like each fieldtype can have different parameters. There is "text_ws", "text", "textTight", etc. Where in Dspace do you pick which one to use during a search??? Example >>> positionIncrementGap="100"> >>> ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" /> >>> preserveOriginal="1" *splitOnNumerics="0" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="0"*/> Tom Avino -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com. To post to this group, send email to dspac...@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> -- >>> Terry Brady >>> Applications Programmer Analyst >>> Georgetown University Library Information Technology >>> http://georgetown-university-libraries.github.io/ >>> 425-298-5498 (Seattle, WA) >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "DSpace Technical Support" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to dspace-tech...@googlegroups.com . >> To post to this group, send email to dspac...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/dspace-tech. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Terry Brady > Applications Programmer Analyst > Georgetown University Library Information Technology > http://georgetown-university-libraries.github.io/ > 425-298-5498 (Seattle, WA) > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] SOLR Configuration (Searching)
Terry, So I think I partially understand. You modified the values in the "textgen" fieldtype in schema.xml. But doesn't the line below define which type to use for a search? Where do you specify to use "textgen" for a search? I hope I am not giving you a headache. On Wednesday, August 30, 2017 at 10:43:07 AM UTC-4, Terry Brady wrote: > > Tom, > > Here is my modified schema.xml for DSpace 5x: > https://gist.github.com/terrywbrady/097164329b2574b9d6bd4bb8433ca37b > > I will be curious if this resolves issue for you. If so, we might want to > propose this as an enhancement. > > Terry > > On Wed, Aug 30, 2017 at 5:43 AM, Tom Avino> wrote: > >> Terry, >> >> Where and how did you change to textgen? I guess that is what I cannot >> find, I see it in schema.xml, but not sure where to change it. >> >> Thanks >> >> On Tuesday, August 29, 2017 at 6:23:19 PM UTC-4, Terry Brady wrote: >>> >>> Unfortunately, I am not aware of any documentation on how to configure >>> this file for DSpace. >>> >>> We found that the term stemming returned far too many false hits, so I >>> changed several of our fields from "text" to "textgen" and the results were >>> improved. >>> >>> >>> https://github.com/DSpace/DSpace/blob/master/dspace/solr/search/conf/schema.xml#L278-L279 >>> >>> On Tue, Aug 29, 2017 at 1:41 PM, Tom Avino wrote: >>> Since we switched from Dspace 3.3 JSPUI to XMLUI, our search is not working the same. If our users type a specific item CA01234, it will not find this even though it is in one of the metadata fields. I am not sure how SOLR works. I have looked what is in schema.xml. I am unclear which fieldtype is used when a search is done. It looks like each fieldtype can have different parameters. There is "text_ws", "text", "textTight", etc. Where in Dspace do you pick which one to use during a search??? Example >>> positionIncrementGap="100"> >>> ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" /> >>> preserveOriginal="1" *splitOnNumerics="0" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="0"*/> Tom Avino -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com. To post to this group, send email to dspac...@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> -- >>> Terry Brady >>> Applications Programmer Analyst >>> Georgetown University Library Information Technology >>> http://georgetown-university-libraries.github.io/ >>> 425-298-5498 (Seattle, WA) >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "DSpace Technical Support" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to dspace-tech...@googlegroups.com . >> To post to this group, send email to dspac...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/dspace-tech. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Terry Brady > Applications Programmer Analyst > Georgetown University Library Information Technology > http://georgetown-university-libraries.github.io/ > 425-298-5498 (Seattle, WA) > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] SOLR Configuration (Searching)
Tom, Here is my modified schema.xml for DSpace 5x: https://gist.github.com/terrywbrady/097164329b2574b9d6bd4bb8433ca37b I will be curious if this resolves issue for you. If so, we might want to propose this as an enhancement. Terry On Wed, Aug 30, 2017 at 5:43 AM, Tom Avinowrote: > Terry, > > Where and how did you change to textgen? I guess that is what I cannot > find, I see it in schema.xml, but not sure where to change it. > > Thanks > > On Tuesday, August 29, 2017 at 6:23:19 PM UTC-4, Terry Brady wrote: >> >> Unfortunately, I am not aware of any documentation on how to configure >> this file for DSpace. >> >> We found that the term stemming returned far too many false hits, so I >> changed several of our fields from "text" to "textgen" and the results were >> improved. >> >> https://github.com/DSpace/DSpace/blob/master/dspace/solr/ >> search/conf/schema.xml#L278-L279 >> >> On Tue, Aug 29, 2017 at 1:41 PM, Tom Avino wrote: >> >>> Since we switched from Dspace 3.3 JSPUI to XMLUI, our search is not >>> working the same. If our users type a specific item CA01234, it will not >>> find this even though it is in one of the metadata fields. I am not sure >>> how SOLR works. I have looked what is in schema.xml. I am unclear which >>> fieldtype is used when a search is done. It looks like each fieldtype can >>> have different parameters. There is "text_ws", "text", "textTight", etc. >>> Where in Dspace do you pick which one to use during a search??? >>> >>> Example >>> >> positionIncrementGap="100"> >>> >>> >>> >> ignoreCase="true" >>> words="stopwords.txt" >>> enablePositionIncrements="true" >>> /> >>> >> preserveOriginal="1" *splitOnNumerics="0" generateWordParts="1" >>> generateNumberParts="1" catenateWords="1" catenateNumbers="1" >>> catenateAll="0" splitOnCaseChange="0"*/> >>> >>> >>> Tom Avino >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "DSpace Technical Support" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to dspace-tech...@googlegroups.com. >>> To post to this group, send email to dspac...@googlegroups.com. >>> Visit this group at https://groups.google.com/group/dspace-tech. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Terry Brady >> Applications Programmer Analyst >> Georgetown University Library Information Technology >> http://georgetown-university-libraries.github.io/ >> 425-298-5498 <(425)%20298-5498> (Seattle, WA) >> > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- Terry Brady Applications Programmer Analyst Georgetown University Library Information Technology http://georgetown-university-libraries.github.io/ 425-298-5498 (Seattle, WA) -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] SOLR Configuration (Searching)
Terry, Where and how did you change to textgen? I guess that is what I cannot find, I see it in schema.xml, but not sure where to change it. Thanks On Tuesday, August 29, 2017 at 6:23:19 PM UTC-4, Terry Brady wrote: > > Unfortunately, I am not aware of any documentation on how to configure > this file for DSpace. > > We found that the term stemming returned far too many false hits, so I > changed several of our fields from "text" to "textgen" and the results were > improved. > > > https://github.com/DSpace/DSpace/blob/master/dspace/solr/search/conf/schema.xml#L278-L279 > > On Tue, Aug 29, 2017 at 1:41 PM, Tom Avino> wrote: > >> Since we switched from Dspace 3.3 JSPUI to XMLUI, our search is not >> working the same. If our users type a specific item CA01234, it will not >> find this even though it is in one of the metadata fields. I am not sure >> how SOLR works. I have looked what is in schema.xml. I am unclear which >> fieldtype is used when a search is done. It looks like each fieldtype can >> have different parameters. There is "text_ws", "text", "textTight", etc. >> Where in Dspace do you pick which one to use during a search??? >> >> Example >> > positionIncrementGap="100"> >> >> >> > ignoreCase="true" >> words="stopwords.txt" >> enablePositionIncrements="true" >> /> >> > preserveOriginal="1" *splitOnNumerics="0" generateWordParts="1" >> generateNumberParts="1" catenateWords="1" catenateNumbers="1" >> catenateAll="0" splitOnCaseChange="0"*/> >> >> >> Tom Avino >> >> -- >> You received this message because you are subscribed to the Google Groups >> "DSpace Technical Support" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to dspace-tech...@googlegroups.com . >> To post to this group, send email to dspac...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/dspace-tech. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Terry Brady > Applications Programmer Analyst > Georgetown University Library Information Technology > http://georgetown-university-libraries.github.io/ > 425-298-5498 (Seattle, WA) > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] SOLR Configuration (Searching)
Unfortunately, I am not aware of any documentation on how to configure this file for DSpace. We found that the term stemming returned far too many false hits, so I changed several of our fields from "text" to "textgen" and the results were improved. https://github.com/DSpace/DSpace/blob/master/dspace/solr/search/conf/schema.xml#L278-L279 On Tue, Aug 29, 2017 at 1:41 PM, Tom Avinowrote: > Since we switched from Dspace 3.3 JSPUI to XMLUI, our search is not > working the same. If our users type a specific item CA01234, it will not > find this even though it is in one of the metadata fields. I am not sure > how SOLR works. I have looked what is in schema.xml. I am unclear which > fieldtype is used when a search is done. It looks like each fieldtype can > have different parameters. There is "text_ws", "text", "textTight", etc. > Where in Dspace do you pick which one to use during a search??? > > Example > positionIncrementGap="100"> > > > ignoreCase="true" > words="stopwords.txt" > enablePositionIncrements="true" > /> > preserveOriginal="1" *splitOnNumerics="0" generateWordParts="1" > generateNumberParts="1" catenateWords="1" catenateNumbers="1" > catenateAll="0" splitOnCaseChange="0"*/> > > > Tom Avino > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- Terry Brady Applications Programmer Analyst Georgetown University Library Information Technology http://georgetown-university-libraries.github.io/ 425-298-5498 (Seattle, WA) -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
[dspace-tech] SOLR Configuration (Searching)
Since we switched from Dspace 3.3 JSPUI to XMLUI, our search is not working the same. If our users type a specific item CA01234, it will not find this even though it is in one of the metadata fields. I am not sure how SOLR works. I have looked what is in schema.xml. I am unclear which fieldtype is used when a search is done. It looks like each fieldtype can have different parameters. There is "text_ws", "text", "textTight", etc. Where in Dspace do you pick which one to use during a search??? Example Tom Avino -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.