Re: Synonym Dictionary
thanks Russ, good link Richard, If MySQL cannot do it natively then you have this functionality also built right into CF with SOLR. Check this http://bloggeraroundthecorner.blogspot.com/2009/08/searching-for-synonyms.html http://bloggeraroundthecorner.blogspot.com/2009/08/searching-for-synonyms.html Russ On Sat, Apr 30, 2011 at 7:42 PM, andy matthews li...@commadelimited.comwrote: ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344210 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Synonym Dictionary
Just to confirm Dave's suggestion, Solr can do this, yes. If you get particularly down and dirty with your configuration, you can have individual fields in your search collection each with their own set of synonyms (amongst many other things). This is not possible using the native CF9 search and index methods and the setup would most certainly take some learning curve time (perhaps overkill for your problem). You can, AFAIK, setup a global synonym list in CF9 that should work fairly smoothly. Here's hoping 9.5 brings some extended Solr goodness :) Dominic On 1 May 2011 18:06, Dave Watts dwa...@figleaf.com wrote: hi dave, had a look at Google Search Appliance and thought it was only for searching our web pages, do you think i could possibly utilize it in some way to get the synonyms for my database searches? Well, as much as I love the appliance, I wasn't really recommending it for your use, as it's fairly expensive if you were to buy it just to solve this one problem. I was just pointing out that Solr/Lucene probably have similar synonym features. That said, yes, the appliance can index all sorts of content in addition to web pages. Database content, filesystem content, CMS/DMS content, Google Apps content, etc, etc. So, you could very easily build a search against your MySQL data that would use synonyms - in fact, we actually do that in the GSA classes I teach. If you have any questions about the Google Search Appliance, feel free to contact me off-list. Fig Leaf Software is a Google Enterprise partner - and the first certified Enterprise training partner - and we sell, implement, and train people on the appliance and on Google Apps. We provide quite a bit of training on behalf of Google, to other Google Enterprise partners, as well as to the public at large. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344154 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Synonym Dictionary
hi dave, had a look at Google Search Appliance and thought it was only for searching our web pages, do you think i could possibly utilize it in some way to get the synonyms for my database searches? Well, as much as I love the appliance, I wasn't really recommending it for your use, as it's fairly expensive if you were to buy it just to solve this one problem. I was just pointing out that Solr/Lucene probably have similar synonym features. That said, yes, the appliance can index all sorts of content in addition to web pages. Database content, filesystem content, CMS/DMS content, Google Apps content, etc, etc. So, you could very easily build a search against your MySQL data that would use synonyms - in fact, we actually do that in the GSA classes I teach. If you have any questions about the Google Search Appliance, feel free to contact me off-list. Fig Leaf Software is a Google Enterprise partner - and the first certified Enterprise training partner - and we sell, implement, and train people on the appliance and on Google Apps. We provide quite a bit of training on behalf of Google, to other Google Enterprise partners, as well as to the public at large. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344114 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Synonym Dictionary
One way (probably isn't the best, but it's workable depending on your site traffic) is to take and create a relationship table. I.E. Cat is related to Feline and Cat is related to Tiger, etc. The issue is.. What do you mean by Related words? How broad of a scope are you taking about? Is Cat related to Dog (both animals) or is Cat related to Cities (housecats)? You need to define your scope more clearly to even see if this is feasible. On Sat, Apr 30, 2011 at 8:26 AM, Richard White rich...@j7is.co.uk wrote: Hi, We use ColdFusion and MySQL (innodb) in our applications and when users search for data we want it to also find records containing related words / synonyms. What is the best way to achieve this? Thanks ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344083 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Synonym Dictionary
We use ColdFusion and MySQL (innodb) in our applications and when users search for data we want it to also find records containing related words / synonyms. What is the best way to achieve this? A good enterprise search solution can do this for you. For example, the Google Search Appliance can do this automatically. I wouldn't be surprised if Solr/Lucene has similar functionality. If you're strictly doing this through database queries, it's up to you to manage a synonym list. This isn't something that relational databases do especially well, either. But if your synonym list is fairly small, it might do it sufficiently well. You could conceivably use a full-text indexing solution against your synonym list, then use the returned record to filter your database search. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344085 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Synonym Dictionary
I'm not up on MySQL, but in MS SQL I use the FULLTEXT functionality. I see this is also available on MySQL: http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html -Original Message- From: Richard White [mailto:rich...@j7is.co.uk] Sent: 30 April 2011 16:27 To: cf-talk Subject: Synonym Dictionary Hi, We use ColdFusion and MySQL (innodb) in our applications and when users search for data we want it to also find records containing related words / synonyms. What is the best way to achieve this? Thanks ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344086 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Synonym Dictionary
thanks for your reply... yes your example is exactly what we want, except as you have indicated the scope is unfortunately large! we basically need our application linked to some form of existing thesaurus that we can query for synonyms, and then in our code make sure it also searches for those words. Basically, our software is used for medical data, and we are finding users typing words such as 'mother' when another user entered the data as 'maternal', therefore the data is not being found. This happens on various different scenarios and is why we are thinking rather than attempting to build our own, we could possibly utilize an existing synonym dictionary plug-in ... if one exists! thanks for any pointers One way (probably isn't the best, but it's workable depending on your site traffic) is to take and create a relationship table. I.E. Cat is related to Feline and Cat is related to Tiger, etc. The issue is.. What do you mean by Related words? How broad of a scope are you taking about? Is Cat related to Dog (both animals) or is Cat related to Cities (housecats)? You need to define your scope more clearly to even see if this is feasible. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344087 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Synonym Dictionary
To create a catalog or do the synonym lookup on the fly you could use Wolfram: http://www.wolframalpha.com/input/?i=synonym+for+bike They have a (commercial) API. The difference with Wolfram as opposed to a Google search is that Wolfram will compute and return the results directly to you rather than provide links to sites with potential results. Stefan On 30 Apr 2011, at 16:26, Richard White wrote: Hi, We use ColdFusion and MySQL (innodb) in our applications and when users search for data we want it to also find records containing related words / synonyms. What is the best way to achieve this? Thanks ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344088 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Synonym Dictionary
thanks for your reply, unfortunately fulltext is only available in MyISAM engine and not InnoDB which our applications uses. I'm not up on MySQL, but in MS SQL I use the FULLTEXT functionality. I see this is also available on MySQL: http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html Hi, We use ColdFusion and MySQL (innodb) in our applications and when users search for data we want it to also find records containing related words / synonyms. What is the best way to achieve this? Thanks ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344089 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Synonym Dictionary
Wolfram sounds like a good fit in that case. I can refer you to the right people if you like. Their pricing is, I found, reasonable. S On 30 Apr 2011, at 16:41, Richard White wrote: thanks for your reply... yes your example is exactly what we want, except as you have indicated the scope is unfortunately large! we basically need our application linked to some form of existing thesaurus that we can query for synonyms, and then in our code make sure it also searches for those words. Basically, our software is used for medical data, and we are finding users typing words such as 'mother' when another user entered the data as 'maternal', therefore the data is not being found. This happens on various different scenarios and is why we are thinking rather than attempting to build our own, we could possibly utilize an existing synonym dictionary plug-in ... if one exists! thanks for any pointers ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344090 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Synonym Dictionary
hi dave, had a look at Google Search Appliance and thought it was only for searching our web pages, do you think i could possibly utilize it in some way to get the synonyms for my database searches? thanks We use ColdFusion and MySQL (innodb) in our applications and when users search for data we want it to also find records containing related words / synonyms. What is the best way to achieve this? A good enterprise search solution can do this for you. For example, the Google Search Appliance can do this automatically. I wouldn't be surprised if Solr/Lucene has similar functionality. If you're strictly doing this through database queries, it's up to you to manage a synonym list. This isn't something that relational databases do especially well, either. But if your synonym list is fairly small, it might do it sufficiently well. You could conceivably use a full-text indexing solution against your synonym list, then use the returned record to filter your database search. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344091 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Synonym Dictionary
wow, had a quick look and instantly impressed. yes would appreciate the referral, thanks Wolfram sounds like a good fit in that case. I can refer you to the right people if you like. Their pricing is, I found, reasonable. S On 30 Apr 2011, at 16:41, Richard White wrote: ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344092 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Synonym Dictionary
You could look into using SOUNDEX in MySQL. -Original Message- From: Richard White [mailto:rich...@j7is.co.uk] Sent: Saturday, April 30, 2011 10:27 AM To: cf-talk Subject: Synonym Dictionary Hi, We use ColdFusion and MySQL (innodb) in our applications and when users search for data we want it to also find records containing related words / synonyms. What is the best way to achieve this? Thanks ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344096 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Synonym Dictionary
Richard, If MySQL cannot do it natively then you have this functionality also built right into CF with SOLR. Check this http://bloggeraroundthecorner.blogspot.com/2009/08/searching-for-synonyms.html http://bloggeraroundthecorner.blogspot.com/2009/08/searching-for-synonyms.html Russ On Sat, Apr 30, 2011 at 7:42 PM, andy matthews li...@commadelimited.comwrote: You could look into using SOUNDEX in MySQL. -Original Message- From: Richard White [mailto:rich...@j7is.co.uk] Sent: Saturday, April 30, 2011 10:27 AM To: cf-talk Subject: Synonym Dictionary Hi, We use ColdFusion and MySQL (innodb) in our applications and when users search for data we want it to also find records containing related words / synonyms. What is the best way to achieve this? Thanks ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344098 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
