Re: is it possible to save the search query?
Hi Amit, Could you please explain more on how to use a user cookie to construct a session. Thanks and regards, Romita From: Amit Nithian anith...@gmail.com To: solr-user@lucene.apache.org solr-user@lucene.apache.org, Date: 11/08/2012 06:21 PM Subject:Re: is it possible to save the search query? Are you trying to do this in real time or offlline? Wouldn't mining your access logs help? It may help to have your front end application pass in some extra parameters that are not interpreted by Solr but are there for stamping purposes for log analysis. One example could be a user id or user cookie or something in case you have to construct sessions. On Wed, Nov 7, 2012 at 10:01 PM, Romita Saha romita.s...@sg.panasonic.comwrote: Hi, The following is the example; 1st query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data ^2 idstart=0rows=11fl=data,id Next query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data id^2start=0rows=11fl=data,id In the 1st query the the field 'data' is boosted by 2. However may be the user was not satisfied with the response. Thus in the next query he boosted the field 'id' by 2. I want to record both the queries and compare between the two, meaning, what are the changes implemented on the 2nd query which are not present in the previous one. Thanks and regards, Romita Saha From: Otis Gospodnetic otis.gospodne...@gmail.com To: solr-user@lucene.apache.org, Date: 11/08/2012 01:35 PM Subject:Re: is it possible to save the search query? Hi, Compare in what sense? An example will help. Otis -- Performance Monitoring - http://sematext.com/spm On Nov 7, 2012 8:45 PM, Romita Saha romita.s...@sg.panasonic.com wrote: Hi All, Is it possible to record a search query in solr and then compare it with the previous search query? Thanks and regards, Romita Saha
Re: is it possible to save the search query?
Hello,
i think you are asking two questions here - i'll see if i can give you some
simple examples for both
1) how can i pull data from a solr search result set and compare it to
another for analysis?
one way - might be to drive the results in to files and then use xslt to
extract relevant information.
here is an example xslt file that pulls specific fields from a result:
?xml version=1.0 encoding=ISO-8859-1?
xsl:stylesheet version=1.0
xmlns:xsl=http://www.w3.org/1999/XSL/Transform;
xsl:output method=text/
xsl:template match=response/result
xsl:for-each select=doc
xsl:text[/xsl:text
xsl:value-of select=str[@name='itemNo']/
xsl:text]/xsl:text
xsl:text,/xsl:text
xsl:value-of select=float[@name='score']/
xsl:text,/xsl:text
xsl:value-of select=int[@name='rankNo']/
xsl:text,/xsl:text
xsl:value-of select=int[@name='partCnt']/
xsl:text#10;/xsl:text
/xsl:for-each
/xsl:template
/xsl:stylesheet
2) how can i embed data in to a solr query, making it easier to do analysis
in the log files?
here is a simple example that bookmarks or brackets transactions in the
logs - used only during stress testing
#!/bin/bash
TYPE=$1
TAG=$2
if [ $TYPE == 1 ]
then
# beginning
curl -v
http://something:1234/boo/core1/select/?q=partImageURL%3A${TAG}-test-beginversion=2.2start=0rows=777indent=on
else
# end
curl -v
http://something:1234/boo/core1/select/?q=partImageURL%3A${TAG}-test-endversion=2.2start=0rows=777indent=on
fi
hopefully this will give you something to start with.
thx
mark
--
View this message in context:
http://lucene.472066.n3.nabble.com/is-it-possible-to-save-the-search-query-tp4018925p4021315.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: is it possible to save the search query?
Hi Mark,
Thanks a lot for your reply. I do not have much knowledge regarding xslt.
Could you kindly specify how do i use xslt to extract relevant
information. Also how can i drive solr response to a file. Any guidance
regarding this would help.
Thanks and regards,
Romita
From: geeky2 gee...@hotmail.com
To: solr-user@lucene.apache.org,
Date: 11/20/2012 10:55 PM
Subject:Re: is it possible to save the search query?
Hello,
i think you are asking two questions here - i'll see if i can give you
some
simple examples for both
1) how can i pull data from a solr search result set and compare it to
another for analysis?
one way - might be to drive the results in to files and then use xslt to
extract relevant information.
here is an example xslt file that pulls specific fields from a result:
?xml version=1.0 encoding=ISO-8859-1?
xsl:stylesheet version=1.0
xmlns:xsl=http://www.w3.org/1999/XSL/Transform;
xsl:output method=text/
xsl:template match=response/result
xsl:for-each select=doc
xsl:text[/xsl:text
xsl:value-of select=str[@name='itemNo']/
xsl:text]/xsl:text
xsl:text,/xsl:text
xsl:value-of select=float[@name='score']/
xsl:text,/xsl:text
xsl:value-of select=int[@name='rankNo']/
xsl:text,/xsl:text
xsl:value-of select=int[@name='partCnt']/
xsl:text#10;/xsl:text
/xsl:for-each
/xsl:template
/xsl:stylesheet
2) how can i embed data in to a solr query, making it easier to do
analysis
in the log files?
here is a simple example that bookmarks or brackets transactions in the
logs - used only during stress testing
#!/bin/bash
TYPE=$1
TAG=$2
if [ $TYPE == 1 ]
then
# beginning
curl -v
http://something:1234/boo/core1/select/?q=partImageURL%3A$
{TAG}-test-beginversion=2.2start=0rows=777indent=on
else
# end
curl -v
http://something:1234/boo/core1/select/?q=partImageURL%3A$
{TAG}-test-endversion=2.2start=0rows=777indent=on
fi
hopefully this will give you something to start with.
thx
mark
--
View this message in context:
http://lucene.472066.n3.nabble.com/is-it-possible-to-save-the-search-query-tp4018925p4021315.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: is it possible to save the search query?
Hi, Thanks for your guidance. I am unable to figure out what is a doc ID and how can i collect all the doc IDs. Thanks and regards, Romita Saha From: Otis Gospodnetic otis.gospodne...@gmail.com To: solr-user@lucene.apache.org, Date: 11/09/2012 12:33 AM Subject:Re: is it possible to save the search query? Hi, Aha, I think I understand. Yes, you could collect all doc IDs from each query and find the differences. There is nothing in Solr that can find those differences or that would store doc IDs of returned hits in the first place, so you would have to implement this yourself. Sematext's Search Analytics service my be of help here in the sense that all data you need (queries, doc IDs, etc.) are collected, so it would be a matter of providing an API to get the data for off-line analysis. But this data collection+diffing is also something you could implement yourself. One thing to think about - what do you do when a query returns a lrge number of hits. Do you really want/need to get IDs for all of them, or only a page at a time. Otis -- Search Analytics - http://sematext.com/search-analytics/index.html Performance Monitoring - http://sematext.com/spm/index.html On Thu, Nov 8, 2012 at 1:01 AM, Romita Saha romita.s...@sg.panasonic.comwrote: Hi, The following is the example; 1st query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data ^2 idstart=0rows=11fl=data,id Next query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data id^2start=0rows=11fl=data,id In the 1st query the the field 'data' is boosted by 2. However may be the user was not satisfied with the response. Thus in the next query he boosted the field 'id' by 2. I want to record both the queries and compare between the two, meaning, what are the changes implemented on the 2nd query which are not present in the previous one. Thanks and regards, Romita Saha From: Otis Gospodnetic otis.gospodne...@gmail.com To: solr-user@lucene.apache.org, Date: 11/08/2012 01:35 PM Subject:Re: is it possible to save the search query? Hi, Compare in what sense? An example will help. Otis -- Performance Monitoring - http://sematext.com/spm On Nov 7, 2012 8:45 PM, Romita Saha romita.s...@sg.panasonic.com wrote: Hi All, Is it possible to record a search query in solr and then compare it with the previous search query? Thanks and regards, Romita Saha
Re: is it possible to save the search query?
Hi, Document ID would be a field in your document. A unique field that you specify when indexing. You can collect it by telling Solr to return it in the search results by including it in the fl= parameter. Otis -- Performance Monitoring - http://sematext.com/spm/index.html Search Analytics - http://sematext.com/search-analytics/index.html On Mon, Nov 19, 2012 at 9:31 PM, Romita Saha romita.s...@sg.panasonic.comwrote: Hi, Thanks for your guidance. I am unable to figure out what is a doc ID and how can i collect all the doc IDs. Thanks and regards, Romita Saha From: Otis Gospodnetic otis.gospodne...@gmail.com To: solr-user@lucene.apache.org, Date: 11/09/2012 12:33 AM Subject:Re: is it possible to save the search query? Hi, Aha, I think I understand. Yes, you could collect all doc IDs from each query and find the differences. There is nothing in Solr that can find those differences or that would store doc IDs of returned hits in the first place, so you would have to implement this yourself. Sematext's Search Analytics service my be of help here in the sense that all data you need (queries, doc IDs, etc.) are collected, so it would be a matter of providing an API to get the data for off-line analysis. But this data collection+diffing is also something you could implement yourself. One thing to think about - what do you do when a query returns a lrge number of hits. Do you really want/need to get IDs for all of them, or only a page at a time. Otis -- Search Analytics - http://sematext.com/search-analytics/index.html Performance Monitoring - http://sematext.com/spm/index.html On Thu, Nov 8, 2012 at 1:01 AM, Romita Saha romita.s...@sg.panasonic.comwrote: Hi, The following is the example; 1st query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data ^2 idstart=0rows=11fl=data,id Next query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data id^2start=0rows=11fl=data,id In the 1st query the the field 'data' is boosted by 2. However may be the user was not satisfied with the response. Thus in the next query he boosted the field 'id' by 2. I want to record both the queries and compare between the two, meaning, what are the changes implemented on the 2nd query which are not present in the previous one. Thanks and regards, Romita Saha From: Otis Gospodnetic otis.gospodne...@gmail.com To: solr-user@lucene.apache.org, Date: 11/08/2012 01:35 PM Subject:Re: is it possible to save the search query? Hi, Compare in what sense? An example will help. Otis -- Performance Monitoring - http://sematext.com/spm On Nov 7, 2012 8:45 PM, Romita Saha romita.s...@sg.panasonic.com wrote: Hi All, Is it possible to record a search query in solr and then compare it with the previous search query? Thanks and regards, Romita Saha
Re: is it possible to save the search query?
Are you trying to do this in real time or offlline? Wouldn't mining your access logs help? It may help to have your front end application pass in some extra parameters that are not interpreted by Solr but are there for stamping purposes for log analysis. One example could be a user id or user cookie or something in case you have to construct sessions. On Wed, Nov 7, 2012 at 10:01 PM, Romita Saha romita.s...@sg.panasonic.comwrote: Hi, The following is the example; 1st query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data ^2 idstart=0rows=11fl=data,id Next query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data id^2start=0rows=11fl=data,id In the 1st query the the field 'data' is boosted by 2. However may be the user was not satisfied with the response. Thus in the next query he boosted the field 'id' by 2. I want to record both the queries and compare between the two, meaning, what are the changes implemented on the 2nd query which are not present in the previous one. Thanks and regards, Romita Saha From: Otis Gospodnetic otis.gospodne...@gmail.com To: solr-user@lucene.apache.org, Date: 11/08/2012 01:35 PM Subject:Re: is it possible to save the search query? Hi, Compare in what sense? An example will help. Otis -- Performance Monitoring - http://sematext.com/spm On Nov 7, 2012 8:45 PM, Romita Saha romita.s...@sg.panasonic.com wrote: Hi All, Is it possible to record a search query in solr and then compare it with the previous search query? Thanks and regards, Romita Saha
Re: is it possible to save the search query?
You can certainly save the results themselves yourself as well as the explanations for scoring and then compare them yourself. Add debugQuery=true to your query and there will be an explain section that gives all the values used in computing the scores of the top documents. -- Jack Krupansky -Original Message- From: Romita Saha Sent: Wednesday, November 07, 2012 10:01 PM To: solr-user@lucene.apache.org Subject: Re: is it possible to save the search query? Hi, The following is the example; 1st query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data^2 idstart=0rows=11fl=data,id Next query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data id^2start=0rows=11fl=data,id In the 1st query the the field 'data' is boosted by 2. However may be the user was not satisfied with the response. Thus in the next query he boosted the field 'id' by 2. I want to record both the queries and compare between the two, meaning, what are the changes implemented on the 2nd query which are not present in the previous one. Thanks and regards, Romita Saha From: Otis Gospodnetic otis.gospodne...@gmail.com To: solr-user@lucene.apache.org, Date: 11/08/2012 01:35 PM Subject:Re: is it possible to save the search query? Hi, Compare in what sense? An example will help. Otis -- Performance Monitoring - http://sematext.com/spm On Nov 7, 2012 8:45 PM, Romita Saha romita.s...@sg.panasonic.com wrote: Hi All, Is it possible to record a search query in solr and then compare it with the previous search query? Thanks and regards, Romita Saha
Re: is it possible to save the search query?
Hi, Aha, I think I understand. Yes, you could collect all doc IDs from each query and find the differences. There is nothing in Solr that can find those differences or that would store doc IDs of returned hits in the first place, so you would have to implement this yourself. Sematext's Search Analytics service my be of help here in the sense that all data you need (queries, doc IDs, etc.) are collected, so it would be a matter of providing an API to get the data for off-line analysis. But this data collection+diffing is also something you could implement yourself. One thing to think about - what do you do when a query returns a lrge number of hits. Do you really want/need to get IDs for all of them, or only a page at a time. Otis -- Search Analytics - http://sematext.com/search-analytics/index.html Performance Monitoring - http://sematext.com/spm/index.html On Thu, Nov 8, 2012 at 1:01 AM, Romita Saha romita.s...@sg.panasonic.comwrote: Hi, The following is the example; 1st query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data ^2 idstart=0rows=11fl=data,id Next query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data id^2start=0rows=11fl=data,id In the 1st query the the field 'data' is boosted by 2. However may be the user was not satisfied with the response. Thus in the next query he boosted the field 'id' by 2. I want to record both the queries and compare between the two, meaning, what are the changes implemented on the 2nd query which are not present in the previous one. Thanks and regards, Romita Saha From: Otis Gospodnetic otis.gospodne...@gmail.com To: solr-user@lucene.apache.org, Date: 11/08/2012 01:35 PM Subject:Re: is it possible to save the search query? Hi, Compare in what sense? An example will help. Otis -- Performance Monitoring - http://sematext.com/spm On Nov 7, 2012 8:45 PM, Romita Saha romita.s...@sg.panasonic.com wrote: Hi All, Is it possible to record a search query in solr and then compare it with the previous search query? Thanks and regards, Romita Saha
Re: is it possible to save the search query?
I think that solr by him self doesn't store the queries (correct me if I'm wrong, about this) but you can accomplish what you want by processing the solr log (its the only way I think). From the solr log you can get the queries and then process the queries according to your needs, and change the boost parameters in your app o solr config. On Nov 8, 2012, at 11:32 AM, Otis Gospodnetic otis.gospodne...@gmail.com wrote: Hi, Aha, I think I understand. Yes, you could collect all doc IDs from each query and find the differences. There is nothing in Solr that can find those differences or that would store doc IDs of returned hits in the first place, so you would have to implement this yourself. Sematext's Search Analytics service my be of help here in the sense that all data you need (queries, doc IDs, etc.) are collected, so it would be a matter of providing an API to get the data for off-line analysis. But this data collection+diffing is also something you could implement yourself. One thing to think about - what do you do when a query returns a lrge number of hits. Do you really want/need to get IDs for all of them, or only a page at a time. Otis -- Search Analytics - http://sematext.com/search-analytics/index.html Performance Monitoring - http://sematext.com/spm/index.html On Thu, Nov 8, 2012 at 1:01 AM, Romita Saha romita.s...@sg.panasonic.comwrote: Hi, The following is the example; 1st query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data ^2 idstart=0rows=11fl=data,id Next query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data id^2start=0rows=11fl=data,id In the 1st query the the field 'data' is boosted by 2. However may be the user was not satisfied with the response. Thus in the next query he boosted the field 'id' by 2. I want to record both the queries and compare between the two, meaning, what are the changes implemented on the 2nd query which are not present in the previous one. Thanks and regards, Romita Saha From: Otis Gospodnetic otis.gospodne...@gmail.com To: solr-user@lucene.apache.org, Date: 11/08/2012 01:35 PM Subject:Re: is it possible to save the search query? Hi, Compare in what sense? An example will help. Otis -- Performance Monitoring - http://sematext.com/spm On Nov 7, 2012 8:45 PM, Romita Saha romita.s...@sg.panasonic.com wrote: Hi All, Is it possible to record a search query in solr and then compare it with the previous search query? Thanks and regards, Romita Saha 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS... CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION http://www.uci.cu http://www.facebook.com/universidad.uci http://www.flickr.com/photos/universidad_uci 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS... CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION http://www.uci.cu http://www.facebook.com/universidad.uci http://www.flickr.com/photos/universidad_uci
Re: is it possible to save the search query?
Hi, Compare in what sense? An example will help. Otis -- Performance Monitoring - http://sematext.com/spm On Nov 7, 2012 8:45 PM, Romita Saha romita.s...@sg.panasonic.com wrote: Hi All, Is it possible to record a search query in solr and then compare it with the previous search query? Thanks and regards, Romita Saha
Re: is it possible to save the search query?
Hi, The following is the example; 1st query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data^2 idstart=0rows=11fl=data,id Next query: http://localhost:8983/solr/db/select/?defType=dismaxdebugQuery=onq=cashier2qf=data id^2start=0rows=11fl=data,id In the 1st query the the field 'data' is boosted by 2. However may be the user was not satisfied with the response. Thus in the next query he boosted the field 'id' by 2. I want to record both the queries and compare between the two, meaning, what are the changes implemented on the 2nd query which are not present in the previous one. Thanks and regards, Romita Saha From: Otis Gospodnetic otis.gospodne...@gmail.com To: solr-user@lucene.apache.org, Date: 11/08/2012 01:35 PM Subject:Re: is it possible to save the search query? Hi, Compare in what sense? An example will help. Otis -- Performance Monitoring - http://sematext.com/spm On Nov 7, 2012 8:45 PM, Romita Saha romita.s...@sg.panasonic.com wrote: Hi All, Is it possible to record a search query in solr and then compare it with the previous search query? Thanks and regards, Romita Saha
