Re: mysolr python client
sounds great for a python project i'm involved in rigth now. I'll take a deeper look on it. thx marco 2011/11/30 Marco Martinez > Hi all, > > For anyone interested, recently I've been using a new Solr client for > Python. It's easy and pretty well documented. If you're interested its site > is: *http://mysolr.redtuna.org/* > * > * > bye! > > Marco Martínez Bautista > http://www.paradigmatecnologico.com > Avenida de Europa, 26. Ática 5. 3ª Planta > 28224 Pozuelo de Alarcón > Tel.: 91 352 59 42 >
Re: solr connection question
ok please don't forget it :) 2010/7/8 Ruben Abad > Jorl, ok tendré que modificar mi petición de vacaciones :( > Rubén Abad > > > On Thu, Jul 8, 2010 at 2:46 PM, ZAROGKIKAS,GIORGOS < > g.zarogki...@multirama.gr> wrote: > > > Hi solr users > > > > I need to know how solr manages the connections when we make a > > request(select update commit) > > Is there any connection pooling or an article to learn about it > connection > > management?? > > How can I log in a file the connections solr server > > > > I have setup my solr 1.4 with tomcat > > > > Thanks in advance > > > > > > > > >
Re: Faceted Search
if you are querying using a http request you can add these two parameters: facet=true facet.field=field_for_faceting and optionally this one to set the max number of facets: facet.limit=facet_limit I don't know if it's what you need... On Fri, Apr 17, 2009 at 6:17 AM, Sajith Weerakoon wrote: > Hi all, > > Can someone of you tell me how to implement a faceted search? > > > > Thanks, > > Regards, > > Sajith Vimukthi Weerakoon. > > > >
Re: indexing txt file
but you need to index the text inside these files, right?. you need to read the text from file and include it into a field into the XML (of course this field must be defined in the schema). you can do it using a script and post then the XML to Solr. what amount/rate of generated text files are you thinking about? On Tue, Apr 14, 2009 at 7:07 PM, Alex Vu wrote: > I just want to be able to index my text file, and other files that carries > the same format but with different IP address, ports, ect. > > I will have the traffic flow running in real-time. Do you think Solr will > be able to index a bunch of my text files in real time? > > On Tue, Apr 14, 2009 at 9:35 AM, Alejandro Gonzalez < > alejandrogonzalezd...@gmail.com> wrote: > > > and i'm not sure of understanding what are u trying to do, but maybe you > > should define a text field and fill it with the text in each file for > > indexing the text in them, or maybe a path to that file if that's what u > > want. > > > > On Tue, Apr 14, 2009 at 6:28 PM, Shalin Shekhar Mangar < > > shalinman...@gmail.com> wrote: > > > > > On Tue, Apr 14, 2009 at 9:44 PM, Alex Vu wrote: > > > > > > > > > > > *schema file is * > > > > > > > > > > > > http://www.w3.org/2001/XMLSchema";> > > > > > > > > > > > > > > > > > > > > > > > > > > > type="xs:string" use="required"/> > > > > type="xs:string" > > > > use="required"/> > > > > > > > use="required"/> > > > > > > > type="xs:string" use="required"/> > > > > > > > use="required"/> > > > > > > > use="required"/> > > > > > > > type="xs:string" use="required"/> > > > > > > > use="required"/> > > > > > > type="xs:string" > > > > use="required"/> > > > > > > > type="xs:string" use="required"/> > > > > > > > use="required"/> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can someone please show me where do I put these files? I'm aware > that > > > the > > > > schema.xsd file goes into the directory conf. What about my xml file, > > and > > > > txt file? > > > > > > > > > > Alex, the Solr schema is not the usual XML Schema (xsd). It is an xml > > file > > > which describes the fields, their analyzers, tokenizers, copyFields, > > > default > > > search field etc. > > > > > > Look into the example schema supplied by Solr (inside > example/solr/conf) > > > directory and modify it according to your needs. > > > > > > -- > > > Regards, > > > Shalin Shekhar Mangar. > > > > > >
Re: indexing txt file
and i'm not sure of understanding what are u trying to do, but maybe you should define a text field and fill it with the text in each file for indexing the text in them, or maybe a path to that file if that's what u want. On Tue, Apr 14, 2009 at 6:28 PM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > On Tue, Apr 14, 2009 at 9:44 PM, Alex Vu wrote: > > > > > *schema file is * > > > > > > http://www.w3.org/2001/XMLSchema";> > > > > > > > > > > > > > type="xs:string" use="required"/> > > > use="required"/> > > > use="required"/> > > > type="xs:string" use="required"/> > > > use="required"/> > > > use="required"/> > > > type="xs:string" use="required"/> > > > use="required"/> > > type="xs:string" > > use="required"/> > > > type="xs:string" use="required"/> > > > use="required"/> > > > > > > > > > > > > > > > > > > Can someone please show me where do I put these files? I'm aware that > the > > schema.xsd file goes into the directory conf. What about my xml file, and > > txt file? > > > > Alex, the Solr schema is not the usual XML Schema (xsd). It is an xml file > which describes the fields, their analyzers, tokenizers, copyFields, > default > search field etc. > > Look into the example schema supplied by Solr (inside example/solr/conf) > directory and modify it according to your needs. > > -- > Regards, > Shalin Shekhar Mangar. >
Re: indexing txt file
now you should post (http post) your xml file (the schema must be in conf folder) to the url in wich it's supossed you have deployed Solr. Don forget to post a commit command after that or you won't see the results: The commit command it's just an xml this way: On Tue, Apr 14, 2009 at 6:14 PM, Alex Vu wrote: > Hi all, > I'm trying to use solr1.3 and trying to index a text file. I wrote a > schema.xsd and a xml file. > > *The content of my text file is * > #src dstprotook > sportdportpktsbytesflowsfirst >atest > 192.168.220.13526.147.238.1466 13283980 > 6 463 1 1237333861.4657640001237333861.664701000 > > *schema file is * > > > http://www.w3.org/2001/XMLSchema";> > > > > > > type="xs:string" use="required"/> > use="required"/> > use="required"/> > type="xs:string" use="required"/> > use="required"/> > use="required"/> > type="xs:string" use="required"/> > use="required"/> > use="required"/> > type="xs:string" use="required"/> > use="required"/> > > > > > > > > > *and my xml file is * > > > http://www.w3.org/2001/XMLSchema-instance"; > > xsi:noNamespaceSchemaLocation="C:\DOCUME~1\tpham\Desktop\networkTraffic.xsd"> > protocolPortNumber="6" ok="1" sourcePort="32439" destinationPort="80" > packets="6" bytes="463" flows="1" initialTimestamp="1237963861.465764000" > terminationTimestamp="1237963861.664701000"/> > protocolPortNumber="17" ok="1" sourcePort="32439" destinationPort="80" > packets="6" bytes="463" flows="1" initialTimestamp="1237963861.465764000" > terminationTimestamp="1237963861.664701000"/> > protocolPortNumber="6" ok="1" sourcePort="32139" destinationPort="80" > packets="6" bytes="463" flows="1" initialTimestamp="1237963861.465764000" > terminationTimestamp="1237963861.664701000"/> > protocolPortNumber="6" ok="1" sourcePort="32839" destinationPort="80" > packets="6" bytes="463" flows="1" initialTimestamp="1237963861.465764000" > terminationTimestamp="1237963861.664701000"/> > protocolPortNumber="17" ok="1" sourcePort="32839" destinationPort="80" > packets="6" bytes="463" flows="1" initialTimestamp="1237963861.465764000" > terminationTimestamp="1237963861.664701000"/> > protocolPortNumber="17" ok="1" sourcePort="32439" destinationPort="80" > packets="6" bytes="463" flows="1" initialTimestamp="1237963861.465764000" > terminationTimestamp="1237963861.664701000"/> > protocolPortNumber="6" ok="1" sourcePort="36839" destinationPort="80" > packets="6" bytes="463" flows="1" initialTimestamp="1237963861.465764000" > terminationTimestamp="1237963861.664701000"/> > protocolPortNumber="6" ok="1" sourcePort="32839" destinationPort="80" > packets="6" bytes="463" flows="1" initialTimestamp="1237963861.465764000" > terminationTimestamp="1237963861.664701000"/> > > > > > Can someone please show me where do I put these files? I'm aware that the > schema.xsd file goes into the directory conf. What about my xml file, and > txt file? > > Thank you, > Alex > > > On Tue, Apr 14, 2009 at 12:37 AM, Alejandro Gonzalez < > alejandrogonzalezd...@gmail.com> wrote: > > > you should construct the xml containing the fields defined in your > > schema.xml and give them the values from the text files. for example if > you > > have an schema defining two fields "title" and "text" you should > construct > > an xml with a field "title" and its value and another called "text" > > containing the body of your doc. then you can post it to Solr you have > > deployed and make a commit an it's done. it's possible to construct an > xml > > defining more than jus t a doc > > > > > > > > > > "doc1 title" > > "doc1 text" > > > > . > > . > > . > > > > "docn title" > > "docn text" > > > > > > > > > > > > 2009/4/14 Noble Paul നോബിള് नोब्ळ् > > > > > what is the cntent of your text file? > > > Solr does not directly index files > > > --Noble > > > > > > On Tue, Apr 14, 2009 at 3:54 AM, Alex Vu wrote: > > > > Hi all, > > > > > > > > Currently I wrote an xml file and schema.xml file. What is the next > > step > > > to > > > > index a txt file? Where should I put my txt file I want to index? > > > > > > > > thank you, > > > > Alex V. > > > > > > > > > > > > > > > > -- > > > --Noble Paul > > > > > >
Re: indexing txt file
you should construct the xml containing the fields defined in your schema.xml and give them the values from the text files. for example if you have an schema defining two fields "title" and "text" you should construct an xml with a field "title" and its value and another called "text" containing the body of your doc. then you can post it to Solr you have deployed and make a commit an it's done. it's possible to construct an xml defining more than jus t a doc "doc1 title" "doc1 text" . . . "docn title" "docn text" 2009/4/14 Noble Paul നോബിള് नोब्ळ् > what is the cntent of your text file? > Solr does not directly index files > --Noble > > On Tue, Apr 14, 2009 at 3:54 AM, Alex Vu wrote: > > Hi all, > > > > Currently I wrote an xml file and schema.xml file. What is the next step > to > > index a txt file? Where should I put my txt file I want to index? > > > > thank you, > > Alex V. > > > > > > -- > --Noble Paul >
Re: Quick Indexing Method???
What about building an XML with text fields as everyones does ? :) On Wed, Apr 1, 2009 at 6:17 PM, Alex Vu wrote: > Hello, > > I am new to Solr. I looked at getting started document. Can somebody show > me how to index text file. I've tried other method, it just takes too > much > time. > > I am aware that Solr take XML files. I'm trying to find the > *quickiest*method to index text, binary, or pcap file. Preferably, > text file. > > Thanx, > Alex V. >
Re: How do I accomplish this (semi-)complicated setup?
try using db for permission management and when u want to make a rep public u just have to add it's id or name to everyuser permissions field. i think you don't need to add any "is_public" field to index, just an id or name field in wich the indexed doc is.So you can pre-filter the reps quering the db obtaining the reps for wich user has permissions and adding this restrictions to the solr query. this way you can't change reps'permissions without re-indexing. so the query for solr if the current user is allowed for search in the 1 and 2 reps should be something like ...rep_id:1OR2... Alex On Wed, Mar 25, 2009 at 8:06 PM, Jesper Nøhr wrote: > OK, we're getting closer. I just have two final questions regarding this > then: > > 1. This would also include all the public repositories, right? If so, > how would such a query look? Some kind of is_public:true AND ...? > > 2. When a repository is made public, the is_public property in the > Solr index needs to reflect this. How can such an update be made > without having to purge and re-index? > > > Jesper > > > On Wed, Mar 25, 2009 at 6:29 PM, Alejandro Gonzalez > wrote: > > ok so u can create a table in a DB where you have a row foreach user and > a > > field with the reps he/she can access. Then you just have to take a look > on > > the db and include the repository name in the index. so you just have to > > control (using query parameters) if the query is done for the right reps > for > > that user. > > > > is it good for u? > > > > > > > > On Wed, Mar 25, 2009 at 6:20 PM, Jesper Nøhr wrote: > > > >> Hm, I must be missing something, then. > >> > >> Consider this. > >> > >> There are three repositories, A and B, C. There are two users, U1 and > U2. > >> > >> Repository A is public, while B and C are private. Only U1 can access > >> B. No one can access C. > >> > >> I index this data, such that Is_Private is true for B. > >> > >> Now, when U2 searches, he will only see data for repo A. This is > correct. > >> > >> When U1 searches, what happens? AFAIK, he will also only see data for > >> A, unless we specify Is_Private:True, but then he will only see data > >> for B (and C, which he doesn't have access to.) > >> > >> Secondly, say we grant U2 access to B. How do we tell Solr that he can > >> see it, then? > >> > >> Sorry if I'm not making much sense here, but I'm quite confused. > >> > >> > >> Jesper > >> > >> > >> > >> On Wed, Mar 25, 2009 at 6:13 PM, Alejandro Gonzalez > >> wrote: > >> > i can't see the problem about that. you can manage your users using a > DB > >> and > >> > keep there the permissions they could have, and create or erase users > >> > without problems. you just have to manage a "working index" field for > >> each > >> > user with repositories' ids he can access. or u can create several > >> indexes > >> > and a users solr index with a multi-valued field with the indexes the > >> user > >> > can access. > >> > > >> > if then u want to turn a private repository into public u just have to > >> > change the permissions field in your DB or users' index. > >> > > >> > On Wed, Mar 25, 2009 at 6:02 PM, Jesper Nøhr > wrote: > >> > > >> >> On Wed, Mar 25, 2009 at 5:57 PM, Eric Pugh > >> >> wrote: > >> >> > You could index the user name or ID, and then in your application > add > >> as > >> >> > filter the username as you pass the query back to Solr. Maybe have > a > >> >> > access_type that is Public or Private, and then for public searches > >> only > >> >> > include the ones that meet the access_type of Public. > >> >> > >> >> That makes sense. Two questions on that: > >> >> > >> >> 1. More than one user can have access to a repository, so how would > >> >> that work? Also, if a user is added/removed, what's the best way to > >> >> keep that in sync? > >> >> > >> >> 2. In the event that a repository that is private, is made public, > how > >> >> easy would it be to run an "UPDATE" so to speak? > >> >> > >> >> > >> >> Jesper > >> >> > >> >>
Re: How do I accomplish this (semi-)complicated setup?
ok so u can create a table in a DB where you have a row foreach user and a field with the reps he/she can access. Then you just have to take a look on the db and include the repository name in the index. so you just have to control (using query parameters) if the query is done for the right reps for that user. is it good for u? On Wed, Mar 25, 2009 at 6:20 PM, Jesper Nøhr wrote: > Hm, I must be missing something, then. > > Consider this. > > There are three repositories, A and B, C. There are two users, U1 and U2. > > Repository A is public, while B and C are private. Only U1 can access > B. No one can access C. > > I index this data, such that Is_Private is true for B. > > Now, when U2 searches, he will only see data for repo A. This is correct. > > When U1 searches, what happens? AFAIK, he will also only see data for > A, unless we specify Is_Private:True, but then he will only see data > for B (and C, which he doesn't have access to.) > > Secondly, say we grant U2 access to B. How do we tell Solr that he can > see it, then? > > Sorry if I'm not making much sense here, but I'm quite confused. > > > Jesper > > > > On Wed, Mar 25, 2009 at 6:13 PM, Alejandro Gonzalez > wrote: > > i can't see the problem about that. you can manage your users using a DB > and > > keep there the permissions they could have, and create or erase users > > without problems. you just have to manage a "working index" field for > each > > user with repositories' ids he can access. or u can create several > indexes > > and a users solr index with a multi-valued field with the indexes the > user > > can access. > > > > if then u want to turn a private repository into public u just have to > > change the permissions field in your DB or users' index. > > > > On Wed, Mar 25, 2009 at 6:02 PM, Jesper Nøhr wrote: > > > >> On Wed, Mar 25, 2009 at 5:57 PM, Eric Pugh > >> wrote: > >> > You could index the user name or ID, and then in your application add > as > >> > filter the username as you pass the query back to Solr. Maybe have a > >> > access_type that is Public or Private, and then for public searches > only > >> > include the ones that meet the access_type of Public. > >> > >> That makes sense. Two questions on that: > >> > >> 1. More than one user can have access to a repository, so how would > >> that work? Also, if a user is added/removed, what's the best way to > >> keep that in sync? > >> > >> 2. In the event that a repository that is private, is made public, how > >> easy would it be to run an "UPDATE" so to speak? > >> > >> > >> Jesper > >> > >> > On Mar 25, 2009, at 12:52 PM, Jesper Nøhr wrote: > >> > > >> >> Hi list, > >> >> > >> >> I've finally settled on Solr, seeing as it has almost everything I > >> >> could want out of the box. > >> >> > >> >> My setup is a complicated one. It will serve as the search backend on > >> >> Bitbucket.org, a mercurial hosting site. We have literally thousands > >> >> of code repositories, as well as users and other data. All this needs > >> >> to be indexed. > >> >> > >> >> The complication comes in when we have private repositories. Only > >> >> select users have access to these, but we still need to index them. > >> >> > >> >> How would I go about accomplishing this? I can't think of a clean way > to > >> >> do it. > >> >> > >> >> Any pointers much appreciated. > >> >> > >> >> > >> >> Jesper > >> > > >> > - > >> > Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | > >> > http://www.opensourceconnections.com > >> > Free/Busy: http://tinyurl.com/eric-cal > >> > > >> > > >> > > >> > > >> > > >> > > >
Re: How do I accomplish this (semi-)complicated setup?
i can't see the problem about that. you can manage your users using a DB and keep there the permissions they could have, and create or erase users without problems. you just have to manage a "working index" field for each user with repositories' ids he can access. or u can create several indexes and a users solr index with a multi-valued field with the indexes the user can access. if then u want to turn a private repository into public u just have to change the permissions field in your DB or users' index. On Wed, Mar 25, 2009 at 6:02 PM, Jesper Nøhr wrote: > On Wed, Mar 25, 2009 at 5:57 PM, Eric Pugh > wrote: > > You could index the user name or ID, and then in your application add as > > filter the username as you pass the query back to Solr. Maybe have a > > access_type that is Public or Private, and then for public searches only > > include the ones that meet the access_type of Public. > > That makes sense. Two questions on that: > > 1. More than one user can have access to a repository, so how would > that work? Also, if a user is added/removed, what's the best way to > keep that in sync? > > 2. In the event that a repository that is private, is made public, how > easy would it be to run an "UPDATE" so to speak? > > > Jesper > > > On Mar 25, 2009, at 12:52 PM, Jesper Nøhr wrote: > > > >> Hi list, > >> > >> I've finally settled on Solr, seeing as it has almost everything I > >> could want out of the box. > >> > >> My setup is a complicated one. It will serve as the search backend on > >> Bitbucket.org, a mercurial hosting site. We have literally thousands > >> of code repositories, as well as users and other data. All this needs > >> to be indexed. > >> > >> The complication comes in when we have private repositories. Only > >> select users have access to these, but we still need to index them. > >> > >> How would I go about accomplishing this? I can't think of a clean way to > >> do it. > >> > >> Any pointers much appreciated. > >> > >> > >> Jesper > > > > - > > Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | > > http://www.opensourceconnections.com > > Free/Busy: http://tinyurl.com/eric-cal > > > > > > > > > > >
Re: How do I accomplish this (semi-)complicated setup?
you can even create separated indexes for private or public access if u need (and place them in separated machines), but i think Eric's suggestion is the best and easier On Wed, Mar 25, 2009 at 5:52 PM, Jesper Nøhr wrote: > Hi list, > > I've finally settled on Solr, seeing as it has almost everything I > could want out of the box. > > My setup is a complicated one. It will serve as the search backend on > Bitbucket.org, a mercurial hosting site. We have literally thousands > of code repositories, as well as users and other data. All this needs > to be indexed. > > The complication comes in when we have private repositories. Only > select users have access to these, but we still need to index them. > > How would I go about accomplishing this? I can't think of a clean way to do > it. > > Any pointers much appreciated. > > > Jesper >
