Using Solr from Github or SVN

2013-03-21 Thread Furkan KAMACI
I want to branch Solr (latest version) at my local and implement some custom codes. After some time(maybe every month) I will merge my code with Solr. However There is code at SVN and Github for Solr and I see that they are not exactly synchronous. Which one do you suggest, do you think that if

Re: Using Solr from Github or SVN

2013-03-21 Thread Furkan KAMACI
AS - www.cominvent.com Solr Training - www.solrtraining.com 21. mars 2013 kl. 10:31 skrev Furkan KAMACI furkankam...@gmail.com: I want to branch Solr (latest version) at my local and implement some custom codes. After some time(maybe every month) I will merge my code with Solr. However There is code at SVN

How can I compile and debug Solr from source code?

2013-03-21 Thread Furkan KAMACI
I use Intellij Idea 12 and Solr 4.1 on a Centos 6.4 64 bit computer. I have opened Solr source code at Intellij IDEA as explained documentation. I want to deploy Solr into Tomcat 7. When I open the project there are configurations set previosly (I used ant idea command before I open the project)

Re: How can I compile and debug Solr from source code?

2013-03-21 Thread Furkan KAMACI
mentioned tomcat. Any ideas? 2013/3/21 Shawn Heisey s...@elyograg.org On 3/21/2013 6:56 AM, Furkan KAMACI wrote: I use Intellij Idea 12 and Solr 4.1 on a Centos 6.4 64 bit computer. I have opened Solr source code at Intellij IDEA as explained documentation. I want to deploy Solr

Re: How can I compile and debug Solr from source code?

2013-03-21 Thread Furkan KAMACI
in IntelliJ too, and for tight development loops, I spend my time doing that instead of running full on Solr. Erik On Mar 21, 2013, at 05:56 , Furkan KAMACI wrote: I use Intellij Idea 12 and Solr 4.1 on a Centos 6.4 64 bit computer. I have opened Solr source code at Intellij

Re: How can I compile and debug Solr from source code?

2013-03-21 Thread Furkan KAMACI
I mean I need that: There is a .war file shipped with Solr source code. How can I regenerate (build my code and generate a .war file) as like that? I will deploy it to Tomcat then? 2013/3/22 Furkan KAMACI furkankam...@gmail.com Your mentioned suggestion is for only example application? Can I

Could not load config for solrconfig.xml

2013-03-21 Thread Furkan KAMACI
I run ant idea command for Solr 4.1.0 and opened source code within Intellij IDEA 12.0.4 and I use Centos 6.4 at my 64 bit computer. I debugged JettySolrRunner (I don't know, I think this is the way to run Solt with Embedd Jetty within my Intellij IDEA.) However I get that error: SEVERE: Unable

Re: How can I compile and debug Solr from source code?

2013-03-21 Thread Furkan KAMACI
Steve On Mar 21, 2013, at 7:38 PM, Furkan KAMACI furkankam...@gmail.com wrote: I mean I need that: There is a .war file shipped with Solr source code. How can I regenerate (build my code and generate a .war file) as like that? I will deploy it to Tomcat then? 2013/3/22 Furkan KAMACI

Re: Could not load config for solrconfig.xml

2013-03-21 Thread Furkan KAMACI
Shoukd I create a collection1 folder as like in the example? On the other hand if I use .war tı deploy how can I resolve that problem too? 2013/3/22 Furkan KAMACI furkankam...@gmail.com I run ant idea command for Solr 4.1.0 and opened source code within Intellij IDEA 12.0.4 and I use Centos

Using Solr For a Real Search Engine

2013-03-22 Thread Furkan KAMACI
If I want to use Solr in a web search engine what kind of strategies should I follow about how to run Solr. I mean I can run it via embedded jetty or use war and deploy to a container? You should consider that I will have heavy work load on my Solr.

NoSuchMethodError updateDocument

2013-03-22 Thread Furkan KAMACI
I use Solr 4.1.0 and Nutch 2.1, Java 1.7.0_17, Tomcat 7.0, Intellij IDEA 12.with a Centos 6.4 at my 64 bit computer. I run that command succesfully: bin/nutch solrindex http://localhost:8080/solr -index However when I run that command: bin/nutch solrindex http://localhost:8080/solr -reindex I

Re: NoSuchMethodError updateDocument

2013-03-22 Thread Furkan KAMACI
:} zkClientTimeout=${zkClientTimeout:15000} core name=collection1 instanceDir=collection1/ /cores On the other hand I run it from my tomcat without using example embedded jetty start.jar. Any ideas? 2013/3/22 Furkan KAMACI furkankam...@gmail.com I use Solr 4.1.0 and Nutch 2.1, Java

Re: NoSuchMethodError updateDocument

2013-03-23 Thread Furkan KAMACI
? BTW: What was your question here regarding Jetty vs Tomcat? -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com Solr Training - www.solrtraining.com 23. mars 2013 kl. 00:50 skrev Furkan KAMACI furkankam...@gmail.com: I just indicated that JVM parameter

NoSuchMethodError SolrIndexSearcher.doc(I)

2013-03-23 Thread Furkan KAMACI
I have just configured my Solr to index nutch crawling data. I run dist-war for Solr and when I deploy my war file from my Intellij IDEA 12.0.4 I get that severe at my logs: Mar 23, 2013 7:14:32 PM org.apache.solr.common.SolrException log SEVERE: null:java.lang.NoSuchMethodError:

Re: NoSuchMethodError SolrIndexSearcher.doc(I)

2013-03-23 Thread Furkan KAMACI
I have indicated: -Dsolr.data.dir as a JVM parameter and error gone. 2013/3/23 Furkan KAMACI furkankam...@gmail.com I have just configured my Solr to index nutch crawling data. I run dist-war for Solr and when I deploy my war file from my Intellij IDEA 12.0.4 I get that severe at my logs

Re: NoSuchMethodError updateDocument

2013-03-23 Thread Furkan KAMACI
Cominvent AS - www.cominvent.com Solr Training - www.solrtraining.com 23. mars 2013 kl. 14:34 skrev Furkan KAMACI furkankam...@gmail.com: Hi Jan; I will check the jar versions. By the way I think that I should create a solr home directory for my application (my application is that: I use Nutch

Re: Recommendation for integration test framework

2013-03-24 Thread Furkan KAMACI
Unrelated about your question you said that: We are utilizing Apache Maven as build management tool I think currently ant + ivy is build and dependency management tools, maven pom is generated via plugin (If I am wrong you can correct it). Are there any plan to move the project based on Maven?

Re: multicore vs multi collection

2013-03-26 Thread Furkan KAMACI
Did you check that document: http://wiki.apache.org/solr/SolrCloud#A_little_about_SolrCores_and_CollectionsIt says: On a single instance, Solr has something called a SolrCorehttp://wiki.apache.org/solr/SolrCorethat is essentially a single index. If you want multiple indexes, you create multiple

Debugging Map Reduce Jobs at Solr

2013-03-26 Thread Furkan KAMACI
Is there any easy way(tools etc.) that I can debug Map Reduce jobs of Solr?

Re: Debugging Map Reduce Jobs at Solr

2013-03-26 Thread Furkan KAMACI
...@gmail.com Hi, Solr doesn't really do MapReduce jobs. Maybe you mean distributed search where queries are dispatched to N servers and then responses are merged/reduced to top N and returned? Otis -- Solr ElasticSearch Support http://sematext.com/ On Tue, Mar 26, 2013 at 6:34 AM, Furkan

There are no SolrCores running. Using the Solr Admin UI currently requires at least one SolrCore.

2013-03-26 Thread Furkan KAMACI
I use Solr 4.2 on Centos 6.4 at AWS and I have deployed solr wars into two different amazon instances at tomcats. *When I run them without solrcloud they are OK.* However I want to use them as solrCloud. I want to start embedded zookeper at one of them. When I run: ps aux | grep catalina I get

Re: There are no SolrCores running. Using the Solr Admin UI currently requires at least one SolrCore.

2013-03-26 Thread Furkan KAMACI
: There must be something off with the jars you are using - a mix of versions or something. - Mark On Mar 26, 2013, at 5:18 PM, Furkan KAMACI furkankam...@gmail.com wrote: I use Solr 4.2 on Centos 6.4 at AWS and I have deployed solr wars into two different amazon instances at tomcats. *When I

SolrCloud On Different AWS Instances With Embedded Zookeeper

2013-03-26 Thread Furkan KAMACI
I have to Amazon Web Services instances. I have set up SolrCloud for them. Solr .wars are deployed into tomcat. When I start solr that runs zookeper, it is OK. It can not find second shard as usual. When I start up second solr it throws error. This is first solr config: JAVA_OPTS=$JAVA_OPTS

Re: multicore vs multi collection

2013-03-26 Thread Furkan KAMACI
it clear than the wiki. How do you create multiple collection which can have different schema? ./zahoor On 26-Mar-2013, at 3:52 PM, Furkan KAMACI furkankam...@gmail.com wrote: Did you check that document: http://wiki.apache.org/solr/SolrCloud#A_little_about_SolrCores_and_CollectionsIt

Re: Loadtesting solr/tomcat7 and tomcat stops responding entirely

2013-03-27 Thread Furkan KAMACI
Hi Nate; This may be out of topic however could you explain that why you want to use Tomcat instead of Jetty or Embedded Jetty? 2013/3/27 Michael Della Bitta michael.della.bi...@appinions.com You're using the blocking IO connector, which isn't so great for heavy loads. Give this a shot...

Re: Setup solrcloud on tomcat

2013-03-28 Thread Furkan KAMACI
First of all, can you check your catalina.out log. It gives the detail about what is wrong. Secondly you can separate such kind of JVM parameters from that solr.xml and put them into a file setenv.sh (you will create it under bin folder of tomcat.) and here is what you should do: #!/bin/sh

Combining Solr Indexes at SolrCloud

2013-03-29 Thread Furkan KAMACI
Let's assume that I have two machine in a SolrCloud that works as a part of cloud. If I want to shutdown one of them an combine its indexes into other how can I do that?

SOAP for Solr indexing mechanism

2013-03-29 Thread Furkan KAMACI
Is there any support for communication over SOAP for Solr indexing mechanism?

Parallel Indexing With Solr?

2013-03-29 Thread Furkan KAMACI
Does Solr allows parallelism (parallel computing) for indexing?

Suggestions for Customizing Solr Admin Page

2013-03-29 Thread Furkan KAMACI
I want to customize Solr Admin Page. I think that I will need more complicated things to manage my cloud. I will separate my Solr cluster into just indexing ones and just response ones. I will index my documents by categorical and I will index them at different collections. In my admin page I

Re: Parallel Indexing With Solr?

2013-03-29 Thread Furkan KAMACI
, 2013 at 5:26 AM, Furkan KAMACI furkankam...@gmail.com wrote: Does Solr allows parallelism (parallel computing) for indexing?

Filtering Search Cloud

2013-04-01 Thread Furkan KAMACI
I want to separate my cloud into two logical parts. One of them is indexer cloud of SolrCloud. Second one is Searcher cloud of SolrCloud. My first question is that. Does separating my cloud system make sense about performance improvement. Because I think that when indexing, searching make time to

Re: Flow Chart of Solr

2013-04-02 Thread Furkan KAMACI
Actually maybe one the most important core thing is that Analysis part at last diagram but there is nothing about it i.e. stamming, lemmitazing etc. at any of them. 2013/4/2 Andre Bois-Crettez andre.b...@kelkoo.com On 04/02/2013 04:20 PM, Koji Sekiguchi wrote: (13/04/02 21:45), Furkan

Re: Flow Chart of Solr

2013-04-02 Thread Furkan KAMACI
://www.sparrowmailapp.com/?sig) On Tuesday, April 2, 2013 at 3:51 PM, Furkan KAMACI wrote: Actually maybe one the most important core thing is that Analysis part at last diagram but there is nothing about it i.e. stamming, lemmitazing etc. at any of them. 2013/4/2 Andre Bois-Crettez

Re: [ANNOUNCE] Solr wiki editing change

2013-04-02 Thread Furkan KAMACI
Hi; Please add FurkanKAMACI to the group. Thanks; Furkan KAMACI 2013/4/2 Steve Rowe sar...@gmail.com On Apr 2, 2013, at 11:23 AM, Ryan Ernst r...@iernst.net wrote: Please add RyanErnst to the contributors group. Thanks! Added to solr wiki ContributorsGroup.

Re: Flow Chart of Solr

2013-04-02 Thread Furkan KAMACI
I think about myself as an example. I have started to make research about Solr just for some weeks. I have learned Solr and its related projects. My next step writing down the main steps Solr. We have separated learning curve of Solr into two main categories. First one is who are using it as out

Re: Flow Chart of Solr

2013-04-03 Thread Furkan KAMACI
So, all in all, is there anybody who can write down just main steps of Solr(including parsing, stemming etc.)? 2013/4/2 Furkan KAMACI furkankam...@gmail.com I think about myself as an example. I have started to make research about Solr just for some weeks. I have learned Solr and its related

Re: Filtering Search Cloud

2013-04-03 Thread Furkan KAMACI
of indexer machines (I don't have any filtering issues right now, I just think that maybe I can need it at future) 2013/4/3 Shawn Heisey s...@elyograg.org On 4/1/2013 3:02 PM, Furkan KAMACI wrote: I want to separate my cloud into two logical parts. One of them is indexer cloud of SolrCloud

Re: Filtering Search Cloud

2013-04-03 Thread Furkan KAMACI
that is indexed from master) from being a response after querying (querying from slaves) ? 2013/4/3 Shawn Heisey s...@elyograg.org On 4/3/2013 1:13 PM, Furkan KAMACI wrote: Shawn, thanks for your detailed explanation. My system will work on high load. I mean I will always index something

Difference Between Indexing and Reindexing

2013-04-03 Thread Furkan KAMACI
OK, This could be a so easy question but I want to learn just a bit more technical detail of it. When I use Nutch to send documents to Solr to be indexing there are two parameters: -index and -reindex. What Solr does at each one different from the other one?

Re: Difference Between Indexing and Reindexing

2013-04-04 Thread Furkan KAMACI
/ On Wed, Apr 3, 2013 at 9:15 PM, Furkan KAMACI furkankam...@gmail.com wrote: OK, This could be a so easy question but I want to learn just a bit more technical detail of it. When I use Nutch to send documents to Solr to be indexing there are two parameters: -index and -reindex. What

Re: Difference Between Indexing and Reindexing

2013-04-04 Thread Furkan KAMACI
, bouncing all over the place without understanding what it is that you are really looking for. More specifically, what exactly is the problem you are trying to solve? -- Jack Krupansky -Original Message- From: Furkan KAMACI Sent: Thursday, April 04, 2013 2:45 AM To: solr-user

Re: Difference Between Indexing and Reindexing

2013-04-04 Thread Furkan KAMACI
I use Nutch 2.1 and using that: bin/nutch solrindex http://localhost:8983/solr -index bin/nutch solrindex http://localhost:8983/solr -reindex 2013/4/4 Gora Mohanty g...@mimirtech.com On 4 April 2013 18:33, Furkan KAMACI furkankam...@gmail.com wrote: I craw webages with Nutch and send them

Re: Difference Between Indexing and Reindexing

2013-04-04 Thread Furkan KAMACI
It may be a deprecated usage(maybe not) but certainly can run -index and -reindex on Nutch 2.1. 2013/4/4 Gora Mohanty g...@mimirtech.com On 4 April 2013 20:16, Gora Mohanty g...@mimirtech.com wrote: On 4 April 2013 19:29, Furkan KAMACI furkankam...@gmail.com wrote: I use Nutch 2.1

Re: Filtering Search Cloud

2013-04-05 Thread Furkan KAMACI
/second? 100,000 documents? 1,000,000,000,000 documents? Best Erick On Wed, Apr 3, 2013 at 5:15 PM, Shawn Heisey s...@elyograg.org wrote: On 4/3/2013 1:52 PM, Furkan KAMACI wrote: Thanks for your explanation, you explained every thing what I need. Just one more question. I see that I can

Re: Flow Chart of Solr

2013-04-05 Thread Furkan KAMACI
scores? In short, you need to be more specific. Don't force us to guess what problem you are trying to solve. -- Jack Krupansky -Original Message- From: Furkan KAMACI Sent: Wednesday, April 03, 2013 6:52 AM To: solr-user@lucene.apache.org Subject: Re: Flow Chart of Solr

Re: Sharing index amongst multiple nodes

2013-04-06 Thread Furkan KAMACI
Hi Daire Mac Mathúna; If there is a way copying one Solr's indexes into another Solr instance, this may also solve the problem. Somebody generates indexes and some of other instances could get a copy of them. At synchronizing process you may eliminate some of indexes at reader instance. So you

Pointing to Hbase for Docuements or Directly Saving Documents at Hbase

2013-04-06 Thread Furkan KAMACI
Hi; First of all should mention that I am new to Solr and making a research about it. What I am trying to do that I will crawl some websites with Nutch and then I will index them with Solr. (Nutch 2.1, Solr-SolrCloud 4.2 ) I wonder about something. I have a cloud of machines that crawls websites

Re: Sharing index amongst multiple nodes

2013-04-06 Thread Furkan KAMACI
I will decline some indexes(index filtering) because of the documents they points. Is it possible? 2013/4/7 Walter Underwood wun...@wunderwood.org This is precisely how Solr replication works. It copies the indexes then does a commit. wunder On Apr 6, 2013, at 2:40 PM, Furkan KAMACI wrote

Re: Sharing index amongst multiple nodes

2013-04-06 Thread Furkan KAMACI
it succeeds, there is no going back. wunder On Apr 6, 2013, at 3:08 PM, Furkan KAMACI wrote: Hi Walter; I am new to Solr and digging into code to understand it. I think that when indexer copies indexes, before the commit it is unsearchable. Where exactly that commit occurs at code and can

Re: Sharing index amongst multiple nodes

2013-04-06 Thread Furkan KAMACI
. wunder On Apr 6, 2013, at 3:58 PM, Furkan KAMACI wrote: Hi Walter; Thanks for your explanation. You said Indexing happens on one Solr server. Is it true even for SolrCloud? 2013/4/7 Walter Underwood wun...@wunderwood.org Indexing happens on one Solr server. After a commit

Prediction About Index Sizes of Solr

2013-04-08 Thread Furkan KAMACI
This may not be a well detailed question but I will try to make it clear. I am crawling web pages and will index them at SolrCloud 4.2. What I want to predict is the index size. I will have approximately 2 billion web pages and I consider each of them will be 100 Kb. I know that it depends on

Solr Admin Page Master Size

2013-04-08 Thread Furkan KAMACI
When I check my Solr Admin Page: Replication (Master) Version Gen Size Master: 1365458125729 5 18.24 MB It is a one shard one computer. What is that 18.24 MB. Does it contains just indexes or indexes, highlights etc. etc.? My solr home folder was 512.7 KB and it has become 22860 KB that is

Average Solr Server Spec.

2013-04-09 Thread Furkan KAMACI
This question may not have a generel answer and may be open ended but is there any commodity server spec. for a usual Solr running machine? I mean what is the average server spesification for a Solr machine (i.e. Hadoop running system it is not recommended to have very big storage capably

Re: How can I set configuration options?

2013-04-09 Thread Furkan KAMACI
Hi Edd; The parameters you mentioned are JVM parameters. There are two ways to define them. First one is if you are using an IDE you can indicate them as JVM parameters. i.e. if you are using Intellij IDEA when you click your Run/Debug configurations there is a line called VM Options. You can

Re: Average Solr Server Spec.

2013-04-09 Thread Furkan KAMACI
HDDs, some local storage, some SAN, some EBS on AWS. etc. Otis -- Solr ElasticSearch Support http://sematext.com/ On Tue, Apr 9, 2013 at 7:04 AM, Furkan KAMACI furkankam...@gmail.com wrote: This question may not have a generel answer and may be open ended

Re: Slow qTime for distributed search

2013-04-09 Thread Furkan KAMACI
Hi Shawn; You say that: *... your documents are about 50KB each. That would translate to an index that's at least 25GB* I know we can not say an exact size but what is the approximately ratio of document size / index size according to your experiences? 2013/4/9 Shawn Heisey s...@elyograg.org

Approximately needed RAM for 5000 query/second at a Solr machine?

2013-04-09 Thread Furkan KAMACI
Are there anybody who can help me about how to guess the approximately needed RAM for 5000 query/second at a Solr machine?

Re: Approximately needed RAM for 5000 query/second at a Solr machine?

2013-04-09 Thread Furkan KAMACI
. The best way to answer is to do a proof of concept implementation and measure it yourself. -- Jack Krupansky -Original Message- From: Furkan KAMACI Sent: Tuesday, April 09, 2013 6:06 PM To: solr-user@lucene.apache.org Subject: Approximately needed RAM for 5000 query/second

Re: Approximately needed RAM for 5000 query/second at a Solr machine?

2013-04-09 Thread Furkan KAMACI
...@wunderwood.org On Apr 9, 2013, at 3:06 PM, Furkan KAMACI wrote: Are there anybody who can help me about how to guess the approximately needed RAM for 5000 query/second at a Solr machine? No. That depends on the kind of queries you have, the size and content of the index, the required

Re: Approximately needed RAM for 5000 query/second at a Solr machine?

2013-04-09 Thread Furkan KAMACI
Thanks for your answer. 2013/4/10 Walter Underwood wun...@wunderwood.org We are using Amazon EC2 M1 Extra Large instances (m1.xlarge). http://aws.amazon.com/ec2/instance-types/ wunder On Apr 9, 2013, at 3:35 PM, Furkan KAMACI wrote: Hi Walter; Firstly thank for your detailed reply

Re: Pushing a whole set of pdf-files to solr

2013-04-09 Thread Furkan KAMACI
Apache Solr 4 Cookbok says that: curl http://localhost:8983/solr/update/extract?literal.id=1commit=true; -F myfile=@cookbook.pdf is that what you want? 2013/4/10 sdspieg sdsp...@mail.ru If anybody could still help me out with this, I'd really appreciate it. Thanks! -- View this message

Re: Approximately needed RAM for 5000 query/second at a Solr machine?

2013-04-09 Thread Furkan KAMACI
use that extra RAM or is it a approximately maximum limit (to have twice size of indexes at machine)? 2013/4/10 Shawn Heisey s...@elyograg.org On 4/9/2013 4:06 PM, Furkan KAMACI wrote: Are there anybody who can help me about how to guess the approximately needed RAM for 5000 query/second

Re: Approximately needed RAM for 5000 query/second at a Solr machine?

2013-04-09 Thread Furkan KAMACI
much I define it as a Java processes. When we think about the indexes at storage and caching them at RAM by OS, is that what you talk about: having more than 5 GB - or - 10 GB RAM for my machine? 2013/4/10 Shawn Heisey s...@elyograg.org On 4/9/2013 7:03 PM, Furkan KAMACI wrote: These are really

Re: Approximately needed RAM for 5000 query/second at a Solr machine?

2013-04-10 Thread Furkan KAMACI
Thank you for your explanations, this will help me to figure out my system. 2013/4/10 Shawn Heisey s...@elyograg.org On 4/9/2013 9:12 PM, Furkan KAMACI wrote: I am sorry but you said: *you need enough free RAM for the OS to cache the maximum amount of disk space all your indexes

Re: Approximately needed RAM for 5000 query/second at a Solr machine?

2013-04-11 Thread Furkan KAMACI
of RAM requirements, you need to create an index, run realistic queries at the installation and measure. Best Erick On Tue, Apr 9, 2013 at 10:32 PM, bigjust bigj...@lambdaphil.es wrote: On 4/9/2013 7:03 PM, Furkan KAMACI wrote: These are really good metrics for me: You say

Re: migration solr 3.5 to 4.1 - JVM GC problems

2013-04-11 Thread Furkan KAMACI
Hi Marc; Could I learn your index size and what is your performance measure as query per second? 2013/4/11 Marc Des Garets marc.desgar...@192.com Big heap because very large number of requests with more than 60 indexes and hundreds of million of documents (all indexes together). My problem

Re: Pointing to Hbase for Docuements or Directly Saving Documents at Hbase

2013-04-11 Thread Furkan KAMACI
process and how highlights are genareted at Solr you are welcome. 2013/4/9 Otis Gospodnetic otis.gospodne...@gmail.com You may also be interested in looking at things like solrbase (on Github). Otis -- Solr ElasticSearch Support http://sematext.com/ On Sat, Apr 6, 2013 at 6:01 PM, Furkan

Re: Approximately needed RAM for 5000 query/second at a Solr machine?

2013-04-11 Thread Furkan KAMACI
tip from Lance Norskog on the mailing list a couple years ago). I'm not sure if the current merge policy implementations still have this issue. -Michael -Original Message- From: Furkan KAMACI [mailto:furkankam...@gmail.com] Sent: Thursday, April 11, 2013 2:44 AM To: solr

Re: Pointing to Hbase for Docuements or Directly Saving Documents at Hbase

2013-04-11 Thread Furkan KAMACI
/ On Thu, Apr 11, 2013 at 10:01 AM, Furkan KAMACI furkankam...@gmail.com wrote: Actually I don't think to store documents at Solr. I want to store just highlights (snippets) at Hbase and I want to retrieve them from Hbase when needed. What do you think about separating just highlights from

Re: Slow qTime for distributed search

2013-04-12 Thread Furkan KAMACI
that even with a small amount of responses it takes a long time (and clearly the merging task on few docs is very short) On Wed, Apr 10, 2013 at 2:50 AM, Shawn Heisey s...@elyograg.org wrote: On 4/9/2013 3:50 PM, Furkan KAMACI wrote: Hi Shawn; You say that: *... your documents

Re: Approximately needed RAM for 5000 query/second at a Solr machine?

2013-04-13 Thread Furkan KAMACI
: On 4/9/2013 7:03 PM, Furkan KAMACI wrote: These are really good metrics for me: You say that RAM size should be at least index size, and it is better to have a RAM size twice the index size (because of worst case scenario). On the other hand let's assume that I have a RAM size

Listing Priority

2013-04-14 Thread Furkan KAMACI
I have crawled some internet pages and indexed them at Solr. When I list my results via Solr I want that: if a page has a URL(my schema includes a field for URL) that ends with .edu, .edu.az or .co.uk I will give more priority to them. How can I do it in a more efficient way at Solr?

Some Questions About Using Solr as Cloud

2013-04-14 Thread Furkan KAMACI
I read wiki and reading SolrGuide of Lucidworks. However I want to clear something in my mind. Here are my questions: 1) Does SolrCloud lets a multi master design (is there any document that I can read about it)? 2) Let's assume that I use multiple cores i.e. core A and core B. Let's assume that

Re: Some Questions About Using Solr as Cloud

2013-04-14 Thread Furkan KAMACI
5) When I use multi core design can I transfer one index data into another core or anywhere else? 6) Does Solr holds old versions of documents or remove them? 2013/4/15 Furkan KAMACI furkankam...@gmail.com I read wiki and reading SolrGuide of Lucidworks. However I want to clear something in my

Re: Some Questions About Using Solr as Cloud

2013-04-15 Thread Furkan KAMACI
Krupansky -Original Message- From: Furkan KAMACI Sent: Sunday, April 14, 2013 7:45 PM To: solr-user@lucene.apache.org Subject: Some Questions About Using Solr as Cloud I read wiki and reading SolrGuide of Lucidworks. However I want to clear something in my mind. Here are my

SolrCloud Leaders

2013-04-15 Thread Furkan KAMACI
Does number of leaders at a SolrCloud is equal to number of shards?

Re: SolrCloud Leaders

2013-04-15 Thread Furkan KAMACI
on disk that continue to exist even if the code is not running. So, in the extreme, the number of leaders can be zero while the number of shards is non-zero on disk. -- Jack Krupansky -Original Message- From: Furkan KAMACI Sent: Monday, April 15, 2013 8:21 AM To: solr-user

Re: SolrCloud Leaders

2013-04-15 Thread Furkan KAMACI
Here writes something: https://support.lucidworks.com/entries/22180608-Solr-HA-DR-overview-3-x-and-4-0-SolrCloud-and says: Both leaders and replicas index items and perform searches. How replicas index items? 2013/4/15 Furkan KAMACI furkankam...@gmail.com Does leaders may response search

Usage of CloudSolrServer?

2013-04-15 Thread Furkan KAMACI
I am reading Lucidworks Solr Guide it says at SolrCloud section: *Read Side Fault Tolerance* With earlier versions of Solr, you had to set up your own load balancer. Now each individual node load balances requests across the replicas in a cluster. You still need a load balancer on the 'outside'

Re: Usage of CloudSolrServer?

2013-04-16 Thread Furkan KAMACI
that into which shard a document will go) than there will be a bottleneck on that leader? 2013/4/15 Shawn Heisey s...@elyograg.org On 4/15/2013 8:05 AM, Furkan KAMACI wrote: My system is as follows: I crawl data with Nutch and send them into SolrCloud. Users will search at Solr. What

Re: Storing Solr Index on NFS

2013-04-16 Thread Furkan KAMACI
Hi Walter; You said: It is not safe to share Solr index files between two Solr servers. Why do you think like that? 2013/4/16 Tim Vaillancourt t...@elementspace.com If centralization of storage is your goal by choosing NFS, iSCSI works reasonably well with SOLR indexes, although good

Re: Usage of CloudSolrServer?

2013-04-16 Thread Furkan KAMACI
) will know about it, without any intervention. Upayavira On Tue, Apr 16, 2013, at 08:36 AM, Furkan KAMACI wrote: Hi Shawn; I am sorry but what kind of Load Balancing is that? I mean does it check whether some leaders are using much CPU or RAM etc.? I think a problem may occur at such kind

Re: Some Questions About Using Solr as Cloud

2013-04-16 Thread Furkan KAMACI
of a document from one core to another, you have to re-index the doc. Best Erick On Mon, Apr 15, 2013 at 7:46 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi Jack; I see that SolrCloud makes everything automated. When I use SolrCloud is it true that: there may be more than one computer

SolrCloud Leader Response Mechanism

2013-04-16 Thread Furkan KAMACI
When a leader responses for a query, does it says that: If I have the data what I am looking for, I should build response with it, otherwise I should find it anywhere. Because it may be long to search it? or does it says I only index the data, I will tell it to other guys to build up the response

Same Shards at Different Machines

2013-04-16 Thread Furkan KAMACI
Is it possible to use same shards at different machines at SolrCloud?

Re: SolrCloud Leader Response Mechanism

2013-04-16 Thread Furkan KAMACI
/gather for you. (Though in some cases that request can be proxied to another node) - Mark On Apr 16, 2013, at 7:48 AM, Furkan KAMACI furkankam...@gmail.com wrote: When a leader responses for a query, does it says that: If I have the data what I am looking for, I should build response

Why indexing and querying performance is better at SolrCloud compared to older versions of Solr?

2013-04-16 Thread Furkan KAMACI
Is there any document that describes why indexing and querying performance is better at SolrCloud compared to older versions of Solr? I was examining that architecture to use: there will be a cloud of Solr that just do indexing and there will be another cloud that copies that indexes into them

Re: Pointing to Hbase for Docuements or Directly Saving Documents at Hbase

2013-04-16 Thread Furkan KAMACI
, 2013 at 11:22 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi Otis; It seems that I should read more about highlights. Is there any where that explains in detail how highlights are generated at Solr? 2013/4/11 Otis Gospodnetic otis.gospodne...@gmail.com Hi, You can't store

When a search query comes to a replica what happens?

2013-04-16 Thread Furkan KAMACI
I want to make it clear in my mind: When a search query comes to a replica what happens? -Does it forwards the search query to leader and leader collects all the data and prepares response (this will cause a performance issue because leader is responsible for indexing at same time) or - replica

How SolrCloud Balance Number of Documents at each Shard?

2013-04-16 Thread Furkan KAMACI
Is it possible that different shards have different number of documents or does SolrCloud balance them? I ask this question because I want to learn the mechanism behind how Solr calculete hash value of the identifier of the document. Is it possible that hash function produces more documents into

Re: When a search query comes to a replica what happens?

2013-04-16 Thread Furkan KAMACI
there it's replicated to non-leader shards. At query time there is no redirect to leader, I imagine, as that would quickly turn leaders into hotspots. Otis -- Solr ElasticSearch Support http://sematext.com/ On Tue, Apr 16, 2013 at 6:01 PM, Furkan KAMACI furkankam...@gmail.com wrote

Re: How SolrCloud Balance Number of Documents at each Shard?

2013-04-16 Thread Furkan KAMACI
bottleneck. 2013/4/17 Otis Gospodnetic otis.gospodne...@gmail.com They won't be exact, but should be close. Are you seeing some *big* differences? Otis -- Solr ElasticSearch Support http://sematext.com/ On Tue, Apr 16, 2013 at 6:11 PM, Furkan KAMACI furkankam...@gmail.com wrote

Re: Pointing to Hbase for Docuements or Directly Saving Documents at Hbase

2013-04-16 Thread Furkan KAMACI
you really know you need it. Otherwise, just store in Solr. Otis -- Solr ElasticSearch Support http://sematext.com/ On Tue, Apr 16, 2013 at 5:32 PM, Furkan KAMACI furkankam...@gmail.com wrote: Hi Otis and Jack; I have made a research about highlights and debugged code. I see

Re: SolrCloud Leader Response Mechanism

2013-04-16 Thread Furkan KAMACI
://sematext.com/ On Tue, Apr 16, 2013 at 5:23 PM, Furkan KAMACI furkankam...@gmail.com wrote: Hi Mark; When I speak with proper terms I want to ask that: is there a data locality of spatial locality ( http://www.roguewave.com/portals/0/products/threadspotter/docs/2011.2

Re: SolrCloud Leader Response Mechanism

2013-04-16 Thread Furkan KAMACI
Replica asks to Zookeper and Leader does not do anything. Thanks for your answer Otis. 2013/4/17 Otis Gospodnetic otis.gospodne...@gmail.com Oui, ZK holds the map. Otis -- Solr ElasticSearch Support http://sematext.com/ On Tue, Apr 16, 2013 at 6:33 PM, Furkan KAMACI furkankam

Re: Storing Solr Index on NFS

2013-04-16 Thread Furkan KAMACI
for the same reason - it's another moving part on the other side of the network. Otis -- Solr ElasticSearch Support http://sematext.com/ On Tue, Apr 16, 2013 at 7:15 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi Walter; You said: It is not safe to share Solr index files

Re: Push/pull model between leader and replica in one shard

2013-04-16 Thread Furkan KAMACI
Really nice presentation. 2013/4/17 Mark Miller markrmil...@gmail.com On Apr 16, 2013, at 1:36 AM, SuoNayi suonayi2...@163.com wrote: Hi, can someone explain more details about what model is used to sync docs between the lead and replica in the shard? The model can be push or

Re: Push/pull model between leader and replica in one shard

2013-04-17 Thread Furkan KAMACI
Hej Mark; What did you use to prepare your presentation, its really nice. 2013/4/17 Furkan KAMACI furkankam...@gmail.com Really nice presentation. 2013/4/17 Mark Miller markrmil...@gmail.com On Apr 16, 2013, at 1:36 AM, SuoNayi suonayi2...@163.com wrote: Hi, can someone explain more

Solr Caching

2013-04-17 Thread Furkan KAMACI
I've just started to read about Solr caching. I want to learn one thing. Let's assume that I have given 4 GB RAM into my Solr application and I have 10 GB RAM. When Solr caching mechanism starts to work, does it use memory from that 4 GB part or lets operating system to cache it from 6 GB part of

  1   2   3   4   5   6   7   8   >