Would like to do it during querying.
Thanks,
Ashish
On Tue, Mar 10, 2015 at 11:07 PM, Alexandre Rafalovitch arafa...@gmail.com
wrote:
Is that during indexing or during query phase?
Indexing has UpdateRequestProcessors (e.g.
http://www.solr-start.com/info/update-request-processors/ )
Query
Actually the reason I did not use the solr script was that I didn't really
get how to make a window service out of it from nssm.exe. I tried doing a
.bat that called solr with start -p 8983 but seems it just loops my command
rather then run it.
Thanks for the help / Karl
On 11 March 2015 at
On 3/10/2015 1:39 PM, Ryan, Michael F. (LNG-DAY) wrote:
You'll need to wrap the date in quotes, since it contains a colon:
String a = speechDate:\1992-07-10T17:33:18Z\;
You could also escape the colons with a backslash. Here's another way
to do it that doesn't require quotes or manual
I tried to run Solr over HDFS following
https://cwiki.apache.org/confluence/display/solr/Running+Solr+on+HDFS
when I was testing map reduce way of index generation.
However, when I run Solr on MapRFS, solr gave error that could not
recognize maprfs:// scheme in URI.
Have anyone met similar
Hi Zheng,
*** I understand that in Solr 5.0, they provide a REST API to do real-time
update of the schema using Curl ** *. Would please help me how to do this?
I need to update both schema.xml and solrconfig.xml in Solr 5.0 in
SolrCloud.
Your help is appreciated..
*Thanks Again..*
On Thu, Mar
Hello,
when I display one of my core's schema, lots of fields appear:
fields:[{
name:_root_,
type:string,
indexed:true,
stored:false},
{
name:_version_,
type:long,
indexed:true,
stored:true},
{
name:id,
Hi,
I understand that in Solr 5.0, they provide a REST API to do real-time
update of the schema using Curl. However, I could not do that for my
eariler version of Solr 4.10.1.
Would like to check, is this function available for the earlier version of
Solr, and is the curl syntax the same as Solr
Hello,
my solr logs say:
INFO - 2015-03-12 08:49:34.900; org.apache.solr.core.RequestHandlers; created
/analysis/document: solr.DocumentAnalysisRequestHandler
WARN - 2015-03-12 08:49:34.919; org.apache.solr.core.SolrResourceLoader; Solr
loaded a deprecated plugin/analysis class
Anyone please suggest
With Regards
Aman Tandon
On Sat, Mar 7, 2015 at 9:55 PM, Aman Tandon amantandon...@gmail.com wrote:
Hi,
Please suggest me what should be the tests which i should run to check the
availability, query time, etc in my solr cloud setup.
With Regards
Aman Tandon
Hi,
I've a field which is being used for result grouping. Here's the field
definition.
field name=ADDedup type=string indexed=true stored=true
multiValued=false required=false omitNorms=true docValues=true/
This started once I did a rolling update from 4.7 to 5.0. I started getting
the
Hi. Erick..
Would please help me distinguish between
Uploading a Configuration Directory and Linking a Collection to a
Configuration Set ?
On Thu, Mar 12, 2015 at 2:01 AM, Nitin Solanki nitinml...@gmail.com wrote:
Thanks a lot Erick.. It will be helpful.
On Wed, Mar 11, 2015 at
Hi,
i want to sort my documents by a given order. The order is defined by a
list of ids.
My current solution is:
list of ids: 15, 5, 1, 10, 3
query: q=*:*fq=(id:((15) OR (5) OR (1) OR (10) OR
(3)))sort=query($idqsort) desc,id ascidqsort=id:((15^5) OR (5^4) OR
(1^3) OR (10^2) OR
I investigated my tomcat7 configuration.
I have founded that we work in BIO mode.
I consider to switch to NIO mode.
what are recommendation in this case?
--
View this message in context:
http://lucene.472066.n3.nabble.com/increase-connections-on-tomcat-tp4192405p4192602.html
Sent from
On 3/11/2015 8:56 AM, SolrUser1543 wrote:
Client application which queries solr needs to increase a number of
simultaneously connections in order to improve performance ( in additional
to get solr results, it needs to get an internal resources like images. )
But this increment has improved
On 3/11/2015 10:53 AM, SolrUser1543 wrote:
does it apply to solr 4.10 ? or only to solr 5 ?
The information I provided is not version-specific. It would apply to
either version you listed and at least some of the previous 4.x versions.
Thanks,
Shawn
Hi,
I am using Oracle 11g2 and we are having a schema where few tables are having
more than 100 million rows (some of them are Varchar2 100 bytes). And we have
to frequently do the LIKE based search on those tables. Sometimes we need to
join the tables also. Insert / Updates are also happening
Hi,
I am not sure but when i am looking into the server/lib directory then i am
able to see the version 8.1 with all those lib files present in that
folder. So i am guessing its version 8.1.
I confirmed it by downloading the new jetty server which was jetty-9.2 and
i found the same version on
Well, the new way will be the only way eventually, so either you learn
the old way then switch or learn it now ;)...
But if you insist you could start with a heap size of 4G like this:
java -Xmx4G -Xms4G -jar start.jar
Best,
Erick
On Wed, Mar 11, 2015 at 1:09 PM, Karl Kildén
You could also check the default memory by starting solr with the -V
parameter for verbose output. It will show your output like this.
If your are startinf solr with script present in bin directory using this
command
*./solr -c -V*
Using Solr root directory: /data/solr/aman/solr_cloud/solr-5.0.0
Yes, Solr 5.0 uses Jetty 8.
FYI, the upcoming release 5.1 will move to Jetty 9.
Also, just in case it matters -- as noted in the 5.0 release notes, the use
of Jetty is now an implementation detail and we might move away from it in
the future -- so you shouldn't be depending on Solr using Jetty
On 3/12/2015 5:03 AM, Pratik Thaker wrote:
I am using Oracle 11g2 and we are having a schema where few tables are having
more than 100 million rows (some of them are Varchar2 100 bytes). And we have
to frequently do the LIKE based search on those tables. Sometimes we need to
join the tables
Hi Shawn,
here is the Jetty Mailing List's reply concerning my question.
Unfortunately, this solution won't work with SOLR Jetty, because its version is
9.
Philippe
--
Just ensure you don't have a /WEB-INF/ directory, and you can
What's your commit strategy? Explicit commits? Soft commits/hard
commits (in solrconfig.xml)?
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On 12 March 2015 at 23:19, Nitin Solanki nitinml...@gmail.com wrote:
Hello,
Hi Alexandre,
*Hard Commit* is :
autoCommit
maxTime${solr.autoCommit.maxTime:3000}/maxTime
openSearcherfalse/openSearcher
/autoCommit
*Soft Commit* is :
autoSoftCommit
maxTime${solr.autoSoftCommit.maxTime:300}/maxTime
/autoSoftCommit
And I am committing 2
Thanks Shawn and Erick for explanation...
On Thu, Mar 12, 2015 at 9:02 PM, Shawn Heisey apa...@elyograg.org wrote:
On 3/12/2015 9:18 AM, Erick Erickson wrote:
By and large, I really never use linking. But it's about associating a
config set
you've _already_ uploaded with a collection.
Hello,
I have written a python script to do 2 documents indexing
each time on Solr. I have 28 GB RAM with 8 CPU.
When I started indexing, at that time 15 GB RAM was freed. While indexing,
all RAM is consumed but **not** a single document is indexed. Why so?
And it through *HTTPError:
Yes, the admin handlers are deprecated because they are now implicit - no
need to specify them in solrconfig. Yeah, the doc is very unclear on that
point, but in CHANGES.TXT: *AdminHandlers is deprecated , /admin/* are
implicitly defined, /get ,/replication and handlers are also implicitly
By and large, I really never use linking. But it's about associating a
config set
you've _already_ uploaded with a collection.
So uploading is pushing the configset from your local machine up to Zookeeper,
and linking is using that uploaded, named configuration with an
arbitrary collection.
But
About 1. Gotta be careful here about what would be promised. You
really _can't_ encrypt the _indexed_ terms in a meaningful way and
still search. And, as you well know, you can reconstruct documents
from the indexed terms. It's lossy, but still coherent enough to give
security folks fits.
For
On 3/12/2015 8:17 AM, phi...@free.fr wrote:
here is the Jetty Mailing List's reply concerning my question.
Unfortunately, this solution won't work with SOLR Jetty, because its version
is 9.
The trunk branch of the Solr source code (version 6.0 development) is
already running Jetty 9.2.9. I
Not unless you can somehow codify that sort order at index time, but
I'm assuming the sort order changes dynamically.
You can also sort by function, but that's not really useful.
Or, if these are relatively short lists, you can sort at the app layer.
Best,
Erick
On Thu, Mar 12, 2015 at 2:16
On 3/12/2015 9:18 AM, Erick Erickson wrote:
By and large, I really never use linking. But it's about associating a
config set
you've _already_ uploaded with a collection.
So uploading is pushing the configset from your local machine up to Zookeeper,
and linking is using that uploaded, named
Actually I ran across a neat IntelliJ plugin that you could install
and directly edit ZK files. And I'm pretty sure there are stand-alone
programs that do this, but they are all outside Solr.
I'm not sure what real time update of the schema is for, would you
(Zheng) explain further? Collections
Hi Hoss,
I created a wrapper class, compiled a jar and included an
org.apache.lucene.codecs.Codec file in META-INF/services in the jar file
with an entry for the wrapper class :HTPostingsFormatWrapper. I created a
collection1/lib directory and put the jar there. (see below)
I'm getting the
On 3/11/2015 4:45 PM, shamik wrote:
field name=DocumentType type=string indexed=true stored=true
multiValued=false required=false omitNorms=true docValues=true
/ 3/11/2015, 2:14:30 PM ERROR SolrDispatchFilter
null:java.lang.IllegalStateException: unexpected docvalues type NONE
for field
Manual optimize is no longer needed for modern Solr. It does great
optimization automatically. The only reason I recommended it here is
to make sure that all segments are brought up to the latest version
and the deleted documents are purged. That's something that also would
happen automatically
For those who have not yet taken the leap to SSD goodness because they are
afraid of flash wear, the burnout test from The Tech Report seems worth a read.
The short story is that they wrote data to the drives until they wore out. All
tested drives survived considerably longer than guaranteed,
what does your schema.xml look like?
what does your solrconfig.xml look like?
what does the document you are indexing look like?
what is the full error with stack trace from your server logs?
details matter.
https://wiki.apache.org/solr/UsingMailingLists
: Date: Thu, 12 Mar 2015 20:27:05
Wow, optimize worked like a charm. This really addressed the docvalues
issue. A follow-up question, is it recommended to run optimize in a
Production Solr index ? Also, in a Sorl cloud mode, do we need to run
optimize on each instance / each shard / any instance ?
Appreciate your help Alex.
--
Hello Alex,
I'm trying to add a new document, using solrj and the error This IndexSchema
is not mutable is raised when inserting the document in the solr index.
My index in solr, is configured with classicSchemaIndexFactory.
If I change it to AutoManaged the insert is done without any problems.
If you cannot trust your root users you probably have bigger problems than with
search... I think it has been suggested to encrypt on codec or directory level
as well. Yep, here is the JIRA
https://issues.apache.org/jira/browse/LUCENE-2228 :)
--
Jan Høydahl, search solution architect
Cominvent
The answer meant it was most likely something user has done not quite
understanding Solr's behavior. Not a bug. I'd ignore that case and
just explain what your issue actually is.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
Do you have any really old segments in that index? Could be worth
trying to optimize them down to one in latest format first.
Like Shawn, this is just a one more idea proposal.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On
Well, I think I've narrowed down the issue. The error is happening when I'm
trying to do a rolling update from Solr 4.7 (which is our current version)
to 5.0 . I'm able to re-produce this couple of times. If I do a fresh index
on a 5.0, it works. Not sure if there's any other way to mitigate it.
Hi guys,
I saw an issue in Jira (https://issues.apache.org/jira/browse/SOLR-7234)
with status Resolved, but the resolution is not identified in the issue.
I am facing the exact same problem.. and not able to identified the
solution.
In the last comment of the issue, is said that this kind
On 3/11/2015 10:45 PM, Aman Tandon wrote:
You may need to manually remove the 127.0.1.1 entries from zookeeper
after you fix the IP address problem.
How to do that?
The zkcli script included with Solr should have everything you need --
getfile, putfile, and clear ... but that would be a
On 3/11/2015 7:38 AM, phi...@free.fr wrote:
does anyone if it is possible to create a directory resource in the
solr-jetty configuration files?
In Tomcat 8, you can do the following:
PostResources
On 3/12/2015 2:00 AM, Zheng Lin Edwin Yeo wrote:
I understand that in Solr 5.0, they provide a REST API to do real-time
update of the schema using Curl. However, I could not do that for my
eariler version of Solr 4.10.1.
Would like to check, is this function available for the earlier version
Wait, step back. This is confusing. What's your real problem you are
trying to solve?
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On 12 March 2015 at 19:50, Mirko Torrisi mirko.torr...@ucdconnect.ie wrote:
Hi folks,
I
I would love to see record level (or even field level) restricted access in
Solr / Lucene.
This should be group level, LDAP like or some rule base (which can be dynamic).
If the solution means having a second core, so be it.
The following is the closest I found:
Thanks for sharing Toke!
Reliability should not be a problem for a Solr cloud environment. A corrupted
index cannot be loaded due to exceptions so the core should not enter an active
state. However, what would happen if parts of the data become corrupted but can
still be processed by the
Hi All,
I am having a solr cloud cluster of 20 nodes with each node having close to
20 Million records and total index size is around 400GB ( 20GB per node X 20
nodes ). I am trying to know the best way to dump out the entire solr data
in say CSV format.
I use successive queries by incrementing
Well, it's cursor or nothing. Well, or some sort of custom code to
manually read Lucene indexes (good luck with deleted items, etc).
I think your understanding is correct.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On 12
On 3/12/2015 12:43 PM, Pedro Figueiredo wrote:
I saw an issue in Jira (https://issues.apache.org/jira/browse/SOLR-7234)
with status Resolved, but the resolution is not identified in the issue.
I am facing the exact same problem.. and not able to identified the
solution.
I believe the problem
Hello Alexandre - if you, and others, allow me to be a bit lazy right now; are
there unit tests that input corrupted segments, where not the structure but the
data is affected, to the codec?
Thanks,
Markus
-Original message-
From:Alexandre Rafalovitch arafa...@gmail.com
Sent:
Hi,
I’m currently working with indexes that need document level security. Based on
the user logged in, query results would omit documents that this user doesn’t
have access to, with LDAP integration and such.
I think that would be nice to have on a future Solr release.
Henrique.
On Mar 12,
Jan - we don't really need any security for our products, nor for most clients.
However, one client does deal with very sensitive data so we proposed to
encrypt the transfer of data and the data on disk through a Lucene Directory.
It won't fill all gaps but it would adhere to such a client's
Lucene 5 has added a lot of various CRCs to catch index corruption
situations. I don't know if it is 'perfect', but there was certainly a
lot of work.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On 12 March 2015 at 18:39,
Hi,
I am indexing documents on Solr 4.10.2. While indexing, I am
getting this error in log -
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
at
Hi,
Things you have mentioned would be useful for our use-case.
On top we've seen these two requests for securing Solr:
1. Encrypting the index (with a customer private key for instance). There
are certainly other ways to go about this, like using virtual private
clouds, but having the feature
Hi,
Securing various Solr APIs has once again surfaced as a discussion in the
developer list. See e.g. SOLR-7236
Would be useful to get some feedback from Solr users about needs in the field.
Please reply to this email and let us know what security aspect(s) would be
most important for your
that would explain it!
Luke tool (http://github.com/DmitryKey/luke) is also useful for such cases
or generally, when in need to check the field contents:
On Wed, Mar 11, 2015 at 12:50 PM, phi...@free.fr wrote:
Hello,
I found the reason: the query to store ymds in SOLR was invalid (json
What is the modern equivalent of Luke?
It is same Luke, but polished:
http://github.com/DmitryKey/luke
On Thu, Mar 12, 2015 at 11:03 AM, phi...@free.fr wrote:
Hello,
my solr logs say:
INFO - 2015-03-12 08:49:34.900; org.apache.solr.core.RequestHandlers;
created /analysis/document:
Just a small correction
If your are startinf solr with script present in bin directory using this
command
*./solr -c -V*
*./solr start -c -V*
With Regards
Aman Tandon
On Thu, Mar 12, 2015 at 4:05 PM, Aman Tandon amantandon...@gmail.com
wrote:
You could also check the default memory by
Well, I don't know this issue to such level of granularity. Perhaps others do.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On 12 March 2015 at 18:57, Markus Jelsma markus.jel...@openindex.io wrote:
Hello Alexandre - if you,
Without cursor, you are rerunning a full search every time. So, slow
down is entirely expected.
With cursor, you do not. It does an internal skip based on cursor
value. I think the sort is there to ensure the value is stable.
Basically, you need to use the cursor.
Regards,
Alex.
Solr
Thanks Alex for explanation. Actually since I am scraping all the contents
from Solr, I am doing a generic query of *:* So I think it should not take
so much time right?
But as you say probably the internal skips using the cursor might be more
efficient than the skip done with increasing the
Are there any results of off-heap cache vs JRE 8 with G1GC?
On 10 March 2015 at 11:13, Alexandre Rafalovitch arafa...@gmail.com wrote:
Ask and you shall receive:
SOLR-7210 Off-Heap filter cache
SOLR-7211 Off-Heap field cache
SOLR-7212 Parameter substitution
SOLR-7214 JSON Facet API
Thanks Alex for quick response. I wanted to avoid reading the lucene index to
prevent complications of merging deleted info. Also I would like to do this
on very frequent basis as well like once in two or three days.
I am wondering if the issues that I faced while scraping the index towards
Hello - i would assume off-heap would out perform any heap based data
structure. G1 is only useful if you deal with very large heaps, and it eats CPU
at the same time. As much as G1 is better than CMS in same cases, you would
still have less wasted CPU time and resp. less STW events.
Anyway.
Any suggestions?
Le 11 mars 2015 à 14:38, phi...@free.fr a écrit :
Hello,
does anyone if it is possible to create a directory resource in the
solr-jetty configuration files?
In Tomcat 8, you can do the following:
PostResources
Hello,
which jetty version does solr 5 integrate?
Cheers,
Philippe
SOLR-1632 will certainly help. But trying to predict whether your core
A or core B will appear first doesn't really seem like a good use of
time. If you actually have a setup like you describe, add debug=all
to your query on both cores and you'll see all the gory detail of how
the scores are
IMO each mega of memory saved has more impact that 0.001 less in latency … an
OOM is killer, a lag of 2 second … is not catastrophic.
—
/Yago Riveiro
On Tue, Mar 10, 2015 at 4:03 PM, Erick Erickson erickerick...@gmail.com
wrote:
Just to pile on:
I admire your bravery! I'll add to the other
You simply cannot compare scores from two separate queries, comparing
them is meaningless.
This appears to be an XY problem, you're asking _how_ to do something
without telling us _what_ the end goal here is.
From your description, I really have no idea what you're trying to do.
You might
Ahhh, ok. When you reloaded the cores, did you do it core-by-core?
Yes, but maybe we reloaded the wrong core or something like that. We
also noticed that the startTime doesn't update in the admin-ui while
switching between cores (you have to reload the page). We still use
4.8.1, so maybe it
76 matches
Mail list logo
