R: defragmentation can improve performance on SATA class 10 disk ~10000 rpm ?

2021-02-22 Thread Danilo Tomasoni
on the memory suggestion by Shawn.. thank you very much. Danilo Danilo Tomasoni Fondazione The Microsoft Research - University of Trento Centre for Computational and Systems Biology (COSBI) Piazza Manifattura 1, 38068 Rovereto (TN), Italy tomas...@cosbi.eu<https://webmail.cosbi.eu/

defragmentation can improve performance on SATA class 10 disk ~10000 rpm ?

2021-02-21 Thread Danilo Tomasoni
to force-optimize or it can be faster? Thank you. Danilo Danilo Tomasoni Fondazione The Microsoft Research - University of Trento Centre for Computational and Systems Biology (COSBI) Piazza Manifattura 1, 38068 Rovereto (TN), Italy tomas...@cosbi.eu<https://webmail.cosbi.eu/owa/redir.asp

R: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!

2021-02-21 Thread Danilo Tomasoni
Congratulations Jan! Danilo Tomasoni Fondazione The Microsoft Research - University of Trento Centre for Computational and Systems Biology (COSBI) Piazza Manifattura 1, 38068 Rovereto (TN), Italy tomas...@cosbi.eu<https://webmail.cosbi.eu/owa/redir.aspx?C=VNXi3_8-qSZ

different interpretation of the same query between solr 7.3.1 and solr 8.4.1

2020-04-07 Thread Danilo Tomasoni
dex/select?indent=off=json=1=true; (solr 8.4.1) curl -X POST -H "Content-type: application/x-www-form-urlencoded" --data @request "http://solr-test.cosbi.eu/solr/COSBIBioIndex/select?indent=off=json=1=true; Any clue on why this is happening? It seems to me that there should be an obvious

Fwd: Re: [SPAM] abnormal cpu usage (solr 7.3.1)

2019-12-11 Thread Danilo Tomasoni
Forwarded Message Subject:Re: [SPAM] abnormal cpu usage (solr 7.3.1) Date: Wed, 11 Dec 2019 17:41:05 +0100 From: Danilo Tomasoni To: Erick Erickson Thank you!. any ideas for the cpu spikes? On 11/12/19 17:28, Erick Erickson wrote: This is pretty useful

Re: [SPAM] abnormal cpu usage (solr 7.3.1)

2019-12-11 Thread Danilo Tomasoni
I'm sorry I forgot the pictures. On 11/12/19 15:20, Danilo Tomasoni wrote: Hello all, we have a solr instance with around 41MLN documents. Recently we stopped our forcemerge policy that ensured only 1 segment was present at query time, because we read here in the ML and elsewhere

merge policy & autocommit

2019-10-28 Thread Danilo Tomasoni
you Danilo -- Danilo Tomasoni Fondazione The Microsoft Research - University of Trento Centre for Computational and Systems Biology (COSBI) Piazza Manifattura 1, 38068 Rovereto (TN), Italy tomas...@cosbi.eu http://www.cosbi.eu As for the European General Data Protection Regulation 2016/679

Re: solr configuration issue

2019-10-25 Thread Danilo Tomasoni
peration. doing it after _every_ document is a poor choice. If you absolutely _must_, at least batch your updates up in groups of, say, 1,000 and open a new searcher after that. Best, Erick On Oct 24, 2019, at 3:52 AM, Danilo Tomasoni wrote: For every document processed, a soft commit is pe

solr configuration issue

2019-10-24 Thread Danilo Tomasoni
to speed up things? I checked the system usage in the solr machine and neither I/O nor CPU are heavily used.. Thanks Danilo -- Danilo Tomasoni Fondazione The Microsoft Research - University of Trento Centre for Computational and Systems Biology (COSBI) Piazza Manifattura 1, 38068 Rovereto (TN

RE: [SPAM] Re: query parsed in different ways in two identical solr instances

2019-06-10 Thread Danilo Tomasoni
Yes I identical because the configuration (solrconfig.xml etc) is identical, just some fields changed. Sorry I was not so precise in the description of the environment. Nice to know it's already fixed. Danilo Tomasoni Fondazione The Microsoft Research - University of Trento Centre

RE: [SPAM] Re: query parsed in different ways in two identical solr instances

2019-06-10 Thread Danilo Tomasoni
w the behaviour is the same! How is that possible? why the schema (and in this case a non-existing field) can influence in such a profound way the behaviour of the query parser? I think that this is a subtle bug and an error should be raised instead of performing an unexpected query. Danilo Tomaso

RE: query parsed in different ways in two identical solr instances

2019-06-10 Thread Danilo Tomasoni
Hello all, maybe I should consider this as a bug and open an issue? Danilo Tomasoni Fondazione The Microsoft Research - University of Trento Centre for Computational and Systems Biology (COSBI) Piazza Manifattura 1, 38068 Rovereto (TN), Italy tomas...@cosbi.eu http://www.cosbi.eu

RE: query parsed in different ways in two identical solr instances

2019-06-07 Thread Danilo Tomasoni
any thoughts on that difference in the solr parsing? is it correct that the first looks like an AND while the second looks like and OR? Thank you Danilo Tomasoni Fondazione The Microsoft Research - University of Trento Centre for Computational and Systems Biology (COSBI) Piazza Manifattura 1

RE: query parsed in different ways in two identical solr instances

2019-06-06 Thread Danilo Tomasoni
ah yes, I'm sure we are using solr 7.3.1 as test (non working) and solr 7.3.0 as 1 (working) 7.3.0 98a6b3d642928b1ac9076c6c5a369472581f7633 - woody - 2018-03-28 14:37:45 vs 7.3.1 ae0705edb59eaa567fe13ed3a222fdadc7153680 - caomanhdat - 2018-05-09 09:30:57 Danilo Tomasoni Fondazione

RE: query parsed in different ways in two identical solr instances

2019-06-06 Thread Danilo Tomasoni
an OR... Danilo Tomasoni Fondazione The Microsoft Research - University of Trento Centre for Computational and Systems Biology (COSBI) Piazza Manifattura 1, 38068 Rovereto (TN), Italy tomas...@cosbi.eu http://www.cosbi.eu As for the European General Data Protection Regulation 2016/679

RE: query parsed in different ways in two identical solr instances

2019-06-06 Thread Danilo Tomasoni
Hello, and thank you for your answer. Attached you will find the two logs for the working solr1 server, and the non-working solr-test server. Danilo Tomasoni Fondazione The Microsoft Research - University of Trento Centre for Computational and Systems Biology (COSBI) Piazza Manifattura 1

query parsed in different ways in two identical solr instances

2019-06-05 Thread Danilo Tomasoni
dline_mesh_terms:\"nonlinear dynam\")~0.1) DisjunctionMaxQuery((medline_chemical_terms:\"numer analysi comput assist\" | medline_mesh_terms:\"numer analysi comput assist\")~0.1) DisjunctionMaxQuery((medline_chemical_terms:\"comput simul\" | medline_mesh_

Re: misteriuos nullpointerexception while adding documents

2019-02-13 Thread Danilo Tomasoni
ma for those fields. Regards, Munendra S N On Tue, Feb 12, 2019 at 2:24 PM Danilo Tomasoni wrote: Hello all, I get this error while uploading my documents with 'set' modifier in json format. My solr version is 7.3.1. I disabled autocommit (both soft and hard), but used to work with a prev

misteriuos nullpointerexception while adding documents

2019-02-12 Thread Danilo Tomasoni
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)     at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:565)     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)     ... 40 more -- Danilo Tomasoni COSBI As for the European General Data Protection Regulation 2016/679 on the

Re: [SPAM] Re: Per-field slop param in eDisMax

2019-01-24 Thread Danilo Tomasoni
ram in eDisMax query parser with Solr 6.0 and 7.6. What I want to do with eDixMax is similar to following in the default query parser. * Query string : "aaa bbb" * Target fields : fieldA(TextField), fieldB(TextField) q=fieldA:"aaa bbb"~2 OR fieldB:"aaa bbb"~5 Anyo

Re: disable auto-commit

2018-12-13 Thread Danilo Tomasoni
these "recovery" commit normally. no it's a single solr instance. Thank you I finally managed to avoid autocommits, the solution was autoSoftCommit/autoCommit maxtime = -1, and till now no OOM. Thank you all guys. On Thu, Dec 13, 2018 at 12:45 PM Danilo Tomasoni wrote: On 13/12/18 10:2

Re: disable auto-commit

2018-12-13 Thread Danilo Tomasoni
) [   x:COSBIBioIndex] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@57d576df commitCommandVersion:0 i double-checked and I don't commit, just update. On Thu, Dec 13, 2018 at 10:15 AM Danilo Tomasoni wrote: Hello I tried setting both autocommit

Re: disable auto-commit

2018-12-12 Thread Danilo Tomasoni
seen any API for that. Thank you for your help Danilo Best, Erick On Wed, Dec 12, 2018 at 4:44 AM Danilo Tomasoni wrote: I want to disable even that. I saw here https://lucene.apache.org/solr/guide/6_6/updatehandlers-in-solrconfig.html that probably to achieve what I want I just need

Re: disable auto-commit

2018-12-12 Thread Danilo Tomasoni
12, 2018 at 3:24 PM Danilo Tomasoni wrote: Hello, I'm experiencing oom while indexing a big amount of documents. The main idea to avoid OOM is to avoid commit (just one big commit at the end). Is this a correct idea? How can I disable autocommit? I've set ${solr.autoCommit.maxTime:-1

disable auto-commit

2018-12-12 Thread Danilo Tomasoni
but it's not sufficient, while indexing I still see documents. Thank you Danilo -- Danilo Tomasoni COSBI As for the European General Data Protection Regulation 2016/679 on the protection of natural persons with regard to the processing of personal data, we inform you that all the data we possess

edismax query with edismax subqueries

2018-12-06 Thread Danilo Tomasoni
uery1': '("butter oil" "dairy" "ghee" "butter" "processed cheese" "cheese" "cream" "sour cream" "whipped cream" )' and the list of OR'ed statements is < 1024 then solr outputs ~118K documents other

Re: solr crashes

2018-12-04 Thread Danilo Tomasoni
RAM. Raise your heap to 24GB and your physical RAM to about 75GB or better 96GB. RAM should be about 3 to 4 times heap size. Regards, Bernd Am 04.12.18 um 13:37 schrieb Danilo Tomasoni: Hello Bernd, Here I list the extra info you requested: - actually the virtual machine has 22GB of RAM

Re: solr crashes

2018-12-04 Thread Danilo Tomasoni
and if so what are they reporting? - Any FullGC in GC logs? Regards, Bernd Am 03.12.18 um 10:09 schrieb Danilo Tomasoni: Hello all, We have a configuration with a single node with 30gb of RAM. We use it to index ~40MLN of documents. We perform queries with edismax parser that contain ofte

solr crashes

2018-12-03 Thread Danilo Tomasoni
ng on here? Otherwise, What kind of solr configuration parameters do you think I need to investigate first? What kind of log lines should I search for to understand what's going on? Thank you Danilo -- Danilo Tomasoni COSBI As for the European General Data Protection Regulatio

Re: OR query strange results

2018-11-19 Thread Danilo Tomasoni
Hello, any ideas on this? Thank you Danilo On 16/11/18 08:48, Danilo Tomasoni wrote: Thank you for your reply Erick. I've thought about termsquery but it doesn't support phrase search AFAIK, and I want to query for near words like "Mycobacterium tuberculosis" and also i would l

Re: OR query strange results

2018-11-15 Thread Danilo Tomasoni
t/fasted way around that would be to use an "fq" clause and the TermsQueryParser. Best, Erick On Thu, Nov 15, 2018 at 7:52 AM Danilo Tomasoni wrote: Hello all, I'm performing some queries with a big list of terms in OR on our solr instance, and this odd situation happened - A. query w

OR query strange results

2018-11-15 Thread Danilo Tomasoni
nthesis, that's to avoid the max boolean clauses error (If you know some other method that allows phrase searches please tell me) Thank you Danilo -- Danilo Tomasoni COSBI As for the European General Data Protection Regulation 2016/679 on the protection of natural persons with regard to

Re: SynonimGraphFilter expands wrong synonims

2018-09-05 Thread Danilo Tomasoni
ou do that, the apostrophe in the document ( ′ ) is not the same symbol ( ' ) you've used in the query and in the synonyms file, so you need to replace it somewhere (in the document and/or in the query) otherwise you won't have any match. HTH Gazza On 05/09/2018 12:19, Danilo Tomasoni wrote: Hello to all

SynonimGraphFilter expands wrong synonims

2018-09-05 Thread Danilo Tomasoni
" that appears as a synonim for the second line. If I remove cytosolic as a synonim from the second line, then again no synonym is expanded. Can you tell me why this happens? I thought that the first line should be expanded since it has a multi-word synonym in it that match exactly t