I had a similar problem and didnt find any solution to use the fields in JSON
Blob for a filter ... Not with DIH.
--
View this message in context:
http://lucene.472066.n3.nabble.com/DIH-Blob-data-tp4168896p4168925.html
Sent from the Solr - User mailing list archive at Nabble.com.
hey.
are sending the cacheImpl in your request? or where are defining it?
cacheImpl=${cache.impl}
if i let this string blank, import fails =(
--
View this message in context:
http://lucene.472066.n3.nabble.com/Using-CachedSqlEntityProcessor-with-delta-imports-in-DIH-tp4091620p4163106.html
thx.
this is a little bit better, but now i got only one row from entity en2 in
my index.
it seems, that the lookup dont work for me =((
--
View this message in context:
http://lucene.472066.n3.nabble.com/DIH-cacheImpl-SortedMapBackedCache-empty-rows-from-sub-entity-tp4162316p4162879.html
i dont know why. but it works if i dont use cacheKey/cacheLookup. But if i
use where it works fine.
http://wiki.apache.org/solr/DataImportHandler#CachedSqlEntityProcessor -
From Example 2:
where=id=en1.id ... strange.
--
View this message in context:
Hello
i am fighting with cacheImpl=SortedMapBackedCache.
I want to refactor my ugly entities and so i try out sub-entities with
caching.
My Problem is that my cached subquery do not return any values from the
select. but why?
thats my entity
entity name=en1 pk=id
Hello my friend :)
i want to monitor the taken time of dih with zabbix. which is the best
indicator of the update-time when you using the dataimporthandler. is time
taken correct or do you think one of request time in mbeans is better? i
dont think that the value of 5minRateReqsPerSecond is a
hello.
is it possible to include some entities with XInclude in my data-config.xml?
i tried with this line:
xi:include href=solr/entity.xml
xmlns:xi=http://www.w3.org/2001/XInclude; /
in my entity.xml is something like:
entity name=name query=SELECT * FROM table/entity
some ideas, why does
hello.
i want to compare two shards each other, because these shards should have
the same index. but this isnt so =(
so i want to find these documents, there are missing in one shard of my both
shards.
my ideas
- distrubuted shard request on my nodes and fire a facet search on my
unique-field.
hello.
My problem is, that i need to calculate a sum of amounts. this amount is in
my index (stored=true). my php script get all values with paging. but if a
request takes too long, jetty is killing this process and i get a broken
pipe.
Which is the best/fastest way to get the values of many
Hey, thx for your reply.
i forgot to say. StatsComponent doesnt work with our application.
too slow and buggy. but i test with this component with version 1.4 ...
maybe some bugfixes in 4.0 ?
this is the reason for calculating the sum on client side and some pages.
but sometimes its too much
Mikhail Khludnev wrote
You can spend some heap for uninverting the index and
utilize wiki.apache.org/solr/StatsComponent
what do you mean with this?
Edward Garrett wrote
how many documents are you working with?
~90 million documents ...
--
View this message in context:
hello.
Which is the best/fastest way to get the value of many fields from index?
My problem is, that i need to calculate a sum of amounts. this amount is in
my index (stored=true). my php script get all values with paging. but if a
request takes too long, jetty is killing this process of export.
Hello.
if my server is running for a while i get some OOM Problems. I think the
problem is, that i running to many cores on one Server with too many
documents.
this is my server concept:
14 cores.
1 with 30 million docs
1 with 22 million docs
1 with growing 25 million docs
1 with 67 million
Hello
i need the host name of my solr-server in my solrconfig.xml
anybody knows the correct variable?
something like ${solr.host} or ${solr.host.name} ...
exists an documantation about ALL available variables in the solr
namespaces?
thx a lot
--
View this message in context:
okay. thx.
i knw this way but its not so nice :P
i set a new variable in my core.properties file which i load in solr.xml for
each core =))
--
View this message in context:
http://lucene.472066.n3.nabble.com/solr-host-name-on-solrconfig-xml-tp3997371p3997652.html
Sent from the Solr - User
Hello.
I have no idea when this error message occurs. Anybody an idea?
no search requests are running in this time.
every minute starts an import, so i think this cause this exception. but
why?
SEVERE: null:org.eclipse.jetty.io.EofException
at
So you want to re-use same SQL sentence in many entities?
Yes
is it necessary to deploy complete solr and lucene for this?
--
View this message in context:
http://lucene.472066.n3.nabble.com/DIH-include-Fieldset-in-query-tp3994798p3995228.html
Sent from the Solr - User mailing list archive at
hello..
i have many big entities in my data-config.xml. in the many entities is the
same query.
the entities look like this:
entity name=name transformer=DateFormatTransformer pk=id
query=
SELECT
field as fielname,
IF(bla NOT NULL, 1, 0) AS blob,
fieldname,
fieldname AS field, ...
more
exists an alternative to waitFlush?
in my setup this command is very usefull for my NRT. is nobody here with the
same problem?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Alternative-to-waitFlush-in-Solr4-0-tp3991489.html
Sent from the Solr - User mailing list archive at
Hello again.
this is my Exception.
with SolrVersion: 4.0.0.2012.04.26.09.00.41
SEVERE: Exception while solr commit.
java.io.FileNotFoundException: _8l.cfs
at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:266)
at
In my older version of solr this was possible, but it seems not possible in
this new =(
--
View this message in context:
http://lucene.472066.n3.nabble.com/FileNotFoundException-during-commit-concurrences-process-tp3991384p3991388.html
Sent from the Solr - User mailing list archive at
Hello.
i running a solr replication. works well, but i need to replicate my
dataimport-properties.
if server1 replicate this file after he create everytime a new file, with
*.timestamp, because the first replication run create this file with wrong
permissions ...
how can is say to solr
my setup includes a asynchron replication.
this means, both are master AND slave at the same time. so i can easy switch
master and slave on the fly without resarting any server with mass of
scripts ... i trigger a replication via cronjob and look everytime, if
server is master or slave. only
my setup includes a asynchron replication.
this means, both are master AND slave at the same time. so i can easy switch
master and slave on the fly without resarting any server with mass of
scripts ... i trigger a replication via cronjob and look everytime, if
server is master or slave. only
is it not fucking possible to import DIH !?!?!? WTF!
i load the new solr 3.6. from website. but when i started solr i got
evertime no DIH found
if i put in my solrconfig
lib dir=../dist/ regex=apache-solr-dataimporthandler-.*\.jar /
AND have this structure
hello..
anyone a idea how i can figure out why my replication failed ? i got no
errors =(
my configuratio is.
2 server! both are master and slave at the same time. only one server makes
updates and is so the master. on slave is started via cron a replication. is
one server crashed, i can
bevore this problem i got this problem
https://issues.apache.org/jira/browse/SOLR-1781
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 8 Cores,
1 Core with 45 Million Documents other Cores 200.000
- Solr1 for
@Shawn Heisey-4
how look your requestHandler of your broker? i think about your idea to do
the same ;)
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 8 Cores,
1 Core with 45 Million Documents other Cores 200.000
i have 8 cores ;-)
i thought that replication is defined in solrconfig.xml and this file is
only load on startup and i cannot change master to slave and slave to master
without restarting the servlet-container ?!?!?!
-
--- System
Hello.
Is it possible to switch master/slave on the fly without restarting the
server?
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 8 Cores,
1 Core with 45 Million Documents other Cores 200.000
- Solr1 for
Did your configuration works ?
i have the same issue and i dont know if it works...
i have 2 servers. each with 2 solr instances (one for updates other for
searching) now i need replication from solr1 to solr2. but what the hell do
solr if master crashed ???
-
Hello.
Which RH do you use to find typing errors like goolge = do you mean
google ?!
I want to use my Autosuggestion EdgeNGram with a clever AutoCorrection!
What do you use ?
-
--- System
One Server, 12 GB RAM, 2 Solr
Why does this not work?
fieldType name=city class=solr.TextField
analyzer
charfilter class=solr.PatternReplaceFilterFactory
pattern=^(\,
)$ replacement= replace=first /
OR
charfilter
okay, thx =)
but i replace it now in my data-config ;)
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 8 Cores,
1 Core with 45 Million Documents other Cores 200.000
- Solr1 for Search-Requests - commit every
I Read this Articel from Mark Miller
http://www.lucidimagination.com/blog/2011/07/11/benchmarking-the-new-solr-%E2%80%98near-realtime%E2%80%99-improvements/
Now i want to know if its useful to update on a new solr version. My version
is: 4.0.0.2010.10.26.08.43.14
I need a really good NRT search
sry.
i need the sum of values of the found documents. e.g. the total amount of
one day. each doc in index has ist own amount.
i try out something with StatsComponent but with 48 Million docs in Index
its to slow.
-
--- System
yes, this way i am using on another part in my application. i hoped, that
exists another way to avoid the way over php
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 8 Cores,
1 Core with 45 Million Documents other
hi thanks for the big reply ;)
i had the idea with the several and small 5M shards too.
and i think thats the next step i have to go, because our biggest index
grows each day with avg. 50K documents.
but make it sense to keep searcher AND updater cores on one big server? i
dont want to use
i think with incident he mean, failures / downtimes / problems with solr !?
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 8 Cores,
1 Core with 45 Million Documents other Cores 200.000
- Solr1 for Search-Requests
which type in the schema.xml do you use.
try out WordDelimiterFilterFactory or some other filters from this site:
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.WordDelimiterFilterFactory
-
--- System
index this field without whitespaces ? XD
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 8 Cores,
1 Core with 45 Million Documents other Cores 200.000
- Solr1 for Search-Requests - commit every Minute - 5GB Xmx
-
http://wiki.apache.org/solr/Join
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 8 Cores,
1 Core with 45 Million Documents other Cores 200.000
- Solr1 for Search-Requests - commit every Minute - 5GB Xmx
- Solr2
okay, thanks a lot.
I thought, that isnt possible to get the month in my case =( i will try out
another way.
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 8 Cores,
1 Core with 45 Million Documents other Cores
thx =)
i think i will save this as an string if ranges really works =)
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 8 Cores,
1 Core with 45 Million Documents other Cores 200.000
- Solr1 for Search-Requests -
is it possible to index a datefield in the format of y-m-d ? i dont need
the timestamp. so i can save me some space.
which ways exists to search with a complex date-filter !?
-
--- System
One Server, 12 GB RAM, 2 Solr
Hello.
i am fighting with the FunctionQuery of Solr.
I try to get a diff of today and an dateField. from this diff, i want do a
modulo from another field with values of 1,3,6,12
in a function somthing like this. ( i know that some functions are not
available in solr)
q={!func}$v2=0v1=(NOW -
okay, thx =)
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for Search-Requests - commit every Minute - 5GB Xmx
- Solr2 for Update-Request -
SELECT CONCAT('tablename_', CAST(cp.id AS CHAR)) AS uniquekey,
cp.id,
cp.fieldname ..,
mp.fieldname,
mp.
FROM consumer AS cp
INNER JOIN morepush AS mp
ON cp.id = mp.id
in my query is a norma JOIN select.
-
--- System
new error message:
i am so an IDIOT !!! SORRRYY XD hehe
i wrote zeroDateTimeBehavOIr and not zeroDateTimeBehavior
...
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores
yes.
thx for your help =)
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for Search-Requests - commit every Minute - 5GB Xmx
- Solr2 for
okay. i didn find the problem =(
it sstill the same shit.
i cannot conver with DateTimeFormater dates form -00-00 =
-MM-dd'T'hh:mm:ss'Z'
i put my date fields into another entity:
entity name=dates query=SELECT * FROM consumer WHERE
id='${main.cp_id}'
field
yes. i put the zeroDateTimeBehavior=convertToNull to my url like:
url=jdbc:mysql://localhost/databaseName?zeroDateTimeBehavoir=convertToNull
ExceptoiN:
May 23, 2011 3:30:22 PM org.apache.solr.handler.dataimport.DataImporter
doFullImport
SEVERE: Full Import failed
the problems are not the empty -00-00 values. the problem is the missing
timestamp at the end of the string !
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other
Hello
i want to index some datefields with this dateformat: -mm-dd. Solr
thwows an exception like this: can not be represented as java.sql.Date
i am unsing ...transformer=DateFormatTransformer
and ...zeroDateTimeBehavoir=convertToNull
how can i say to DIH to convert this fields in correct
did you mean something like this ?
DATE_FORMAT(cp.field, '%Y-%m-%di %H:%i:%s') AS field ???
i think i need to add the timestamp to my date fields? or not ?
why cannot DIH handle with this ?
-
--- System
One Server, 12
entity name=foo pk=cp_id transformer=DateFormatTransformer
query=SELECT ...,
...some fields ...
cp.start_date_1,
cp.start_date_2,
cp.end_date_1,
cp.end_date_2,
.. some other fields ..
FROM ...
/entity
okay, i found the problem.
i put the fields two times in my data-config ;-)
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for
what do you mean with segments-number ?
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for Search-Requests - commit every Minute - 5GB Xmx
-
Hello.
How can i set the timezone oft java in my java properties ?
my problem is, that in the dataimport-properties is a wrong timezone and i
dont know how to set the correct timezone ... !?!? thx
-
--- System
One
Hello.
How can i change the default value of optimize in DIH to false ?
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for Search-Requests -
yes, but as default! i dont want to set it from me to false.
i need not after ervery commit an optimize and i want it default=false !
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million
hello.
my optimize is taking tooo much time and sometimes i started a optimize but
i dont want it ... :/ stupit, i know.
is it possible to abort a runnung optimize ?
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7
The current limitation or pause is when the ram buffer is flushing to disk
- when an optimize starts and is running ~4 hours, you say, that DIH is
flushing the doc`s during this pause into the index ?
-
--- System
One
is it necessary to update for solr ?
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for Search-Requests - commit every Minute - 5GB Xmx
-
hello.
my NRT-Search is not correctly configured =(
2 Solr-Instances. one searcher and one updater
the updater start every minute an update of around 3000 documents. and the
searcher start an commit ervery minute to refresh the index and read the new
doc`s
these are my Cache values for an 36
i start a commit on searcher-Core with:
.../core/update?commit=truewaitFlush=false
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for
my filterCache has a warmupTime from ~6000 ... but my config is like this:
LRU Cache(maxSize=3000, initialSize=50, autowarmCount=50 ...)
should i set maxSize to 50 or similar value ?
-
--- System
One Server, 12 GB RAM, 2
oooh. my queryResultCache has a warmupTime from 54000 = ~1 Minute
any suggestions ??
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for
i fighting with the same problem but with jetty.
its in this case necessary to delete also the jetty work-DIR ???
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores
Hello.
When is start an optimize (which takes more than 4 hours) no updates from
DIH are possible.
i thougt solr is copy the hole index and then start an optimize from the
copy and not lock the index and optimize this ... =(
any way to do both in the same time ?
-
Make sure streaming is on.
-- how to check ?
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for Search-Requests - commit every Minute - 5GB
is it possible to use StreamingUpdateSolrServer with a php application ?
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for Search-Requests -
Hello.
i want to change my full-imports from DIH to use of Java and
StreamingUpdateSolrServer ...
is in the wiki a little how to or something similar ?
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1
why is solr copy my complete index to somewhere when i start an delta-import?
i copy one core, start an full-import from 35Million docs and then start an
delta-import from the last hour (~2000Docs).
dih/solr need start to copy the hole index... why ? i think he is copy the
index, because my
Hello again ;-)
after a full-import from 36M Doc`s my delta import dont work fine.
if i starts my delta (which runs on another core very fast) the commit need
vry long.
I think, that solr copys the hole index and commit the new documents in the
index and then reduce the index size after
i have the same problem. any resolutions ?
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for Search-Requests - commit every Minute - 5GB Xmx
Hello.
i get sometimes much results, and solr or jetty give me the error.
EVERE: java.lang.IllegalStateException: Form too large1787345100
numfound ist 94000, not really much, but i get the a double-value from each
doc and calculate the sum over php. when i put the query into browser, a
http://yonik.wordpress.com/2010/07/29/csv-output-for-solr/
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for Search-Requests - commit every
Hello,
my problem is, that after a full-import solr reserved all of my RAM and my
delta-imports need about 1 hour for less than 5000 small documents.
How can i start GarbageCollector to get the RAM back ?
-
--- System
i run an full-import via DIH, 35 Million Documents, i dont restart solr. my
cronjob start automaticly an delta. if i restart solr, delta obtain in ~10
seconds ...
free -m show me how many RAM is beeing used and with top. the server is
only for solr, so no other processes are using my RAM.
-
okay, i installed an monitor, jconsole and jvisualvm. how can i see with
this, where my probem is ?
what data are needed ? :/
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million
it'll negatively impact the desired goal of low latency new index readers?
- yes, i think so, thats the reason because i dont understand the
wiki-article ...
i set the warmupCount to 500 and i got no error messages, that solr isnt
available ...
but solr-stats.jsp show me a warmuptime of
okay, not the time ... the items ...
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for Search-Requests - commit every Minute - 5GB Xmx
-
Maybe the article is out of date?
- maybe .. i dont know
in my case it make no sense and i use another configuration ...
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents
i am using solr for NRT with this version of solr ...
Solr Specification Version: 4.0.0.2010.10.26.08.43.14
Solr Implementation Version: 4.0-2010-10-26_08-05-39 1027394 - hudson -
2010-10-26 08:43:14
Lucene Specification Version: 4.0-2010-10-26_08-05-39
Lucene Implementation Version:
question: http://wiki.apache.org/solr/NearRealtimeSearchTuning
'PERFORMANCE WARNING: Overlapping onDeckSearchers=x
i got this message.
in my solrconfig.xml: maxWarmingSearchers=4, if i set this to 1 or 2 i got
exception. with 4 i got nothing, but the Performance Warning. the
wiki-articel
Are you using shards or have everything in same index?
- shards == distributed Search over several cores ? = yes, but not always.
but in generally not.
What problem did you experience with the StatsCompnent?
- if i use stats on my 34Million Index, no matter how many docs founded, the
sum takes
i am using NRT, and the caches are not always warmed, i think this is almost
a problem !?
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for
I tried to create an NRT like in the wiki but i got some problems with
autowarming and ondeckSearchers.
ervery minute i start a delta of one core and the other core start every
minute a commit of the index to search for it.
wiki says ... = 1 Searcher and fitlerCache warmupCount=3600. with this
make it sense to update solr for getting SOLR-571 ???
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for Search-Requests - commit every Minute
Hello.
i have 34.000.000 documents in my index and each doc have a field with a
double-value. i want the sum of these fields. i testet it with the
statscomponent but this is not usable. !! so i get all my values directly
from solr, from the index and with php-sum() i get my sum.
that works fine
my error is, that solr is not reachable with a ping.
ping over php-HttpRequest ...
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for
Hello.
i have the field reason_1 and reason_2. this two fields is in my schema one
dynamicField: dynamicField name=reason_* type=textgen indexed=true
stored=false/
i copy this field in my text-default search field: copyField
source=reason_* dest=text/
And in a new field reason: copyField
the fieldType is textgen.
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with 31 Million Documents other Cores 100.000
- Solr1 for Search-Requests - commit every Minute - 4GB Xmx
- Solr2 for
the documents havent the same uniquekey, only reason is the same.
i cannot show the exactly search request, because of privacy policy...
the query is like that:
reason_1: firstname lastname,
reason_2: 1234, 02.02.2011
-- in field reason: firstname lastname, 1234, 02.02.2011
the search
Hello.
I do every 2 Minutes a Delta and if one Core (of 7) is running a delta, solr
isnt available. when i look in the logFile the ping comes in this time, when
DocBuilder is running ...
Feb 15, 2011 11:49:20 AM org.apache.solr.handler.dataimport.DocBuilder
doDelta
INFO: Delta Import completed
or change the index to a compound-index
solrconfig.xml: useCompoundFiletrue/useCompoundFile
so solr creates one index file and not thousands.
-
--- System
One Server, 12 GB RAM, 2 Solr Instances, 7 Cores,
1 Core with
i found the problem.
DIH or i think the JDBC-Driver casting 0 and 1 to boolean, if the field in
database from type (tinyint(1)).
iam using tow fields with type of tinyint(1) and tinyint(2) -.-
-
--- System
One Server, 12
Hello-
i am using shard-requests over several cores.
each core has his own index and own schema. but every core have the field
status !
regularly the status is 0 or 1. but one core can have the status: 0, 1 OR 2
-- the status field type i used is string but string make the cores (with
only 0
1 - 100 of 229 matches
Mail list logo