Hi Shawn,
I am new to solr and I have set up a cloud cluster of 1 shard and 3
collections one 2 servers. I am facing the same issue. I am using
CloudSolrClient client = new
CloudSolrClient.Builder(zkUrls,Optional.empty()).build(), to create my
client.
and then I fire import command using,
clien
On 12/6/2017 1:38 AM, Mahmoud Almokadem wrote:
> I'm already using the admin UI and get URL for fetching the status of
> dataimporter from network console and tried it outside the admin UI. Admin
> UI have the same behavior, when I pressed on execute the status messages
> are swapped between "not
Thanks Shawn,
I'm already using the admin UI and get URL for fetching the status of
dataimporter from network console and tried it outside the admin UI. Admin
UI have the same behavior, when I pressed on execute the status messages
are swapped between "not started", "started and indexing", "compl
On 12/3/2017 9:27 AM, Mahmoud Almokadem wrote:
We're facing an issue related to the dataimporter status on new Admin UI
(7.0.1).
Calling to the API
http://solrip/solr/collection/dataimport?_=1512314812090&command=status&indent=on&wt=json
returns different status despite the importer is running
or could i use a filter in schema.xml where i define a fieldtype and use some
filter that understands xpath?
On 4. Sep 2013, at 11:52 AM, Shalin Shekhar Mangar wrote:
> No that wouldn't work. It seems that you probably need a custom
> Transformer to extract the right div content. I do not know i
No that wouldn't work. It seems that you probably need a custom
Transformer to extract the right div content. I do not know if
TikaEntityProcessor supports such a thing.
On Wed, Sep 4, 2013 at 12:38 PM, Andreas Owen wrote:
> so could i just nest it in a XPathEntityProcessor to filter the html or
so could i just nest it in a XPathEntityProcessor to filter the html or is
there something like xpath for tika?
but now i dont know how to pass the text to tika, what do i put in url and
datasou
I don't know much about Tika but in the example data-config.xml that
you posted, the "xpath" attribute on the field "text" won't work
because the xpath attribute is used only by a XPathEntityProcessor.
On Thu, Aug 29, 2013 at 10:20 PM, Andreas Owen wrote:
> I want tika to only index the content i
i changed following line (xpath):
On 22. Aug 2013, at 10:06 PM, Alexandre Rafalovitch wrote:
> Ah. That's because Tika processor does not support path extraction. You
> need to nest one more level.
>
> Regards,
> Alex
> On 22 Aug 2013 13:34, "Andreas Owen" wrote:
>
>> i can do it like th
ok but i'm not doing any path extraction, at least i don't think so.
htmlMapper="identity" isn't preserving html
it's reading the content of the pages but it's not putting it into "text_test"
and "text". it's only in "text_test" the copyField isn't working.
data-config.xml:
Ah. That's because Tika processor does not support path extraction. You
need to nest one more level.
Regards,
Alex
On 22 Aug 2013 13:34, "Andreas Owen" wrote:
> i can do it like this but then the content isn't copied to text. it's just
> in text_test
>
> url="${rec.path}${rec.file}" dataS
i can do it like this but then the content isn't copied to text. it's just in
text_test
On 22. Aug 2013, at 6:12 PM, Andreas Owen wrote:
> i put it in the tika-entity as attribute, but it doesn't change anything. my
> bigger concern is why text_test isn't populated at all
i put it in the tika-entity as attribute, but it doesn't change anything. my
bigger concern is why text_test isn't populated at all
On 22. Aug 2013, at 5:27 PM, Alexandre Rafalovitch wrote:
> Can you try SOLR-4530 switch:
> https://issues.apache.org/jira/browse/SOLR-4530
>
> Specifically, setti
Can you try SOLR-4530 switch:
https://issues.apache.org/jira/browse/SOLR-4530
Specifically, setting htmlMapper="identity" on the entity definition. This
will tell Tika to send full HTML rather than a seriously stripped one.
Regards,
Alex.
Personal website: http://www.outerthoughts.com/
LinkedIn:
i have tried post.jar and it works when i set the literal.id in solrconfig.xml.
i can't pass the id with post.jar (-Dparams=literal.id=abc) because i get a
error: "could not find or load main class .id=abc".
On 20. Jul 2013, at 7:05 PM, Andreas Owen wrote:
> path was set text wasn't, but it do
path was set text wasn't, but it doesn't make a difference. my importer says 1
row fetched, 0 docs processed, 0 docs skipped. i don't understand how it can
have 2 docs indexed with such a output.
On 20. Jul 2013, at 12:47 PM, Shalin Shekhar Mangar wrote:
> Are the "path" and "text" fields set
Are the "path" and "text" fields set to "stored" in the schema.xml?
On Sat, Jul 20, 2013 at 3:37 PM, Andreas Owen wrote:
> they are in my schema, path is typed correctly the others are default
> fields which already exist. all the other fields are populated and i can
> search for them, just pat
they are in my schema, path is typed correctly the others are default fields
which already exist. all the other fields are populated and i can search for
them, just path and text aren't.
On 19. Jul 2013, at 6:16 PM, Alexandre Rafalovitch wrote:
> Dumb question: they are in your schema? Spelled
Dumb question: they are in your schema? Spelled right, in the right
section, using types also defined? Can you populate them by hand with a CSV
file and post.jar?
Regards,
Alex.
Personal website: http://www.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is
On Mon, Oct 25, 2010 at 10:12 AM, Dario Rigolin
wrote:
> Looking at DataImporter I'm not sure if it's possible to import using a
> standard ... xml document representing a document add operation.
> Generating is quite expensive in my application and I have
> cached
> all those documents into a te
ecleare for the preImportDeleteQuery.
Thanks for your time.
-Original Message-
From: Bilgin Ibryam [mailto:bibr...@gmail.com]
Sent: mercoledì 14 luglio 2010 14.46
To: solr-user@lucene.apache.org
Subject: Re: DataImporter
Is it possible that you have the same IDs in both entities?
Could you show h
Is it possible that you have the same IDs in both entities?
Could you show here your entity mappings?
Bilgin Ibryam
On Wed, Jul 14, 2010 at 11:48 AM, Amdebirhan, Samson, VF-Group <
samson.amdebir...@vodafone.com> wrote:
> Hi all,
>
>
>
> Can someone help me in this ?
>
>
>
> Importing 2 differen
On Thu, Apr 16, 2009 at 10:31 AM, Mani Kumar wrote:
> Aah, Bryan you got it ... Thanks!
> Noble: so i can hope that it'll be fixed soon :) thank you for fixing it
> ...
> please lemme know when its done..
>
This is fixed in trunk. The next nightly build should have this fix.
--
Regards,
Shalin
Aah, Bryan you got it ... Thanks!
Noble: so i can hope that it'll be fixed soon :) thank you for fixing it ...
please lemme know when its done..
Thanks!
Mani Kumar
2009/4/16 Noble Paul നോബിള് नोब्ळ्
> Hi Bryan,
> Thanks a lot. It is invoking the wrong method
>
> it should have been
> bsz = con
Hi Bryan,
Thanks a lot. It is invoking the wrong method
it should have been
bsz = context.getVariableResolver().replaceTokens(bsz);
it was a silly mistake
--Noble
On Thu, Apr 16, 2009 at 2:13 AM, Bryan Talbot wrote:
> I think there is a bug in the 1.4 daily builds of data import handler which
I think there is a bug in the 1.4 daily builds of data import handler
which is causing the batchSize parameter to be ignored. This was
probably introduced with more recent patches to resolve variables.
The affected code is in JdbcDataSource.java
String bsz = initProps.getProperty("batch
DIH streams 1 row at a time.
DIH is just a component in Solr. Solr indexing also takes a lot of memory
On Tue, Apr 14, 2009 at 12:02 PM, Mani Kumar wrote:
> Yes its throwing the same OOM error and from same place...
> yes i will try increasing the size ... just curious : how this dataimport
> wo
Yes its throwing the same OOM error and from same place...
yes i will try increasing the size ... just curious : how this dataimport
works?
Does it loads the whole table into memory?
Is there any estimate about how much memory it needs to create index for 1GB
of data.
thx
mani
On Tue, Apr 14, 2
On Tue, Apr 14, 2009 at 11:36 AM, Mani Kumar wrote:
> Hi Shalin:
> yes i tried with batchSize="-1" parameter as well
>
> here the config i tried with
>
>
>
> driver="com.mysql.jdbc.Driver"
> url="jdbc:mysql://localhost/mydb_development"
> user="root" password="**" />
>
>
> I hope i have u
Hi Shalin:
yes i tried with batchSize="-1" parameter as well
here the config i tried with
I hope i have used batchSize parameter @ right place.
Thanks!
Mani Kumar
On Tue, Apr 14, 2009 at 11:24 AM, Shalin Shekhar Mangar <
s
On Tue, Apr 14, 2009 at 11:18 AM, Mani Kumar wrote:
> Here is the stack trace:
>
> notice in stack trace * "at
> com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1749)"*
>
> It looks like that its trying to read whole table into memory at a time. n
> thts y getting OOM.
>
>
Mani, the data-
Hi Noble:
But the question is how much memory? is there any rules or something like
that? so that i can estimate the how much memory it requires?
Yeah i can increase it upto 800MB max will try it and let you know
Thanks!
Mani
2009/4/14 Noble Paul നോബിള് नोब्ळ्
> DIH itself may not be con
Here is the stack trace:
notice in stack trace * "at
com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1749)"*
It looks like that its trying to read whole table into memory at a time. n
thts y getting OOM.
Apr 14, 2009 11:15:01 AM org.apache.solr.handler.dataimport.DataImporter
doFullImpo
DIH itself may not be consuming so much memory. It also includes the
memory used by Solr.
Do you have a hard limit on 400MB , is it not possible to increase it?
On Tue, Apr 14, 2009 at 11:09 AM, Mani Kumar wrote:
> Hi ILAN:
>
> Only one query is required to generate a document ...
> Here is my
Hi ILAN:
Only one query is required to generate a document ...
Here is my data-config.xml
and other useful info:
mysql> select * from items
+--+
| count(*) |
+--+
| 900051 |
+--+
1 row in set (0.00 sec)
Each
Depending on your dataset and how your queries look you may very likely
need to increase to a larger heap size. How many queries and rows are
required for each of your documents to be generated?
Ilan
On 4/13/09 12:21 PM, Mani Kumar wrote:
Hi Shalin:
Thanks for quick response!
By defaults i
Hi Shalin:
Thanks for quick response!
By defaults it was set to 1.93 MB.
But i also tried it with following command:
$ ./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M
I also tried tricks given on
http://wiki.apache.org/solr/DataImportHandlerFaq page.
what should i try next ?
On Mon, Apr 13, 2009 at 11:57 PM, Mani Kumar wrote:
> Hi All,
> I am trying to setup a Solr instance on my macbook.
>
> I get following errors when m trying to do a full db import ... please help
> me on this
>
> java.lang.OutOfMemoryError: Java heap space
>at
>
> org.apache.solr.handler.d
I am using Tomcat ...
On Mon, Apr 13, 2009 at 11:57 PM, Mani Kumar wrote:
> Hi All,
> I am trying to setup a Solr instance on my macbook.
>
> I get following errors when m trying to do a full db import ... please help
> me on this
>
> Apr 13, 2009 11:53:28 PM
> org.apache.solr.handler.dataimport.
Currently there is nothing . There is a hackish way to achieve it.
DIH allows to read values from request params and use it in the templates.
eg: query="select * from atable where id > ${dataimporter.request.last_id}"
so, DIH must be invoked with the extra request param last_id like this
http://:/
40 matches
Mail list logo
