[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13693019#comment-13693019
]
Otis Gospodnetic commented on SOLR-1301:
[~kanarsky] - yes, take Solr results and
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13688330#comment-13688330
]
Alexander Kanarsky commented on SOLR-1301:
--
[~otis], do you mean to use the Solr
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13679070#comment-13679070
]
Mark Miller commented on SOLR-1301:
---
Yeah, we have taken this issue as a starting point
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13678948#comment-13678948
]
Otis Gospodnetic commented on SOLR-1301:
Noticed this is issue #11 in terms of
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214152#comment-13214152
]
Alexander Kanarsky commented on SOLR-1301:
--
OK, so I changed the patch to work
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13155291#comment-13155291
]
Mark Johnson commented on SOLR-1301:
Has anyone updated this contrib to work with the
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13109190#comment-13109190
]
Alexander Kanarsky commented on SOLR-1301:
--
Viktors, can you increase the number
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13106900#comment-13106900
]
Viktors Rotanovs commented on SOLR-1301:
Beware: with ZIP option enabled, this
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13090429#comment-13090429
]
Alexander Kanarsky commented on SOLR-1301:
--
Mark, I planned to add some unit tests
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13088829#comment-13088829
]
Mark Johnson commented on SOLR-1301:
It appears that this issue has fallen by the
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13088860#comment-13088860
]
Mark Johnson commented on SOLR-1301:
Also does anyone have the json converter listed in
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030586#comment-13030586
]
Lance Norskog commented on SOLR-1301:
-
Hadoop contains something called MR, a unit test
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981930#action_12981930
]
Alexander Kanarsky commented on SOLR-1301:
--
Note for the Hadoop 0.21 users: the
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12920402#action_12920402
]
Dhruv Bansal commented on SOLR-1301:
I am unable to compile SOLR 1.4.1 after patching
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12912497#action_12912497
]
Jason Rutherglen commented on SOLR-1301:
Alexander,
I think we'll need to use
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12907347#action_12907347
]
Alexander Kanarsky commented on SOLR-1301:
--
Grant, sure. Will do this in a next
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12902645#action_12902645
]
Daniel Ivan Pizarro commented on SOLR-1301:
---
I'm getting the following error:
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12897507#action_12897507
]
Alexander Kanarsky commented on SOLR-1301:
--
Mathias, I did not test the patch for
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12896514#action_12896514
]
Mathias Walter commented on SOLR-1301:
--
I tried this patch with Hadoop 0.20.2. It works
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12875949#action_12875949
]
Jason Rutherglen commented on SOLR-1301:
bq. Matt, I think Viktors mentioned the
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12873577#action_12873577
]
Koji Sekiguchi commented on SOLR-1301:
--
We are using this patch (Andrzej version +
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12873290#action_12873290
]
Marc Sturlese commented on SOLR-1301:
-
Can someone tell me wich org.apache.commons.csv
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12869977#action_12869977
]
Matt Revelle commented on SOLR-1301:
Viktors: That must have been a regression from
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12864593#action_12864593
]
Jason Rutherglen commented on SOLR-1301:
Matt, Can you post a patch including the
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12859853#action_12859853
]
Jason Rutherglen commented on SOLR-1301:
Matt, interesting. I'm most concerned
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12833108#action_12833108
]
Jason Rutherglen commented on SOLR-1301:
There still seems to be a bug where the
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12829200#action_12829200
]
Jason Rutherglen commented on SOLR-1301:
In production the latest patch does not
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12828915#action_12828915
]
shyjuThomas commented on SOLR-1301:
---
I have a need to perform Solr indexing in MapReduce
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12828961#action_12828961
]
Ted Dunning commented on SOLR-1301:
---
{quote}
Based on these observation, I have few
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12828172#action_12828172
]
Jason Rutherglen commented on SOLR-1301:
There's a bug caused by the latest change:
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12828232#action_12828232
]
Kay Kay commented on SOLR-1301:
---
Did the latest patch involve an upgrade of the hdfs / patched
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12828356#action_12828356
]
Kevin Peterson commented on SOLR-1301:
--
I pointed you in the wrong direction. It isn't
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12828368#action_12828368
]
Jason Rutherglen commented on SOLR-1301:
I'm testing deleting the temp dir in
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12806547#action_12806547
]
Ted Dunning commented on SOLR-1301:
---
It is critical to put indexes in the task local area
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12800720#action_12800720
]
Grant Ingersoll commented on SOLR-1301:
---
Seems like this would make the most sense as
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12800723#action_12800723
]
Grant Ingersoll commented on SOLR-1301:
---
bq. Furthermore, by using an
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12800746#action_12800746
]
Andrzej Bialecki commented on SOLR-1301:
-
bq. I'm curious about the not sending
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12800748#action_12800748
]
Grant Ingersoll commented on SOLR-1301:
---
bq. Hmm, I don't think this would make sense
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12800756#action_12800756
]
Jason Rutherglen commented on SOLR-1301:
Andrzej's model works great in production.
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12800760#action_12800760
]
Grant Ingersoll commented on SOLR-1301:
---
Don't confuse the ZK stuff for search w/ the
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12800758#action_12800758
]
Andrzej Bialecki commented on SOLR-1301:
-
Iff we somehow could get a mapping
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12800775#action_12800775
]
Jason Rutherglen commented on SOLR-1301:
{quote}What I meant was the Hadoop job
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12800785#action_12800785
]
Grant Ingersoll commented on SOLR-1301:
---
I don't follow how sending docs to a suite of
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12800802#action_12800802
]
Jason Rutherglen commented on SOLR-1301:
bq. Hadoop streaming the output of the
This can also be a big performance win. Jason Venner reports significant
index and cluster start time improvements by indexing to local disk, zipping
and then uploading the resulting zip file. Hadoop has significant file open
overhead so moving one zip file wins big over many index component
On 2010-01-15 20:13, Ted Dunning wrote:
This can also be a big performance win. Jason Venner reports significant
index and cluster start time improvements by indexing to local disk, zipping
and then uploading the resulting zip file. Hadoop has significant file open
overhead so moving one zip
Zipping cores/shards is in the latest patch...
On Fri, Jan 15, 2010 at 11:22 AM, Andrzej Bialecki a...@getopt.org wrote:
On 2010-01-15 20:13, Ted Dunning wrote:
This can also be a big performance win. Jason Venner reports significant
index and cluster start time improvements by indexing to
I can see why that is a win over the existing, but I still don't get why it
wouldn't be faster just to index to a suite of Solr master indexers and save
all this file slogging around. But, I guess that is a separate patch all
together.
On Jan 15, 2010, at 2:35 PM, Jason Rutherglen wrote:
The reason I would a major speed win when expect indexing to local disk and
copying later is that you get much more efficient reading of documents with
normal hadoop mechanisms. Throwing documents to the various Solr master
indexers is bound to be slower than having 20 machines reading at local
Copying files ala HDFS is trivial because it's sequential,
Lucene merging isn't, so scaling merging over 20 machines vs 4 Solr
has clear advantages... That and on-demand expandability, so I
can reindex 2 terabytes of data in half a day vs weeks or more
with 4 Solr masters has compelling
We index comparable amounts of data in a few hours.
On Fri, Jan 15, 2010 at 1:08 PM, Jason Rutherglen
jason.rutherg...@gmail.com wrote:
That and on-demand expandability, so I
can reindex 2 terabytes of data in half a day vs weeks or more
with 4 Solr masters has compelling advantages.
--
Makes sense. Interesting exercise to think about.
On Jan 15, 2010, at 4:08 PM, Jason Rutherglen wrote:
Copying files ala HDFS is trivial because it's sequential,
Lucene merging isn't, so scaling merging over 20 machines vs 4 Solr
has clear advantages... That and on-demand expandability, so I
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12764717#action_12764717
]
Jason Venner (www.prohadoop.com) commented on SOLR-1301:
I need to
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12764745#action_12764745
]
Jason Rutherglen commented on SOLR-1301:
Thanks for the update Jason. It runs
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12760309#action_12760309
]
Jason Rutherglen commented on SOLR-1301:
We need to include the schema.xml in the
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12754276#action_12754276
]
jv ning commented on SOLR-1301:
---
I have an updated version that uses a sequence number, to
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12753731#action_12753731
]
Jason Rutherglen commented on SOLR-1301:
Should we add ThreadedIndexWriter (from
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12753739#action_12753739
]
Yonik Seeley commented on SOLR-1301:
I don't know anything about ThreadedIndexWriter,
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12753754#action_12753754
]
jv ning commented on SOLR-1301:
---
Within a Map/Reduce task, there is usually a significant
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12753756#action_12753756
]
jv ning commented on SOLR-1301:
---
My notes on the patchupdate were in a README.txt that didn't
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12753765#action_12753765
]
Jason Rutherglen commented on SOLR-1301:
Yonik,
It looks like
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12753769#action_12753769
]
Jason Rutherglen commented on SOLR-1301:
{quote} In the ideal world, the Map/Reduce
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12753317#action_12753317
]
Jason Rutherglen commented on SOLR-1301:
I think we can parallelize the indexing in
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12753332#action_12753332
]
jv ning commented on SOLR-1301:
---
In my case, I have 6 tasks per machine, but only 4 disks, so
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12748935#action_12748935
]
jv ning commented on SOLR-1301:
---
I have used this at a decent scale, and will be adding a few
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12747492#action_12747492
]
jv ning commented on SOLR-1301:
---
Anyone using this patch set?
What are people using for the
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12747525#action_12747525
]
Jason Rutherglen commented on SOLR-1301:
Jv,
I've used the patch. It works, though
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12747542#action_12747542
]
jv ning commented on SOLR-1301:
---
currently you pass the directory of your solr conf/lib to the
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12737568#action_12737568
]
Ken Krugler commented on SOLR-1301:
---
Hi Jason,
Re Katta, you're right that it doesn't
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12737253#action_12737253
]
Andrzej Bialecki commented on SOLR-1301:
-
This patch is intended to work with Solr
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12737299#action_12737299
]
Jason Rutherglen commented on SOLR-1301:
Andrzej,
* Are you going to add a way to
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12737306#action_12737306
]
Andrzej Bialecki commented on SOLR-1301:
-
bq. Are you going to add a way to
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12736944#action_12736944
]
Jason Rutherglen commented on SOLR-1301:
I think we'll want to integrate this patch
[
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12734376#action_12734376
]
Jason Rutherglen commented on SOLR-1301:
I downloaded the patch. I'd like to be
74 matches
Mail list logo
