HI Erik,
Thank you for your reply. I wrote into a file. Allmy love to cat * > file.
I structured my JSON in a format that I would like to upload into solr.
defined a schema.xml and solrconfig.xml and took it from there. Initiallu I
uploaded a 1G with post then I got a bit to over zealous I
Yep, going to smaller files (and possibly feeding
them through multiple clients) is the way to go
here.
You could also use a SolrJ client which would be
more efficient, here's a place to start. Admittedly
it doesn't parse JSON, but should give you an idea
of how you could go about it if you
Hi,
I am relatively new to Solr and have usage query. I have a 20 GB JSON file
which I want to upload into my solr. Do I have to form smaller chunks or is
there a way to upload the whole thing in one go?
I am getting the following error with bin/post:
Entering auto mode. File endings considered
Let's back up quite a ways here. Where did the 20G file come from?
Indexing files in JSON requires that they follow a very specific format,
Solr doesn't index arbitrary JSON files.
With that out of the way, yes, 20G is unlikely to work without tweaking
some parameters in both solrconfig.xml
