Timeout error during commit

[Steven White]

Hi folks,

I'm indexing about 1 billion records (each are small Solr doc, no more than
20 bytes each).  The logic is basically as follows:

    while (data-of-1-billion) {
        read-1000-items from DB
        at-100-items send 100 items to Solr: i.e.:
solrConnection.add(docs);
    }
    solrConnection.commit()

I'm seeing the following expection from SolrJ:

org.apache.solr.client.solrj.SolrServerException: Timeout occured while
waiting response from server at: http://localhost:8983/solr/test_data

Looking at Solr's log, I see this:

INFO  - 2016-01-15 21:15:34.836; [   test_data]
org.apache.solr.update.processor.LogUpdateProcessor; [test_data]
webapp=/solr path=/update params={wt=xml&version=2.2} {add=[, ... (101
adds)]} 0 5172

Which tells me it took Solr a bit over 5 sec. to complete the commit.

Now when I created the Solr connection, I used 5 seconds like so:

    solrClient.setConnectionTimeout(5000;
  solrClient.setSoTimeout(5000);

Two questions:

1) Is the time out error because of my use of 5000?
2) Should I be calling "solrConnection.commit()" every now and than inside
the loop?

Thanks

Steve