To Issue 5: unfortunately, there are some hidden NPEs in the bulk request API.
Check this https://github.com/jprante/elasticsearch-support/blob/master/src/test/java/org/xbib/elasticsearch/support/various/NPETest.java if you find a pattern that matches your case. Jörg On Tue, Jul 22, 2014 at 8:00 PM, Alain Désilets <[email protected]> wrote: > Thx Jörg, > > Your comments helped me get quite a bit further. Here is an updated > version of my code: > > https://gist.github.com/alaindesilets/aec9492890c37075fa4e > > > On Monday, 21 July 2014 13:13:58 UTC-4, Jörg Prante wrote: >> >> To issue 1: you create a single node cluster without index, and a client >> of it. >> > > Duh! Don't know how I could have missed that. So now, I added a > method createIndex() which creates the index if it doesn't exist, and sets > number_of_replicas to 2. It doesn't define mappings since the ES doc says > that default mappings will be automatically generated if none are specified. > > Note however that even with that change, I still couldn't see the new > index in Marvel/Sense, when I instantiated the client with > method makeClientFromEmbeddedNode(). But if I instantiate the client > through a TransportClient, i.e. by invoking > method makeClientFromTransportClient(), then Marvel/Sense sees the new > index. > > Using a TransportClient instead of a client obtained from an embedded node > also greatly accelerated the client creation. Instead of taking > 8 secs to > create the client, it takes < 1 sec. Not sure why. > > >> >> To issue 2: you see the UnavailableShardsException caused by a timeout >> while indexing to a replica shard. This means, you may have set up a single >> node cluster, but with replica level 1 (default) which needs 2 nodes for >> indexing. Maybe there was once another node joining the cluster and ES >> wants it back abut it never came (after 60 secs). Then ES returns the >> timeout error. Maybe replica level 0 helps. You should also check the >> cluster health. A color of green shows everything works, yellow means there >> are too few nodes to satisfy the replica condition, and read means the >> cluster is not in a consistent/usable state. >> > > I tried setting number_replica=0 and number_replica=1 in createIndex(), > but I still got the org.elasticsearch.action.UnavailableShardsException > error if I instantiate the client with makeClientFromEmbeddedNode(). But I > don't get the error if I instantiate the client with > makeClientFromTransportClient(), independantly of the number of replicas I > specify. > > >> To issue 3: not sure what clusterName() means. I would use settings and >> add a field "cluster.name". Maybe it is equivalent. You must ensure you >> use the same "cluster.name" setting throughout all nodes and clients. >> You also can not reuse data from clusters that had other names (look into >> the "data" folder) >> > > I have now moved that code to a method called > makeClientFromNamedClusterNode(). I haven't been able to make that work at > all. > > To issue 4: ES takes ~5 secs for discovery, the zen modules pings and >> waits for responding master nodes by default. If you just test locally on >> your developer machine, you should disable zen. Most easily by disabling >> networking at all, by using NodeBuilder.nodeBuilder().local(true)... >> > > Not sure I understand what that means, but in any case, using a > TransportClient seems to address that issue. > > So, all in all, I feel I am in pretty good shape now. Thanks for the help. > > There is one new Issue I an now encountering, when I try to do a bulk > indexing, namely: > > Issue 5: If I uncomment the call to index100NewBeersInOneBatch(), I get > the following: > > == Indexing 100 new beer objects as in one batch... (Elapsed so far: 1.0 > seconds) > Adding beer no 0 to the bulk request. (Elapsed so far: 1.0 seconds) > Exception in thread "main" java.lang.NullPointerException > at org.elasticsearch.action.bulk.BulkRequest.internalAdd( > BulkRequest.java:129) > at org.elasticsearch.action.bulk.BulkRequest.add(BulkRequest.java: > 118) > at org.elasticsearch.action.bulk.BulkRequestBuilder.add( > BulkRequestBuilder.java:52) > at ca.nrc.ElasticSearch.ElasticSearchDemo. > index100NewBeersInOneBatch(ElasticSearchDemo.java:158) > at ca.nrc.ElasticSearch.ElasticSearchDemo.main(ElasticSearchDemo. > java:72) > > > Any thoughts on what might be going on there? > > Thx. > > Alain > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/c3f6c329-4dc4-4714-8e6b-bae1b7fa1f51%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/c3f6c329-4dc4-4714-8e6b-bae1b7fa1f51%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEpsONV0b40mhtepRs7ksnzQqX_%2BjPovVPZupOGKiK7JA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
