ok, i installed 0.90.10 version of elasticsearch instead and rebooted,
mounted the image again and tried to index the files
it seems to fail to get _meta from fs mydocs, not sure what im doing wrong.
im running
java version "1.7.0_55"
elasticsearch 0.90.10
fsriver 1.0.0
curl 7.35.0
php 5.5.9
output of var/log/..
[2014-05-05 19:58:33,545][INFO ][node ] [Rax]
version[0.90.10], pid[2066], build[0a5781f/2014-01-10T10:18:37Z]
[2014-05-05 19:58:33,545][INFO ][node ] [Rax]
initializing ...
[2014-05-05 19:58:33,585][INFO ][plugins ] [Rax] loaded
[river-fs], sites []
[2014-05-05 19:58:35,307][INFO ][node ] [Rax]
initialized
[2014-05-05 19:58:35,308][INFO ][node ] [Rax] starting
...
[2014-05-05 19:58:35,363][INFO ][transport ] [Rax]
bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
192.168.1.65:9300]}
[2014-05-05 19:58:38,426][INFO ][cluster.service ] [Rax]
new_master [Rax][jG-qvr0jRTyMmvR9-NkrYA][inet[/192.168.1.65:9300]], reason:
zen-disco-join (elected_as_master)
[2014-05-05 19:58:38,454][INFO ][discovery ] [Rax]
elasticsearch/jG-qvr0jRTyMmvR9-NkrYA
[2014-05-05 19:58:38,469][INFO ][http ] [Rax]
bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/
192.168.1.65:9200]}
[2014-05-05 19:58:38,470][INFO ][node ] [Rax] started
[2014-05-05 19:58:38,554][INFO ][gateway ] [Rax] recovered
[0] indices into cluster_state
[2014-05-05 19:58:42,667][INFO ][node ] [Rax] stopping
...
[2014-05-05 19:58:42,683][INFO ][node ] [Rax] stopped
[2014-05-05 19:58:42,683][INFO ][node ] [Rax] closing
...
[2014-05-05 19:58:42,714][INFO ][node ] [Rax] closed
[2014-05-05 19:59:27,032][INFO ][node ] [N'Gabthoth]
version[0.90.10], pid[1142], build[0a5781f/2014-01-10T10:18:37Z]
[2014-05-05 19:59:27,053][INFO ][node ] [N'Gabthoth]
initializing ...
[2014-05-05 19:59:27,626][INFO ][plugins ] [N'Gabthoth]
loaded [river-fs], sites []
[2014-05-05 19:59:30,355][INFO ][node ] [N'Gabthoth]
initialized
[2014-05-05 19:59:30,355][INFO ][node ] [N'Gabthoth]
starting ...
[2014-05-05 19:59:30,486][INFO ][transport ] [N'Gabthoth]
bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
192.168.1.65:9300]}
[2014-05-05 19:59:33,658][INFO ][cluster.service ] [N'Gabthoth]
new_master [N'Gabthoth][SBeWzhj0Sp2VIDME6n9khA][inet[/192.168.1.65:9300]],
reason: zen-disco-join (elected_as_master)
[2014-05-05 19:59:33,789][INFO ][discovery ] [N'Gabthoth]
elasticsearch/SBeWzhj0Sp2VIDME6n9khA
[2014-05-05 19:59:33,799][INFO ][http ] [N'Gabthoth]
bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/
192.168.1.65:9200]}
[2014-05-05 19:59:33,800][INFO ][node ] [N'Gabthoth]
started
[2014-05-05 19:59:33,908][INFO ][gateway ] [N'Gabthoth]
recovered [0] indices into cluster_state
[2014-05-05 19:59:57,578][INFO ][cluster.metadata ] [N'Gabthoth]
[mydocs] creating index, cause [api], shards [5]/[1], mappings []
[2014-05-05 20:00:10,153][INFO ][cluster.metadata ] [N'Gabthoth]
[_river] creating index, cause [auto(index api)], shards [1]/[1], mappings
[]
[2014-05-05 20:00:10,311][INFO ][cluster.metadata ] [N'Gabthoth]
[_river] update_mapping [mydocs] (dynamic)
[2014-05-05 20:00:10,325][INFO ][river.routing ] [N'Gabthoth] no
river _meta document found, retrying in 1000 ms
[2014-05-05 20:00:11,343][WARN
][fr.pilato.elasticsearch.river.fs.river.FsRiver] [N'Gabthoth] [fs][mydocs]
`fs.name` attribute is deprecated. Don't use it anymore.
[2014-05-05 20:00:11,344][INFO
][fr.pilato.elasticsearch.river.fs.river.FsRiver] [N'Gabthoth] [fs][mydocs]
Starting fs river scanning
[2014-05-05 20:00:11,389][INFO ][cluster.metadata ] [N'Gabthoth]
[index] creating index, cause [api], shards [5]/[1], mappings []
[2014-05-05 20:00:11,492][WARN ][river ] [N'Gabthoth]
failed to get _meta from [fs]/[mydocs]
java.lang.NoSuchMethodError:
org.elasticsearch.action.admin.cluster.state.ClusterStateRequestBuilder.setIndices([Ljava/lang/String;)Lorg/elasticsearch/action/admin/cluster/state/ClusterStateRequestBuilder;
at
fr.pilato.elasticsearch.river.fs.river.FsRiver.isMappingExist(FsRiver.java:303)
at
fr.pilato.elasticsearch.river.fs.river.FsRiver.pushMapping(FsRiver.java:318)
at
fr.pilato.elasticsearch.river.fs.river.FsRiver.start(FsRiver.java:231)
at
org.elasticsearch.river.RiversService.createRiver(RiversService.java:147)
at
org.elasticsearch.river.RiversService$ApplyRivers$2.onResponse(RiversService.java:272)
at
org.elasticsearch.river.RiversService$ApplyRivers$2.onResponse(RiversService.java:266)
at
org.elasticsearch.action.support.TransportAction$ThreadedActionListener$1.run(TransportAction.java:93)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-05-05 19:13 GMT+02:00 David Pilato <[email protected]>:
> fsriver 1.0.0 does not work with elasticsearch 0.90.7
>
> Look here for older versions:
> https://github.com/dadoonet/fsriver/tree/es-0.90
>
>
>
> --
> *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
> @dadoonet <https://twitter.com/dadoonet> |
> @elasticsearchfr<https://twitter.com/elasticsearchfr>
>
>
> Le 5 mai 2014 à 18:41:03, John Ottosson ([email protected]) a
> écrit:
>
> the url is not the issue, i renamed it in this text.
>
>
> 2014-05-05 18:39 GMT+02:00 John Ottosson <[email protected]>:
>
>> more..
>>
>> [2014-05-05 18:16:35,335][INFO ][node ] [Valerie Cooper] version[1.1.1],
>> pid[1258], build[f1585f0/2014-04-16T14:27:12Z]
>> [2014-05-05 18:16:35,336][INFO ][node ] [Valerie Cooper] initializing ...
>> [2014-05-05 18:16:35,897][INFO ][plugins ] [Valerie Cooper] loaded
>> [river-fs], sites []
>> [2014-05-05 18:16:39,112][INFO ][node ] [Valerie Cooper] initialized
>> [2014-05-05 18:16:39,112][INFO ][node ] [Valerie Cooper] starting ...
>> [2014-05-05 18:16:39,286][INFO ][transport ] [Valerie Cooper]
>> bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
>> 192.168.1.65:9300]}
>> [2014-05-05 18:16:42,415][INFO ][cluster.service ] [Valerie Cooper]
>> new_master [Valerie
>> Cooper][fxeE0FUbR-WR1_0-vx_RZw][duster][inet[/192.168.1.65:9300]],
>> reason:
>> zen-disco-join (elected_as_master)
>> [2014-05-05 18:16:42,450][INFO ][discovery ] [Valerie Cooper]
>> elasticsearch/fxeE0FUbR-WR1_0-vx_RZw
>> [2014-05-05 18:16:42,470][INFO ][http ] [Valerie Cooper] bound_address
>> {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.1.65:9200]}
>> [2014-05-05 18:16:42,541][INFO ][gateway ] [Valerie Cooper] recovered [0]
>> indices into cluster_state
>> [2014-05-05 18:16:42,542][INFO ][node ] [Valerie Cooper] started
>> [2014-05-05 18:21:40,646][INFO ][cluster.metadata ] [Valerie Cooper]
>> [_river] creating index, cause [auto(index api)], shards [1]/[1], mappings
>> []
>> [2014-05-05 18:21:40,959][INFO ][cluster.metadata ] [Valerie Cooper]
>> [_river] update_mapping [mydocs] (dynamic)
>> [2014-05-05 18:21:41,983][WARN ][river.routing ] [Valerie Cooper] no
>> river type provided for [_river], ignoring...
>> [2014-05-05 18:23:11,827][DEBUG][action.index ] [Valerie Cooper]
>> [_river][0], node[fxeE0FUbR-WR1_0-vx_RZw], [P], s[STARTED]: Failed to
>> execute [index {[_river][
>> mydocs][_meta], source[{
>> "type": "fs",
>> "fs": {
>> "name": "mnt",
>> "url": "/mnt/filer/"
>>
>>
>>
>> },
>> "index": {
>> "index": "index",
>> "type": "doc"
>> }
>> ]}]
>> org.elasticsearch.index.mapper.MapperParsingException: failed to parse
>> at
>> org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:540)
>> at
>> org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:462)
>> at
>> org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:384)
>> at
>> org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:203)
>> at
>> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:556)
>> at
>> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:426)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.elasticsearch.common.jackson.core.JsonParseException:
>> Unexpected end-of-input: expected close marker for OBJECT (from [Source:
>> [B@35580b3a; line: 1, column: 0])
>> at [Source: [B@35580b3a; line: 13, column: 145]
>> at
>> org.elasticsearch.common.jackson.core.JsonParser._constructError(JsonParser.java:1524)
>> at
>> org.elasticsearch.common.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:557)
>> at
>> org.elasticsearch.common.jackson.core.base.ParserMinimalBase._reportInvalidEOF(ParserMinimalBase.java:487)
>> at
>> org.elasticsearch.common.jackson.core.base.ParserBase._handleEOF(ParserBase.java:487)
>> at
>> org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._skipWSOrEnd(UTF8StreamJsonParser.java:2588)
>> at
>> org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:649)
>> at
>> org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:50)
>> at
>> org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:471)
>> at
>> org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:515)
>> ... 8 more
>>
>>
>>
>> 2014-05-05 18:29 GMT+02:00 John Ottosson <[email protected]>:
>>
>> Hi, I tried reinstalling elasticsearch and fsriver, when trying to
>>> connect to localhost it failed, perhaps some setting i missed. it
>>> said curl: (7) Failed to connect to localhost port 9200: Connection
>>> refused, i used the ip adress instead and got this in the
>>> var/log/elasticsearch/elasticsearch.log
>>>
>>> [2014-05-05 18:16:35,335][INFO ][node ] [Valerie Cooper] version[1.1.1],
>>> pid[1258], build[f1585f0/2014-04-16T14:27:12Z]
>>> [2014-05-05 18:16:35,336][INFO ][node ] [Valerie Cooper] initializing ...
>>> [2014-05-05 18:16:35,897][INFO ][plugins ] [Valerie Cooper] loaded
>>> [river-fs], sites []
>>> [2014-05-05 18:16:39,112][INFO ][node ] [Valerie Cooper] initialized
>>> [2014-05-05 18:16:39,112][INFO ][node ] [Valerie Cooper] starting ...
>>> [2014-05-05 18:16:39,286][INFO ][transport ] [Valerie Cooper]
>>> bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
>>> 192.168.1.65:9300]}
>>> [2014-05-05 18:16:42,415][INFO ][cluster.service ] [Valerie Cooper]
>>> new_master [Valerie
>>> Cooper][fxeE0FUbR-WR1_0-vx_RZw][duster][inet[/192.168.1.65:9300]],
>>> reason: zen-disco-join (elected_as_master)
>>> [2014-05-05 18:16:42,450][INFO ][discovery ] [Valerie Cooper]
>>> elasticsearch/fxeE0FUbR-WR1_0-vx_RZw
>>> [2014-05-05 18:16:42,470][INFO ][http ] [Valerie Cooper] bound_address
>>> {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.1.65:9200
>>> ]}
>>> [2014-05-05 18:16:42,541][INFO ][gateway ] [Valerie Cooper] recovered
>>> [0] indices into cluster_state
>>> [2014-05-05 18:16:42,542][INFO ][node ] [Valerie Cooper] started
>>> [2014-05-05 18:21:40,646][INFO ][cluster.metadata ] [Valerie Cooper]
>>> [_river] creating index, cause [auto(index api)], shards [1]/[1], mappings
>>> []
>>> [2014-05-05 18:21:40,959][INFO ][cluster.metadata ] [Valerie Cooper]
>>> [_river] update_mapping [mydocs] (dynamic)
>>> [2014-05-05 18:21:41,983][WARN ][river.routing ] [Valerie Cooper] no
>>> river type provided for [_river], ignoring...
>>> [2014-05-05 18:23:11,827][DEBUG][action.index ] [Valerie Cooper]
>>> [_river][0], node[fxeE0FUbR-WR1_0-vx_RZw], [P], s[STARTED]: Failed to
>>> execute [index {[_river][mydocs][_meta], source[{
>>>
>>> "type": "fs",
>>> "fs": {
>>> "name": "mnt",
>>> "url": "/mnt/files/"
>>>
>>>
>>> },
>>> "index": {
>>> "index": "index",
>>> "type": "doc"
>>> }
>>> ]}]
>>> org.elasticsearch.index.mapper.MapperParsingException: failed to parse
>>> at
>>> org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:540)
>>> at
>>> org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:462)
>>> at
>>> org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:384)
>>> at
>>> org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:203)
>>> at
>>> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:556)
>>> at
>>> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:426)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>> Caused by: org.elasticsearch.common.jackson.core.JsonParseException:
>>> Unexpected end-of-input: expected close marker for OBJECT (from [Source:
>>> [B@35580b3a; line: 1, column: 0])
>>> at [Source: [B@35580b3a; line: 13, column: 145]
>>> at
>>> org.elasticsearch.common.jackson.core.JsonParser._constructError(JsonParser.java:1524)
>>> at
>>> org.elasticsearch.common.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:557)
>>> at
>>> org.elasticsearch.common.jackson.core.base.ParserMinimalBase._reportInvalidEOF(ParserMinimalBase.java:487)
>>> at
>>> org.elasticsearch.common.jackson.core.base.ParserBase._handleEOF(ParserBase.java:487)
>>> at
>>> org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._skipWSOrEnd(UTF8StreamJsonParser.java:2588)
>>> at
>>> org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:649)
>>> at
>>> org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:50)
>>> at
>>> org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:471)
>>> at
>>> org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:515)
>>> ... 8 more
>>>
>>>
>>>
>>>
>>> 2014-05-05 16:42 GMT+02:00 David Pilato <[email protected]>:
>>>
>>> Do you see anything in logs?
>>>>
>>>> --
>>>> David ;-)
>>>> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>>>>
>>>> Le 5 mai 2014 à 15:50, John Ottosson <[email protected]> a
>>>> écrit :
>>>>
>>>> Ive got some problems getting fsriver to index my files, ive got a
>>>> diskimage mounted on /mnt/ and it contains a folder files containing files.
>>>> I am running this on a single server for testing purposes.
>>>>
>>>> When I try to index the files in the folder with:
>>>>
>>>> curl -XPUT 'localhost:9200/_river/mydocs/_meta' -d '{
>>>> "type": "fs",
>>>> "fs": {
>>>> "name": "mnt",
>>>> "url": "/mnt/files/"
>>>>
>>>> },
>>>> "index": {
>>>> "index": "index",
>>>> "type": "doc"
>>>> }
>>>> }'
>>>>
>>>> the folder contains 1gig of data in various formats and the command
>>>> only produces:
>>>>
>>>> {"ok":true,"_index":"_river","_type":"mydocs","_id":"_meta","_version":17}
>>>>
>>>> i can see that it updates the version but it does not produce any index
>>>> to my knowledge.
>>>> GET /_river/mydocs/_search returns with a blank.
>>>>
>>>> curl -XGET 'http://localhost:9200/'
>>>> {
>>>> "ok" : true,
>>>> "status" : 200,
>>>> "name" : "Sigyn",
>>>> "version" : {
>>>> "number" : "0.90.7",
>>>> "build_hash" : "36897d07dadcb70886db7f149e645ed3d44eb5f2",
>>>> "build_timestamp" : "2013-11-13T12:06:54Z",
>>>> "build_snapshot" : false,
>>>> "lucene_version" : "4.5.1"
>>>> },
>>>> "tagline" : "You Know, for Search"
>>>>
>>>> fsriver version 1.0.0
>>>>
>>>>
>>>> Thankful for any help with this matter.
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "elasticsearch" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>>
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/elasticsearch/30b9b34f-c30a-4105-b632-dbd927713280%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/30b9b34f-c30a-4105-b632-dbd927713280%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>> --
>>>> You received this message because you are subscribed to a topic in the
>>>> Google Groups "elasticsearch" group.
>>>> To unsubscribe from this topic, visit
>>>> https://groups.google.com/d/topic/elasticsearch/lTh_NpkkO4Q/unsubscribe
>>>> .
>>>> To unsubscribe from this group and all its topics, send an email to
>>>> [email protected].
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/elasticsearch/1826A460-98E9-4905-8897-650C22B2D11D%40pilato.fr<https://groups.google.com/d/msgid/elasticsearch/1826A460-98E9-4905-8897-650C22B2D11D%40pilato.fr?utm_medium=email&utm_source=footer>.
>>>>
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAJBsOReskjidZYtwfy9AMWO5PW7ucKNEgQ5UTxjcpxzdeHbTpA%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CAJBsOReskjidZYtwfy9AMWO5PW7ucKNEgQ5UTxjcpxzdeHbTpA%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elasticsearch/lTh_NpkkO4Q/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/etPan.5367c6ba.7545e146.5c36%40MacBook-Air-de-David.local<https://groups.google.com/d/msgid/elasticsearch/etPan.5367c6ba.7545e146.5c36%40MacBook-Air-de-David.local?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAJBsORfJRh3rRD02h_%3DgBU0sEv9Yxjqt0OsVwT0RWoVyss1P7Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
