Hi. I'm quite new to elasticsearch, so far it's all been going great but I've run into a wall and after a few days of no progress I thought it was time to ask for help.
I'm trying to create a replacement search solution for a CMS system, one of the requirements is that it needs to index binary files. The mapping-attachments plugin appears to be just the thing, but I'm struggling to get it to work. I've tried this with ElasticSearch 1.3x and Mapper Attachements 2.3.2 and ElasticSearch 1.4x and Mapper Attachments 2.4.2 running under Windows. I have no errors in the log, the plugin appears to be loading correctly, so I assume I'm doing something wrong with my requests. I've simplified my requests down to the most basic level I can, and the issue still occurs. Testing has been done with the Postman extension in Chrome. But I've converted my posts to curl requests to help anyone who might want to try this on Linux. The Base64 file is a .txt file with some English text from the BBC News site. Create test index curl -XPUT 'http://localhost:9200/test/' Response { "acknowledged": true } Create mapping for person curl -XPUT 'http://localhost:9200/test/_mapping/person' -d '{ "person" : { "properties" : { "my_attachment" : { "type" : "attachment" } } } }' Response { "acknowledged": true } Get mapping for person curl -XGET 'http://localhost:9200/test/_mapping/person' Response { "test": { "mappings": { "person": { "properties": { "my_attachment": { "type": "attachment", "path": "full", "fields": { "my_attachment": { "type": "string" }, "author": { "type": "string" }, "title": { "type": "string" }, "name": { "type": "string" }, "date": { "type": "date", "format": "dateOptionalTime" }, "keywords": { "type": "string" }, "content_type": { "type": "string" }, "content_length": { "type": "integer" }, "language": { "type": "string" } } } } } } } } This looks good, I have meta data fields for the file in the mapping Create person id 1 curl -XPUT 'http://localhost:9200/test/person/1' -d '{ "my_attachment" : "Rm9ybWVyIGNlbGVicml0eSBwdWJsaWNpc3QgTWF4IENsaWZmb3JkIGhhcyBoYWQgYW4gYXBwZWFsIGFnYWluc3QgaGlzIGVpZ2h0LXllYXIgc2VudGVuY2UgZm9yIHNleCBvZmZlbmNlcyByZWplY3RlZCBieSB0aGUgQ291cnQgb2YgQXBwZWFsLg0KDQpUaGUgY291cnQgcnVsZWQgdGhlIHNlbnRlbmNlIGhhbmRlZCB0byBDbGlmZm9yZCBlYXJsaWVyIHRoaXMgeWVhciB3YXMganVzdGlmaWVkIGFuZCBjb3JyZWN0Lg0KDQpDbGlmZm9yZCB3YXMgY29udmljdGVkIGluIEFwcmlsIG9mIGVpZ2h0IGhpc3RvcmljYWwgaW5kZWNlbnQgYXNzYXVsdHMgb24gd29tZW4gYW5kIG9uIGdpcmxzIGFzIHlvdW5nIGFzIDE1Lg0KDQpIaXMgbGF3eWVyIGhhZCBhcmd1ZWQgdGhlIHNlbnRlbmNlIHdhcyAidW5mYWlyIiBhbmQgY2xhaW1lZCBDbGlmZm9yZCB3YXMgbm90IGEgdGhyZWF0IHRvIHdvbWVuLg==" }' Response { "_index": "test", "_type": "person", "_id": "1", "_version": 1, "created": true } Looks good, let's get that record back Get person id 1 curl -XGET 'http://localhost:9200/test/person/1' { "_index": "test", "_type": "person", "_id": "1", "_version": 1, "found": true, "_source": { "my_attachment": "Rm9ybWVyIGNlbGVicml0eSBwdWJsaWNpc3QgTWF4IENsaWZmb3JkIGhhcyBoYWQgYW4gYXBwZWFsIGFnYWluc3QgaGlzIGVpZ2h0LXllYXIgc2VudGVuY2UgZm9yIHNleCBvZmZlbmNlcyByZWplY3RlZCBieSB0aGUgQ291cnQgb2YgQXBwZWFsLg0KDQpUaGUgY291cnQgcnVsZWQgdGhlIHNlbnRlbmNlIGhhbmRlZCB0byBDbGlmZm9yZCBlYXJsaWVyIHRoaXMgeWVhciB3YXMganVzdGlmaWVkIGFuZCBjb3JyZWN0Lg0KDQpDbGlmZm9yZCB3YXMgY29udmljdGVkIGluIEFwcmlsIG9mIGVpZ2h0IGhpc3RvcmljYWwgaW5kZWNlbnQgYXNzYXVsdHMgb24gd29tZW4gYW5kIG9uIGdpcmxzIGFzIHlvdW5nIGFzIDE1Lg0KDQpIaXMgbGF3eWVyIGhhZCBhcmd1ZWQgdGhlIHNlbnRlbmNlIHdhcyAidW5mYWlyIiBhbmQgY2xhaW1lZCBDbGlmZm9yZCB3YXMgbm90IGEgdGhyZWF0IHRvIHdvbWVuLg==" } } Attachment has been added as a string, and there are no additional meta data fields Here's my system info got via curl -XGET 'http://localhost:9200/_nodes' { "cluster_name": "elasticsearch", "nodes": { "QWhhRNIOTUWX_1OxGSJOvA": { "name": "Franz Kafka", "transport_address": "inet[/192.168.76.148:9300]", "host": "WIN-23CNBGGKSSE", "ip": "192.168.76.148", "version": "1.4.0", "build": "bc94bd8", "http_address": "inet[/192.168.76.148:9200]", "settings": { "node": { "name": "Franz Kafka" }, "client": { "type": "node" }, "http": { "cors": { "enabled": "true", "allow-origin": "/https?:\\/\\/local.kibana(:[0-9]+)?/" } }, "name": "Franz Kafka", "path": { "data": "c:\\apps\\elasticsearch\\data", "work": "c:\\apps\\elasticsearch", "home": "c:\\apps\\elasticsearch", "conf": "c:\\apps\\elasticsearch\\config", "logs": "c:/apps/elasticsearch/logs" }, "cluster": { "name": "elasticsearch" }, "config": "c:\\apps\\elasticsearch\\config\\elasticsearch.yml", "plugin": { "mandatory": "mapper-attachments" } }, "os": { "refresh_interval_in_millis": 1000, "available_processors": 4, "cpu": { "vendor": "Intel", "model": "Xeon", "mhz": 2666, "total_cores": 4, "total_sockets": 1, "cores_per_socket": 4, "cache_size_in_bytes": -1 }, "mem": { "total_in_bytes": 8589402112 }, "swap": { "total_in_bytes": 17176915968 } }, "process": { "refresh_interval_in_millis": 1000, "id": 6048, "max_file_descriptors": -1, "mlockall": false }, "jvm": { "pid": 6048, "version": "1.7.0_71", "vm_name": "Java HotSpot(TM) 64-Bit Server VM", "vm_version": "24.71-b01", "vm_vendor": "Oracle Corporation", "start_time_in_millis": 1415361000462, "mem": { "heap_init_in_bytes": 268435456, "heap_max_in_bytes": 1038876672, "non_heap_init_in_bytes": 24313856, "non_heap_max_in_bytes": 136314880, "direct_max_in_bytes": 1038876672 }, "gc_collectors": [ "ParNew", "ConcurrentMarkSweep" ], "memory_pools": [ "Code Cache", "Par Eden Space", "Par Survivor Space", "CMS Old Gen", "CMS Perm Gen" ] }, "thread_pool": { "generic": { "type": "cached", "keep_alive": "30s", "queue_size": -1 }, "index": { "type": "fixed", "min": 4, "max": 4, "queue_size": "200" }, "bench": { "type": "scaling", "min": 1, "max": 2, "keep_alive": "5m", "queue_size": -1 }, "get": { "type": "fixed", "min": 4, "max": 4, "queue_size": "1k" }, "snapshot": { "type": "scaling", "min": 1, "max": 2, "keep_alive": "5m", "queue_size": -1 }, "merge": { "type": "scaling", "min": 1, "max": 2, "keep_alive": "5m", "queue_size": -1 }, "suggest": { "type": "fixed", "min": 4, "max": 4, "queue_size": "1k" }, "bulk": { "type": "fixed", "min": 4, "max": 4, "queue_size": "50" }, "optimize": { "type": "fixed", "min": 1, "max": 1, "queue_size": -1 }, "warmer": { "type": "scaling", "min": 1, "max": 2, "keep_alive": "5m", "queue_size": -1 }, "flush": { "type": "scaling", "min": 1, "max": 2, "keep_alive": "5m", "queue_size": -1 }, "search": { "type": "fixed", "min": 12, "max": 12, "queue_size": "1k" }, "listener": { "type": "fixed", "min": 2, "max": 2, "queue_size": -1 }, "percolate": { "type": "fixed", "min": 4, "max": 4, "queue_size": "1k" }, "management": { "type": "scaling", "min": 1, "max": 5, "keep_alive": "5m", "queue_size": -1 }, "refresh": { "type": "scaling", "min": 1, "max": 2, "keep_alive": "5m", "queue_size": -1 } }, "network": { "refresh_interval_in_millis": 5000, "primary_interface": { "address": "192.168.76.148", "name": "eth6", "mac_address": "00:0C:29:80:70:CA" } }, "transport": { "bound_address": "inet[/0:0:0:0:0:0:0:0:9300]", "publish_address": "inet[/192.168.76.148:9300]" }, "http": { "bound_address": "inet[/0:0:0:0:0:0:0:0:9200]", "publish_address": "inet[/192.168.76.148:9200]", "max_content_length_in_bytes": 104857600 }, "plugins": [ { "name": "mapper-attachments", "version": "2.4.1", "description": "Adds the attachment type allowing to parse difference attachment formats", "jvm": true, "site": false }, { "name": "kopf", "version": "1.3.7", "description": "kopf - simple web administration tool for ElasticSearch", "url": "/_plugin/kopf/", "jvm": false, "site": true } ] } } } And my elasticsearch log from startup. [2014-11-07 13:23:59,256][INFO ][node ] [Franz Kafka] version[1.4.0], pid[6928], build[bc94bd8/2014-11-05T14:26:12Z] [2014-11-07 13:23:59,256][INFO ][node ] [Franz Kafka] initializing ... [2014-11-07 13:23:59,319][INFO ][plugins ] [Franz Kafka] loaded [mapper-attachments], sites [kopf] [2014-11-07 13:24:03,503][INFO ][node ] [Franz Kafka] initialized [2014-11-07 13:24:03,503][INFO ][node ] [Franz Kafka] starting ... [2014-11-07 13:24:03,643][INFO ][transport ] [Franz Kafka] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.76.148:9300]} [2014-11-07 13:24:03,784][INFO ][discovery ] [Franz Kafka] elasticsearch/A4ONWcVyRIiJxaVw0Mm0uA [2014-11-07 13:24:07,566][INFO ][cluster.service ] [Franz Kafka] new_master [Franz Kafka][A4ONWcVyRIiJxaVw0Mm0uA][WIN-23CNBGGKSSE][inet[/192.168.76.148:9300]], reason: zen-disco-join (elected_as_master) [2014-11-07 13:24:07,705][INFO ][http ] [Franz Kafka] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.76.148:9200]} [2014-11-07 13:24:07,705][INFO ][node ] [Franz Kafka] started [2014-11-07 13:24:08,416][INFO ][gateway ] [Franz Kafka] recovered [1] indices into cluster_state I've also set Mapper Attachment as a mandatory plugin in the config, so it's definitely loading as the node starts up ok. I'd really appreciate some help on this. I'm sure I'm making some newbie mistake with the mapping or something, but the documentation isn't helping me here. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/95bcd0b7-844a-40b5-93cf-dce2ea4bc284%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
