Re: Diagnosing a slow query

2014-08-01 Thread ryan chou
i would rather suggest you separate the nodes roles. you could divide them into 3 node roles. 1.Index node : for index data 2.load balance : for query , you could maximize load balance node memory for avoiding the GC consume too much memory. 3.data node : just for store the data. and optimize

Re: Unable to install plugin

2014-08-01 Thread Mark Walkom
Do you have a /usr/share/elasticsearch/bin/plugin file? If so what happens if you run it with the -h flag? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 1 August 2014 15:57, sa...@systeminsights.com wrote: Hey,

Re: index size impact on search performance?

2014-08-01 Thread Mark Walkom
You can create one every hour if you want, daily is good. You may have to deal with that in the code however it makes management a lot easier. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 1 August 2014 14:35,

hadoop as storage

2014-08-01 Thread navdeep agarwal
i want to know can we store elasticsearch indexes in hdfs ,and elasticsearch can use these indexes to cater searches . as in solr we have have hdfs directory factory ,do we have similar in elastic search . -- You received this message because you are subscribed to the Google Groups

Re: Help needed understanding analyzer behavior

2014-08-01 Thread Sina Tamanna
When I develope custom analyzers I use Analyze API to test it and understand the tokens that will be indexed. Take a look at http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-analyze.html -- You received this message because you are subscribed to the Google

Pull Requests mounting?

2014-08-01 Thread James Green
Hi, A cursory glance at the GitHub pages shows a build-up of pull requests: ElasticSearch: 135 Kibana: 64 Is there a movement to get these merged or otherwise cleaned up? We are waiting on at least one of these but lack hope in the face of the volumes pending. Thanks, James -- You received

Re: Pull Requests mounting?

2014-08-01 Thread Clinton Gormley
Hi James A cursory glance at the GitHub pages shows a build-up of pull requests: ElasticSearch: 135 Kibana: 64 Is there a movement to get these merged or otherwise cleaned up? We are waiting on at least one of these but lack hope in the face of the volumes pending. We have a

Re: Cosine Similarity ElasticSearch

2014-08-01 Thread RickDast
There is someone that can help us? Thank you very much! -- View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Cosine-Similarity-ElasticSearch-tp4060620p4061039.html Sent from the ElasticSearch Users mailing list archive at Nabble.com. -- You received this message

Re: Pull Requests mounting?

2014-08-01 Thread James Green
Thanks Clint, my alarm was based on the lack of pulse activity over at Kibana. On 1 August 2014 09:37, Clinton Gormley clinton.gorm...@elasticsearch.com wrote: Hi James A cursory glance at the GitHub pages shows a build-up of pull requests: ElasticSearch: 135 Kibana: 64 Is there a

Re: Cosine Similarity ElasticSearch

2014-08-01 Thread vineeth mohan
Hello Rick , MLT query translates the source document into a normal boolean terms query. This query is fired against the index and a normal TF IDF is used to find the score. AFAIK this is the working of MLT. Thanks Vineeth On Fri, Aug 1, 2014 at 2:11 PM, RickDast chicco...@gmail.com

Help with creating mapping for my products

2014-08-01 Thread Radu Gurita
I have this product JSON document that I generate from my database. { product: { name: My product, description : My product description, variations: [ { sku: BLARED10, price: 10.15, attributes : {

Re: Remote access through SSH

2014-08-01 Thread Pawan Sharma
Why dont you try ssh -i ~/.ssh/id_rsa xx.xxx.xxx.xxx 'curl -XGET localhost:9200/index/_mapping/type' cheers On Fri, Aug 1, 2014 at 3:43 AM, Patrick Proniewski elasticsea...@patpro.net wrote: Hi, Looks like your problem is not related to ES. It's a networking problem. You ES server

ES-Hadoop data query liveness near real-time quantified?

2014-08-01 Thread Pierre WP
I have a question about what near real-time means exactly, in a quantified way, when described this way on the ES-hadoop home page: We are happy to report that es-hadoop is being used in multiple data-intensive environments; in a recent example, a large financial institute that stores all of

Re: ES-Hadoop data query liveness near real-time quantified?

2014-08-01 Thread Costin Leau
Using the RTC definitions, ES, Hadoop, the JVM and the popular OS themselves are soft/near real-time systems - so if you are coming from a hard/firm RT system, you can safely assume that everything (and again not just ES) is soft. As a tangent, very few systems are hard RT (ES is neither a nuclear

Re: index size impact on search performance?

2014-08-01 Thread David Pilato
But did you try what I wrote? You still see a lot of IOs on the second call? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 1 août 2014 à 12:30, Wang Yong cnwangy...@gmail.com a écrit : I just found this great blog about range filter:

Re: Logging every query

2014-08-01 Thread Alejandro de la Viña
Any ideas? I've tried everything and still nothing. On Wednesday, July 30, 2014 2:29:55 PM UTC-3, Alejandro de la Viña wrote: 've got an enviroment set on Dev that should keep a log with every query ran, but it's not writing anything. I'm using the slow-log feature for it... These are my

Disabling key_as_string

2014-08-01 Thread James Griffin
Hi, We're performing some aggregations with size=0 on a lot of data, and I'm looking to optimise the transfer over the wire. Our responses look like this: buckets: [ - { - key: 15, - key_as_string: 15, - doc_count: 8183 }, - { - key: 100, -

Re: Disabling key_as_string

2014-08-01 Thread Colin Goodheart-Smithe
Hi James, This issue was brought up recently in the github issues list[1] and has been fixed[2]. It should be available in the version 1.3.0. [1] https://github.com/elasticsearch/elasticsearch/issues/6655 [2] https://github.com/elasticsearch/elasticsearch/pull/6830 Regards, Colin On

Re: Cosine Similarity ElasticSearch

2014-08-01 Thread Alex S.V.
Hi, I found some native script codings from Igor Motov here: https://github.com/imotov/elasticsearch-native-script-example/blob/master/src/main/java/org/elasticsearch/examples/nativescript/script/CosineSimilarityScoreScript.java and now playing with it Alex On Friday, August 1, 2014

setup multiple cluster on different machines

2014-08-01 Thread ali koyuncu
I setup two clusters on same machine succesfully. Now I should configure cluster on different machine and I don't know how it makes. I didn't find guide for this. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group

boosting query howto?

2014-08-01 Thread Bernd Fehling
I tried the following query { fields: [ score, title, price ], boosting: { positive: { term: { text: einzelhandel } }, negative: { term: { price: 100 } }, negative_boost: 0.0025, boost: 200 } } What I'm trying

Re: Disabling key_as_string

2014-08-01 Thread James Griffin
Hi Colin, Thanks for getting back to me so quick. I'm running my query against a 1.3.1 cluster and still seeing key_as_string, even though no format parameter is in the query. My query is below: { query: { filtered: { query: { match_all: {} } } }, aggs: {

Re: hadoop as storage

2014-08-01 Thread Adrien Grand
Hi, We don't support storing indices on HDFS. However, we have a plugin that integrates Elasticsearch with Hadoop and allows to get data back and forth between Hadoop and Elasticsearch easily: http://www.elasticsearch.org/overview/hadoop/ For the record, indices stored in HDFS would be

Re: Disabling key_as_string

2014-08-01 Thread James Griffin
Hi, I dug into that PR and it seems it was solved for histograms but not for terms aggregations. I presume this is a new bug, then? Are there other aggregation types this should be solved for? Thanks, James On Friday, August 1, 2014 2:04:52 PM UTC+1, James Griffin wrote: Hi Colin,

update return 500 and npe

2014-08-01 Thread fancyerii
I am using 0.90.13. $ curl -XGET http://localhost:9200/blog/article/1?pretty { _index : blog, _type : article, _id : 1, _version : 3, exists : true, _source : {title:New version of Elasticsearch released!,content:new content,tags:[announce,elasticsearch,release],counter:1} }

Re: boosting query howto?

2014-08-01 Thread joergpra...@gmail.com
Have you tried boosting boolean query clauses? http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_boosting_query_clauses.html Jörg On Fri, Aug 1, 2014 at 2:39 PM, Bernd Fehling bernd.fehl...@gmail.com wrote: I tried the following query { fields: [ score, title,

Re: Recommendations needed for large ELK system design

2014-08-01 Thread Alex
Ok thank you Mark, you've been extremely helpful and we now have a better idea about what we're doing! -Alex On Thursday, 31 July 2014 23:57:26 UTC+1, Mark Walkom wrote: 1 - Curator FTW. 2 - Masters handle cluster state, shard allocation and a whole bunch of other stuff around managing the

Re: Constant High (~99%) CPU on 1 of 5 Nodes in Cluster

2014-08-01 Thread Greg Murnane
From the Marvel image, it looks like the heap utilization isn't dropping periodically as it does on the other nodes. Can you verify that GC is behaving nicely while this occurs? -- The information transmitted in this email is intended only for the person(s) or entity to which it is addressed

Re: Disabling key_as_string

2014-08-01 Thread Colin Goodheart-Smithe
Sorry, I assumed you were using the histogram aggregation. It was this aggregation that the fixed was applied to. I have opened issue #7125 https://github.com/elasticsearch/elasticsearch/issues/7125 to resolve this in the terms aggregation. On Friday, 1 August 2014 14:04:52 UTC+1, James

Re: Disabling key_as_string

2014-08-01 Thread James Griffin
Cracking, thanks Colin On Friday, August 1, 2014 3:28:45 PM UTC+1, Colin Goodheart-Smithe wrote: Sorry, I assumed you were using the histogram aggregation. It was this aggregation that the fixed was applied to. I have opened issue #7125

Re: Pull Requests mounting?

2014-08-01 Thread Rashid Khan
Hi James, We're keeping an eye on the progress of a few high priority pull requests that we're really excited about. We've also been working on a number of experimental new features that we'll be pushing out soon. At that point we'll be cleaning up the outstanding issues. Thanks On Friday,

Need: SAP BPC BW Techno Functional- Plymouth, MI.

2014-08-01 Thread Suresh Dabla
Hi There ! Hope you are doing great. Below is the position of SAP BPC BW Techno Functional-Plymouth, MI. Kindly go through the job description and let me know if you have any available consultant. *Job Title* : SAP BPC BW Techno Functional *Relevant Experience (Yrs) * : At least 1

Elasticsearch throwing exception while starting on Redhat

2014-08-01 Thread mailtov16
Hi, I am trying to install elasticsearch on Redhat. I have downloaded and installed latest elasticsearch from the website. While starting elasticsearch , I am getting following exception: [root@common ~]# /usr/share/elasticsearch/bin/elasticsearch Failed to configure logging...

enforcing types

2014-08-01 Thread eunever32
Hi, This is something I have discovered is happening... if I have a long type I can still store numbers such as 2, 4 and 1.2 Then when I retrieve the type is not long but string and float;( In other words in Java when I examine my values I can have 2 (type long) , 4 (type string) , 1.2 (type

Can't create geo_shape multi field

2014-08-01 Thread Owain Braddick
Hi, I am trying to create a mapping with a multi field where the default field type is geo_shape and it doesn't seem to work. When I retrieve the mapping it just appears as a normal geo_shape field. Is this a bug or is it deliberately not possible to do this? If it's not possible, should there

New Install on small box - OOME with no data loaded?!

2014-08-01 Thread Tony Su
Running in a VMware VM ES 1.3 from the official repo On an openSUSE box using the CentOS RPM as follows zypper ar http://packages.elasticsearch.org/elasticsearch/1.3/centos Elasticsearch_1.3_repository openJDK 1.7 *# systemctl status* elasticsearch.service - Starts and stops a single

Re: Can't create geo_shape multi field

2014-08-01 Thread David Pilato
IMHO it's a bug. I think you should open an issue. --  David Pilato | Technical Advocate | Elasticsearch.com @dadoonet | @elasticsearchfr Le 1 août 2014 à 17:29:35, Owain Braddick (owainbradd...@gmail.com) a écrit: Hi, I am trying to create a mapping with a multi field where the default

Re: enforcing types

2014-08-01 Thread David Pilato
Did you try to define a mapping for your field? --  David Pilato | Technical Advocate | Elasticsearch.com @dadoonet | @elasticsearchfr Le 1 août 2014 à 17:25:53, euneve...@gmail.com (euneve...@gmail.com) a écrit: Hi, This is something I have discovered is happening... if I have a long type I

Re: Issue (Bug?) with field_value_factor and Search Template

2014-08-01 Thread Germán Carrillo
It seems I was taking too literally the Search Template documentation [1] (or perhaps it lacks a small note on search paths). If I use what is stated in [1] I get the error I mentioned in the first post of this thread. Nonetheless, If I specify the full path in the POST query, ES returns what is

Re: Can't create geo_shape multi field

2014-08-01 Thread Owain Braddick
Thanks David. Raised as https://github.com/elasticsearch/elasticsearch/issues/7128. On Friday, August 1, 2014 4:45:01 PM UTC+1, David Pilato wrote: IMHO it's a bug. I think you should open an issue. -- *David Pilato* | *Technical Advocate* | *Elasticsearch.com* @dadoonet

Completion Suggestion Not working as documented.

2014-08-01 Thread Madhavan Ramachandran
Hi All, I am new to Elastic Search. I try to implement the completion suggester as documented in http://www.elasticsearch.org/blog/you-complete-me/; with sample data as below. Instead of hotel.. i choose region and locations.. us-en-us-chicago us-en-us-chicago, central loop us-en-us-chicago,

Re: Complete suggestion Issue

2014-08-01 Thread Madhavan Ramachandran
Does anyone provide any suggestion for the below observation? any kind of help is much appreciated. Regards Madhavan.TR On Thursday, July 31, 2014 2:45:18 PM UTC-5, Madhavan Ramachandran wrote: Hi I have created index for typeahead aka completion suggestion in ES. I feeded the below data

Re: EsRejectedExecutionException

2014-08-01 Thread Ivan Brusic
Your stack trace indicates that your search pool was exhausted, not your admin action thread pool. I don't think that restarting servers will cause searches to execute, but hopefully someone else would chime in and correct me. Do you have that many searches going on? -- Ivan On Fri, Aug 1,

High count of threadpools getting created

2014-08-01 Thread John D. Ament
Hi all Based on another thread I started, I'm embedding ES into my application. Starting today, and I'm sure there's something that got thrown into the mix causing this, we have started seeing high counts of thread pools getting created and the application never fully booting. I see this in

Re: New Install on small box - OOME with no data loaded?!

2014-08-01 Thread Tony Su
Resolved the problem. Although openJDK1.7 was installed, for some reason the system was configured with openJDK 1.5. Installed Oracle SE JDK 1.7 and no problems. Tony On Friday, August 1, 2014 8:40:02 AM UTC-7, Tony Su wrote: Running in a VMware VM ES 1.3 from the official repo On an

Re: Help with creating mapping for my products

2014-08-01 Thread Ivan Brusic
With inner dccuments, you cannot query two different attributes for the same inner object. Internally, Elasticsearch/Lucene will flatten all the inner properties into one array. You would need to switch to nested documents or even parent/child documents in order to query on a single deeper

Re: [Ann] JDBC River Plugin for ElasticSearch

2014-08-01 Thread Madhavan Ramachandran
Hi JP I setup a single node ES 1.2.1 on windows 2008 R2. Need to know the JDBC river version that support to import data from Microsoft SQL Server 2012. Regards Madhavan.TR On Monday, July 28, 2014 8:59:45 AM UTC-5, Santosh B wrote: Thanks a lot for sharing On Mon, Jul 28, 2014 at

Re: enforcing types

2014-08-01 Thread eunever32
okay so when I don't have a mapping and I do a GET _mapping it returns this: { test1: { mappings: { type2: { properties: { my_int: { type: long } } Is that not the same as having a mapping? Then to check the theory I

AS400 Developer -Framingham, MA

2014-08-01 Thread Suresh Dabla
Hi There, Hope you are doing great. Kindly let me know about your *consultant *is he available for below mentioned position. *AS400 Developer * *Work Location: Framingham, MA * *Duration: 6+ Months* *Mandatory Technical / Functional Skills * •6+ years of experience in

Re: Pull Requests mounting?

2014-08-01 Thread Ivan Brusic
Once I hit 3 open pull requests, I stopped submitting them. :) I am down to only 1 open (from last year), but my confidence is still somewhat low. Most of my changes have been small, mainly because I am hesitant about dedicating more time for something that might be ignored. I need to simply

Re: Dealing with spam in this forum

2014-08-01 Thread Ivan Brusic
I just deleted a couple of non-related job postings and banned the poster. Going forward, is there a consensus among the community about whether or not job postings should be allowed? I do not mind postings that come directly from companies, especially those whose existing developers are already

Re: Dealing with spam in this forum

2014-08-01 Thread Lukáš Vlček
Ivan, I would not care much if the frequency is low but I think Elasticsearch Linkedin group ( https://www.linkedin.com/groups?home=gid=3393294trk=anet_ug_hm) could be more logical target for job postings. Lukas On Fri, Aug 1, 2014 at 8:54 PM, Ivan Brusic i...@brusic.com wrote: I just

Curl -xget works, but fail when use RESTful client in java

2014-08-01 Thread Chia-Eng Chang
I tried to send http/get to my elasticsearch server.if I query: curl 'http://localhost:9200/index/_search?scroll=1msize=50pretty' -d '{query : {match_all : {}}}' it works perfect. But when I tried to use jersy to build my client, I did the follwoing: public class

Re: enforcing types

2014-08-01 Thread eunever32
The problem happens with java and python api when the typechecking throws exceptions For example when we use java to read the values mentioned : test2/lll/1 my_int = 1 class java.lang.String test2/lll/2 my_int = 2 class java.lang.Integer test2/lll/3 my_int = 1.2 class java.lang.Double *So you

Re: elasticsearch NoShardAvailableActionException

2014-08-01 Thread Jeff Dupont
I'm getting the same issue using Node.js. Looking for any solutions at this point as a restart of the server never seems to work. On Friday, July 6, 2012 4:09:58 PM UTC-7, Yuhan Zhang wrote: Hi all, looks like I am still able to retrieve data from localhost:9200 through curl, but when I

Re: Curl -xget works, but fail when use RESTful client in java

2014-08-01 Thread Chia-Eng Chang
Got the answer! It turns out that just add an between the url and encoded json body. String string1={\query\ : {\match_all\ : {}}}; WebResource webResource = client .resource(http://localhost:9200/obd2/_search?scroll=1msize=50+ URLEncoder.encode(string1)); then the http/get was successfully

raw field in top N query not working

2014-08-01 Thread Eitan Vesely
Hi all, i have a field named msg in my index that holds an error message string. when trying to count the top N errors i set the field to msg.raw yet i still see that the terms are all the words comprising the strings rather than the whole string as i would expect. (i did not set this field

Re: Dealing with spam in this forum

2014-08-01 Thread Mark Walkom
I agree with Ivan and Lukas, though recruiters with ES related jobs should be ok as long as it's not a generic BCC like the ones Ivan deleted. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 2 August 2014 05:05,

Re: Elasticsearch throwing exception while starting on Redhat

2014-08-01 Thread Mark Walkom
You should really use service to handle starting it - (sudo) service elasticsearch start This will include the correct configurations, /etc/elasticsearch/elasticsearch.yml and /etc/default/elasticsearch. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email:

Filter on array entries?

2014-08-01 Thread Ben Woosley
Hey Guys, I have some articles that are associated with groups, and I want to select all articles that have a certain group id associated, but elasticsearch isn't returning the results in a query. I've tried a number of different approaches and I hope someone here can tell me what I'm doing

Re: update return 500 and npe

2014-08-01 Thread fancyerii
I found the problem, it should be json object curl -XPOST http://localhost:9200/blog/article/1/_update -d '{script:ctx._source.counter += 2}' On Friday, August 1, 2014 9:40:44 PM UTC+8, fanc...@gmail.com wrote: I am using 0.90.13. $ curl -XGET http://localhost:9200/blog/article/1?pretty

No route to self

2014-08-01 Thread Matt Hughes
Originally I was getting a bunch of No Route to Host errors and tracked it down to being out of file handles. I have fixed the file handle, problem, but I still keep getting No route to host errors; the odd thing is, the error says it can't connect to itself: These logs are *from*

Re: No route to self

2014-08-01 Thread Mark Walkom
Can you put your config into a gist/pastebin? ES isn't binding to loopback is it? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 2 August 2014 11:51, Matt Hughes hughes.m...@gmail.com wrote: Originally I was

ES reports 4 nodes but only 3 severs. One node is used twice

2014-08-01 Thread David Montgomery
Hi, ES is showing 4 nodes but I only have 3. How do I resolve? One address is used twice 172.31.6.204 { ok : true, cluster_name : elasticsearch, nodes : { Fi5sM17uTku9LJYYDtKglQ : { name : Beautiful Dreamer, transport_address : inet[/172.31.6.204:9301], hostname :

Re: ES reports 4 nodes but only 3 severs. One node is used twice

2014-08-01 Thread Mark Walkom
Are you using logstash perhaps? If not then check the number of ES processes on your .204 node as there could be two instances running. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 2 August 2014 12:58, David

Re: No route to self

2014-08-01 Thread Matt Hughes
No, not loopback. I should caveat that this is running in a docker container. The only values I specify in my config are: network.publish_host: 10.52.207.36 discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: 10.52.207.32,10.52.207.33,10.52.207.36 [2014-08-02

Re: No route to self

2014-08-01 Thread Matt Hughes
Just a bit more info to show the port is bound on the machine. [hostos ~]$ ss -aen6 State Recv-Q Send-Q Local Address:PortPeer Address:Port LISTEN 0 128 :::111 :::* ino:13048