Re: Hit/Token Properties Advanced Scoring

2014-07-23 Thread Ivan Brusic
1. You can retrieve the term position, offset and payload using function score scripts: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-advanced-scripting.html 2: There are a couple of proposed solutions that would store the data in another index that is joined with

Re: Could not initialize class org.elasticsearch.index.codec.postingsformat.PostingFormats when packaging everything into single jar

2014-07-23 Thread Ramesh Kantam
Hi ia m ramesh... I'm getting the following error message while creating river for couchdb.. {error:IndexCreationException[[_river] failed to create index]; nested: NoClassDefFoundError[Could not initialize class org.elasticsearch.index.codec.postingsformat.PostingFormats]; ,status:500} Can

how can I run query with aggs field in kibana3

2014-07-23 Thread Moshe Beeri
I am trying to run this kind of query {query : {query_string : {query : project:240026 AND timestamp:[0 TO *]}}} -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to

Node Client with bulk request indefinitely blocked thread when ClusterBlockException is being thrown

2014-07-23 Thread Cosmin-Radu Vasii
I am using the dataless NodeClient to connect to my cluster (version is 1.1.1). Everything is working ok, except when failures occur. The scenario is the following: -I have an application java based which connects to ES Cluster (application is started and the cluster is up and running) -I

Node Client settings

2014-07-23 Thread Cosmin-Radu Vasii
Which setting are important for the node client? For my data nodes I disabled swapping, increased the memory and the file descriptors number and I understand why I have to do all of this (Lucene, I/O ops etc). But on the node client, which is dataless, do I need to consider applying such

Re: Is ES capable of doing pagination?

2014-07-23 Thread PrasathRajan
can somebody suggest.. Is it possible to jump over Page Nos [forward/backward]while using Scan/Scroll for Pagination?.. Or any other effective way available for pagination?. -- View this message in context:

Re: JVM crash on 64 bit SPARC with Elasticsearch 1.2.2 due to unaligned memory access

2014-07-23 Thread David Roberts
Just wanted to say thanks for fixing this so quickly. I can see the code change is already in the 1.2 branch. On 22 July 2014 16:35, Adrien Grand adrien.gr...@elasticsearch.com wrote: Agreed that this is an issue! I opened https://github.com/elasticsearch/elasticsearch/issues/6962 On Tue,

kibana dashboard save failed

2014-07-23 Thread jin201407
Hello. I just downloaded kibana-3.1.0 and open sample dashboard in kibana. then I tried to save dashboard but there was save file error. I attached the error screenshot. Is there any one who had the same problem? Thanks in advance. -- You received this message because you are subscribed

one index has different _type and different _type have same field with different type will disturb?

2014-07-23 Thread xu piao
i have one index with different _type like http://localhost:9200/matrix/group and http://localhost:9200/matrix/user the two _type all have 'region' field but with different type. group : - region: { - type: string - store: true - analyzer: ik } } user: - region:

Re: kibana dashboard save failed

2014-07-23 Thread Mark Walkom
Can you kibana install talk to elasticsearch ok? Are you seeing other data via it? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 23 July 2014 19:52, jin201...@gmail.com wrote: Hello. I just downloaded

Re: Questions RE: ES Script Written in JavaScript

2014-07-23 Thread Roland Dunn
OK, answering a couple of my own questions: 1. Return value: _score e.g. _score = Math.sqrt(retscore); 2. Accessing nested: easy enough: _source.things[i] 3. Debugging. Dunno. On Tuesday, 22 July 2014 17:17:40 UTC+1, Roland Dunn wrote: Hi, Wonder if anyone could help. I've managed to

how would you design the store model in Elasticsearch for user behavior data

2014-07-23 Thread panfei
user behavior data like this(transformed to JSON): {uid:user001, action:click, context: {level:21,ip:222.222.222.222, val:87}} {uid:user002, action:click, context: {level:28,ip:222.222.222.221,val:96}} #1 {uid:user002, action:buy, context: {level:28,ip:222.222.222.221,val:abc}} #2 ... 1. here

Sort search results by Bayesian Average

2014-07-23 Thread Nick T
Hi, I have an interesting use case which I posted up on Stack Overflow. If anyone has the time to take a look and share their thoughts that would be great: http://stackoverflow.com/questions/24885143/elasticsearch-bayesian-average. Cheers -- You received this message because you are

[ANN] Elasticsearch Twitter River plugin 2.2.0 released

2014-07-23 Thread Elasticsearch Team
Heya, We are pleased to announce the release of the Elasticsearch Twitter River plugin, version 2.2.0. The Twitter River plugin allows index twitter stream using elasticsearch rivers feature. https://github.com/elasticsearch/elasticsearch-river-twitter/ Release Notes -

Re: [ERROR][bootstrap] {1.2.2}: Initialization Failed ... - NullPointerException[null]

2014-07-23 Thread vjbangis
I tried EC2 discovery with the following settings, still keeps elected_as_master with the other node. discovery.type: ec2 discovery.ec2.groups: security groups discovery.ec2.host_type: private_ip cloud.aws.region: ap-southeast-1 cloud.aws.access_key: [access key] cloud.aws.secret_key: [security

Re: [ANN] Elasticsearch Twitter River plugin 2.2.0 released

2014-07-23 Thread James Green
I was told a week or two ago on IRC that rivers were deprecated in favour of external data sources like Logstash. Is this not correct? On 23 July 2014 13:00, Elasticsearch Team da...@pilato.fr wrote: Heya, We are pleased to announce the release of the *Elasticsearch Twitter River plugin*,

Re: [ANN] Elasticsearch Twitter River plugin 2.2.0 released

2014-07-23 Thread David Pilato
Rivers will be deprecated in favor of logstash inputs. Deprecated does not mean removed yet. So in the meantime we still try to keep up to date official plugins. But yes, you should prefer using if possible logstash twitter input (http://logstash.net/docs/1.4.2/inputs/twitter) --  David Pilato 

Even Shard Distribution?

2014-07-23 Thread michael
Hey guys, We've recently set up a 5 node ES cluster, serving our 6-shards / 1-replica index (we chose 6 back when we only had 3 nodes). We sometimes find a highly uneven distribution of shards across the nodes. For example, when we had 3 nodes, 4/6 of the index lived on 1 node, 2/6 lived on

Re: how would you design the store model in Elasticsearch for user behavior data

2014-07-23 Thread panfei
is there any way to convert data type without using the script mechinism ? 2014-07-23 17:22 GMT+08:00 panfei cnwe...@gmail.com: user behavior data like this(transformed to JSON): {uid:user001, action:click, context: {level:21,ip:222.222.222.222, val:87}} {uid:user002, action:click,

Re: how would you design the store model in Elasticsearch for user behavior data

2014-07-23 Thread panfei
{ size: 0, query: { filtered: { filter: { regexp: { who: [0-9]+ } } } }, aggs: { max_who: { max: { script: Double.parseDouble(_source.who) }

Re: Even Shard Distribution?

2014-07-23 Thread Nikolas Everett
For the 0/6 node are you sure you don't have some configuration preventing shards from allocating there? We use this: http://git.wikimedia.org/blob/operations%2Fpuppet.git/d2e2989bbafc7f7f730efacaa652a05bec3ef541/modules%2Felasticsearch%2Ftemplates%2Felasticsearch.yml.erb#L420 but its is designed

Is there a better way to achieve my goal than having multiple completion suggesters on a single index?

2014-07-23 Thread Gordon Rankin
I have an index of photos and need to return completion suggestions based on several of the fields: - Tags - Place - Country - Date The simplest way to do this of course would be to create one completion suggester and simply feed the various inputs into it when indexing.

Re: Even Shard Distribution?

2014-07-23 Thread michael
Thanks for that, Nik. I'm okay with evenly spreading all the indices, rather than just the one I'm having issues with. I'll give your config a try! Def no special configurations on that one. We didn't even realize there was such a thing as allocation configuration up until yesterday (after the

Re: Even Shard Distribution?

2014-07-23 Thread Nikolas Everett
On Wed, Jul 23, 2014 at 9:21 AM, mich...@modernmast.com wrote: Thanks for that, Nik. I'm okay with evenly spreading all the indices, rather than just the one I'm having issues with. I'll give your config a try! Def no special configurations on that one. We didn't even realize there was such

Sudden high OS Load, then ES VM disappears

2014-07-23 Thread michael
Hey all! I'm having some a serious problem with my ES cluster. Every now and then, when writing to the cluster, a machine (or two) will suddenly spike up on OS Load, writing will come to a screeching halt (5s for 1k docs, as opposed to ~100ms normally), and then shortly after, the VM that was

Re: Even Shard Distribution?

2014-07-23 Thread michael
Got it. To be honest, I was pretty sure of that, up until this AM, when that same OS Load spike happened again. But this time, the shards were allocated more evenly. So I'm not sure that's even the problem any more. I just posted a new post with more information about the load spike issue.

Re: Is there a better way to achieve my goal than having multiple completion suggesters on a single index?

2014-07-23 Thread Adrien Grand
Hi Gordon, Given your requirements, I think you are doing the right thing. There is no particular concern wrt querying multiple suggesters at the same time. On Wed, Jul 23, 2014 at 3:20 PM, Gordon Rankin gor...@ripplesoup.com wrote: I have an index of photos and need to return completion

Re: Is ES capable of doing pagination?

2014-07-23 Thread Adrien Grand
Hi Prasath, Scan and scroll can only move forward. If you want to have previous/next buttons, you would typically run the query once again with different values of `from` and `size`. On Tue, Jul 22, 2014 at 3:16 PM, PrasathRajan prasanth.sunr...@gmail.com wrote: can somebody suggest.. Is it

Re: Unstable cluster - suspect illegal state: trying to move shard from primary mode to replica mode

2014-07-23 Thread Mohamed Lrhazi
I think am running into this same issue, even after upgrading to 1.2.2. Did you stabilize your cluster? Thanks, Mohamed. On Saturday, May 24, 2014 5:05:55 AM UTC-4, Robin Clarke wrote: And found this error too in one of the nodes which left the cluster: java.lang.NullPointerException

Re: Sudden high OS Load, then ES VM disappears

2014-07-23 Thread michael
One additional piece of information -- the .yml conf file we use: https://gist.github.com/schonfeld/ef8f012eb0775be202ce On Wednesday, July 23, 2014 9:31:45 AM UTC-4, mic...@modernmast.com wrote: Hey all! I'm having some a serious problem with my ES cluster. Every now and then, when

Re: Is there a better way to achieve my goal than having multiple completion suggesters on a single index?

2014-07-23 Thread Gordon Rankin
Thanks Adrien... Thanks for your speedy response. I'm very new to Elasticsearch so it's good to know I am doing the write thing. I guess i'll continue as I am unless anyone else can think of any reason not to. Cheers! On Wednesday, July 23, 2014 2:34:08 PM UTC+1, Adrien Grand wrote: Hi

Re: Sudden high OS Load, then ES VM disappears

2014-07-23 Thread Nikolas Everett
I'm not sure what OS Load is in this context but I'm guessing it is load average. The shape of the memory usage graph indicates that the orange node might be stuck in a garbage collection storm - the numbers for heap aren't going up and down - just staying constant while the load is pretty high.

Re: Is ES capable of doing pagination?

2014-07-23 Thread Nikolas Everett
Scan/scroll is also not for exposing to web scale users. Fine for tens of users, not for millions. There is non-trivial cost on the cluster during scan/scroll. For the most part we just use from and size. There is a setting called preference that might be worth looking at if you expect lots of

non existing scripts execute - what is going on

2014-07-23 Thread Adrian
My windows test machine where I have ES installed has restarted automatically several times during the day. I was testing some custom scripts in ES. After restart I went back to continue my tests and realized that any script I told ES to run it would do the same thing and always give me

Re: Sudden high OS Load, then ES VM disappears

2014-07-23 Thread michael
Looking at the JVM GC graphs, I do see some increases there, but not sure those are enough to cause this storm? https://lh6.googleusercontent.com/-4wVrdN5UNRY/U8_DuSsh15I/AAk/prHDyOwB_gE/s1600/Screenshot+2014-07-23+10.05.10.png The disk graphs in Marvel don't show anything out of the

Add / Remove nodes in cluster, good practice question

2014-07-23 Thread Pierre-Vincent Ledoux
Hi, We have a little cluster of 2 nodes, hosting 4 indexes of about 1.5M documents each, replicated on both nodes. Those 2 nodes are on VPS that are stored on the same physical host. As it represents a single point of failure, we have decided to start a new VPS on a different host. What is

Sortting based on combination of ES score and two fields

2014-07-23 Thread M_20
Hi, How can I sort the result based on summation of several fields? For my application, I want to sort the results based on ( ElasticSearch score + field_1 + field_2 ) In fact: Final Score = ElasticSearch score + field_1 + field_2 It seems kinda recursive, but I was wondering ES is able to do

When to use multiple clusters

2014-07-23 Thread Alex Kehayias
I have several large indices (100M docs) on the same cluster. Is there any advice of when it is appropriate to separate into multiple clusters vs one large one? Each index has a slightly different usage profile (read vs write heavy, update vs insert). How many indices would you recommend for a

Re: Sortting based on combination of ES score and two fields

2014-07-23 Thread M_20
It seems this is what I need script_score : { script : _score + doc['my_numeric_field_1'].value + doc['my_numeric_field_2'].value } Am I right?? On Wednesday, July 23, 2014 9:29:22 AM UTC-5, M_20 wrote: Hi, How can I sort the result based on summation of several fields? For my

Using _msearch with suggesters only?

2014-07-23 Thread Gordon Rankin
I have to query several completion suggesters at the same time. This is easy to do using the _suggest api. However if I want to query multiple suggesters on different indexes I have two choices: 1. Perform multiple http requests using the _suggest api 2. Use the _msearch api. I am

shingle filter for sub phrase matching

2014-07-23 Thread Nick Tackes
I have created a gist with an analyzer that uses filter shingle in attempt to match sub phrases. For instance I have entries in the table with discrete phrases like EGFR Lung Cancer Lung Cancer and I want to match these when searching the phrase 'EGFR related lung cancer My

How Recover mistaken delete index?

2014-07-23 Thread 闫旭
Dear All! How can i recover deleted indux? or how to backup the index? Thanks Best Regard! -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to

Re: Sudden high OS Load, then ES VM disappears

2014-07-23 Thread Nikolas Everett
On Wed, Jul 23, 2014 at 10:19 AM, mich...@modernmast.com wrote: Looking at the JVM GC graphs, I do see some increases there, but not sure those are enough to cause this storm?

kibana panel not refreshed when query changes

2014-07-23 Thread paco
hi, how can i force my custom kibana panel to be filtered by a query? now it only gets filtered if I save the dashboard with the new query and then I refresh the page ¿maybe something lacks in the code in order for renderinge getting invoke after query changes? -- You received this message

Increase writing speed from hive

2014-07-23 Thread Sakthi
How to increase writing speed to ES from hive? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this

Re: How Recover mistaken delete index?

2014-07-23 Thread Benoit Gagnon
To create and restore backups, use the Snapshots And Restore http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-snapshots.html APIs. On Wednesday, July 23, 2014 10:58:23 AM UTC-4, 闫旭 wrote: Dear All! How can i recover deleted indux? or how to backup the index?

Re: Sudden high OS Load, then ES VM disappears

2014-07-23 Thread michael
No, the VM does not response to curl requests. Closest thing I found to that read bytes in the API was the _cluster/stats endpoint -- https://gist.github.com/schonfeld/d45401e44f5961c38502 Were you referring to a different endpoint? What're your thoughts re angry hardware? Insufficient

Re: shingle filter for sub phrase matching

2014-07-23 Thread Nick Tackes
#create a test index with shingle mapping curl -XPUT localhost:9200/test -d '{ settings:{ index:{ analysis:{ analyzer:{ analyzer_shingle:{ tokenizer:standard, filter:[standard, lowercase, filter_stop,

IMAPRiver plugin attachemnt index issue

2014-07-23 Thread Gabriel Kapitany
Hi all, I have installed elasticserarch 1.2.1, and IMAPRiver plugin elasticsearch-river-imap-0.0.7-b20 with elasticsearch-mapper-attachments-2.2.0-SNAPSHOT. [2014-07-23 11:56:44,304][INFO ][node ] [Shiver Man] version[1.2.1], pid[28748], build[6c95b75/2014-06-03T15:02:52Z]

Re: IMAPRiver plugin attachemnt index issue

2014-07-23 Thread David Pilato
How do you know that it has not been indexed? You can't rely on the _source field. It sounds like attachment are here in attachments field. It looks good to me. May be you should check that the mapping is correct and attachments field has attachment type? --  David Pilato | Technical Advocate 

Array type limitations?

2014-07-23 Thread michael
Hey guys. I'm curious to know what are the limitations of an array type field? I'm using ES to store an array of social-network follower IDs for each of my users, and this can sometimes get big (10M+ items). Is this okay with arrays? Or should I be using something else like a nested type? My

Re: IMAPRiver plugin attachemnt index issue

2014-07-23 Thread Gabriel Kapitany
Hi David, I have checked if the attachment type matches the attachment type: contentType : text/csv; charset=us-ascii and it does. A search by a keyword in the the attachment brings up nothing. However if I do a mime64 decode on the attachment (content filed) shows that the attachment is

Re: IMAPRiver plugin attachemnt index issue

2014-07-23 Thread David Pilato
What kind of query are you running? --  David Pilato | Technical Advocate | Elasticsearch.com @dadoonet | @elasticsearchfr Le 23 juillet 2014 à 19:02:31, Gabriel Kapitany (gkapit...@gmail.com) a écrit: Hi David, I have checked if the attachment type matches the attachment type: contentType :

Re: IMAPRiver plugin attachemnt index issue

2014-07-23 Thread Gabriel Kapitany
query: curl -XGET 'http://localhost:9200/imapriverdata/_search' -d '{ query : { match : { _all : alpha } } } ' response: {took:3,timed_out:false,_shards:{total:5,successful:5,failed:0},hits:{total:0,max_score:null,hits:[]}} Thanks, Gabriel On Wednesday, July 23, 2014 12:31:31

Re: IMAPRiver plugin attachemnt index issue

2014-07-23 Thread David Pilato
Could you try: curl -XGET 'http://localhost:9200/imapriverdata/_search' -d '{   query : {         match : { attachments : alpha }     } } ' Also what gives: GET /imapriverdata/_mapping?pretty --  David Pilato | Technical Advocate | Elasticsearch.com @dadoonet | @elasticsearchfr Le 23 juillet

Re: Help with Synonyms

2014-07-23 Thread Daniel Yim
Ivan, thank you feeding my curiosity! The first one really gave me an a-ha! moment when I saw the images of the synonym matching as directed graphs. It put some insight as to why my multi-token synonyms were being expanded a certain way. On Tuesday, July 22, 2014 4:37:45 PM UTC-5, Ivan Brusic

Re: Sudden high OS Load, then ES VM disappears

2014-07-23 Thread Nikolas Everett
Heap size isn't total memory size. Its size for java to allocate stuff. There are tons of other memory costs but the rule of thumb is to set heap to no more then 30GB and around half of physical memory. I imagine docker is complicating things. I'm not sure what docker does with memory mapped

Re: IMAPRiver plugin attachemnt index issue

2014-07-23 Thread Gabriel Kapitany
Changing the query from *_all* to* attachments* doesn't change the result and the second query returns: { imapriverdata : { mappings : { imapriverstate : { properties : { errormsg : { type : string }, exists : { type :

AWS t2.medium vs. Linode 4GB

2014-07-23 Thread Aivis Silins
Hello, Today I did performance tests (some kind of) between Linode 4GB and AWS t2.medium - just to see which one is faster in this situation. Test results are quite similar between these two - which is a bit interesting (or probably not - let me know what do you think), because AWS t2.medium

Re: one index has different _type and different _type have same field with different type will disturb?

2014-07-23 Thread Ivan Brusic
In the end, all the documents end up in the same Lucene index, and while Lucene is schema-less, all similarly named fields must be the same type. Types are useful in Elasticsearch to separate different type configurations, but will fail on similarly named fields. There is some work being done to

Re: Sudden high OS Load, then ES VM disappears

2014-07-23 Thread joergpra...@gmail.com
How do you estimate the memory to configure? Here is my rough estimation: Docker limit = 10G Kernel, OS services etc. ~ 1G OS filesystem cache for ES ~50% of 10G ~ 5G ES JVM + direct buffer + heap = 10G - 1G - 5G ~ 4G So when you estimate for ES JVM + direct buffers ~1G, you have left 3G

Re: IMAPRiver plugin attachemnt index issue

2014-07-23 Thread David Pilato
That's what I thought. attachments does not have attachment type.           attachments : {             properties : {               content : {                 type : string               },               contentType : {                 type : string               },               filename : {  

Re: AWS t2.medium vs. Linode 4GB

2014-07-23 Thread arshpreet singh
On Wed, Jul 23, 2014 at 11:07 PM, Aivis Silins aivis.sil...@gmail.com wrote: Hello, Today I did performance tests (some kind of) between Linode 4GB and AWS t2.medium - just to see which one is faster in this situation. Test results are quite similar between these two - which is a bit

Re: one index has different _type and different _type have same field with different type will disturb?

2014-07-23 Thread xu piao
my es log is : org.elasticsearch.transport.RemoteTransportException: [Yuriko Oyama][inet[/10.0.8.102:19300]][search/phase/query] Caused by: org.elasticsearch.search.query.QueryPhaseExecutionException: [matrix][1]: query[filtered(name:北京 西路^20.0 tags:北京 西路^5.0 intra:北京 西路 region:北京 西路^2.0

SIREn plugin for nested documents

2014-07-23 Thread Ivan Brusic
Has anyone else seen this plugin? http://siren.solutions/siren/overview/ There was some discussion between one of the developers and Jorg a while back, so I guess this is the outcome. Have not tried it yet, but I will give it a shot this weekend. I am hoping that it can fix a longstanding issue

Re: IMAPRiver plugin attachemnt index issue

2014-07-23 Thread Gabriel Kapitany
Hey Dave, Thanks a lot for all your help, Gabriel On Wednesday, July 23, 2014 12:31:31 PM UTC-4, Gabriel Kapitany wrote: Hi all, I have installed elasticserarch 1.2.1, and IMAPRiver plugin elasticsearch-river-imap-0.0.7-b20 with elasticsearch-mapper-attachments-2.2.0-SNAPSHOT.

How do I set up ElasticSearch nodes on EC2 so they have consistent DNS entries?

2014-07-23 Thread Ryan V
I have 3 ElasticSearch nodes in a cluster on AWS EC2. My client apps use connection pooling and have the public IP addresses for all 3 nodes in their config files. The problem I have is that EC2 seems to occasionally reassign public IP addresses for these instances. They also change if I

Re: SIREn plugin for nested documents

2014-07-23 Thread joergpra...@gmail.com
I noticed Siren has an example of 1000 library catalog records from British Library prepared in JSON https://github.com/sindicetech/siren/blob/master/siren-elasticsearch-demo/src/example/datasets/bnb/ From what it seems, Siren can index a tree (semi-structured data), using positional nodes, then

Re: SIREn plugin for nested documents

2014-07-23 Thread Brian
Thanks for the link. Unfortunately, Chrome on Mac OS (latest versions of each) causes this web page to blank and redisplay continually. Can't read it; hope you can. In a previous life, I created a search engine that handled parent/child relationships with blindingly fast performance. One trick

Re: Get top n parents based on sum of score of its children

2014-07-23 Thread Maxime Nay
Hi, I came up with something but I am not sure this is optimal. Can you let me know what I could improve? { query: { bool: { must: [ {has_child : { type : books, score_mode: sum, query : {

long String retrieved as empty; short String retrieved fine; why ES?

2014-07-23 Thread Adrian
I use this script to inspect data in various docs from ES. { query: { match_all: {} }, sort: { _script: { script: if(doc['site'].values.contains(12)){return 'foo'}else{return doc['dataX'].values }, type: string, order: desc } } } The only important part of

Re: long String retrieved as empty; short String retrieved fine; why ES?

2014-07-23 Thread Adrian
Correction: I get somewhere around 220 characters in that String NOT 40-50 as I originally mentioned. On Wednesday, 23 July 2014 15:36:10 UTC-4, Adrian wrote: I use this script to inspect data in various docs from ES. { query: { match_all: {} }, sort: { _script: {

Re: Sorting on a custom script field in Java

2014-07-23 Thread M_20
Hi Igor, I wrote a code to sort on custom script( based on ES documentation http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-sort.html ), but it doesn't work. Here are the code and the error message. Any suggestion? / journalscorenormal -- the field's

Can I filter with exact phrases?

2014-07-23 Thread IronMike
How can I exclude exact phrases with a filter? Lets say I want to search for heaven is blue but exclude exact phrase nature books. I understand I can use a bool query with must must_not, but is there a way to filter exact phrases like the example below, but instead of terms, I use

Re: Can I filter with exact phrases?

2014-07-23 Thread Ivan Brusic
You can wrap any query with a query filter: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-filter.html -- Ivan On Wed, Jul 23, 2014 at 1:52 PM, IronMike sabdall...@gmail.com wrote: How can I exclude exact phrases with a filter? Lets say I want to

Re: Sorting on a custom script field in Java

2014-07-23 Thread M_20
I found the error !! enable dynamic scripting in config file http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html On Wednesday, July 23, 2014 3:05:11 PM UTC-5, M_20 wrote: Hi Igor, I wrote a code to sort on custom script( based on

[ANN] Elasticsearch ICU Analysis plugin 2.3.0 released

2014-07-23 Thread Elasticsearch Team
Heya, We are pleased to announce the release of the Elasticsearch ICU Analysis plugin, version 2.3.0. The ICU Analysis plugin integrates Lucene ICU module into elasticsearch, adding ICU relates analysis components.. https://github.com/elasticsearch/elasticsearch-analysis-icu/ Release Notes

[ANN] Elasticsearch Stempel (Polish) Analysis plugin 2.3.0 released

2014-07-23 Thread Elasticsearch Team
Heya, We are pleased to announce the release of the Elasticsearch Stempel (Polish) Analysis plugin, version 2.3.0. The Stempel (Polish) Analysis plugin integrates Lucene stempel (polish) analysis module into elasticsearch.. https://github.com/elasticsearch/elasticsearch-analysis-stempel/

[ANN] Elasticsearch Japanese (kuromoji) Analysis plugin 2.3.0 released

2014-07-23 Thread Elasticsearch Team
Heya, We are pleased to announce the release of the Elasticsearch Japanese (kuromoji) Analysis plugin, version 2.3.0. The Japanese (kuromoji) Analysis plugin integrates Lucene kuromoji analysis module into elasticsearch.. https://github.com/elasticsearch/elasticsearch-analysis-kuromoji/

[ANN] Elasticsearch Smart Chinese Analysis plugin 2.3.0 released

2014-07-23 Thread Elasticsearch Team
Heya, We are pleased to announce the release of the Elasticsearch Smart Chinese Analysis plugin, version 2.3.0. Smart Chinese Analysis plugin integrates Lucene Smart Chinese analysis module into elasticsearch.. https://github.com/elasticsearch/elasticsearch-analysis-smartcn/ Release Notes

[ANN] Elasticsearch Phonetic Analysis plugin 2.3.0 released

2014-07-23 Thread Elasticsearch Team
Heya, We are pleased to announce the release of the Elasticsearch Phonetic Analysis plugin, version 2.3.0. The Phonetic Analysis plugin integrates phonetic token filter analysis with elasticsearch.. https://github.com/elasticsearch/elasticsearch-analysis-phonetic/ Release Notes -

Filtr range on id field faster?

2014-07-23 Thread sada
Hi! I'm new in ES. I have a quick question. Does filtr range on _id field work faster than on another field mapped to integer? -- View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Filtr-range-on-id-field-faster-tp4060481.html Sent from the ElasticSearch Users

[ANN] Elasticsearch Python language plugin 2.3.0 released

2014-07-23 Thread Elasticsearch Team
Heya, We are pleased to announce the release of the Elasticsearch Python language plugin, version 2.3.0. The Python language plugin allows to have python as the language of scripts to execute.. https://github.com/elasticsearch/elasticsearch-lang-python/ Release Notes -

Re: Array type limitations?

2014-07-23 Thread joergpra...@gmail.com
If you want to use more than 1024 terms, you will hit the Lucene max clause limit. Managing array is not a good idea with 10M+ items. You'd have to iterate it by yourself for appending/modifying which will take a very long time (and space). Maybe you find this interesting for your model design

JDK version 1.7.0_65?

2014-07-23 Thread Greg Brown
The current ES docs reference Java 7 u60: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/_installation.html#_installation Lucene seems to just say u55 or higher: http://lucene.apache.org/core/4_9_0/SYSTEM_REQUIREMENTS.html Has anyone used/verified ES with OpenJDK 7u65?

Re: Array type limitations?

2014-07-23 Thread michael
Thanks for the response, Jörg. When I filter by follower_ids, I actually use Elasticsearch's terms lookup feature, so I never run into the 1024 max clause limit. That said, because I append 5k IDs to that field at a time, you are correct -- appending 5k IDs to an array with millions of

Re: Creating Dynamic Dashboard with Kibana

2014-07-23 Thread Andre Encarnacao
I am actually wondering the same thing. Is there a way to query ES from within a Kibana dashboard script in order to make decisions on how to layout our dashboard? I tried using the ElasticSearch-js API (copy and pasted the entire API into the dashboard script) but there's no way to use the

[ANN] Elasticsearch JavaScript language plugin 2.3.0 released

2014-07-23 Thread Elasticsearch Team
Heya, We are pleased to announce the release of the Elasticsearch JavaScript language plugin, version 2.3.0. The JavaScript language plugin allows to have javascript as the language of scripts to execute.. https://github.com/elasticsearch/elasticsearch-lang-javascript/ Release Notes -

Re: Add / Remove nodes in cluster, good practice question

2014-07-23 Thread Mark Walkom
If this is production you really want an odd number of nodes to reduce potential split brain issues. However in your case, just add the new node to the cluster, let it replicate across, then shutdown the node you no longer want. Any impact will be minimal. Regards, Mark Walkom Infrastructure

Re: When to use multiple clusters

2014-07-23 Thread Mark Walkom
Depends what your hardware profiles are like, and a bunch of other things related to you and your environment. eg If you have high end servers then it makes sense to put your heavy read/write indexes into a cluster on those, then leave the rest for more average machines. We have multiple clusters

Re: When to use multiple clusters

2014-07-23 Thread Alex Kehayias
Thanks Mark! We're deploying on EC2 (always a good time). Seems like the mixture of different indices that have different usage profiles is leading to some performance issues that a dedicated cluster would be more appropriate for. On Wednesday, July 23, 2014 7:04:34 PM UTC-4, Mark Walkom

ES 1.3.0 and 1.2.3 released

2014-07-23 Thread Mark Walkom
http://www.elasticsearch.org/blog/elasticsearch-1-3-0-released/ Today, we are happy to announce the release of *Elasticsearch 1.3.0*, based on* Lucene 4.9*, along with a bugfix release of *Elasticsearch 1.2.3*. You can download them and read the full changes list here: - Latest stable

Resuming Interrupted Reindexing Process

2014-07-23 Thread Sarang Zargar
We are reindexing an index with following attributes: - Size ~3TB - Ever increasing (new records are pumped at high volume during peak hours) Is there a clean way to resume an interrupted reindexing process? Does elastic search provides a way to persist the scrolls? If we can use

Re: Add / Remove nodes in cluster, good practice question

2014-07-23 Thread Nikolas Everett
1. Fix the unicast discovery list to contain the two nodes you'll end up with. Make sure the cluster name matches. 2. Start the new node. 3. Watch it join the cluster. If it doesn't then check step 1 and repeat 2 until it works. 4. Use the allocation filtering API to ban all shards from the node

Structure of Kibanas Micro Analysis query.

2014-07-23 Thread Finn Poitier
Kindly enough, the queries being used in Kibana are mostly shown via the information icon i. But unfortunately there´s no query shown for the micro analyses of specific fields and I wonder about their structure actually. Let me explain: For example the micro analysis for the field

Re: one index has different _type and different _type have same field with different type will disturb?

2014-07-23 Thread xu piao
in order to save this problem, i have try many ways to search. these ways is ok : { bool : { must : { query_string : { query : \采莲新村\, fields : [ name^200, tags^5, intra^1, region^2, address^2 ] } }, minimum_should_match : 1 } } { bool : { must

Re: ES 1.3.0 and 1.2.3 released

2014-07-23 Thread vjbangis
is the bigdesk plugin still supported in the new release? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view

Re: [ERROR][bootstrap] {1.2.2}: Initialization Failed ... - NullPointerException[null]

2014-07-23 Thread vjbangis
Finally! the node can now added/remove to the cluster with only four lines cluster.name: Cluster node.name: [node?] discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: [node?] May next question is, where can I dump the data sources (using S3 or EBS), why? Then, how

prevent 'match_phrase' from evaluating score

2014-07-23 Thread 陳智清
Hello, Is it possible to have a 'match_phrase' query which does not do any scoring during search? I know that using 'query filter' or wraps a query in 'constant_score query' return documents with constant score. But as I know, what they do is cutting of the evaluated score, i.e. the score

Re: prevent 'match_phrase' from evaluating score

2014-07-23 Thread 陳智清
In fact, not only 'match_phrase', we want to eliminate scoring of all queries. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to

  1   2   >