I also meant to mention that I commented out the credentials in the yml file, I was still getting same message, meaning that credentials aren't probably getting read for whatever reason.
On Friday, July 4, 2014 10:46:58 AM UTC-4, sabdalla80 wrote: > > My cluster has two instances, I have the same setup on both instances. To > answer Ross's question, I use the curl command to register the repository > from the instance(s) itself. Do I need to do anything else on the > instance(s) as far as AWS credentials? > > Here is what I have: > Enter ##################### Elasticsearch Configuration Example > ##################### > > > # This file contains an overview of various configuration settings, > > # targeted at operations staff. Application developers should > > # consult the guide at <http://elasticsearch.org/guide>. > > # > > # The installation procedure is covered at > > # < > http://elasticsearch.org/guide/en/elasticsearch/reference/current/setup.html > >. > > # > > # Elasticsearch comes with reasonable defaults for most settings, > > # so you can try it out without bothering with configuration. > > # > > # Most of the time, these defaults are just fine for running a production > > # cluster. If you're fine-tuning your cluster, or wondering about the > > # effect of certain configuration option, please _do ask_ on the > > # mailing list or IRC channel [http://elasticsearch.org/community]. > > > # Any element in the configuration can be replaced with environment > variables > > # by placing them in ${...} notation. For example: > > # > > # node.rack: ${RACK_ENV_VAR} > > > # For information on supported formats and syntax for the config file, see > > # < > http://elasticsearch.org/guide/en/elasticsearch/reference/current/setup-configuration.html > > > > > > ################################### Cluster > ################################### > > > # Cluster name identifies your cluster for auto-discovery. If you're > running > > # multiple clusters on the same network, make sure you're using unique > names. > > # > > cluster.name: rexCluster > > #################################### Node > ##################################### > > > # Node names are generated dynamically on startup, so you're relieved > > # from configuring them manually. You can tie this node to a specific name: > > # > > node.name: "node 1" > > > # Every node can be configured to allow or deny being eligible as the > master, > > # and to allow or deny to store the data. > > # > > # Allow this node to be eligible as a master node (enabled by default): > > # > > node.master: true > > # > > # Allow this node to store data (enabled by default): > > # > > # node.data: true > > > # You can exploit these settings to design advanced cluster topologies. > > # > > # 1. You want this node to never become a master node, only to hold data. > > # This will be the "workhorse" of your cluster. > > # > > # node.master: false > > # node.data: true > > # > > # 2. You want this node to only serve as a master: to not store any data > and > > # to have free resources. This will be the "coordinator" of your > cluster. > > # > > # node.master: true > > # node.data: false > > # > > # 3. You want this node to be neither master nor data node, but > > # to act as a "search load balancer" (fetching data from nodes, > > # aggregating results, etc.) > > # > > # node.master: false > > # node.data: false > > > # Use the Cluster Health API [http://localhost:9200/_cluster/health], the > > # Node Info API [http://localhost:9200/_nodes] or GUI tools > > # such as <http://www.elasticsearch.org/overview/marvel/>, > > # <http://github.com/karmi/elasticsearch-paramedic>, > > # <http://github.com/lukas-vlcek/bigdesk> and > > # <http://mobz.github.com/elasticsearch-head> to inspect the cluster > state. > > > # A node can have generic attributes associated with it, which can later > be used > > # for customized shard allocation filtering, or allocation awareness. An > attribute > > # Path to directory containing configuration (this file and logging.yml): > > # > > path.conf: /etc/elasticsearch > > > # Path to directory where to store index data allocated for this node. > > # > > # path.data: /path/to/data > > # > > # Can optionally include more than one location, causing data to be > striped across > > # the locations (a la RAID 0) on a file level, favouring locations with > most free > > # space on creation. For example: > > # > > path.data: /opt/cores/elasticsearch/data > > > # Path to temporary files: > > # > > path.work: /opt/cores/elasticsearch/work > > > # Path to log files: > > # > > path.logs: /opt/cores/elasticsearch/logs > > > # Path to where plugins are installed: > > # > > # path.plugins: /path/to/plugins > > ################################### Memory > #################################### > > > # Elasticsearch performs poorly when JVM starts swapping: you should > ensure that > > # it _never_ swaps. > > # > > # Set this property to true to lock the memory: > > # > > bootstrap.mlockall: true > > ################################## Discovery > ################################## > > > # Discovery infrastructure ensures nodes can be found within a cluster > > # and master node is elected. Multicast discovery is the default. > > > # Set to ensure a node sees N other master eligible nodes to be considered > > # operational within the cluster. Its recommended to set it to a higher > value > > # than 1 when running more than 2 nodes in the cluster. > > # > > discovery.zen.minimum_master_nodes: 2 > > > # Set the time to wait for ping responses from other nodes when > discovering. > > # Set this option to a higher value on a slow or congested network > > # to minimize discovery failures: > > # > > discovery.zen.ping.timeout: 3s > > > # For more information, see > > # < > http://elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html > > > > > # Unicast discovery allows to explicitly control which nodes will be used > > # to discover the cluster. It can be used when multicast is not present, > > # or to restrict the cluster communication-wise. > > # > > # 1. Disable multicast discovery (enabled by default): > > # > > discovery.zen.ping.multicast.enabled: false > > # > > # 2. Configure an initial list of master nodes in the cluster > > # to perform discovery when new nodes (master or data) are started: > > # > > discovery.zen.ping.unicast.hosts: ["IP1","IP2"] > > > # EC2 discovery allows to use AWS EC2 API in order to perform discovery. > > # > > # You have to install the cloud-aws plugin for enabling the EC2 discovery. > > # > > # For more information, see > > # < > http://elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-ec2.html > > > > # > > # See <http://elasticsearch.org/tutorials/elasticsearch-on-ec2/> > > # for a step-by-step tutorial. > > cloud: > > > aws: > > access_key: XXXXX > > secret_key: YYYYYYYYYYY > > discovery: > > type: ec2 > > > > On Friday, July 4, 2014 2:27:48 AM UTC-4, David Pilato wrote: >> >> Agreed. Could you share your elasticsearch.yml file without touching >> anything but only replacing Key/secret? >> >> Keep the formating. >> >> -- >> David ;-) >> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs >> >> >> Le 4 juil. 2014 à 06:51, Ross Simpson <[email protected]> a écrit : >> >> That specific exception (com.amazonaws.AmazonClientException) is thrown >> by the AWS client libraries, and it means the library couldn't find your >> AWS credentials. I'm not sure why, as the details in your original post >> look correct. >> >> FWIW, S3 snapshots are working well for me. Here's my setup: >> ES 1.1.1 >> AWS cloud plugin 2.1.0 >> >> elasticsearch.yml: >> cloud.aws.access_key: ... >> cloud.aws.secret_key: ...... >> >> Repo registration: >> $ curl -XPUT 'http://localhost:9200/_snapshot/es-backups' -d >> '{"type":"s3","settings":{"compress":"true","base_path":"prod_backups","region":"us-east","bucket":"..."}}' >> {"acknowledged":true} >> >> In your latest post, it looks like you're running the command on a remote >> ES host (10.211.154.24). Does that specific host have the AWS credentials >> in its ES config? Snapshotting will require that *all* nodes in the >> cluster have the AWS credentials, because they will each be writing to S3. >> >> Are there any relevant entries in the ES logs from startup? >> >> >> >> On Friday, 4 July 2014 04:25:44 UTC+10, sabdalla80 wrote: >>> >>> I installed latest ES version too "1.2.1", still getting same error >>> { >>> "error": "RemoteTransportException[[node >>> 2][inet[/10.211.154.24:9300]][cluster/repository/put]]; nested: >>> RepositoryException[[es_repository] failed to create repository]; nested: >>> CreationException[Guice creation errors:\n\n1) Error injecting constructor, >>> com.amazonaws.AmazonClientException: Unable to load AWS credentials from >>> any provider in the chain\n at >>> org.elasticsearch.repositories.s3.S3Repository.<init>()\n at >>> org.elasticsearch.repositories.s3.S3Repository\n at >>> Key[type=org.elasticsearch.repositories.Repository, annotation=[none]]\n\n1 >>> error]; nested: AmazonClientException[Unable to load AWS credentials from >>> any provider in the chain]; ", >>> "status": 500 >>> } >>> >>> Any ideas? I would appreciate some feedback on how to figure out this >>> problem because I would like to backup our index to S3. >>> >>> On Wednesday, July 2, 2014 3:36:58 PM UTC-4, sabdalla80 wrote: >>>> >>>> Unfortunately, I tried with and without the region setting, no >>>> difference. >>>> >>>> On Tuesday, July 1, 2014 7:43:21 PM UTC-4, Glen Smith wrote: >>>>> >>>>> I'm not sure it matters, but I noticed you aren't setting a region in >>>>> either your config or when registering your repo. >>>>> >>>>> On Tuesday, July 1, 2014 7:08:28 PM UTC-4, sabdalla80 wrote: >>>>>> >>>>>> I am not sure the version is the problem, I guess I can upgrade from >>>>>> V1.1 to latest. >>>>>> "Not able to load credential from supply chain", Any idea this error >>>>>> is generated, Is there any other place that my credentials need to be >>>>>> besides .yml file? >>>>>> Note, I am able to write/read to S3 remotely, so I don't have any >>>>>> priviliges problems that I can think of. >>>>>> >>>>>> On Tuesday, July 1, 2014 4:44:17 PM UTC-4, David Pilato wrote: >>>>>>> >>>>>>> I think 2.1.1 should work fine as well. >>>>>>> >>>>>>> That said, you should upgrade to latest 1.1 (or 1.2)... >>>>>>> >>>>>>> -- >>>>>>> David ;-) >>>>>>> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs >>>>>>> >>>>>>> >>>>>>> Le 1 juil. 2014 à 22:13, Glen Smith <[email protected]> a écrit : >>>>>>> >>>>>>> According to >>>>>>> https://github.com/elasticsearch/elasticsearch-cloud-aws/tree/es-1.1 >>>>>>> you should use v2.1.0 of the plugin with ES 1.1.0. >>>>>>> >>>>>>> On Tuesday, July 1, 2014 9:03:04 AM UTC-4, sabdalla80 wrote: >>>>>>>> >>>>>>>> I am having a problem setting up backup and restore part of AWS on >>>>>>>> S3. >>>>>>>> I have 2.1.1 AWS plugin & ElasticSearch V1.1.0 >>>>>>>> >>>>>>>> My yml: >>>>>>>> >>>>>>>> cloud: >>>>>>>> aws: >>>>>>>> access_key: ######### >>>>>>>> secret_key: ##################### >>>>>>>> discovery: >>>>>>>> type: ec2 >>>>>>>> >>>>>>>> When I try to register a repository: >>>>>>>> >>>>>>>> PUT /_snapshot/es_repository{ >>>>>>>> >>>>>>>> "type": "s3", >>>>>>>> >>>>>>>> "settings": { >>>>>>>> >>>>>>>> "bucket": "esbucket" >>>>>>>> >>>>>>>> }} >>>>>>>> >>>>>>>> >>>>>>>> I get this error, it complains about loading my credentials! Is >>>>>>>> this ElasticSearch problem or AWS? >>>>>>>> >>>>>>>> Note I am running as root user "ubuntu" on Ec2 and also running AWS >>>>>>>> with root privileges as opposed to IAM role, not sure if it's a >>>>>>>> problem or >>>>>>>> not. >>>>>>>> "error": "RepositoryException[[es_repository] failed to create >>>>>>>> repository]; nested: CreationException[Guice creation errors:\n\n1) >>>>>>>> Error >>>>>>>> injecting constructor, com.amazonaws.AmazonClientException: Unable to >>>>>>>> load >>>>>>>> AWS credentials from any provider in the chain\n at >>>>>>>> org.elasticsearch.repositories.s3.S3Repository.<init>(Unknown >>>>>>>> Source)\n >>>>>>>> while locating org.elasticsearch.repositories.s3.S3Repository\n >>>>>>>> while >>>>>>>> locating org.elasticsearch.repositories.Repository\n\n1 error]; >>>>>>>> nested: >>>>>>>> AmazonClientException[Unable to load AWS credentials from any provider >>>>>>>> in >>>>>>>> the chain]; ", >>>>>>>> "status": 500 >>>>>>>> }ode here... >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "elasticsearch" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/elasticsearch/e7db355a-7710-4408-80de-60960fd16d1d%40googlegroups.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/elasticsearch/e7db355a-7710-4408-80de-60960fd16d1d%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/a3df1eb8-b6a2-4400-ba8c-58e1b2b33385%40googlegroups.com >> >> <https://groups.google.com/d/msgid/elasticsearch/a3df1eb8-b6a2-4400-ba8c-58e1b2b33385%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ef64e5a5-9295-45f4-934a-6847bc46326e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
