Re: Not able to run Riak CS on Raspberry Pi 3
Henry:
It looks like there is a typo on the following line:
> {add_paths,["/home/pi/src/riak-cs-2.1.1/rel/riak-cs/lib/riak_cs-2.1.1/ebin"$
Your advanced.config file should be similar to this:
> {riak_kv, [
>
> {add_paths,["/home/pi/src/riak-cs-2.1.1/rel/riak-cs/lib/riak_cs-2.1.1/ebin"]},
>{storage_backend, riak_cs_kv_multi_backend},
>{multi_backend_prefix_list, [{<<"0b:">>, be_blocks}]},
>{multi_backend_default, be_default},
>{multi_backend, [
>{be_default, riak_kv_eleveldb_backend, [
>{data_root,"/home/pi/src/riak/deps/eleveldb"}
>]},
>{be_blocks, riak_kv_bitcask_backend, [
>{data_root,"/home/pi/src/riak/deps/bitcask"}
> ]}
>]}
> ]}.
Please let me know if that configuration change gets you going.
Regards,
Charlie Voiselle
> On Apr 13, 2017, at 7:20 AM, Henry- Norbert Cocos <co...@stud.fra-uas.de>
> wrote:
>
> Hi List,
>
> i am having problems running Riak CS on the Raspberry Pi 3. I installed it
> from source and made the necassary changes to the "riak.config" and the
> "advanced.config".
> It currently shows me, that it has issues running the configuration.
>
> I'm also having troubles to find information on how to specify the leveldb
> and bitcask modules in the "advanced.config" file.
>
> I was looking for information in the following links:
>
> http://docs.basho.com/riak/cs/2.1.1/cookbooks/configuration/riak-for-cs/
>
> http://docs.basho.com/riak/cs/2.1.1/tutorials/fast-track/local-testing-environment/
>
> Below the config file and the error:
>
> Regards,
> Henry
>
>
> -
> Here the current advanced.config:
>
>
> {riak_kv, [
> {add_paths,["/home/pi/src/riak-cs-2.1.1/rel/riak-cs/lib/riak_cs-2.1.1/ebin"$
>{storage_backend, riak_cs_kv_multi_backend},
>{multi_backend_prefix_list, [{<<"0b:">>, be_blocks}]},
>{multi_backend_default, be_default},
>{multi_backend, [
>{be_default, riak_kv_eleveldb_backend, [
>{data_root,"/home/pi/src/riak/deps/eleveldb"}
>]},
>{be_blocks, riak_kv_bitcask_backend, [
>{data_root,"/home/pi/src/riak/deps/bitcask"}
>]}
>]}
> ]}.
>
>
> ---
> The error i am getting after running "sudo bin/riak config generate -l debug":
>
> 13:15:10.017 [info] /home/pi/src/riak/rel/riak/bin/../etc/advanced.config
> detected, overlaying proplists
> escript: exception error: no function clause matching
>
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Production deployment requirements for memory backend storage
Neeraj: Thanks for you interest in Riak. I will copy your questions into this email for reference and answer them inline. 1. What is the “platform_data_dir” used for when memory is used as storage backend? Is it only needed for active anti-entropy and cluster metadata? Do I need to persist this data i.e. if a node goes down and restarts in this configuration, is persistence of data in “platform_data_dir” required. As you have pointed out, the platform_data_dir contains more than just the actual data stored in the cluster. There are three folders that must be persisted for a node to remain a member of a cluster and to not create issues with the sizes of the vector clocks internal to the objects. They are: ring - The binary files that describe the cluster and the vnode ownership mappings. Deleting this folder will cause the node to start up and create a new default ring. This default ring will allocate 100% of the partitions to that node. This is non-fatal and is resolved by rejoining the node to the cluster. This extra work can be avoided by persisting the ring file properly. cluster_meta - This folder contains the properties for bucket types and typed custom buckets. kv_vnode - This folder contains generated actor-ids for each Riak vnode. The routine loss of this directory will cause orphaned vnode actor-ids to potentially accumulate in objects’ vclocks. Active anti-entropy is a process to prevent bit-rot in long-lived data. Since your questions we concerning ephemeral data, we would recommend that it be disabled because there are overheads in creating and maintaining the trees that make no sense for ephemeral data. 2. What is the minimum memory requirement of an empty Riak node in this configuration? On a sample node that I brought up, an empty Riak KV 2.2.3, the beam.smp process was using 1.5 gb of RAM with an empty memory backend and AAE-disabled. 3. What is the minimum disk and CPU requirement of a Riak node in this configuration? There are a few variables that dictate how much actual disk throughput you will use in a Riak cluster that only uses the memory backend-logging overhead, ring changes, and cluster metadata changes. Logging throughput is determined by the general health of the cluster and is minimal in clusters that are well-behaved. The logfiles themselves have configurable size caps and set numbers of rotations (by default 5 logs capped at 50mb for each file). There are some other logfiles that are not managed by lager and they can grow beyond these expected limits. If you are building nodes optimized for storage, you will want to monitor the size of this folder and trim it as appropriate. The ring is a data structure that is used to hold information about the cluster’s membership, the node capabilities, MDC replication configuration, and the legacy custom bucket metadata. In stable clusters that are using no custom buckets the impact of writes to the ring is negligible; however there are certain antipatterns involving the creation of a large number of buckets with custom properties in the “default” bucket type that will bloat the ring file and result in a large amount of ring gossip. Finally, Riak bucket types and their properties as well as the custom bucket properties of typed buckets is stored in cluster-metadata. This backend is a dets-based store that uses hashtree comparisons to maintain consistency across members of the cluster. This backend’s storage also depends on the amount and speed with which you create metadata within your cluster. There is more generically-applicable information about [cluster capacity planning] <http://docs.basho.com/riak/kv/2.2.3/setup/planning/cluster-capacity/> in the Riak KV documentation. Thanks again for your interest, Charlie Voiselle Sr. Product Manager, Riak KV/Clients Basho Technologies @angrycub [cluster capacity planning] - http://docs.basho.com/riak/kv/2.2.3/setup/planning/cluster-capacity/ <http://docs.basho.com/riak/kv/2.2.3/setup/planning/cluster-capacity/> > On Apr 10, 2017, at 3:30 PM, Neeraj Poddar <n.pod...@f5.com> wrote: > > Hello, > > I wanted to understand the production requirements for using Riak as a > non-persistent ephemeral data store. In particular the following questions > relate to using Riak with “memory” configured as storage backend: > > 1. What is the “platform_data_dir” used for when memory is used as > storage backend? Is it only needed for active anti-entropy and cluster > metadata? Do I need to persist this data i.e. if a node goes down and > restarts in this configuration, is persistence of data in “platform_data_dir” > required. > 2. What is the minimum memory requirement of an empty Riak node in this > configuration? > 3. What is the minimum disk and CPU requirement of a Riak node in this > configuration? > > -- > Regards, > Neeraj Podda
[ANN] Riak KV 2.2.3 is now available
Today we are pleased to release Riak KV 2.2.3. Riak KV 2.2.3 is the latest bug-fix package for our current feature release, and continues Basho’s commitment to our customers and community to develop and deliver a highly resilient, high-performance, key-value database. In fixing a riak and riak-admin issue <https://github.com/basho/node_package/pull/210>, we inadvertently used a Bash-specific script variable ($RANDOM), causing the riak attach and riak top commands to fail on Ubuntu when /bin/sh is aliased to Dash. We opted to release a full bugfix version because of a small shell script bug that could impact all platforms; there is no impact to the operation and data in the riak cluster. These errors could cause certain riak-admin and riak commands to fail to start. This bugfix removes the use of $RANDOM, allowing all riak and riak-admin commands to function properly in Dash. Information about the specific fixes in this release can be found in the release notes[1]. Downloading Riak KV 2.2.3 Open Source users can download packages from the Downloads page[2]. Riak KV Enterprise customers can download Riak KV from your Zendesk page (valid Basho Zendesk account is required). If you run Riak KV on AWS, you can also update to using the latest AMI. As always, we would like to hear your feedback, suggestions or bug reports. Best regards, Charlie Voiselle Senior Product Manager - Riak KV, Basho Technologies @angrycub [issue] - https://github.com/basho/node_package/pull/210 <https://github.com/basho/node_package/pull/210> [1] - http://docs.basho.com/riak/kv/2.2.3/release-notes/ <http://docs.basho.com/riak/kv/2.2.3/release-notes/> [2] - http://docs.basho.com/riak/kv/2.2.3/downloads/ <http://docs.basho.com/riak/kv/2.2.3/downloads/>___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
[ANN] - Issues with Riak KV 2.2.2 packages for Ubuntu
We have identified a late-breaking issue specific to the Ubuntu builds. We have pulled the packages from their hosted locations in the interim, and they will be available shortly. I will send an email to the list once we have resolved this issue. Thanks, Charlie Voiselle Senior Product Manager - Riak KV, Basho Technologies @angrycub ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
[ANN] Riak KV 2.2.2 now available
Today we are pleased to release Riak KV 2.2.2. Riak KV 2.2.2 is the latest bug-fix package for our current feature release, and continues Basho’s commitment to our customers and community to develop and deliver a highly resilient, high-performance, key-value database. In KV 2.2.1 we fixed <https://github.com/basho/node_package/pull/183> an issue <https://github.com/basho/riak/issues/509> with nodetool, allowing it to work with SSL options in vm.args. However, in changing NODETOOL to an alias, four riak-admin commands (handoff, get, set, and describe) that used exec to allow them to run properly on all of our supported operating systems, now produce the following error message when they are run: [root@node1 ~]# riak-admin handoff /usr/sbin/riak-admin: line : exec: NODETOOL: not found This bugfix release resolves the issues with exec and NODETOOL, allowing handoff, get, set, and describe to function properly. We have located the broken QA test that misidentified these commands as working and have corrected it for future releases. Information about the specific fixes in this release can be found in the release notes <http://docs.basho.com/riak/kv/2.0.9/release-notes/>[1]. Downloading Riak KV 2.2.2 Open Source users can download packages from the Downloads page <http://docs.basho.com/riak/kv/2.2.2/downloads/>[2]. Riak KV Enterprise customers can download Riak KV from your Zendesk page (valid Basho Zendesk account is required). If you run Riak KV on AWS, you can also update to using the latest AMI. As always, we would like to hear your feedback, suggestions or bug reports. Best regards, Charlie Voiselle Senior Product Manager - Riak KV, Basho Technologies @angrycub [fixed] - https://github.com/basho/node_package/pull/183 <https://github.com/basho/node_package/pull/183> [issue] - https://github.com/basho/riak/issues/509 <https://github.com/basho/riak/issues/509> [1] - http://docs.basho.com/riak/kv/2.2.2/release-notes/ <http://docs.basho.com/riak/kv/2.2.2/release-notes/> [2] - http://docs.basho.com/riak/kv/2.2.2/downloads/ <http://docs.basho.com/riak/kv/2.2.2/downloads/> ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
[ANN] Riak KV 2.0.9 now available
Today we are pleased to release Riak KV 2.0.9. Riak KV 2.0.9 is the latest release in our long-term support version, and continues Basho’s commitment to our customers and community to develop and deliver a highly resilient, high-performance, key-value database. In KV 2.0.8 we fixed <https://github.com/basho/node_package/pull/183> an issue <https://github.com/basho/riak/issues/509> with nodetool, allowing it to work with SSL options in vm.args. However, in changing NODETOOL to an alias, four riak-admin commands (handoff, get, set, and describe) that used exec to allow them to run properly on all of our supported operating systems, now produce the following error message when they are run: [root@node1 ~]# riak-admin handoff /usr/sbin/riak-admin: line : exec: NODETOOL: not found This bugfix release resolves the issues with exec and NODETOOL, allowing handoff, get, set, and describe to function properly. We have located the broken QA test that misidentified these commands as working and have corrected it for future releases. Information about the specific fixes in this release can be found in the release notes <http://docs.basho.com/riak/kv/2.0.9/release-notes/>[1]. Riak KV 2.2.1 is impacted by a similar issue, and will be corrected by an upcoming 2.2.2 release. Downloading Riak KV 2.0.9 Open Source users can download packages from the Downloads page <http://docs.basho.com/riak/kv/2.0.9/downloads/>[2]. Riak KV Enterprise customers can download Riak KV from your Zendesk page (valid Basho Zendesk account is required). If you run Riak KV on AWS, you can also update to using the latest AMI. As always, we would like to hear your feedback, suggestions or bug reports. Best regards, Charlie Voiselle Senior Product Manager - Riak KV, Basho Technologies @angrycub [fixed] - https://github.com/basho/node_package/pull/183 <https://github.com/basho/node_package/pull/183> [issue] - https://github.com/basho/riak/issues/509 <https://github.com/basho/riak/issues/509> [1] - http://docs.basho.com/riak/kv/2.0.9/release-notes/ <http://docs.basho.com/riak/kv/2.0.9/release-notes/> [2] - http://docs.basho.com/riak/kv/2.0.9/downloads/ <http://docs.basho.com/riak/kv/2.0.9/downloads/> ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
[ANN] Riak KV 2.2.1 now available
Today we are pleased to release Riak KV 2.2.1. Riak KV 2.2.1 is the latest bugfix release, and continues Basho’s commitment to our customers and community to develop and deliver a highly resilient, high-performance, key-value database. Information about the specific fixes in this release can be found in the release notes[1] <http://docs.basho.com/riak/kv/2.2.1/release-notes/>. Downloading Riak KV 2.2.1 Open Source users can download packages from the Downloads page[2] <http://docs.basho.com/riak/kv/2.2.1/downloads/>. Riak KV Enterprise customers can download Riak KV from your Zendesk page (valid Basho Zendesk account is required). If you run Riak KV on AWS, you can also update to using the latest AMI once it is released. As always, we would like to hear your feedback, suggestions or bug reports. Best regards, Charlie Voiselle Senior Product Manager - Riak KV, Basho Technologies @angrycub [1] - http://docs.basho.com/riak/kv/2.2.1/release-notes/ <http://docs.basho.com/riak/kv/2.2.1/release-notes/>[2] - http://docs.basho.com/riak/kv/2.2.1/downloads/ <http://docs.basho.com/riak/kv/2.2.1/downloads/>___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Handoffs are too slow after netsplit
Andrey: Another thing that can stall handoff is running commands that reset the vnode activity timer. The `riak-admin vnode-status` command will reset the activity timer and should never be run more frequently that then vnode inactivity timeout; if you do, that can permanently stall handoff. We have seen this before at a customer site where they were collecting the statics from the vnode-status command into their metrics system. Regards, Charlie Voiselle > On Feb 23, 2017, at 7:14 AM, Douglas Rohrer <droh...@basho.com> wrote: > > Andrey: > > It's waiting for 60 seconds, literally... > > See > https://github.com/basho/riak_core/search?utf8=%E2%9C%93=vnode_inactivity_timeout > > <https://github.com/basho/riak_core/search?utf8=%E2%9C%93=vnode_inactivity_timeout> > - handoff is not initiated until a vnode has been inactive for the specified > inactivity period. > > For demonstration purposes, if you want to reduce this time, you could set > the riak_core.vnode_inactivity_timeout period lower ,which can be set in > advanced.config. Also note that, depending on the backend you use, it's > possible to have other settings set lower than the vnode inactivity timeout, > you can actually prevent handoff completely - see > http://docs.basho.com/riak/kv/2.2.0/setup/planning/backend/bitcask/#sync-strategy > > <http://docs.basho.com/riak/kv/2.2.0/setup/planning/backend/bitcask/#sync-strategy>, > for examnple. > > Hope this helps. > > Doug > > On Thu, Feb 23, 2017 at 6:40 AM Andrey Ershov <andrers...@gmail.com > <mailto:andrers...@gmail.com>> wrote: > Hi, guys! > > I'd like to follow up on handoffs behaviour after netsplit. The problem is > that right after network partition is healed, "riak-admin transfers" command > says that there are X partitions waiting transfer from one node to another, > and Y partitions waiting transfer in the opposite direction. What are they > waiting for? Active transfers section is always empty. It takes about 1 > minute for transfer to occur. I've increased transfer_limit to 100 and it > does not help. > Also I've tried to attach to Erlang VM and execute > riak_core_vnode_manager:force_handoff() on each node. This command returns > 'ok'. But seems that it does not work right after network is healed. After > some time 30-60 s, force_handoff() works as expected, but actually it's the > same latency as in auto handoff case. > > So what is it waiting for? Any ideas? > > I'm preparing real-time coding demo to be shown on the conference. So it's > too much time to wait for 1 minute for handoff to occur just for a couple of > keys... > -- > Thanks, > Andrey > ___ > riak-users mailing list > riak-users@lists.basho.com <mailto:riak-users@lists.basho.com> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com> > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
[ANN] Riak KV operating system packaging deprecations
As part of our release process, we regularly review our list of supported operating systems(OS) and OS versions for additions and deprecations. Deprecations fall into one of two categories: 1) deprecation of an aging OS version 2) deprecation of an entire operating system. When it comes to the deprecation of aging OS versions, we closely monitor end-of-life notices from OS vendors to determine when to stop building for a specific OS version. Stability and availability are the reasons you chose Riak KV, and without active vendor support for an underlying operating system, both of these values can be compromised. The deprecation of an entire OS is a more difficult decision because we would love to have a Riak package for any platform where someone wants to run Riak. However, attempting to support too many platforms risks inadequate support for some platforms. Thus, we monitor statistics for which operating systems are most frequently used by our customers and community members. We have found a few that are significantly less popular among Riak users. Basho will be removing official support for these operating systems to allow us to focus more attention on the ones most often deployed. So, let’s get to the details: Starting with Riak KV 2.0.8, we have stopped building packages for SmartOS 1.3 and Debian 6. For all future Riak KV releases, we will cease building packages for all SmartOS versions, all Solaris versions, and all Fedora Core versions. Basho customers with any questions are encouraged to contact either their account managers or the Basho Client Services <https://help.basho.com/> team. Basho community members can contact us via the riak-users mailing list. Best regards, Charlie Voiselle Senior Product Manager - Riak KV, Basho Technologies @angrycub ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
[ANN] Announcing Riak KV 2.2.0
Today we are pleased to release Riak KV 2.2.0. Riak KV 2.2 continues Basho’s commitment to our customers and community to develop and deliver a highly resilient, high-performance, key-value database. The new features include: * Global Object Expiration in LevelDB * LZ4 compression for LevelDB * Support for HyperLogLog Data Types * Improved Active Anti-Entropy Performance * Cluster Job Controls * Improvements to Riak Search (up to 5x faster ingest) * Support for Debian 8 and Ubuntu 16 To get more information about the features, please review the blog post announcing Riak KV 2.2 <http://basho.com/posts/technical/riak-kv-2-2-release-highlights/> [1] and the Release Notes <http://docs.basho.com/riak/kv/2.2.0/release-notes/> [2]. Also, make sure you take a look at the Upgrade <http://docs.basho.com/riak/kv/2.2.0/setup/upgrading/cluster/> [3]/Downgrade <https://docs.basho.com/riak/kv/2.2.0/setup/downgrade/> [4] information when planning your upgrade. For a demo or to talk to one of our Riak Solution Architects, please contact us <http://basho.com/contact/>. Finally, thanks to the many Basho team members for their continued efforts working with our community and our customers on this new release of Riak KV. Charlie Voiselle Senior Product Manager - Riak KV, Basho Technologies @angrycub [1] - http://basho.com/posts/technical/riak-kv-2-2-release-highlights/ [2] - http://docs.basho.com/riak/kv/2.2.0/release-notes/ [3] - http://docs.basho.com/riak/kv/2.2.0/setup/upgrading/cluster/ [4] - http://docs.basho.com/riak/kv/2.2.0/setup/downgrade/___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak KV evolution
Ricardo: Thank you for your question, and I want to reassure you that Riak KV is still very much under active development. Work that is being done in the Riak TS codebase is being used to improve Riak KV where it applies. Riak KV 2.2 is coming soon and will include these new features: Global Object TTL in eLevelDB LZ4 Compression in eLevelDB Debian 8 and Ubuntu 16 support Switches to disable certain operations in Riak (Key Listing, 2i, Search) HyperLogLog Data Type We have also been working to enhance performance and pay-down technical debt in the product. Some examples of this are: Inconsistent hashing of equivalent objects with AAE Riak Search performance optimizations Upgrading Riak Search's internal Solr version to 4.10.4 As to the future, while we don't provide public roadmaps, some of the themes coming in Riak will be: Advanced data lifecycle management Enhancements to Multi-data Center Replication I sincerely appreciate your feedback. Regards, Charlie Voiselle Product Manager, Riak KV and Clients > On Oct 3, 2016, at 1:34 PM, Ricardo Mayerhofer <ricardo@gmail.com> wrote: > > Hi everyone, > Despite seeing new components being added to Basho data solutions, which is > great (e.g. Riak TS) I wonder about Riak KV evolution. > > Since 2013 there's no major release of Riak KV and not many features were > added since then. Is Riak KV still a priority to Basho? > > Some features that I miss: > TTL per bucket or per key, > Secondary index in Bitcask or TTL in LevelDB. > Be able to drop an entire bucket. > Is there a Roadmap of Riak KV or a list of features coming? > > -- > Ricardo Mayerhofer > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak S2 error
Michael:
If that is the entire contents of your advanced.config file, you will need to
wrap all of that in an erlang list delimiter. []. For example
[
{riak_kv, [
{add_paths, "/usr/lib64/riak-cs/lib/riak_cs-2.1.0/ebin"},
{storage_backend, riak_cs_kv_multi_backend},
{multi_backend_prefix_list, [{<<"0b:">>, be_blocks}]},
{multi_backend_default, be_default},
{multi_backend, [
{be_default, riak_kv_eleveldb_backend, [
{total_leveldb_mem_percent, 30},
{data_root, "/var/lib/riak/leveldb"}
]},
{be_blocks, riak_kv_bitcask_backend, [
{data_root, "/var/lib/riak/bitcask"}
]}
]}
]}
].
Give that a shot and let us know if it works for you!
Thanks,
Charlie Voiselle
> On Jan 7, 2016, at 11:53 AM, Michael Walsh <m...@walshnetwork.com> wrote:
>
> Sure thing.
>
> And thanks!
>
>
> advanced.config:
> {riak_kv, [
> {add_paths, "/usr/lib64/riak-cs/lib/riak_cs-2.1.0/ebin"},
> {storage_backend, riak_cs_kv_multi_backend},
> {multi_backend_prefix_list, [{<<"0b:">>, be_blocks}]},
> {multi_backend_default, be_default},
> {multi_backend, [
> {be_default, riak_kv_eleveldb_backend, [
> {total_leveldb_mem_percent, 30},
> {data_root, "/var/lib/riak/leveldb"}
> ]},
> {be_blocks, riak_kv_bitcask_backend, [
> {data_root, "/var/lib/riak/bitcask"}
> ]}
> ]}
> ]}.
>
> On Wed, Jan 6, 2016 at 5:44 PM, Shunichi Shinohara <sh...@basho.com
> <mailto:sh...@basho.com>> wrote:
> Hi Michael,
>
> Could you provide all the result of "riak config generate -l debug" as well as
> riak.conf and advanced.config?
>
> Thanks,
> Shino
>
> 2016-01-07 9:05 GMT+09:00 Michael Walsh <m...@walshnetwork.com
> <mailto:m...@walshnetwork.com>>:
> > All,
> >
> > I'm trying to set up a RIak S2 instance to integrate with KV and I'm getting
> > the following cuttlefish error out of advanced.config. Any suggestions on
> > where I'm going wrong?
> >
> > notes: erlang error was fixed. All paths are correct.
> >
> > Thanks!
> >
> > -Michael Walsh
> >
> > $riak config generate -l debug
> >
> > ...
> > {riak_kv,
> > [{add_paths,
> >
> > "/usr/lib64/riak-cs/lib/riak_cs-2.1.0/ebin"},
> >{storage_backend,riak_cs_kv_multi_backend},
> >{multi_backend_prefix_list,
> > [{<<"0b:">>,be_blocks}]},
> >{multi_backend_default,be_default},
> >{multi_backend,
> > [{be_default,riak_kv_eleveldb_backend,
> > [{total_leveldb_mem_percent,30},
> >{data_root,"/var/lib/riak/leveldb"}]},
> > {be_blocks,riak_kv_bitcask_backend,
> >
> > [{data_root,"/var/lib/riak/bitcask"}]}]}]}) (lists.erl, line 1247)
> > in function cuttlefish_escript:engage_cuttlefish/1
> > (src/cuttlefish_escript.erl, line 375)
> > in call from cuttlefish_escript:generate/1 (src/cuttlefish_escript.erl,
> > line 258)
> > in call from escript:run/2 (escript.erl, line 747)
> > in call from escript:start/1 (escript.erl, line 277)
> > in call from init:start_it/1
> > in call from init:start_em/1
> >
> > ___
> > riak-users mailing list
> > riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> > <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
> >
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Memory Errors when using RiakCS
Ben:
Awesome! Happy that that got you sorted out.
-cv
> On Nov 24, 2015, at 11:57 AM, Ben Rudolph <brudo...@dimagi.com> wrote:
>
> Right you are:
> 2015-11-23 18:11:19.260 [warning]
> <0.142.0>@riak_cs_app:check_bucket_props:128 Unable to verify moss.access
> bucket settings (disconnected).
> 2015-11-23 18:11:19.261 [warning]
> <0.142.0>@riak_cs_app:check_bucket_props:128 Unable to verify moss.storage
> bucket settings (disconnected).
> 2015-11-23 18:11:19.261 [warning]
> <0.142.0>@riak_cs_app:check_bucket_props:128 Unable to verify moss.buckets
> bucket settings (disconnected).
> 2015-11-23 18:11:19.261 [error] <0.142.0>@riak_cs_app:sanity_check:70 Could
> not verify bucket properties. Error was disconnected.
> 2015-11-23 18:11:19.261 [error] <0.141.0> CRASH REPORT Process <0.141.0> with
> 0 neighbours exited with reason:
> {error_verifying_props,{riak_cs_app,start,[normal,[]]}} in
> application_master:init/4 line 133
> 2015-11-23 18:11:19.262 [info] <0.7.0> Application riak_cs exited with
> reason: {error_verifying_props,{riak_cs_app,start,[normal,[]]}}
>
> Dang, sorry I think I was misled by the `config OK` signal at the beginning
> of the error.
>
> On Tue, Nov 24, 2015 at 11:40 AM, Charlie Voiselle <cvoise...@basho.com
> <mailto:cvoise...@basho.com>> wrote:
> Ben:
>
> It is not uncommon to see the
>> [os_mon] memory supervisor port (memsup): Erlang has closed
> message when the Erlang VM shuts down. Would it be possible for you to
> share your console.log file to the list? There is probably another
> configuration error causing your riak-cs nodes to shut down automatically.
> Misconfigured default bucket types is one such problem that will cause a
> riak-cs node to start and then stop immediately.
>
> Thanks,
> Charlie Voiselle
> Basho, Client Services.
>
>
>> On Nov 23, 2015, at 1:30 PM, Ben Rudolph <brudo...@dimagi.com
>> <mailto:brudo...@dimagi.com>> wrote:
>>
>> Hi,
>>
>> We've been trying to setup RiakCS for storing files on our website,
>> http://commcarehq.org/ <http://commcarehq.org/>. It's been difficult for us
>> to deploy because we've been having issues with our servers not being able
>> to start Riak due to memory errors. Here's what a traceback looks like when
>> I try and start Riak with `riak-console`:
>>
>> config is OK
>> -config /var/lib/riak-cs/generated.configs/app.2015.11.23.18.11.17.config
>> -args_file /var/lib/riak-cs/generated.configs/vm.2015.11.23.18.11.17.args
>> -vm_args /var/lib/riak-cs/generated.configs/vm.2015.11.23.18.11.17.args
>> Exec: /usr/lib/riak-cs/erts-5.10.3/bin/erlexec -boot
>> /usr/lib/riak-cs/releases/2.0.1/riak-cs -config
>> /var/lib/riak-cs/generated.configs/app.2015.11.23.18.11.17.config -args_file
>> /var/lib/riak-cs/generated.configs/vm.2015.11.23.18.11.17.args -vm_args
>> /var/lib/riak-cs/generated.configs/vm.2015.11.23.18.11.17.args
>> -pa /usr/lib/riak-cs/lib/basho-patches -- console
>> Root: /usr/lib/riak-cs
>> Erlang R16B02_basho8 (erts-5.10.3) [source] [64-bit] [smp:8:8]
>> [async-threads:64] [kernel-poll:true] [frame-pointer]
>>
>> =INFO REPORT 23-Nov-2015::18:11:18 ===
>> alarm_handler: {set,{system_memory_high_watermark,[]}}
>> =INFO REPORT 23-Nov-2015::18:11:18 ===
>> opening log file: "/var/log/riak-cs/access.log.2015_11_23_18"
>> Eshell V5.10.3 (abort with ^G)
>> (**)1> [os_mon] memory supervisor port (memsup): Erlang has closed
>>
>> The machines we are trying to deploy to have 8GB of RAM and also running a
>> few other services, but have 3-4GB of RAM available. Are there any
>> configuration params we can use so it consumes less memory? I've taken a
>> look at the `system_memory_high_watermark` which is the error we see
>> (http://docs.basho.com/riak/latest/ops/running/recovery/errors/
>> <http://docs.basho.com/riak/latest/ops/running/recovery/errors/>). Our
>> configs can be found here in case that's useful:
>>
>> https://github.com/dimagi/commcarehq-ansible/tree/riak/ansible/roles/riakcs/templates
>>
>> <https://github.com/dimagi/commcarehq-ansible/tree/riak/ansible/roles/riakcs/templates>
>>
>> Any insight would be useful! Thank you!
>>
>> Ben
>>
>>
>>
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
>
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak doesn't start anymore after server crash - no errors logged
Another thing that I have seen cause riak_kv to stall like this is LevelDB
corruption preventing vnodes from starting properly. I use this command to
look for LOG files with the word error in them. It is also worth checking the
anti_entropy levelDB LOG files as well.
find . -name "LOG" -exec grep -l -i 'error' {} \;
-cv
> On Nov 11, 2015, at 7:05 PM, Steven Sylvester wrote:
>
> Hi Markus,
>
> One thing that stands out in what you pasted is the following line:
>
> 2015-11-09 11:31:45.263 [info] <0.73.0> alarm_handler:
> {set,{{disk_almost_full,"/"},[]}}
>
> Check the output of `df` and add nodes/disk as necessary to keep your disk
> usage at a manageable level. Our docs recommend keeping disk use below 60%:
> http://docs.basho.com/riak/latest/ops/building/planning/cluster/#Disk
>
>
> Disk capacity issues can sneak up on you due to some backend operations that
> require a lot of space all at once, such as Bitcask merging or AAE. *Bad
> things* tend to happen to a Riak cluster with full disks, including data
> loss/corruption.
>
> Other than that, are there any changes that have been made to the environment
> lately? Have you performed any upgrades recently?
>
> Check the output of `ps aux` and kill any processes related to Riak, such as
> any instances of `beam.smp` before attempting to start the node again.
>
> If the node won't start after these troubleshooting steps, run `riak console`
> which will attempt to start the node and put you into an Erlang attach
> shell. Monitor the messages that are printed to stdout and if/when it seems
> to hang, paste the output of the whole thing here and we can take a look.
>
> --
> Steven Sylvester, CISSP
> Client Services Engineer
> Basho Technologies
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Unable to start riak in ubuntu after changing IP address
Sambit: If you change the `nodename` on a node after starting it, you will create a condition where the node will no longer be able to find its name in the ring at startup and the startup will fail. If this is a single node cluster, you can just wipe the contents of `/var/lib/riak/ring` and a new ring will be generated. If you need to do this for multiple nodes, I would encourage you to review http://docs.basho.com/riak/latest/ops/running/nodes/renaming/ http://docs.basho.com/riak/latest/ops/running/nodes/renaming/ which is fairly exhaustive on the subject of renaming nodes. Hope this gets you back up and going, Charlie Voiselle Client Services, Basho On May 25, 2015, at 11:43 AM, Vitaly E 13vitam...@gmail.com wrote: Which version of Riak are you using? Do you see any error messages? Also, what Java program are you reffering to? Can you post a link please? Regards, Vitaly On Fri, May 22, 2015 at 9:29 AM, sambit sambitte...@gmail.com mailto:sambitte...@gmail.com wrote: Hi I have successfully installed Riak in ubuntu in my vmware virtual box in windows 7. It was working fine when I was applying some command from terminal. I want to use Riak from my Eclipse from Windows environment. I am using the java program provided by Riak site. I changed the riak IP address, after changing I am unable to start Riak. Please help me in this regard. I have made the following changes in riak.conf file. nodename = riak@192.168.17.11 mailto:riak@192.168.17.11 listener.http.internal = 192.168.17.11:8098 http://192.168.17.11:8098/ I refered to the following sites. http://stackoverflow.com/questions/13859906/riak-node-no-longer-working-after-changing-ip-address http://stackoverflow.com/questions/13859906/riak-node-no-longer-working-after-changing-ip-address http://docs.basho.com/riak/1.2.0/references/Command-Line-Tools---riak-admin/#reip http://docs.basho.com/riak/1.2.0/references/Command-Line-Tools---riak-admin/#reip Please help to start Riak and suggest me how to run the built-in java program from Eclipse in windows 7. -- View this message in context: http://riak-users.197444.n3.nabble.com/Unable-to-start-riak-in-ubuntu-after-changing-IP-address-tp4033065.html http://riak-users.197444.n3.nabble.com/Unable-to-start-riak-in-ubuntu-after-changing-IP-address-tp4033065.html Sent from the Riak Users mailing list archive at Nabble.com. ___ riak-users mailing list riak-users@lists.basho.com mailto:riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: uneven disk distribution
Johnny: Something else to look for would be any errors in the console.log related to Bitcask merging. It would be interesting to see if the unusual disk utilization was related to a specific partition. If it is, you could consider removing that particular partition and running riak_kv:repair to restore the replicas from the adjacent partitions. I can provide more information if you find that to be the case. Regards, Charlie Voiselle Client Services, Basho On May 14, 2015 10:06 PM, Engel Sanchez en...@basho.com mailto:en...@basho.com wrote: Hi Johnny. Make sure that the configuration on that node is not different to the others. For example, it could be configured to never merge Bitcask files, so that space could never be reclaimed. http://docs.basho.com/riak/latest/ops/advanced/backends/bitcask/#Configuring-Bitcask http://docs.basho.com/riak/latest/ops/advanced/backends/bitcask/#Configuring-Bitcask On Thu, May 14, 2015 at 4:31 PM, Johnny Tan johnnyd...@gmail.com mailto:johnnyd...@gmail.com wrote: We have a 6-node test riak cluster. One of the nodes seems to be using far more disk: staging-riak001.pp /dev/sda3 15G 6.3G 7.2G 47% / staging-riak002.pp /dev/sda3 15G 6.4G 7.1G 48% / staging-riak003.pp /dev/sda3 15G 6.1G 7.5G 45% / staging-riak004.pp /dev/sda3 15G 14G 266M 99% / staging-riak005.pp /dev/sda3 15G 5.8G 7.7G 44% / staging-riak006.pp /dev/sda3 15G 6.3G 7.3G 47% / Specifically, /var/lib/riak/bitcask is using up most of that space. It seems to have files in there that are much older than any of the other nodes. We've done maintenance of various sort on this cluster -- as the name indicates, we use it as a staging ground before we go to production. I don't recall a specific issue per se, but I wouldn't rule it out. Is there a way to figure out if there's an underlying issue here, or whether some of this disk space is not really current and can somehow be purged? What info would help answer those questions? johnny ___ riak-users mailing list riak-users@lists.basho.com mailto:riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com mailto:riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: No WM route
This error is usually caused when `admin.listener` is configured in
riak-cs.conf. Either remove that line from the configuration file, or use
that IP and port for your create user request.
Please let me know if that helps you out
Charlie Voiselle
Basho Client Services
On Apr 23, 2015 8:46 AM, Amit Anand aan...@viimed.com wrote:
Hi all Im very new to Riak CS so please bear with me! Finally managed to
get Riak, Riak CS and Stanchion all running and now Im trying to add the
admin user. When I run my curl command I get nothing back and in the logs I
see this error:
2015-04-23 11:37:18.024 [error] 0.94.0 No WM route: 'POST' /riak-cs/user
{7,{user-agent,{'User-Agent',curl/7.29.0},{host,{'Host',
10.7.2.113:8080
},{accept,{'Accept',*/*},nil,{content-type,{'Content-Type',application/json},{content-length,{'Content-Length',45},nil,nil},nil}},nil},{x-rcs-rewrite-path,{x-rcs-rewrite-path,/riak-cs/user},{x-rcs-raw-url,{x-rcs-raw-url,/riak-cs/user},nil,nil},nil}}}
I have set anonymous user to on in the riak-cs.conf AS WELL as to true in
the advanced.config. I tried with just in .conf and that didnt work either.
Would anybody have any ideas on how to get this work I would really
appreciate it. Thanks and sorry if Im asking a basic question I just cant
figure it out.
Im running CentOS 7 and:
[root@riakcs riak-cs]# riak version
2.1.0
[root@riakcs riak-cs]# riak-cs version
2.0.0
[root@riakcs riak-cs]# stanchion version
2.0.0
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: cannot connect to DB
Since you changed the node name on a running node, you will not be able to stop it. Two choices here. Reset the nodename in `/etc/riak/vm.args` back to `riak@127.0.0.1 mailto:riak@127.0.0.1` long enough to run the `riak stop` command, or just `kill -9 beam.smp` which will forcibly shutdown the riak process. Once you have the node stopped, you will need to delete the contents of the `/var/lib/riak/ring` folder otherwise the node will not start up. This only needs to be done once if the node has been renamed. Once your cluster has been created properly, you will not want to delete the ring file without specific guidance to do so. Verify that the node name is back to `riak@192.168.1.55 mailto:riak@192.168.1.55` in `/etc/riak/vm.args` and that all of the IPs in `/etc/riak/app.config` are set appropriately before restarting Riak Once you have verified this, you should be able to start Riak and connect to it from other nodes. As Siculars, pointed out, there is fairly comprehensive documentation of the config files, their locations, and all of the settings at the Riak Documentation website. Make sure you use the version selector to view the documentation for the version that you have installed Cheers, Charlie Voiselle Basho Client Services On Jan 12, 2015, at 8:58 AM, Ildar Alishev ildaralis...@gmail.com wrote: i have changed nodename and now it says root@salo:~# riak stop Node 'riak@192.168.1.55 mailto:riak@192.168.1.55' not responding to pings. Node is not running! root@salo:~# 12 янв. 2015 г., в 16:53, Ciprian Manea cipr...@basho.com mailto:cipr...@basho.com написал(а): Hi Ildar, Please have a look at the configuration files: /etc/riak/app.config and /etc/riak/vm.config By default Riak binds to localhost, but you can change that using the following snippet: export riakIP=$(ifconfig eth0 | grep 'inet addr' | cut -d: -f2 | cut -d' ' -f1) sudo sed -i s/127.0.0.1/$riakIP/ http://127.0.0.1/$riakIP/ /etc/riak/app.config /etc/riak/vm.args Regards, Ciprian On Mon, Jan 12, 2015 at 3:51 PM, Ildar Alishev ildaralis...@gmail.com mailto:ildaralis...@gmail.com wrote: Also maybe someone can help me, i have installed riak using apt-get install riak, i don’t understand where did it installed? How i can create node there? Thank you! 12 янв. 2015 г., в 16:37, Alexander Sicular sicul...@gmail.com mailto:sicul...@gmail.com написал(а): Hi Ildar, Please take a look at the docs, http://docs.basho.com/riak/latest/ops/building/basic-cluster-setup/ http://docs.basho.com/riak/latest/ops/building/basic-cluster-setup/ , you need to set up your IP address most likely. -Alexander @siculars http://siculars.posthaven.com http://siculars.posthaven.com/ Sent from my iRotaryPhone On Jan 12, 2015, at 06:56, Ildar Alishev ildaralis...@gmail.com mailto:ildaralis...@gmail.com wrote: Hello. Have started RIAK in UBUNTU64 14.04, trying to connect from another computer inside local network, but cannot connect. Maybe i should set up ipadress somewhere? ___ riak-users mailing list riak-users@lists.basho.com mailto:riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com mailto:riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: invalid hintfiles after upgrde
Invalid hint files will be rebuilt automatically during the Bitcask merge process. I believe you are seeing a one-time consequence of switching to signed hint files during the upgrade. Not to worry, they will be regenerated through normal merge activity. Regards, -cv On Jan 17, 2014 6:56 AM, Jan Zahradka - MALL.cz jan.zahra...@mall.cz wrote: Hi everybody, I have a 3 node Riak 1.1.1 cluster running on Centos 6.3. I wanted to upgrade it to the most recent version and I was told that first I have to upgrade to version 1.3.2 and then I can upgrade to 1.4.X version. So I followed the instructions in Rolling upgrades guide (stop, upgrade package, start, status, wait-for-service riak_kv). After upgrading the package (1.1.1 - 1.3.2) and starting new version I could see messages like this in console.log : 2014-01-17 12:58:31.638 [error] 0.658.0 Hintfile '/var/lib/riak/bitcask/ 707914855582156101004909840846949587645842325504/26.bitcask.hint' invalid 2014-01-17 12:58:31.974 [error] 0.662.0 Hintfile '/var/lib/riak/bitcask/ 981946412581700398168100746981252653831329677312/20.bitcask.hint' invalid 2014-01-17 12:58:32.043 [error] 0.661.0 Hintfile '/var/lib/riak/bitcask/ 890602560248518965780370444936484965102833893376/20.bitcask.hint' invalid 2014-01-17 12:58:32.064 [error] 0.663.0 Hintfile '/var/lib/riak/bitcask/ 1004782375664995756265033322492444576013453623296/22.bitcask.hint' invalid etc etc. I upgraded the whole cluster and it is functional. I read that hinfiles are not fatal for running, but I would like to have it clear. Does anybody have any advice? Is it possible to rebuild the hintfiles? i tried to google it but no result ;o( (or maybe I'm not as googler as I thought ;o)) Thank You very much Jan ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: anti_entropy_expire
Edgar:
Could you attach the AAE section of your app.config? I’d like to look into
this issue further for you. Something I think you might be running into is
https://github.com/basho/riak_core/pull/483.
The issue of concern is that the LevelDB bloom filter is not enabled properly
for the instance into which the AAE data is stored. You can mitigate this
particular issue by adding {use_bloomfilter, true} as shown below:
%% The LevelDB options used by AAE to generate the LevelDB-backed
%% on-disk hashtrees.
{anti_entropy_leveldb_opts, [{write_buffer_size, 4194304},
{max_open_files, 20}]},
Becomes:
%% The LevelDB options used by AAE to generate the LevelDB-backed
%% on-disk hashtrees.
{anti_entropy_leveldb_opts, [{write_buffer_size, 4194304},
{use_bloomfilter, true},
{max_open_files, 20}]},
This might not solve your specific problem, but it will certainly improve your
AAE performance.
Thanks,
Charlie Voiselle
On Dec 31, 2013, at 12:04 PM, Edgar Veiga edgarmve...@gmail.com wrote:
Hey guys!
Nothing on this one?
Btw: Happy new year :)
On 27 December 2013 22:35, Edgar Veiga edgarmve...@gmail.com wrote:
This is a du -hs * of the riak folder:
44G anti_entropy
1.1M kv_vnode
252G leveldb
124K ring
It's a 6 machine cluster, so ~1512G of levelDB.
Thanks for the tip, I'll upgrade in a near future!
Best regards
On 27 December 2013 21:41, Matthew Von-Maszewski matth...@basho.com wrote:
I have a query out to the developer that can better respond to your follow-up
questions. It might be Monday before we get a reply due to the holidays.
Do you happen to know how much data is in the leveldb dataset and/or one
vnode? Not sure it will change the response, but might be nice to have that
info available.
Matthew
P.S. Unrelated to your question: Riak 1.4.4 is available for download. It
has a couple of nice bug fixes for leveldb.
On Dec 27, 2013, at 2:08 PM, Edgar Veiga edgarmve...@gmail.com wrote:
Ok, thanks for confirming!
Is it normal, that this action affects the overall state of the cluster? On
the 26th It started the regeneration and the the response times of the
cluster raised to never seen values. It was a day of heavy traffic but
everything was going quite ok until it started the regeneration process..
Have you got any advices about changing those app.config values? My cluster
is running smoothly for the past 6 months and I don't want to start all over
again :)
Best Regards
On 27 December 2013 18:56, Matthew Von-Maszewski matth...@basho.com wrote:
Yes. Confirmed.
There are options available in app.config to control how often this occurs
and how many vnodes rehash at once: defaults are every 7 days and two
vnodes per server at a time.
Matthew Von-Maszewski
On Dec 27, 2013, at 13:50, Edgar Veiga edgarmve...@gmail.com wrote:
Hi!
I've been trying to find what may be the cause of this.
Every once in a week, all the nodes in my riak cluster start to do some
kind of operation that lasts at least for two days.
You can watch a sample of my munin logs regarding the last week in here:
https://cloudup.com/imWiBwaC6fm
Take a look at the days 19 and 20, and now it has started again on the 26...
I'm suspecting that this may be caused by the aae hash trees being
regenerated, as you say in your documentation:
For added protection, Riak periodically (default: once a week) clears and
regenerates all hash trees from the on-disk K/V data.
Can you confirm me that this may be the root of the problem and if it's
normal for the action to last for two days?
I'm using riak 1.4.2 on 6 machines, with centOS. The backend is levelDB.
Best Regards,
Edgar Veiga
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: different versions in upgrade
Louis-Philippe et al: You can follow the rolling upgrade procedure to upgrade a node from 1.2 to 1.4.x directly. The note in the instructions only concerns upgrading from 1.0 to 1.4. No need to stop at 1.3.2. Thanks, Charlie Voiselle On Aug 13, 2013, at 9:23 AM, Guido Medina guido.med...@temetra.com wrote: Also, in theory if you have at least 5 nodes in the cluster one node down at a time doesn't stop your cluster from working properly. You could do the following node by node which I have done several times: Stop Riak on the upgrading node and in another node mark the upgrading node as down (riak-admin down riak@upgrading-node) Upgrade Riak on that node to version 1.3.2, start it up and wait till it is completely operative (riak_kv is up and all transfers are finished, check by typing riak-admin transfers) For Riak 1.3.2 once the step is over, type riak-admin reformat-indexes and tail -f /var/log/riak/console.log which should be done really fast if there isn't anything to fix. Do 1 to 3 per node. Do 1 and 2 but for for Riak 1.4.1. HTH, Guido. On 13/08/13 13:50, Guido Medina wrote: Same here, except that Riak 1.3.2 did that for me automatically. As Jeremiah mentioned, you should go first to 1.3.2 on all nodes, per node the first time Riak starts it will take some time upgrading the 2i indexes storage format, if you see any weirdness then execute riak-admin reformat-indexes as soon as you upgrade a node, 1 by 1. Before you even start read the release notes for each major version besides the rolling upgrade doc: Riak 1.3.2: https://github.com/basho/riak/blob/1.3/RELEASE-NOTES.md Riak 1.4.1: https://github.com/basho/riak/blob/1.4/RELEASE-NOTES.md HTH, Guido. On 13/08/13 13:41, Bhuwan Chawla wrote: Having done a similar upgrade, a gotcha to keep in mind: Note for Secondary Index users If you use Riak's Secondary Indexes and are upgrading from a version prior to Riak version 1.3.1, you need to reformat the indexes using the riak-admin reformat-indexes command On Tue, Aug 13, 2013 at 8:36 AM, Jeremiah Peschka jeremiah.pesc...@gmail.com wrote: From http://docs.basho.com/riak/latest/ops/running/rolling-upgrades/ it looks like you should upgrade to 1.3.2 and then 1.4.1 Depending on how badly you need the extra capacity, it would probably be better to start by upgrading all nodes and then adding the new one. -- Jeremiah Peschka - Founder, Brent Ozar Unlimited MCITP: SQL Server 2008, MVP Cloudera Certified Developer for Apache Hadoop On Aug 13, 2013, at 5:06 AM, Louis-Philippe Perron lpper...@gmail.com wrote: Hi riak peoples, I'm in the process of adding a new node to a aging (1 node) cluster. I would like to know what would be the prefered incrementing upgrade to get all my nodes on the latest riak version. The best scenario would also have the least downtime. The old node is at riak version 1.2.1. My actual plan is: - install riak 1.4.1 on the new node - add the new 1.4.1 node to the old 1.2.1 cluster. - bring the 1.2.1 node offline - upgrade the 1.2.1 node to 1.4.1 - put the upgraded node back online will this work? thanks! ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Utility functions
Kevin:
As mentioned in the Riak MapReduce documentation, you can find more information
about the pre-built MR jobs here:
https://github.com/basho/riak_kv/blob/master/src/riak_kv_mapreduce.erl --
Erlang Module that contains all of the pre-built Erlang MapReduce Phases
https://github.com/basho/riak_kv/blob/master/priv/mapred_builtins.js --
JavaScript source of the pre-built JavaScript MapReduce phases
Hope this helps,
Charle Voiselle
On Feb 8, 2013, at 10:41 AM, rkevinbur...@charter.net wrote:
I was looking at CorrugatedIrons website and I see code like:
var query = new RiakMapReduceQuery()
.Inputs(BucketName)
.MapJs(m = m.Source(@function(o){return[1];}))
.ReduceJs(m = m.Name(@Riak.reduceSum).Keep(true));
I am assuming that functions like Riak.reduceSum are general utility
functions that are part of the Riak distro. Is there a list of these
somewhere documenting their use and purpose? There seems to be both
JavaScript and Erlang methods. I would be interested in both. Thank you.
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: about hash calculation in reference page at docs.basho.com
Donald:
Since the list of partitions can easily be segmented using a filter on
partition values
that are greater than the hashed value, it makes for a simple determination of
the ordered
preflist.
For Example: Hash = 1045375627425331784151332358177649483819648417632
Parition| P H | List |
+---+---+
0 | False | B |
182687704666362864775460604089535377456991567872| False | B |
365375409332725729550921208179070754913983135744| False | B |
548063113999088594326381812268606132370974703616| False | B |
730750818665451459101842416358141509827966271488| False | B |
913438523331814323877303020447676887284957839360| False | B |
1096126227998177188652763624537212264741949407232 | True | A |
1278813932664540053428224228626747642198940975104 | True | A |
+---+---+
Then so long as order is maintained, your sorted preflist is simply A ++ B.
It would be a (if only slightly) more complex algorithm, requiring
determination by
lookahead if you were in the correct partition, in order to have the partitions
names be
starting values.
To answer your second question, the partitions are named by dividing the
integer keyspace
by the num partitions to create an incrementor[1]. This incrementor is used as
the
stepping factor in a sequence from 0 to RINGTOP value --
`trunc(math:pow(2,160)-1)`.
You can call these functions yourself in the `riak attach` as follows:
(dev1@127.0.0.1)1 Inc = chash:ring_increment(8).
182687704666362864775460604089535377456991567872
(dev1@127.0.0.1)2 [{IndexAsInt} || IndexAsInt -lists:seq(0,( trunc(
math:pow(2,160) -1)
-1),Inc)].
[{0},
{182687704666362864775460604089535377456991567872},
{365375409332725729550921208179070754913983135744},
{548063113999088594326381812268606132370974703616},
{730750818665451459101842416358141509827966271488},
{913438523331814323877303020447676887284957839360},
{1096126227998177188652763624537212264741949407232},
{1278813932664540053428224228626747642198940975104}]}
Hope this helps!
Charlie Voiselle
---
[0] https://github.com/basho/riak_core/blob/master/src/chash.erl#L88
[1] https://github.com/basho/riak_core/blob/master/src/chash.erl#L162
On Jan 29, 2013, at 9:47 AM, Donald Yan dyan@gmail.com wrote:
Hi Charlie,
That really helps to clear the confusion. Thanks again!
Then it comes to the next question :-) Just curious to know: is there a
special reason why using the end value as the partition's name? Normally,
most people I think would pick the beginning value as the partition's name in
common logic.
And a second question related to the above question is how this end value is
picked: partition named 0 is actually ending with number 0, and so forth
for the other partitions. This is like the 160-bit space got right-shifted
one number relative to 0-started numbering system. Is there a special reason
for it in Riak, or simply want to use 1-started numbering system.
wr, donald
On Mon, Jan 28, 2013 at 3:21 PM, Charlie Voiselle cvoise...@basho.com wrote:
Donald:
Since we are traversing the ring in a clockwise fashion, the data is written
into the next partition you would run into moving forward. You can see how
the partitions are chosen in chash:ordered_from/2 at
https://github.com/basho/riak_core/blob/master/src/chash.erl#L138. The
preference list is built using the partitions that come after the hashed
value.
If you consider the partition's name to be the end value that the partition
holds, the an object that hashes to
1045375627425331784151332358177649483819648417632 being assigned to
1096126227998177188652763624537212264741949407232 makes sense. That partition
would hold items that hash from
913438523331814323877303020447676887284957839361 to
1096126227998177188652763624537212264741949407232
Does this help?
Thanks,
Charlie Voiselle
On Jan 24, 2013, at 12:05 AM, Donald Yan dyan@gmail.com wrote:
Hi there,
I'm reading this page at
http://docs.basho.com/riak/latest/references/appendices/concepts/Replication/#Understanding-replication-by-example
and have a question about the example given in the page. I quote it here:
The DocIdx hash is a 160 bit integer:
(dev1@127.0.0.1)5 I:160/integer = DocIdx.
183,28,67,173,80,128,26,94,190,198,65,15,27,243,135,127,121,101,255,96
(dev1@127.0.0.1)6 I.
1045375627425331784151332358177649483819648417632
The node looks up the hashed key in the ring which returns a list of
“preferred” partitions for the given key.
(node1@127.0.0.1) Preflist = riak_core_ring:preflist(DocIdx, Ring).
[{1096126227998177188652763624537212264741949407232, 'dev1@127.0.0.1'},
{1278813932664540053428224228626747642198940975104, 'dev2@127.0.0.1'},
{0, 'dev1
Re: Riak startup failing.
Kevin: First things first. What version of Riak are you running? The `riak-admin cluster` commands are available as of Riak 1.2.0. Next, if you have already renamed all of the nodes, you will need to use the `reip` method to modify all of the ring files on each of the nodes. If you are able, it's easier to roll back the changes to the configuration and get the nodes starting again. Once they are started and running, here's a writeup of the scheme you would use to do the rolling configuration changes: https://gist.github.com/4566736 Thanks, Charlie Voiselle On Dec 29, 2012, at 1:02 PM, Kevin Burton rkevinbur...@charter.net wrote: Here is what I found out (I am still stuck) https://github.com/basho/basho_docs/issues/28 The step says to rename or remove the data/ring directory. My data directory is /var/lib/riak. If I rename or remove that directory then no commands work. I have assumed it meant to rename or remove just the ring directory which is for me /var/lib/riak/ring. When the node is restarted the ring directory is recreated. When I issue riak-admin down (I have to specify the node name since it is required) I get errors/warnings: ~$ sudo riak-admin down r...@xxx.xxx.xxx.xxx Attempting to restart script through sudo -H -u riak Node is not running! I am not sure what it means to join the existing cluster on the first node. Is this riak-admin cluster join? For the first node I get an error ~$ sudo riak-admin cluster join r...@xxx.xxx.xxx.xxx Attempting to restart script through sudo -H -u riak Join failed. Try again in a few moments. There is no ‘force-replace’ argument to riak-admin and that command seems to be deprecated. There is a force-replace for riak-admin cluster but it takes two nodes as arguments which I am not sure what to put there. What should I use? These steps seem to get me further but still there are gaps (at least for me). ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak and host names
Deepak: When you name a node in app.config with -name it has to have a '.' in it, like r...@hostname.net As you have surmised, you can get around that if you use the -sname argument instead. They have to be done consistently. In your example, had you used the -sname argument, `riak@riaknode1` would work. Making a host entry `riaknode1.priv` that points to the local address would work with the -name argument. The inportant thing about -name and -sname is that they can't mix within a cluster. Cluster replace is designed to replace a node with a new one and transfer all the partitions. You can cheat and use it to rename a node though. The process to do this would look like the following: Stop the node to rename with `riak stop` Mark it 'down' from another node in the cluster using `riak-admin down «old nodename». Rename the node in vm.args. Delete the ring directory. Start the node with `riak start`. It will come up as a single instance which you can verify with `riak-admin member-status`. Join the node to the cluster with `riak-admin cluster join «cluster nodename» ` Set it to replace the old instance of itself with `riak-admin cluster replace «old nodename» «new nodename» Plan the changes with `riak-admin cluster plan` Commit the changes with `riak-admin cluster commit` As you can see, this is a very large effort, so best to use hostnames that aren't moving around. Apologies for you getting this twice, Deepak. I failed to reply to the list as well. Hope this makes sense... Charlie On Jan 1, 2013, at 2:43 PM, Deepak Balasubramanyam deepak.b...@gmail.com wrote: I took the AWS EC2 riak image for a spin today. I have a query regarding riak nodes and how they behave when the machine reboots. When an EC2 instance reboots, the internal ip / internal DNS / external DNS change. This renders the app.config and -name argument on vm.args incorrect. I was exploring solutions to deal with this problem. 1. Preventive measures Someone on this thread dated May 2011 suggested using host file entries that point to the local internal IP address. That does not seem to work. Riak fails with the following error when I add a new entry to /etc/hosts and configure vm.args with -name riak@riaknode1 Hostname riaknode1 is illegal I confirmed that riaknode1 pings correctly before starting riak. I guess erlang tries to match the hostname of the system resulting in this failure ? Can anyone throw some light on this ? 2. Use -sname Is starting the erlang VM with the sname flag an option if it will help prevent the 'illegal hostname' error ? Disclaimer: My knowledge of erlang is close to zilch, so sorry if that option sounded like something you could dismiss easily :) 3. Use cluster replace a. I understand that the IPs in app.config and vm.args can be replaced with the correct IP on a restart and using a subsequent 'cluster replace' command will do. Will executing the 'cluster plan' and 'cluster commit' commands now produce network chatter ? b . What happens if 2 nodes go down and one was joined with the other. They both have 2 different IP addresses on restart. How will 'cluster replace' work now ? Do let me know your thoughts. Thanks -Deepak ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak and host names
Matt: You would need to use (or implement your own) DNS service that you could programmatically access--Route 53 has an API that you could use to create DNS entries that point to the internal addresses of your nodes. In very carefully re-reading the thread Deepak mentions, one problem that will occur is that each node needs to be able to resolve the other nodes by name also. The only way for this to occur reasonably, would be to register the internal addresses with a single point that they share. Some examples of free services that you might use for this are DynDns[1], DNSDynamic[2], or DNS-O-Matic[3]. I have also seen some projects floating around the web that might enable you to create a self-hosted dynamic DNS like opendyn[4] and GnuDIP[5]; however, I have had no occasion to use something like this in my own environment. Some additional discussion about creating your own Dynamic DNS server is also at http://unix.stackexchange.com/questions/29049/how-to-create-a-custom-dynamic-dns-solution Hope this helps! Charlie [1] http://www.dyn.com [2] http://www.dnsdynamic.org [3] http://www.dnsomatic.com [4] http://code.google.com/p/opendyn/ [5] http://gnudip2.sourceforge.net/ On Jan 7, 2013, at 5:00 PM, Matt Black matt.bl...@jbadigital.com wrote: Thanks for this Charlie. I'm running a production Riak cluster on AWS which runs constantly, and I've been wondering how I might be able to easliy stop and start AWS nodes for a testing and benchmarking cluster (to save on cost). By using the 'riaknode1.priv' hostname method you describe, would I be able to stop and then restart a whole cluster of nodes at once? (As described by Deepak, AWS assigns new IPs when a VM starts). Thanks Matt On 8 January 2013 01:31, Charlie Voiselle cvoise...@basho.com wrote: Deepak: When you name a node in app.config with -name it has to have a '.' in it, like r...@hostname.net As you have surmised, you can get around that if you use the -sname argument instead. They have to be done consistently. In your example, had you used the -sname argument, `riak@riaknode1` would work. Making a host entry `riaknode1.priv` that points to the local address would work with the -name argument. The inportant thing about -name and -sname is that they can't mix within a cluster. Cluster replace is designed to replace a node with a new one and transfer all the partitions. You can cheat and use it to rename a node though. The process to do this would look like the following: Stop the node to rename with `riak stop` Mark it 'down' from another node in the cluster using `riak-admin down «old nodename». Rename the node in vm.args. Delete the ring directory. Start the node with `riak start`. It will come up as a single instance which you can verify with `riak-admin member-status`. Join the node to the cluster with `riak-admin cluster join «cluster nodename» ` Set it to replace the old instance of itself with `riak-admin cluster replace «old nodename» «new nodename» Plan the changes with `riak-admin cluster plan` Commit the changes with `riak-admin cluster commit` As you can see, this is a very large effort, so best to use hostnames that aren't moving around. Apologies for you getting this twice, Deepak. I failed to reply to the list as well. Hope this makes sense... Charlie On Jan 1, 2013, at 2:43 PM, Deepak Balasubramanyam deepak.b...@gmail.com wrote: I took the AWS EC2 riak image for a spin today. I have a query regarding riak nodes and how they behave when the machine reboots. When an EC2 instance reboots, the internal ip / internal DNS / external DNS change. This renders the app.config and -name argument on vm.args incorrect. I was exploring solutions to deal with this problem. 1. Preventive measures Someone on this thread dated May 2011 suggested using host file entries that point to the local internal IP address. That does not seem to work. Riak fails with the following error when I add a new entry to /etc/hosts and configure vm.args with -name riak@riaknode1 Hostname riaknode1 is illegal I confirmed that riaknode1 pings correctly before starting riak. I guess erlang tries to match the hostname of the system resulting in this failure ? Can anyone throw some light on this ? 2. Use -sname Is starting the erlang VM with the sname flag an option if it will help prevent the 'illegal hostname' error ? Disclaimer: My knowledge of erlang is close to zilch, so sorry if that option sounded like something you could dismiss easily :) 3. Use cluster replace a. I understand that the IPs in app.config and vm.args can be replaced with the correct IP on a restart and using a subsequent 'cluster replace' command will do. Will executing the 'cluster plan' and 'cluster commit' commands now produce network chatter ? b . What happens if 2 nodes go down and one was joined with the other
