Re: [DRBD-user] building v9

[Yannis Milios]

Thanks Christoph for the very clear explanation, I think that it was a
piece which was missing for all of us!

B.R.
Yannis

On Thu, 10 Dec 2020 at 12:07, Christoph Böhmwalder <
christoph.boehmwal...@linbit.com> wrote:

> Hi Pierre,
>
> As much as we may want it, DRBD's coccinelle-based compat system is not
> a general purpose solution. We can't guarantee that DRBD will build for
> any given kernel – there is simply too much going on in the block layer
> and other parts of the kernel, so that we cannnot possibly cover for all
> those different combinations (and still expect DRBD to work as intended).
>
> So we have made a bit of a compromise: we build and test DRBD for a
> defined set of kernels. These are deemed "most interesting", according
> to the opinion of LINBIT and our customers. Namely, we currently build
> for these 125 kernels on the amd64 architecture at time of writing:
>
> Distribution  | Kernel Version
>   | --
> amazonlinux2.0-amd64  | 4.14.128-112.105.amzn2
> debian-buster-amd64   | 4.19.0-5; 4.19.0-6; 4.19.0-8
> debian-jessie-amd64   | 3.16.0-4; 3.16.0-5; 3.16.0-6
> debian-stretch-amd64  | 4.9.0-6; 4.9.0-7; 4.9.0-8; 4.9.0-9; 4.9.0-11
> oracle6.0-amd64   | 4.1.12-124.26.3.el6uek; 4.1.12-124.21.1.el6uek
> oracle7.0-amd64   | 4.14.35-1844.1.3.el7uek; 4.1.12-94.3.9.el7uek;
> 4.1.12-124.26.10.el7uek; 4.14.35-1902.4.8.el7uek; 4.14.35-1818.3.3.el7uek
> oracle8.0-amd64   | 5.4.17-2011.0.7.el8uek
> rhel6.10-amd64| 2.6.32-754.el6
> rhel6.6-amd64 | 2.6.32-504.el6
> rhel6.7-amd64 | 2.6.32-573.1.1.el6
> rhel6.8-amd64 | 2.6.32-642.1.1.el6
> rhel6.9-amd64 | 2.6.32-696.el6; 2.6.32-696.23.1.el6;
> 2.6.32-696.30.1.el6
> rhel7-xen-amd64   | 4.9.188-35.el7; 4.9.199-35.el7; 4.9.206-36.el7;
> 4.9.212-36.el7; 4.9.215-36.el7
> rhel7.0-amd64 | 3.10.0-123.20.1.el7
> rhel7.1-amd64 | 3.10.0-229.1.2.el7
> rhel7.2-amd64 | 3.10.0-327.el7
> rhel7.3-amd64 | 3.10.0-514.6.2.el7; 3.10.0-514.36.5.el7
> rhel7.4-amd64 | 3.10.0-693.el7; 3.10.0-693.17.1.el7;
> 3.10.0-693.21.1.el7
> rhel7.5-amd64 | 3.10.0-862.el7
> rhel7.6-amd64 | 3.10.0-957.el7
> rhel7.7-amd64 | 3.10.0-1049.el7; 3.10.0-1062.el7
> rhel7.8-amd64 | 3.10.0-1127.el7
> rhel7.9-amd64 | 3.10.0-1160.el7
> rhel8.0-amd64 | 4.18.0-80.1.2.el8_0
> rhel8.1-amd64 | 4.18.0-147.el8
> rhel8.2-amd64 | 4.18.0-193.el8
> rhel8.3-amd64 | 4.18.0-240.1.1.el8_3
> sles11-sp4-amd64  | 3.0.101-108.13.1
> sles12-sp2-amd64  | 4.4.74-92.38.1
> sles12-sp3-amd64  | 4.4.92-6.30.1
> sles12-sp4-amd64  | 4.12.14-95.3.1
> sles12-sp5-amd64  | 4.12.14-120.1
> sles15-sp0-amd64  | 4.12.14-25.25.1
> sles15-sp1-amd64  | 4.12.14-197.29
> sles15-sp2-amd64  | 5.3.18-22.2
> ubuntu-bionic-amd64   | ✗ 5.3.0-1034-aws; ✗ 5.3.0-1035-aws;
> 5.4.0-1025-aws; 5.4.0-1028-aws; 5.4.0-1029-aws; 5.4.0-1030-aws;
> 4.15.0-1007-aws
> ubuntu-bionic-amd64   | ✗ 5.3.0-1035-azure; ✗ 5.4.0-1023-azure;
> 5.4.0-1025-azure; 5.4.0-1026-azure; 5.4.0-1031-azure; 5.4.0-1032-azure;
> 4.15.0-1009-azure
> ubuntu-bionic-amd64   | 4.15.0-112-lowlatency
> ubuntu-bionic-amd64   | ✗ 4.15.0-118; ✗ 4.15.0-121; 4.15.0-122;
> 4.15.0-123; 4.15.0-124; 4.15.0-126; 4.15.0-20
> ubuntu-focal-amd64| ✗ 5.4.0-1022-aws; ✗ 5.4.0-1024-aws;
> 5.4.0-1025-aws; 5.4.0-1028-aws; 5.4.0-1029-aws; 5.4.0-1030-aws;
> 5.4.0-1009-aws
> ubuntu-focal-amd64| ✗ 5.4.0-1022-azure; ✗ 5.4.0-1023-azure;
> 5.4.0-1025-azure; 5.4.0-1026-azure; 5.4.0-1031-azure; 5.4.0-1032-azure;
> 5.4.0-1010-azure
> ubuntu-focal-amd64| ✗ 5.4.0-51; ✗ 5.4.0-52; 5.4.0-48; 5.4.0-53;
> 5.4.0-54; 5.4.0-56; 5.4.0-26
> ubuntu-trusty-amd64   | 4.4.0-1022-aws
> ubuntu-trusty-amd64   | 3.13.0-129; 3.13.0-133; 3.13.0-139; 3.13.0-142;
> 3.13.0-149
> ubuntu-xenial-amd64   | 4.4.0-1092-aws; 4.4.0-1098-aws; 4.4.0--aws;
> 4.4.0-1114-aws; 4.4.0-1117-aws
> ubuntu-xenial-amd64   | 4.13.0-1018-azure; 4.15.0-1036-azure;
> 4.15.0-1040-azure
> ubuntu-xenial-amd64   | ✗ 4.4.0-190; 4.4.0-193; 4.4.0-194; 4.15.0-120;
> 4.15.0-123; 4.4.0-197
> xenserver6.5-amd64| 3.10.41-323
> xenserver7.1-amd64| 4.4.27-572.565306
> xenserver7.2-amd64| 4.4.52-2.1
> xenserver8.0-amd64| 4.19.19-5.0.8
>
>
> Using one of these kernels will give you the smoothest experience when
> building DRBD. We actually pre-compute all compat patches for these
> kernels and put them in our release tarballs. This means that, if one of
> these kernels is detected, you will not need spatch at all and just need
> to apply a plain patch.
>
> For a lucky set of other kernels, we have made SPAAS (spatch as a
> service). This sends a "fingerprint" of your currently running kernel's
> capabilities to LINBIT, where we can then build a compat patch
> specifically for that kernel. This also works sometimes, but again, we
> cannot possibly guarantee that this works for any given kernel (not to
> mention test it so

Re: [DRBD-user] building v9

[Yannis Milios]

I tested building on  5.8.0-31-generic (Ubuntu Focal) and I'm getting the
same error as you do, so I'd assume that drbd cannot build against that
kernel at the moment.
I have reverted back to 5.4.0-48-generic which seems to be ok.

On Thu, 10 Dec 2020 at 05:59, Pierre-Philipp Braun 
wrote:

> Hey.  Here is some update on my attempts to build the v9 module on
> Ubuntu.  I managed to build it on Slackware Linux before, but for some
> reason all my attempts are failing on Ubuntu/focal.
>
> Ubuntu 20.04.1 LTS
>
> 5.8.0-31-generic vs. 5.9.11.xenreiser4
>
> /usr/local/bin/spatch --version
> spatch version 1.0.8-00177-g28737419 compiled with OCaml version 4.08.1
> Flags passed to the configure script: [none]
> OCaml scripting support: yes
> Python scripting support: yes
> Syntax of regular expressions: Str
>
> ---
>
> apt install drbd-dkms #
> https://launchpad.net/~linbit/+archive/ubuntu/linbit-drbd9-stack
> cat /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/make.log
>
> [...]
> Successfully connected to SPAAS
> ('d35a4b17210dab1336de2725b997f300e9acd297')
>   % Total% Received % Xferd  Average Speed   TimeTime Time
> Current
>  Dload  Upload   Total   SpentLeft
> Speed
> 100  48190 00  4819  0  70867 --:--:-- --:--:-- --:--:--
> 69840
>   You can create a new .tgz including this pre-computed compat patch
>   by calling "make unpatch ; echo
> drbd-9.0.25-1/drbd/drbd-kernel-compat/cocci_cache/19859231f3d13f02c91b7f06baf8e98c/compat.patch
> >>.filelist ; make tgz"
>   PATCH
>   CC [M]
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_dax_pmem.o
>   CC [M]
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_debugfs.o
>   CC [M]
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_bitmap.o
>   CC [M]  /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_proc.o
>   CC [M]
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_sender.o
>   CC [M]
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_receiver.o
>   CC [M]  /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_req.o
>   CC [M]
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_actlog.o
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_bitmap.c: In
> function 'bm_realloc_pages':
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_bitmap.c:368:15:
> error: too many arguments to function '__vmalloc'
>   368 |   new_pages = __vmalloc(bytes,
>   |   ^
> In file included from
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_bitmap.c:16:
> ./include/linux/vmalloc.h:111:14: note: declared here
>   111 | extern void *__vmalloc(unsigned long size, gfp_t gfp_mask);
>   |  ^
> make[2]: *** [scripts/Makefile.build:290:
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_bitmap.o] Error 1
> make[2]: *** Waiting for unfinished jobs
> make[1]: *** [Makefile:1780:
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd] Error 2
> make: *** [Makefile:132: kbuild] Error 2
> make: Leaving directory
> '/var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd'
>
> ---
>
> same goes with my customized 5.9.11.xenreiser4 kernel
>
> Successfully connected to SPAAS
> ('d35a4b17210dab1336de2725b997f300e9acd297')
>   % Total% Received % Xferd  Average Speed   TimeTime Time
> Current
>  Dload  Upload   Total   SpentLeft
> Speed
> 100  48190 00  4819  0  64253 --:--:-- --:--:-- --:--:--
> 63407
>   You can create a new .tgz including this pre-computed compat patch
>   by calling "make unpatch ; echo
> drbd-9.0.25-1/drbd/drbd-kernel-compat/cocci_cache/19859231f3d13f02c91b7f06baf8e98c/compat.patch
> >>.filelist ; make tgz"
>   PATCH
>   CC [M]
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_dax_pmem.o
>   CC [M]
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_debugfs.o
>   CC [M]
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_bitmap.o
>   CC [M]  /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_proc.o
>   CC [M]
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_sender.o
>   CC [M]
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_receiver.o
>   CC [M]  /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_req.o
>   CC [M]
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_actlog.o
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_bitmap.c: In
> function 'bm_realloc_pages':
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_bitmap.c:368:15:
> error: too many arguments to function '__vmalloc'
>   368 |   new_pages = __vmalloc(bytes,
>   |   ^
> In file included from
> /var/lib/dkms/drbd/9.0.25-1ppa1~focal1/build/src/drbd/drbd_bitmap.c:16:
> ./include/linux/vmalloc.h:111:14: note: declared here
>   111 | extern void *__vmalloc(unsigned long size, gfp_t gfp_mask);
>   |  

Re: [DRBD-user] drbd-9.0.26-rc1

[Yannis Milios]

Attaching another build issue on Arch, kernel 5.9.6-arch1-1 (x86_64)  ...

DKMS make.log for drbd-9.0.26-0rc1 for kernel 5.9.6-arch1-1 (x86_64)
Fri Nov 13 10:14:12 GMT 2020
make: Entering directory '/var/lib/dkms/drbd/9.0.26-0rc1/build/src/drbd'

Calling toplevel makefile of kernel source tree, which I believe is in
KDIR=/lib/modules/5.9.6-arch1-1/build

make -C /lib/modules/5.9.6-arch1-1/build
M=/var/lib/dkms/drbd/9.0.26-0rc1/build/src/drbd  modules
  COMPAT  __vmalloc_has_2_params
  COMPAT  alloc_workqueue_takes_fmt
  COMPAT  before_4_13_kernel_read
  COMPAT  blkdev_issue_zeroout_discard
  COMPAT  drbd_release_returns_void
  COMPAT  genl_policy_in_ops
  COMPAT  have_SHASH_DESC_ON_STACK
  COMPAT  have_WB_congested_enum
  COMPAT  have_allow_kernel_signal
  COMPAT  have_atomic_dec_if_positive_linux
  COMPAT  have_atomic_in_flight
  COMPAT  have_bd_claim_by_disk
  COMPAT  have_bd_unlink_disk_holder
  COMPAT  have_bdi_congested_fn
  COMPAT  have_bio_bi_bdev
  COMPAT  have_bio_bi_disk
  COMPAT  have_bio_bi_error
  COMPAT  have_bio_bi_opf
  COMPAT  have_bio_bi_status
  COMPAT  have_bio_clone_fast
  COMPAT  have_bio_flush
  COMPAT  have_bio_free
  COMPAT  have_bio_op_shift
  COMPAT  have_bio_rw
  COMPAT  have_bio_set_op_attrs
  COMPAT  have_bio_start_io_acct
  COMPAT  have_bioset_create_front_pad
  COMPAT  have_bioset_init
  COMPAT  have_bioset_need_bvecs
  COMPAT  have_blk_check_plugged
  COMPAT  have_blk_qc_t_make_request
  COMPAT  have_blk_queue_flag_set
  COMPAT  have_blk_queue_make_request
  COMPAT  have_blk_queue_merge_bvec
  COMPAT  have_blk_queue_plugged
  COMPAT  have_blk_queue_split_bio
  COMPAT  have_blk_queue_split_q_bio
  COMPAT  have_blk_queue_split_q_bio_bioset
  COMPAT  have_blk_queue_update_readahead
  COMPAT  have_blk_queue_write_cache
  COMPAT  have_blkdev_get_by_path
  COMPAT  have_d_inode
  COMPAT  have_fallthrough
  COMPAT  have_file_inode
  COMPAT  have_generic_start_io_acct_q_rw_sect_part
  COMPAT  have_generic_start_io_acct_rw_sect_part
  COMPAT  have_genl_family_parallel_ops
  COMPAT  have_ib_cq_init_attr
  COMPAT  have_ib_get_dma_mr
  COMPAT  have_idr_alloc
  COMPAT  have_idr_is_empty
  COMPAT  have_inode_lock
  COMPAT  have_ktime_to_timespec64
  COMPAT  have_kvfree
  COMPAT  have_max_send_recv_sge
  COMPAT  have_netlink_cb_portid
  COMPAT  have_nla_nest_start_noflag
  COMPAT  have_nla_parse_deprecated
  COMPAT  have_nla_put_64bit
  COMPAT  have_part_stat_h
  COMPAT  have_pointer_backing_dev_info
  COMPAT  have_prandom_u32
  COMPAT  have_proc_create_single
  COMPAT  have_ratelimit_state_init
  COMPAT  have_rb_augment_functions
  COMPAT  have_refcount_inc
  COMPAT  have_req_hardbarrier
  COMPAT  have_req_noidle
  COMPAT  have_req_nounmap
  COMPAT  have_req_op_write
  COMPAT  have_req_op_write_same
  COMPAT  have_req_op_write_zeroes
  COMPAT  have_req_prio
  COMPAT  have_req_write
  COMPAT  have_req_write_same
  COMPAT  have_revalidate_disk_size
  COMPAT  have_security_netlink_recv
  COMPAT  have_sendpage_ok
  COMPAT  have_shash_desc_zero
  COMPAT  have_signed_nla_put
  COMPAT  have_simple_positive
  COMPAT  have_struct_bvec_iter
  COMPAT  have_struct_kernel_param_ops
  COMPAT  have_struct_size
  COMPAT  have_submit_bio
  COMPAT  have_submit_bio_noacct
  COMPAT  have_tcp_sock_set_cork
  COMPAT  have_tcp_sock_set_nodelay
  COMPAT  have_tcp_sock_set_quickack
  COMPAT  have_time64_to_tm
  COMPAT  have_timer_setup
  COMPAT  have_void_make_request
  COMPAT  hlist_for_each_entry_has_three_parameters
  COMPAT  ib_alloc_pd_has_2_params
  COMPAT  ib_device_has_ops
  COMPAT  ib_post_send_const_params
  COMPAT  ib_query_device_has_3_params
  COMPAT  kmap_atomic_page_only
  COMPAT  need_make_request_recursion
  COMPAT  queue_limits_has_discard_zeroes_data
  COMPAT  rdma_create_id_has_net_ns
  COMPAT  sock_create_kern_has_five_parameters
  COMPAT  sock_ops_returns_addr_len
  UPD /var/lib/dkms/drbd/9.0.26-0rc1/build/src/drbd/compat.5.9.6-arch1.h
  UPD /var/lib/dkms/drbd/9.0.26-0rc1/build/src/drbd/compat.h
./drbd-kernel-compat/gen_compat_patch.sh: line 12: spatch: command not found
./drbd-kernel-compat/gen_compat_patch.sh: line 45: hash: spatch: not found
  INFO: no suitable spatch found; trying spatch-as-a-service;
  be patient, may take up to 10 minutes
  if it is in the server side cache it might only take a second
  SPAAS5470857a83aa446cd991bcef3cdf804a
Successfully connected to SPAAS ('d35a4b17210dab1336de2725b997f300e9acd297')
  % Total% Received % Xferd  Average Speed   TimeTime Time
 Current
 Dload  Upload   Total   SpentLeft
 Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:--
  0
100 101280  46000  5528  12169  14624 --:--:-- --:--:-- --:--:--
26936
  You can create a new .tgz including this pre-computed compat patch
  by calling "make unpatch ; echo
drbd-9.0.26-0rc1/drbd/drbd-kernel-compat/cocci_cache/5470857a83aa446cd991bcef3cdf804a/compat.patch
>>.filelist ; make tgz"
  PATCH
patching fi

Re: [DRBD-user] [DRBD-announce] linstor-server 1.10.0 release

[Yannis Milios]

Hello,

Quick question, just wondering how will "auto-evict" affect a 3 node
linstor cluster with a replica number of 2? Say that node1 goes down for
more than an 1h, linstor will try to replace its drbd resources to either
node2 or node3 assuming that the redundacy level falls below 2 and there's
enough free space in the backing device on remaining nodes (will diskless
clients count in this case?).

How will linstor respond in the case where node1 comes back online?  Will
it just restore back drbd resources on it ? or it will just reject that
node from being a part of the cluster and in this case the node will have
to be rejoined to the cluster.

Thank you,
Yannis

On Mon, 9 Nov 2020 at 11:16, Rene Peinthor  wrote:

> Hi!
>
> This release brings 2 new added features, auto-evict and configurable ETCD
> prefixes:
>
> Auto-Evict:
> If a satellite has no connection to the controller for more than an hour,
> the controller will mark that node as EVICTED and remove all its
> DRBD-resources. Should this cause the total number of replicas for those
> resources to fall below a user-set minimum, it will then try to place new
> replicas on other satellites to keep enough replicas available.
>
> ETCD-prefixes:
> You can now configure the used ETCD prefix within the linstor.toml file,
> of course this needs to be done before the first start of the controller.
> As a little drawback(cleanup) of this change, it isn't possible anymore to
> directly upgrade ETCD backed Linstor-Controller installation from a version
> prior < 1.4.3. If you have such a situation
> upgrade to 1.9.0 first and then to 1.10.0.
>
> linstor-server 1.10.0
> 
>  * Added auto-evict feature
>  * ETCD prefix is now configurable (migration only works now starting from
> version 1.4.3)
>  * Block IO can now be throttled also by iops
>  * Fixed REST-API single snapshot filtering
>  * Fixed drbd-events2 parsing race condition
>  * Fixed toggle-disk doesn't work if an unrelated node is offline
>  * Fixed race-condition in auto-tiebreaker
>  * Fixed usage of wait for snapshot-shipping
>  * REST-API version 1.5.0
>
> https://www.linbit.com/downloads/linstor/linstor-server-1.10.0.tar.gz
>
> Linstor PPA:
> https://launchpad.net/~linbit/+archive/ubuntu/linbit-drbd9-stack
>
> Cheers,
> Rene
> ___
> drbd-announce mailing list
> drbd-annou...@lists.linbit.com
> https://lists.linbit.com/mailman/listinfo/drbd-announce
>
-- 
Sent from Gmail Mobile
___
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
https://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Remove DRBD w/out Data Loss

[Yannis Milios]

You mean completely removing DRBD while preserving the data on its backing
device ? That should work out of the box, without any extra effort, as DRBD
works as a transparent layer and it does not modify the data on the backing
device.

Yannis




On Mon, 31 Aug 2020 at 09:39, Eric Robinson  wrote:

>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> I may have missed this answer when I checked the manual, but we need to
> convert a DRBD cluster node into standalone server without losing the data.
> Is that possible?
>
>
>
>
>
> --Eric
>
>
>
>
>
>
>
>
>
>
>
>
>
> Disclaimer : This email and any files transmitted with it are confidential
> and intended solely for intended recipients. If you are not the named
> addressee you should not disseminate, distribute, copy or alter this email.
> Any views or opinions presented in this
>
> email are solely those of the author and might not represent those of
> Physician Select Management. Warning: Although Physician Select Management
> has taken reasonable precautions to ensure no viruses are present in this
> email, the company cannot accept responsibility
>
> for any loss or damage arising from the use of this email or attachments.
>
>
>
>
>
> ___
>
> Star us on GITHUB: https://github.com/LINBIT
>
> drbd-user mailing list
>
> drbd-user@lists.linbit.com
>
> https://lists.linbit.com/mailman/listinfo/drbd-user
>
> --
Sent from Gmail Mobile
___
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
https://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] cross version sync failing

[Yannis Milios]

Indeed, but that normally shouldn't be a problem (?). For example, the
combination of centos6 (drbd8) and centos7(drbd9) works without issues.


On Tue, 26 May 2020 at 11:48, Trevor Hemsley  wrote:

> On 26/05/2020 11:41, Yannis Milios wrote:
> > centos6 <-> centos7 [OK]
> > centos7 <-> centos8 [OK]
> > centos6 <-> centos8 [NOT_OK]  Error reported: "error receiving
> > ReportState, e: -5 l: 0!"
>
> You have DRBD 8.4 on CentOS 6 and DRBD 9 on the others.
>
> Trevor
>
>
> *Disclaimer*
>
> The information contained in this communication from the sender is
> confidential. It is intended solely for use by the recipient and others
> authorized to receive it. If you are not the recipient, you are hereby
> notified that any disclosure, copying, distribution or taking action in
> relation of the contents of this information is strictly prohibited and may
> be unlawful.
>
> This email has been scanned for viruses and malware, and may have been
> automatically archived by *Mimecast Ltd*, an innovator in Software as a
> Service (SaaS) for business. Providing a *safer* and *more useful* place
> for your human generated data. Specializing in; Security, archiving and
> compliance. To find out more Click Here
> <http://www.mimecast.com/products/>.
>
-- 
Sent from Gmail Mobile
___
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
https://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] cross version sync failing

[Yannis Milios]

I confirmed this as well, details below...

---
centos 6.10
kernel 2.6.32-754.el6.x86_64
drbd84-utils-9.5.0-1.el6.elrepo.x86_64
kmod-drbd84-8.4.11-1.el6_10.elrepo.x86_64



centos 7.8.2003
kernel 3.10.0-1127.8.2.el7.x86_64
drbd90-utils-9.12.2-1.el7.elrepo.x86_64
kmod-drbd90-9.0.22-2.el7_8.elrepo.x86_64
-

-
centos 8.1.1911
kernel 4.18.0-147.8.1.el8_1.x86_64
kmod-drbd90-9.0.21-2.el8_1.elrepo.x86_64
drbd90-utils-9.10.0-2.el8.elrepo.x86_64
---

Combinations:

centos6 <-> centos7 [OK]
centos7 <-> centos8 [OK]
centos6 <-> centos8 [NOT_OK]  Error reported: "error receiving ReportState,
e: -5 l: 0!"

Migrating from centos6 to centos7 first and then after from centos7 to
centos8 seems to be the only option for now.


On Mon, 25 May 2020 at 06:29, Rupa Schomaker  wrote:

> I have a pair of servers that are running current centos 6 (kernel
> versions below) which I'm working on upgrading to centos 8.  I would like
> to run for a period of time with one node running centos 6 and one node
> running centos 8 so that I can failback to centos 6 in the case we have
> unknown application issues.
>
> In my testbed, I have a centos 6 server that has a drbd volume that was
> originally synced with another centos 6 server.  I then connected that to a
> centos 7 server which ran w/out problem, but when I try to connect to a
> centos 8 server, I get a protocol error.
>
> Below are configs/logs.  Any suggestions?
>
> node 0: c61
> uname: Linux centos6-test-1.novalocal 2.6.32-754.29.1.el6.x86_64 #1 SMP
> Mon Apr 27 15:30:33 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
> kernel: kernel-2.6.32-754.29.1.el6.x86_64
> kmod: kmod-drbd84-8.4.11-1.el6_10.elrepo.x86_64
>
> [root@centos6-test-1 schomarx]# drbdadm dump
> # /etc/drbd.conf
> global {
> usage-count yes;
> cmd-timeout-medium 600;
> cmd-timeout-long 0;
> }
>
> common {
> }
>
> # resource gitlab on centos6-test-1.novalocal: not ignored, not stacked
> # defined at /etc/drbd.d/gitlab.res:1
> resource gitlab {
> on centos6-test-1.novalocal {
> device   /dev/drbd0 minor 0;
> disk /dev/vdb;
> meta-diskinternal;
> address  ipv4 192.168.0.25:7788;
> }
> on centos8-test-2.novalocal {
> device   /dev/drbd0 minor 0;
> disk /dev/vdb;
> meta-diskinternal;
> address  ipv4 192.168.0.22:7788;
> }
> net {
> protocol   C;
> verify-alg   sha256;
> }
> }
>
> node1: c82
> uanme: Linux centos8-test-2.novalocal 4.18.0-147.8.1.el8_1.x86_64 #1 SMP
> Thu Apr 9 13:49:54 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
> kernel: kernel-4.18.0-147.8.1.el8_1.x86_64
> kmod: kmod-25-13.el8.x86_64
>
> [root@centos8-test-2 schomarx]# drbdadm dump
> # /etc/drbd.conf
> global {
> usage-count no;
> }
>
> common {
> net {
> protocol   C;
> }
> }
>
> # resource gitlab on centos8-test-2.novalocal: not ignored, not stacked
> # defined at /etc/drbd.d/gitlab.res:1
> resource gitlab {
> device   /dev/drbd0 minor 0;
> on centos6-test-1.novalocal {
> node-id 1;
> disk /dev/vdb;
> address  ipv4 192.168.0.25:7788;
> }
> on centos8-test-2.novalocal {
> node-id 0;
> disk /dev/vdc;
> address  ipv4 192.168.0.22:7788;
> }
> net {
> verify-alg   sha256;
> }
> }
>
> 
>
> On centos8-test-2, I have /dev/vdb and /dev/vdc.  /dev/vdb is the working
> backing store that was originally on centos 6, worked on centos 7
> using drbd90 and then moved to centos 8.  /dev/vdc is a brand new metadata
> 90 backing store.  Neither are able to establish a connection
> without a protocol error.
>
> I've pasted the logs from each type.
>
> For /dev/vdb which has data from a prior sync:
>
> === centos6-test-1 ===
> May 20 13:23:47 centos6-test-1 kernel: drbd gitlab: conn( BrokenPipe ->
> Unconnected )
> May 20 13:23:47 centos6-test-1 kernel: drbd gitlab: receiver terminated
> May 20 13:23:47 centos6-test-1 kernel: drbd gitlab: Restarting receiver
> thread
> May 20 13:23:47 centos6-test-1 kernel: drbd gitlab: receiver (re)started
> May 20 13:23:47 centos6-test-1 kernel: drbd gitlab: conn( Unconnected ->
> WFConnection )
> May 20 13:23:47 centos6-test-1 kernel: drbd gitlab: Handshake successful:
> Agreed network protocol version 101
> May 20 13:23:47 centos6-test-1 kernel: drbd gitlab: Feature flags enabled
> on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES.
> May 20 13:23:47 centos6-test-1 kernel: drbd gitlab: conn( WFConnection ->
> WFReportParams )
> May 20 13:23:47 centos6-test-1 kern

Re: [DRBD-user] Linstor/DRBD9 : Initial sync stuck at 30Mbps

[Yannis Milios]

What happens if you temporarily disconnect the drbd resource in Primary
mode from the rest  during the "move" process? Does that speed up the
process ? If yes, then you will have to tune drbd sync parameters for the
10Gbit link.

G.

On Mon, 6 May 2019 at 14:35, Julien Escario 
wrote:

> Hello,
> Sorry if this as already been answered : I check the archives and found
> similar issues with DRBD8 but none with linstor and DRBD9.
>
> This is not really a 'problem' but more a config/performance misconfig :
> when I 'move' a ressource from another storage backend to linstor
> storage on Proxmox, the sync is maxed at 3MB/s (completely flat).
>
> Drbdtop reports this :
> Sent: total:23.8MiB Per/Sec:3.0MiB
>
> Let me first confirm my hardware is (hopefully) capable of doing far
> more (10Gbps network and full SSD ZFS storage with nvme cache).
>
> So it seems I made a misconf somewhere.
>
> I tried to change a few values :
> # linstor controller drbd-options   --max-buffers=36864
> --rcvbuf-size=2097152   --sndbuf-size=1048576
> # linstor controller drbd-options   --c-fill-target=10240
> --c-max-rate=737280   --c-min-rate=20480   --c-plan-ahead=10
>
> That was described as optimal for 10Gbps network on some howtos I found.
>
> Just in case it wasn't applied on the fly, I ran drbdadm adjust on the
> resource (both nodes).
>
> Values are stored in /var/lib/linstor.d/linstor_common.conf file.
>
> No speed change in this sync.
>
> If I run 2 similar syncs at the same time, each of them is stuck at 3MB/s.
>
> What did I miss ?
>
> Best regards,
> Julien Escario
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] LINSTOR on PROXMOX : How to deploy ressources on a new satellite?

[Yannis Milios]

> Are you saying this needs to be done for every single resource potentially
> hundreds of vm's with multiple disks attached? This sounds like a huge pita.
>

Yes.  However, I did a test. I temporarily reduced redundancy level in
/etc/pve/storage.cfg and the created a new VM in PVE.
Then I added the resource to the additional node by using 'linstor resource
create' command. Finally I checked the properties of the resource and I
noticed that the two important keys, 'PeerSlots' and 'StorPoolName' were
automatically added to the newly added resource, so I would assume that
this is not an issue anymore...

Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] LINSTOR on PROXMOX : How to deploy ressources on a new satellite?

[Yannis Milios]

You have to specify which storage pool to use for the resource, otherwise
it will default to 'DfltStorPool', which does not exist. So that would be
something like this...

$ linstor resource create pve3 vm-400-disk-1 --storage-pool 


It might be also wise to check and compare the settings of the resource on
an existing node, and then add any missing entries for the resource on the
new node. For example ..

$ linstor rd lp pve2 vm-400-disk-1#This will show the settings for
> vm-400-disk-1 on node pve2



> $ linstor rd lp pve3 vm-400-disk-1#This will show the settings for
> vm-400-disk-1 on node pve3


Compare settings, and if needed add any missing entries. This is only
needed for the existing resources.
Any new resources (VMs) you create on Proxmox will be automatically created
with the correct settings (assuming that you have increased redundancy from
2 to 3 in /etc/pve/storage.cfg).

P.S
I strongly recommend using 'linstor interactive' mode to familiarise
yourself to linstor command line parameters.

Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox broken after upgrading to PVE/libpve-storage-perl/stable 5.0-32

[Yannis Milios]

> Hello,
> Yannis, did you managed to get rid of this warning ? Same thing here
> since last upgrade.
>
> Nothing really bad happening except this annoying warning ...


No, I haven’t, but doesn’t bother me as soon as everything is working
properly.

My understanding is that this warning  is set from PVE, rather than Linbit
side. So currently there must be two possible “solutions”.
First, inspect the code on the recently updated pve-storage plugin and
remove the entries referring to that warning. This is hackish and should be
avoided, especially as any changes will be overwritten by future plugin
updates.
Second, Linbit must update linstor-proxmox plugin to match the new PVE
storage API (2.0).

I’m sure that Linbit is currently working on the second, so I would say
let’s be patient.

Yannis
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Linstor-server 0.7.3/linstor-client 0.7.2 release

[Yannis Milios]

>
> > I don't really understand why drbd8.4 module was loaded (from pve kernel
> > package) instead of drbd-dkms (aka 9) module ...
> >
> > Just ran dpkg-reconfigure drbd-dkms and rebooted servers to check if
> > correct version is loaded at boot time.
>

Personally, I choose to use 'apt-mark hold' to prevent new kernels from
installing automatically. I prefer to have control when to install a new
kernel for such reason.
Someone mentioned that there's a 'pveupgrade' command which shows more
information during 'apt dist-upgrade', but I have never tried it.


> It is pretty simple, it builds/depmods the .kos and puts them in an
> "update" directory in the /lib/modules (every distribution like to call
> this "update" dir differently, even Debian vs. Ubuntu IIRC).
>

If Proxmox was shipping DRBD9 in their kernel (instead of DRBD8), as they
were doing initially, then DKMS wouldn't be needed at all, but well, that's
their decision ...
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox broken after upgrading to PVE/libpve-storage-perl/stable 5.0-32

[Yannis Milios]

Upgraded to linstor-proxmox (3.0.2-3) and seems to be working well with
libpve-storage-perl  (5.0-32).
There's a warning notification during live migrates about the upgraded
storage API, but at the end the process is completed successfully..

"Plugin "PVE::Storage::Custom::LINSTORPlugin" is implementing an older
storage API, an upgrade is recommended"

Thanks for the fast response.

Regards,
Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox broken after upgrading to PVE/libpve-storage-perl/stable 5.0-32

[Yannis Milios]

Ok, I used "dist-upgrade" because that's what Proxmox recommends to do when
upgrading their systems.

On Tue, 27 Nov 2018 at 15:41, Roland Kammerer 
wrote:

> On Tue, Nov 27, 2018 at 03:22:07PM +0000, Yannis Milios wrote:
> > Hi Roland,
> >
> > I just did a simple ‘apt dist-upgrade’ and the rest followed ...
>
> Ah yes, that is how "dist-upgrade" works. "upgrade" would not have done
> that. Anyways, there are "-3" versions of the plugins out now and one
> should be save as long as "libpve-storage-perl" is not 5.0-31. '<' and
> '>' are ok.
>
> Regards, rck
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox broken after upgrading to PVE/libpve-storage-perl/stable 5.0-32

[Yannis Milios]

Hi Roland,

I just did a simple ‘apt dist-upgrade’ and the rest followed ...

Yannis


On Tue, 27 Nov 2018 at 14:58, Roland Kammerer 
wrote:

> On Tue, Nov 27, 2018 at 02:13:58PM +0000, Yannis Milios wrote:
> > Just for the record, Proxmox has released libpve-storage-perl/stable
> 5.0-32
> > today, which seems to break linstor-proxmox plugin.
> >
> > Reverting to libpve-storage-perl/stable 5.0-30 and reinstalling
> > linstor-proxmox package fixes the problem.
> >
> > During the upgrade, the following action is taking place ...
> >
> > Removing linstor-proxmox (3.0.2-2) ...
> > Removing 'diversion of /usr/share/perl5/PVE/Storage/DRBDPlugin.pm to
> > /usr/share/perl5/PVE/Storage/DRBDPlugin.pm.proxmox by linstor-proxmox'
> >
> > I guess this is  related to the recent changes in PVE storage API  and
> the
> > hard coded dependency of linstor-proxmox to libpve-storage-perl
> > (<= 5.0-30) ?
>
> How did you do that update? Manual dist-upgrade? Or is that "what
> proxmox does"?
>
> Yes, if it gets removed on dist-upgrade, then the plugin does not work
> anymore :). Wolfgang from PVE was kind enough to un-break old plugins in
> 042dd4be1f0e8c58940550d84572a890e5071904 (before 5.0-32 happened). So
> what I need to do is only exclude "-31", or drop the whole "version"
> match again and just pretend "-31" has never happened on actual stable
> systems. Something like this. Anyways, I will prepare a new new
> packaging release of the DRBD plugins ASAP.
>
> Thanks for mentioning it!
>
> Regards, rck
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

[DRBD-user] linstor-proxmox broken after upgrading to PVE/libpve-storage-perl/stable 5.0-32

[Yannis Milios]

Just for the record, Proxmox has released libpve-storage-perl/stable 5.0-32
today, which seems to break linstor-proxmox plugin.

Reverting to libpve-storage-perl/stable 5.0-30 and reinstalling
linstor-proxmox package fixes the problem.

During the upgrade, the following action is taking place ...

Removing linstor-proxmox (3.0.2-2) ...
Removing 'diversion of /usr/share/perl5/PVE/Storage/DRBDPlugin.pm to
/usr/share/perl5/PVE/Storage/DRBDPlugin.pm.proxmox by linstor-proxmox'

I guess this is  related to the recent changes in PVE storage API  and the
hard coded dependency of linstor-proxmox to libpve-storage-perl
(<= 5.0-30) ?

Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Linstor-Proxmox plugin 3.0.2-1 on PVE 5.2-11

[Yannis Milios]

Looks like they did a change on  pve-storage >= 5.0-31 (on pvetest
currently), which can potentially break things on the upcoming "stable"
version (see Wolfgang's previous e-mail) ... :)


On Thu, 15 Nov 2018 at 14:36, Roland Kammerer 
wrote:

> On Thu, Nov 15, 2018 at 10:45:43AM +, Yannis Milios wrote:
> >  This probably has nothing to do with DRBD, better to confirm on PVE
> > forum/ML.
> >
> > > Versions :
> > > - PVE 5.2-11
> > >
> > > I'm using the latest versions of both LINSTOR/PVE, no issues here.
> >
> > Just a thought, I noticed your pve-manager version is 5.2-11 where
> normally
> > it should be 5.2-10, if you are using pve no subscription repo.
> > Are you sure you have not messed up package versions by switching between
> > pve test -> pve no subscription ?
>
> Looks like it. I mean it can't be that package alone I guess, because
> who would break the API and smuggle that into a packaging revision bump?
>
> Where does that 5.2-11 come from? Will it transition to "stable" and
> break things? Or will that be something we have to fix for PM 6?
>
> The only thing I currently know is that it works with lastest PM 5.
>
> Regards, rck
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Linstor-Proxmox plugin 3.0.2-1 on PVE 5.2-11

[Yannis Milios]

 This probably has nothing to do with DRBD, better to confirm on PVE
forum/ML.

> Versions :
> - PVE 5.2-11
>
> I'm using the latest versions of both LINSTOR/PVE, no issues here.

Just a thought, I noticed your pve-manager version is 5.2-11 where normally
it should be 5.2-10, if you are using pve no subscription repo.
Are you sure you have not messed up package versions by switching between
pve test -> pve no subscription ?

Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox controller toggle tests

[Yannis Milios]

>
> As far as I know, Proxmox does not need 3 nodes and/or a quorum, and the
> LINSTOR controller does not care either.
>

Thanks for confirming this Robert.
In my experience, Promox requires a minimum 3 nodes when HA is
enabled/required. When HA is enabled, and one of the 2 cluster nodes goes
down, then HA will not function properly, due to not enough remaining votes
for a working,healthy cluster.
Since LINSTOR controller functionality in Proxmox is based on HA (VM), that
will (indirectly) affect also LINSTOR availability, as the HA VM used to
host it, will stop functioning as soon as Proxmox cluster (corosync) looses
the quorum majority, hence my recommendation of 3 nodes.

Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] DRBD9 + PROXMOX 5.2 - can't initialise drbd with drbdmanage

[Yannis Milios]

It’s definitely in a much better state than drbdmanage is and AFAIK people
are already using it in production.



On Sat, 10 Nov 2018 at 12:32, Sebastien CHATEAU-DUTIER <
sebast...@chateau-dutier.com> wrote:

> Dear Milios,
>
> Thanks for your help, I will make test with linstor but it's for
> production use in little time, is it OK to use linstor for production ?
>
> Regards
> Le 10/11/2018 à 13:27, Yannis Milios a écrit :
>
> drbdmanage is end of life, please use linstor instead (see drbd
> documentation on how to configure it on proxmox).
>
>
>
> On Sat, 10 Nov 2018 at 12:18, Sebastien CHATEAU-DUTIER <
> sebast...@chateau-dutier.com> wrote:
>
>> Dear all,
>>
>> On fresh install of proxmox 5.2 (2 nodes up to date with installation of
>> drbdmanage-proxmox with the offical documentation)
>>
>> I have one VG drbdpool created on nodes but I can't initialise drbd, I
>> have this error :
>>
>> root@pve1:~# drbdmanage init 192.168.72.100
>>
>> You are going to initialize a new drbdmanage cluster.
>> CAUTION! Note that:
>>* Any previous drbdmanage cluster information may be removed
>>* Any remaining resources managed by a previous drbdmanage installation
>>  that still exist on this system will no longer be managed by
>> drbdmanage
>>
>> Confirm:
>>
>>yes/no: yes
>> Error: External command failed:
>> drbdsetup new-resource .drbdctrl 0
>> Command output:
>>(stdout)
>> Command exited with exit_code 20
>>
>> Initialization failed
>>
>>
>> As anyone have an idea of what is the reel error ?
>>
>> Thanks in advance
>>
>> Best regards.
>>
>> Sebastien
>>
>> ___
>> drbd-user mailing list
>> drbd-user@lists.linbit.com
>> http://lists.linbit.com/mailman/listinfo/drbd-user
>>
> --
> Sent from Gmail Mobile
>
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] DRBD9 + PROXMOX 5.2 - can't initialise drbd with drbdmanage

[Yannis Milios]

drbdmanage is end of life, please use linstor instead (see drbd
documentation on how to configure it on proxmox).



On Sat, 10 Nov 2018 at 12:18, Sebastien CHATEAU-DUTIER <
sebast...@chateau-dutier.com> wrote:

> Dear all,
>
> On fresh install of proxmox 5.2 (2 nodes up to date with installation of
> drbdmanage-proxmox with the offical documentation)
>
> I have one VG drbdpool created on nodes but I can't initialise drbd, I
> have this error :
>
> root@pve1:~# drbdmanage init 192.168.72.100
>
> You are going to initialize a new drbdmanage cluster.
> CAUTION! Note that:
>* Any previous drbdmanage cluster information may be removed
>* Any remaining resources managed by a previous drbdmanage installation
>  that still exist on this system will no longer be managed by
> drbdmanage
>
> Confirm:
>
>yes/no: yes
> Error: External command failed:
> drbdsetup new-resource .drbdctrl 0
> Command output:
>(stdout)
> Command exited with exit_code 20
>
> Initialization failed
>
>
> As anyone have an idea of what is the reel error ?
>
> Thanks in advance
>
> Best regards.
>
> Sebastien
>
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox controller toggle tests

[Yannis Milios]

You need at least 3 nodes to have a proper working cluster (i.e quorum).

In addition, check drbd/linstor documentation on how to create a linstor vm
in pve, it will save you time in doing all those steps manually..

Yannis

On Sat, 10 Nov 2018 at 12:18, Greb  wrote:

> Hello,
>
> I did the same tests on another platform and I found the same problems.
> Do you have any idea about this problem or will there be something to
> check?
>
> thank you,
>
> Greb
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] (DRBD 9) promote secondary to primary with primary crashed

[Yannis Milios]

Try adding --force parameter in drbdadm command.



On Friday, November 2, 2018, Daniel Hertanu  wrote:

> Hi,
>
> I'm running two nodes with DRBD 9 and I want to simulate a primary node
> crash followed by restoring the access to the data on the secondary node
> left.
>
> So, having the sync done between the hosts I'm disconnecting the network
> to the primary to simulate the crash.
> All my attempts to promote the secondary to primary so I can mount the
> drbd device and access the data have failed. Usually I'm getting this:
>
> [root@server2-drbd ~]# drbdadm primary resource01
> resource01: State change failed: (-2) Need access to UpToDate data
> Command 'drbdsetup primary resource01' terminated with exit code 17
>
> Can anyone help me?
>
> Thanks,
>
> Daniel
>
>

-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] 8 Zettabytes out-of-sync?

[Yannis Milios]

On Fri, 2 Nov 2018 at 09:25, Jarno Elonen  wrote:

>
> This is getting quite worrisome. Is anyone else experiencing this with
> DRBD 9? Is it something really wrong in my setup, or are there perhaps some
> known instabilities in DRBD 9.0.15-1?
>

Yes, I have been facing this as well on all "recent" versions of DRBD9,
currently I'm on 9.0.16-1, on some of the resources. The way I usually get
that sorted by disconnecting,discarding the secondaries, but yes I would
agree with you that it looks a bit worrisome...

Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] slow sync speed

[Yannis Milios]

Just a quick note ..

You are correct, it shouldn't be required (v8.9.10) and I was surprised
> with that too.
>

In the DRBD documentation, it is stated that ...

"When multiple DRBD resources share a single replication/synchronization
network, synchronization with a fixed rate may not be an optimal approach.
So, in DRBD 8.4.0 the variable-rate synchronization was enabled by default."

..and..

"In a few, very restricted situations[4], it might make sense to just use
some fixed synchronization rate. In this case, first of all you need to
turn the dynamic sync rate controller off, by using c-plan-ahead 0;."

..by observing your configuration, it looks like you added that option
since the first time, hence no surprises here, you explicitly decided to
disable variable sync rate ... :)
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox-3.0.0-rc1

[Yannis Milios]

Just came across this, not sure if it's a bug or a feature 

When snapshoting a VM, with RAM checkbox disabled, a snapshot is created on
the node where the VM is running on.
When snapshoting a VM, with RAM checkbox enabled,a new drbd resource is
being created in the following format...
vm-101-state-snap1, and it's synced across nodes. Is this an expected
behavior? Normally, in both cases, a  local
snapshot should be created by the backend storage (LVM or ZFS), only on the
node where the VM is running.
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] DRBD 9 without Linstor

[Yannis Milios]

You can, but your life will be miserable without LINSTOR managing the
resources (hence the existence of it in the first place)  ... :)

On Wed, 3 Oct 2018 at 13:29, M. Jahanzeb Khan 
wrote:

> Hello,
>
> I would like to know that is it possible to use drbd 9 without using
> Linstor on top of LVM ?
> I have a 3 nodes server and I was using drbdmanage before. But now I just
> want to use drbd 9 without any additional tools.
>
>
> Best regards,
> Jaz
>
>
> 
>  Virus-free.
> www.avast.com
> 
> <#m_-7213948685694810749_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Softlockup when using 9.0.15-1 version

[Yannis Milios]

Not sure if it's related, but I had a similar issue on one of my PVE hosts
recently. I have the same kernel installed on all (3) nodes, but this
machine was locking up after a few minutes with the softlockup messages you
were getting. The only way to recover was to hard reboot the machine.
Managed to get it working by adding "nmi_watchdog=0" in GRUB options, as
per PVE wiki instructions. Not sure if DRBD or the kernel (tried with
different kernel as well, with no luck), or something else was causing
this. So far, works fine. I'm using DRBD 9.0.15-1 kernel module on all
nodes.

On Sun, 30 Sep 2018 at 12:45, kvaps  wrote:

> Hi, ok. Here is my kernel traces for the 4.15.18-5-pve kernel, I use
> it with both: bionic and xenial ubuntu systems.
> There is also out-of-tree mlnx_en driver installed (version 4.3-1.0.1)
> I use VFIO devices connected via bonding for DRBD communication.
> As I said before the problem occurs only with 9.0.15-1 kernel module,
> and there is no problems on 9.0.14-1 version.
>
> I'm not sure but maybe it is connected with version mismatch between
> 9.0.14-1 and 9.0.15-1 during replication process is running?
>
>
> ---
>
> [  673.805377] NMI watchdog: Watchdog detected hard LOCKUP on cpu 7
> [  683.134748] NMI watchdog: Watchdog detected hard LOCKUP on cpu 0
> [  688.597202] watchdog: BUG: soft lockup - CPU#4 stuck for 23s!
> [kworker/4:2:3483]
> [  689.539292] Kernel panic - not syncing: softlockup: hung tasks
> [  689.822358] CPU: 4 PID: 3483 Comm: kworker/4:2 Tainted: P
> O L   4.15.18-5-pve #1
> [  690.223400] Hardware name: HP ProLiant m710x Server
> Cartridge/ProLiant m710x Server Cartridge, BIOS H07 05/10/2018
> [  690.726310] Workqueue: events wait_rcu_exp_gp
> [  690.937639] Call Trace:
> [  691.055597]  
> [  691.152632]  dump_stack+0x63/0x8b
> [  691.314120]  panic+0xe4/0x244
> [  691.457890]  watchdog_timer_fn+0x21c/0x230
> [  691.656303]  ? watchdog+0x30/0x30
> [  691.816636]  __hrtimer_run_queues+0xe7/0x220
> [  692.023631]  hrtimer_interrupt+0xa3/0x1e0
> [  692.218525]  smp_apic_timer_interrupt+0x6f/0x130
> [  692.442133]  apic_timer_interrupt+0x84/0x90
> [  692.644699]  
> [  692.746094] RIP: 0010:smp_call_function_single+0x8e/0x100
> [  693.007742] RSP: 0018:bb8a07443d80 EFLAGS: 0202 ORIG_RAX:
> ff11
> [  693.375128] RAX: 0001 RBX: bb4a9200 RCX:
> 
> [  693.721604] RDX: bb4a9200 RSI: ba0feac0 RDI:
> 0005
> [  694.068179] RBP: bb8a07443dd0 R08: 96bf81523880 R09:
> 0005
> [  694.414734] R10: bb8a07443df8 R11: 0280 R12:
> 005e
> [  694.760804] R13: bb4a9200 R14: 00a0 R15:
> 0001
> [  695.107153]  ? rcu_barrier_func+0x50/0x50
> [  695.301204]  sync_rcu_exp_select_cpus+0x2ad/0x420
> [  695.529610]  ? cpumask_next+0x1b/0x20
> [  695.707302]  ? sync_rcu_exp_select_cpus+0x2ad/0x420
> [  695.944146]  ? rcu_barrier_func+0x50/0x50
> [  696.138698]  wait_rcu_exp_gp+0x20/0x30
> [  696.320144]  process_one_work+0x1e0/0x400
> [  696.514658]  worker_thread+0x4b/0x420
> [  696.692225]  kthread+0x105/0x140
> [  696.848570]  ? process_one_work+0x400/0x400
> [  697.051278]  ? kthread_create_worker_on_cpu+0x70/0x70
> [  697.296928]  ? do_syscall_64+0x73/0x130
> [  697.483045]  ? SyS_exit_group+0x14/0x20
> [  697.669355]  ret_from_fork+0x35/0x40
> [  698.924648] Shutting down cpus with NMI
> [  699.110150] Kernel Offset: 0x3900 from 0x8100
> (relocation range: 0x8000-0xbfff)
> [  699.638057] Rebooting in 10 seconds..
> [  709.736369] ACPI MEMORY or I/O RESET_REG.
>
>
> ---
>
> [ 9596.201090] watchdog: BUG: soft lockup - CPU#5 stuck for 23s!
> [systemd:12398]
> [ 9596.548566] Kernel panic - not syncing: softlockup: hung tasks
> [ 9596.831632] CPU: 5 PID: 12398 Comm: systemd Tainted: P   O
> L   4.15.18-5-pve #1
> [ 9597.220358] Hardware name: HP ProLiant m710x Server
> Cartridge/ProLiant m710x Server Cartridge, BIOS H07 05/10/2018
> [ 9597.724072] Call Trace:
> [ 9597.842902]  
> [ 9597.940248]  dump_stack+0x63/0x8b
> [ 9598.100587]  panic+0xe4/0x244
> [ 9598.244267]  watchdog_timer_fn+0x21c/0x230
> [ 9598.442988]  ? watchdog+0x30/0x30
> [ 9598.603634]  __hrtimer_run_queues+0xe7/0x220
> [ 9598.811388]  hrtimer_interrupt+0xa3/0x1e0
> [ 9599.006050]  smp_apic_timer_interrupt+0x6f/0x130
> [ 9599.230283]  apic_timer_interrupt+0x84/0x90
> [ 9599.433052]  
> [ 9599.534382] RIP: 0010:smp_call_function_many+0x1f9/0x260
> [ 9599.791986] RSP: 0018:ab5ca6027b40 EFLAGS: 0202 ORIG_RAX:
> ff11
> [ 9600.155228] RAX:  RBX: a00801563900 RCX:
> a008014288e0
> [ 9600.501585] RDX: 0001 RSI:  RDI:
> a007c0844070
> [ 9600.847949] RBP: ab5ca6027b78 R08:  R09:
> 000

[DRBD-user] How to split network traffic on LINSTOR cluster

[Yannis Milios]

Hello,

I've got some questions in regards to splitting/separating the DRBD network
traffic on a 3 node LINSTOR/PVE cluster.

Initially, both LINSTOR and DRBD traffic were using the "Default" network
(i.e 10.10.10.0/24), which was set during the initial cluster setup.
Now, I used 'linstor n i c' to create an additional interface, (i.e
nic10G), which is listening on network "10.10.20.0/24".
During the creation, I did *not* set the "-p" parameter for setting the
port for Satellite connections. So, currently the cluster looks like this...

+--+
| Node| NodeType   | Addresses   |
State   |
|--|
| linstor-controller | CONTROLLER | 10.10.10.1:3376 (PLAIN)|
Unknown |
| pve1| COMBINED   | 10.10.10.2:3366 (PLAIN),10.10.20.1 |
Online  |
| pve2| COMBINED   | 10.10.10.3:3366 (PLAIN),10.10.20.2  |
Online  |
| pve3| COMBINED   | 10.10.10.4:3366 (PLAIN),10.10.20.3  |
Online  |
+--+

Then, I set "PrefNic" property for the two "storage-pools" on nodes pve1
and pve2 (pve3 is a Diskless node).
If I create a new VM, LINSTOR correctly sets the DRBD traffic to pass
through the new network (nic10G). I can verify this by observing DRBD
resource file for the newly created VM.

However, I'm not sure how to do the same on the existing VM/DRBD resources,
which still replicate over the "Default" network ? Do I need to remove/add
each resource, on each node,  or there is a smarter way to do it ?

Another thing that I noticed is that, the Diskless node (pve3), always
connects over the "Default" network, even on the newly created resources.
This probably makes sense, as the "PrefNic" can be set at the "storage
pool" level. As the "Diskless" node does not have one, then LINSTOR uses
the "Default"
network for it? Is there a way to configure a Diskless node to use the
"nic10G" interface (as Diskful nodes do) for the DRBD replication ?

The reason I did not set the "-p" parameter when I created "nic10G" net
interface, is because I want LINSTOR traffic to remain on "Default"
network, where the "DRBD" traffic to go through "nic10G".

Last question, does "linstor-controller" VM need to listen on both
"Default" and "nic10G" network ? Normally only "Default" net should be
needed, as that's the one configured for "Satellite" <-> "Controller"
communication. Am I right on this ?


Regards,
Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Linstor v0.6.2

[Yannis Milios]

I rolled back to 0.5.0 as well, but I didn't had to restore the database.
But backing up the DB is a must anyways ...
Currently doing the test upgrades in a test (VM lab) cluster, so no harm on
the production cluster.

On Mon, 3 Sep 2018 at 15:51, Roberto Resoli  wrote:

> Il 03/09/2018 16:32, Yannis Milios ha scritto:
> > Just tried 0.6.2-1 on PVE and it seems to fail with a different error
> > this time (Migration Failed!)
> >
> >
> https://privatebin.net/?832d42e56c2734a9#0ZBY7DAQhLAbLSmzc62rokuKSVOkduBAO28lt0UIqrA=
>
> Yes, moreover, downgrading to 0.5.0 is quite annoying, and a backup copy
> of linstor db is required.
>
> rob
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Linstor v0.6.2

[Yannis Milios]

Just tried 0.6.2-1 on PVE and it seems to fail with a different error this
time (Migration Failed!)

https://privatebin.net/?832d42e56c2734a9#0ZBY7DAQhLAbLSmzc62rokuKSVOkduBAO28lt0UIqrA=

On Mon, 3 Sep 2018 at 14:31, Rene Peinthor  wrote:

> Hi again!
>
> While further testing the latest version upgrade migration, we noticed
> we missed another table migration, that are fixed now in the v0.6.2
> release.
>
> linstor-server
> --
>  * Fix: FreespaceMgr migration was still missing some entries in SECURITY
> ACL table
>  * Fix: Controller trying to connect to none satellite nodes
>
> http://www.linbit.com/downloads/linstor/linstor-server-0.6.2.tar.gz
>
> Best regards,
> Rene
>
> --
> René Peinthor - Software Developer
> rene.peint...@linbit.com
>
> LIN BIT  | Keeping
> the Digital World Running
> DRBD HA - Disaster Recovery - Software-defined Storage
> t  / f
>  / in
>  / y
>  / g+
> 
>
> DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Linstor v0.6.0 release

[Yannis Milios]

FYI, I tested upgrading linstor-controller and linstor-client from 0.5.0-1
to 0.6.0-1 on Proxmox and it's failing to start the service, with the error
below...

Reported error:
> ===
>
> Category:   RuntimeException
> Class name: LinStorSqlRuntimeException
> Class canonical name:
> com.linbit.linstor.LinStorSqlRuntimeException
> Generated at:   Method 'loadAllFreeSpaceMgrs', Source
> file 'StorPoolDataGenericDbDriver.java', Line #142
>
> Error message:  The stored free space manager name
> 'SYSTEM:' could not be restored
>

Full error report:
https://privatebin.net/?84048d3498f59ebd#Kiopv2nA+U76vlEkQ/m7frEftv8LM/akv5T7tbNHxxY=

Was able to reproduce it on two different clusters, the workaround was to
downgrade to 0.5.0-1 .

Regards,
Y



On Fri, 31 Aug 2018 at 15:24, Rene Peinthor 
wrote:

> Hi All!
>
> Linstor v0.6 is out!
>
> First thing to note is that the client command for creating storage pools
> was slightly changed, the driver is now the first positional argument e.g.
> `linstor storage-pool create lvm drbdnode sp_name`
>
> Adding a backing disk to a diskless node, while it is online, while it
> is under IO, while it is primary, is a cool feature that might come handy
> in various situations. E.g.
>
> - You want have more read IOPs for a VM
> - You want to reduce network-load when your VM reads form a virtual disk
> - You want to increase redundancy of your data set
>
> Of course the inverse operation (remove a disk, changing a resource to a
> diskless one) while it is online and primary is available with this
> release as well.
>
> linstor-server 0.6.0
> 
>  * Feature: Convert diskless DRBD to diskful online (and reverse)
>  * Feature: Add Swordfish driver support
>  * Fix hanging resource-definition delete (if it was empty)
>  * Fix incorrect lvm thin pool total capacity
>  * Fix snapshot names from migrated resource
>
> http://www.linbit.com/downloads/linstor/linstor-server-0.6.0.tar.gz
>
> linstor-client/python-linstor 0.6.0
> ---
>  * Add command for disk to diskless
>  * Client accepts multiple controllers
>  * Storage pool create command changed
>  * Added swordfish driver to storage pools
>  * Node list now shows also ports
>  * Fixes and updates to the drbdmanage migrate script
>
> http://www.linbit.com/downloads/linstor/python-linstor-0.6.0.tar.gz
> http://www.linbit.com/downloads/linstor/linstor-client-0.6.0.tar.gz
>
> Linstor PPA:
> https://launchpad.net/~linbit/+archive/ubuntu/linbit-drbd9-stack
>
> Best regards,
> Rene
>
> --
> René Peinthor - Software Developer
> rene.peint...@linbit.com
>
> LINBIT | Keeping the Digital World Running
> DRBD HA - Disaster Recovery - Software-defined Storage
> t / f / in / y / g+
>
> DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox-2.9.0

[Yannis Milios]

>
> Do we agree on that?


Yes, thanks for clairying ...



> And that is the problem we have to fix. The linstor satellite deletes
> its resource files from /var/lib/linstor.d on startup. So
> linstor-satellite.service and drbd.service started more or less at the
> same time. The satellite deleted the res file, drbd.service could not
> bring it up, the rest is obvious.


Good to know that this is a "known" issue and that you are working for a
solution ... :-)


> The satellite just deleted it. If that would not have happened,
>
drbd.service would have brought it up, and the controller VM would have
> started (assuming the rest works expected). The key thing here is that
> we do *not* need to controller to start the controller VM.


Got it.



> That is good to know. Basically with that you avoided that the res file
> got deleted, because it was at a different location ;-).
>

True .. :)


> In general we do not want that. We want the resource to be under linstor
> control. For example if you add a cluster nodes, it would be nice to
> just add that node and assign the resource to it.


I agree, that will be a much better approach.

Regards,
Y
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox-2.9.0

[Yannis Milios]

>
> This is just what came to my mind as a solution, but what storage pool
> you used for controller vm resource? Is it the linstor-managed one or
> another? In the former case, i guess that the controller vm resource is
> a sort of 'foreign body'.


 I used the DRBD storage pool (the same used by LINSTOR). The trick is
that, by defining the "controllervm " in /etc/pve/storage.cfg ,
linstor-proxmox plugin is clever enough to understand that is specific VM
is *not* handled by LINSTOR, so it defaults to use "classic" drbd approach
instead...

Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox-2.9.0

[Yannis Milios]

Sorry for interrupting, but I wanted to share my experience with this as
well (see below) ...


> What do you mean by that? The DRBD resource (== storage for the
> controller VM) is brought up by the drbd.service and can then be
> auto-promoted. The plugin code ignores that VM. The Proxmox HA service
> should do its job and start the VM on one node. So again, where exactly
> does that chain break in your setup?
>

In my case, I had success in making the LINSTOR controller HA in two
different clusters, but by following a slightly different approach.

In documentation, it's stated the following ...

"The basic idea is to execute the LINSTOR controller within a VM that is
controlled by Proxmox and
its HA features, where the storage resides on DRBD managed by LINSTOR
itself."

So by this, I understand that the LINSTOR controller VM will be managed by
PVE by using its HA feature, all good so far.
Then it's stated that, the storage that will be used to store this VM will
be DRBD, which will be managed by LINSTOR itself.


 ^^^
Question: Isn't the above a paradox? How can PVE HA start a machine which
is stored in a shared storage, which the only way to make it available, is
by starting this HA VM  in the first place?

My tests showed that whenever PVE tried to start this HA VM, it failed,
because the underlying storage (DRBD) was not available  (and yes,
drbd.service was set to "enabled" and was "started" in all nodes).
Checked inside /var/lib/linstor.d to see if the resource file for this VM
was in there, but it wasn't. Actually, there were no resource files at all
in there, apart from "linstor_common.conf".

Question: Where should drbd.service find the resource configuration file
for the LINSTOR controller VM? Inside /var/lib/linstor.d or in /etc/drbd.d ?

For the former, LINSTOR should auto generate that resource file, but it
doesn't, because the controller is not available (yet) ?. For the latter,
someone can manually create a .res file (the classic DRBD approach), copy
it to all nodes and do the required backed storage preparations before
deploying the LINSTOR controller VM.

I used the latter method and so far seems to work properly (manually
created .res files in /etc/drbd.d). As, Proxmox supports multiple storage
types, storing LINSTOR controller VM in a NFS storage for example, should
be an alternative, easier to setup option.
Almost all Proxmox cluster setups, have some type of NFS server (for
storing ISOs,templates etc) anyways, but I understand that Linbit want to
offer a purely DRBD solution, without the need to use 3rd party stuff.

No matter which option you'll choose, as soon as the LINSTOR controller HA
VM is up and running, all the rest should run properly without issues.

Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

[DRBD-user] Linstor | Failed to restore snapshot

[Yannis Milios]

I have created a snapshot for a VM by using Proxmox web interface, by using
the previous version of linstor-proxmox plugin (2.8-1).
Today, I upgraded to version 2.9.0-1 , not sure if this problem is
affecting this version as well.

When trying to restore the snapshot from the command line to a new resource
(vm-120-disk-1), as described in the documentation,
I'm getting the error below...

The snapshot was taken on a LVM Thin backend.

Caused by:
==

Description:
Failed to restore snapshot [snap_vm-122-disk-1_snap1] from volume
[vm-122-disk-1_0] to volume [vm-120-disk-1_0] Command 'lvcreate
--snapshot --name vm-120-disk-1_0
drbdpool/vm-122-disk-1_0_snap_vm-122-disk-1_snap1' returned with
exitcode 5.

Standard out:
  Using default stripesize 64.00 KiB.


Error message:
  Snapshot origin LV vm-122-disk-1_0_snap_vm-122-disk-1_snap1 not
found in Volume group drbdpool.


...Where the actual snapshot name in the drbdpool VG is (note the two 00
instead of four above):

LV Namevm-122-disk-1_00_snap_vm-122-disk-1_snap1

Could this be a bug ?

Thanks
Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Linstor | Peer's disk size too small

[Yannis Milios]

Ok, so for the time being I will have to stay with the 2 LVM backed nodes
or perhaps "convert" the 3rd node to LVM as well (even though that's
something I would like to avoid for now).

Thanks for your replies.

Regards,
Yannis


On Fri, 24 Aug 2018 at 11:23, Robert Altnoeder 
wrote:

> On 08/24/2018 12:21 PM, Yannis Milios wrote:
> > Thanks for you answer.
> >
> > It should be possible to avoid this by setting an explicit size
> > for the
> > DRBD volume in the resource configuration file, so that DRBD will
> only
> > use that much space even if more is available.
> >
> >
> > Do you mean by manually editing the resource configuration files in
> > /var/lib/linstor.d or by using a specific sub command of Linstor
> > client to achieve the same result ?
> > If I do those modifications manually on the resource files, will they
> > be overwritten by Linstor at some point ?
>
> No, we'll have to change this in LINSTOR. Manual changes to the files
> are lost when the file is overwritten.
>
> br,
> Robert
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Linstor | Peer's disk size too small

[Yannis Milios]

Thanks for you answer.

It should be possible to avoid this by setting an explicit size for the
> DRBD volume in the resource configuration file, so that DRBD will only
> use that much space even if more is available.


Do you mean by manually editing the resource configuration files in
/var/lib/linstor.d or by using a specific sub command of Linstor client to
achieve the same result ?
If I do those modifications manually on the resource files, will they be
overwritten by Linstor at some point ?

Thanks,
Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

[DRBD-user] Linstor | Peer's disk size too small

[Yannis Milios]

Hello,

Trying to create a new resource by using Linstor, on a 3 node cluster. Two
of the nodes are using LVM thin as storage backend and one of them is using
ZFS.

I created a test RD, then a VD with a size of 10GB. Then, I create the
resource on the LVM backed nodes by using 'linstor r c  test' and
'linstor r c  test' and the command succeeds.
However, when I do the same for the 3rd node (ZFS backed), it hangs forever
and I have to issue ctrl-c to cancel the operation.If I add the '--async'
option, it succeeds and I can see that the ZVOL is created for the
requested resource, but when observing the resource via drbdtop, the 3rd
node does not sync properly with the other two.
On the dmesg log in drbdtop for this resource I get the following (pve3 is
the 3rd node):
"[81307.730199] drbd test/0 drbd1000 pve3: The peer's disk size is too
small! (20971520 < 20975152 sectors)

This is reproducible in all my attempts to create any random resource on
the 3rd node. Could this be happening due to the alignment issues that
Robert mentioned on the previous post?

For now, the workaround for me has been to create the resources on LVM
backed nodes, leaving the 3rd node (ZFS) as a diskless node.
When I was using drbdmanage for managing the same cluster, I never had this
problem.

Any suggestions ?

Regards,
Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] DM to Linstor migration issue

[Yannis Milios]

Sorry, you are right, attaching it here ...
https://privatebin.net/?f272071f6eb44ba1#qzYulRsVxV3CsEKe4LrNlVrAKGbB5x7DbMDr6Q1rbao=

On Wed, 22 Aug 2018 at 12:59, Roland Kammerer 
wrote:

> On Wed, Aug 22, 2018 at 12:34:32PM +0100, Yannis Milios wrote:
> > Hi Roland,
> >
> >
> > > Do you still have the migration script? Could you post the part for
> that
> > > resource? Would be interesting which value the script tried to set.
> > >
> > >
> > Yes, I do. You can find it here
> >
> https://privatebin.net/?a12ad8f1c97bcb15#XLlAENrDGQ7OYn/Mq4Uvq7vwZuZ+jyjRBLIUPMepYgE=
>
> That is a dump of the control volume. Thanks for that, that is also
> good, but I meant the "migration.sh" (or however you called it) script
> that gets generated from "lionstor dm-migrate ctrldump.json
> migration.sh".
>
> Regards, rck
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] DM to Linstor migration issue

[Yannis Milios]

Hi Roland,


> Do you still have the migration script? Could you post the part for that
> resource? Would be interesting which value the script tried to set.
>
>
Yes, I do. You can find it here
https://privatebin.net/?a12ad8f1c97bcb15#XLlAENrDGQ7OYn/Mq4Uvq7vwZuZ+jyjRBLIUPMepYgE=

The problem in my case, was not just in one resource, but on all of them.
In short, none of the resources could  come online. To fix that, I had to
resize the volume definitions for each resource.


> Anything uncommon? Did you change the max number of peers or anything?
>

Not sure to be honest. I have done several modifications on this cluster
over the time,but perhaps I can give you some clues, maybe the answer is
somewhere in there ... :-)

- Initially, all 3 nodes were using ZFS Thin as a DRBD backend. Now, 2
nodes are using LVM Thin, and 1 ZFS.
- All resources were created automatically by drbdmanage-proxmox plugin,
sometimes with redundancy 2 and sometimes with redundancy 3 (I was playing
around with this option).
- There were occasions, where a resource which initially  was created with
drbdmanage-proxmox plugin with redundancy 2, later it was manually assigned
to the 3rd node, manually by using drbdmanage command, in order to have
redundancy of 3.
- IIRC in only one occasion, I had to manually export DRBD metadata from a
resource, modify the max-peers option from 1 to 7 and then restore import
it back. Not sure why it was set to 1 in the first place, but yes I had to
do this modification, otherwise the peers were refusing to sync.

It is good hat there is a fix and you guys managed to migrate. I still
> wonder why this did not trigger in my tests.
>

 As you can see from the above, perhaps my setup is not the ideal to go to
conclusions, but still, I would accept if some of the resources had failed,
but not all ?!. Maybe Roberto can also give some tips from his setup?

Thanks for the good work!

Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] DM to Linstor migration issue

[Yannis Milios]

On Tue, 21 Aug 2018 at 10:23, Robert Altnoeder 
wrote:

You could try to resize the volume (volume-definition set-size) to match
> those 33587200 KiB that you see as the expected value, which will
> effectively make it somewhat larger than that. If the peer count is
> indeed different from what LINSTOR thinks it should be, that might cause
> a mismatch between the net size reported by LINSTOR and the actual net
> size, but it should at least make the volume usable until we can come up
> with a fix for any issues caused by different peer count.
>

That's fantastic! Thanks for the tip and for the explanation. Managed to
have some good progress after doing what you suggested...

Regards,
Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

[DRBD-user] DM to Linstor migration issue

[Yannis Milios]

Hello,

I was testing DM to Linstor migration script by following the steps in the
documentation on a 3 node test cluster.
The migration script was completed successfully, resources and volume
definitions were created normally.
However, when rebooting the 3 nodes, none of the DRBD resources comes up
(drbdtop shows an empty list and the same applies to drbdadm,drbdsetup etc).
Checked the error reports that are listed by using the 'linstor
error-reports list', and I can see multiple occurrences of the following
error:

Reported error:
===

Description:
Initialization of storage for resource 'vm-115-disk-1' volume 0 failed
Cause:
Storage volume 0 of resource 'vm-115-disk-1' too large. Expected
33561640KiB, but was : 33587200KiB.


Same error is being logged for all VMs and I suspect that this is
preventing the resources from coming up. Any ideas how to solve this?

Thanks,
Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Linstor | changing cluster's interface ip addresses

[Yannis Milios]

Many thanks, this is going to be a very useful option...

Yannis



On Mon, 20 Aug 2018 at 17:27, Robert Altnoeder 
wrote:

> On 08/20/2018 05:19 PM, Yannis Milios wrote:
> > Quick question. I noticed the following command in Linstor:
> >
> > 'linstor node interface modify'
> >
> > By using this command someone can modify the network interface/ip
> > address that Linstor is listening on.
> >
> > Let's assume that we have already created a 3 node DRBD/Linstor
> > cluster in the ip range 10.10.10.0/24 <http://10.10.10.0/24>, and
> > later on we decide to move to different (faster) network interfaces
> > within the ip range of 10.10.20.0/24 <http://10.10.20.0/24>.
> >
> > Will such scenario be possible with Linstor, *without* recreating the
> > whole cluster ?
>
> Yes.
>
> It is not well-tested yet, but that is the intention behind it. The
> details are a bit more complex (e.g., the controller can select the
> network interface that it uses for communication with the satellite, and
> that can be a different one than those used for connecting DRBD
> resources, etc...).
>
> I have used this command in the past to change resource configurations
> from Ethernet to Infiniband, from IPv4 to IPv6, etc., so that's all
> supposed to work.
>
> > Will the above command also update drbd resource configuration files
> > to the new ip addresses *without* the need to re-create them ?
>
> Yes
>
> br,
> Robert
>
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

[DRBD-user] Linstor | changing cluster's interface ip addresses

[Yannis Milios]

Hello,

Quick question. I noticed the following command in Linstor:

'linstor node interface modify'

By using this command someone can modify the network interface/ip address
that Linstor is listening on.

Let's assume that we have already created a 3 node DRBD/Linstor cluster in
the ip range 10.10.10.0/24, and later on we decide to move to different
(faster) network interfaces within the ip range of 10.10.20.0/24.

Will such scenario be possible with Linstor, *without* recreating the whole
cluster ?
Will the above command also update drbd resource configuration files to the
new ip addresses *without* the need to re-create them ?

Thanks,
Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Migration from drbdmanage to linstor?

[Yannis Milios]

There's a migration plan from DM to LINSTOR on the way, check the following
post (Roland's comment) ...

https://lists.gt.net/drbd/users/29928?search_string=linstor%20migration;#29928

On Wed, 15 Aug 2018 at 10:22, Frank Rust  wrote:

> Hi all,
> since drbdmanage will reach its end-of-life at the end of this year, it is
> time to think about migration to linstor.
>
> I have a small cluster with 7 nodes, three of them with disks of about
> 30TB each.
> The storage is about 70% filled.
>
> Since it is no option to start linstor from scratch, how would I migrate
> all of my storage to the new system? All documents I see start with "create
> drbdpool"...
>
> Thanks in advance, Frank
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Extent XXX beyond end of bitmap!

[Yannis Milios]

Does this happen on both nodes? What’s the status of the backing device
(lvm) ? Can you post the exact versions for both kernel module and utils?
Any clue in the logs?

On Tue, 14 Aug 2018 at 06:57, Oleksiy Evin  wrote:

> Hi,
>
> We have DRBD 8.4 over LVM volume setup on CentOS7.
>
> After a forcibly rebooted both nodes we've got DRBD resources in a
> Diskless state and unable to attach any of it or make it primary.
>
> # drbd-overview
> 0:master-drbd/0  Connected Secondary/Secondary Diskless/Diskless
>
> # drbdadm attach all
> extent 19136522 beyond end of bitmap!
> extent 19143798 beyond end of bitmap!
> extent 19151565 beyond end of bitmap!
>
> ../shared/drbdmeta.c:2279:apply_al: ASSERT(bm_pos - bm_on_disk_pos <=
> chunk - extents_size) failed.
>
> I've tried to invalidate or connect discarding the data (drbdadm connect
> --discard-my-data all), nothing helped.
>
> I suppose that this is some corruption happened with activity log. Is
> there any way to resolve/discard the latest changes to make the resource up
> to date?
> Regards,
> Oleksiy Evin
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] DRBD and ProxMox asymmetric cluster

[Yannis Milios]

This has to be asked in PVE mailing list, nothing to do with DRBD.

Check PVE documentation for how HA works (keyword: nofailback).

AFIAIK PVE's HA manager has no way to interact with the underlying storage.
It will just attempt to start the resource wherever you have set it to do
so.If the undetlying storage (DRBD in this case) is ready to do that, it
will do it. I assume you are using DRBD8 in dual primary mode (with all
risks that implies, if improperly configured).




On Thursday, August 9, 2018, Alexander Trilissky <
alexander.trilis...@gmail.com> wrote:

> Hello!
>
> I have an asymmetric ProxMox 5.2 cluster with two main nodes (node-A and
> node-B) and one small qorum node (node-C).
>
> My aim is to get a HA cluster with DRBD-driven storage on main nodes. It
> should act as follows:
>
> 1)In working state all nodes are up, DRBD resources are UpToDate,
> guest VM is running on one of main nodes.
>
> 2)When Passive main node (B) fails, guest VM works as usual on Active
> main node (A). When Passive main node restores, DRBD resource synchronizes
> automatically.
>
> 3)When Active main node (A) fails, PM HA restarts guest VM on Passive
> Main node (B).
>
> And now the interesting:
>
> 4)If the rest main node (B) fails or was gracefully shut down, and
> after that we start the first main node (A), PM HA starts guest VM on
> this node with old data on disk. So we lost data changes on the main node
> B.
>
> Is there any way to allow PM HA to start guest only on the node with last
> activated resource in case of not fully mashed cluster, and to disable this
> if the node has old replica? Is third small node can help with it?
>


-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] DRBD 9 and internal metadisk v08

[Yannis Milios]

I'm not familiar with opensuse, but just guessing...
Perhaps there was a kernel update on your system and you forgot to
recompile drbd9 kernel module against it?

Normally, this should be handled automatically by dkms depending on how the
system is configured.

Try rebooting on an older kernel to verify if this is the case. If so,
download the kernel headers for the new kernel and compile drbd kernel
module for the new kernel.

On Tuesday, August 7, 2018, Josef Karliak  wrote:

>   Good morning,
>   I use drbd 9.0 on opensuse leap15, which comes with drbd 9.0. Also drbd
> 8.4 is a part of this package. After reboot the kernel module did not
> loaded and I've this server in the error - I found that the error was in
> the metadisk with v09 and now the system use drbd 8.4, so it expects
> metadisk with version v08.
>   I created new metadisk, I was informed about existing v09, but what can
> I do. Now I'm syncing.
>   There is a question - how do I tell drbd utils that I want to use drbd
> 9, not 8.4 ?
>   Thanks and best regards
>   J.Karliak
>
>   Installed rpms:
> drbd-utils-9.3.1-lp150.1.1.x86_64
> yast2-drbd-4.0.3-lp150.2.1.noarch
> drbd-9.0.14+git.62f906cf-lp150.87.5.x86_64
>
>
>
> --
> Ma domena pouziva zabezpeceni a kontrolu SPF (www.openspf.org) a
> DomainKeys/DKIM (s ADSP) a implementaci DMARC. Pokud mate problemy s
> dorucenim emailu, zacnete pouzivat metody overeni puvody emailu
> zminene vyse. Dekuji.
> My domain use SPF (www.openspf.org) and DomainKeys/DKIM (with ADSP)
> policy and implementation of the DMARC. If you've problem with sending
> emails to me, start using email origin methods mentioned above. Thank
> you.



-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox-2.8

[Yannis Milios]

> Yes, "start" is pretty obvious and in the article. Sure, "enable" is
> also a good idea, but the interesting thing is: Did you really have to
> "unmask" it?
>
>
AFAIR yes, I had to unmask the service in order to enable and then
eventually start it. But perhaps this was true when I installed
linstor-server package? not sure.
Perhaps when installing linstor-controller package, you don't have to do
this step...will have to re-check..
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox-2.8

[Yannis Milios]

On Mon, 30 Jul 2018 at 09:09, Roland Kammerer 
wrote:

> On Fri, Jul 27, 2018 at 01:52:55PM +0100, Yannis Milios wrote:
> > One last thing I forgot to mention in the last post is ...
> >
> > When creating a VM or CT via PVE webgui it fails with the below:
> >
> >
> https://privatebin.net/?dd4373728501c9eb#FsTXbEfRh43WIV4q7tO5wnm0HdW0O/gJbwavrYCgkeE=
>
> Okay, that is something for the LINSTOR people to look into. Maybe that
> happend because of the "linstor-server" vs.
> "linstor-controller/linstor-satellite" confusion and the according
> service was not started there.
>

Yes, that's correct. It happens only with linstor-server package. As soon
as you remove it and install linstor-satellite and linstor-controller, it
works as expected.
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox-2.8

[Yannis Milios]

>
> > However, in your blog post you mention
> > linstor-controller,linstor-satellite and linstor-client.
>
> That is what you should do.
>

You are right, that's what I ended to do and now everything works perfectly.
In addition, I had to enable/start linstor-satellite service on all
satellite nodes and unmask/start linstor controller service on the
controller node.


> Forget about the "linstor-server" package. Never ever use it (on Debian
> based systems).
>
>
Ok, will do.

BR
Y
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox-2.8

[Yannis Milios]

> Satellite and Controller are quite obvious, Combined is a node that runs
> a Satellite and may sometimes run a Controller, Auxiliary is a node that
> runs neither but is registered for other reasons, this is mostly
> reserved for future features.
>

Can these 'roles' be modified afterwards once set? or they are static, i.e
do we have to remove node and re-add it with another role specification.


> There is a NodeLost API and a corresponding command for it.


Is there a way for an admin to access API and run this command or it's for
dev only use?

it is expected that a system
> administrator will clean up a resource manually if automatic cleanup
> does not work,


I presume here you mean cleaning up resource LVs,ZVOLs etc. Sure, this is
true as long as the node is accessible.


> ...and as soon as LINSTOR detects that the resource has been
> cleaned up properly, it will disappear from LINSTOR's database if the
> resource was marked for deletion.
>

Nice.

There are however no plans to add any force flags like in drbdmanage to
> resource management (or similar) commands, because that frequently
> caused massive desyncs of drbdmanage's state and the real state of
> backend storage resource, as it was frequently misused by
> administrators, who also often expected the various "force" options to
> do something completely different than they actually did.
>

True ...


> Deleting the database will cause LINSTOR to initialize a new database.
> The database could be anywhere depending on how LINSTOR was installed,
> where it currently is can be found out by looking at the connection-url
> setting in controller's database.cfg file.
>

In my case it's in /opt/linstor-server/database.cfg and the entry is..
jdbc:h2:/opt/linstor-server/linstordb

Are you saying that deleting /opt/linstor-server/linstordb will reset all
settings and cause LINSTOR to create a new database file?


> This is supposed to be managed by a cluster resource manager like
> pacemaker.
> Obviously, in a multi-controller HA environment, the controller database
> must be available on all nodes, and there are various possibilities to
> ensure it is
>

Thanks, I think it has already been mentioned that for Proxmox, this will
be in the form of a HA VM appliance, which will be provided by LINBIT.


>
> I'll leave answering the package-related questions to our packaging
> experts.
>

Thanks again..

BR
Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox-2.8

[Yannis Milios]

One last thing I forgot to mention in the last post is ...

When creating a VM or CT via PVE webgui it fails with the below:

https://privatebin.net/?dd4373728501c9eb#FsTXbEfRh43WIV4q7tO5wnm0HdW0O/gJbwavrYCgkeE=

Did some investigation on the linstor side and realised as a possible
problem to be the following:

root@pve3:~# linstor r c pve2 vm-101-disk-1
ERROR:
Description:
The default storage pool 'DfltStorPool' for resource 'vm-101-disk-1'
for volume number '0' is not deployed on node 'pve2'.
Details:
The resource which should be deployed had at least one volume
definition (volume number '0') which LinStor tried to automatically create.
The default storage pool's name for this new volume was looked for in its
volume definition's properties, its resource's properties, its node's
properties and finally in a system wide default storage pool name defined
by the LinStor controller.
Node: pve2, Resource: vm-101-disk-1

If I specify the '--storage-pool drbdpool' option on 'linstor r c pve2
vm-101-disk-1' , then the resource is being assigned properly to the
cluster node.

Could this be the problem that PVE fails as well  ?

Where 'DfltStorPool' is being used for ? Is it ok to delete it and leave
only 'drbdpool' as SPD ?

Thanks
Y
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox-2.8

[Yannis Milios]

Thanks for the explanation, this was helpful. Currently testing on a 'lab'
environment.

I've got some questions, most are related to linstor itself and not
linstor-proxmox specific, hopefully this is the correct thread to expand
these questions...

- What's the difference between installing linstor-server package only
(which includes linstor-controller and linstor-satellite) and by installing
linstor-controller, linstor-satellite separately ?
In Linstor documentation, it is mentioned that linstor-server package
should be installed on all nodes. However, in your blog post you mention
linstor-controller,linstor-satellite and linstor-client.
Then later, you mention 'systemctl start linstor-server' which does not
exist if you don't install linstor-package. If you try to install
controller,satellite and server at the same time, the installation fails
with an error in creating controller and satellite systemd units. Which of
the above is the correct approach ?

- 3 nodes in the cluster(A,B,C), all configured as 'Combined' nodes, nodeC
acts as a controller.
 Let's assume that nodeA fails and it will not come up any soon, so I want
to remove it from the   cluster.To accomplish that I use  "linstor node
delete " . The problem is that the node   (which appears as OFFLINE)
it never gets deleted from the cluster. Obviously the controller, is
awaiting for the dead node's confirmation and refuses to remove its entry
if it doesn't. Is there any way to force   remove the dead node from the
database ?
 Same applies when deleting a RD,R,VD from the same node. In DM there was a
(-f) force option,   which was useful in such situations.

- Is there any option to wipe all cluster information, similar to
"drbdmanage uninit" in order to start  from scratch? Purging all
linstor packages does not seem to reset this information.

- If nodeC (controller) dies, then logically must decide which of the
surviving nodes will replace it, let's say nodeB is selected as controller
node. After starting linstor-controller service on nodeB and giving
"linstor n l" , there are no nodes cluster nodes in the list. Does this
mean we have to re-create the cluster from scratch (guess no) or there's a
way to import the config from the dead nodeC?

thanks in advance,
Yannis

Short answer: somehow if you really know what your are doing. No don't
> do that.
>
> because:
> - you can not use both plugins at the same time. Both claim the "drbd"
>   name. Long story, it has to be like this. Hardcoded "drbd" in
>   Plugin.pm which is out of our control.
> - DM/LS would not overwrite each others res files, but depending on your
>   configuration/default ports/minors, the results (one res file from DM,
>   one unrelated from LINSTOR might conflict because of port/minor
>   collisions).
>
> So if you want to test the LINSTOR stuff/plugin, do it in a "lab".
>
> Migration will be possible, also "soon" (testing the plugin and linstor
> makes this soon sooner ;-) ). Roughly it will be a DM export of the DB +
> a linstor (client) command that reads that json dump and generates
> linstor commands to add these resources to the LINSTOR DB (with the
> existing ports/minors,...). LINSTOR is then clever enough to not create
> new meta-data, it will see that these resources are up and fine. This
> will be a documented procedure for which steps you do in what order.
>
> Regards, rck
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] linstor-proxmox-2.8

[Yannis Milios]

Quick question, can we use Linstor side-by-side with DM, without affecting
one the other ?
This may be good for testing or perhaps for migrating DM resources to
Linstor in the future ?
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] [DRBD-9.0.15-0rc1] Resource "stuck" during live migration

[Yannis Milios]

I've switched pve1,pve2 to lvm thin recently just for testing and left pve3
with zfs as a storage back end. However, I really miss some cool zfs
features, compared to lvm thin, like on-the-fly compression of zero blocks
and its fast,low cost, point in time snapshots... What I don't miss though,
is zfs memory consumtion compared to lvm thin :-)

On Thu, Jul 26, 2018 at 8:26 AM Roland Kammerer 
wrote:

> On Wed, Jul 25, 2018 at 08:49:02PM +0100, Yannis Milios wrote:
> > Hello,
> >
> > Currently testing 9.0.15-0rc1 on a 3 node PVE cluster.
> >
> > Pkg versions:
> > --
> > cat /proc/drbd
> > version: 9.0.15-0rc1 (api:2/proto:86-114)
> > GIT-hash: fc844fc366933c60f7303694ca1dea734dcb39bb build by root@pve1,
> > 2018-07-23 18:47:08
> > Transports (api:16): tcp (9.0.15-0rc1)
> > ii  python-drbdmanage 0.99.18-1
> > ii  drbdmanage-proxmox2.2-1
> > ii  drbd-utils9.5.0-1
> > -
> > Resource=vm-122-disk-1
> > Replica count=3
> > PVE nodes=pve1,pve2,pve3
> > Resource is active on pve2 (Primary), the rest two nodes (pve1,pve2) are
> > Secondary.
> >
> > Tried to live migrate the VM from pve2 to pve3 and the process stuck just
> > before starting. By inspecting dmesg on both nodes (pve2,pve3), I get the
> > following crash..
> >
> >
> > pve2 (Primary) node:
> >
> https://privatebin.net/?fb5435a42b431af2#4xZpd9D5bYnB000+H3K0noZmkX20fTwGSziv5oO/Zlg=
> >
> > pve3(Secondary)node:
> >
> https://privatebin.net/?d3b1638fecb6728f#2StXbwDPT0JlFUKf686RJiR+4hl52jEmmij2UTtnSjs=
> >
>
> We will look into it closer. For now I saw "zfs" in the second trace and
> stopped. It is so freaking broken, it is not funny any more (it craps
> out with all kinds of BS in our internal infrastructure as well). For
> example we had to go back to a xenial kernel because the bionic ones zfs
> is that broken :-/ 
>
> Regards, rck
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

[DRBD-user] [DRBD-9.0.15-0rc1] Resource "stuck" during live migration

[Yannis Milios]

Hello,

Currently testing 9.0.15-0rc1 on a 3 node PVE cluster.

Pkg versions:
--
cat /proc/drbd
version: 9.0.15-0rc1 (api:2/proto:86-114)
GIT-hash: fc844fc366933c60f7303694ca1dea734dcb39bb build by root@pve1,
2018-07-23 18:47:08
Transports (api:16): tcp (9.0.15-0rc1)
ii  python-drbdmanage 0.99.18-1
ii  drbdmanage-proxmox2.2-1
ii  drbd-utils9.5.0-1
-
Resource=vm-122-disk-1
Replica count=3
PVE nodes=pve1,pve2,pve3
Resource is active on pve2 (Primary), the rest two nodes (pve1,pve2) are
Secondary.

Tried to live migrate the VM from pve2 to pve3 and the process stuck just
before starting. By inspecting dmesg on both nodes (pve2,pve3), I get the
following crash..


pve2 (Primary) node:
https://privatebin.net/?fb5435a42b431af2#4xZpd9D5bYnB000+H3K0noZmkX20fTwGSziv5oO/Zlg=

pve3(Secondary)node:
https://privatebin.net/?d3b1638fecb6728f#2StXbwDPT0JlFUKf686RJiR+4hl52jEmmij2UTtnSjs=

Cancelled the migration, but it now it's impossible to change the state of
the DRBD resource (vm-122-disk-1), in any way (switch from Primary to
Secondary, Disconnect, bring down the resource etc) on pve3 or pve2.

root@pve3:~# drbdadm down vm-122-disk-1
vm-122-disk-1: State change failed: (-12) Device is held open by someone
additional info from kernel:
failed to demote
Command 'drbdsetup down vm-122-disk-1' terminated with exit code 11

Can't find any apparent process locking the specific resource on pve3 by
using lsof.

Is there a way to recover from this without rebooting the each node ?

Thanks
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

[DRBD-user] drbdmanage-handlers out-of-sync ?

[Yannis Milios]

Hello,

Currently, drbdmanage-handlers command does not seem to support
"out-of-sync" handler. I'm aware that drbdmanage is about to be replaced by
linstor, just wondering if there is a possibility to be added to drbdmanage
somehow ?

Thanks
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Intro questions

[Yannis Milios]

I believe that most of your questions can be answered by reading the DRBD
user's guide..

https://docs.linbit.com/docs/users-guide-8.4/#p-intro

There are 2 versions of DRBD available at the moment, DRBD8 (for up to 2
nodes) and DRBD9 (for 2+ nodes).

On Mon, Jun 25, 2018 at 7:07 AM Alex  wrote:

> Hi,
>
> I posted this message a few days ago and I'm not sure it was received.
> I'm really hoping someone can help.
>
> I'm trying to understand how drbd works and how all the different
> pieces go together. I first installed drbd on Linux many, many years
> ago, but it appears to be quite different and obviously more mature.
>
> I have a mail server with about 3TB of disk space for users. I'd like
> to create a completely different system on a completely different
> network and somehow provide a single interface for users to access
> their email.
>
> Does drbd typically work with a front-end system that helps to manage
> which system is active?
>
> Ideally I'd like a system where I can advertise a single IP address in
> DNS and have traffic routed to the currently active system but I don't
> really understand how all the pieces go together.
>
> Does drbd typically involve NFS? LVM?
>
> Is there a general overview doc that provides a description of typical
> configurations?
>
> What are the services that are available as open source as compared
> with the ones offered commercially by linbit?
>
> Thanks,
> Alex
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] [proxmox]Move disk to drbd storage fails...

[Yannis Milios]

Looks like the new gmail has messed up the message, re sending in plain text...

create full clone of drive scsi1 (local-zfs:vm-108-disk-2)
pong
Operation completed successfully
Operation completed successfully
pong
Operation completed successfully
Operation completed successfully
pong
Operation completed successfully
pong
Operation completed successfully
Operation completed successfully
transferred: 0 bytes remaining: 34359738368 bytes total: 34359738368
bytes progression: 0.00 %
qemu-img: Could not open '/dev/drbd/by-res/vm-108-disk-1/0': Could not
open '/dev/drbd/by-res/vm-108-disk-1/0': No such file or directory
pong
Operation completed successfully
Operation completed successfully
TASK ERROR: storage migration failed: copy failed: command
'/usr/bin/qemu-img convert -p -n -T none -f raw -O raw
/dev/zvol/rpool/data/vm-108-disk-2 /dev/drbd/by-res/vm-108-disk-1/0'
failed: exit code 1
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] [proxmox]Move disk to drbd storage fails...

[Yannis Milios]

This does not seem to work for me either, I'm on the latest pve and
drbd9/drbdmanage versions. For me, it fails with a different kind of error
though.
I've executed the command on the "leader" node. The source vm disk is local
zfs but I tried it also with a qemu raw image and had the same result.

Task viewer: VM 108 - Move disk
OutputStatus
Stop
create full clone of drive scsi1 (local-zfs:vm-108-disk-2)
pong
Operation completed successfully
Operation completed successfully
pong
Operation completed successfully
Operation completed successfully
pong
Operation completed successfully
pong
Operation completed successfully
Operation completed successfully
transferred: 0 bytes remaining: 34359738368 bytes total: 34359738368 bytes
progression: 0.00 %
qemu-img: Could not open '/dev/drbd/by-res/vm-108-disk-1/0': Could not open
'/dev/drbd/by-res/vm-108-disk-1/0': No such file or directory
pong
Operation completed successfully
Operation completed successfully
TASK ERROR: storage migration failed: copy failed: command
'/usr/bin/qemu-img convert -p -n -T none -f raw -O raw
/dev/zvol/rpool/data/vm-108-disk-2 /dev/drbd/by-res/vm-108-disk-1/0'
failed: exit code 1



On Wed, Jun 6, 2018 at 8:30 AM Roland Kammerer 
wrote:

> On Tue, Jun 05, 2018 at 03:46:10PM +0200, Christophe NOUVEL wrote:
> > Hi all,
> >
> > Our new 2 nodes proxmox cluster (PVE 5.2) + drbd9 (Linbit repository)
> works
> > fine.
> >
> > But when we try to move an existing disk, from an external USB hard drive
> > to DRB storage, we have :
> >
> > create full clone of drive ide1 (local-lvm:vm-107-disk-1)
> > pong
> > Operation completed successfully
> > Operation completed successfully
> > pong
> > Operation completed successfully
> > "." is not a valid unit!
> > usage: drbdmanage add-volume [-h] [-m MINOR] [-d DEPLOY] [-s SITE] name
> size
> > ...
> > Valid units: pib,gb,kib,kb,g,mb,k,mib,m,pb,p,t,tb,tib,gib
> > pong
> > Operation completed successfully
> > pong
> > Operation completed successfully
> > Operation completed successfully
> > drive mirror is starting for drive-ide1
> > drive-ide1: Cancelling block job
> > drive-ide1: Done.
> > pong
> > Operation completed successfully
> > Operation completed successfully
> > TASK ERROR: storage migration failed: mirroring error: VM 107 qmp command
> > 'drive-mirror' failed - Could not open
> '/dev/drbd/by-res/vm-107-disk-3/0':
> > No such file or directory
> >
> > We found a similar problem here :
> > http://lists.linbit.com/pipermail/drbd-user/2017-September/023679.html
> >
> > And tried to patch /usr/share/perl5/PVE/Storage/Custom/DRBDplugin.pm, but
> > nothing changed...
> > I even tried to hide this file, and nothing changes again??
>
> 1. Don't manually patch something
> 2. To me that looks like a bug that got fixed by now probably more than
> a year ago, IIRC. Back then the bug was that this "debug output"
> including '.' was printend on stdout, and then got mixed into other
> commands.
>
> So my best guess is you are using very very outdated versions.
>
> dpkg -l | grep drbd # would help
>
> Regards, rck
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] PingAck did not arrive in time.

[Yannis Milios]

Two things:

- I would use drbd8 instead of drbd9 for a 2 node setup.
- I would first test with 1 nic instead of 2.

On Wed, May 23, 2018 at 11:01 AM, Dirk Bonenkamp - ProActive <
d...@proactive.nl> wrote:

> Hi List,
>
> I'm struggling with a new DRBD9 setup. It's a simple Master/Slave setup.
> I'm running Ubuntu 16.04 LTS with the DRBD9 packages from the Launchpad
> PPA.
>
> I'm running some DRBD8 systems in production for quite some years, so I
> have some experience. This setup is very similar, the only major
> difference is that this is DRBD9 and I use LUKS encrypted partitions as
> backend.
>
> I keep running into this 'PingAck did not arrive in time.' error, which
> points to network issues if I am correct (see complete log snippet
> below). This error occurs when I try to reattach the secondary node
> after a reboot. Initial sync works fine.
>
> The servers are interconnected with 2 10Gb NICs. I had bonding & jumbo
> frames configured, but deactivated all this, to no avail. I've also
> stripped the DRBD configuration to the bare minimum (see below).
>
> I've tested the connection with iperf and some other tools and it seems
> just fine.
>
> Could somebody point me in the right direction?
>
> Thank you in advance, regards,
>
> Dirk Bonenkamp
>
> syslog messages:
>
> May 23 11:31:56 data2 kernel: [  704.111755] drbd: loading out-of-tree
> module taints kernel.
> May 23 11:31:56 data2 kernel: [  704.112290] drbd: module verification
> failed: signature and/or required key missing - tainting kernel
> May 23 11:31:56 data2 kernel: [  704.127677] drbd: initialized. Version:
> 9.0.14-1 (api:2/proto:86-113)
> May 23 11:31:56 data2 kernel: [  704.127680] drbd: GIT-hash:
> 62f906cf44ef02a30ce0c148fec223b40c51c533 build by root@data2, 2018-05-23
> 09:19:54
> May 23 11:31:56 data2 kernel: [  704.127683] drbd: registered as block
> device major 147
> May 23 11:31:56 data2 kernel: [  704.153565] drbd r0: Starting worker
> thread (from drbdsetup [4495])
> May 23 11:31:56 data2 kernel: [  704.183031] drbd r0/0 drbd0: disk(
> Diskless -> Attaching )
> May 23 11:31:56 data2 kernel: [  704.183066] drbd r0/0 drbd0: Maximum
> number of peer devices = 1
> May 23 11:31:56 data2 kernel: [  704.183293] drbd r0: Method to ensure
> write ordering: flush
> May 23 11:31:56 data2 kernel: [  704.183308] drbd r0/0 drbd0:
> drbd_bm_resize called with capacity == 273437203064
> May 23 11:31:58 data2 kernel: [  706.508228] drbd r0/0 drbd0: resync
> bitmap: bits=34179650383 words=534057038 pages=1043081
> May 23 11:31:58 data2 kernel: [  706.508234] drbd r0/0 drbd0: size = 127
> TB (136718601532 KB)
> May 23 11:31:58 data2 kernel: [  706.508236] drbd r0/0 drbd0: size = 127
> TB (136718601532 KB)
> May 23 11:32:10 data2 kernel: [  717.890420] drbd r0/0 drbd0: recounting
> of set bits took additional 1256ms
> May 23 11:32:10 data2 kernel: [  717.890435] drbd r0/0 drbd0: disk(
> Attaching -> Outdated )
> May 23 11:32:10 data2 kernel: [  717.890439] drbd r0/0 drbd0: attached
> to current UUID: 244DD61D2781DF44
> May 23 11:32:10 data2 kernel: [  717.918473] drbd r0 data1: Starting
> sender thread (from drbdsetup [4544])
> May 23 11:32:10 data2 kernel: [  717.922534] drbd r0 data1: conn(
> StandAlone -> Unconnected )
> May 23 11:32:10 data2 kernel: [  717.922820] drbd r0 data1: Starting
> receiver thread (from drbd_w_r0 [4498])
> May 23 11:32:10 data2 kernel: [  717.922973] drbd r0 data1: conn(
> Unconnected -> Connecting )
> May 23 11:32:10 data2 kernel: [  718.421219] drbd r0 data1: Handshake to
> peer 1 successful: Agreed network protocol version 113
> May 23 11:32:10 data2 kernel: [  718.421229] drbd r0 data1: Feature
> flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
> WRITE_ZEROES.
> May 23 11:32:10 data2 kernel: [  718.421259] drbd r0 data1: Starting
> ack_recv thread (from drbd_r_r0 [4550])
> May 23 11:32:10 data2 kernel: [  718.424095] drbd r0: Preparing
> cluster-wide state change 1205605755 (0->1 499/146)
> May 23 11:32:10 data2 kernel: [  718.437172] drbd r0: State change
> 1205605755: primary_nodes=2, weak_nodes=FFFC
> May 23 11:32:10 data2 kernel: [  718.437185] drbd r0: Aborting
> cluster-wide state change 1205605755 (12ms) rv = -22
> May 23 11:32:12 data2 kernel: [  719.896223] drbd r0: Preparing
> cluster-wide state change 445952355 (0->1 499/146)
> May 23 11:32:12 data2 kernel: [  719.896498] drbd r0: State change
> 445952355: primary_nodes=2, weak_nodes=FFFC
> May 23 11:32:12 data2 kernel: [  719.896508] drbd r0: Committing
> cluster-wide state change 445952355 (0ms)
> May 23 11:32:12 data2 kernel: [  719.896541] drbd r0 data1: conn(
> Connecting -> Connected ) peer( Unknown -> Primary )
> May 23 11:32:12 data2 kernel: [  719.912186] drbd r0/0 drbd0 data1:
> drbd_sync_handshake:
> May 23 11:32:12 data2 kernel: [  719.912198] drbd r0/0 drbd0 data1: self
> 244DD61D2781DF44:::
> bits:52035 flags:20
> May 23 11:32:12 data2 kernel: [  719.

Re: [DRBD-user] cannot remove snapshot

[Yannis Milios]

First make sure your DRBD cluster is in healthy state, that is, all drbd
control volumes are in "normal" status.

Then, I would use "drbdmanage resume-all" to resume all pending operations
first. If that fails, then I would use "drbdmanage remove-snapshot -f
resource snapshot" to force remove then snapshot from DRBD cluster.

On Wed, May 16, 2018 at 11:36 AM, Michał Szamocki 
wrote:

> Hello,
>
> My cluster shows a snapshot I can't remove:
>
> # drbdmanage s
> +---
> -+
> | Resource  | Name  |
>   | State |
> |---
> -|
> | vm-112-disk-1 | snap_vm-112-disk-1_vzdump |
>   |   n/a |
> +---
> -+
>
> How can I remove information about snapshot from ctrlvol?
>
>
> It was already removed on LVM level:
>
> # lvs | grep vm-112
>   vm-112-disk-1_00 drbdpool Vwi-aotz-- 120,12g drbdthinpool94,31
>
>
> I can't create new snapshot:
>
> # /usr/bin/drbdmanage create-snapshot snap_vm-112-disk-1_vzdump
> vm-112-disk-1 adelaide
> Error: Object already exists
>
> I found errors in syslog:
> May 12 07:52:00 adelaide drbdmanaged[3404]: ERROR  Failed to remove
> snapshot snap_vm-112-disk-1_vzdump #0 block device
> 'vm-112-disk-1.snap_vm-112-disk-1_vzdump_00'
> May 12 07:54:16 adelaide drbdmanaged[3404]: ERROR  Failed to remove
> snapshot snap_vm-112-disk-1_vzdump #0 block device
> 'vm-112-disk-1.snap_vm-112-disk-1_vzdump_00'
>
> BR,
> Michał Szamocki
>
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Strange drbdtop results

[Yannis Milios]

>drbdtop on a resource with detailed >status give me OutOfSync on some
>nodes.

I try to adjust all resources without any success on solving this problem


That can be solved by “disconnect/connect” the resource that has out of
sync blocks.
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] One resource per disk?

[Yannis Milios]

I would prefer the 2nd option. Ideally all disks would be members of a
RAID(10?) array, with  DRBD sitting on top for the replication, and LVM for
managing the volume.
Another option would be ZFS managing the disks and the volume, while DRBD
sitting on top for the replication. This very same scenario would also
apply for LVM (thin or thick).
There's no right or wrong, depends on what your needs are.

I would avoid having single points of failure though, like single drives in
any case...

On Tue, May 1, 2018 at 3:12 PM, Gandalf Corvotempesta <
gandalf.corvotempe...@gmail.com> wrote:

> Hi to all
> Let's assume 3 servers with 12 disks each
> Would you create one resource per disk and then manage them with something
> like LVM or a single resource from a huge volume over all disks?
>
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] failed exit code 1

[Yannis Milios]

There's a dedicated repo for PVE, you can use that one instead of building
from source.
You can find more here:
https://docs.linbit.com/docs/users-guide-9.0/#ch-proxmox

On Mon, Apr 30, 2018 at 2:12 PM, Sebastian Blajszczak <
sebastian.blajszc...@addmore.de> wrote:

> It failed, because I had to compile it manually:
>
> [r...@leia.comp.local~]$# dpkg -l | grep drbd-utils
>
> ii  drbd-utils   9.1.1+linbit-1
> amd64RAID 1 over TCP/IP for Linux (user utilities)
>
>
>
> I´m on debian 9. Is there an easier way to update it?
>
>
>
> Wit best regards
>
>
>
> *Von:* Yannis Milios 
> *Gesendet:* Montag, 30. April 2018 14:17
> *An:* Sebastian Blajszczak 
> *Cc:* drbd-user@lists.linbit.com
> *Betreff:* Re: [DRBD-user] failed exit code 1
>
>
>
> When you say "it failed", how did it exactly failed ? Anything on the logs
> ?
>
>
>
> I'm on PVE5 as well, but did not have any issues updating both utils and
> kmod.
>
>
>
> dpkg -l | grep drbd-utils
> ii  drbd-utils   9.3.1-1
> amd64RAID 1 over TCP/IP for Linux (user utilities)
>
>
>
> dpkg -l | grep drbd-dkms
> ii  drbd-dkms9.0.13-1
>  all  RAID 1 over TCP/IP for Linux module source
>
>
>
>
>
> On Thu, Apr 26, 2018 at 1:19 PM, Sebastian Blajszczak <
> sebastian.blajszc...@addmore.de> wrote:
>
> Hello,
>
>
>
> I tried to update the drbd-utils from 9.1.1 to 9.3
>
> It failed, and I still on 9.1.1, for me OK, but:
>
>
>
> Now when I trying to start a VM on this machine I´m getting this error:
>
>
>
> kvm: -drive file=/dev/drbd/by-res/vm-106-disk-1/0,if=none,id=drive-
> virtio0,cache=writethrough,format=raw,aio=threads,detect-zeroes=on: Could
> not open '/dev/drbd/by-res/vm-106-disk-1/0': No such file or directory
>
>
>
> start failed: command '/usr/bin/kvm -id 106 -name LDAP-FB -chardev
> 'socket,id=qmp,path=/var/run/qemu-server/106.qmp,server,nowait' -mon
> 'chardev=qmp,mode=control' -pidfile /var/run/qemu-server/106.pid -daemonize
> -smp '1,sockets=1,cores=1,maxcpus=1' -nodefaults -boot
> 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/
> share/qemu-server/bootsplash.jpg' -vga std -vnc
> unix:/var/run/qemu-server/106.vnc,x509,password -cpu
> kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce -m 256 -device
> 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device
> 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device
> 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device
> 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 
> 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3'
> -iscsi 'initiator-name=iqn.1993-08.org.debian:01:308de83551e6' -drive
> 'file=/dev/drbd/by-res/vm-106-disk-1/0,if=none,id=drive-
> virtio0,cache=writethrough,format=raw,aio=threads,detect-zeroes=on'
> -device 
> 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100'
> -netdev 'type=tap,id=net0,ifname=tap106i0,script=/var/lib/qemu-
> server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on'
> -device 'virtio-net-pci,mac=E6:9D:26:C9:18:41,netdev=net0,bus=pci.
> 0,addr=0x12,id=net0,bootindex=300'' failed: exit code 1
>
>
>
> I´m on drbd 9.0.13
>
> Proxmox 5
>
>
>
> I can migrate VMs offline and start them on the other node2 (did not
> update node 2), all other things are running. Drbd is up to date and
> syncing.
>
> How can I fix this error?
>
>
>
> With best regards,
>
> Sebastian
>
>
>
>
>
>
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
>
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] failed exit code 1

[Yannis Milios]

When you say "it failed", how did it exactly failed ? Anything on the logs ?

I'm on PVE5 as well, but did not have any issues updating both utils and
kmod.

dpkg -l | grep drbd-utils
> ii  drbd-utils   9.3.1-1
> amd64RAID 1 over TCP/IP for Linux (user utilities)


dpkg -l | grep drbd-dkms
> ii  drbd-dkms9.0.13-1
>  all  RAID 1 over TCP/IP for Linux module source



On Thu, Apr 26, 2018 at 1:19 PM, Sebastian Blajszczak <
sebastian.blajszc...@addmore.de> wrote:

> Hello,
>
>
>
> I tried to update the drbd-utils from 9.1.1 to 9.3
>
> It failed, and I still on 9.1.1, for me OK, but:
>
>
>
> Now when I trying to start a VM on this machine I´m getting this error:
>
>
>
> kvm: -drive file=/dev/drbd/by-res/vm-106-disk-1/0,if=none,id=drive-
> virtio0,cache=writethrough,format=raw,aio=threads,detect-zeroes=on: Could
> not open '/dev/drbd/by-res/vm-106-disk-1/0': No such file or directory
>
>
>
> start failed: command '/usr/bin/kvm -id 106 -name LDAP-FB -chardev
> 'socket,id=qmp,path=/var/run/qemu-server/106.qmp,server,nowait' -mon
> 'chardev=qmp,mode=control' -pidfile /var/run/qemu-server/106.pid -daemonize
> -smp '1,sockets=1,cores=1,maxcpus=1' -nodefaults -boot
> 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/
> share/qemu-server/bootsplash.jpg' -vga std -vnc
> unix:/var/run/qemu-server/106.vnc,x509,password -cpu
> kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce -m 256 -device
> 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device
> 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device
> 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device
> 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 
> 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3'
> -iscsi 'initiator-name=iqn.1993-08.org.debian:01:308de83551e6' -drive
> 'file=/dev/drbd/by-res/vm-106-disk-1/0,if=none,id=drive-
> virtio0,cache=writethrough,format=raw,aio=threads,detect-zeroes=on'
> -device 
> 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100'
> -netdev 'type=tap,id=net0,ifname=tap106i0,script=/var/lib/qemu-
> server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on'
> -device 'virtio-net-pci,mac=E6:9D:26:C9:18:41,netdev=net0,bus=pci.
> 0,addr=0x12,id=net0,bootindex=300'' failed: exit code 1
>
>
>
> I´m on drbd 9.0.13
>
> Proxmox 5
>
>
>
> I can migrate VMs offline and start them on the other node2 (did not
> update node 2), all other things are running. Drbd is up to date and
> syncing.
>
> How can I fix this error?
>
>
>
> With best regards,
>
> Sebastian
>
>
>
>
>
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Not enough free bitmap slots when assigning a resource on an additional node

[Yannis Milios]

After digging a bit more in both user's guide and in ML, managed to assign
the resource on the 3rd node.
It was required to manually dump,modify and restore the metadata.
I had to increase the value of the 'max-peers' option, which for this
particular resource was set to '0' for some reason.

All other resources seem to have the correct values.


On Mon, Apr 9, 2018 at 2:44 PM, Yannis Milios 
wrote:

> Hello,
>
> On a 3 node/zfs backed drbd9 cluster, while trying to assign-resource on
> an additional node, I'm getting "Not enough free bitmap slots" and the
> resync does not start.
>
> Removing/reassigning the resource does not help either. I couldn't find
> enough information about this error when searching ML archives.
>
> Any ideas what is causing this ?
>
> Thanks
>
> Some logs:
> 
> │[137368.869743] drbd vm-122-disk-1: Preparing cluster-wide state change
> 3649068569 (0->-1 3/1)
> │
> │[137368.870076] drbd vm-122-disk-1: State change 3649068569:
> primary_nodes=5, weak_nodes=FFF8
>│
> │[137368.870078] drbd vm-122-disk-1: Committing cluster-wide state change
> 3649068569 (0ms)
>│
> │[137368.870082] drbd vm-122-disk-1: role( Secondary -> Primary )
>
>   │
> │[142124.120066] drbd vm-122-disk-1 pve1: Preparing remote state change
> 152764597 (primary_nodes=1, weak_nodes=FFFC)
>  │
> │[142124.120284] drbd vm-122-disk-1 pve1: Committing remote state change
> 152764597
>  │
> │[142124.120289] drbd vm-122-disk-1: State change failed: Refusing to be
> Outdated while Connected
> │
> │[142124.120399] drbd vm-122-disk-1/0 drbd103: Failed: disk( UpToDate ->
> Outdated )
> │
> │[142124.120410] drbd vm-122-disk-1: FATAL: Local commit of prepared
> 152764597 failed!
>  │
> │[142124.350767] drbd vm-122-disk-1: Preparing cluster-wide state change
> 3667315219 (0->1 496/16)
> │
> │[142124.350948] drbd vm-122-disk-1: State change 3667315219:
> primary_nodes=5, weak_nodes=FFFB
>│
> │[142124.350949] drbd vm-122-disk-1 pve2: Cluster is now split
>
>│
> │[142124.350950] drbd vm-122-disk-1: Committing cluster-wide state change
> 3667315219 (0ms)
>│
> │[142124.350970] drbd vm-122-disk-1 pve2: conn( Connected -> Disconnecting
> ) peer( Secondary -> Unknown )
>   │
> │[142124.350973] drbd vm-122-disk-1/0 drbd103 pve2: pdsk( Diskless ->
> DUnknown ) repl( Established -> Off )
> │
> │[142124.350994] drbd vm-122-disk-1 pve2: ack_receiver terminated
>
>   │
> │[142124.350996] drbd vm-122-disk-1 pve2: Terminating ack_recv thread
>
>   │
> │[142124.467248] drbd vm-122-disk-1 pve2: Connection closed
>
>   │
> │[142124.467428] drbd vm-122-disk-1 pve2: conn( Disconnecting ->
> StandAlone )
> │
> │[142124.467432] drbd vm-122-disk-1 pve2: Terminating receiver thread
>
>   │
> │[142124.467504] drbd vm-122-disk-1 pve2: Terminating sender thread
>
>   │
> │[142133.966538] drbd vm-122-disk-1 pve2: Starting sender thread (from
> drbdsetup [1502468])
>   │
> │[142133.968349] drbd vm-122-disk-1/0 drbd103 pve2: Not enough free bitmap
> slots
>│
> │[142135.077014] drbd vm-122-disk-1/0 drbd103 pve2: Not enough free bitmap
> slots
>│
> │[142227.488145] drbd vm-122-disk-1/0 drbd103 pve2: Not enough free bitmap
> slots
>│
> │[142232.509729] drbd vm-122-disk-1/0 drbd103 pve2: Not enough free bitmap
> slots
>│
> │[142272.526580] drbd vm-122-disk-1/0 drbd103 pve2: Not enough free bitmap
> slots
>│
> │[143361.840005] drbd vm-122-disk-1 pve2: Terminating sender thread
>
>   │
> │[143505.002551] drbd vm-122-disk-1 pve2: Starting sender thread (from
> drbdsetup [3396832])
>   │
> │[143505.004330] drbd vm-122-disk-1/0 drbd103 pve2: Not enough free bitmap
> slots
>│
> │[143506.090369] drbd vm-122-disk-1/0 drbd103 pve2: Not enough free bitmap
> slots
>│
> │[143633.980319] drbd vm-122-disk-1 pve2: Terminating sender thread
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

[DRBD-user] Not enough free bitmap slots when assigning a resource on an additional node

[Yannis Milios]

Hello,

On a 3 node/zfs backed drbd9 cluster, while trying to assign-resource on an
additional node, I'm getting "Not enough free bitmap slots" and the resync
does not start.

Removing/reassigning the resource does not help either. I couldn't find
enough information about this error when searching ML archives.

Any ideas what is causing this ?

Thanks

Some logs:

│[137368.869743] drbd vm-122-disk-1: Preparing cluster-wide state change
3649068569 (0->-1 3/1)
│
│[137368.870076] drbd vm-122-disk-1: State change 3649068569:
primary_nodes=5, weak_nodes=FFF8
   │
│[137368.870078] drbd vm-122-disk-1: Committing cluster-wide state change
3649068569 (0ms)
   │
│[137368.870082] drbd vm-122-disk-1: role( Secondary -> Primary )

  │
│[142124.120066] drbd vm-122-disk-1 pve1: Preparing remote state change
152764597 (primary_nodes=1, weak_nodes=FFFC)
 │
│[142124.120284] drbd vm-122-disk-1 pve1: Committing remote state change
152764597
 │
│[142124.120289] drbd vm-122-disk-1: State change failed: Refusing to be
Outdated while Connected
│
│[142124.120399] drbd vm-122-disk-1/0 drbd103: Failed: disk( UpToDate ->
Outdated )
│
│[142124.120410] drbd vm-122-disk-1: FATAL: Local commit of prepared
152764597 failed!
 │
│[142124.350767] drbd vm-122-disk-1: Preparing cluster-wide state change
3667315219 (0->1 496/16)
│
│[142124.350948] drbd vm-122-disk-1: State change 3667315219:
primary_nodes=5, weak_nodes=FFFB
   │
│[142124.350949] drbd vm-122-disk-1 pve2: Cluster is now split

 │
│[142124.350950] drbd vm-122-disk-1: Committing cluster-wide state change
3667315219 (0ms)
   │
│[142124.350970] drbd vm-122-disk-1 pve2: conn( Connected -> Disconnecting
) peer( Secondary -> Unknown )
  │
│[142124.350973] drbd vm-122-disk-1/0 drbd103 pve2: pdsk( Diskless ->
DUnknown ) repl( Established -> Off )
│
│[142124.350994] drbd vm-122-disk-1 pve2: ack_receiver terminated

  │
│[142124.350996] drbd vm-122-disk-1 pve2: Terminating ack_recv thread

  │
│[142124.467248] drbd vm-122-disk-1 pve2: Connection closed

  │
│[142124.467428] drbd vm-122-disk-1 pve2: conn( Disconnecting -> StandAlone
)
  │
│[142124.467432] drbd vm-122-disk-1 pve2: Terminating receiver thread

  │
│[142124.467504] drbd vm-122-disk-1 pve2: Terminating sender thread

  │
│[142133.966538] drbd vm-122-disk-1 pve2: Starting sender thread (from
drbdsetup [1502468])
  │
│[142133.968349] drbd vm-122-disk-1/0 drbd103 pve2: Not enough free bitmap
slots
   │
│[142135.077014] drbd vm-122-disk-1/0 drbd103 pve2: Not enough free bitmap
slots
   │
│[142227.488145] drbd vm-122-disk-1/0 drbd103 pve2: Not enough free bitmap
slots
   │
│[142232.509729] drbd vm-122-disk-1/0 drbd103 pve2: Not enough free bitmap
slots
   │
│[142272.526580] drbd vm-122-disk-1/0 drbd103 pve2: Not enough free bitmap
slots
   │
│[143361.840005] drbd vm-122-disk-1 pve2: Terminating sender thread

  │
│[143505.002551] drbd vm-122-disk-1 pve2: Starting sender thread (from
drbdsetup [3396832])
  │
│[143505.004330] drbd vm-122-disk-1/0 drbd103 pve2: Not enough free bitmap
slots
   │
│[143506.090369] drbd vm-122-disk-1/0 drbd103 pve2: Not enough free bitmap
slots
   │
│[143633.980319] drbd vm-122-disk-1 pve2: Terminating sender thread
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Unable to init a node

[Yannis Milios]

>
> Yes, I have three nodes :
> root@dmz-pve1:~ # drbdmanage list-nodes
> +---
> -+
> | Name | Pool Size | Pool Free |
> | State |
> |---
> -|
> | dmz-pve1 |   unknown |   unknown |
> |ok |
> | dmz-pve2 |   unknown |   unknown |
> |ok |
> | dmz-pve3 |   unknown |   unknown |
> |ok |
> +---
> -+
> But pool size is unknown, I have a VG named drbdpool
>

Check your /etc/ drbdmanaged.cfg file and make sure that the storage type
defined  there, is configured properly.
Is it thin or thick LVM? If it's thin LVM how did you create it ?


- The control volume is happy and connected to the other nodes?
>
> Yes
>   LV Path/dev/drbdpool/.drbdctrl_0
>   LV Name.drbdctrl_0
>   VG Namedrbdpool
>   LV Path/dev/drbdpool/.drbdctrl_1
>   LV Name.drbdctrl_1
>   VG Namedrbdpool
> root@dmz-pve1:~ #  drbdadm status
> .drbdctrl role:Primary
>   volume:0 disk:UpToDate
>   volume:1 disk:UpToDate
>   dmz-pve2 role:Secondary
> volume:0 peer-disk:UpToDate
> volume:1 peer-disk:UpToDate
>   dmz-pve3 role:Secondary
> volume:0 peer-disk:UpToDate
> volume:1 peer-disk:UpToDate
> root@dmz-pve1:~ #  drbd-overview 2>/dev/null
>  0:.drbdctrl/0  Connected(3*) Prim(dmz-pve1)/Seco(dmz-pve2,dmz-pve3)
> UpTo(dmz-pve1)/UpTo(dmz-pve2,dmz-pve3)
>  1:.drbdctrl/1  Connected(3*) Prim(dmz-pve1)/Seco(dmz-pve2,dmz-pve3)
> UpTo(dmz-pve1)/UpTo(dmz-pve3,dmz-pve2)
>
>
Yes, control volumes seem to be OK and drbd nodes too, but that's unrelated
to the storage which you have dedicated to DRBD (/etc/drbdmanaged.cfg). You
need to make sure that, that one is also OK.



>
> - You can create resources on the command line and assign them?
>
> I don't understand your question.
> I try to create virtual machine on one node of the proxmox cluster via the
> proxmox web interface.
> I need information on the pve storage.cfg
>

Before adding/removing VMs based on DRBD storage, you have to make sure
that DRBD storage works first. This means you need to check drbd pool
status, and use   drbdmanage add-resource, drbdmanage add-volume commands
in CLI to add/remove DRBD volumes. If that does not work, then proxmox gui
will not work as well.


Y
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Upgrade PVE 4.4 to 5.1 DRB9

[Yannis Milios]

>>  Do you get new headers automatically when there is a new Proxmox kernel?

My experience has been that, no, PVE does not install the pve-headers-
automatically after each kernel upgrade (apt dist-upgrade).
So, what I need to do is 2 additional steps:

- Install pve-headers- matching the new kernel.
- Run apt install drbd-dkms --reinstall

Y
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Upgrade PVE 4.4 to 5.1 DRB9

[Yannis Milios]

A simple ‘apt install drbd-dkms —reinstall’ should work as well. Remember
repeating this task each time you upgrade your kernel/headers.


Y


-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Unable to init a node

[Yannis Milios]

>>  root@dmz-pve1:~ # modinfo drbd
>> filename:   /lib/modules/4.13.13-6-pve/kernel/drivers/block/drbd/
drbd.ko
>> alias:  block-major-147-*
>> license:GPL
>> version:8.4.7   < (this is wrong!)


Since version 5, PVE ships with drbd8 kernel module (see version section
above) . That's the reason you're getting errors during drbd cluster
initialization.
You will have to install drbd-dkms package from Linbit's repository in
order to get drbd9 kernel module installed. Make sure you install
pve-headers-x package before doing that, otherwise drbd9 kernel module
will fail to build.

Y
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Problems with LVM over DRBD.

[Yannis Milios]

Are you using drbd in dual-primary mode ? Can you post the configuration of
drbd resource (vm) and the output of "cat /proc/drbd" ?
Any clue in the logs ?

Y



On Fri, Feb 16, 2018 at 10:16 AM, Carles Xavier Munyoz Baldó <
car...@unlimitedmail.org> wrote:

> Hi,
> We have a two nodes Proxmox cluster, using as storage LVM virtual disks
> over DRBD.
>
> Some days ago we reinstalled this cluster from scratch with the last
> Proxomox version. For avoid service interruption we first reinstalled
> one of the nodes and started all virtual machines in it and then
> reinstalled the other node.
>
> Up here all right.
>
> We have the DRBD storage syncrhonized and updated in the second node but
> when we put the DRBD storage in primary role in this second node, it is
> not able to see nothing related with the LVM virtual disks. pvscan,
> vgscan or lvmscan return nothing about the LVM virtual disks created
> over the DRBD unit.
>
> We have verified that the filters in the /etc/lvm/lvm.conf file are
> correct. Indeed this file is identical in both cluster nodes.
>
> We don't understand why the reinstaled second node is not able to see
> the LVM virtual disks over the DRBD unit.
> May someone help us with this problem?
>
> Thank you very much in advance.
> Best regards.
> 
> SOLTECSIS SOLUCIONES TECNOLOGICAS, S.L.
> Carles Xavier Munyoz Baldó
> Departamento de I+D+I
> Tel./Fax: 966 446 046
> cmun...@soltecsis.com
> www.soltecsis.com
> 
>
> ---
> La información contenida en este e-mail es confidencial,
> siendo para uso exclusivo del destinatario arriba mencionado.
> Le informamos que está totalmente prohibida cualquier
> utilización, divulgación, distribución y/o reproducción de
> esta comunicación sin autorización expresa en virtud de la
> legislación vigente. Si ha recibido este mensaje por error,
> le rogamos nos lo notifique inmediatamente por la misma vía
> y proceda a su eliminación.
> ---
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Proxmox repo release.gpg expired

[Yannis Milios]

Thanks for the hint, works now ...


Y

On Thu, Feb 8, 2018 at 10:43 AM, Lars Ellenberg 
wrote:

> On Thu, Feb 08, 2018 at 10:06:00AM +0100, Christoph Lechleitner wrote:
> > Am 08.02.18 um 09:38 schrieb Yannis Milios:
> > > Can you please renew proxmox repo Release.gpg file ?
>
> The signature is fine.
> You need to refresh your keyring.
> How? See below.
>
> > > W: An error occurred during the signature verification. The repository
> > > is not updated and the previous index files will be used. GPG error:
> > > http://packages.linbit.com/proxmox proxmox-5 Release: The following
> > > signatures were invalid: EXPKEYSIG 53B3B037282B6E23 LINBIT Package and
> > > Repository Signing Key (2017)
> > > W: Failed to fetch
> > > http://packages.linbit.com/proxmox/dists/proxmox-5/Release.gpg  The
> > > following signatures were invalid: EXPKEYSIG 53B3B037282B6E23 LINBIT
> > > Package and Repository Signing Key (2017)
> > > W: Some index files failed to download. They have been ignored, or old
> > > ones used instead.
> >
> > Just in case this might help:
> >
> > Debian Stretch introduced 2 new requirements for signing keys and
> > signatures:
>
> Why would you think it would?
> Both requirements are fullfilled, and not being complained about.
> (Also someone would have noticed that before,
> stretch was not released yesterday).
>
> It complains about an *EXPKEYSIG*
> And there is also the "2017" as a hint.
> so: key expiry. Which the subject already says as well.
>
> We usually do not create a new key,
> but simply extend the validity for an other year,
> and also add the "current year" uid.
>
> Your system apparently did not notice the extended validity.
> You can help:
>
> apt-key list --verbose LINBIT
> | pub   dsa1024 2008-11-13 [SC] [expired: 2018-02-01]
> |32A7 46AD 3ACF B7EB 9A18  8D19 53B3 B037 282B 6E23
> | uid   [ expired] LINBIT Package and Repository Signing Key (2017)
> | ...
> | sub   elg2048 2008-11-13 [E] [expired: 2018-02-01]
> | gpg: Note: signature key 53B3B037282B6E23 expired Don 01 Feb 2018
> 11:49:14 CET
>
>
> apt-key adv --keyserver keyserver.ubuntu.com --recv-keys
> 0x53B3B037282B6E23
> (or your keyserver of choice)
> | gpg: key 53B3B037282B6E23: "LINBIT Package and Repository Signing Key
> (2018)" 1 new user ID
> | gpg: key 53B3B037282B6E23: "LINBIT Package and Repository Signing Key
> (2018)" 11 new signatures
> ...
>
> apt-key list --verbose LINBIT
> | pub   dsa1024 2008-11-13 [SC] [expires: 2019-02-01]
> |   32A7 46AD 3ACF B7EB 9A18  8D19 53B3 B037 282B 6E23
> | uid   [ unknown] LINBIT Package and Repository Signing Key (2018)
> | ...
> | sub   elg2048 2008-11-13 [E] [expires: 2019-02-01]
>
> Yay.
>
> --
> : Lars Ellenberg
> : LINBIT | Keeping the Digital World Running
> : DRBD -- Heartbeat -- Corosync -- Pacemaker
>
> DRBD® and LINBIT® are registered trademarks of LINBIT
> __
> please don't Cc me, but send to list -- I'm subscribed
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

[DRBD-user] Proxmox repo release.gpg expired

[Yannis Milios]

Can you please renew proxmox repo Release.gpg file ?

Thanks

W: An error occurred during the signature verification. The repository is
not updated and the previous index files will be used. GPG error:
http://packages.linbit.com/proxmox proxmox-5 Release: The following
signatures were invalid: EXPKEYSIG 53B3B037282B6E23 LINBIT Package and
Repository Signing Key (2017)
W: Failed to fetch
http://packages.linbit.com/proxmox/dists/proxmox-5/Release.gpg  The
following signatures were invalid: EXPKEYSIG 53B3B037282B6E23 LINBIT
Package and Repository Signing Key (2017)
W: Some index files failed to download. They have been ignored, or old ones
used instead.


Y
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Understanding "classic" 3 node set up.

[Yannis Milios]

Did you fixed the typo that Peter mentioned on his last post ? That should
be "internal" and not "inetrnal". Then must copy the resource file on all 3
nodes.


> >>  meta-disk   inetrnal;
>
>

After fixing that you should be able to create the metadata and initialize
the stacked device by:

On trk-kvm-01 (assuming that this is the primary node):
drbdadm create-md --stacked convirt-U

On trk-kvm-03:
drbdadm create-md convirt-U#this will create the required metadata on
the underlying device (i.e /dev/trk-kvm-03-vg/convirt). No need to transfer
metadata manually from trk-kvm-01)

On trk-kvm-01:
drbdadm --stacked adjust convirt-U

On trk-kvm-03:
drbdadm adjust convirt-U

On trk-kvm-01:
drbdadm --stacked -- --overwrite-data-of-peer primary convirt-U
cat /proc/drbd



Y
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Can't initialize DRBD9 in Proxmox 5.0 (drbdsetup new-resource exit code20)

[Yannis Milios]

Follow the steps in DRBD9 documentation carefully, in order to create and
initialise properly your DRBD9 cluster.

https://docs.linbit.com/docs/users-guide-9.0/#s-proxmox-overview

- Make sure you have installed properly drbd9 kernel modules/drbd-utils.
- Make sure you have created a VG named drbdpool *before* initialising the
cluster.
- Make sure you have configured  /etc/drbdmanaged.cfg properly.

On Thu, Jan 18, 2018 at 9:47 AM, Borja Muñoz 
wrote:

> Hello,
>
> I'm trying to setup DRBD9 in a new Proxmox 5.0 Cluster but I can't make it
> work. I have read some threads like this but the solution is not fully
> clear at all, so:
>
> The problem is the following:
>
> drbdmanage init
>
> *You are going to initialize a new drbdmanage cluster.*
> *CAUTION! Note that:*
> *  * Any previous drbdmanage cluster information may be removed*
> *  * Any remaining resources managed by a previous drbdmanage installation*
> *that still exist on this system will no longer be managed by
> drbdmanage*
>
> *Confirm:*
> *  yes/no: yes*
> *Error: External command failed:*
> *drbdsetup new-resource .drbdctrl 0*
> *Command output:*
> *  (stdout) *
> *Command exited with exit_code 20*
> *Initialization failed*
>
> I have installed the latest packages with: apt-get install drbd-dkms
> dbrd-manager pve-headers etc and I have loaded the module
> * dkms status*
> *drbd, 9.0.11-1: added*
>
> modprobe drbd
>
> # modinfo drbd
> *filename:   /lib/modules/4.10.17-2-pve/updates/dkms/drbd.ko*
> *alias:  block-major-147-**
> *license:GPL*
> *version:9.0.11-1*
> *description:drbd - Distributed Replicated Block Device v9.0.11-1*
> *author: Philipp Reisner  , Lars
> Ellenberg  *
> *srcversion: 9ACAAF3E3C9B36A172843DE*
> *depends:libcrc32c*
> *vermagic:   4.10.17-2-pve SMP mod_unload modversions *
> *parm:   enable_faults:int*
> *parm:   fault_rate:int*
> *parm:   fault_count:int*
> *parm:   fault_devs:int*
> *parm:   disable_sendpage:bool*
> *parm:   allow_oos:DONT USE! (bool)*
> *parm:   minor_count:Approximate number of drbd devices (1-255)
> (uint)*
>
> *parm:   usermode_helper:string *
>
> I don't know what to do anymore.
> Thanks
>
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] online verification: how to query completion status?

[Yannis Milios]

>> prior to drbd 9.x I could easily check how far online verification has come
by reading /proc/drbd.

That has been deprecated  on 9.x. To achieve something similar you will
have to either use drbdtop   or
otherwise you can have a look on this

thread.

Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Control deployment?

[Yannis Milios]

Sorry, now that I read your email more carefully, I get what you mean...

Obviously the quickest solution is to set replica to 3, but I guess that
you don’t want to do that, 1st because you don’t want to sacrfice storage
space and 2nd because you will still have the problem when you add a
4th,5th node and so on...

Not sure if drbd9 plugin is able to “detect” where the vm will be located
 during the initial resource creation so it can assign a diskful resource
below it. Maybe one of devs can give a better answer to this...

And to answer my part, yes I do that manually as well... :)

Yannis
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Control deployment?

[Yannis Milios]

>>(Now I am solving this by migrate vm to node1, unassigned vm-image from
node3, assign vm-image to node 3, migrate vm to node3, unassigned vm-image
from node1, whats awful, error prone and somewhat waste of time)

Why you are doing all these steps manually?! This is something that PVE
handles automatically via the DRBD9 plugin. Just a simple offline/online
live migration of VM from one node to the other should be enough to set
drbd resources in the correct mode...

Yannis
On Thu, 2 Nov 2017 at 13:40, Frank Rust  wrote:

> Hi all,
> I have a question concerning deployment.
>
> Lets assume I have cluster of three nodes of drbd9, all running
> virtualisation software (e.g. proxmox)
>
> node1 has 30TB of free storage
> node2 has 20TB of free storage
> node3 has 10TB of free storage.
>
> When I create a vm, residing on node3 with 80GB disk vm-image and
> redundancy 2, the system will put the image to node1 and node2, and assign
> it to node3 diskless.
> That’s working fine of course, but I would prefer to have the image
> preferably on the node running the vm. So in case of network failure the
> machine would have at least a standalone disk.
>
> But now I have the vm-image on node1 and node2 and diskless on node3. I
> can not simply "drbdmanage assign vm-image node3" because it is already
> there…
>
> Is there any reasonable way to get along with this?
>
> (Now I am solving this by migrate vm to node1, unassigned vm-image from
> node3, assign vm-image to node 3, migrate vm to node3, unassigned vm-image
> from node1, whats awful, error prone and somewhat waste of time)
>
> Best regards,
> Frank
>
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] query regarding existing data replication

[Yannis Milios]

If you mean that you want to preserve your existing data, then yes it's
possible. Read carefully this section in documentation:

http://docs.linbit.com/docs/users-guide-8.4/#s-prepare-storage

...and specifically the section  "It is not necessary for this storage area
to be empty before you create a DRBD resource from it. In fact it is a
common use case to create a two-node cluster from a previously
non-redundant single-server system using DRBD (some caveats apply — please
refer to DRBD meta data if you are planning to do this)."

Yannis

On Wed, Oct 11, 2017 at 5:42 PM, Sandeep  wrote:

> Hi,
> am new to DRBD and plan to replicate "application data" .
> wanted to understand - as to how to setup drbd when disks have
> "existing application data" .
>
> drbd document talks about "initial device synchronization" - how can i
> ensure the synchronization does remove/change data on the source ?
>
>  Regards
>
>  Sandeep S
>
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Some info

[Yannis Milios]

Are you planning to use DRBD8 or DRBD9?

DRBD8 is limited to 2 nodes(max 3).
DRBD9 can scale to multiple nodes.

For DRBD8 the most common setup is RAID -> DRBD -> LVM  or  RAID -> LVM ->
DRBD
It’s management is way easier than DRBD9.

The most common DRBD9 setups are RAID -> LVM (thin or thick) -> DRBD  or
 HDD  ->  ZFS (thin or thick)  ->  DRBD.
Complicated management...

On Wed, 11 Oct 2017 at 20:52, Gandalf Corvotempesta <
gandalf.corvotempe...@gmail.com> wrote:

> 2017-10-11 21:22 GMT+02:00 Adam Goryachev <
> mailingli...@websitemanagers.com.au>:
> > You can also do that with raid + lvm + drbd... you just need to create a
> new
> > drbd as you add a new LV, and also resize the drbd after you resize the
> LV.
>
> I prefere to keep drbd as minimum. I'm much more familiar with LVM.
> If not an issue, i prefere to keep the number of drbd resources as bare
> minimum.
>
> > If both drives fail on one node, then raid will pass the disk errors up
> to
> > DRBD, which will mark the local storage as down, and yes, it will read
> all
> > needed data from remote node (writes are always sent to the remote node).
> > You would probably want to migrate the remote node to primary as quickly
> as
> > possible, and then work on fixing the storage.
>
> Why should I migrate the remote node to primary? Any advantage?
>
> > Yes, it is not some bizarre configuration that has never been seen
> before.
> > You also haven't mentioned the size of your proposed raid, nor what size
> you
> > are planning on growing it to?
>
> Currently, I'm planning to start with 2TB disks. I don't think to go
> over 10-12TB
>
> > Yes, you will always want multiple network paths between the two nodes,
> and
> > also fencing. bonding can be used to improve performance, but you should
> > *also* have an additional network or serial or other connection between
> the
> > two nodes which is used for fencing.
>
> Ok.
>
> Any "bare-metal" distribution with DRBD or detailed guide on how to
> implement HA?
> Something like FreeNAS, or similiar.
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] DRBDv9 with iSCSI as scaleout SAN

[Yannis Milios]

In addition, as long as you're using proxmox, it would be way easier to
setup the native drbd9 plugin for proxmox instead of using the iscsi
method. In this case  both drbd and proxmox should be hosted on the same
servers (hyper-converged setup). Each vm will reside in a separate drbd9
resource/volume and you can control the redundancy level as well.

On Tue, Oct 3, 2017 at 1:04 PM, Adam Goryachev <
mailingli...@websitemanagers.com.au> wrote:

> Note, all the below relates to my uses of DRBD 8.4 in production. I'm
> assuming most of it will be equally applicable to DRBD9.
>
>
> On 3/10/17 19:52, Gandalf Corvotempesta wrote:
>
>> Just trying to figure out if drbd9 can do the job.
>>
>> Requirement: a scale-out storage for VMs image hosting (and other
>> services, but they would be made by creating, in example, an NFS VM on
>> top of DRBD)
>>
>> Let's assume a 3-nodes DRBDv9 cluster.
>> I would like to share this cluster by using iSCSI (or better protocol, if
>> any).
>> Multiple proxmox nodes sharing this drbd cluster.
>>
>> Probably, one drbd resource is created for each VM.
>>
>> Now, some question:
>>
>> how can I ensure that my iSCSI target is redundant across all nodes in
>> the cluster ?
>>
> What do you mean by redundant? You only have a single iscsi server, this
> is the current DRBD primary server. You would use heartbeat or similar to
> automatically stop the iscsi server, change primary to a different server,
> and then start iscsi server on that machine. Your iscsi clients will get no
> response during this time, which looks like a disk stall. Note, it's
> important to ensure you do this in the correct order:
> 1) Remove IP address (or firewall so that no response is sent back, no
> ICMP port closed message, no TCP packets, nothing at all).
> 2) Stop iscsi service
> 3) Change to secondary
> 4) Change other server to primary
> 5) Start iscsi service on new primary server
> 6) Add IP address, or fix firewall to allow traffic in/out.
>
>> When I have to add a fourth or fifth node to drbd cluster, should I
>> replicate the iscsi target configuration on both ?
>>
> Yes, you must ensure the iscsi config is identical on every server which
> could potentially become primary.
>
>> Will the drbd resources automatically rebalanced across the new nodes ?
>>
> I'm not sure, I suspect you are considering to make one of your DRBD nodes
> primary for some of the resources, and another primary for a different
> group of resources, and then somehow your peers will work out which primary
> to talk to for their iscsi service. This could be possible (thinking,
> definitely you will want to test this first).
>
> Consider if each DRBD resource will have a dedicated IP address. You will
> need to somehow dynamically configure the iscsi service (it is possible
> with iet and messing around in /proc) to listen on this extra IP, and serve
> this extra resource. Doing this individually for each resource (ie, the
> above 6 steps would be repeated once for each resource). However, I wonder
> if this would get you any significant benefit? All data will still need to
> be written to all servers, though I suppose reads will be better balanced
> than an all on one primary system.
>
> Should I change something in the iscsi/proxmox configuration after the
>> rebalance or is it transparent?
>>
> I'm thinking yes... I suspect your heartbeat layer will need to manage
> these changes for you.
>
>> Any pitfalls or drawbacks ?
>>
> Lots make sure you test a lot... including any and all failure
> modes you can think of, as well as a complete failure (all nodes die and
> recover).
>
> Hopefully someone with more hands on experience with DRBD9 can comment
> further
>
> Regards,
> Adam
>
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] drbdmanage quorum control

[Yannis Milios]

Thanks for clarifying this ...

Regards,
Yannis

On Tue, Oct 3, 2017 at 12:30 PM, Roland Kammerer  wrote:

> On Tue, Oct 03, 2017 at 12:05:50PM +0100, Yannis Milios wrote:
> > I think you have to use 'drbdmanage reelect' command to reelect a new
> > leader first.
> >
> > man drbdmanage-reelect
>
> In general that is a bad idea, and I regret that I exposed it as a
> subcommand and did not hide it behind a
> "--no-you-dont-want-that-unless-you-are-rck" where it then sill asks you
> to prove the Riemann hypothesis before continuing...
>
> > On Mon, Oct 2, 2017 at 2:12 PM, Jason Fitzpatrick <
> jayfitzpatr...@gmail.com>
> > wrote:
> >
> > > Hi all
> > >
> > > I am trying to get my head around the quorum-control features within
> > > drbdmanage,
> > >
> > > I have deliberately crashed my cluster, and spun up one node, and as
> > > expected I am unable to get drbdmanage to start due to the lack of
> > > quorum,,
> > >
> > > I was under the impression that I should have been able to override
> > > the quorum state and get the drbdmanaged process online using DBUS /
> > > manually calling the service, but am drawing a blank..
> > >
> > > for the sake of this example it is a 2 node cluster node1 is online
> > > and node2 is still powered off,
> > >
> > > [root@node1]# drbdmanage quorum-control --override ignore node2
> > > Modifying quorum state of node 'node2':
> > > Waiting for server: ...
> > > Error: Startup not successful (no quorum? not *both* nodes up in a 2
> > > node cluster?)
> > > Error: Startup not successful (no quorum? not *both* nodes up in a 2
> > > node cluster?)
> > >
> > > Any advice?
>
> Bring back the second node. In two node clusters that is the only clean
> way to bring back the cluster. If you want quorum, get >=3 nodes.
> Period. In two node clusters both have to be up. "reelect" is a last
> resort command for the absolute worst case to bring up a 2 node cluster
> where only one node survived and the other one is gone beyond repair.
> "reelect" with a forced win alters internal state to make that possible.
> It does not revert that internal state if, for whatever reason the
> second node then shows up again. You would have to restart the "reelect"
> node to get it then in a sane internal state again.
>
> tl;tr: If you want quorum: >=3 nodes. Don't use "reelect" to force wins.
>
> Regards, rck
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] drbdmanage quorum control

[Yannis Milios]

I think you have to use 'drbdmanage reelect' command to reelect a new
leader first.

man drbdmanage-reelect

Yannis



On Mon, Oct 2, 2017 at 2:12 PM, Jason Fitzpatrick 
wrote:

> Hi all
>
> I am trying to get my head around the quorum-control features within
> drbdmanage,
>
> I have deliberately crashed my cluster, and spun up one node, and as
> expected I am unable to get drbdmanage to start due to the lack of
> quorum,,
>
> I was under the impression that I should have been able to override
> the quorum state and get the drbdmanaged process online using DBUS /
> manually calling the service, but am drawing a blank..
>
> for the sake of this example it is a 2 node cluster node1 is online
> and node2 is still powered off,
>
> [root@node1]# drbdmanage quorum-control --override ignore node2
> Modifying quorum state of node 'node2':
> Waiting for server: ...
> Error: Startup not successful (no quorum? not *both* nodes up in a 2
> node cluster?)
> Error: Startup not successful (no quorum? not *both* nodes up in a 2
> node cluster?)
>
> Any advice?
>
> Thanks
>
> Jay
>
> --
>
> "The only difference between saints and sinners is that every saint
> has a past while every sinner has a future. "
> — Oscar Wilde
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] drbdmanage-proxmox v2.0

[Yannis Milios]

Tried, and it seems working, but only when taking snapshots of a vm
> which resides on a leader node.
>
>
I can reproduce this as well...
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] drbdmanage-proxmox v2.0

[Yannis Milios]

Sure, I get your point. Hope to see you come to an agreement again, there
must be a way :) (don't want to start flame wars).


On Wed, 20 Sep 2017 at 08:35, Roland Kammerer 
wrote:

> On Tue, Sep 19, 2017 at 05:26:02PM +0100, Yannis Milios wrote:
> > > I would also like to ask you as community which features you would like
> > > to see in future releases.
> >
> > * I guess integrating a simple 'cluster health status' page in PVE gui
> > is not a part of your development, right ?  I mean something like
> > showing the output of 'drbdadm status' or 'drbd-overview' in the gui.
>
> It certainly was not and I'm not sure how this would work for an
> external plugin. So far I did not see any hooks where a custom plugin
> could realize that functionality. (Not saying it does not exist, but I
> could not find it). So if there is some other "Custom" directory like
> "PVE/Storage/Custom", then it would be easy. But I really do not intend
> to modify/divert existing files in the proxmox packages (besides what we
> already did to get our plugin to Custom). Modifying files that don't
> belong to you is bad style from a packaging perspective and we had our
> fair share of "political discussions" ;-). Actually, I also would not be
> happy if someone diverts files in our packages...
>
> So, if there is a non-intrusive way to expose that information I did not
> find so far, fine. Otherwise I don't see that happen, sorry.
>
> Regards, rck
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] drbdmanage-proxmox v2.0

[Yannis Milios]

> I would also like to ask you as community which features you would like
> to see in future releases.

* I guess integrating a simple 'cluster health status' page in PVE gui is
not a part of your development, right ?
  I mean something like showing the output of 'drbdadm status' or
'drbd-overview' in the gui.
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] drbdmanage-proxmox v2.0

[Yannis Milios]

>
> In this release we added:
> - resize support
> - creating and deleting of snapshots.

 Still, we consider creating (and deleting)
> snapshots a good addition.


Perfect! thanks a lot


> I would also like to ask you as community which features you would like
> to see in future releases, except the obvious snapshot rollback
> functionality.
>
>
Personally, I'm not a fan of 'rollback'. I prefer the already available
'Clone' method (via CLI) of rolling back :) I believe it's much faster and
more convenient.
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] drbd cluster initialize

[Yannis Milios]

If this is a clean pve installation, then you need to enable their
repository first. By default it's not there ...

https://pve.proxmox.com/wiki/Package_Repositories#_proxmox_ve_no_subscription_repository

After adding it, you'll be able to find the kernel headers by doing a
'apt-get update' and then 'apt-cache search pve-headers*'.

Yannis



On Mon, Sep 18, 2017 at 1:55 PM, Roland Kammerer  wrote:

> On Mon, Sep 18, 2017 at 03:29:04PM +0300, Tsirkas Georgios wrote:
> > The current Proxmox5 repository, doesnt have kernel-headers which need
> for
> > drbd-dkms
>
> I have pve5 running with drbd-dkms...
>
> The headers are called pve-headers-something.
>
> Regards, rck
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Promote client-only to complete node ?

[Yannis Milios]

Usually when I need that, I live migrate the vm from the client node to the
other node and then I use drbdmanage unassign/assign to convert client to a
'normal' satellite node with local storage. Then wait for sync to complete
and move the vm back to the original node (if necessary).

Yannis

On Wed, 13 Sep 2017 at 08:27, Julien Escario 
wrote:

> Le 12/09/2017 à 11:39, Roland Kammerer a écrit :
> > On Tue, Sep 12, 2017 at 09:49:26AM +0200, Julien Escario wrote:
> >> Hello,
> >> I'm trying to 'promote' a client node to have a local copy of datas but
> can't
> >> find any reference to such command in manual.
> >>
> >> I tried :
> >> vm-260-disk-1 role:Secondary
> >>   disk:Diskless
> >>   vm4 role:Secondary
> >> peer-disk:UpToDate
> >>   vm5 role:Secondary
> >> peer-disk:UpToDate
> >>
> >> This one is in secondary state (VM not running) so I could
> unassign/assign this
> >> ressource to this node without problem.
> >>
> >> But if the ressource is already in primary state, any way to ask for a
> local
> >> copy of datas with drbdmanage ?
> >
> > Definitely not with drbdmanage. You have to get that node in Secondary
> > state and then drbdmanage unassign/assign.
>
> Thanks for this clear answer ! Perhaps something to add to the todo list ?
> From my point of view, technically, nothing seems to block this feature
> but in
> term of code base, this is probably harder.
>
> Best regards,
> Julien Escario
>
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Receiving a message "disk size of peer is too small"

[Yannis Milios]

>The partition takes all the disk, but in the new one, when I created it,
was a little smaller.

That's the case then. What about leaving the new disk unpartitioned and
using it as /dev/sdb (instead of /dev/sdb1) ?

Otherwise I guess 2 options remain:

- Buy a bigger disk
- Shrink the original in order to match the size of the new (dangerous):

https://docs.linbit.com/doc/users-guide-83/s-resizing/

-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Receiving a message "disk size of peer is too small"

[Yannis Milios]

On Tue, Sep 12, 2017 at 1:50 AM, José Andrés Matamoros Guevara <
amatamo...@ie-networks.com> wrote:

> I have two different servers with drbd replication. One disk failed in one
> of the servers. I have replaced it with a newer disk with physical sector
> size of 4096 bytes. The old one is 512 bytes.
>

That shouldn't matter..


>
>
> Now, I’m receiving a message saying “the disk size of peer is too small”
> and the resources don’t connect. If both disks are of the same size, is
> there any configuration I can make to make it work?
>
>
Depends how DRBD resource has been configured. Can you provide the .res
files ?


>
>
> I’m pasting the fdisk print information of both disks. Can I create a
> partition of the same size in the new disk? What can I do to solve this
> problem?
>
>
If the DRBD resource is configured to use partitions instead of a full
disk, then yes you need to create an equal or bigger partition on the new
disk (destionation).

Yannis
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] [DRBD9] on freshly created cluster, slave does not sync with master

[Yannis Milios]

Hi,

Control volumes on the slave node, 10.9.4.192 , show up in 'Inconsistent'
state 'before-adjust'. However 'after-adjust' they show up in 'Uptodate'
state on all nodes, that is the expected state.

Have you tried issuing 'drbdmanage restart -rq' on the slave node and then
'drbdmanage n' to check the status ? How about restarting the slave node ?

Yannis


On Thu, 7 Sep 2017 at 13:05, Zbigniew Kostrzewa 
wrote:

> Hi all,
>
> I am trying to setup 3-node cluster but constantly one of the nodes fail
> to sync with master. Here is what I do:
>
> 1. I create 3 VMs with CentOS 7.2 minimal (`3.10.0-327.el7.x86_64`)
> 2. I install DRBD9 RPMs I have built myself from following tags: drbd:
> 9.0.9, drbd-utils: 9.1.0, drbdmanage: 0.99.10
> 3. I setup SSH passwordless authentication between all 3 nodes
> 4. I setup hostname and add all hostnames to /etc/hosts on each node
> 5. On master, I run `drbdmanage init [MASTER_IP]`
> 6. On each slave I run `drbdmanage add-node [SLAVE_HOSTNAME] [SLAVE_IP]`
>
> ...and what happens is that first slave synchronizes with master
> properly but the second one hangs in "Connecting" state and is marked on
> master as "StandAlone":
>
> .drbdctrl role:Primary
>volume:0 disk:UpToDate
>volume:1 disk:UpToDate
>node-10-9-4-137 role:Secondary
>  volume:0 peer-disk:UpToDate
>  volume:1 peer-disk:UpToDate
>node-10-9-4-192 connection:StandAlone
>
> According to user's guide, this should work out-of-the-box but somehow
> it doesn't. Am I missing something? Should I do some pre-setup that's
> not mentioned in the user's guide?
>
> I've collected output from drbdadm, drbdsetup, dbus-monitor --system and
> syslog, if anyone would be willing to help me with this I have put the
> logs on github:
> - master:
>
> https://github.com/localghost/issues/tree/master/drbd/node_does_not_sync/before_adjust/10.9.4.119/logs
> - slaves:
>  * successful:
>
> https://github.com/localghost/issues/tree/master/drbd/node_does_not_sync/before_adjust/10.9.4.137/logs
>  * failing:
>
> https://github.com/localghost/issues/tree/master/drbd/node_does_not_sync/before_adjust/10.9.4.192/logs
>
> Thanks,
> Zbigniew Kostrzewa
>
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-- 
Sent from Gmail Mobile
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] drbdmanage for Proxmox 5.0

[Yannis Milios]

Yes, it's planned, check the below discussion:

https://lists.gt.net/drbd/users/29178

On Wed, Sep 6, 2017 at 2:48 PM, Michał Szamocki  wrote:

> Hello,
>
> Are there any plans to support Proxmox 5.0?
>
> Greetings,
>
> Michał Szamocki
> Cirrus - Aedificaremus Tibi
> WWW: http://www.cirrus.pl
> ___
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] DRBD over ZFS - or the other way around?

[Yannis Milios]

...I mean by cloning it first, since snapshot does not appear as blockdev
to the system but the clone does.

On Wed, Sep 6, 2017 at 2:58 PM, Yannis Milios 
wrote:

> Even in that case I would prefer to assemble a new DRBD device ontop of
> the ZVOL snapshot and then mount the DRBD device instead :)
>
> On Wed, Sep 6, 2017 at 2:56 PM, Gionatan Danti  wrote:
>
>> On 06/09/2017 15:31, Yannis Milios wrote:
>>
>>> If your topology is like the following:  HDD -> ZFS (ZVOL) -> DRBD ->
>>> XFS then I believe it should make sense to always mount at the DRBD level
>>> and not at the ZVOL level which happens to be the underlying blockdev for
>>> DRBD.
>>>
>> Sure! Directly mounting the DRBD-backing ZVOL would, at the bare minumum,
>> ruin the replication with the peer.
>>
>> I was speaking about mounting ZVOLs *snapshots* to access previous data
>> version.
>>
>> Regards.
>>
>>
>> --
>> Danti Gionatan
>> Supporto Tecnico
>> Assyoma S.r.l. - www.assyoma.it
>> email: g.da...@assyoma.it - i...@assyoma.it
>> GPG public key ID: FF5F32A8
>>
>
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] DRBD over ZFS - or the other way around?

[Yannis Milios]

Even in that case I would prefer to assemble a new DRBD device ontop of the
ZVOL snapshot and then mount the DRBD device instead :)

On Wed, Sep 6, 2017 at 2:56 PM, Gionatan Danti  wrote:

> On 06/09/2017 15:31, Yannis Milios wrote:
>
>> If your topology is like the following:  HDD -> ZFS (ZVOL) -> DRBD -> XFS
>> then I believe it should make sense to always mount at the DRBD level and
>> not at the ZVOL level which happens to be the underlying blockdev for DRBD.
>>
> Sure! Directly mounting the DRBD-backing ZVOL would, at the bare minumum,
> ruin the replication with the peer.
>
> I was speaking about mounting ZVOLs *snapshots* to access previous data
> version.
>
> Regards.
>
>
> --
> Danti Gionatan
> Supporto Tecnico
> Assyoma S.r.l. - www.assyoma.it
> email: g.da...@assyoma.it - i...@assyoma.it
> GPG public key ID: FF5F32A8
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] DRBD over ZFS - or the other way around?

[Yannis Milios]

If your topology is like the following:  HDD -> ZFS (ZVOL) -> DRBD -> XFS
then I believe it should make sense to always mount at the DRBD level and
not at the ZVOL level which happens to be the underlying blockdev for DRBD.

On Wed, Sep 6, 2017 at 12:28 PM, Jan Schermer  wrote:

> Not sure you can mount snapshot (I always create a clone).
> However I never saw anything about “drbd” filesystem - what distribution
> is this? Apparently it tries to be too clever…
> Try creating a clone and mounting it instead, it’s safer anyway (saw bug
> in issue tracker that ZFS panics if you try to write to the snapshot or
> something like that…)
>
> Other than that - yes, this should work fine.
>
> Jan
>
>
> > On 6 Sep 2017, at 13:23, Gionatan Danti  wrote:
> >
> > On 19/08/2017 10:24, Yannis Milios wrote:
> >> Option (b) seems more suitable for a 2 node drbd8 cluster in a
> primary/secondary setup. Haven't tried it so I cannot tell if there are any
> clurpits. My only concern in such setup would be if drbd corrupts silently
> the data on the lower level and zfs is not aware of that.Also, if you are
> *not* going to use live migration, and you can affort loosing some seconds
> of data on the secondary node in favor of better performance in the primary
> node, then you could consider using protocol A instead of C for the
> replication link.
> >
> > Hi all,
> > I "revive" this old thread to let you know I settled to use DRBD 8.4 on
> top of ZVOLs.
> >
> > I have a question for anyone using DRBD on top of a snapshot-capable
> backend (eg: ZFS, LVM, etc)...
> >
> > When snapshotting a DRBD block device, trying to mount it (the snapshot,
> not the original volume!) results in the following error message:
> >
> > [root@master7 tank]# mount /dev/zvol/tank/vol1\@snap1 /mnt/
> > mount: unknown filesystem type 'drbd'
> >
> > To successfully mount the snapshot volume, I need to specify the volume
> filesystem, for example (the other options are xfs-specific):
> >
> > [root@master7 tank]# mount -t xfs /dev/zvol/tank/vol1\@snap1 /mnt/ -o
> ro,norecovery,nouuid
> >
> > Is that the right approach? Or I am missing something?
> > Thanks.
> >
> > --
> > Danti Gionatan
> > Supporto Tecnico
> > Assyoma S.r.l. - www.assyoma.it
> > email: g.da...@assyoma.it - i...@assyoma.it
> > GPG public key ID: FF5F32A8
> > ___
> > drbd-user mailing list
> > drbd-user@lists.linbit.com
> > http://lists.linbit.com/mailman/listinfo/drbd-user
>
>
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user