Re: [DRBD-user] Some question about linstor

[yang zhenyu]

Hi，Robert
Now I use lvm storage pool to test. But there are still some problems. Is
there any problem with my operation?

[root@node-4 proto]# linstor list-node
╭╮
┊ Node   ┊ NodeType ┊ IPs┊ State ┊
╞╡
┊ node-1 ┊ COMBINED ┊ 10.10.0.72 ┊ ok┊
┊ node-4 ┊ COMBINED ┊ 10.10.0.75 ┊ ok┊
╰╯

[root@node-4 proto]# linstor list-resource
╭─╮
┊ ResourceName ┊ Node   ┊ Port ┊ Device   ┊ State ┊
╞┄╡
┊ rstdfn   ┊ node-1 ┊ 7000 ┊ drbdpool ┊ok ┊
┊ rstdfn   ┊ node-4 ┊ 7000 ┊ drbdpool ┊ok ┊
╰─╯

[root@node-4 proto]# linstor list-storage-pool
╭─╮
┊ StoragePool ┊ Node   ┊ Driver┊ DriverDevice ┊ SupportsSnapshots ┊
╞┄╡
┊ lvm ┊ node-1 ┊ LvmDriver ┊ drbdpool ┊ false ┊
┊ lvm ┊ node-4 ┊ LvmDriver ┊ drbdpool ┊ false ┊
╰─╯

[root@node-4 proto]# vgdisplay

 --- Volume group ---
  VG Name   drbdpool
  System ID
  Formatlvm2
  Metadata Areas1
  Metadata Sequence No  1
  VG Access read/write
  VG Status resizable
  MAX LV0
  Cur LV0
  Open LV   0
  Max PV0
  Cur PV1
  Act PV1
  VG Size   <30.00 GiB
  PE Size   4.00 MiB
  Total PE  7679
  Alloc PE / Size   0 / 0
  Free  PE / Size   7679 / <30.00 GiB
  VG UUID   QpdD46-Q4dh-qS7e-pSsX-JLos-gEJr-MX96EU

When I create volume, some error in log:

[root@node-4 proto]# linstor list-volume
╭──╮
┊ Node   ┊ Resource ┊ VolumeNr ┊ MinorNr ┊   State ┊
╞┄┄╡
┊ node-1 ┊ rstdfn   ┊ 0┊ 1000┊ not present ┊
┊ node-1 ┊ rstdfn   ┊ 1┊ 1001┊ not present ┊
┊ node-4 ┊ rstdfn   ┊ 0┊ 1000┊ not present ┊
┊ node-4 ┊ rstdfn   ┊ 1┊ 1001┊ not present ┊
╰──╯

Feb 24 15:09:16 node-4 Satellite: [MainWorkerPool_0009] ERROR
LINSTOR/Satellite - Missing required namespace for backing storage [Report
number A910AC3-01]

[root@node-4 linstor-server]# vi logs/ErrorReport-A910AC3-01.log

ERROR REPORT A910AC3-01



Application:LINBIT® LINSTOR
Module: Satellite
Version:0.1.1
Build ID:   b257fa2de27a92ee33a1d5a81f6da30ddca950e7
Build time: Wed Feb 14 12:49:01 UTC 2018
Error time: 2018-02-24 15:09:16



Reported error:
===

Category:   Error
Class name: ImplementationError
Class canonical name:   com.linbit.ImplementationError
Generated at:   Method 'reconfigureStorageDriver',
Source file 'StorPoolData.java', Line #304

Error message:  Missing required namespace for backing
storage

Call backtrace:

Method   Native Class:Line number
reconfigureStorageDriver N
 com.linbit.linstor.StorPoolData:304
getVolumeStorageDriver   N
 com.linbit.linstor.core.DrbdDeviceHandler:510
evaluateStorageVolumeN
 com.linbit.linstor.core.DrbdDeviceHandler:589
createResource   N
 com.linbit.linstor.core.DrbdDeviceHandler:756
dispatchResource N
 com.linbit.linstor.core.DrbdDeviceHandler:176
run  N
 com.linbit.linstor.core.DeviceManagerImpl$DeviceHandlerInvocation:904
run  N
 com.linbit.WorkerPool$WorkerThread:179


END OF ERROR REPORT.


2018-02-23 18:15 GMT+08:00 Robert Altnoeder :

> On 02/23/2018 10:55 AM, yang zhenyu wrote:
> >
> > [root@node-1 linstor-server]# zfs list
> > NAME USED AVAIL REFER MOUNTPOINT
> > linstorpool 384K 1.76T 96K /linstorpool
> >
> > Then I create a storage-pool, I don't understand the args
> > 'driver_device' means what?
> > Is a zfs zvol name? or zvol device name? like zd0...
>
> The driver_device is the name of a zpool that will be used to create
> storage in.
>
> However, the ZFS driver does not yet work properly, and it will probably
> only be fixed a few weeks from now because other higher priority
> development tasks are 

Re: [DRBD-user] Some question about linstor

[Robert Altnoeder]

On 02/23/2018 10:55 AM, yang zhenyu wrote:
>
> [root@node-1 linstor-server]# zfs list
> NAME USED AVAIL REFER MOUNTPOINT
> linstorpool 384K 1.76T 96K /linstorpool
>
> Then I create a storage-pool, I don't understand the args
> 'driver_device' means what?
> Is a zfs zvol name? or zvol device name? like zd0...

The driver_device is the name of a zpool that will be used to create
storage in.

However, the ZFS driver does not yet work properly, and it will probably
only be fixed a few weeks from now because other higher priority
development tasks are scheduled before fixing it.
It might work to some extent if a dummy volume is created in the zpool
before trying to use the zpool with LINSTOR.

> [root@node-1 linstor-server]# linstor create-resource -s zfs -d rstdfn
> node-1
> [root@node-1 linstor-server]# linstor create-resource -s zfs -d rstdfn
> node-4
> [root@node-1 linstor-server]# linstor list-resource
> ╭─╮
> ┊ ResourceName ┊ Node ┊ Port ┊ Device ┊ State ┊
> ╞┄╡
> ┊ rstdfn ┊ node-1 ┊ 7000 ┊ DISKLESS ┊ ok ┊
> ┊ rstdfn ┊ node-4 ┊ 7000 ┊ DISKLESS ┊ ok ┊
> ╰─╯
>
> Why this Device is DISKLESS?

Because you set it to diskless by using the '-d' option on the
create-resource command.

> [root@node-1 linstor-server]# linstor create-volume-definition -d 2
> rstdfn 30G

-d (aka deploy) is a leftover from drbdmanage that is not implemented
and does not belong in creation of volume definitions, this will be
removed in future versions.
(actually, the entire user interface will probably change significantly
in future versions)

> I do not know where is the problem. Is it network communication
> problems? But I `telnet node-4 3366` is no problem.

Information about most errors that occur can be found in LINSTOR's error
reports in its "logs" directory.

best regards,
-- 
Robert Altnoeder
+43 1 817 82 92 0
robert.altnoe...@linbit.com

LINBIT | Keeping The Digital World Running
DRBD - Corosync - Pacemaker
f /  t /  in /  g+

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Some question about linstor

[yang zhenyu]

Hi，Robert

Now the node status is ok, but integration zfs is still a problem, so I
still need your help.

Here are my steps:

[root@node-1 linstor-server]# linstor list-node
╭─╮
┊ Node ┊ NodeType ┊ IPs ┊ State ┊
╞┄╡
┊ node-1 ┊ COMBINED ┊ 10.10.0.72 ┊ ok ┊
┊ node-4 ┊ SATELLITE ┊ 10.10.0.75 ┊ ok ┊
╰─╯

I hava a zfs pool named linstorpool.

[root@node-1 linstor-server]# zfs list
NAME USED AVAIL REFER MOUNTPOINT
linstorpool 384K 1.76T 96K /linstorpool

Then I create a storage-pool, I don't understand the args 'driver_device'
means what?
Is a zfs zvol name? or zvol device name? like zd0...

usage: linstor create-storage-pool [-h]
name node_name {lvm,lvmthin,zfs}
driver_device

Here I am free to use a name 'test1'.

linstor create-storage-pool zfs node-1 zfs test1
[root@node-1 linstor-server]# linstor list-storage-pool
╭─╮
┊ StoragePool ┊ Node ┊ Driver ┊ DriverDevice ┊ SupportsSnapshots ┊
╞┄╡
┊ zfs ┊ node-1 ┊ ZfsDriver ┊ test1 ┊ true ┊
╰─╯
root@node-1 linstor-server]# linstor create-resource-definition rstdfn
[root@node-1 linstor-server]# linstor list-resource-definition
╭─╮
┊ ResourceName ┊ Port ┊ State ┊
╞┄╡
┊ rstdfn ┊ 7000 ┊ ok ┊
╰─╯
[root@node-1 linstor-server]# linstor create-resource -s zfs -d rstdfn
node-1
[root@node-1 linstor-server]# linstor create-resource -s zfs -d rstdfn
node-4
[root@node-1 linstor-server]# linstor list-resource
╭─╮
┊ ResourceName ┊ Node ┊ Port ┊ Device ┊ State ┊
╞┄╡
┊ rstdfn ┊ node-1 ┊ 7000 ┊ DISKLESS ┊ ok ┊
┊ rstdfn ┊ node-4 ┊ 7000 ┊ DISKLESS ┊ ok ┊
╰─╯

Why this Device is DISKLESS?

[root@node-1 linstor-server]# linstor create-volume-definition -d 2 rstdfn
30G
SUCCESS:
New volume definition with number '0' of resource definition 'rstdfn'
created.
WARNING:
Description:
Satellite reported an error during fullSync. This change will NOT be
delivered to satellte 'node-4' until the error is resolved. Reconnect the
satellite to the controller to remove this blockade.
Details:
Resource definition: rstdfn, Volume number: 0

[root@node-1 ~]# tailf /var/log/messages
Feb 23 17:38:52 node-1 Satellite: Adjusting resource rstdfn
Feb 23 17:38:52 node-1 kernel: drbd rstdfn: Starting worker thread (from
drbdsetup [4730])
Feb 23 17:38:52 node-1 kernel: drbd rstdfn node-4: Starting sender thread
(from drbdsetup [4735])
Feb 23 17:38:52 node-1 kernel: drbd rstdfn node-4: conn( StandAlone ->
Unconnected )
Feb 23 17:38:52 node-1 kernel: drbd rstdfn node-4: Starting receiver thread
(from drbd_w_rstdfn [4731])
Feb 23 17:38:52 node-1 kernel: drbd rstdfn node-4: conn( Unconnected ->
Connecting )

[root@node-1 linstor-server]# linstor list-volume
╭──╮
┊ Node ┊ Resource ┊ VolumeNr ┊ MinorNr ┊ State ┊
╞┄┄╡
┊ node-1 ┊ rstdfn ┊ 0 ┊ 1000 ┊ not present ┊
┊ node-4 ┊ rstdfn ┊ 0 ┊ 1000 ┊ unknown ┊
╰──╯

[root@node-1 linstor-server]# drbdadm status
rstdfn role:Secondary
disk:Diskless
node-4 connection:Connecting

[root@node-4 opt]# drbdadm status
# No currently configured DRBD found.

[root@node-4 opt]# cat /etc/drbd.d/rstdfn.res
cat: /etc/drbd.d/rstdfn.res: No such file or directory

I do not know where is the problem. Is it network communication problems?
But I `telnet node-4 3366` is no problem.

2018-02-22 19:02 GMT+08:00 Robert Altnoeder :

> Hello,
>
> On 02/22/2018 11:35 AM, yang zhenyu wrote:
> > [...]
> >
> > The following is my understanding:
> >
> > 1. The LINSTOR Controller node corresponds to the drbdmanage pure
> > controller, there is no storage.
> > 2. The LINSTOR Combined node corresponds to the drbdmanage controller
> > with storage.
> > 3. The LINSTOR Satellites node  corresponds to the drbdmanage
> > satellites with storage.
>
> LINSTOR comprises two modules, the controller and the satellite.
>
> Only the controller manages the configuration, e.g., all resource
> definitions, storage pools, network interfaces of nodes, etc. are
> defined on the controller.
>
> Only the satellites deal with actual storage, with DRBD, etc., and the
> satellites obtain information about the configuration of their resources
> from the active controller (the controller contacts all registered
> satellites).
>
> Only one controller is active at a time, but multiple installations of
> the controller may be provided so that if the active controller node
> crashes, the controller can be started on 

Re: [DRBD-user] Some question about linstor

[yang zhenyu]

Thanks, this really bothers me. Because there is no documentation, I
can only use the command or code to guide the use, if there is
obsolete code or command, it will be more trouble.

2018-02-22 17:05 GMT+08:00 Roland Kammerer :
> On Thu, Feb 22, 2018 at 02:09:39PM +0800, yang zhenyu wrote:
>> 3. Why some cmd return: 'This command is deprecated or not implemented', why?
>
> This is the one I can answer, I leave the rest to the linstor people.
> This is more or less a leftover from the old drbdmanage client. I
> started the linstor client (now mainly developed by Rene) as a "fork" of
> the drbdmanage client. I kept all the commands where I thought that we maybe
> need them in linstor as well and hardwired them to this "not
> implemented/deprecated" functions. So this means that they are not yet
> implemented or that they will be removed eventually.
>
> In general thanks for testing, the whole linstor thing is currently
> under heavy development. So except for specific use cases not everything
> is "production ready"...
>
> Regards, rck
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Some question about linstor

[yang zhenyu]

Hi，Robert

Thanks for your reply. I am a cloud compute developer. My company has
the intent of code-level cooperation. They should have been contacted
LINBIT office. But now I have to do some basic research first, confirm
that we want.

Through your reply, I probably understand some. I still need to experiment.

The following is my understanding:

1. The LINSTOR Controller node corresponds to the drbdmanage pure
controller, there is no storage.
2. The LINSTOR Combined node corresponds to the drbdmanage controller
with storage.
3. The LINSTOR Satellites node  corresponds to the drbdmanage
satellites with storage.
4. The LINSTOR Auxiliary node corresponds to the drbdmanage client node?
5. Only the control node to access the database? Currently linstor
only use the derby database? So I need a shared storage for multiple
control nodes. But I do not understand what is the ues of cluster
manage software at here? For example pacemaker.

Thanks again, if I have other questions I will ask you again.

2018-02-22 17:32 GMT+08:00 Robert Altnoeder :
> Hello,
>
> LINSTOR is still in an experimental phase of its development, so it is
> probably only useful to developers at this point.
>
> On 02/22/2018 07:09 AM, yang zhenyu wrote:
>> And did not find the corresponding guidance document.
> There is no documentation available for LINSTOR at this point
>> My questions:
>> 1. How to create a cluster of drbd.?
> Basically, you need the controller installed on at least one node and
> satellites installed on those nodes that run DRBD. If a controller and a
> satellite are running on the same node, that's known as a combined node.
>> 2. When I create a controller node, but it's state always is 'offline'. why?
> Online/offline refers to the satellite module on a node. Only one
> controller is online at any time. You need a cluster manager and either
> a database on shared storage, on replicated storage, or a replicated
> database, or a centralized database server for failover between multiple
> controllers.
>> 3. Why some cmd return: 'This command is deprecated or not implemented', why?
> Parts of the current command line client were ported from the older
> drbdmanage, and many old command entries still exist but serve no
> purpose. As far as I know, those have been removed in some newer version
> that might not have been pushed to public repositories yet.
>> 4. Can you give me some guide doc about using linstor?
> There is virtually zero documentation on LINSTOR available at this
> point. The only ones using it right now is a handful of people who are
> either in LINBIT or closely working together with us on planning future
> integration of LINSTOR into their projects.
>> [root@node-1 ~]# linstor list-node
>> ╭╮
>> ┊ Node   ┊ NodeType   ┊ IPs┊ State   ┊
>> ╞╡
>> ┊ node-1 ┊ CONTROLLER ┊ 10.10.0.72 ┊ OFFLINE ┊
>> ┊ node-4 ┊ CONTROLLER ┊ 10.10.0.75 ┊ OFFLINE ┊
>> ╰╯
> With only two controllers you can not create any resources. You'd need
> satellite nodes or combined nodes for the controllers or combined nodes
> to work with.
>> I use zfs:
> The ZFS storage driver is still untested (other than a few unit tests),
> so at this point, noone knows whether or not it actually works on any
> system other than its developer's workstation. The developer told me
> there are probably still some parts missing to make it work on most systems.
>> When I run the cmd in node-4:
>> [root@node-4 ~]# linstor list-node
>> ╭───╮
>> ┊ Node ┊ NodeType ┊ IPs ┊ State ┊
>> ╞┄┄┄╡
>> ╰───╯
>>
>> How to connect node-1 and node-4?
> Currently, the LINSTOR client must always be connected to the controller
> (it may be redirected there when attempting to connect it to a
> satellite), which connects to its registered satellites. Satellites do
> not see the entire configuration, they basically see only those objects
> that they need to know about. Controllers (re)connect to satellites
> automatically.
>
> Thanks for your interest and for giving it a try, but unfortunately it
> is probably still too early for the LINSTOR software to be useful to you
> (unless you are planning to set up some joint strategic project with
> LINBIT, if that's your goal, please contact our office directly).
> LINSTOR is still being developed and many things are still missing (the
> most important one probably being documentation).
>
> Best regards,
> --
> Robert Altnoeder
> +43 1 817 82 92 0
> robert.altnoe...@linbit.com
>
> LINBIT | Keeping The Digital World Running
> DRBD - Corosync - Pacemaker
> f /  t /  in /  g+
>
> DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Some question about linstor

[Robert Altnoeder]

Hello,

On 02/22/2018 11:35 AM, yang zhenyu wrote:
> [...]
>
> The following is my understanding:
>
> 1. The LINSTOR Controller node corresponds to the drbdmanage pure
> controller, there is no storage.
> 2. The LINSTOR Combined node corresponds to the drbdmanage controller
> with storage.
> 3. The LINSTOR Satellites node  corresponds to the drbdmanage
> satellites with storage.

LINSTOR comprises two modules, the controller and the satellite.

Only the controller manages the configuration, e.g., all resource
definitions, storage pools, network interfaces of nodes, etc. are
defined on the controller.

Only the satellites deal with actual storage, with DRBD, etc., and the
satellites obtain information about the configuration of their resources
from the active controller (the controller contacts all registered
satellites).

Only one controller is active at a time, but multiple installations of
the controller may be provided so that if the active controller node
crashes, the controller can be started on another node. This kind of
failover is managed by a cluster manager such as Pacemaker.

A combined node is a node that can run the controller module and the
satellite module. This is required if one of the storage nodes is also
running the controller module. In this case, the controller module and
the satellite module will be running on that node at the same time, and
the controller will connect to the satellite on the local node the same
way it connects to remote satellites.

The controller can also be installed on platforms that do not support
the kind of storage that the satellites support, or that do not support
DRBD (e.g., you could run the controller on an AIX or HP/UX box and have
it control satellites that run on Linux, and in the future also Windows
with DRBD for Windows).

> 4. The LINSTOR Auxiliary node corresponds to the drbdmanage client node?

Auxiliary nodes are more or less disabled nodes, they do not run the
LINSTOR software. They are needed for future features, as of now, they
are not very useful for anything.

> 5. Only the control node to access the database?

Only one controller is connected to the database at any time.

> Currently linstor only use the derby database? So I need a shared storage for 
> multiple
> control nodes.

Technically, the controller is able to connect to a
centralized/networked database (we run one of our development systems
with such a configuration), but normally it runs an Apache Derby
embedded database in the controller module's process. The database layer
is exchangable, but Derby is the only driver that is implemented as of
today.

> But I do not understand what is the ues of cluster manage software at here? 
> For example pacemaker.

It can be used for automatic failover to another controller if the
active controller node fails.

best regards,
-- 
Robert Altnoeder
+43 1 817 82 92 0
robert.altnoe...@linbit.com

LINBIT | Keeping The Digital World Running
DRBD - Corosync - Pacemaker
f /  t /  in /  g+

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Some question about linstor

[Robert Altnoeder]

Hello,

LINSTOR is still in an experimental phase of its development, so it is
probably only useful to developers at this point.

On 02/22/2018 07:09 AM, yang zhenyu wrote:
> And did not find the corresponding guidance document.
There is no documentation available for LINSTOR at this point
> My questions:
> 1. How to create a cluster of drbd.?
Basically, you need the controller installed on at least one node and
satellites installed on those nodes that run DRBD. If a controller and a
satellite are running on the same node, that's known as a combined node.
> 2. When I create a controller node, but it's state always is 'offline'. why?
Online/offline refers to the satellite module on a node. Only one
controller is online at any time. You need a cluster manager and either
a database on shared storage, on replicated storage, or a replicated
database, or a centralized database server for failover between multiple
controllers.
> 3. Why some cmd return: 'This command is deprecated or not implemented', why?
Parts of the current command line client were ported from the older
drbdmanage, and many old command entries still exist but serve no
purpose. As far as I know, those have been removed in some newer version
that might not have been pushed to public repositories yet.
> 4. Can you give me some guide doc about using linstor?
There is virtually zero documentation on LINSTOR available at this
point. The only ones using it right now is a handful of people who are
either in LINBIT or closely working together with us on planning future
integration of LINSTOR into their projects.
> [root@node-1 ~]# linstor list-node
> ╭╮
> ┊ Node   ┊ NodeType   ┊ IPs┊ State   ┊
> ╞╡
> ┊ node-1 ┊ CONTROLLER ┊ 10.10.0.72 ┊ OFFLINE ┊
> ┊ node-4 ┊ CONTROLLER ┊ 10.10.0.75 ┊ OFFLINE ┊
> ╰╯
With only two controllers you can not create any resources. You'd need
satellite nodes or combined nodes for the controllers or combined nodes
to work with.
> I use zfs:
The ZFS storage driver is still untested (other than a few unit tests),
so at this point, noone knows whether or not it actually works on any
system other than its developer's workstation. The developer told me
there are probably still some parts missing to make it work on most systems.
> When I run the cmd in node-4:
> [root@node-4 ~]# linstor list-node
> ╭───╮
> ┊ Node ┊ NodeType ┊ IPs ┊ State ┊
> ╞┄┄┄╡
> ╰───╯
>
> How to connect node-1 and node-4?
Currently, the LINSTOR client must always be connected to the controller
(it may be redirected there when attempting to connect it to a
satellite), which connects to its registered satellites. Satellites do
not see the entire configuration, they basically see only those objects
that they need to know about. Controllers (re)connect to satellites
automatically.

Thanks for your interest and for giving it a try, but unfortunately it
is probably still too early for the LINSTOR software to be useful to you
(unless you are planning to set up some joint strategic project with
LINBIT, if that's your goal, please contact our office directly).
LINSTOR is still being developed and many things are still missing (the
most important one probably being documentation).

Best regards,
-- 
Robert Altnoeder
+43 1 817 82 92 0
robert.altnoe...@linbit.com

LINBIT | Keeping The Digital World Running
DRBD - Corosync - Pacemaker
f /  t /  in /  g+

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] Some question about linstor

[Roland Kammerer]

On Thu, Feb 22, 2018 at 02:09:39PM +0800, yang zhenyu wrote:
> 3. Why some cmd return: 'This command is deprecated or not implemented', why?

This is the one I can answer, I leave the rest to the linstor people.
This is more or less a leftover from the old drbdmanage client. I
started the linstor client (now mainly developed by Rene) as a "fork" of
the drbdmanage client. I kept all the commands where I thought that we maybe
need them in linstor as well and hardwired them to this "not
implemented/deprecated" functions. So this means that they are not yet
implemented or that they will be removed eventually.

In general thanks for testing, the whole linstor thing is currently
under heavy development. So except for specific use cases not everything
is "production ready"...

Regards, rck
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user