Re: [RFC PATCH 19/28] lkl tools: host lib: virtio block device

2015-11-08 Thread Richard Weinberger 
Am 08.11.2015 um 05:15 schrieb Octavian Purdila:
> On Sat, Nov 7, 2015 at 2:24 PM, Richard Weinberger
>  wrote:
>> On Tue, Nov 3, 2015 at 9:20 PM, Octavian Purdila
>>  wrote:
>>> Host independent implementation for virtio block devices. The host
>>> dependent part of the host library must provide an implementation for
>>> lkl_dev_block_ops.
>>>
>>> Disks can be added to the LKL configuration via lkl_disk_add(), a new
>>> LKL application API.
>>>
>>> Signed-off-by: Octavian Purdila 
>>> ---
>>>  tools/lkl/include/lkl.h  |  20 
>>>  tools/lkl/include/lkl_host.h |  21 
>>>  tools/lkl/lib/virtio_blk.c   | 116 
>>> +++
>>>  3 files changed, 157 insertions(+)
>>>  create mode 100644 tools/lkl/lib/virtio_blk.c
>>
> 
> Hi Richard,
> 
>> Can you please outline what the differences between this driver and
>> UML's block driver are?
>>
> 
> LKL actually uses the standard virtio block driver, it does not
> implement a new (Linux kernel) driver.
> 
> This patch is the implementation for the host side device (in virtio's
> spec lingo).
> 
> Or maybe I misunderstood your question?

I just realized, that virtio does not depend on PCI, I thought you've 
implemented something
on your own. That said, UML could also use virtio block.

>> While UML and LKL have different goals they could at least share some 
>> drivers.
>> UML also has networking drivers you could reuse.
>> Maybe it would make sense to integrate LKL completely into arch/um if
>> we find a nice way
>> to combine them.
>> CONFIG_UML_LIBRARY, hmm?
>> Would be a nice opportunity to clear away some dung from arch/um and
>> refactor it. :-)
>>
> 
> Yeah, that would be nice :) I think the key part is to define the
> right host operations (in LKL terms) that can support UML. I'll have
> to spend some time to study UML's internals a bit to see if that would
> be doable.

arch/um/os-Linux/ is a good place to start. It implements most things on the 
host side.
Drivers like UML'S block driver always have foo_user.c and foo_kern.c part.

In case of any questions, just ask!

Thanks,
//richard
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [RFC PATCH 19/28] lkl tools: host lib: virtio block device

2015-11-08 Thread Richard Weinberger 
Am 08.11.2015 um 05:15 schrieb Octavian Purdila:
> On Sat, Nov 7, 2015 at 2:24 PM, Richard Weinberger
>  wrote:
>> On Tue, Nov 3, 2015 at 9:20 PM, Octavian Purdila
>>  wrote:
>>> Host independent implementation for virtio block devices. The host
>>> dependent part of the host library must provide an implementation for
>>> lkl_dev_block_ops.
>>>
>>> Disks can be added to the LKL configuration via lkl_disk_add(), a new
>>> LKL application API.
>>>
>>> Signed-off-by: Octavian Purdila 
>>> ---
>>>  tools/lkl/include/lkl.h  |  20 
>>>  tools/lkl/include/lkl_host.h |  21 
>>>  tools/lkl/lib/virtio_blk.c   | 116 
>>> +++
>>>  3 files changed, 157 insertions(+)
>>>  create mode 100644 tools/lkl/lib/virtio_blk.c
>>
> 
> Hi Richard,
> 
>> Can you please outline what the differences between this driver and
>> UML's block driver are?
>>
> 
> LKL actually uses the standard virtio block driver, it does not
> implement a new (Linux kernel) driver.
> 
> This patch is the implementation for the host side device (in virtio's
> spec lingo).
> 
> Or maybe I misunderstood your question?

I just realized, that virtio does not depend on PCI, I thought you've 
implemented something
on your own. That said, UML could also use virtio block.

>> While UML and LKL have different goals they could at least share some 
>> drivers.
>> UML also has networking drivers you could reuse.
>> Maybe it would make sense to integrate LKL completely into arch/um if
>> we find a nice way
>> to combine them.
>> CONFIG_UML_LIBRARY, hmm?
>> Would be a nice opportunity to clear away some dung from arch/um and
>> refactor it. :-)
>>
> 
> Yeah, that would be nice :) I think the key part is to define the
> right host operations (in LKL terms) that can support UML. I'll have
> to spend some time to study UML's internals a bit to see if that would
> be doable.

arch/um/os-Linux/ is a good place to start. It implements most things on the 
host side.
Drivers like UML'S block driver always have foo_user.c and foo_kern.c part.

In case of any questions, just ask!

Thanks,
//richard
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [RFC PATCH 19/28] lkl tools: host lib: virtio block device

2015-11-07 Thread Octavian Purdila 
On Sat, Nov 7, 2015 at 2:24 PM, Richard Weinberger
 wrote:
> On Tue, Nov 3, 2015 at 9:20 PM, Octavian Purdila
>  wrote:
>> Host independent implementation for virtio block devices. The host
>> dependent part of the host library must provide an implementation for
>> lkl_dev_block_ops.
>>
>> Disks can be added to the LKL configuration via lkl_disk_add(), a new
>> LKL application API.
>>
>> Signed-off-by: Octavian Purdila 
>> ---
>>  tools/lkl/include/lkl.h  |  20 
>>  tools/lkl/include/lkl_host.h |  21 
>>  tools/lkl/lib/virtio_blk.c   | 116 
>> +++
>>  3 files changed, 157 insertions(+)
>>  create mode 100644 tools/lkl/lib/virtio_blk.c
>

Hi Richard,

> Can you please outline what the differences between this driver and
> UML's block driver are?
>

LKL actually uses the standard virtio block driver, it does not
implement a new (Linux kernel) driver.

This patch is the implementation for the host side device (in virtio's
spec lingo).

Or maybe I misunderstood your question?

> While UML and LKL have different goals they could at least share some drivers.
> UML also has networking drivers you could reuse.
> Maybe it would make sense to integrate LKL completely into arch/um if
> we find a nice way
> to combine them.
> CONFIG_UML_LIBRARY, hmm?
> Would be a nice opportunity to clear away some dung from arch/um and
> refactor it. :-)
>

Yeah, that would be nice :) I think the key part is to define the
right host operations (in LKL terms) that can support UML. I'll have
to spend some time to study UML's internals a bit to see if that would
be doable.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [RFC PATCH 19/28] lkl tools: host lib: virtio block device

2015-11-07 Thread Richard Weinberger 
On Tue, Nov 3, 2015 at 9:20 PM, Octavian Purdila
 wrote:
> Host independent implementation for virtio block devices. The host
> dependent part of the host library must provide an implementation for
> lkl_dev_block_ops.
>
> Disks can be added to the LKL configuration via lkl_disk_add(), a new
> LKL application API.
>
> Signed-off-by: Octavian Purdila 
> ---
>  tools/lkl/include/lkl.h  |  20 
>  tools/lkl/include/lkl_host.h |  21 
>  tools/lkl/lib/virtio_blk.c   | 116 
> +++
>  3 files changed, 157 insertions(+)
>  create mode 100644 tools/lkl/lib/virtio_blk.c

Can you please outline what the differences between this driver and
UML's block driver are?

While UML and LKL have different goals they could at least share some drivers.
UML also has networking drivers you could reuse.
Maybe it would make sense to integrate LKL completely into arch/um if
we find a nice way
to combine them.
CONFIG_UML_LIBRARY, hmm?
Would be a nice opportunity to clear away some dung from arch/um and
refactor it. :-)

-- 
Thanks,
//richard
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [RFC PATCH 19/28] lkl tools: host lib: virtio block device

2015-11-07 Thread Richard Weinberger 
On Tue, Nov 3, 2015 at 9:20 PM, Octavian Purdila
 wrote:
> Host independent implementation for virtio block devices. The host
> dependent part of the host library must provide an implementation for
> lkl_dev_block_ops.
>
> Disks can be added to the LKL configuration via lkl_disk_add(), a new
> LKL application API.
>
> Signed-off-by: Octavian Purdila 
> ---
>  tools/lkl/include/lkl.h  |  20 
>  tools/lkl/include/lkl_host.h |  21 
>  tools/lkl/lib/virtio_blk.c   | 116 
> +++
>  3 files changed, 157 insertions(+)
>  create mode 100644 tools/lkl/lib/virtio_blk.c

Can you please outline what the differences between this driver and
UML's block driver are?

While UML and LKL have different goals they could at least share some drivers.
UML also has networking drivers you could reuse.
Maybe it would make sense to integrate LKL completely into arch/um if
we find a nice way
to combine them.
CONFIG_UML_LIBRARY, hmm?
Would be a nice opportunity to clear away some dung from arch/um and
refactor it. :-)

-- 
Thanks,
//richard
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [RFC PATCH 19/28] lkl tools: host lib: virtio block device

2015-11-07 Thread Octavian Purdila 
On Sat, Nov 7, 2015 at 2:24 PM, Richard Weinberger
 wrote:
> On Tue, Nov 3, 2015 at 9:20 PM, Octavian Purdila
>  wrote:
>> Host independent implementation for virtio block devices. The host
>> dependent part of the host library must provide an implementation for
>> lkl_dev_block_ops.
>>
>> Disks can be added to the LKL configuration via lkl_disk_add(), a new
>> LKL application API.
>>
>> Signed-off-by: Octavian Purdila 
>> ---
>>  tools/lkl/include/lkl.h  |  20 
>>  tools/lkl/include/lkl_host.h |  21 
>>  tools/lkl/lib/virtio_blk.c   | 116 
>> +++
>>  3 files changed, 157 insertions(+)
>>  create mode 100644 tools/lkl/lib/virtio_blk.c
>

Hi Richard,

> Can you please outline what the differences between this driver and
> UML's block driver are?
>

LKL actually uses the standard virtio block driver, it does not
implement a new (Linux kernel) driver.

This patch is the implementation for the host side device (in virtio's
spec lingo).

Or maybe I misunderstood your question?

> While UML and LKL have different goals they could at least share some drivers.
> UML also has networking drivers you could reuse.
> Maybe it would make sense to integrate LKL completely into arch/um if
> we find a nice way
> to combine them.
> CONFIG_UML_LIBRARY, hmm?
> Would be a nice opportunity to clear away some dung from arch/um and
> refactor it. :-)
>

Yeah, that would be nice :) I think the key part is to define the
right host operations (in LKL terms) that can support UML. I'll have
to spend some time to study UML's internals a bit to see if that would
be doable.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[RFC PATCH 19/28] lkl tools: host lib: virtio block device

2015-11-03 Thread Octavian Purdila 
Host independent implementation for virtio block devices. The host
dependent part of the host library must provide an implementation for
lkl_dev_block_ops.

Disks can be added to the LKL configuration via lkl_disk_add(), a new
LKL application API.

Signed-off-by: Octavian Purdila 
---
 tools/lkl/include/lkl.h  |  20 
 tools/lkl/include/lkl_host.h |  21 
 tools/lkl/lib/virtio_blk.c   | 116 +++
 3 files changed, 157 insertions(+)
 create mode 100644 tools/lkl/lib/virtio_blk.c

diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index 958614d..0c30b23 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -20,4 +20,24 @@ static inline long lkl_sys_lseek(unsigned int fd, 
__lkl__kernel_loff_t off,
  */
 const char *lkl_strerror(int err);
 
+/**
+ * lkl_disk_backstore - host dependend disk backstore
+ *
+ * @fd - an open file descriptor that can be used by preadv/pwritev; used by
+ * POSIX hosts
+ */
+union lkl_disk_backstore {
+   int fd;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * Must be called before calling lkl_start_kernel.
+ *
+ * @backstore - the disk backstore
+ * @returns a disk id (0 is valid) or a strictly negative value in case of 
error
+ */
+int lkl_disk_add(union lkl_disk_backstore backstore);
+
 #endif
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
index 26d3e43..2dafaa8 100644
--- a/tools/lkl/include/lkl_host.h
+++ b/tools/lkl/include/lkl_host.h
@@ -20,4 +20,25 @@ struct lkl_dev_buf {
unsigned int len;
 };
 
+extern struct lkl_dev_blk_ops lkl_dev_blk_ops;
+
+#define LKL_DEV_BLK_TYPE_READ  0
+#define LKL_DEV_BLK_TYPE_WRITE 1
+#define LKL_DEV_BLK_TYPE_FLUSH 4
+#define LKL_DEV_BLK_TYPE_FLUSH_OUT 5
+
+struct lkl_dev_blk_ops {
+   int (*get_capacity)(union lkl_disk_backstore bs,
+   unsigned long long *res);
+   void (*request)(union lkl_disk_backstore bs, unsigned int type,
+   unsigned int prio, unsigned long long sector,
+   struct lkl_dev_buf *bufs, int count);
+};
+
+#define LKL_DEV_BLK_STATUS_OK  0
+#define LKL_DEV_BLK_STATUS_IOERR   1
+#define LKL_DEV_BLK_STATUS_UNSUP   2
+
+void lkl_dev_blk_complete(struct lkl_dev_buf *bufs, unsigned char status,
+ int len);
 #endif
diff --git a/tools/lkl/lib/virtio_blk.c b/tools/lkl/lib/virtio_blk.c
new file mode 100644
index 000..3262f42
--- /dev/null
+++ b/tools/lkl/lib/virtio_blk.c
@@ -0,0 +1,116 @@
+#include 
+#include "virtio.h"
+
+struct virtio_blk_dev {
+   struct virtio_dev dev;
+   struct {
+   uint64_t capacity;
+   } config;
+   struct lkl_dev_blk_ops *ops;
+   union lkl_disk_backstore backstore;
+};
+
+struct virtio_blk_req_header {
+   uint32_t type;
+   uint32_t prio;
+   uint64_t sector;
+};
+
+struct virtio_blk_req_trailer {
+   uint8_t status;
+};
+
+static int blk_check_features(uint32_t features)
+{
+   if (!features)
+   return 0;
+
+   return -LKL_EINVAL;
+}
+
+void lkl_dev_blk_complete(struct lkl_dev_buf *bufs, unsigned char status,
+ int len)
+{
+   struct virtio_dev_req *req;
+   struct virtio_blk_req_trailer *f;
+
+   req = container_of(bufs - 1, struct virtio_dev_req, buf);
+
+   if (req->buf_count < 2) {
+   lkl_printf("virtio_blk: no status buf\n");
+   return;
+   }
+
+   if (req->buf[req->buf_count - 1].len != sizeof(*f)) {
+   lkl_printf("virtio_blk: bad status buf\n");
+   } else {
+   f = req->buf[req->buf_count - 1].addr;
+   f->status = status;
+   }
+
+   virtio_dev_complete(req, len);
+}
+
+static void blk_queue(struct virtio_dev *dev, struct virtio_dev_req *req)
+{
+   struct virtio_blk_req_header *h;
+   struct virtio_blk_dev *blk_dev;
+
+   if (req->buf[0].len != sizeof(struct virtio_blk_req_header)) {
+   lkl_printf("virtio_blk: bad header buf\n");
+   lkl_dev_blk_complete(>buf[1], LKL_DEV_BLK_STATUS_UNSUP, 0);
+   return;
+   }
+
+   h = req->buf[0].addr;
+   blk_dev = container_of(dev, struct virtio_blk_dev, dev);
+
+   blk_dev->ops->request(blk_dev->backstore, le32toh(h->type),
+ le32toh(h->prio), le32toh(h->sector),
+ >buf[1], req->buf_count - 2);
+}
+
+static struct virtio_dev_ops blk_ops = {
+   .check_features = blk_check_features,
+   .queue = blk_queue,
+};
+
+int lkl_disk_add(union lkl_disk_backstore backstore)
+{
+   struct virtio_blk_dev *dev;
+   unsigned long long capacity;
+   int ret;
+   static int count;
+
+   dev = lkl_host_ops.mem_alloc(sizeof(*dev));
+   if (!dev)
+   return -LKL_ENOMEM;
+
+   dev->dev.device_id = 2;
+   dev->dev.vendor_id = 0;
+

[RFC PATCH 19/28] lkl tools: host lib: virtio block device

2015-11-03 Thread Octavian Purdila 
Host independent implementation for virtio block devices. The host
dependent part of the host library must provide an implementation for
lkl_dev_block_ops.

Disks can be added to the LKL configuration via lkl_disk_add(), a new
LKL application API.

Signed-off-by: Octavian Purdila 
---
 tools/lkl/include/lkl.h  |  20 
 tools/lkl/include/lkl_host.h |  21 
 tools/lkl/lib/virtio_blk.c   | 116 +++
 3 files changed, 157 insertions(+)
 create mode 100644 tools/lkl/lib/virtio_blk.c

diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index 958614d..0c30b23 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -20,4 +20,24 @@ static inline long lkl_sys_lseek(unsigned int fd, 
__lkl__kernel_loff_t off,
  */
 const char *lkl_strerror(int err);
 
+/**
+ * lkl_disk_backstore - host dependend disk backstore
+ *
+ * @fd - an open file descriptor that can be used by preadv/pwritev; used by
+ * POSIX hosts
+ */
+union lkl_disk_backstore {
+   int fd;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * Must be called before calling lkl_start_kernel.
+ *
+ * @backstore - the disk backstore
+ * @returns a disk id (0 is valid) or a strictly negative value in case of 
error
+ */
+int lkl_disk_add(union lkl_disk_backstore backstore);
+
 #endif
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
index 26d3e43..2dafaa8 100644
--- a/tools/lkl/include/lkl_host.h
+++ b/tools/lkl/include/lkl_host.h
@@ -20,4 +20,25 @@ struct lkl_dev_buf {
unsigned int len;
 };
 
+extern struct lkl_dev_blk_ops lkl_dev_blk_ops;
+
+#define LKL_DEV_BLK_TYPE_READ  0
+#define LKL_DEV_BLK_TYPE_WRITE 1
+#define LKL_DEV_BLK_TYPE_FLUSH 4
+#define LKL_DEV_BLK_TYPE_FLUSH_OUT 5
+
+struct lkl_dev_blk_ops {
+   int (*get_capacity)(union lkl_disk_backstore bs,
+   unsigned long long *res);
+   void (*request)(union lkl_disk_backstore bs, unsigned int type,
+   unsigned int prio, unsigned long long sector,
+   struct lkl_dev_buf *bufs, int count);
+};
+
+#define LKL_DEV_BLK_STATUS_OK  0
+#define LKL_DEV_BLK_STATUS_IOERR   1
+#define LKL_DEV_BLK_STATUS_UNSUP   2
+
+void lkl_dev_blk_complete(struct lkl_dev_buf *bufs, unsigned char status,
+ int len);
 #endif
diff --git a/tools/lkl/lib/virtio_blk.c b/tools/lkl/lib/virtio_blk.c
new file mode 100644
index 000..3262f42
--- /dev/null
+++ b/tools/lkl/lib/virtio_blk.c
@@ -0,0 +1,116 @@
+#include 
+#include "virtio.h"
+
+struct virtio_blk_dev {
+   struct virtio_dev dev;
+   struct {
+   uint64_t capacity;
+   } config;
+   struct lkl_dev_blk_ops *ops;
+   union lkl_disk_backstore backstore;
+};
+
+struct virtio_blk_req_header {
+   uint32_t type;
+   uint32_t prio;
+   uint64_t sector;
+};
+
+struct virtio_blk_req_trailer {
+   uint8_t status;
+};
+
+static int blk_check_features(uint32_t features)
+{
+   if (!features)
+   return 0;
+
+   return -LKL_EINVAL;
+}
+
+void lkl_dev_blk_complete(struct lkl_dev_buf *bufs, unsigned char status,
+ int len)
+{
+   struct virtio_dev_req *req;
+   struct virtio_blk_req_trailer *f;
+
+   req = container_of(bufs - 1, struct virtio_dev_req, buf);
+
+   if (req->buf_count < 2) {
+   lkl_printf("virtio_blk: no status buf\n");
+   return;
+   }
+
+   if (req->buf[req->buf_count - 1].len != sizeof(*f)) {
+   lkl_printf("virtio_blk: bad status buf\n");
+   } else {
+   f = req->buf[req->buf_count - 1].addr;
+   f->status = status;
+   }
+
+   virtio_dev_complete(req, len);
+}
+
+static void blk_queue(struct virtio_dev *dev, struct virtio_dev_req *req)
+{
+   struct virtio_blk_req_header *h;
+   struct virtio_blk_dev *blk_dev;
+
+   if (req->buf[0].len != sizeof(struct virtio_blk_req_header)) {
+   lkl_printf("virtio_blk: bad header buf\n");
+   lkl_dev_blk_complete(>buf[1], LKL_DEV_BLK_STATUS_UNSUP, 0);
+   return;
+   }
+
+   h = req->buf[0].addr;
+   blk_dev = container_of(dev, struct virtio_blk_dev, dev);
+
+   blk_dev->ops->request(blk_dev->backstore, le32toh(h->type),
+ le32toh(h->prio), le32toh(h->sector),
+ >buf[1], req->buf_count - 2);
+}
+
+static struct virtio_dev_ops blk_ops = {
+   .check_features = blk_check_features,
+   .queue = blk_queue,
+};
+
+int lkl_disk_add(union lkl_disk_backstore backstore)
+{
+   struct virtio_blk_dev *dev;
+   unsigned long long capacity;
+   int ret;
+   static int count;
+
+   dev = lkl_host_ops.mem_alloc(sizeof(*dev));
+   if (!dev)
+   return -LKL_ENOMEM;
+
+   dev->dev.device_id = 2;
+   dev->dev.vendor_id =