Re: [PATCH] fs/coda: remove useless varible 'err'
On Fri, Nov 06, 2020 at 10:57:08PM +0800, Alex Shi wrote: > The variable is unused and cause gcc warning: > fs/coda/file.c:241:6: warning: variable ‘err’ set but not used > [-Wunused-but-set-variable] > So let's remove it. > Signed-off-by: Alex Shi > Cc: Jan Harkes > Cc: c...@cs.cmu.edu > Cc: codal...@coda.cs.cmu.edu > Cc: linux-kernel@vger.kernel.org Looks good to me. Acked-by: Jan Harkes > --- > fs/coda/file.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/fs/coda/file.c b/fs/coda/file.c > index 128d63df5bfb..31a7ba383cb2 100644 > --- a/fs/coda/file.c > +++ b/fs/coda/file.c > @@ -238,11 +238,10 @@ int coda_release(struct inode *coda_inode, struct file > *coda_file) > struct coda_file_info *cfi; > struct coda_inode_info *cii; > struct inode *host_inode; > - int err; > > cfi = coda_ftoc(coda_file); > > - err = venus_close(coda_inode->i_sb, coda_i2f(coda_inode), > + venus_close(coda_inode->i_sb, coda_i2f(coda_inode), > coda_flags, coda_file->f_cred->fsuid); > > host_inode = file_inode(cfi->cfi_container); > -- > 1.8.3.1 > >
Re: [PATCH v3 00/29] Convert files to ReST - part 2
On Tue, Apr 28, 2020 at 03:09:51PM -0400, Jonathan Corbet wrote: > So I'm happy to merge this set, but there is one thing that worries me a > bit... > > > fs/coda/Kconfig |2 +- > > I'd feel a bit better if I could get an ack or two from filesystem folks > before I venture that far out of my own yard...what say you all? I acked the Coda parts on the first iteration of this patch. I have no problem with you merging them. Jan
Re: [RFC PATCH 1/6] vfs, coda: Fix the lack of locking in FID replacement inode rehashing
That looks good to me. Acked-by: Jan Harkes On April 18, 2019 10:06:07 AM EDT, David Howells wrote: >- but, as the comment notes, this really needs some locking. > >Fix this by moving the core part of the code to fs/inode.c and
Re: fs/coda oops bisected to (925b9cd1b8) "locking/rwsem: Make owner store task pointer of last owni
On Sun, Mar 31, 2019 at 03:13:47PM -0400, Jan Harkes wrote:
> On Sun, Mar 31, 2019 at 02:14:13PM -0400, Waiman Long wrote:
> > One possibility is that there is a previous reference to the memory
> > currently occupied by the spinlock. If the memory location is previously
> > part of a rwsem structure and someone is still using it, you may get
> > memory corruption.
>
> Ah, I hadn't even thought of that possibility. Good, it will open up
First of all, I have to thank you for your original patch because
otherwise I probably would never have discovered that something was
seriously wrong. Your patch made the problem visible.
I ended up changing 'owner' to '_RET_IP_' and dumping the value of the
clobbered coda inode spinlock and surrounding memory and found that the
'culprit' is in ext4_filemap_fault and despite it being in ext4, it is
still a Coda specific problem.
Effectively Coda overlays other filesystems' inodes for mmap, but
the vma->vm_file still points at Coda's file. So when we use
file_inode() in ext4_filemap_fault we end up with the Coda inode instead
of the ext4 inode and when trying to grab ext4's mmap_sem we really just
scribble over the memory region that happens to contain the Coda inode
spinlock. A fix is to use vm_file->f_mapping->host instead of
file_inode(vm_file).
Of course everyone looks at ext4 as a canonical example so this problem
has spread pretty much everywhere and I'm wondering how to best resolve
this.
- change file_inode() to follow file->f_mapping->host
would fix most places, but maybe f_mapping is not always guaranteed to
point at a usable place?
- change Coda's mmap to replace vma->vm_file with the host file
we'd probably no longer get notified when the last reference to the
host file goes away, so we'd call coda_release and notify userspace on
close() even when there are still active mmap regions.
- fix every in-tree file system to use vma->vm_file->f_mapping->host.
Jan
diff --git a/fs/ext4/file.c b/fs/ext4/file.c
index 69d65d49837b..122d691d3eda 100644
--- a/fs/ext4/file.c
+++ b/fs/ext4/file.c
@@ -284,7 +284,7 @@ static vm_fault_t ext4_dax_huge_fault(struct vm_fault *vmf,
vm_fault_t result;
int retries = 0;
handle_t *handle = NULL;
- struct inode *inode = file_inode(vmf->vma->vm_file);
+ struct inode *inode = vmf->vma->vm_file->f_mapping->host;
struct super_block *sb = inode->i_sb;
/*
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index b54b261ded36..62a0025ce7f8 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -6211,7 +6211,7 @@ vm_fault_t ext4_page_mkwrite(struct vm_fault *vmf)
int err;
vm_fault_t ret;
struct file *file = vma->vm_file;
- struct inode *inode = file_inode(file);
+ struct inode *inode = file->f_mapping->host;
struct address_space *mapping = inode->i_mapping;
handle_t *handle;
get_block_t *get_block;
@@ -6302,7 +6302,7 @@ vm_fault_t ext4_page_mkwrite(struct vm_fault *vmf)
vm_fault_t ext4_filemap_fault(struct vm_fault *vmf)
{
- struct inode *inode = file_inode(vmf->vma->vm_file);
+ struct inode *inode = vmf->vma->vm_file->f_mapping->host;
vm_fault_t ret;
down_read(_I(inode)->i_mmap_sem);
Re: fs/coda oops bisected to (925b9cd1b8) "locking/rwsem: Make owner store task pointer of last owni
On Sun, Mar 31, 2019 at 02:14:13PM -0400, Waiman Long wrote: > On 03/31/2019 12:00 AM, Jan Harkes wrote: > > On Fri, Mar 29, 2019 at 05:53:22PM +, Waiman Long wrote: > >> On 03/29/2019 12:10 PM, Jan Harkes wrote: > >>> I knew I definitely had never seen this problem with the stable kernel > >>> on Ubuntu xenial (4.4) so I bisected between v4.4 and v5.1-rc2 and ended > >>> up at > >>> > >>> # first bad commit: [925b9cd1b89a94b7124d128c80dfc48f78a63098] > >>> # locking/rwsem: Make owner store task pointer of last owning reader > >>> > >>> When I revert this particular commit on 5.1-rc2, I am not able to > >>> reproduce the problem anymore. > >> Without CONFIG_DEBUG_RWSEMS, the only behavioral change of this patch is > >> to do an unconditional write of task_structure pointer into sem->owner > >> after acquiring the read lock in down_read(). Before this patch, it does > > > > I tried with just that change, but that is not at fault. It is also hard > > to believe we have a use-after-free issue, because we are using a > > spinlock on the inode that is held in place by the file we are releasing. > > One possibility is that there is a previous reference to the memory > currently occupied by the spinlock. If the memory location is previously > part of a rwsem structure and someone is still using it, you may get > memory corruption. Ah, I hadn't even thought of that possibility. Good, it will open up some new places to look at. Since I can open, read, write and close files in Coda without problems and so far it only seems to be related to closing the fd as a result of an executable process exiting. > > - unsigned long val = (unsigned long)owner | RWSEM_READER_OWNED > > -| > > RWSEM_ANONYMOUSLY_OWNED; > > + unsigned long val = RWSEM_READER_OWNED | > > RWSEM_ANONYMOUSLY_OWNED; > > The change above clears all the upper bytes of owner to 0, while the > original code will write some non-zero values to the upper bytes. Yes, and write locks will still write to all bits of the owner field. So the clobber must then be caused by something trying to grab a read lock on a rw_semaphore that has been freed, and whose memory got reused by the inode that was allocated when starting the executable. That seems like a specific enough thing that I hopefully will be able to find it. > > I'll continue digging if I can find a reason why. So far I've only found > > one place where rwsem->owner is modified while not holding a lock, but > > changing that doesn't make a difference for my particular case. ... > The rwsem is still locked. The above code releases the current task from > being the lock holder of the rwsem from lockdep's perspective. Later on Ah that explains why that change made no difference. I'm unfamiliar with the details of the locking code and assumed it was releasing some sort of write lock there, my bad. Jan
Re: fs/coda oops bisected to (925b9cd1b8) "locking/rwsem: Make owner store task pointer of last owni
On Fri, Mar 29, 2019 at 05:53:22PM +, Waiman Long wrote:
> On 03/29/2019 12:10 PM, Jan Harkes wrote:
> > I knew I definitely had never seen this problem with the stable kernel
> > on Ubuntu xenial (4.4) so I bisected between v4.4 and v5.1-rc2 and ended
> > up at
> >
> > # first bad commit: [925b9cd1b89a94b7124d128c80dfc48f78a63098]
> > # locking/rwsem: Make owner store task pointer of last owning reader
> >
> > When I revert this particular commit on 5.1-rc2, I am not able to
> > reproduce the problem anymore.
>
> Without CONFIG_DEBUG_RWSEMS, the only behavioral change of this patch is
> to do an unconditional write of task_structure pointer into sem->owner
> after acquiring the read lock in down_read(). Before this patch, it does
I tried with just that change, but that is not at fault. It is also hard
to believe we have a use-after-free issue, because we are using a
spinlock on the inode that is held in place by the file we are releasing.
After trying various variations the minimal change that fixes the soft
lockup is as follows. Without this patch I get a reliable lockup, with
the patch everything works as expected.
diff --git a/kernel/locking/rwsem.h b/kernel/locking/rwsem.h
index bad2bca..0cc437d 100644
--- a/kernel/locking/rwsem.h
+++ b/kernel/locking/rwsem.h
@@ -61,8 +61,7 @@ static inline void rwsem_clear_owner(struct rw_semaphore
*sem)
static inline void __rwsem_set_reader_owned(struct rw_semaphore *sem,
struct task_struct *owner)
{
- unsigned long val = (unsigned long)owner | RWSEM_READER_OWNED
-| RWSEM_ANONYMOUSLY_OWNED;
+ unsigned long val = RWSEM_READER_OWNED | RWSEM_ANONYMOUSLY_OWNED;
WRITE_ONCE(sem->owner, (struct task_struct *)val);
}
I'll continue digging if I can find a reason why. So far I've only found
one place where rwsem->owner is modified while not holding a lock, but
changing that doesn't make a difference for my particular case.
diff --git a/include/linux/percpu-rwsem.h b/include/linux/percpu-rwsem.h
index 03cb4b6f842e..fe696a8b57f3 100644
--- a/include/linux/percpu-rwsem.h
+++ b/include/linux/percpu-rwsem.h
@@ -114,11 +114,11 @@ extern void percpu_free_rwsem(struct
percpu_rw_semaphore *);
static inline void percpu_rwsem_release(struct percpu_rw_semaphore *sem,
bool read, unsigned long ip)
{
- lock_release(>rw_sem.dep_map, 1, ip);
#ifdef CONFIG_RWSEM_SPIN_ON_OWNER
if (!read)
sem->rw_sem.owner = RWSEM_OWNER_UNKNOWN;
#endif
+ lock_release(>rw_sem.dep_map, 1, ip);
}
static inline void percpu_rwsem_acquire(struct percpu_rw_semaphore *sem,
Jan
fs/coda oops bisected to (925b9cd1b8) "locking/rwsem: Make owner store task pointer of last owning reader"
I was testing Coda on the 5.1-rc2 kernel and noticed that when I run a binary out of /coda, the binary would never exit and the system would detect a soft lockup. I narrowed it down to a very simple reproducible case of running a statically linked executable (busybox) from /coda with the cwd outside of Coda, so the only Coda file reference is from the executable itself. I knew I definitely had never seen this problem with the stable kernel on Ubuntu xenial (4.4) so I bisected between v4.4 and v5.1-rc2 and ended up at # first bad commit: [925b9cd1b89a94b7124d128c80dfc48f78a63098] # locking/rwsem: Make owner store task pointer of last owning reader When I revert this particular commit on 5.1-rc2, I am not able to reproduce the problem anymore. The puzzling thing to me is that a lot of that particular patch touches codepaths that are not even enabled in the kernels that I run, because I do not have CONFIG_RWSEM_DEBUG enabled. $ grep RWSEM .config CONFIG_RWSEM_XCHGADD_ALGORITHM=y CONFIG_RWSEM_SPIN_ON_OWNER=y # CONFIG_DEBUG_RWSEMS is not set And this patch is for rwsem, while my soft lockup is on a spinlock. So either I have a race in fs/coda that got somehow uncovered by this patch, or something else is going on here but I have not been able to figure it out. Jan Here is the Oops that reliably triggers on any kernel after 925b9cd1b8 when I execute a binary from /coda. It repeats a second time 28 seconds later and another 5 seconds later is followed by "rcu_sched self-detected stall on CPU" which I include as well. [ 64.145934] watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [busybox:1814] [ 64.145938] Modules linked in: snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_pcm kvm_intel kvm irqbypass crct10dif_pclmul snd_hwdep crc32_pclmul ghash_clmulni_intel snd_seq snd_timer aesni_intel joydev snd_seq_device aes_x86_64 crypto_simd cryptd glue_helper snd input_leds soundcore serio_raw i2c_piix4 mac_hid parport_pc ppdev lp parport coda hid_generic usbhid hid qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm psmouse pata_acpi floppy [ 64.145958] CPU: 4 PID: 1814 Comm: busybox Not tainted 4.19.0-rc3+ #18 [ 64.145959] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 64.145964] RIP: 0010:native_queued_spin_lock_slowpath+0x21/0x1c0 [ 64.145965] Code: 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 0f 1f 44 00 00 ba 01 00 00 00 8b 07 85 c0 75 0a f0 0f b1 17 85 c0 75 f2 f3 c3 f3 90 ec 81 fe 00 01 00 00 0f 84 25 01 00 00 81 e6 00 ff ff ff 75 44 [ 64.145966] RSP: 0018:bab781dabdf0 EFLAGS: 0282 ORIG_RAX: ff13 [ 64.145967] RAX: 9491 RBX: 949179f30c00 RCX: [ 64.145967] RDX: 0001 RSI: 9491 RDI: 9490777dca04 [ 64.145968] RBP: 9490777dca08 R08: 00025f80 R09: c01db0ed [ 64.145968] R10: bab781dabe30 R11: 0001 R12: 949176c2a220 [ 64.145969] R13: 9490777dca04 R14: 94917950edc8 R15: 9490777dca08 [ 64.145970] FS: 01957880() GS:94917bb0() knlGS: [ 64.145970] CS: 0010 DS: ES: CR0: 80050033 [ 64.145971] CR2: 00415c20 CR3: 7720a001 CR4: 00160ee0 [ 64.145974] Call Trace: [ 64.145979] _raw_spin_lock+0x1d/0x20 [ 64.145982] coda_release+0x66/0xd0 [coda] [ 64.145986] __fput+0xe2/0x210 [ 64.145989] task_work_run+0x86/0xa0 [ 64.145990] do_exit+0x2d7/0xb30 [ 64.145994] ? handle_mm_fault+0xfa/0x200 [ 64.145995] do_group_exit+0x3a/0xa0 [ 64.145996] __x64_sys_exit_group+0x14/0x20 [ 64.145998] do_syscall_64+0x55/0x100 [ 64.145999] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 64.146001] RIP: 0033:0x453988 [ 64.146005] Code: Bad RIP value. [ 64.146005] RSP: 002b:7ffcd39246a8 EFLAGS: 0246 ORIG_RAX: 00e7 [ 64.146006] RAX: ffda RBX: RCX: 00453988 [ 64.146007] RDX: RSI: 003c RDI: [ 64.146007] RBP: 005a79b0 R08: 00e7 R09: ffe0 [ 64.146008] R10: 0022 R11: 0246 R12: 0001 [ 64.146008] R13: 007c1640 R14: 005a0fc1 R15: ... [ 97.001204] rcu: INFO: rcu_sched self-detected stall on CPU [ 97.001211] rcu: 4-: (14997 ticks this GP) idle=916/1/0x4002 softirq=2901/2901 fqs=6623 [ 97.001211] rcu: (t=15000 jiffies g=3281 q=10605) [ 97.001213] NMI backtrace for cpu 4 [ 97.001215] CPU: 4 PID: 1814 Comm: busybox Tainted: G L 4.19.0-rc3+ #18 [ 97.001216] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 97.001216] Call Trace: [ 97.001219] [ 97.001223] dump_stack+0x5c/0x7b [ 97.001236] nmi_cpu_backtrace+0x8a/0x90 [ 97.001239] ? lapic_can_unplug_cpu+0xa0/0xa0 [
Re: [PATCH 2] coda: get rid of CODA_FREE()
On Thu, Feb 14, 2019 at 11:38:22AM +0300, Dan Carpenter wrote: > The CODA_FREE() macro just calls kvfree(). We can call that directly > instead. Added these to my (getting long) list of things that I really should get upstreamed sometime soon now. Jan
Re: [PATCH] fs: coda: fix a double-fetch case in coda_psdev_write
This has come up before, and as you noticed nothing in that header is looked at
after that second copy.
Also if the user space cache manager was trying to be malicious it wouldn't
have to resort to multi-threaded race conditions. It could make a setuid root
shell appear, or many other things a file system can do.
Jan
On December 25, 2018 2:03:51 PM EST, Kangjie Lu wrote:
>"hdr" has been copied in from user space and "hdr.opcode" is checked.
>The code copies it again. User space data between the two copies is
>subject to modification if the user-space code is multithreaded and
>malicious. The modification may invalidate the check. In the original
>code, "hdr.opcode" is not used after the second copy, so it is actually
>safe with the current code. However, in case future code may use
>"hdr.opcode", let's avoid copying the header from user space again.
>
>Signed-off-by: Kangjie Lu
>---
> fs/coda/psdev.c | 41 ++---
> 1 file changed, 22 insertions(+), 19 deletions(-)
>
>diff --git a/fs/coda/psdev.c b/fs/coda/psdev.c
>index c5234c21b539..fb4d1c654773 100644
>--- a/fs/coda/psdev.c
>+++ b/fs/coda/psdev.c
>@@ -97,35 +97,38 @@ static long coda_psdev_ioctl(struct file * filp,
>unsigned int cmd, unsigned long
>static ssize_t coda_psdev_write(struct file *file, const char __user
>*buf,
> size_t nbytes, loff_t *off)
> {
>-struct venus_comm *vcp = (struct venus_comm *)
>file->private_data;
>-struct upc_req *req = NULL;
>-struct upc_req *tmp;
>+ struct venus_comm *vcp = (struct venus_comm *) file->private_data;
>+ struct upc_req *req = NULL;
>+ struct upc_req *tmp;
> struct list_head *lh;
> struct coda_in_hdr hdr;
> ssize_t retval = 0, count = 0;
> int error;
>
>-/* Peek at the opcode, uniquefier */
>+ /* Peek at the opcode, uniquefier */
> if (copy_from_user(, buf, 2 * sizeof(u_long)))
>- return -EFAULT;
>+ return -EFAULT;
>
>-if (DOWNCALL(hdr.opcode)) {
>+ if (DOWNCALL(hdr.opcode)) {
> union outputArgs *dcbuf;
> int size = sizeof(*dcbuf);
>
> if ( nbytes < sizeof(struct coda_out_hdr) ) {
> pr_warn("coda_downcall opc %d uniq %d, not enough!\n",
>- hdr.opcode, hdr.unique);
>+ hdr.opcode, hdr.unique);
> count = nbytes;
> goto out;
> }
> if ( nbytes > size ) {
> pr_warn("downcall opc %d, uniq %d, too much!",
>- hdr.opcode, hdr.unique);
>- nbytes = size;
>+ hdr.opcode, hdr.unique);
>+ nbytes = size;
> }
> CODA_ALLOC(dcbuf, union outputArgs *, nbytes);
>- if (copy_from_user(dcbuf, buf, nbytes)) {
>+ *((struct coda_in_hdr *)dcbuf) = hdr;
>+ if (copy_from_user(dcbuf + sizeof(hdr),
>+ buf + sizeof(hdr),
>+ nbytes - sizeof(hdr))) {
> CODA_FREE(dcbuf, nbytes);
> retval = -EFAULT;
> goto out;
>@@ -137,14 +140,14 @@ static ssize_t coda_psdev_write(struct file
>*file, const char __user *buf,
> CODA_FREE(dcbuf, nbytes);
> if (error) {
> pr_warn("%s: coda_downcall error: %d\n",
>- __func__, error);
>+ __func__, error);
> retval = error;
> goto out;
> }
> count = nbytes;
> goto out;
> }
>-
>+
> /* Look for the message on the processing queue. */
> mutex_lock(>vc_mutex);
> list_for_each(lh, >vc_processing) {
>@@ -159,19 +162,19 @@ static ssize_t coda_psdev_write(struct file
>*file, const char __user *buf,
>
> if (!req) {
> pr_warn("%s: msg (%d, %d) not found\n",
>- __func__, hdr.opcode, hdr.unique);
>+ __func__, hdr.opcode, hdr.unique);
> retval = -ESRCH;
> goto out;
> }
>
>-/* move data into response buffer. */
>+ /* move data into response buffer. */
> if (req->uc_outSize < nbytes) {
> pr_warn("%s: too much cnt: %d, cnt: %ld, opc: %d, uniq: %d.\n",
>- __func__, req->uc_outSize, (long)nbytes,
>- hdr.opcode, hdr.unique);
>+ __func__, req->uc_outSize, (long)nbytes,
>+ hdr.opcode, hdr.unique);
> nbytes = req->uc_outSize; /* don't have more space! */
> }
>-if (copy_from_user(req->uc_data, buf, nbytes)) {
>+ if
Re: [PATCH 1/3] VFS: introduce MAY_ACT_AS_OWNER
Same for Coda.
uid/gid/mode don't mean anything, access is based on the directory ACL and the
authentication token that is held by the userspace cache manager and ultimately
decided by the servers.
Unless someone broke this recently and made permission checks uid based I would
expect no change. If this is broken by a recent commit I expect something
similar to what NFS is trying to do by allowing the actual check to be passed
down.
Jan
On October 4, 2018 10:10:13 AM EDT, David Howells wrote:
>NeilBrown wrote:
>
>> diff --git a/fs/afs/security.c b/fs/afs/security.c
>> index 81dfedb7879f..ac2e39de8bff 100644
>> --- a/fs/afs/security.c
>> +++ b/fs/afs/security.c
>> @@ -349,6 +349,16 @@ int afs_permission(struct inode *inode, int
>mask)
>> if (mask & MAY_NOT_BLOCK)
>> return -ECHILD;
>>
>> +/* Short-circuit for owner */
>> +if (mask & MAY_ACT_AS_OWNER) {
>> +if (inode_owner_or_capable(inode))
>
>You don't know that inode->i_uid in meaningful. You may have noticed
>that
>afs_permission() ignores i_uid and i_gid entirely. It queries the
>server (if
>this information is not otherwise cached) to ask what permits the user
>is
>granted - where the user identity is defined by the key returned from
>afs_request_key()[*].
>
>So, NAK for the afs piece.
>
>David
>
>[*] If there's no appropriate key, anonymous permits will be used.
Re: [PATCH 1/3] VFS: introduce MAY_ACT_AS_OWNER
Same for Coda.
uid/gid/mode don't mean anything, access is based on the directory ACL and the
authentication token that is held by the userspace cache manager and ultimately
decided by the servers.
Unless someone broke this recently and made permission checks uid based I would
expect no change. If this is broken by a recent commit I expect something
similar to what NFS is trying to do by allowing the actual check to be passed
down.
Jan
On October 4, 2018 10:10:13 AM EDT, David Howells wrote:
>NeilBrown wrote:
>
>> diff --git a/fs/afs/security.c b/fs/afs/security.c
>> index 81dfedb7879f..ac2e39de8bff 100644
>> --- a/fs/afs/security.c
>> +++ b/fs/afs/security.c
>> @@ -349,6 +349,16 @@ int afs_permission(struct inode *inode, int
>mask)
>> if (mask & MAY_NOT_BLOCK)
>> return -ECHILD;
>>
>> +/* Short-circuit for owner */
>> +if (mask & MAY_ACT_AS_OWNER) {
>> +if (inode_owner_or_capable(inode))
>
>You don't know that inode->i_uid in meaningful. You may have noticed
>that
>afs_permission() ignores i_uid and i_gid entirely. It queries the
>server (if
>this information is not otherwise cached) to ask what permits the user
>is
>granted - where the user identity is defined by the key returned from
>afs_request_key()[*].
>
>So, NAK for the afs piece.
>
>David
>
>[*] If there's no appropriate key, anonymous permits will be used.
Re: [PATCH] fs: fix access beyond unterminated strings in prints
On Fri, Sep 28, 2018 at 09:00:48PM +0300, Amir Goldstein wrote: > I chose not to split the patches per fs in the hope that maintainers > would quickly ack the patch and ask you to carry it for them. Ack for Coda. I actually have a patch queued somewhere that dropped printing the name and only prints the length, but this works too. Jan
Re: [PATCH] fs: fix access beyond unterminated strings in prints
On Fri, Sep 28, 2018 at 09:00:48PM +0300, Amir Goldstein wrote: > I chose not to split the patches per fs in the hope that maintainers > would quickly ack the patch and ask you to carry it for them. Ack for Coda. I actually have a patch queued somewhere that dropped printing the name and only prints the length, but this works too. Jan
Re: [PATCH 05/11] UAPI: coda: Don't use internal kernel structs in UAPI
On Thu, Sep 06, 2018 at 01:52:29PM +0200, Yann Droneaud wrote: > Hi, > > Le jeudi 06 septembre 2018 à 08:13 +0100, David Howells a écrit : > > Yann Droneaud wrote: > > > > > This structure should not have been exposed to userspace in the > > > first > > > place: it's unusable by userspace as it is. It was incorrect to > > > have it > > > outside of #ifdef __KERNEL__ before commit 607ca46e97a1b ... > > > ... > > > All CODA_REQ_* defines internals to kernel side and not exchanged > > > with > > > userspace. > > > > > > Please move them back to > > > > Is there any reason coda_psdev.h needs to be in include/linux/ rather > > than fs/coda/? > > > > It's a valid concern. > > At first I thought the first lines (see below) could have been useful > for userspace: > > #define CODA_PSDEV_MAJOR 67 > #define MAX_CODADEVS 5/* how many do we allow */ Nope, userspace just tries to open /dev/cfs0, or a manually configured alternative. We have only used linux/coda.h, and actually carry our own copy of that file which is kept in sync manually, which is why there are all those ifdefs for different systems in there. This all originates from the time of the 2.1.x kernels when Coda was built externally. > But the file was unsuable for a long long time so we can assume it's > usage by userspace is deprecated, then we could remove it from UAPI, > and moves its content back to include/linux. > > As one could see include/linux/coda_psdev.h is not used outside of > fs/coda, moving the header here as you suggests seems to be the correct > solution. Agreed. Jan
Re: [PATCH 05/11] UAPI: coda: Don't use internal kernel structs in UAPI
On Thu, Sep 06, 2018 at 01:52:29PM +0200, Yann Droneaud wrote: > Hi, > > Le jeudi 06 septembre 2018 à 08:13 +0100, David Howells a écrit : > > Yann Droneaud wrote: > > > > > This structure should not have been exposed to userspace in the > > > first > > > place: it's unusable by userspace as it is. It was incorrect to > > > have it > > > outside of #ifdef __KERNEL__ before commit 607ca46e97a1b ... > > > ... > > > All CODA_REQ_* defines internals to kernel side and not exchanged > > > with > > > userspace. > > > > > > Please move them back to > > > > Is there any reason coda_psdev.h needs to be in include/linux/ rather > > than fs/coda/? > > > > It's a valid concern. > > At first I thought the first lines (see below) could have been useful > for userspace: > > #define CODA_PSDEV_MAJOR 67 > #define MAX_CODADEVS 5/* how many do we allow */ Nope, userspace just tries to open /dev/cfs0, or a manually configured alternative. We have only used linux/coda.h, and actually carry our own copy of that file which is kept in sync manually, which is why there are all those ifdefs for different systems in there. This all originates from the time of the 2.1.x kernels when Coda was built externally. > But the file was unsuable for a long long time so we can assume it's > usage by userspace is deprecated, then we could remove it from UAPI, > and moves its content back to include/linux. > > As one could see include/linux/coda_psdev.h is not used outside of > fs/coda, moving the header here as you suggests seems to be the correct > solution. Agreed. Jan
Re: [PATCH 05/11] UAPI: coda: Don't use internal kernel structs in UAPI
On Wed, Sep 05, 2018 at 07:12:37PM +0200, Yann Droneaud wrote: > Le mercredi 05 septembre 2018 à 16:55 +0100, David Howells a écrit : > > The size and layout of internal kernel structures may not be relied > > upon outside of the kernel and may even change in a containerised > > environment if a container image is frozen and shifted to another > > machine. > > > > Excise these from Coda's upc_req struct. ... > > This structure should not have been exposed to userspace in the first > place: it's unusable by userspace as it is. It was incorrect to have it > outside of #ifdef __KERNEL__ before commit 607ca46e97a1b ... ... > So the structure can be moved back to . I found a year old patch that clearly fell through the cracks that fixes this exact thing. https://lkml.org/lkml/2017/8/6/186 Jan
Re: [PATCH 05/11] UAPI: coda: Don't use internal kernel structs in UAPI
On Wed, Sep 05, 2018 at 07:12:37PM +0200, Yann Droneaud wrote: > Le mercredi 05 septembre 2018 à 16:55 +0100, David Howells a écrit : > > The size and layout of internal kernel structures may not be relied > > upon outside of the kernel and may even change in a containerised > > environment if a container image is frozen and shifted to another > > machine. > > > > Excise these from Coda's upc_req struct. ... > > This structure should not have been exposed to userspace in the first > place: it's unusable by userspace as it is. It was incorrect to have it > outside of #ifdef __KERNEL__ before commit 607ca46e97a1b ... ... > So the structure can be moved back to . I found a year old patch that clearly fell through the cracks that fixes this exact thing. https://lkml.org/lkml/2017/8/6/186 Jan
Re: [PATCH 05/11] UAPI: coda: Don't use internal kernel structs in UAPI
On Wed, Sep 05, 2018 at 04:55:10PM +0100, David Howells wrote: > The size and layout of internal kernel structures may not be relied upon > outside of the kernel and may even change in a containerised environment if > a container image is frozen and shifted to another machine. > > Excise these from Coda's upc_req struct. Argh, that won't work. I still have to look at where this structure is used exactly, but... Either this structure is used by the messages that the kernel sends to userspace, in which case we don't want the kernel to pack the larger structure that includes a list_head and a wait_queue_head_t in the message while userspace reads as if it was a smaller structure without those. But my gut feeling is that this is not part of the upcall request messages and never gets to userspace and as such shouldn't be in uapi to begin with. Jan
Re: [PATCH 05/11] UAPI: coda: Don't use internal kernel structs in UAPI
On Wed, Sep 05, 2018 at 04:55:10PM +0100, David Howells wrote: > The size and layout of internal kernel structures may not be relied upon > outside of the kernel and may even change in a containerised environment if > a container image is frozen and shifted to another machine. > > Excise these from Coda's upc_req struct. Argh, that won't work. I still have to look at where this structure is used exactly, but... Either this structure is used by the messages that the kernel sends to userspace, in which case we don't want the kernel to pack the larger structure that includes a list_head and a wait_queue_head_t in the message while userspace reads as if it was a smaller structure without those. But my gut feeling is that this is not part of the upcall request messages and never gets to userspace and as such shouldn't be in uapi to begin with. Jan
[PATCH] coda: fix 'kernel memory exposure attempt' in fsync
When an application called fsync on a file in Coda a small request with just the file identifier was allocated, but the declared length was set to the size of union of all possible upcall requests. This bug has been around for a very long time and is now caught by the extra checking in usercopy that was introduced in Linux-4.8. The exposure happens when the Coda cache manager process reads the fsync upcall request at which point it is killed. As a result there is nobody servicing any further upcalls, trapping any processes that try to access the mounted Coda filesystem. Cc: sta...@vger.kernel.org Signed-off-by: Jan Harkes <jahar...@cs.cmu.edu> --- fs/coda/upcall.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fs/coda/upcall.c b/fs/coda/upcall.c index e82357c89979..8cf16d8c5261 100644 --- a/fs/coda/upcall.c +++ b/fs/coda/upcall.c @@ -446,8 +446,7 @@ int venus_fsync(struct super_block *sb, struct CodaFid *fid) UPARG(CODA_FSYNC); inp->coda_fsync.VFid = *fid; - error = coda_upcall(coda_vcp(sb), sizeof(union inputArgs), - , inp); + error = coda_upcall(coda_vcp(sb), insize, , inp); CODA_FREE(inp, insize); return error; -- 2.14.2
[PATCH] coda: fix 'kernel memory exposure attempt' in fsync
When an application called fsync on a file in Coda a small request with just the file identifier was allocated, but the declared length was set to the size of union of all possible upcall requests. This bug has been around for a very long time and is now caught by the extra checking in usercopy that was introduced in Linux-4.8. The exposure happens when the Coda cache manager process reads the fsync upcall request at which point it is killed. As a result there is nobody servicing any further upcalls, trapping any processes that try to access the mounted Coda filesystem. Cc: sta...@vger.kernel.org Signed-off-by: Jan Harkes --- fs/coda/upcall.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fs/coda/upcall.c b/fs/coda/upcall.c index e82357c89979..8cf16d8c5261 100644 --- a/fs/coda/upcall.c +++ b/fs/coda/upcall.c @@ -446,8 +446,7 @@ int venus_fsync(struct super_block *sb, struct CodaFid *fid) UPARG(CODA_FSYNC); inp->coda_fsync.VFid = *fid; - error = coda_upcall(coda_vcp(sb), sizeof(union inputArgs), - , inp); + error = coda_upcall(coda_vcp(sb), insize, , inp); CODA_FREE(inp, insize); return error; -- 2.14.2
Re: [PATCH] fs/coda: ensure the header peeked at is the same in the actual message
On Sat, Sep 23, 2017 at 10:35:45PM -0400, Meng Xu wrote: > Hi Jaharkes and Coda filesystem developers, > > I am resending the email on a potential race condition bug I found in the > Coda filesystem as well as the patch I propose. Please feel free to comment > whether you think this is a serious problem and whether the patch will work. > Thank you. Hi, It does look like there is a potential race there but I don't believe it is very serious because, - The userspace Coda cache manager component (Coda client) is using co-routines instead of true multithreading. So even if someone tries to exploit this through a vulnerability through the Coda client the whole process is blocked on the write systemcall and there is no other thread of execution available that can try to change the fields. - If someone can exploit the Coda client, they already have achieved root, so no need to try to exploit anything in the kernel. - If someone can replace the opcode and unique id, they can replace anything else in the message, with much worse results. For instance by rewriting the response to an open call from success to failure the attacker is able to cause a file descriptor leak because the Coda client believes there is an open file, while the kernel and application believe the open failed, etc. Now there are two types of messages the Coda client writes to the kernel device. - Downcalls which provide hints to trigger cache revalidations, here the unique id is ignored and the opcode mostly indicates how severe the cache invalidation is (only a single object, whole subtree, etc.) If someone wants to mess with that rewriting the file identifier that is in the message body is more useful and the worst they can do is make stale data stick in the kernel cache. I notice that your patch does not address the copy_from_user for downcalls around line 128. - The other type are the upcall responses, file system requests are blocked until the Coda client returns a response with the matching unique id from the request. This match is performed at line 158, which is after the peek, but before the second copy_from_user. Once the matching request has been found and dequeued we do not look at the unique id anymore. Because the response is correlated to a specific request and we do not re-copy the opcode from the response to the req->uc_opcode field it doesn't really matter what the opcode in the response was at this point because all decisions are based on the request opcode. Jan
Re: [PATCH] fs/coda: ensure the header peeked at is the same in the actual message
On Sat, Sep 23, 2017 at 10:35:45PM -0400, Meng Xu wrote: > Hi Jaharkes and Coda filesystem developers, > > I am resending the email on a potential race condition bug I found in the > Coda filesystem as well as the patch I propose. Please feel free to comment > whether you think this is a serious problem and whether the patch will work. > Thank you. Hi, It does look like there is a potential race there but I don't believe it is very serious because, - The userspace Coda cache manager component (Coda client) is using co-routines instead of true multithreading. So even if someone tries to exploit this through a vulnerability through the Coda client the whole process is blocked on the write systemcall and there is no other thread of execution available that can try to change the fields. - If someone can exploit the Coda client, they already have achieved root, so no need to try to exploit anything in the kernel. - If someone can replace the opcode and unique id, they can replace anything else in the message, with much worse results. For instance by rewriting the response to an open call from success to failure the attacker is able to cause a file descriptor leak because the Coda client believes there is an open file, while the kernel and application believe the open failed, etc. Now there are two types of messages the Coda client writes to the kernel device. - Downcalls which provide hints to trigger cache revalidations, here the unique id is ignored and the opcode mostly indicates how severe the cache invalidation is (only a single object, whole subtree, etc.) If someone wants to mess with that rewriting the file identifier that is in the message body is more useful and the worst they can do is make stale data stick in the kernel cache. I notice that your patch does not address the copy_from_user for downcalls around line 128. - The other type are the upcall responses, file system requests are blocked until the Coda client returns a response with the matching unique id from the request. This match is performed at line 158, which is after the peek, but before the second copy_from_user. Once the matching request has been found and dequeued we do not look at the unique id anymore. Because the response is correlated to a specific request and we do not re-copy the opcode from the response to the req->uc_opcode field it doesn't really matter what the opcode in the response was at this point because all decisions are based on the request opcode. Jan
Re: [PATCH 03/10] coda: honor AT_STATX_DONT_SYNC
On Wed, Sep 20, 2017 at 10:39:58AM +0200, Miklos Szeredi wrote:
> The description of this flag says "Don't sync attributes with the server".
> In other words: always use the attributes cached in the kernel and don't
> send network or local messages to refresh the attributes.
What is the use case for this AT_STATX_DONT_SYNC flag?
I'm asking because the Coda userspace client potentially has attributes
that are not cached in the kernel but can be (re-)validated without
network communication. So if we just care about avoiding network
traffic we could propagate the flag up to userspace. If we want to avoid
context switches, disk I/O and only check on what happens to be cached
in the kernel the current approach works fine.
Jan
> Signed-off-by: Miklos Szeredi <mszer...@redhat.com>
> Cc: Jan Harkes <jahar...@cs.cmu.edu>
> ---
> fs/coda/inode.c | 5 -
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/fs/coda/inode.c b/fs/coda/inode.c
> index 6058df380cc0..734672b2cbdc 100644
> --- a/fs/coda/inode.c
> +++ b/fs/coda/inode.c
> @@ -255,7 +255,10 @@ static void coda_evict_inode(struct inode *inode)
> int coda_getattr(const struct path *path, struct kstat *stat,
>u32 request_mask, unsigned int flags)
> {
> - int err = coda_revalidate_inode(d_inode(path->dentry));
> + int err = 0;
> +
> + if (!(flags & AT_STATX_DONT_SYNC))
> + err = coda_revalidate_inode(d_inode(path->dentry));
> if (!err)
> generic_fillattr(d_inode(path->dentry), stat);
> return err;
> --
> 2.5.5
>
>
Re: [PATCH 03/10] coda: honor AT_STATX_DONT_SYNC
On Wed, Sep 20, 2017 at 10:39:58AM +0200, Miklos Szeredi wrote:
> The description of this flag says "Don't sync attributes with the server".
> In other words: always use the attributes cached in the kernel and don't
> send network or local messages to refresh the attributes.
What is the use case for this AT_STATX_DONT_SYNC flag?
I'm asking because the Coda userspace client potentially has attributes
that are not cached in the kernel but can be (re-)validated without
network communication. So if we just care about avoiding network
traffic we could propagate the flag up to userspace. If we want to avoid
context switches, disk I/O and only check on what happens to be cached
in the kernel the current approach works fine.
Jan
> Signed-off-by: Miklos Szeredi
> Cc: Jan Harkes
> ---
> fs/coda/inode.c | 5 -
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/fs/coda/inode.c b/fs/coda/inode.c
> index 6058df380cc0..734672b2cbdc 100644
> --- a/fs/coda/inode.c
> +++ b/fs/coda/inode.c
> @@ -255,7 +255,10 @@ static void coda_evict_inode(struct inode *inode)
> int coda_getattr(const struct path *path, struct kstat *stat,
>u32 request_mask, unsigned int flags)
> {
> - int err = coda_revalidate_inode(d_inode(path->dentry));
> + int err = 0;
> +
> + if (!(flags & AT_STATX_DONT_SYNC))
> + err = coda_revalidate_inode(d_inode(path->dentry));
> if (!err)
> generic_fillattr(d_inode(path->dentry), stat);
> return err;
> --
> 2.5.5
>
>
Re: coda's use of file->f_mapping and inode->i_mapping
On Wed, Aug 02, 2017 at 11:05:31AM -0400, Jeff Layton wrote: > The weird bit is that in coda_file_mmap, we then do this: > > coda_file->f_mapping = host_file->f_mapping; > if (coda_inode->i_mapping == _inode->i_data) > coda_inode->i_mapping = host_inode->i_mapping; > > > What is the significance of mmap on coda files? If you want to monkey > around with the i_mapping and f_mapping, wouldn't it make more sense to > do so at open() time? Normal read and write calls go through the Coda kernel module and are passed along to the underlying 'host / container' file. However with mmap you only get told that the mapping is created and the following page faults are handled directly by following the file's f_mapping/i_mapping pointer. We don't need to set up this pointer if the user never calls mmap. It is possible that when different users open the same file object they are given a different container file. Although Coda's userspace as far as I know doesn't actually do this, this case was 'anticipated' as a possibility. By delaying the monkeying around to the point of mmap, open/read/write for files works reliably and only if a second mmap is attempted where the second coda_file handle for the same coda_inode happens to have a different host_file associated with it do we return EBUSY. Jan
Re: coda's use of file->f_mapping and inode->i_mapping
On Wed, Aug 02, 2017 at 11:05:31AM -0400, Jeff Layton wrote: > The weird bit is that in coda_file_mmap, we then do this: > > coda_file->f_mapping = host_file->f_mapping; > if (coda_inode->i_mapping == _inode->i_data) > coda_inode->i_mapping = host_inode->i_mapping; > > > What is the significance of mmap on coda files? If you want to monkey > around with the i_mapping and f_mapping, wouldn't it make more sense to > do so at open() time? Normal read and write calls go through the Coda kernel module and are passed along to the underlying 'host / container' file. However with mmap you only get told that the mapping is created and the following page faults are handled directly by following the file's f_mapping/i_mapping pointer. We don't need to set up this pointer if the user never calls mmap. It is possible that when different users open the same file object they are given a different container file. Although Coda's userspace as far as I know doesn't actually do this, this case was 'anticipated' as a possibility. By delaying the monkeying around to the point of mmap, open/read/write for files works reliably and only if a second mmap is attempted where the second coda_file handle for the same coda_inode happens to have a different host_file associated with it do we return EBUSY. Jan
Re: [PATCH 96/98] HACK include/uapi/linux/coda_psdev.h: fix compilation in userspace
On Sat, May 30, 2015 at 05:39:28PM +0200, Mikko Rapeli wrote: > Include linux/coda.h for caddr_t and use unsigned short type directly. > Userspace headers do not have list_head and wait_queue_head_t so just > ifdef them away which is a HACK. Any ideas how to fix this properly? I grepped the Coda userspace sources and it doesn't look like this particular struct is used there anyway, it is only used by the kernel module to track which requests are waiting to be read by the Coda userspace application and after that which requests are waiting for a response. I guess a proper fix would be to move this struct to a non-uapi header, or maybe even to the (probably) only C file in the kernel where it is used. Jan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 96/98] HACK include/uapi/linux/coda_psdev.h: fix compilation in userspace
On Sat, May 30, 2015 at 05:39:28PM +0200, Mikko Rapeli wrote: Include linux/coda.h for caddr_t and use unsigned short type directly. Userspace headers do not have list_head and wait_queue_head_t so just ifdef them away which is a HACK. Any ideas how to fix this properly? I grepped the Coda userspace sources and it doesn't look like this particular struct is used there anyway, it is only used by the kernel module to track which requests are waiting to be read by the Coda userspace application and after that which requests are waiting for a response. I guess a proper fix would be to move this struct to a non-uapi header, or maybe even to the (probably) only C file in the kernel where it is used. Jan -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 1/1 linux-next] fs/coda/upcall.c: remove UPARG flow control macro
On Mon, Apr 27, 2015 at 07:23:06PM +0200, Fabian Frederick wrote:
> Move UPARG code to alloc_upcall() and test errors/return in callsites.
> This patch removes flow control in macros which must be avoided.
> (See Documentation/CodingStyle)
At first glance this is not a correct patch.
UPARG allocates a buffer that is large enough to hold both the upcall
message going to userspace as well as the incoming reply message. At
first glance this patch only seems to allocate insize. It may still be
doing the right thing because insize is defined at the call level as
MAX(sizeof(inarg), sizeof(outarg)). This it looks like quite a few
changes for an untested patch and not enough for an actual cleanup of
the related code.
Jan
> ---
> This is untested.
>
> fs/coda/upcall.c | 107
> +++
> 1 file changed, 69 insertions(+), 38 deletions(-)
>
> diff --git a/fs/coda/upcall.c b/fs/coda/upcall.c
> index 9b1ffaa..7524f630 100644
> --- a/fs/coda/upcall.c
> +++ b/fs/coda/upcall.c
> @@ -41,30 +41,24 @@
> static int coda_upcall(struct venus_comm *vc, int inSize, int *outSize,
> union inputArgs *buffer);
>
> -static void *alloc_upcall(int opcode, int size)
> +static int alloc_upcall(int opcode, int insize, int *outsize,
> + union inputArgs **inp, union outputArgs **outp)
> {
> - union inputArgs *inp;
> + CODA_ALLOC(*inp, union inputArgs *, insize);
> + if (!*inp)
> + return -ENOMEM;
>
> - CODA_ALLOC(inp, union inputArgs *, size);
> -if (!inp)
> - return ERR_PTR(-ENOMEM);
...
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 1/1 linux-next] fs/coda/upcall.c: remove UPARG flow control macro
On Mon, Apr 27, 2015 at 07:23:06PM +0200, Fabian Frederick wrote:
Move UPARG code to alloc_upcall() and test errors/return in callsites.
This patch removes flow control in macros which must be avoided.
(See Documentation/CodingStyle)
At first glance this is not a correct patch.
UPARG allocates a buffer that is large enough to hold both the upcall
message going to userspace as well as the incoming reply message. At
first glance this patch only seems to allocate insize. It may still be
doing the right thing because insize is defined at the call level as
MAX(sizeof(inarg), sizeof(outarg)). This it looks like quite a few
changes for an untested patch and not enough for an actual cleanup of
the related code.
Jan
---
This is untested.
fs/coda/upcall.c | 107
+++
1 file changed, 69 insertions(+), 38 deletions(-)
diff --git a/fs/coda/upcall.c b/fs/coda/upcall.c
index 9b1ffaa..7524f630 100644
--- a/fs/coda/upcall.c
+++ b/fs/coda/upcall.c
@@ -41,30 +41,24 @@
static int coda_upcall(struct venus_comm *vc, int inSize, int *outSize,
union inputArgs *buffer);
-static void *alloc_upcall(int opcode, int size)
+static int alloc_upcall(int opcode, int insize, int *outsize,
+ union inputArgs **inp, union outputArgs **outp)
{
- union inputArgs *inp;
+ CODA_ALLOC(*inp, union inputArgs *, insize);
+ if (!*inp)
+ return -ENOMEM;
- CODA_ALLOC(inp, union inputArgs *, size);
-if (!inp)
- return ERR_PTR(-ENOMEM);
...
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 1/3] FS/CODA: replace printk by pr_foo()
On Tue, May 13, 2014 at 06:49:09PM +0200, Fabian Frederick wrote: > No level printk converted to pr_warn or pr_info > > Cc: Jan Harkes > Cc: Andrew Morton > Signed-off-by: Fabian Frederick > --- Ack, those are nice cleanups for those ugly printk's. Looks like it will shave off a few KB of memory too by making the strings shorter. Jan > fs/coda/cnode.c | 4 ++-- > fs/coda/coda_linux.h | 2 +- > fs/coda/dir.c| 12 ++-- > fs/coda/inode.c | 29 +++-- > fs/coda/psdev.c | 34 +- > fs/coda/upcall.c | 14 +++--- > 6 files changed, 48 insertions(+), 47 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 1/3] FS/CODA: replace printk by pr_foo()
On Tue, May 13, 2014 at 06:49:09PM +0200, Fabian Frederick wrote: No level printk converted to pr_warn or pr_info Cc: Jan Harkes jahar...@cs.cmu.edu Cc: Andrew Morton a...@linux-foundation.org Signed-off-by: Fabian Frederick f...@skynet.be --- Ack, those are nice cleanups for those ugly printk's. Looks like it will shave off a few KB of memory too by making the strings shorter. Jan fs/coda/cnode.c | 4 ++-- fs/coda/coda_linux.h | 2 +- fs/coda/dir.c| 12 ++-- fs/coda/inode.c | 29 +++-- fs/coda/psdev.c | 34 +- fs/coda/upcall.c | 14 +++--- 6 files changed, 48 insertions(+), 47 deletions(-) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH review 18/85] coda: Restrict coda messages to the initial user namespace
On Wed, Feb 13, 2013 at 09:51:07AM -0800, Eric W. Biederman wrote: > From: "Eric W. Biederman" > > Remove the slight chance that uids and gids in coda messages will be > interpreted in the wrong user namespace. Awesome, I was wondering how to handle uid's from different namespaces cleanly in Coda's userspace daemon without resorting to exporting kuids to userspace. But this is a much more elegant solution, every namespace that cares about Coda should just run their own mountpoint and userspace daemon. Jan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH review 18/85] coda: Restrict coda messages to the initial user namespace
On Wed, Feb 13, 2013 at 09:51:07AM -0800, Eric W. Biederman wrote: From: Eric W. Biederman ebied...@xmission.com Remove the slight chance that uids and gids in coda messages will be interpreted in the wrong user namespace. Awesome, I was wondering how to handle uid's from different namespaces cleanly in Coda's userspace daemon without resorting to exporting kuids to userspace. But this is a much more elegant solution, every namespace that cares about Coda should just run their own mountpoint and userspace daemon. Jan -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] fs/coda: remove static inline forward declarations
On Wed, Feb 13, 2008 at 02:22:43PM +0200, Ilpo J?rvinen wrote: > They're defined later on in the same file with bodies and > nothingin between needs them. > > Signed-off-by: Ilpo J?rvinen <[EMAIL PROTECTED]> Acked-by: Jan Harkes <[EMAIL PROTECTED]> Looks good, getting rid of such cruft is always useful. Jan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [GIT PATCH] suspend patches for 2.6.25-rc0
On Fri, Feb 08, 2008 at 12:14:40AM +0100, Rafael J. Wysocki wrote: > I haven't revert the changes. It's only moved the notifier chain calls from > the ioctls to open/release in user.c . Ah my bad, sorry about the false alarm. Jan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [GIT PATCH] suspend patches for 2.6.25-rc0
On Fri, Feb 01, 2008 at 06:20:56PM -0500, Len Brown wrote: > please pull from: > > git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6.git suspend > > This will update the files shown below. The following patch, commit c3e94d899c864e558f938f9845ddb8c2e5d5ccd0 Author: Alan Stern <[EMAIL PROTECTED]> Date: Mon Nov 19 23:38:25 2007 +0100 Hibernation: Add PM_RESTORE_PREPARE and PM_POST_RESTORE notifiers (rev. 2) Seems to have reverted some code related to the PM_HIBERNATION_PREPARE notifiers in kernel/power/user.c that were added in 2.6.23 by the following patch, commit b10d911749d37dccfa5873d2088aea3f074b9e45 Author: Rafael J. Wysocki <[EMAIL PROTECTED]> Date: Thu Jul 19 01:47:36 2007 -0700 PM: introduce hibernation and suspend notifiers I'm pretty sure this was not the intent of the new patch because it only tries to extend upon the former functionality and not replace it. Jan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [GIT PATCH] suspend patches for 2.6.25-rc0
On Fri, Feb 01, 2008 at 06:20:56PM -0500, Len Brown wrote: please pull from: git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6.git suspend This will update the files shown below. The following patch, commit c3e94d899c864e558f938f9845ddb8c2e5d5ccd0 Author: Alan Stern [EMAIL PROTECTED] Date: Mon Nov 19 23:38:25 2007 +0100 Hibernation: Add PM_RESTORE_PREPARE and PM_POST_RESTORE notifiers (rev. 2) Seems to have reverted some code related to the PM_HIBERNATION_PREPARE notifiers in kernel/power/user.c that were added in 2.6.23 by the following patch, commit b10d911749d37dccfa5873d2088aea3f074b9e45 Author: Rafael J. Wysocki [EMAIL PROTECTED] Date: Thu Jul 19 01:47:36 2007 -0700 PM: introduce hibernation and suspend notifiers I'm pretty sure this was not the intent of the new patch because it only tries to extend upon the former functionality and not replace it. Jan -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [GIT PATCH] suspend patches for 2.6.25-rc0
On Fri, Feb 08, 2008 at 12:14:40AM +0100, Rafael J. Wysocki wrote: I haven't revert the changes. It's only moved the notifier chain calls from the ioctls to open/release in user.c . Ah my bad, sorry about the false alarm. Jan -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Networked filesystems vs backing_dev_info
On Sat, Oct 27, 2007 at 11:34:26AM +0200, Peter Zijlstra wrote: > I had me a little look at bdi usage in networked filesystems. > > NFS, CIFS, (smbfs), AFS, CODA and NCP > > And of those, NFS is the only one that I could find that creates > backing_dev_info structures. The rest seems to fall back to > default_backing_dev_info. While a file is opened in Coda we associate the open file handle with a local cache file. All read and write operations are redirected to this local file and we even redirect inode->i_mapping. Actual reads and writes are completely handled by the underlying file system. We send the new file contents back to the servers only after all local references have been released (last-close semantics). As a result, there is no need for backing_dev_info structures in Coda, if any congestion control is needed it will be handled by the underlying file system where our locally cached copies are stored. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Networked filesystems vs backing_dev_info
On Sat, Oct 27, 2007 at 11:34:26AM +0200, Peter Zijlstra wrote: I had me a little look at bdi usage in networked filesystems. NFS, CIFS, (smbfs), AFS, CODA and NCP And of those, NFS is the only one that I could find that creates backing_dev_info structures. The rest seems to fall back to default_backing_dev_info. While a file is opened in Coda we associate the open file handle with a local cache file. All read and write operations are redirected to this local file and we even redirect inode-i_mapping. Actual reads and writes are completely handled by the underlying file system. We send the new file contents back to the servers only after all local references have been released (last-close semantics). As a result, there is no need for backing_dev_info structures in Coda, if any congestion control is needed it will be handled by the underlying file system where our locally cached copies are stored. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 2/4] Fix mainline filesystems to handle ATTR_KILL_ bits correctly
On Mon, Aug 20, 2007 at 04:53:22PM -0400, Jeff Layton wrote: > This should fix all of the filesystems in the mainline kernels to handle > ATTR_KILL_SUID and ATTR_KILL_SGID correctly. For most of them, this is > just a matter of making sure that they call generic_attrkill early in > the setattr inode op. > > Signed-off-by: Jeff Layton <[EMAIL PROTECTED]> Coda part looks fine to me. Couldn't test it beyond 'it compiles and doesn't oops', because our userspace client unconditionally strips setuid/setgid bits anyways. Acked-by: Jan Harkes <[EMAIL PROTECTED]> - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 2/4] Fix mainline filesystems to handle ATTR_KILL_ bits correctly
On Mon, Aug 20, 2007 at 04:53:22PM -0400, Jeff Layton wrote: This should fix all of the filesystems in the mainline kernels to handle ATTR_KILL_SUID and ATTR_KILL_SGID correctly. For most of them, this is just a matter of making sure that they call generic_attrkill early in the setattr inode op. Signed-off-by: Jeff Layton [EMAIL PROTECTED] Coda part looks fine to me. Couldn't test it beyond 'it compiles and doesn't oops', because our userspace client unconditionally strips setuid/setgid bits anyways. Acked-by: Jan Harkes [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: wrong order of arguments of ->readdir()
On Mon, Jul 16, 2007 at 09:49:30PM +0100, Al Viro wrote: > > c) Jan's patch was not cc'ed to any mailing list > > ... said patch (presumably being an obvious fix of obvious roothole) > had not been sent to mainline immediately. The exploiting application would have to run in the role of the userspace cache manager. In other words, you need to, - open /dev/cfs0 - mount the Coda filesystem and pass the file handle of the opened cfs0 device as one of the mount options Both of these actions already require root, and if you can get this far without root priviledges it would be considerably easier to just hand a setuid root shell through the mounted Coda file system. So I didn't consider this a critical root hole. > > d) Al's patch was not sent to the maintainer. Nor to me. Nor was it > >staged in any tree which I can get at so that I could inform people of > >the upcoming conflict/duplication/etc. > > Coda is still maintained? That's certainly news to me - I'm not being > sarcastic, but my impression had been that coda got abandoned several > years ago. Well, in that case someone forgot to update the MAINTAINERS file. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: wrong order of arguments of ->readdir()
On Mon, Jul 16, 2007 at 01:44:24PM -0700, Andrew Morton wrote:
> On Sun, 15 Jul 2007 23:59:03 GMT
> Linux Kernel Mailing List wrote:
> >
> > wrong order of arguments of ->readdir()
> >
> > Shows how many people are testing coda - the bug had been there for 5
> > years
> > and results of stepping on it are not subtle.
> >
> > Signed-off-by: Al Viro <[EMAIL PROTECTED]>
> > Signed-off-by: Linus Torvalds <[EMAIL PROTECTED]>
> > ---
> > fs/coda/dir.c |2 +-
> > 1 files changed, 1 insertions(+), 1 deletions(-)
> >
> > diff --git a/fs/coda/dir.c b/fs/coda/dir.c
> > index 9ddf5ed..898a86d 100644
> > --- a/fs/coda/dir.c
> > +++ b/fs/coda/dir.c
> > @@ -470,7 +470,7 @@ int coda_readdir(struct file *coda_file, void *dirent,
> > filldir_t filldir)
> >
> > ret = -ENOENT;
> > if (!IS_DEADDIR(host_inode)) {
> > - ret = host_file->f_op->readdir(host_file, filldir,
> > dirent);
> > + ret = host_file->f_op->readdir(host_file, dirent,
> > filldir);
> > file_accessed(host_file);
> > }
> > }
>
> Today is not being a good day.
>
> This bug was already fixed. The following mistakes were made:
This bug never affected Coda clients, the relevant code path was used by
'potemkin' which was an extremely stripped down client that was used to
test the kernel module during it's initial development. It simply
exported an existing directory tree under the /coda namespace.
I noticed the bug when I moved the place where we lock the container
file which was triggering the lockdep warning and corrected the code. It
isn't a code path that our userspace uses, and considering how long the
bug has been present nobody else used either so I actually didn't even
remember fixing that bug when I was cherry picking patches out of my
development tree to pass upstream.
> a) Jan's patch was misleadingly titled "coda: avoid lockdep warning in
>coda_readdir"
>
> b) Jan's patch had no changelog
>
> c) Jen's patch was not cc'ed to any mailing list
>
> d) Al's patch was not sent to the maintainer. Nor to me. Nor was it
>staged in any tree which I can get at so that I could inform people of
>the upcoming conflict/duplication/etc.
I didn't realize that as the maintainer for Coda and only touching code
within fs/coda, I was expected to CC' the mailing lists. Especially
surprising since there have been quite a few patches to fs/coda on which
I haven't even been CC'd and which were never sent any mailing list
either. I see them only after they appear in the main kernel during the
merge window, which means I pretty much don't get a chance to update and
retest my development work before the window is closed again.
My patches weren't written last week, several of them have been around
since at least June 2005 and are probably even older than that,
http://www.coda.cs.cmu.edu/cgi-bin/gitweb.cgi?p=linux-coda.git;a=shortlog
Yes that is an externally built Coda kernel module, which most Coda
users who have a problem with the in-kernel one switch to. Even with
some of the known bugs, the in-kernel version works fine for most use
cases.
So this time I cherrypicked a limited subset of fixes and cleanups that
I think should go upstream. The cherry picks were for the most part
rewrites by looking at diffs between 2.6.22-rc and the external module
code that I maintain (which is why I didn't have nice changelogs).
> Guys (and I mean all guys): things are really shit at present. Please try
> to do better.
You can check with git how many patches went into fs/coda that I have
not been CC'd on, this problem has been ongoing for a while. I don't
mind it too much, clearly the patches were mostly minor changes which
did not break Coda.
Jan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: wrong order of arguments of -readdir()
On Mon, Jul 16, 2007 at 01:44:24PM -0700, Andrew Morton wrote:
On Sun, 15 Jul 2007 23:59:03 GMT
Linux Kernel Mailing List linux-kernel@vger.kernel.org wrote:
wrong order of arguments of -readdir()
Shows how many people are testing coda - the bug had been there for 5
years
and results of stepping on it are not subtle.
Signed-off-by: Al Viro [EMAIL PROTECTED]
Signed-off-by: Linus Torvalds [EMAIL PROTECTED]
---
fs/coda/dir.c |2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/fs/coda/dir.c b/fs/coda/dir.c
index 9ddf5ed..898a86d 100644
--- a/fs/coda/dir.c
+++ b/fs/coda/dir.c
@@ -470,7 +470,7 @@ int coda_readdir(struct file *coda_file, void *dirent,
filldir_t filldir)
ret = -ENOENT;
if (!IS_DEADDIR(host_inode)) {
- ret = host_file-f_op-readdir(host_file, filldir,
dirent);
+ ret = host_file-f_op-readdir(host_file, dirent,
filldir);
file_accessed(host_file);
}
}
Today is not being a good day.
This bug was already fixed. The following mistakes were made:
This bug never affected Coda clients, the relevant code path was used by
'potemkin' which was an extremely stripped down client that was used to
test the kernel module during it's initial development. It simply
exported an existing directory tree under the /coda namespace.
I noticed the bug when I moved the place where we lock the container
file which was triggering the lockdep warning and corrected the code. It
isn't a code path that our userspace uses, and considering how long the
bug has been present nobody else used either so I actually didn't even
remember fixing that bug when I was cherry picking patches out of my
development tree to pass upstream.
a) Jan's patch was misleadingly titled coda: avoid lockdep warning in
coda_readdir
b) Jan's patch had no changelog
c) Jen's patch was not cc'ed to any mailing list
d) Al's patch was not sent to the maintainer. Nor to me. Nor was it
staged in any tree which I can get at so that I could inform people of
the upcoming conflict/duplication/etc.
I didn't realize that as the maintainer for Coda and only touching code
within fs/coda, I was expected to CC' the mailing lists. Especially
surprising since there have been quite a few patches to fs/coda on which
I haven't even been CC'd and which were never sent any mailing list
either. I see them only after they appear in the main kernel during the
merge window, which means I pretty much don't get a chance to update and
retest my development work before the window is closed again.
My patches weren't written last week, several of them have been around
since at least June 2005 and are probably even older than that,
http://www.coda.cs.cmu.edu/cgi-bin/gitweb.cgi?p=linux-coda.git;a=shortlog
Yes that is an externally built Coda kernel module, which most Coda
users who have a problem with the in-kernel one switch to. Even with
some of the known bugs, the in-kernel version works fine for most use
cases.
So this time I cherrypicked a limited subset of fixes and cleanups that
I think should go upstream. The cherry picks were for the most part
rewrites by looking at diffs between 2.6.22-rc and the external module
code that I maintain (which is why I didn't have nice changelogs).
Guys (and I mean all guys): things are really shit at present. Please try
to do better.
You can check with git how many patches went into fs/coda that I have
not been CC'd on, this problem has been ongoing for a while. I don't
mind it too much, clearly the patches were mostly minor changes which
did not break Coda.
Jan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: wrong order of arguments of -readdir()
On Mon, Jul 16, 2007 at 09:49:30PM +0100, Al Viro wrote: c) Jan's patch was not cc'ed to any mailing list ... said patch (presumably being an obvious fix of obvious roothole) had not been sent to mainline immediately. The exploiting application would have to run in the role of the userspace cache manager. In other words, you need to, - open /dev/cfs0 - mount the Coda filesystem and pass the file handle of the opened cfs0 device as one of the mount options Both of these actions already require root, and if you can get this far without root priviledges it would be considerably easier to just hand a setuid root shell through the mounted Coda file system. So I didn't consider this a critical root hole. d) Al's patch was not sent to the maintainer. Nor to me. Nor was it staged in any tree which I can get at so that I could inform people of the upcoming conflict/duplication/etc. Coda is still maintained? That's certainly news to me - I'm not being sarcastic, but my impression had been that coda got abandoned several years ago. Well, in that case someone forgot to update the MAINTAINERS file. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: how about mutual compatibility between Linux's GPLv2 and GPLv3?
On Wed, Jun 27, 2007 at 08:08:08PM -0300, Alexandre Oliva wrote: > On Jun 26, 2007, Jan Harkes <[EMAIL PROTECTED]> wrote: > > > GPLv2 section 3. > > The source code for a work means the preferred form of the work for > > making modifications to it. > > > I believe this states that the source code has to be in the preferred > > form for making modifications and not some other form that sucks rocks. > > Yes, but in the scenario I proposed, the source code *is* in the > preferred form for making modifications, it just so happens to be > behind a barrier you cannot trespass. This is not different from > shipping binaries and sources in a CD inside a locked box that you > can't open. You've received both, but how is the fact that you can't > reach the source code (or the binaries) a violation of the GPL in this > case? That was explained in next paragraph where I said that preventing access sure sounds like a restriction on copying and modification, which is where the GPL violation would be. > As for making modifications, I'd like to take this opportunity to > withdraw, for purposes of interpretation, my earlier agreement that > TiVo permits modification, even though it doesn't permit modification > in place. I don't see any distinction in US copyright law between > modification in place and modification by creating a separate work. That doesn't really matter because the GPLv2 makes a distinction between source code and binary object code. For instance in section 1 where it says that you may copy and distribute the program's source code. In section 2 where it talks about modifications it at first doesn't seem to distinguish between source and binary, but it doesn't grant the right that the resulting modified program will actually work. And yes, you can still modify the kernel binary on the Tivo harddrive, it just won't boot with the standard bootloader. But there is an explicit no warranty clause in the GPLv2. Which I believe is a good thing. If the license tried to enforce usability, i.e. "continued functioning of the modified object code is in no case prevented or interfered with", then any time a new release of the program causes a regression for some users this would be a license violation. > This distinction makes sense for some cases of modification of > software, but it doesn't make sense for other copyrightable works, > such as sculptures, paintings, drawings, etc. > > When you modify a sculpture, you're modifying it in place, and this > requires as much copyright permission as creating a modified copy of > it. Both count as modification. And when you scribble in the margin of a book you are also modifying it, so what. I don't think you even need copyright permission, although you will probably get into trouble if the book was borrowed from a library. I don't see the relevance in the context of this discussion. > And if TiVo creates artificial > obstacles to your modifying the copy of GPLed programs that ships in > its devices, then I believe it counts as a restriction on > modification. You can modify the source, recompile it, even binary patch the kernel on the Tivo's harddrive. None of that is restricted. What is restricted is the freedom to run the modified object code on Tivo's hardware. That right is not granted by the GPLv2, and fitness for any purpose is explicitly disclaimed in the NO WARRANTY section. ... skipped a whole bunch, you actually repeated yourself a couple of times but the same answer applied, GPLv2 does not grant the right to be able to run the (modified) program on their hardware, and none of the granted rights, to copy, distribute and modify, are restricted ... > Here are variations of another scenario I proposed back then: > > - Tivoizing device ships with tivoized software, regardless of whether > the corresponding sources are accessible to the end user or hidden > inside the box, in a fully-encrypted disk that only that machine can > read. I'm assuming no written offer for access to the software is included. Source are inaccessible, restricted the recipients rights to copy, distribute and modify the source code, GPLv2 violation. otherwise, source is accessible and recipient can copy, distribute and modify. > - Network authenticates the device with help of the built-in crypto > device, and device requests feature-complete binaries in encrypted > network sessions. It's in the fature-complete binaries that the most > valuable improvements made by the tivoizer are, that they don't want > to share with anyone else. Sources *are* available behind the same > authentication machinery you don't can't get past. Whether the device > chooses to download them into the encrypted device you can't get to , > to dispose of them right away or even leave the
Re: how about mutual compatibility between Linux's GPLv2 and GPLv3?
On Wed, Jun 27, 2007 at 08:08:08PM -0300, Alexandre Oliva wrote: On Jun 26, 2007, Jan Harkes [EMAIL PROTECTED] wrote: GPLv2 section 3. The source code for a work means the preferred form of the work for making modifications to it. I believe this states that the source code has to be in the preferred form for making modifications and not some other form that sucks rocks. Yes, but in the scenario I proposed, the source code *is* in the preferred form for making modifications, it just so happens to be behind a barrier you cannot trespass. This is not different from shipping binaries and sources in a CD inside a locked box that you can't open. You've received both, but how is the fact that you can't reach the source code (or the binaries) a violation of the GPL in this case? That was explained in next paragraph where I said that preventing access sure sounds like a restriction on copying and modification, which is where the GPL violation would be. As for making modifications, I'd like to take this opportunity to withdraw, for purposes of interpretation, my earlier agreement that TiVo permits modification, even though it doesn't permit modification in place. I don't see any distinction in US copyright law between modification in place and modification by creating a separate work. That doesn't really matter because the GPLv2 makes a distinction between source code and binary object code. For instance in section 1 where it says that you may copy and distribute the program's source code. In section 2 where it talks about modifications it at first doesn't seem to distinguish between source and binary, but it doesn't grant the right that the resulting modified program will actually work. And yes, you can still modify the kernel binary on the Tivo harddrive, it just won't boot with the standard bootloader. But there is an explicit no warranty clause in the GPLv2. Which I believe is a good thing. If the license tried to enforce usability, i.e. continued functioning of the modified object code is in no case prevented or interfered with, then any time a new release of the program causes a regression for some users this would be a license violation. This distinction makes sense for some cases of modification of software, but it doesn't make sense for other copyrightable works, such as sculptures, paintings, drawings, etc. When you modify a sculpture, you're modifying it in place, and this requires as much copyright permission as creating a modified copy of it. Both count as modification. And when you scribble in the margin of a book you are also modifying it, so what. I don't think you even need copyright permission, although you will probably get into trouble if the book was borrowed from a library. I don't see the relevance in the context of this discussion. And if TiVo creates artificial obstacles to your modifying the copy of GPLed programs that ships in its devices, then I believe it counts as a restriction on modification. You can modify the source, recompile it, even binary patch the kernel on the Tivo's harddrive. None of that is restricted. What is restricted is the freedom to run the modified object code on Tivo's hardware. That right is not granted by the GPLv2, and fitness for any purpose is explicitly disclaimed in the NO WARRANTY section. ... skipped a whole bunch, you actually repeated yourself a couple of times but the same answer applied, GPLv2 does not grant the right to be able to run the (modified) program on their hardware, and none of the granted rights, to copy, distribute and modify, are restricted ... Here are variations of another scenario I proposed back then: - Tivoizing device ships with tivoized software, regardless of whether the corresponding sources are accessible to the end user or hidden inside the box, in a fully-encrypted disk that only that machine can read. I'm assuming no written offer for access to the software is included. Source are inaccessible, restricted the recipients rights to copy, distribute and modify the source code, GPLv2 violation. otherwise, source is accessible and recipient can copy, distribute and modify. - Network authenticates the device with help of the built-in crypto device, and device requests feature-complete binaries in encrypted network sessions. It's in the fature-complete binaries that the most valuable improvements made by the tivoizer are, that they don't want to share with anyone else. Sources *are* available behind the same authentication machinery you don't can't get past. Whether the device chooses to download them into the encrypted device you can't get to , to dispose of them right away or even leave them around, or it simply refrains from downloading the sources because it doesn't need them, the end result is the same: you've received the sources over the network, but you can't get to them. If it doesn't download
Re: how about mutual compatibility between Linux's GPLv2 and GPLv3?
On Tue, Jun 26, 2007 at 04:47:34AM -0300, Alexandre Oliva wrote: > On Jun 26, 2007, Alexandre Oliva <[EMAIL PROTECTED]> wrote: > > On Jun 26, 2007, Jan Harkes <[EMAIL PROTECTED]> wrote: > >> You could argue that they do not restrict copying, distribution > >> and modification of the sources in general, only of the specific copy > >> they distribute. > > > "We don't oppose that you do any of these things, once you get the > > source code. We just make it difficult (hopefully impossible) that > > you'll get to the source code in the first place." > > Actually, they could even claim you don't need the source code to make > modifications. The license even says the source code is the most > convenient form to make modifications, not the only form. Now, that > the others suck rocks is not their fault, is it? ;-) GPLv2 section 3. The source code for a work means the preferred form of the work for making modifications to it. I believe this states that the source code has to be in the preferred form for making modifications and not some other form that sucks rocks. As far as your argument that making it difficult or impossible to obtain the source code could be in compliance. I don't see how (intentionally) making something difficult could not be interpreted as a restriction so it still violates section 6 of the GPLv2. And yes, I do realize that you intentionally tried to come up with your example to somehow bring tivoization to the source code. However although the GPLv2 doesn't address the freedom to use (and specifically does not grants the user a right to run a modified version of the program on some specific piece of hardware), it does (try) to address the recipients rights to copy, distribute and modify. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: how about mutual compatibility between Linux's GPLv2 and GPLv3?
On Tue, Jun 26, 2007 at 04:47:34AM -0300, Alexandre Oliva wrote: On Jun 26, 2007, Alexandre Oliva [EMAIL PROTECTED] wrote: On Jun 26, 2007, Jan Harkes [EMAIL PROTECTED] wrote: You could argue that they do not restrict copying, distribution and modification of the sources in general, only of the specific copy they distribute. We don't oppose that you do any of these things, once you get the source code. We just make it difficult (hopefully impossible) that you'll get to the source code in the first place. Actually, they could even claim you don't need the source code to make modifications. The license even says the source code is the most convenient form to make modifications, not the only form. Now, that the others suck rocks is not their fault, is it? ;-) GPLv2 section 3. The source code for a work means the preferred form of the work for making modifications to it. I believe this states that the source code has to be in the preferred form for making modifications and not some other form that sucks rocks. As far as your argument that making it difficult or impossible to obtain the source code could be in compliance. I don't see how (intentionally) making something difficult could not be interpreted as a restriction so it still violates section 6 of the GPLv2. And yes, I do realize that you intentionally tried to come up with your example to somehow bring tivoization to the source code. However although the GPLv2 doesn't address the freedom to use (and specifically does not grants the user a right to run a modified version of the program on some specific piece of hardware), it does (try) to address the recipients rights to copy, distribute and modify. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: how about mutual compatibility between Linux's GPLv2 and GPLv3?
On Mon, Jun 25, 2007 at 04:54:52PM -0300, Alexandre Oliva wrote: > Consider this scenario: vendor tivoizes Linux in the device, and > includes the corresponding sources only in a partition that is > theoretically accessible using the shipped kernel, but that nothing in > the software available in the machine will let you get to. Further, > sources (like everything else on disk) are encrypted, and you can only Interesting scenario, it seems to comply with GPLv2 on the surface. If that kernel doesn't actually allow access and wipes the source partition to use it as swap on first boot, then no machine is actually capable of reading the source. So it isn't really machine readable. Another gripe is that encrypted media are not customarily used for software interchange. So that's 2 (minor) strikes where this method of distribution doesn't seem to match the language of section 3a. You also cannot interpret the encrypted partition as source code because a bit further down in section 3, it defines source code as, "The source code for a work means the preferred form of the work for making modifications to it." So now we get to section 6. The recipient receives a license to copy, distribute or modify. You may not impose further restrictions on these rights granted herein. You could argue that they do not restrict copying, distribution and modification of the sources in general, only of the specific copy they distribute. However here we go back to section 2 which states that their modified copy is a derived work which must be licensed under the GPLv2, so that would make it specific enough that recipients have in fact been granted the right to copy, distribute and modify the copy of the source of that corresponds to the distributed binaries, which is restricted because of the encryption which prevents the user to copy, distribute or modify the source code. > Does anyone think this is permitted by the letter of GPLv2? No. > How are the sources passed on in this way going to benefit the user or > the community? Not a really interesting question if the method of distribution violates the letter of the GPLv2, is it? They get sued for copyright infringement because they are not in compliance with section 3 and the sources are released as a result. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: how about mutual compatibility between Linux's GPLv2 and GPLv3?
On Mon, Jun 25, 2007 at 04:54:52PM -0300, Alexandre Oliva wrote: Consider this scenario: vendor tivoizes Linux in the device, and includes the corresponding sources only in a partition that is theoretically accessible using the shipped kernel, but that nothing in the software available in the machine will let you get to. Further, sources (like everything else on disk) are encrypted, and you can only Interesting scenario, it seems to comply with GPLv2 on the surface. If that kernel doesn't actually allow access and wipes the source partition to use it as swap on first boot, then no machine is actually capable of reading the source. So it isn't really machine readable. Another gripe is that encrypted media are not customarily used for software interchange. So that's 2 (minor) strikes where this method of distribution doesn't seem to match the language of section 3a. You also cannot interpret the encrypted partition as source code because a bit further down in section 3, it defines source code as, The source code for a work means the preferred form of the work for making modifications to it. So now we get to section 6. The recipient receives a license to copy, distribute or modify. You may not impose further restrictions on these rights granted herein. You could argue that they do not restrict copying, distribution and modification of the sources in general, only of the specific copy they distribute. However here we go back to section 2 which states that their modified copy is a derived work which must be licensed under the GPLv2, so that would make it specific enough that recipients have in fact been granted the right to copy, distribute and modify the copy of the source of that corresponds to the distributed binaries, which is restricted because of the encryption which prevents the user to copy, distribute or modify the source code. Does anyone think this is permitted by the letter of GPLv2? No. How are the sources passed on in this way going to benefit the user or the community? Not a really interesting question if the method of distribution violates the letter of the GPLv2, is it? They get sued for copyright infringement because they are not in compliance with section 3 and the sources are released as a result. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: how about mutual compatibility between Linux's GPLv2 and GPLv3?
On Fri, Jun 22, 2007 at 01:14:27AM -0300, Alexandre Oliva wrote: > On Jun 21, 2007, Jan Harkes <[EMAIL PROTECTED]> wrote: > > On Thu, Jun 21, 2007 at 08:23:57PM -0300, Alexandre Oliva wrote: > >> It's not like anyone can safely tivoize devices with GPLv2 already, > > > So you really didn't pay any attention to anything people told you? > > Yes. Particularly to what Alan Cox and his legal counsel told me. A > single copyright holder able and willing to enforce the license > against tivoization is enough. What part of this don't you > understand? > > > The license does not grant the right that you will be able to run the > > software on any particular platform or whether it will even work at all. > > It doesn't grant TiVo the right to distribute the program without > corresponding source code. > > They fail to distribute the source code to the functional signature > derived from the kernel binary. > > Kaboom! A signature is not a creative work and as such not covered by copyright. At the moment the only protection that the signature/key has is that of a trade secret, the GPLv2 does not cover that type of intellectual propery and does not grant the licensee access to trade secrets. Show me any case law that indicates otherwise. Maybe the content producers will at some point manage to establish that encryption keys and signatures are somehow copyrightable, but until a court of law makes that determination there is no kaboom here. > As for the right to run the program, I've also explained why in > Brazilian copyright law this is a right granted by the license, > because the license says that right is unrestricted. > > Kaboom! No, the license says that it does not address the right to run a program, and states that the license does not impose restrictions. That is quite different from saying that the right is unrestricted. So again, not much of a kaboom here either. > > A mutual compatibility agreement (sublicense) would effectively > > terminate any rights granted by the GPLv2, > > Additional permissions to combine are not permission to relicense. > Look at section 13 of GPLv3dd4 and of Affero GPLv3dd1. That's the > sort of additional permission I'm talking about here. I'm talking about the GPLv2, so it doesn't matter what the GPLv3 says wrt. additional permissions. Even if GPLv2 and GPLv3 grant the same freedoms (permissions), the collective work would have additional restrictions because of the GPLv3 code and as such the combined work would result in a sublicensing of the GPLv2 licensed code, which is explicitly not permitted. > The same kind of additional permission that GPLed projects that use > openssl adopt. If they are the original author they can make that decision, just like authors can dual-license, or decide to license their code as GPLv2+. It is kind of funny how they phrase the exception as granting permission to link against OpenSSL, where in reality they accept the added restriction that results from the advertising clause of the BSD license. (i.e. instead of granting you an additional freedom, they chose to remove the freedom to modify the part of the code that advertises). However with the openssl case there is no tight coupling because openssl is a separate library. Some people have argued that the LGPL was never really necessary because (unlike static libraries) shared libraries are still separable from the GPL'd program. Furthermore, those projects are not pulling individual source files from into their project. There are also alternative crypto libraries (gnutls, nessie) which can in many cases easily replace the openssl dependency. Finally the original author accepts the additional restriction that comes from the BSD + advertising clause, while the GPLv2 authors do not accept the additional restrictions they would inherit from the GPLv3 otherwise they would re-license their code to GPLv2+. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: how about mutual compatibility between Linux's GPLv2 and GPLv3?
On Thu, Jun 21, 2007 at 08:23:57PM -0300, Alexandre Oliva wrote: > On Jun 21, 2007, "David Schwartz" <[EMAIL PROTECTED]> wrote: > > >> > Wouldn't that defeat the entire purpose of the GPLv3? Couldn't > >> > I take any > >> > GPLv3 program, combine it with a few lines of Linux code, and > >> > Tivoize the > >> > result? > > >> No. This is not permission to relicense. This is permission to > >> combine. Each author still gets to enforce the terms of her own code. > > > This makes no sense. We are not talking mere aggregation here, we are > > talking developmental convergence. If I wrote some code that was in the > > Linux kernel, how can I enforce the terms of my code (guaranteed write to > > Tivoize) in the derivative work that it becomes mixed with? > > In just the same way you'd enforce it today: with help from a lawyer > who understands these issues that you clearly don't understand. > > >> So a tivoizer would have to take out the code licensed under the > >> GPLv3, and use only the code that permits tivoization. Same as today, > >> but without the possibility of cooperation for licensees who don't > >> tivoize. > > > I am baffled how this could possibly work. You understand that the GPLv2 > > specifically guarantees that any derivative work will be Tivoizable and the > > people who chose the GPLv2 specifically want it that way? > > Yes. And the GPLv2 code would remain that way. > > If GPLv3 had this provision I suggested, and you wanted to cooperate > with some other project that offered GPLv3 drivers, then you could use > them by adding the mutual-cooperation provision I suggested. > > Of course you're free to not want to cooperate with anyone else who > doesn't share your opinion. That's your call. > > > If I chose the GPLv2 over the GPLv3 as a conscious choice, that means I want > > people to be able to Tivoize any derivative work made from my work that is > > distributed. > > This provision was not intended to prevent anyone from tivoizing your > work or derived works thereof. It was only intended to enable you to > use code from GPLv3 projects as long as these GPLv3 projects could > also use your code. Mutual cooperation, as opposed to no cooperation > whatsoever. > > I *think* lawyers would probably recommend you to keep code under > different licenses in separate files, like you already do with code > licensed under GPLv2-compatible licenses. > > I *think* that, with this pair of mutual cooperation provisions, you > could even use code licensed under the Apache 2.0 license. And > OpenSolaris drivers, if it's licensed under GPLv3. > > And you wouldn't be departing from your intent to enable people to > tivoize your code, which you currently choose not to enforce even > though GPLv2 might very well enable you to; you could keep on > cooperating with people who understand GPLv2 doesn't permit > tivoization, and you'd be able to establish mutual cooperation with > people who choose a license that makes this point clear. > > It's not like anyone can safely tivoize devices with GPLv2 already, so > refusing to cooperate with GPLv3 on these grounds buys you nothing. > It is only a public statement of refusal to cooperate, with you are > entitled to make, even if it comes off as silly because you chose a > license that already contains the provisions for "complete > corresponding source code" and "no further restrictions on the > exercise of the rights granted by the license" that tivoizers fail to > comply with. So you really didn't pay any attention to anything people told you? Ok, here are the relevant parts from GPLv2, The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION ^^^ 0. ... Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. ... 6. ... You may not impose any further restrictions on the recipients' exercise of the rights granted herein. ... 11. ... THERE IS NO WARRANTY ... INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. ... The license does not grant the right that you will be able to run the software on any particular platform or whether it will even work at all. You are only granted the rights to _COPY_, _DISTRIBUTE_, and _MODIFY_. Tivo in no way limits your ability to exercise any of these rights. > FWIW, all of my (very few) contributions to Linux were made with the > intent of not permitting tivoization or any form of restricting users > freedoms. I guess this means, from now on, you'd stop
Re: how about mutual compatibility between Linux's GPLv2 and GPLv3?
On Thu, Jun 21, 2007 at 08:23:57PM -0300, Alexandre Oliva wrote: On Jun 21, 2007, David Schwartz [EMAIL PROTECTED] wrote: Wouldn't that defeat the entire purpose of the GPLv3? Couldn't I take any GPLv3 program, combine it with a few lines of Linux code, and Tivoize the result? No. This is not permission to relicense. This is permission to combine. Each author still gets to enforce the terms of her own code. This makes no sense. We are not talking mere aggregation here, we are talking developmental convergence. If I wrote some code that was in the Linux kernel, how can I enforce the terms of my code (guaranteed write to Tivoize) in the derivative work that it becomes mixed with? In just the same way you'd enforce it today: with help from a lawyer who understands these issues that you clearly don't understand. So a tivoizer would have to take out the code licensed under the GPLv3, and use only the code that permits tivoization. Same as today, but without the possibility of cooperation for licensees who don't tivoize. I am baffled how this could possibly work. You understand that the GPLv2 specifically guarantees that any derivative work will be Tivoizable and the people who chose the GPLv2 specifically want it that way? Yes. And the GPLv2 code would remain that way. If GPLv3 had this provision I suggested, and you wanted to cooperate with some other project that offered GPLv3 drivers, then you could use them by adding the mutual-cooperation provision I suggested. Of course you're free to not want to cooperate with anyone else who doesn't share your opinion. That's your call. If I chose the GPLv2 over the GPLv3 as a conscious choice, that means I want people to be able to Tivoize any derivative work made from my work that is distributed. This provision was not intended to prevent anyone from tivoizing your work or derived works thereof. It was only intended to enable you to use code from GPLv3 projects as long as these GPLv3 projects could also use your code. Mutual cooperation, as opposed to no cooperation whatsoever. I *think* lawyers would probably recommend you to keep code under different licenses in separate files, like you already do with code licensed under GPLv2-compatible licenses. I *think* that, with this pair of mutual cooperation provisions, you could even use code licensed under the Apache 2.0 license. And OpenSolaris drivers, if it's licensed under GPLv3. And you wouldn't be departing from your intent to enable people to tivoize your code, which you currently choose not to enforce even though GPLv2 might very well enable you to; you could keep on cooperating with people who understand GPLv2 doesn't permit tivoization, and you'd be able to establish mutual cooperation with people who choose a license that makes this point clear. It's not like anyone can safely tivoize devices with GPLv2 already, so refusing to cooperate with GPLv3 on these grounds buys you nothing. It is only a public statement of refusal to cooperate, with you are entitled to make, even if it comes off as silly because you chose a license that already contains the provisions for complete corresponding source code and no further restrictions on the exercise of the rights granted by the license that tivoizers fail to comply with. So you really didn't pay any attention to anything people told you? Ok, here are the relevant parts from GPLv2, The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION ^^^ 0. ... Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. ... 6. ... You may not impose any further restrictions on the recipients' exercise of the rights granted herein. ... 11. ... THERE IS NO WARRANTY ... INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. ... The license does not grant the right that you will be able to run the software on any particular platform or whether it will even work at all. You are only granted the rights to _COPY_, _DISTRIBUTE_, and _MODIFY_. Tivo in no way limits your ability to exercise any of these rights. FWIW, all of my (very few) contributions to Linux were made with the intent of not permitting tivoization or any form of restricting users freedoms. I guess this means, from now on, you'd stop accepting my contributions and take out whatever contributions I've made, since Only if the terms of your
Re: how about mutual compatibility between Linux's GPLv2 and GPLv3?
On Fri, Jun 22, 2007 at 01:14:27AM -0300, Alexandre Oliva wrote: On Jun 21, 2007, Jan Harkes [EMAIL PROTECTED] wrote: On Thu, Jun 21, 2007 at 08:23:57PM -0300, Alexandre Oliva wrote: It's not like anyone can safely tivoize devices with GPLv2 already, So you really didn't pay any attention to anything people told you? Yes. Particularly to what Alan Cox and his legal counsel told me. A single copyright holder able and willing to enforce the license against tivoization is enough. What part of this don't you understand? The license does not grant the right that you will be able to run the software on any particular platform or whether it will even work at all. It doesn't grant TiVo the right to distribute the program without corresponding source code. They fail to distribute the source code to the functional signature derived from the kernel binary. Kaboom! A signature is not a creative work and as such not covered by copyright. At the moment the only protection that the signature/key has is that of a trade secret, the GPLv2 does not cover that type of intellectual propery and does not grant the licensee access to trade secrets. Show me any case law that indicates otherwise. Maybe the content producers will at some point manage to establish that encryption keys and signatures are somehow copyrightable, but until a court of law makes that determination there is no kaboom here. As for the right to run the program, I've also explained why in Brazilian copyright law this is a right granted by the license, because the license says that right is unrestricted. Kaboom! No, the license says that it does not address the right to run a program, and states that the license does not impose restrictions. That is quite different from saying that the right is unrestricted. So again, not much of a kaboom here either. A mutual compatibility agreement (sublicense) would effectively terminate any rights granted by the GPLv2, Additional permissions to combine are not permission to relicense. Look at section 13 of GPLv3dd4 and of Affero GPLv3dd1. That's the sort of additional permission I'm talking about here. I'm talking about the GPLv2, so it doesn't matter what the GPLv3 says wrt. additional permissions. Even if GPLv2 and GPLv3 grant the same freedoms (permissions), the collective work would have additional restrictions because of the GPLv3 code and as such the combined work would result in a sublicensing of the GPLv2 licensed code, which is explicitly not permitted. The same kind of additional permission that GPLed projects that use openssl adopt. If they are the original author they can make that decision, just like authors can dual-license, or decide to license their code as GPLv2+. It is kind of funny how they phrase the exception as granting permission to link against OpenSSL, where in reality they accept the added restriction that results from the advertising clause of the BSD license. (i.e. instead of granting you an additional freedom, they chose to remove the freedom to modify the part of the code that advertises). However with the openssl case there is no tight coupling because openssl is a separate library. Some people have argued that the LGPL was never really necessary because (unlike static libraries) shared libraries are still separable from the GPL'd program. Furthermore, those projects are not pulling individual source files from into their project. There are also alternative crypto libraries (gnutls, nessie) which can in many cases easily replace the openssl dependency. Finally the original author accepts the additional restriction that comes from the BSD + advertising clause, while the GPLv2 authors do not accept the additional restrictions they would inherit from the GPLv3 otherwise they would re-license their code to GPLv2+. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
On Tue, Jun 19, 2007 at 06:20:24PM -0300, Alexandre Oliva wrote: > On Jun 19, 2007, Jan Harkes <[EMAIL PROTECTED]> wrote: > > and which will most likely make GPLv3 software unusable for various > > applications ranging from medical equipment to financial transaction > > systems (and probably others) > > Not unusable, except perhaps for the one example about credit card > terminals presented so far. > > > is there to just make it a _bit_ more inconvenient for vendors to > > implement a tivo-like scheme? > > I'm not sure they find it to be "just a bit". > > Point is to keep Free Software Free freedoms, and ROM doesn't make it > non-Free, so this provision is a means to ensure the compliance with > the wishes of users who want their software to not be used in ways > that make it non-Free. You keep referring to the four freedoms so I googled for them and found http://www.gnu.org/philosophy/free-sw.html So which of the freedoms did Tivo take away? * The freedom to run the program, for any purpose (freedom 0). * The freedom to study how the program works, and adapt it to your needs (freedom 1). Access to the source code is a precondition for this. * The freedom to redistribute copies so you can help your neighbor (freedom 2). * The freedom to improve the program, and release your improvements to the public, so that the whole community benefits (freedom 3). Access to the source code is a precondition for this. It doesn't seem to me they took away freedoms 1, 2 or 3. They released the source to any free software components and we can study, modify, redistribute, improve and release our improvements for the benefit of the whole community. btw. freedom 3 seems to be just repeating what we already got from freedoms 1 and 2. So the only one we could differ in opinion about is freedom 0. I would say that they in no way are limiting my use of the Linux kernel (which is the part I mostly care about) I can run the program for any purpose I see fit. What if I want to run mythtv on my PC at home? Tivo has no control whether or not I can do so even when my kernel contains any of their modification or improvements, so I claim that I in fact retained freedom 0. Your position (and I hope I will get this right), is that they should allow you to run the program for any purpose _on the hardware it is installed on_. Interpreted that way, the whole modification part doesn't even come into play, your freedom is taken away if you cannot even run your own software on top of the already installed kernel. This 'freedom 0' the way (I hope) you are interpreting it, could clearly never have been protected by the GPLv2 since it explicitly does not cover "Activities other than copying, distribution and modification" (GPLv2, term 0) However the GPLv3 does not seem to address this point either, the whole discussion in section 5 about user products and installation information completely misses the fact that none of the 'four freedoms' (which I assume formed the foundation for the license) is about allowing a user to install the program on some particular piece of hardware and that is exactly where I think all this anti-tivoization language is going wrong. It is clearly having a hard time pinning down the exact requirements for something that was not well defined in the first place. The real issue with Tivo isn't that they signed their kernel or their initrd, but that they do not allow you to use that kernel for any purpose, for instance run mythtv or a webserver on their unmodified kernel. Maybe the GPLv3 shouldn't try to talk about license keys or installation information and then heap on some exceptions for non-consumer devices or rom-based implementations and then some further legaleze patches to close the really obvious loopholes. Maybe it if it actually addressed the fact that you want to be able to use the distributed software for any purpose you see fit by allowing you to run your own applications on the kernel they distributed. I still believe they do allow me to use the program for any purpose as they can not limit my use of the Linux kernel whether or not my copy contains any of their contributions, so excuse my while I'll go enjoy some more of my freedom 0. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Versioning file system
On Tue, Jun 19, 2007 at 03:13:33PM -0700, H. Peter Anvin wrote: > [EMAIL PROTECTED] wrote: > > > > the only trouble I ever had with the .snapshot approach is when tar or > > find would decend down into the .snapshot when I didn't really intend > > for it to do so. > > > > Netapp optionally made .snapshot not show up in readdir, which solved > that problem. > > I have a bigger issue with it starting with only one dot, which is > traditionally used for user configuration information. I think > ..snapshot would have been a better choice, and by extension leaving > double-dot filenames as filesystem namespace (I know that contradicts > POSIX, but the only namespace available in POSIX is /../ which is a > global namespace.) Still, anything that depends on increasing the length of file or path names to refer to different versions will encounter problems for long file names and deep paths because there is an upper limit on file and path name lengths. It may be possible to use namespaces where an application can change the view it (and it's children) will have on the storage by switching to a different namespace and tagging it with for instance a specific version or date we're interested in. Not sure if we actually pass namespace information down to the file systems yet though, last time I checked they were only visible for the VFS. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
On Tue, Jun 19, 2007 at 02:40:59AM -0300, Alexandre Oliva wrote: > > The actual software is mailed to you on a credit card sized > > ROM when you activate service. ... > The GPLv3 won't remove every way in which people who want/need to stop > the user from making changes to the software could accomplishing this > (ROM). It will just make this a bit more inconvenient, such that > vendors that have the option respect users' freedoms, and those that > find it too inconvenient respect the wishes of users who don't want > their software turned non-free. I am trying to read that last sentence and it just doesn't seem to make any sense. Or are you saying that all that anti-tivoization language that adds complex requirements which change depending on the market some device happens to be sold in and which will most likely make GPLv3 software unusable for various applications ranging from medical equipment to financial transaction systems (and probably others) is there to just make it a _bit_ more inconvenient for vendors to implement a tivo-like scheme? So what exactly is the point of all this then? Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
On Tue, Jun 19, 2007 at 02:40:59AM -0300, Alexandre Oliva wrote: The actual software is mailed to you on a credit card sized ROM when you activate service. ... The GPLv3 won't remove every way in which people who want/need to stop the user from making changes to the software could accomplishing this (ROM). It will just make this a bit more inconvenient, such that vendors that have the option respect users' freedoms, and those that find it too inconvenient respect the wishes of users who don't want their software turned non-free. I am trying to read that last sentence and it just doesn't seem to make any sense. Or are you saying that all that anti-tivoization language that adds complex requirements which change depending on the market some device happens to be sold in and which will most likely make GPLv3 software unusable for various applications ranging from medical equipment to financial transaction systems (and probably others) is there to just make it a _bit_ more inconvenient for vendors to implement a tivo-like scheme? So what exactly is the point of all this then? Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Versioning file system
On Tue, Jun 19, 2007 at 03:13:33PM -0700, H. Peter Anvin wrote: [EMAIL PROTECTED] wrote: the only trouble I ever had with the .snapshot approach is when tar or find would decend down into the .snapshot when I didn't really intend for it to do so. Netapp optionally made .snapshot not show up in readdir, which solved that problem. I have a bigger issue with it starting with only one dot, which is traditionally used for user configuration information. I think ..snapshot would have been a better choice, and by extension leaving double-dot filenames as filesystem namespace (I know that contradicts POSIX, but the only namespace available in POSIX is /../ which is a global namespace.) Still, anything that depends on increasing the length of file or path names to refer to different versions will encounter problems for long file names and deep paths because there is an upper limit on file and path name lengths. It may be possible to use namespaces where an application can change the view it (and it's children) will have on the storage by switching to a different namespace and tagging it with for instance a specific version or date we're interested in. Not sure if we actually pass namespace information down to the file systems yet though, last time I checked they were only visible for the VFS. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
On Tue, Jun 19, 2007 at 06:20:24PM -0300, Alexandre Oliva wrote: On Jun 19, 2007, Jan Harkes [EMAIL PROTECTED] wrote: and which will most likely make GPLv3 software unusable for various applications ranging from medical equipment to financial transaction systems (and probably others) Not unusable, except perhaps for the one example about credit card terminals presented so far. is there to just make it a _bit_ more inconvenient for vendors to implement a tivo-like scheme? I'm not sure they find it to be just a bit. Point is to keep Free Software Free freedoms, and ROM doesn't make it non-Free, so this provision is a means to ensure the compliance with the wishes of users who want their software to not be used in ways that make it non-Free. You keep referring to the four freedoms so I googled for them and found http://www.gnu.org/philosophy/free-sw.html So which of the freedoms did Tivo take away? * The freedom to run the program, for any purpose (freedom 0). * The freedom to study how the program works, and adapt it to your needs (freedom 1). Access to the source code is a precondition for this. * The freedom to redistribute copies so you can help your neighbor (freedom 2). * The freedom to improve the program, and release your improvements to the public, so that the whole community benefits (freedom 3). Access to the source code is a precondition for this. It doesn't seem to me they took away freedoms 1, 2 or 3. They released the source to any free software components and we can study, modify, redistribute, improve and release our improvements for the benefit of the whole community. btw. freedom 3 seems to be just repeating what we already got from freedoms 1 and 2. So the only one we could differ in opinion about is freedom 0. I would say that they in no way are limiting my use of the Linux kernel (which is the part I mostly care about) I can run the program for any purpose I see fit. What if I want to run mythtv on my PC at home? Tivo has no control whether or not I can do so even when my kernel contains any of their modification or improvements, so I claim that I in fact retained freedom 0. Your position (and I hope I will get this right), is that they should allow you to run the program for any purpose _on the hardware it is installed on_. Interpreted that way, the whole modification part doesn't even come into play, your freedom is taken away if you cannot even run your own software on top of the already installed kernel. This 'freedom 0' the way (I hope) you are interpreting it, could clearly never have been protected by the GPLv2 since it explicitly does not cover Activities other than copying, distribution and modification (GPLv2, term 0) However the GPLv3 does not seem to address this point either, the whole discussion in section 5 about user products and installation information completely misses the fact that none of the 'four freedoms' (which I assume formed the foundation for the license) is about allowing a user to install the program on some particular piece of hardware and that is exactly where I think all this anti-tivoization language is going wrong. It is clearly having a hard time pinning down the exact requirements for something that was not well defined in the first place. The real issue with Tivo isn't that they signed their kernel or their initrd, but that they do not allow you to use that kernel for any purpose, for instance run mythtv or a webserver on their unmodified kernel. Maybe the GPLv3 shouldn't try to talk about license keys or installation information and then heap on some exceptions for non-consumer devices or rom-based implementations and then some further legaleze patches to close the really obvious loopholes. Maybe it if it actually addressed the fact that you want to be able to use the distributed software for any purpose you see fit by allowing you to run your own applications on the kernel they distributed. I still believe they do allow me to use the program for any purpose as they can not limit my use of the Linux kernel whether or not my copy contains any of their contributions, so excuse my while I'll go enjoy some more of my freedom 0. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
On Mon, Jun 18, 2007 at 08:31:30PM -0300, Alexandre Oliva wrote: > On Jun 18, 2007, Linus Torvalds <[EMAIL PROTECTED]> wrote: > > In the GPLv3 world, we have already discussed in this thread how you can > > follow the GPLv3 by making the TECHNICALLY INFERIOR choice of using a ROM > > instead of using a flash device. > > Yes. This is one option that doesn't bring any benefits to anyone. > It maintains the status quo for users and the community, but it loses > the ability for the vendor to upgrade, fix or otherwise control the > users. Bad for the vendor. Not really, Tivo could simply sell you a box without any installed software. The actual software is mailed to you on a credit card sized ROM when you activate service. When they want to (or need to) update the software they send out a new ROM card, maybe yearly as part of the service subscription renewal. The box could even be sold by third party vendors, I think they may even have started off that way, my old Series 1 had a big Philips logo on it. So now we make sure that this hardware refuses to boot any unsigned code, but it wasn't shipped containing GPLv3 software, so it's license terms simply does not apply. The software is shipped on a ROM card which can no longer be modified by the manufacturer or any third party, so it would seem to comply with the GPLv3. I can even imagine that the hardware is really general purpose but the ROM is encrypted so that only the BIOS/bootloader can unlock it. So the GPLv3 seems to fall short on actually preventing tivoization. It just requires an extra layer of indirection, ship hardware seperately from software. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
On Mon, Jun 18, 2007 at 08:31:30PM -0300, Alexandre Oliva wrote: On Jun 18, 2007, Linus Torvalds [EMAIL PROTECTED] wrote: In the GPLv3 world, we have already discussed in this thread how you can follow the GPLv3 by making the TECHNICALLY INFERIOR choice of using a ROM instead of using a flash device. Yes. This is one option that doesn't bring any benefits to anyone. It maintains the status quo for users and the community, but it loses the ability for the vendor to upgrade, fix or otherwise control the users. Bad for the vendor. Not really, Tivo could simply sell you a box without any installed software. The actual software is mailed to you on a credit card sized ROM when you activate service. When they want to (or need to) update the software they send out a new ROM card, maybe yearly as part of the service subscription renewal. The box could even be sold by third party vendors, I think they may even have started off that way, my old Series 1 had a big Philips logo on it. So now we make sure that this hardware refuses to boot any unsigned code, but it wasn't shipped containing GPLv3 software, so it's license terms simply does not apply. The software is shipped on a ROM card which can no longer be modified by the manufacturer or any third party, so it would seem to comply with the GPLv3. I can even imagine that the hardware is really general purpose but the ROM is encrypted so that only the BIOS/bootloader can unlock it. So the GPLv3 seems to fall short on actually preventing tivoization. It just requires an extra layer of indirection, ship hardware seperately from software. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
On Sun, Jun 17, 2007 at 05:17:57AM -0300, Alexandre Oliva wrote: > Just make the tivoization machinery require two keys: one that the > vendor keeps, one that the vendor gives to the user (maybe without > ever knowing it). Neither one can install modifications alone, but > the user can approve modifications by the vendor, and the vendor can > approve modifications by the user. This is still not ideal, but it at > least doesn't permit the vendor to remove features from under the > user. So what features has Tivo removed (or threatened to remove) from the GPL licensed parts? I think at some point they disabled the _undocumented_ skip feature in their own proprietary software, and ended up reenabling it when a lot of customers complained. Even if those customers had the ability to replace any GPL licensed parts, it would not have reenabled the feature. And it was an undocumented easter egg type thing at that, it isn't like they widely announced it in their advertising or as a selling point. So Google is using Linux right. What if they remove some feature? (let's pick a randomg one, i.e. phone number lookup) Should I get a keycard for their machine room to fix the problem, or maybe we should use some secret sharing mechanism to prevent them from removing the feature. > RMS does not want TiVo (or anyone else) to disrespect users' freedoms, > and installing technical measures to prevent users from adapting the > software to suit their needs and running their modifications is > disrespecting users freedoms. So if Tivo would allow you to boot your own kernel, but keeps the harddisk encrypted if the booted kernel does not have the right signature? In such a case you can run your own kernel and if you replace the harddrive you can install all the applications you might want. You cannot however use their software, any of the recorded content or obtain any further guide data/service updates. And how is that any different from taking an off-the-shelf PC and booting your own kernel with Tivo's modifications? Or really different from the current situation. Tivo complied in as far as they made GPL licensed code available, you can examine it, modify it, compile it. You just can't use it _in combination with TiVo's own software and service_. I didn't think the GPLv2 covered anything related to use and you have retained all the freedoms the GPL promises. > That he is not opposed to the idea of TiVo using a GPLv3 kernel is > easy to see, if you take the time to read the draft instead of > spreading false assumptions about it: > > this requirement does not apply if neither you nor any third party > retains the ability to install modified object code on the User > Product So they keep the system locked down, but include perl/python/emacs and distribute updates in the form of scripts/source code which are either interpreted or compiled to a ramfs filesystem at boot. Time to add another exception? Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
On Sun, Jun 17, 2007 at 05:17:57AM -0300, Alexandre Oliva wrote: Just make the tivoization machinery require two keys: one that the vendor keeps, one that the vendor gives to the user (maybe without ever knowing it). Neither one can install modifications alone, but the user can approve modifications by the vendor, and the vendor can approve modifications by the user. This is still not ideal, but it at least doesn't permit the vendor to remove features from under the user. So what features has Tivo removed (or threatened to remove) from the GPL licensed parts? I think at some point they disabled the _undocumented_ skip feature in their own proprietary software, and ended up reenabling it when a lot of customers complained. Even if those customers had the ability to replace any GPL licensed parts, it would not have reenabled the feature. And it was an undocumented easter egg type thing at that, it isn't like they widely announced it in their advertising or as a selling point. So Google is using Linux right. What if they remove some feature? (let's pick a randomg one, i.e. phone number lookup) Should I get a keycard for their machine room to fix the problem, or maybe we should use some secret sharing mechanism to prevent them from removing the feature. RMS does not want TiVo (or anyone else) to disrespect users' freedoms, and installing technical measures to prevent users from adapting the software to suit their needs and running their modifications is disrespecting users freedoms. So if Tivo would allow you to boot your own kernel, but keeps the harddisk encrypted if the booted kernel does not have the right signature? In such a case you can run your own kernel and if you replace the harddrive you can install all the applications you might want. You cannot however use their software, any of the recorded content or obtain any further guide data/service updates. And how is that any different from taking an off-the-shelf PC and booting your own kernel with Tivo's modifications? Or really different from the current situation. Tivo complied in as far as they made GPL licensed code available, you can examine it, modify it, compile it. You just can't use it _in combination with TiVo's own software and service_. I didn't think the GPLv2 covered anything related to use and you have retained all the freedoms the GPL promises. That he is not opposed to the idea of TiVo using a GPLv3 kernel is easy to see, if you take the time to read the draft instead of spreading false assumptions about it: this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product So they keep the system locked down, but include perl/python/emacs and distribute updates in the form of scripts/source code which are either interpreted or compiled to a ramfs filesystem at boot. Time to add another exception? Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Versioning file system
On Sat, Jun 16, 2007 at 02:03:49PM -0600, Jeffrey V. Merkey wrote: > Jan Harkes wrote: > >implementation, just a high level description. Finally advising anyone > >(who is not an actual patent lawyer that could correctly interpret the > >language and scope of a patent) to go search out patents seems pretty > >bad advice. That can only result in not even attempting to research some > >potentially new and innovative approach. > > > >Researching prior published work in the area is considerably more > >helpful. Especially when something is complex beyond belief it has > >probably attracted various researchers over time and there are most > >likely various different solutions that have been explored previously. > >Such existing work can form a good basis for further work. > > When you get into the recycling issues with storage, the patents come > into play. Also, using the file name to reference revisions is already > the subject of a patent previously filed (I no longer own the patent, I > sold them to Canopy). There is a third one about to be issued. Congratulations on obtaining those patents, I hope they will be used wisely. I am however not a patent lawyer and as such in no position to evaluate their claims. As a more useful response, the original poster may want to look at some of the prior work in this area, I just picked a couple, (Cedar File System from Xerox PARC) A Caching File System for a Programmer's Workstation (1985) Michael D. Schroeder, David K. Gifford, Roger M. Needham (Vax/VMS System Software Handbook) (TOPS-20 User's Manual) (Plan 9 (file system)) Plan 9 from Bell Labs (1990) Rob Pike, Dave Presotto, Sean Dorward, Bob Flandrena, Ken Thompson, Howard Trickey, Phil Winterbottom (Elephant File System) Elephant: The File System that Never Forgets (1999) Douglas J. Santry, Michael J. Feeley, Norman C. Hutchinson Workshop on Hot Topics in Operating Systems Deciding when to forget in the Elephant file system (1999) Douglas S. Santry, Michael J. Feeley, Norman C. Hutchinson, Alistair C. Veitch, Ross W. Carton, Jacob Otir (Ext3Cow) Ext3cow: The Design, Implementation, and Analysis of Metadata for a Time-Shifting File System (2003) Zachary N. J. Peterson, Randal C. Burns Sites like portal.acm.org and citeseer.ist.psu.edu are good places to find copies of these papers. They also provide links to other work that either is cited by, or cites these papers which is a convenient way to find other papers in this area. Researching, designing and implementing such systems is a lot of fun, admittedly often more fun than long term debugging and/or maintaining, but that is life. Don't get too upset if the end result cannot be included in the main kernel. Just start over from scratch, you may just end up with an even better design the second time around. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Versioning file system
On Sat, Jun 16, 2007 at 04:12:14AM -0600, Jeffrey V. Merkey wrote: > Jeffrey V. Merkey wrote: > >over). There's also another patent filed as well. It's a noble > >effort to do a free version, but be aware there's some big guns with > >patents out there already, not to mention doing this is complex beyond > >belief. > > I reviewed your sample implementation, and it appears to infringe 3 > patents already.You should do some research on this. First of all, you are responding to someone in the UK, I thought they didn't even have software patents over there. Second, I didn't see any implementation, just a high level description. Finally advising anyone (who is not an actual patent lawyer that could correctly interpret the language and scope of a patent) to go search out patents seems pretty bad advice. That can only result in not even attempting to research some potentially new and innovative approach. Researching prior published work in the area is considerably more helpful. Especially when something is complex beyond belief it has probably attracted various researchers over time and there are most likely various different solutions that have been explored previously. Such existing work can form a good basis for further work. Finally, even if there are patents they could be too limited in scope, overly broad, can be invalidated due to prior art. It may also be possible that a patent holder has no problem granting a royalty free license for a GPL licensed implementation. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Versioning file system
On Sat, Jun 16, 2007 at 04:12:14AM -0600, Jeffrey V. Merkey wrote: Jeffrey V. Merkey wrote: over). There's also another patent filed as well. It's a noble effort to do a free version, but be aware there's some big guns with patents out there already, not to mention doing this is complex beyond belief. I reviewed your sample implementation, and it appears to infringe 3 patents already.You should do some research on this. First of all, you are responding to someone in the UK, I thought they didn't even have software patents over there. Second, I didn't see any implementation, just a high level description. Finally advising anyone (who is not an actual patent lawyer that could correctly interpret the language and scope of a patent) to go search out patents seems pretty bad advice. That can only result in not even attempting to research some potentially new and innovative approach. Researching prior published work in the area is considerably more helpful. Especially when something is complex beyond belief it has probably attracted various researchers over time and there are most likely various different solutions that have been explored previously. Such existing work can form a good basis for further work. Finally, even if there are patents they could be too limited in scope, overly broad, can be invalidated due to prior art. It may also be possible that a patent holder has no problem granting a royalty free license for a GPL licensed implementation. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Versioning file system
On Sat, Jun 16, 2007 at 02:03:49PM -0600, Jeffrey V. Merkey wrote: Jan Harkes wrote: implementation, just a high level description. Finally advising anyone (who is not an actual patent lawyer that could correctly interpret the language and scope of a patent) to go search out patents seems pretty bad advice. That can only result in not even attempting to research some potentially new and innovative approach. Researching prior published work in the area is considerably more helpful. Especially when something is complex beyond belief it has probably attracted various researchers over time and there are most likely various different solutions that have been explored previously. Such existing work can form a good basis for further work. When you get into the recycling issues with storage, the patents come into play. Also, using the file name to reference revisions is already the subject of a patent previously filed (I no longer own the patent, I sold them to Canopy). There is a third one about to be issued. Congratulations on obtaining those patents, I hope they will be used wisely. I am however not a patent lawyer and as such in no position to evaluate their claims. As a more useful response, the original poster may want to look at some of the prior work in this area, I just picked a couple, (Cedar File System from Xerox PARC) A Caching File System for a Programmer's Workstation (1985) Michael D. Schroeder, David K. Gifford, Roger M. Needham (Vax/VMS System Software Handbook) (TOPS-20 User's Manual) (Plan 9 (file system)) Plan 9 from Bell Labs (1990) Rob Pike, Dave Presotto, Sean Dorward, Bob Flandrena, Ken Thompson, Howard Trickey, Phil Winterbottom (Elephant File System) Elephant: The File System that Never Forgets (1999) Douglas J. Santry, Michael J. Feeley, Norman C. Hutchinson Workshop on Hot Topics in Operating Systems Deciding when to forget in the Elephant file system (1999) Douglas S. Santry, Michael J. Feeley, Norman C. Hutchinson, Alistair C. Veitch, Ross W. Carton, Jacob Otir (Ext3Cow) Ext3cow: The Design, Implementation, and Analysis of Metadata for a Time-Shifting File System (2003) Zachary N. J. Peterson, Randal C. Burns Sites like portal.acm.org and citeseer.ist.psu.edu are good places to find copies of these papers. They also provide links to other work that either is cited by, or cites these papers which is a convenient way to find other papers in this area. Researching, designing and implementing such systems is a lot of fun, admittedly often more fun than long term debugging and/or maintaining, but that is life. Don't get too upset if the end result cannot be included in the main kernel. Just start over from scratch, you may just end up with an even better design the second time around. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
On Fri, Jun 15, 2007 at 12:02:11PM +0200, Bernd Paysan wrote: > > the GPLv3 license. That's not their fault, it's the fault of people > > who wrote the GPLv3 license, promulgated the GPLv3 license, and who is > > attempting to convince everyone that the GPLv3 license is the only > > valid license for Right Thinking FSF automatons to use. > > Ah no, it's their fault. The GPLv2 always was clear that there will be some > future releases of the GPL, and that you should keep "upgrading" possible. ... > enough. I try to give up. I suggest everyone who has some assertions about > what the GPLv2 does read it through and find the place where it says so. Ok, open your local copy of the GPL (you should have a copy in /usr/share/common-licences/GPL if you use Debian). Now read along with me... |GNU GENERAL PUBLIC LICENSE | Version 2, June 1991 Ok, we have that settled, this license describes GPL version 2, specifically, no if's or later's. Bunch of legal mumbo jumbo follows until we reach... | END OF TERMS AND CONDITIONS And we've reached the end of the license. Now it it followed up by some helpful text for the reader... |How to Apply These Terms to Your New Programs ... | To do so, attach the following notices to the program. It is safest | to attach them to the start of each source file to most effectively | convey the exclusion of warranty; and each file should have at least | the "copyright" line and a pointer to where the full notice is found. | | |Copyright (C) | |This program is free software; you can redistribute it and/or modify |it under the terms of the GNU General Public License as published by |the Free Software Foundation; either version 2 of the License, or |(at your option) any later version. Look, some example boiler plate that is suggested to be placed in the source code _to most effectively convey_ that the code is GPL licensed. So clearly there must be less effective ways, one of which may just be to include a COPYING or LICENSE file along with the distributed source. And the author of this example boiler place is clearly 'talking' to the the copyright owner/licensor (i.e. not the licencee), giving suggestions about adding contact information and even possible commands that could be added to interactive programs. So that '(at your option)' may just be a suggestion to the licensor, or maybe not. But that isn't really here not there. Files that don't have such boiler plate do not convey the terms of the license as effectively, but they are still covered by the license that was included with the kernel, which is clearly version 2. Now ofcourse we were looking at the official GPLv2, so I ran a diff against the COPYING file in the kernel tree. The only differences I can see are the included clarification by Linus at the top, some formatting differences and the fact that the FSF version seems to refer to the 'GNU Lesser General Public License', while Linus's version has, 'GNU Library General Public License'. > Unfortunately, I haven't seen GPL citations from the Linus-fanboy curve, > only suggestions that the GPL "does not say something" which it clearly > does. I hope this helps. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
On Fri, Jun 15, 2007 at 12:02:11PM +0200, Bernd Paysan wrote: the GPLv3 license. That's not their fault, it's the fault of people who wrote the GPLv3 license, promulgated the GPLv3 license, and who is attempting to convince everyone that the GPLv3 license is the only valid license for Right Thinking FSF automatons to use. Ah no, it's their fault. The GPLv2 always was clear that there will be some future releases of the GPL, and that you should keep upgrading possible. ... enough. I try to give up. I suggest everyone who has some assertions about what the GPLv2 does read it through and find the place where it says so. Ok, open your local copy of the GPL (you should have a copy in /usr/share/common-licences/GPL if you use Debian). Now read along with me... |GNU GENERAL PUBLIC LICENSE | Version 2, June 1991 Ok, we have that settled, this license describes GPL version 2, specifically, no if's or later's. Bunch of legal mumbo jumbo follows until we reach... | END OF TERMS AND CONDITIONS And we've reached the end of the license. Now it it followed up by some helpful text for the reader... |How to Apply These Terms to Your New Programs ... | To do so, attach the following notices to the program. It is safest | to attach them to the start of each source file to most effectively | convey the exclusion of warranty; and each file should have at least | the copyright line and a pointer to where the full notice is found. | |one line to give the program's name and a brief idea of what it does. |Copyright (C) year name of author | |This program is free software; you can redistribute it and/or modify |it under the terms of the GNU General Public License as published by |the Free Software Foundation; either version 2 of the License, or |(at your option) any later version. Look, some example boiler plate that is suggested to be placed in the source code _to most effectively convey_ that the code is GPL licensed. So clearly there must be less effective ways, one of which may just be to include a COPYING or LICENSE file along with the distributed source. And the author of this example boiler place is clearly 'talking' to the the copyright owner/licensor (i.e. not the licencee), giving suggestions about adding contact information and even possible commands that could be added to interactive programs. So that '(at your option)' may just be a suggestion to the licensor, or maybe not. But that isn't really here not there. Files that don't have such boiler plate do not convey the terms of the license as effectively, but they are still covered by the license that was included with the kernel, which is clearly version 2. Now ofcourse we were looking at the official GPLv2, so I ran a diff against the COPYING file in the kernel tree. The only differences I can see are the included clarification by Linus at the top, some formatting differences and the fact that the FSF version seems to refer to the 'GNU Lesser General Public License', while Linus's version has, 'GNU Library General Public License'. Unfortunately, I haven't seen GPL citations from the Linus-fanboy curve, only suggestions that the GPL does not say something which it clearly does. I hope this helps. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
On Thu, Jun 14, 2007 at 12:28:34PM -0700, David Schwartz wrote: > > The GPL applies to "the Program" which in this case is the Linux kernel > > as a whole and it in fact does indicate a specific version. All code > > submitted and included in this program has has been submitted with the > > understanding that the work as a whole is specifically licensed as > > GPLv2. Some authors have granted additional rights, such as dual BSD/GPL > > or GPLv2 and later and explicitly added such a notice. > > Since the Linux kernel as a whole does not have a single author, it is > impossible to license it as a whole. Nobody has the authority to do that. > (The GPL is not a copyright assignment type license.) > > Fortunately, the GPL clears this up: > > "Each time you redistribute the Program (or any work based on the > Program), the recipient automatically receives a license from the > original licensor to copy, distribute or modify the Program subject to > these terms and conditions. You may not impose any further > restrictions on the recipients' exercise of the rights granted herein. > You are not responsible for enforcing compliance by third parties to > this License." > > Linus cannot impose any further restrictions on the recipients' exercise of > the rights granted. > > When you download a copy of the Linux kernel, you do not receive one license > because nobody could grant you one license. You receive a logically separate > license from each original licensor. You receive from Linus only a license > to his contributions. > > Note that you cannot take a GPLv2+ work and redistribute it as GPLv3 only. > You can license your contributions as GPLv3 only of course. However, each > recipient still receives a GPLv2+ license to the parts that were originally > licensed that way. The people you distribute the work from receive licenses > from the original licensors to those parts, and you have no right to modify > that license. (See GPL section 6, quoted above.) You have a good point. It can be argued that contributions before 2.4.0-test8 were in fact GPLv2+, but anything after that point has clearly been contributed as GPLv2 only. So now we have a bunch of pre-2.4.0-test8 code that may possibly be v2+ and files that explicitly state v2+ in their boiler plate. However many of these files may have had additional contributions from other authors which (unless otherwise specified) were GPLv2-only. And because v2 and v3 are incompatible, all those files with v2-only contributions will become v2-only when version 3 is released. Of course it may be that all those copyright owners do not mind re-releasing their copyrighted code as v2+, but they will have to be contacted. Several maintainers did pay attention to such details. I once submitted a patch that among others touched reiserfs, and I promptly got a friendly email from Hans asking me to sign off any rights he needed to re-release the related code under a different license, so he made sure the combined work wouldn't end up GPLv2 only. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
On Thu, Jun 14, 2007 at 12:28:34PM -0700, David Schwartz wrote: The GPL applies to the Program which in this case is the Linux kernel as a whole and it in fact does indicate a specific version. All code submitted and included in this program has has been submitted with the understanding that the work as a whole is specifically licensed as GPLv2. Some authors have granted additional rights, such as dual BSD/GPL or GPLv2 and later and explicitly added such a notice. Since the Linux kernel as a whole does not have a single author, it is impossible to license it as a whole. Nobody has the authority to do that. (The GPL is not a copyright assignment type license.) Fortunately, the GPL clears this up: Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. Linus cannot impose any further restrictions on the recipients' exercise of the rights granted. When you download a copy of the Linux kernel, you do not receive one license because nobody could grant you one license. You receive a logically separate license from each original licensor. You receive from Linus only a license to his contributions. Note that you cannot take a GPLv2+ work and redistribute it as GPLv3 only. You can license your contributions as GPLv3 only of course. However, each recipient still receives a GPLv2+ license to the parts that were originally licensed that way. The people you distribute the work from receive licenses from the original licensors to those parts, and you have no right to modify that license. (See GPL section 6, quoted above.) You have a good point. It can be argued that contributions before 2.4.0-test8 were in fact GPLv2+, but anything after that point has clearly been contributed as GPLv2 only. So now we have a bunch of pre-2.4.0-test8 code that may possibly be v2+ and files that explicitly state v2+ in their boiler plate. However many of these files may have had additional contributions from other authors which (unless otherwise specified) were GPLv2-only. And because v2 and v3 are incompatible, all those files with v2-only contributions will become v2-only when version 3 is released. Of course it may be that all those copyright owners do not mind re-releasing their copyrighted code as v2+, but they will have to be contacted. Several maintainers did pay attention to such details. I once submitted a patch that among others touched reiserfs, and I promptly got a friendly email from Hans asking me to sign off any rights he needed to re-release the related code under a different license, so he made sure the combined work wouldn't end up GPLv2 only. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
On Wed, Jun 13, 2007 at 04:24:36PM +0200, Bernd Paysan wrote: > There's no point of discussing that the Linux kernel *as a whole* (as > compilation) currently is under GPLv2 only, since it sais so, and a few > files also explicitely say so. The whole combination is GPLv2 only, but > most parts aren't. You claim that any source files without a notices are 'any version of the GPL'. But I read the license and you are totally wrong about that. The GPL applies to "the Program" which in this case is the Linux kernel as a whole and it in fact does indicate a specific version. All code submitted and included in this program has has been submitted with the understanding that the work as a whole is specifically licensed as GPLv2. Some authors have granted additional rights, such as dual BSD/GPL or GPLv2 and later and explicitly added such a notice. All other code is simply copyrighted, and the only available license is the GPLv2. Take for example fs/inode.c. Notice how it doesn't have GPL boilerplate, but it is clearly indicating that it is copyrighted. So taking that file by itself out of the context of the kernel and then distributing it would clearly be a copyright violation. The only one reason you can distribute that code is because of the GPLv2 that covers the Linux kernel (i.e. "the Program"). > > > So my conclusion is: If you, as contributor to the Linux kernel, want > > > to make clear that your work really is GPLv2 only, you have to do that > > > yourself, you have to add a notice like above to files where you > > > exclusively own copyright. The kernel is explicitly licensed as GPLv2, any contributions (source files/parts of the work) that wish to grant additional rights have to specify so explicitly, and not the other way around however much you'd like that. > patches as the mainline kernel), I'm free to choose under which conditions > I redistribute it, given that it's compatible with the conditions the > original authors have chosen. Most of them have said nothing (other than > implicitely that it's ok for them to put it under GPL, as they haven't > opposed to inclusion into the Linux kernel), some have said GPLv2 or later, Reread section 9 and consider that "the Program" is the Linux kernel, which does explicitly state a version and does not include the "and any later" option. Any source that does not explicitly specify additional rights is GPLv2. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
On Wed, Jun 13, 2007 at 04:24:36PM +0200, Bernd Paysan wrote: There's no point of discussing that the Linux kernel *as a whole* (as compilation) currently is under GPLv2 only, since it sais so, and a few files also explicitely say so. The whole combination is GPLv2 only, but most parts aren't. You claim that any source files without a notices are 'any version of the GPL'. But I read the license and you are totally wrong about that. The GPL applies to the Program which in this case is the Linux kernel as a whole and it in fact does indicate a specific version. All code submitted and included in this program has has been submitted with the understanding that the work as a whole is specifically licensed as GPLv2. Some authors have granted additional rights, such as dual BSD/GPL or GPLv2 and later and explicitly added such a notice. All other code is simply copyrighted, and the only available license is the GPLv2. Take for example fs/inode.c. Notice how it doesn't have GPL boilerplate, but it is clearly indicating that it is copyrighted. So taking that file by itself out of the context of the kernel and then distributing it would clearly be a copyright violation. The only one reason you can distribute that code is because of the GPLv2 that covers the Linux kernel (i.e. the Program). So my conclusion is: If you, as contributor to the Linux kernel, want to make clear that your work really is GPLv2 only, you have to do that yourself, you have to add a notice like above to files where you exclusively own copyright. The kernel is explicitly licensed as GPLv2, any contributions (source files/parts of the work) that wish to grant additional rights have to specify so explicitly, and not the other way around however much you'd like that. patches as the mainline kernel), I'm free to choose under which conditions I redistribute it, given that it's compatible with the conditions the original authors have chosen. Most of them have said nothing (other than implicitely that it's ok for them to put it under GPL, as they haven't opposed to inclusion into the Linux kernel), some have said GPLv2 or later, Reread section 9 and consider that the Program is the Linux kernel, which does explicitly state a version and does not include the and any later option. Any source that does not explicitly specify additional rights is GPLv2. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Reiser4. BEST FILESYSTEM EVER.
Since you decide to publically respond to a private email, but not only you did not 'discuss' anything I wrote and in fact cut out most of the useful information in my reply I guess I will have to repeat my observations. Once I send out this email, I'll just add you to my friendly killfile (as well as this thread's subject/msgids) so don't bother continuing you one-sided 'discussions' on this topic. On Fri, Apr 06, 2007 at 07:47:36PM -0700, [EMAIL PROTECTED] wrote: > > Do you really have to repeat the results in every email you sent? > > Damn, I did it again. WHY DO YOU CARE? Because I saw them the first time around. And although the performance difference on those micro benchmarks seems quite impressive, I'm not convinced. > Look, its simple, I am (among other things) discussing these results, so > people need to see them. However, you do not discuss, you just repeat, and repeat, and repeat. But for what reason. Do you want an actual discussion, or do you hate the reiserfs developers so much that you want to antagonize any and all other Linux file systems developers? > > > Don't you agree, that "If they are accurate, THEN they are obviously > > > very relevant." > > > > Not everyone does. I care mostly about reliability and availability > > neither of which are shown by your results. > > Actually, to some extent, bonnie++ tests the reliability of the > filesystem, eg, NTFS-3g usually fails. > > By the way, I have pulled the plug on my REISER4 system, a number of > times now, and it recovers without problem. Very nice for you. I guess you have also not been hit by out of memory conditions or failing partial writes. So I guess we can ignore the patch that was just sent a day or two ago to the mailing list because you succesfully pulled the plug, a number of times at that. > > > I have set up a Reiser4 partition with gzip compression, here is the > > > difference in disk usage of a typical Debian installation on two 10GB > > > partitions, one with Reiser3 and the other with Reiser4. > > > > > > debian:/# df > > > Filesystem 1K-blocks Used Available Use% Mounted on > > > /dev/sda3 10490104 6379164 4110940 61% /3 > > > /dev/sda7 9967960 2632488 7335472 27% /7 > > ... > > > Partition 3 is Reiser3 -- uses 6.4 GB. > > > Partition 7 is Reiser4 -- uses 2.6 GB. > > > > > > So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3 > > > 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same > > > info). > > > > Wow, consider me totally and completely, unimpressed. > > Here is the part of my email that you seemed to totally ignore, You've just saved yourself $3.80, now go get yourself a latte. (see. http://tomayko.com/weblog/2006/09/11/that-dilbert-cartoon) Seriously, disk storage is getting less expensive all the time, you can already buy a 250GB SATA drive for $70. Also, compression doesn't help once you store already compressed data such as jpeg images, mp3 files, or mpeg2/4/divx video files. Not only are the savings non-existant, but we still end up with the disadvantage that corruption propagates to multiple files because of the compression in the file system. And if it doesn't propagate across multiple files, the compression can't be all that good since it can't benefit as much from similarity between files. So if that is the case and you really want to save diskspace you almost have to look at read-only compressed filesystems such as cramfs, squashfs, zisofs, cloop and various other variants in combination with a unionfs overlay to get read/write functionality. But in the end everything is a tradeoff. You can save diskspace, but increase the cost of corruption. Use a better compression algorithm, but that uses more CPU or which is visible in performance of the application. This can be offset by caching more, and being lazier with writebacks, but that hurts on-disk consistency, creates more memory pressure (more swapout/paging) and generally slows down other applications that aren't actually accessing the disk. Having a fast multi-core cpu and lots of memory helps a lot, but at some point what tradeoff did we just make, we saved a couple of dollars not having to buy a larger disk, but we spend considerably more on the more expensive cpu and memory. > Wow, consider me totally impressed by your AMAZING BIAS. > > Would you like to tell me why you are SO BIASED against REISER4. And that is the reponse I get, I thought you wanted discussion, but clearly you don't care about any meaningful discussion. Your goal seems to be to make sure that other developers end up ignoring any thread that has reiser in the subject. And even if they are not biased and welcome a discussion, you will just call them out on it, because clearly if they want to discuss something they aren't totally with it, so they have to be
Re: Reiser4. BEST FILESYSTEM EVER.
Since you decide to publically respond to a private email, but not only you did not 'discuss' anything I wrote and in fact cut out most of the useful information in my reply I guess I will have to repeat my observations. Once I send out this email, I'll just add you to my friendly killfile (as well as this thread's subject/msgids) so don't bother continuing you one-sided 'discussions' on this topic. On Fri, Apr 06, 2007 at 07:47:36PM -0700, [EMAIL PROTECTED] wrote: Do you really have to repeat the results in every email you sent? Damn, I did it again. WHY DO YOU CARE? Because I saw them the first time around. And although the performance difference on those micro benchmarks seems quite impressive, I'm not convinced. Look, its simple, I am (among other things) discussing these results, so people need to see them. However, you do not discuss, you just repeat, and repeat, and repeat. But for what reason. Do you want an actual discussion, or do you hate the reiserfs developers so much that you want to antagonize any and all other Linux file systems developers? Don't you agree, that If they are accurate, THEN they are obviously very relevant. Not everyone does. I care mostly about reliability and availability neither of which are shown by your results. Actually, to some extent, bonnie++ tests the reliability of the filesystem, eg, NTFS-3g usually fails. By the way, I have pulled the plug on my REISER4 system, a number of times now, and it recovers without problem. Very nice for you. I guess you have also not been hit by out of memory conditions or failing partial writes. So I guess we can ignore the patch that was just sent a day or two ago to the mailing list because you succesfully pulled the plug, a number of times at that. I have set up a Reiser4 partition with gzip compression, here is the difference in disk usage of a typical Debian installation on two 10GB partitions, one with Reiser3 and the other with Reiser4. debian:/# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda3 10490104 6379164 4110940 61% /3 /dev/sda7 9967960 2632488 7335472 27% /7 ... Partition 3 is Reiser3 -- uses 6.4 GB. Partition 7 is Reiser4 -- uses 2.6 GB. So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same info). Wow, consider me totally and completely, unimpressed. Here is the part of my email that you seemed to totally ignore, You've just saved yourself $3.80, now go get yourself a latte. (see. http://tomayko.com/weblog/2006/09/11/that-dilbert-cartoon) Seriously, disk storage is getting less expensive all the time, you can already buy a 250GB SATA drive for $70. Also, compression doesn't help once you store already compressed data such as jpeg images, mp3 files, or mpeg2/4/divx video files. Not only are the savings non-existant, but we still end up with the disadvantage that corruption propagates to multiple files because of the compression in the file system. And if it doesn't propagate across multiple files, the compression can't be all that good since it can't benefit as much from similarity between files. So if that is the case and you really want to save diskspace you almost have to look at read-only compressed filesystems such as cramfs, squashfs, zisofs, cloop and various other variants in combination with a unionfs overlay to get read/write functionality. But in the end everything is a tradeoff. You can save diskspace, but increase the cost of corruption. Use a better compression algorithm, but that uses more CPU or which is visible in performance of the application. This can be offset by caching more, and being lazier with writebacks, but that hurts on-disk consistency, creates more memory pressure (more swapout/paging) and generally slows down other applications that aren't actually accessing the disk. Having a fast multi-core cpu and lots of memory helps a lot, but at some point what tradeoff did we just make, we saved a couple of dollars not having to buy a larger disk, but we spend considerably more on the more expensive cpu and memory. Wow, consider me totally impressed by your AMAZING BIAS. Would you like to tell me why you are SO BIASED against REISER4. And that is the reponse I get, I thought you wanted discussion, but clearly you don't care about any meaningful discussion. Your goal seems to be to make sure that other developers end up ignoring any thread that has reiser in the subject. And even if they are not biased and welcome a discussion, you will just call them out on it, because clearly if they want to discuss something they aren't totally with it, so they have to be totally BIASED against REISER4. At least it looks like we agree on something, I
Re: Finding hardlinks
On Mon, Jan 01, 2007 at 11:47:06PM +0100, Mikulas Patocka wrote: > >Anyway, cp -a is not the only application that wants to do hardlink > >detection. > > I tested programs for ino_t collision (I intentionally injected it) and > found that CP from coreutils 6.7 fails to copy directories but displays > error messages (coreutils 5 work fine). MC and ARJ skip directories with > colliding ino_t and pretend that operation completed successfuly. FTS > library fails to walk directories returning FTS_DC error. Diffutils, find, > grep fail to search directories with coliding inode numbers. Tar seems > tolerant except incremental backup (which I didn't try). All programs > except diff were tolerant to coliding ino_t on files. Thanks for testing so many programs, but... did the files/symlinks with colliding inode number have i_nlink > 1? Or did you also have directories with colliding inode numbers. It looks like you've introduced hardlinked directories in your test which are definitely not supported, in fact it will probably cause not only issues for userspace programs, but also locking and garbage collection issues in the kernel's dcache. I'm surprised you're seeing so many problems. The only find problem that I am aware of is the one where it assumes that there will be only i_nlink-2 subdirectories in a given directory, this optimization can be disabled with -noleaf. The only problems I've encountered with ino_t collisions are archivers and other programs that recursively try to copy a tree while preserving hardlinks. And in all cases these seem to have no problem with such collisions as long as i_nlink == 1. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Finding hardlinks
On Mon, Jan 01, 2007 at 11:47:06PM +0100, Mikulas Patocka wrote: Anyway, cp -a is not the only application that wants to do hardlink detection. I tested programs for ino_t collision (I intentionally injected it) and found that CP from coreutils 6.7 fails to copy directories but displays error messages (coreutils 5 work fine). MC and ARJ skip directories with colliding ino_t and pretend that operation completed successfuly. FTS library fails to walk directories returning FTS_DC error. Diffutils, find, grep fail to search directories with coliding inode numbers. Tar seems tolerant except incremental backup (which I didn't try). All programs except diff were tolerant to coliding ino_t on files. Thanks for testing so many programs, but... did the files/symlinks with colliding inode number have i_nlink 1? Or did you also have directories with colliding inode numbers. It looks like you've introduced hardlinked directories in your test which are definitely not supported, in fact it will probably cause not only issues for userspace programs, but also locking and garbage collection issues in the kernel's dcache. I'm surprised you're seeing so many problems. The only find problem that I am aware of is the one where it assumes that there will be only i_nlink-2 subdirectories in a given directory, this optimization can be disabled with -noleaf. The only problems I've encountered with ino_t collisions are archivers and other programs that recursively try to copy a tree while preserving hardlinks. And in all cases these seem to have no problem with such collisions as long as i_nlink == 1. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Finding hardlinks
On Fri, Dec 22, 2006 at 12:49:42AM +0100, Mikulas Patocka wrote: > On Thu, 21 Dec 2006, Jan Harkes wrote: > >On Wed, Dec 20, 2006 at 12:44:42PM +0100, Miklos Szeredi wrote: > >>The stat64.st_ino field is 64bit, so AFAICS you'd only need to extend > >>the kstat.ino field to 64bit and fix those filesystems to fill in > >>kstat correctly. > > > >Coda actually uses 128-bit file identifiers internally, so 64-bits > >really doesn't cut it. Since the 128-bit space is used pretty sparsely > > The problem is that if inode number collision happens occasionally, you > get data corruption with cp -a command --- it will just copy one file and > hardlink the other. Our 128-bit space is fairly sparse and there is some regularity so we optimized the hash to minimize the chance on collisions. This is also useful for iget5_locked, each 32-bit ino_t is effectively a hash bucket in our case and avoiding collisions makes the lookup in the inode cache more efficient. Another part is that only few applications actually care about hardlinks (cp -a, rsync, tar/afio). All of these already could miss some files or create false hardlinks when files in the tree are renamed during the copy. We also have a special atomic volume snapshot function that is used to create a backup, which backs up additional attributes that are not visible through the standard POSIX/vfs api (directory acls, creator/owner information, version-vector information for conflict detection and resolution) I've also found that most applications that care about hardlinks already have a check whether the link count is greater than one and the object is not a directory. This is probably done more for efficiency, it would be a waste of memory to track every object as a possible hardlink. And because Coda already restrict hardlinks in many cases they end up not being used very much, or get converted by a cross-directory rename to COW objects which of course have nlink == 1. > If user (or script) doesn't specify that flag, it doesn't help. I think > the best solution for these filesystems would be either to add new syscall > int is_hardlink(char *filename1, char *filename2) > (but I know adding syscall bloat may be objectionable) > or add new field in statvfs ST_HAS_BROKEN_INO_T, that applications can > test and disable hardlink processing. BROKEN_INO_T sounds a bit harsh, and something like that would really have to be incorporated in the SuS standard for it to be useful. It also would require all applications to be updated to check for this flag. On the other hand if we don't worry about this flag we just have to fix the few applications that do not yet check that nlink>1 && !IS_DIR. Those applications would probably appreciate the resulting reduced memory requirements and performance increase because they end up with considerably fewer candidates in their internal list of potential hardlinked objects. Of course this doesn't solve the problem for some filesystem with larger than 64-bit object identifiers that wants to support normal hardlinked files. But adding a BROKEN_INO_T flag doesn't solve it either, since the backup/copy would not perform hardlink processing in which case such a file system can just as well always pretend that i_nlink for files is always one. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Finding hardlinks
On Wed, Dec 20, 2006 at 12:44:42PM +0100, Miklos Szeredi wrote: > The stat64.st_ino field is 64bit, so AFAICS you'd only need to extend > the kstat.ino field to 64bit and fix those filesystems to fill in > kstat correctly. Coda actually uses 128-bit file identifiers internally, so 64-bits really doesn't cut it. Since the 128-bit space is used pretty sparsely there is a hash which avoids most collistions in 32-bit i_ino space, but not completely. I can also imagine that at some point someone wants to implement a git-based filesystem where it would be more natural to use 160-bit SHA1 hashes as unique object identifiers. But Coda only allow hardlinks within a single directory and if someone renames a hardlinked file and one of the names ends up in a different directory we implicitly create a copy of the object. This actually leverages off of the way we handle volume snapshots and the fact that we use whole file caching and writes, so we only copy the metadata while the data is 'copy-on-write'. I'm considering changing the way we handle hardlinks by having link(2) always create a new object with copy-on-write semantics (i.e. replacing link with some sort of a copyfile operation). This way we can get rid of several special cases like the cross-directory rename. It also avoids problems when the various replicas of an object are found to be inconsistent and we allow the user to expand the file. On expansion a file becomes a directory that contains all the objects on individual replicas. Handling the expansion in a dcache friendly way is nasty enough as is and complicated by the fact that we really don't want such an expansion to result in hard-linked directories, so we are forced to inventing new unique object identifiers, etc. Again, not having hardlinks would simplify things somewhat here. Any application that tries to be smart enough to keep track of which files are hardlinked should (in my opinion) also have a way to disable this behaviour. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Finding hardlinks
On Wed, Dec 20, 2006 at 12:44:42PM +0100, Miklos Szeredi wrote: The stat64.st_ino field is 64bit, so AFAICS you'd only need to extend the kstat.ino field to 64bit and fix those filesystems to fill in kstat correctly. Coda actually uses 128-bit file identifiers internally, so 64-bits really doesn't cut it. Since the 128-bit space is used pretty sparsely there is a hash which avoids most collistions in 32-bit i_ino space, but not completely. I can also imagine that at some point someone wants to implement a git-based filesystem where it would be more natural to use 160-bit SHA1 hashes as unique object identifiers. But Coda only allow hardlinks within a single directory and if someone renames a hardlinked file and one of the names ends up in a different directory we implicitly create a copy of the object. This actually leverages off of the way we handle volume snapshots and the fact that we use whole file caching and writes, so we only copy the metadata while the data is 'copy-on-write'. I'm considering changing the way we handle hardlinks by having link(2) always create a new object with copy-on-write semantics (i.e. replacing link with some sort of a copyfile operation). This way we can get rid of several special cases like the cross-directory rename. It also avoids problems when the various replicas of an object are found to be inconsistent and we allow the user to expand the file. On expansion a file becomes a directory that contains all the objects on individual replicas. Handling the expansion in a dcache friendly way is nasty enough as is and complicated by the fact that we really don't want such an expansion to result in hard-linked directories, so we are forced to inventing new unique object identifiers, etc. Again, not having hardlinks would simplify things somewhat here. Any application that tries to be smart enough to keep track of which files are hardlinked should (in my opinion) also have a way to disable this behaviour. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Finding hardlinks
On Fri, Dec 22, 2006 at 12:49:42AM +0100, Mikulas Patocka wrote: On Thu, 21 Dec 2006, Jan Harkes wrote: On Wed, Dec 20, 2006 at 12:44:42PM +0100, Miklos Szeredi wrote: The stat64.st_ino field is 64bit, so AFAICS you'd only need to extend the kstat.ino field to 64bit and fix those filesystems to fill in kstat correctly. Coda actually uses 128-bit file identifiers internally, so 64-bits really doesn't cut it. Since the 128-bit space is used pretty sparsely The problem is that if inode number collision happens occasionally, you get data corruption with cp -a command --- it will just copy one file and hardlink the other. Our 128-bit space is fairly sparse and there is some regularity so we optimized the hash to minimize the chance on collisions. This is also useful for iget5_locked, each 32-bit ino_t is effectively a hash bucket in our case and avoiding collisions makes the lookup in the inode cache more efficient. Another part is that only few applications actually care about hardlinks (cp -a, rsync, tar/afio). All of these already could miss some files or create false hardlinks when files in the tree are renamed during the copy. We also have a special atomic volume snapshot function that is used to create a backup, which backs up additional attributes that are not visible through the standard POSIX/vfs api (directory acls, creator/owner information, version-vector information for conflict detection and resolution) I've also found that most applications that care about hardlinks already have a check whether the link count is greater than one and the object is not a directory. This is probably done more for efficiency, it would be a waste of memory to track every object as a possible hardlink. And because Coda already restrict hardlinks in many cases they end up not being used very much, or get converted by a cross-directory rename to COW objects which of course have nlink == 1. If user (or script) doesn't specify that flag, it doesn't help. I think the best solution for these filesystems would be either to add new syscall int is_hardlink(char *filename1, char *filename2) (but I know adding syscall bloat may be objectionable) or add new field in statvfs ST_HAS_BROKEN_INO_T, that applications can test and disable hardlink processing. BROKEN_INO_T sounds a bit harsh, and something like that would really have to be incorporated in the SuS standard for it to be useful. It also would require all applications to be updated to check for this flag. On the other hand if we don't worry about this flag we just have to fix the few applications that do not yet check that nlink1 !IS_DIR. Those applications would probably appreciate the resulting reduced memory requirements and performance increase because they end up with considerably fewer candidates in their internal list of potential hardlinked objects. Of course this doesn't solve the problem for some filesystem with larger than 64-bit object identifiers that wants to support normal hardlinked files. But adding a BROKEN_INO_T flag doesn't solve it either, since the backup/copy would not perform hardlink processing in which case such a file system can just as well always pretend that i_nlink for files is always one. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiser4 plugins
On Wed, Jul 06, 2005 at 09:33:13PM -0500, David Masover wrote: > And speaking of which, the only doomsday scenario (running out of RAM) > that I can think of with this scheme is if we have a ton of hardlinks to > the same file and we try to move one of them. But this scales linearly > with the number of hardlinks, I think. Maybe not quite, but certainly > not exponentially. > > The only other doomsday scenario is if we have a ludicrously deep tree. rename(a/b, c/b), if a and c are identical. End result, either you deadlock trying to lock the same object twice, or you end up removing b since the target of a rename is unlinked. The VFS uses dentries, there is one per hardlinked object, and they have a single parent only. So a/b and c/b are represented by the same inode, but they have completely different dentry-aliases associated with them. Similar things for removal, we can safely remove a file, but not a directory that still has children, if you have 'meta-files' hanging of a file, you'd have to get rid of the metadata objects first. If you want to use a file as a directory, it probably will need the same restrictions as a directory if you expect the dcache and VFS locking to work correctly. So that means, _no hardlinks to files_ (the file system could internally implement copy-on-write type links, or use a content addressable storage to deal with diskspace issues) and the file system probably has to d_unhash/destroy metadata objects before it can unlink the file object, etc. > To make this work in real usage, not DOS testing, we really need both of > those, and even then I'm not sure it can work. What's the maximum > number of hardlinks supported to a single file? I believe nlink is a 16-bit value, so that would be either 32K or 64K depending on signedness. > What's the maximum tree depth? Last time I checked, PATH_MAX is 4096, so that would be about 2048 single character directory names. > Can these be limited to prevent actual DOS attempts? Is that the 'nobody needs more than 64KB of memory' kind of DOS? Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiser4 plugins
On Wed, Jul 06, 2005 at 09:33:13PM -0500, David Masover wrote: And speaking of which, the only doomsday scenario (running out of RAM) that I can think of with this scheme is if we have a ton of hardlinks to the same file and we try to move one of them. But this scales linearly with the number of hardlinks, I think. Maybe not quite, but certainly not exponentially. The only other doomsday scenario is if we have a ludicrously deep tree. rename(a/b, c/b), if a and c are identical. End result, either you deadlock trying to lock the same object twice, or you end up removing b since the target of a rename is unlinked. The VFS uses dentries, there is one per hardlinked object, and they have a single parent only. So a/b and c/b are represented by the same inode, but they have completely different dentry-aliases associated with them. Similar things for removal, we can safely remove a file, but not a directory that still has children, if you have 'meta-files' hanging of a file, you'd have to get rid of the metadata objects first. If you want to use a file as a directory, it probably will need the same restrictions as a directory if you expect the dcache and VFS locking to work correctly. So that means, _no hardlinks to files_ (the file system could internally implement copy-on-write type links, or use a content addressable storage to deal with diskspace issues) and the file system probably has to d_unhash/destroy metadata objects before it can unlink the file object, etc. To make this work in real usage, not DOS testing, we really need both of those, and even then I'm not sure it can work. What's the maximum number of hardlinks supported to a single file? I believe nlink is a 16-bit value, so that would be either 32K or 64K depending on signedness. What's the maximum tree depth? Last time I checked, PATH_MAX is 4096, so that would be about 2048 single character directory names. Can these be limited to prevent actual DOS attempts? Is that the 'nobody needs more than 64KB of memory' kind of DOS? Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Linux-2.6.11 can't disable CAD
On Thu, Apr 07, 2005 at 11:16:14AM -0400, Richard B. Johnson wrote: > In the not-too distant past, one could disable Ctl-Alt-DEL. > Can't do it anymore. ... > Observe that reboot() returns 0 and `strace` understands what > parameters were passed. The result is that, if I hit Ctl-Alt-Del, > `init` will still execute the shutdown-order (INIT 0). Actually, if CAD is enabled in the kernel, it will just reboot. If CAD is disabled in the kernel a SIGINT is sent to pid 1 (/sbin/init). So what you probably had in the not-too-distant past was a disabled CAD in the kernel _and_ you had modified the following line in /etc/inittab, # What to do when CTRL-ALT-DEL is pressed. ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now AFAIK this hasn't ever really changed. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Linux-2.6.11 can't disable CAD
On Thu, Apr 07, 2005 at 11:16:14AM -0400, Richard B. Johnson wrote: In the not-too distant past, one could disable Ctl-Alt-DEL. Can't do it anymore. ... Observe that reboot() returns 0 and `strace` understands what parameters were passed. The result is that, if I hit Ctl-Alt-Del, `init` will still execute the shutdown-order (INIT 0). Actually, if CAD is enabled in the kernel, it will just reboot. If CAD is disabled in the kernel a SIGINT is sent to pid 1 (/sbin/init). So what you probably had in the not-too-distant past was a disabled CAD in the kernel _and_ you had modified the following line in /etc/inittab, # What to do when CTRL-ALT-DEL is pressed. ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now AFAIK this hasn't ever really changed. Jan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [patch 0/5] Hotplug firmware loader for Keyspan usb-serial driver
On Tue, Apr 05, 2005 at 10:45:30PM +0200, Kay Sievers wrote: > On Tue, 2005-04-05 at 15:38 -0400, Jan Harkes wrote: > > Here is another stab at making the keyspan firmware easily loadable with > > hotplug. Differences from the previous version, > > > > - keep the IHEX parser into a separate module. > > - added a fw-y and fw-m install targets to kbuild which will install a > > driver's firmware files in /lib/modules/`uname -r`/firmware. > > > > 01 - Add lib/ihex_parser.ko. > > Oh, I just see now that it's a EZ-USB device. Did you try adapting the > driver to use fxload(8)? to load the firmware. I have a USB-Modem that > works perfect with loading ez-firmware that way. Never heard of fxload before. I just tried it by moving the firmware directory out of the way so that hotplug wouldn't try to initialize the device behind my back. # fxload -v -t fx -D /proc/bus/usb/002/011 -I keyspan-usa19qi.fw microcontroller type: fx single stage: load on-chip memory open RAM hexfile image keyspan-usa19qi.fw stop CPU write on-chip, addr 0x0033 len3 (0x0003) write on-chip, addr 0x001a len4 (0x0004) write on-chip, addr 0x0003 len 23 (0x0017) write on-chip, addr 0x0023 len3 (0x0003) write on-chip, addr 0x0046 len 192 (0x00c0) write on-chip, addr 0x0043 len3 (0x0003) write on-chip, addr 0x len3 (0x0003) write on-chip, addr 0x0026 len 12 (0x000c) write on-chip, addr 0x0106 len 960 (0x03c0) write on-chip, addr 0x04c6 len 960 (0x03c0) write on-chip, addr 0x0886 len 960 (0x03c0) write on-chip, addr 0x0c46 len 905 (0x0389) ... WROTE: 4028 bytes, 12 segments, avg 335 reset CPU # dmesg | tail keyspan 2-1:1.0: device disconnected usb 2-1: new full speed USB device using uhci_hcd and address 11 keyspan 2-1:1.0: Keyspan - (without firmware) converter detected usb 2-1: Required firmware image (keyspan-usa19qi.fw) unavailable. >>> fxload was used at this point usb 2-1: USB disconnect, address 11 keyspan 2-1:1.0: device disconnected usb 2-1: new full speed USB device using uhci_hcd and address 12 usb 2-1: configuration #1 chosen from 2 choices keyspan 2-1:1.0: Keyspan 1 port adapter converter detected usb 2-1: Keyspan 1 port adapter converter now attached to ttyUSB0 Looks like it works, so I guess the pre-numeration stuff in the driver is really not necessary. We just need a bunch of hotplug rules to load the correct firmware whenever an uninitialized device is plugged in and some way to distribute the firmware images. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[patch 3/5] Hotplug firmware loader for Keyspan usb-serial driver
Simple program to convert the keyspan firmware header files to IHEX
formatted files that can be loaded with hotplug.
This is really only needed once to convert the existing keyspan firmware
headers, which is what the next patch does.
Index: linux/drivers/usb/serial/dumpfw.c
===
--- /dev/null 1970-01-01 00:00:00.0 +
+++ linux/drivers/usb/serial/dumpfw.c 2005-03-10 23:16:37.240765747 -0500
@@ -0,0 +1,98 @@
+/*
+ * Convert keyspan firmware header files to Intel HEX format
+ * cc -I/usr/src/linux/drivers/usb/serial -o dumpfw dumpfw.c
+ */
+#include
+#include
+
+//#include "keyspan.h" /* ezusb_hex_record */
+
+struct ezusb_hex_record {
+uint16_t address;
+uint8_t size;
+uint8_t data[64];
+};
+
+#include "keyspan_usa28_fw.h"
+#include "keyspan_usa28x_fw.h"
+#include "keyspan_usa28xa_fw.h"
+#include "keyspan_usa28xb_fw.h"
+#include "keyspan_usa19_fw.h"
+#include "keyspan_usa19qi_fw.h"
+#include "keyspan_mpr_fw.h"
+#include "keyspan_usa19qw_fw.h"
+#include "keyspan_usa18x_fw.h"
+#include "keyspan_usa19w_fw.h"
+#include "keyspan_usa49w_fw.h"
+#include "keyspan_usa49wlc_fw.h"
+
+char *boilerplate = "\
+# This firmware for the Keyspan %s USB-to-Serial adapter is\n\
+#\n\
+#Copyright (C) 1999-2003\n\
+#Keyspan, A division of InnoSys Incorporated (\"Keyspan\")\n\
+#\n\
+# as an unpublished work. This notice does not imply unrestricted or\n\
+# public access to the source code from which this firmware image is\n\
+# derived. Except as noted below this firmware image may not be\n\
+# reproduced, used, sold or transferred to any third party without\n\
+# Keyspan's prior written consent. All Rights Reserved.\n\
+#\n\
+# Permission is hereby granted for the distribution of this firmware\n\
+# image as part of a Linux or other Open Source operating system kernel\n\
+# in text or binary form as required.\n\
+#\n\
+# This firmware may not be modified and may only be used with\n\
+# Keyspan hardware. Distribution of this firmware, in whole or in\n\
+# part, requires the inclusion of this statement.\n\
+#\n";
+
+void dumpfw(char *name, const struct ezusb_hex_record *record)
+{
+char fw_name[20];
+char NAME[10];
+uint16_t address, i;
+uint8_t crc;
+FILE *f;
+
+for (i = 0; name[i] != '\0'; i++)
+ NAME[i] = toupper(name[i]);
+NAME[i] = '\0';
+
+sprintf(fw_name, "keyspan-%s.fw", name);
+printf("writing %s\n", fw_name);
+
+f = fopen(fw_name, "w");
+fprintf(f, boilerplate, NAME);
+
+while (record->address != 0x) {
+ crc = record->size + (record->address >> 8) + record->address;
+ fprintf(f, ":%02X%04X00", record->size, record->address);
+ for (i = 0; i < record->size; i++) {
+ fprintf(f, "%02X", record->data[i]);
+ crc += record->data[i];
+ }
+ fprintf(f, "%02X\n", (uint8_t)-crc);
+ record++;
+}
+fprintf(f, ":0001FF\n");
+
+fclose(f);
+}
+
+int main(int argc, char **argv)
+{
+dumpfw("mpr", keyspan_mpr_firmware);
+dumpfw("usa18x", keyspan_usa18x_firmware);
+dumpfw("usa19",keyspan_usa19_firmware);
+dumpfw("usa19qi", keyspan_usa19qi_firmware);
+dumpfw("usa19qw", keyspan_usa19qw_firmware);
+dumpfw("usa19w", keyspan_usa19w_firmware);
+dumpfw("usa28",keyspan_usa28_firmware);
+dumpfw("usa28x", keyspan_usa28x_firmware);
+dumpfw("usa28xa", keyspan_usa28xa_firmware);
+dumpfw("usa28xb", keyspan_usa28xb_firmware);
+dumpfw("usa49w", keyspan_usa49w_firmware);
+dumpfw("usa49wlc", keyspan_usa49wlc_firmware);
+}
+
--
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[patch 5/5] Hotplug firmware loader for Keyspan usb-serial driver
Add "make install_firmware" to the kbuild environment. Signed-off-by: Jan Harkes <[EMAIL PROTECTED]> Makefile| 33 + drivers/usb/serial/Makefile |5 + scripts/Makefile.fwinst | 34 ++ 3 files changed, 72 insertions(+) Index: linux/scripts/Makefile.fwinst === --- /dev/null 1970-01-01 00:00:00.0 + +++ linux/scripts/Makefile.fwinst 2005-04-05 14:36:56.0 -0400 @@ -0,0 +1,34 @@ +# == +# Installing firmware +# == + +src := $(obj) + +.PHONY: __fwinst +__fwinst: + +-include .config +include $(if $(wildcard $(obj)/Kbuild), $(obj)/Kbuild, $(obj)/Makefile) +include scripts/Makefile.lib + +firmware := $(fw-y) $(fw-m) +firmware := $(addprefix $(src)/, $(firmware)) + +# = + +.PHONY: $(firmware) +__fwinst: $(firmware) $(subdir-ym) + @: + +quiet_cmd_firmware_install = INSTALL $@ + cmd_firmware_install = mkdir -p $(2); cp $@ $(2) + +$(firmware): + $(call cmd,firmware_install,$(MODLIB)/firmware) + +# = + +.PHONY: $(subdir-ym) +$(subdir-ym): + $(Q)$(MAKE) -f $(if $(KBUILD_SRC),$(srctree)/)scripts/Makefile.fwinst obj=$@ + Index: linux/Makefile === --- linux.orig/Makefile 2005-03-29 16:19:14.0 -0500 +++ linux/Makefile 2005-04-05 14:42:42.0 -0400 @@ -927,6 +927,21 @@ modules modules_install: FORCE endif # CONFIG_MODULES +# --- +# Firmware + +fwinst-dirs := $(addprefix _fwinst_,$(vmlinux-dirs)) + +.PHONY: $(fwinst-dirs) _fwinst_ firmware_install +$(fwinst-dirs): + $(Q)$(MAKE) $(fwinst)=$(patsubst _fwinst_%,%,$@) + +_fwinst_: + @rm -rf $(MODLIB)/firmware + @mkdir -p $(MODLIB)/firmware + +firmware_install: _fwinst_ $(fwinst-dirs) + # Generate asm-offsets.h # --- @@ -1042,6 +1057,7 @@ help: @echo '* vmlinux - Build the bare kernel' @echo '* modules - Build all modules' @echo ' modules_install - Install all modules' + @echo ' firmware_install- Install all firmware' @echo ' dir/- Build all files in dir and below' @echo ' dir/file.[ois] - Build specified target only' @echo ' rpm - Build a kernel as an RPM package' @@ -1101,6 +1117,10 @@ else # KBUILD_EXTMOD # make M=dir modules_install # Install the modules build in the module directory # Assumes install directory is already created +# make M=dir firmware_install +# Install the firmware in the firmware directory +# Assumes install directory is already created + # We are always building modules KBUILD_MODULES := 1 @@ -1129,6 +1149,13 @@ modules: $(module-dirs) modules_install: $(Q)$(MAKE) -rR -f $(srctree)/scripts/Makefile.modinst +fwinst-dirs := $(addprefix _fwinst_,$(KBUILD_EXTMOD)) +.PHONY: $(fwinst-dirs) firmware_install +$(fwinst-dirs): + $(Q)$(MAKE) $(fwinst)=$(patsubst _fwinst_%,%,$@) + +firmware_install: $(fwinst-dirs) + clean-dirs := $(addprefix _clean_,$(KBUILD_EXTMOD)) .PHONY: $(clean-dirs) clean @@ -1149,6 +1176,7 @@ help: @echo '' @echo ' modules - default target, build the module(s)' @echo ' modules_install - install the module' + @echo ' firmware_install- install firmware' @echo ' clean - remove generated files in module directory only' @echo '' endif # KBUILD_EXTMOD @@ -1346,6 +1374,11 @@ build := -f $(if $(KBUILD_SRC),$(srctree # $(Q)$(MAKE) $(clean)=dir clean := -f $(if $(KBUILD_SRC),$(srctree)/)scripts/Makefile.clean obj +# Shorthand for $(Q)$(MAKE) -f scripts/Makefile.fwinst obj=dir +# Usage: +# $(Q)$(MAKE) $(fwinst)=dir +fwinst := -f $(if $(KBUILD_SRC),$(srctree)/)scripts/Makefile.fwinst obj + # $(call descend,,) # Recursively call a sub-make in with target # Usage is deprecated, because make does not see this as an invocation of make. Index: linux/drivers/usb/serial/Makefile === --- linux.orig/drivers/usb/serial/Makefile 2005-03-10 16:01:14.0 -0500 +++ linux/drivers/usb/serial/Makefile 2005-04-05 13:01:54.0 -0400 @@ -36,3 +36,8 @@ obj-$(CONFIG_USB_SERIAL_VISOR)+= viso obj-$(CONFIG_USB_SERIAL_WHITEHEAT) += whiteheat.o obj-$(CONFIG_U
[patch 2/5] Hotplug firmware loader for Keyspan usb-serial driver
Convert the keyspan USB serial driver to use request_firmware and
firmware_load_ihex.
Signed-off-by: Jan Harkes <[EMAIL PROTECTED]>
Kconfig | 90 --
keyspan.c | 132 +++---
keyspan.h | 84 ---
3 files changed, 59 insertions(+), 247 deletions(-)
Index: linux/drivers/usb/serial/keyspan.c
===
--- linux.orig/drivers/usb/serial/keyspan.c 2005-04-05 15:08:37.0
-0400
+++ linux/drivers/usb/serial/keyspan.c 2005-04-05 15:23:47.0 -0400
@@ -28,6 +28,9 @@
Change History
+2005mar10 Jan Harkes
+ Use generic request_firmware/load_ihex functions.
+
2003sep04 LPM (Keyspan) add support for new single port product USA19HS.
Improve setup message handling for all devices.
@@ -106,6 +109,8 @@
#include
#include
#include
+#include
+#include
#include
#include
#include "usb-serial.h"
@@ -1165,13 +1170,28 @@ static void keyspan_close(struct usb_ser
port->tty = NULL;
}
+static int keyspan_fw_load(struct ihex_record *record, void *arg)
+{
+ struct usb_serial *serial = (struct usb_serial *)arg;
+ int ret;
+
+ ret = ezusb_writememory(serial, record->address, record->data,
+ record->len, 0xa0);
+ if (ret < 0) {
+ dev_err(>dev->dev,
+ "ezusb_writememory failed for Keyspan"
+ "firmware (%d %08X %p %d)\n",
+ ret, record->address, record->data, record->len);
+ }
+ return ret;
+}
/* download the firmware to a pre-renumeration device */
static int keyspan_fake_startup (struct usb_serial *serial)
{
int response;
- const struct ezusb_hex_record *record;
- char*fw_name;
+ char*model, fw_name[20];
+ const struct firmware *fw;
dbg("Keyspan startup version %04x product %04x",
le16_to_cpu(serial->dev->descriptor.bcdDevice),
@@ -1182,97 +1202,43 @@ static int keyspan_fake_startup (struct
return(1);
}
- /* Select firmware image on the basis of idProduct */
switch (le16_to_cpu(serial->dev->descriptor.idProduct)) {
- case keyspan_usa28_pre_product_id:
- record = _usa28_firmware[0];
- fw_name = "USA28";
- break;
-
- case keyspan_usa28x_pre_product_id:
- record = _usa28x_firmware[0];
- fw_name = "USA28X";
- break;
-
- case keyspan_usa28xa_pre_product_id:
- record = _usa28xa_firmware[0];
- fw_name = "USA28XA";
- break;
-
- case keyspan_usa28xb_pre_product_id:
- record = _usa28xb_firmware[0];
- fw_name = "USA28XB";
- break;
-
- case keyspan_usa19_pre_product_id:
- record = _usa19_firmware[0];
- fw_name = "USA19";
- break;
-
- case keyspan_usa19qi_pre_product_id:
- record = _usa19qi_firmware[0];
- fw_name = "USA19QI";
- break;
-
- case keyspan_mpr_pre_product_id:
- record = _mpr_firmware[0];
- fw_name = "MPR";
- break;
-
- case keyspan_usa19qw_pre_product_id:
- record = _usa19qw_firmware[0];
- fw_name = "USA19QI";
- break;
-
- case keyspan_usa18x_pre_product_id:
- record = _usa18x_firmware[0];
- fw_name = "USA18X";
- break;
-
- case keyspan_usa19w_pre_product_id:
- record = _usa19w_firmware[0];
- fw_name = "USA19W";
- break;
-
- case keyspan_usa49w_pre_product_id:
- record = _usa49w_firmware[0];
- fw_name = "USA49W";
- break;
-
- case keyspan_usa49wlc_pre_product_id:
- record = _usa49wlc_firmware[0];
- fw_name = "USA49WLC";
- break;
-
+ case keyspan_usa18x_pre_product_id: model = "usa18x"; break;
+ case keyspan_usa19_pre_product_id:model = "usa19";break;
+ case keyspan_usa19qi_pre_product_id: model = "usa19qi"; break;
+ case keyspan_mpr_pre_product_id: model = "mpr"; break;
+ case keyspan_usa19qw_pre_product_i
[patch 1/5] Hotplug firmware loader for Keyspan usb-serial driver
A simple Intel HEX format parser/loader.
Signed-off-by: Jan Harkes <[EMAIL PROTECTED]>
include/linux/ihex_parser.h | 23 +
lib/Kconfig |8 +
lib/Makefile|2
lib/ihex_parser.c | 181
4 files changed, 214 insertions(+)
Index: linux/lib/Makefile
===
--- linux.orig/lib/Makefile 2005-04-05 15:21:30.0 -0400
+++ linux/lib/Makefile 2005-04-05 15:21:45.0 -0400
@@ -34,6 +34,8 @@ obj-$(CONFIG_ZLIB_INFLATE) += zlib_infla
obj-$(CONFIG_ZLIB_DEFLATE) += zlib_deflate/
obj-$(CONFIG_REED_SOLOMON) += reed_solomon/
+obj-$(CONFIG_IHEX_PARSER) += ihex_parser.o
+
hostprogs-y:= gen_crc32table
clean-files:= crc32table.h
Index: linux/lib/ihex_parser.c
===
--- /dev/null 1970-01-01 00:00:00.0 +
+++ linux/lib/ihex_parser.c 2005-04-05 15:29:55.0 -0400
@@ -0,0 +1,181 @@
+/*
+ * Parser/loader for IHEX formatted data.
+ *
+ * Copyright (C) 2005 Jan Harkes <[EMAIL PROTECTED]>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ */
+
+#include
+#include
+#include
+
+MODULE_AUTHOR("Jan Harkes <[EMAIL PROTECTED]>");
+MODULE_DESCRIPTION("IHEX formatted data parser");
+MODULE_LICENSE("GPL");
+
+#ifndef QUIET
+#define dbg(msg, ...) printk(KERN_WARNING "%s: line %d: " msg, __FUNCTION__,
line, ## __VA_ARGS__)
+#else
+#define dbg(msg, ...) do {} while(0)
+#endif
+
+/**
+ * nibble/hex are little helpers to parse hexadecimal numbers to a byte value
+ **/
+static __u8 nibble(const __u8 n)
+{
+ if (n >= '0' && n <= '9') return n - '0';
+ else if (n >= 'A' && n <= 'F') return n - ('A' - 10);
+ else if (n >= 'a' && n <= 'f') return n - ('a' - 10);
+ return 0;
+}
+static __u8 hex(const __u8 *data, __u8 *crc)
+{
+ __u8 val = (nibble(data[0]) << 4) | nibble(data[1]);
+ *crc += val;
+ return val;
+}
+
+/**
+ * load_ihex:
+ *
+ * Description:
+ * @data is expected to point at a block of Intel HEX formatted data.
+ * @size is the size of the data block.
+ *
+ * @load will be called for each record that is found in the IHEX data.
+ *
+ * @context will be passed on to @load.
+ *
+ **/
+int load_ihex(const u8 *data, const size_t size, void *context,
+ int (*load)(struct ihex_record *record, void *context))
+{
+ struct ihex_record *record;
+ __u32 offset = 0;
+ __u8 type, crc = 0;
+ int i, j, err = 0;
+ int line = 1;
+
+ record = kmalloc(sizeof(*record), GFP_KERNEL);
+ if (!record) {
+ dbg("Allocation failed\n");
+ return -ENOMEM;
+ }
+
+ i = 0;
+next_record:
+ /* search for the start of record character */
+ while (i < size) {
+ if (data[i] == '\n') line++;
+ if (data[i++] == ':') break;
+ }
+
+ /* Minimum record length would be about 10 characters */
+ if (i + 10 > size) {
+ dbg("Can't find valid record\n");
+ err = -EINVAL;
+ goto done;
+ }
+
+ record->len = hex(data + i, );
+ i += 2;
+
+ /* now check if we have enough data to read everything */
+ if (i + 8 + (record->len * 2) > size) {
+ dbg("Not enough data to read complete record\n");
+ err = -EINVAL;
+ goto done;
+ }
+
+ record->address = hex(data + i, ) << 8; i += 2;
+ record->address |= hex(data + i, ); i += 2;
+ record->address += offset;
+ type = hex(data + i, ); i += 2;
+
+ for (j = 0; j < record->len; j++, i += 2)
+ record->data[j] = hex(data + i, );
+
+ /* check CRC */
+ (void)hex(data + i, ); i += 2;
+ if (crc != 0) {
+ dbg("CRC failure\n");
+ err = -EINVAL;
+ goto done;
+ }
+
+ /* Done reading the record */
+ switch (type) {
+ case 0:
+ /* old style EOF record? */
+ if (!record->len)
+ break;
+
+ err = load(record, context);
+ if (err < 0) {
+ dbg("Firmware load failed (err %d)\n", err);
+ break;
+ } else
+ err = 0;
+ goto next_record;
+
+ case 1: /* End-Of-File Record */
+ if (record->address || record->len) {
+ dbg("Bad EOF record (type 01) format\n");
+
[patch 0/5] Hotplug firmware loader for Keyspan usb-serial driver
Here is another stab at making the keyspan firmware easily loadable with hotplug. Differences from the previous version, - keep the IHEX parser into a separate module. - added a fw-y and fw-m install targets to kbuild which will install a driver's firmware files in /lib/modules/`uname -r`/firmware. 01 - Add lib/ihex_parser.ko. 02 - Adapt drivers/usb/serial/keyspan.ko to use request_firmware/load_ihex. 03 - Program used to dump the firmware headers to IHEX formatted files. 04 - Result of converting the firmware. 05 - Add "make install_firmware" and the related fw-$(CONFIG_FOO) stuff. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/04] Load keyspan firmware with hotplug
On Tue, Apr 05, 2005 at 04:36:31PM +0200, Marcel Holtmann wrote: > > > I agree with Dmitry on this point. The IHEX parser should not be inside > > > firmware_class.c. What about using keyspan_ihex.[ch] for it? > > > > That's what I had originally, actually called firmware_ihex.ko, since > > the IHEX format parser is not in any way keyspan specific and there are > > several usb-serial converters that seem to use the same IHEX->.h trick > > which could trivially be modified to use this loader. > > > > But the compiled parser fairly small (< 2KB) and adding it to the > > existing module didn't effectively add any size to the firmware_class > > module since things are rounded to a page boundary anyways. > > so it seems that this is usb-serial specific at the moment. Then I would I really don't see the point you are trying to argue. I said, sure it is possible to make it a separate module, that is what my initial implementation was. Why do you want to pigeon-hole it with anything but the existing firmware loading code? It is _not_ usb-serial specific, in fact once the device is initialized this isn't even needed. And the initialization as far as I can see uses little or no usb-serial code. It happens that many usb-serial devices are built around the ezusb chipset, which in turn seems to be a 8051-based microcontroller. The compilers for such microcontrollers seem to generate IHEX formatted output possibly because eprom burners generally support the format. > it up at the moment. People are also working on a replacement for the > current request_firmware(), because the needs are changing. Try to keep > it close with the usb-serial for now. What? I find the existing request_firmware fairly simple and straightforward, it has a very KISS-like quality to it, it is nice and small and even the userspace support is trivial. I only saw a mention about 'replacing' it in the current thread which mostly involved complaints but didn't actually see anyone volunteering to start working on such a replacement. If a driver wants to load 5 different chunks, just call request_firmware 5 times (i.e. drivers/bluetooth/bcm203x.c). If the data is a single binary blob, just ask for the single binary blob. In this case there seems to be some structure to the blob that I wanted to preserve, and that would either be some custom binary format that contains []... tuples, which ofcourse causes problems for our big-endian brothers, or a similar ascii format, where the IHEX is not only pretty much standardized, it is trivial to parse and even adds checksum information. The only thing that I see missing right now is a change to the makefiles to have in-tree firmware files get installed in /lib/modules/`uname -r`/firmware or some similar place. Ideally people would add a line like, fw-$(CONFIG_FOO) = foo-firmware-blob.fw And make install could drop it a place where hotplug can find it. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/04] Load keyspan firmware with hotplug
On Tue, Apr 05, 2005 at 11:22:06AM +0200, Marcel Holtmann wrote: > I agree with Dmitry on this point. The IHEX parser should not be inside > firmware_class.c. What about using keyspan_ihex.[ch] for it? That's what I had originally, actually called firmware_ihex.ko, since the IHEX format parser is not in any way keyspan specific and there are several usb-serial converters that seem to use the same IHEX->.h trick which could trivially be modified to use this loader. But the compiled parser fairly small (< 2KB) and adding it to the existing module didn't effectively add any size to the firmware_class module since things are rounded to a page boundary anyways. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 00/04] Load keyspan firmware with hotplug
On Tue, Apr 05, 2005 at 10:32:38AM +0200, Kay Sievers wrote: > On Mon, 2005-04-04 at 23:51 -0500, Dmitry Torokhov wrote: > > Firmware loader is format-agnostic, I think having IHEX parser in a separate > > file would be better... > > Why should this be in-kernel at all? Convert the firmware into a binary > blob or do it in the userspace request. Because the keyspan headers seem to have a specific sequence of operations, something like 'load these 5 bytes at offset 10, load this byte at offset 0, etc.' I don't know if there is some magic in that sequence. Without knowing what is in the firmware, it looks to me like a little bit of bootstrap code is loaded first. Jan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[patch 1/5] Hotplug firmware loader for Keyspan usb-serial driver
A simple Intel HEX format parser/loader.
Signed-off-by: Jan Harkes [EMAIL PROTECTED]
include/linux/ihex_parser.h | 23 +
lib/Kconfig |8 +
lib/Makefile|2
lib/ihex_parser.c | 181
4 files changed, 214 insertions(+)
Index: linux/lib/Makefile
===
--- linux.orig/lib/Makefile 2005-04-05 15:21:30.0 -0400
+++ linux/lib/Makefile 2005-04-05 15:21:45.0 -0400
@@ -34,6 +34,8 @@ obj-$(CONFIG_ZLIB_INFLATE) += zlib_infla
obj-$(CONFIG_ZLIB_DEFLATE) += zlib_deflate/
obj-$(CONFIG_REED_SOLOMON) += reed_solomon/
+obj-$(CONFIG_IHEX_PARSER) += ihex_parser.o
+
hostprogs-y:= gen_crc32table
clean-files:= crc32table.h
Index: linux/lib/ihex_parser.c
===
--- /dev/null 1970-01-01 00:00:00.0 +
+++ linux/lib/ihex_parser.c 2005-04-05 15:29:55.0 -0400
@@ -0,0 +1,181 @@
+/*
+ * Parser/loader for IHEX formatted data.
+ *
+ * Copyright (C) 2005 Jan Harkes [EMAIL PROTECTED]
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ */
+
+#include linux/module.h
+#include linux/slab.h
+#include linux/ihex_parser.h
+
+MODULE_AUTHOR(Jan Harkes [EMAIL PROTECTED]);
+MODULE_DESCRIPTION(IHEX formatted data parser);
+MODULE_LICENSE(GPL);
+
+#ifndef QUIET
+#define dbg(msg, ...) printk(KERN_WARNING %s: line %d: msg, __FUNCTION__,
line, ## __VA_ARGS__)
+#else
+#define dbg(msg, ...) do {} while(0)
+#endif
+
+/**
+ * nibble/hex are little helpers to parse hexadecimal numbers to a byte value
+ **/
+static __u8 nibble(const __u8 n)
+{
+ if (n = '0' n = '9') return n - '0';
+ else if (n = 'A' n = 'F') return n - ('A' - 10);
+ else if (n = 'a' n = 'f') return n - ('a' - 10);
+ return 0;
+}
+static __u8 hex(const __u8 *data, __u8 *crc)
+{
+ __u8 val = (nibble(data[0]) 4) | nibble(data[1]);
+ *crc += val;
+ return val;
+}
+
+/**
+ * load_ihex:
+ *
+ * Description:
+ * @data is expected to point at a block of Intel HEX formatted data.
+ * @size is the size of the data block.
+ *
+ * @load will be called for each record that is found in the IHEX data.
+ *
+ * @context will be passed on to @load.
+ *
+ **/
+int load_ihex(const u8 *data, const size_t size, void *context,
+ int (*load)(struct ihex_record *record, void *context))
+{
+ struct ihex_record *record;
+ __u32 offset = 0;
+ __u8 type, crc = 0;
+ int i, j, err = 0;
+ int line = 1;
+
+ record = kmalloc(sizeof(*record), GFP_KERNEL);
+ if (!record) {
+ dbg(Allocation failed\n);
+ return -ENOMEM;
+ }
+
+ i = 0;
+next_record:
+ /* search for the start of record character */
+ while (i size) {
+ if (data[i] == '\n') line++;
+ if (data[i++] == ':') break;
+ }
+
+ /* Minimum record length would be about 10 characters */
+ if (i + 10 size) {
+ dbg(Can't find valid record\n);
+ err = -EINVAL;
+ goto done;
+ }
+
+ record-len = hex(data + i, crc);
+ i += 2;
+
+ /* now check if we have enough data to read everything */
+ if (i + 8 + (record-len * 2) size) {
+ dbg(Not enough data to read complete record\n);
+ err = -EINVAL;
+ goto done;
+ }
+
+ record-address = hex(data + i, crc) 8; i += 2;
+ record-address |= hex(data + i, crc); i += 2;
+ record-address += offset;
+ type = hex(data + i, crc); i += 2;
+
+ for (j = 0; j record-len; j++, i += 2)
+ record-data[j] = hex(data + i, crc);
+
+ /* check CRC */
+ (void)hex(data + i, crc); i += 2;
+ if (crc != 0) {
+ dbg(CRC failure\n);
+ err = -EINVAL;
+ goto done;
+ }
+
+ /* Done reading the record */
+ switch (type) {
+ case 0:
+ /* old style EOF record? */
+ if (!record-len)
+ break;
+
+ err = load(record, context);
+ if (err 0) {
+ dbg(Firmware load failed (err %d)\n, err);
+ break;
+ } else
+ err = 0;
+ goto next_record;
+
+ case 1: /* End-Of-File Record */
+ if (record-address || record-len) {
+ dbg(Bad EOF record (type 01) format\n);
+ err = -EINVAL;
+ }
+ break;
+
+ case 2: /* Extended Segment Address Record (HEX86) */
+ case 4: /* Extended Linear Address Record (HEX386
