Re: [Pkg-zfsonlinux-devel] any news/reply regarding ZFS in NEW?
On 02/03/14 06:56, Turbo Fredriksson wrote: > On Mar 2, 2014, at 4:52 AM, Dimitri John Ledkov wrote: > >> > Hostile binary takeover is not allowed - that is two separate source >> > packages should not build the same binary package names, even if on >> > different architectures. > Ok, sounds reasonable when you say it like that. I'd still appreciate a link > to the policy for that. One possible example of theoretical breakage is to run the command "apt-get source libzfs1", right now it downloads the kfreebsd/zfsutils sources, but I don't know what will happen when zfs-linux is allowed into the archives. Is apt intelligent enough to pick the source corresponding to the binary package of the host arch? signature.asc Description: OpenPGP digital signature
Re: [Pkg-zfsonlinux-devel] any news/reply regarding ZFS in NEW?
On 28/02/14 17:58, John Paul Adrian Glaubitz wrote: >> > However, I will wait for a resolution from ftp-master before >> > resuming my work on the package, because there is the possibility >> > of ftp-master not allowing the upload and I don't like to waste my >> > time. > Just because your package is rejected doesn't mean you can't get it > into unstable at all. Packages are rejected all the time. It just > means the package is not *yet* fit for ACCEPT. What I'm afraid of is ftp-masters rejecting the package for license issues (CDDL-GPL). If the ftp-masters reject the package on a license issue basis this would mean that zfs-linux won't get into Debian. So I rather will wait for this before resuming my work on the current package. I think the license isn't a problem at all because zfs-dkms only ships source code (no binary distributed). And the binary utilities distributed on zfsutils don't depend on any CDDL-incompatible library/package. signature.asc Description: OpenPGP digital signature
Re: [Pkg-zfsonlinux-devel] any news/reply regarding ZFS in NEW?
On 28/02/14 17:23, John Paul Adrian Glaubitz wrote: > On 02/28/2014 04:13 PM, Turbo Fredriksson wrote: >> Is it ok/allowed to upload a new package, even though the initial one is >> still stuck in incoming? > > I suggest asking the FTP masters to mark the package as REJECT if you > want to change something again. As long the package is still stuck > in NEW (not incoming, this is where the package goes once it's > been ACCEPTED), you can always have it rejected. > > It's the cleaner solution in my opinion instead of uselessly bumping > the package revision to fix minor issues before the package isn't > even ACCEPTED. > > Adrian > I advise against this. The upload is to experimental, is OK if the package has RC bugs. Let the ftp-master team accept the package first, and once that is done we can upload a better version to unstable. In the meanwhile you can continue working on the package repository as usual. However, I will wait for a resolution from ftp-master before resuming my work on the package, because there is the possibility of ftp-master not allowing the upload and I don't like to waste my time. Regards! signature.asc Description: OpenPGP digital signature
Re: Porting valgrind to Debian/kFreeBSD
On 07/03/13 19:51, Jeff Epler wrote: > On Thu, Mar 07, 2013 at 05:49:55PM +0100, Petr Salinger wrote: Valgrind does appear to be aware of sysarch, implementing sysarch(AMD64_SET_FSBASE) in coregrind/m_syswrap/syswrap-amd64-freebsd.c. >>> >>> Aha. eglibc is testing that the syscall succeeds, which is indicated by >>> setting RAX to 0. >> >> The standard kernel in a success case sets RAX to zero >> and clears carry flag. In error case the carry flag is set >> and RAX contains error number. >>t >> It probably should be >> >> SET_STATUS_Success2((0L, tst->arch.vex.guest_RDX ); > > As you suggest, this change also works (replacement of my earlier change, not > cumulative with it): > > - SET_STATUS_Success2((ULong)*p, tst->arch.vex.guest_RDX ); > + SET_STATUS_Success2(0, tst->arch.vex.guest_RDX ); > > Jeff > Wow! You are a genius :) I confirm that it works perfectly with any of the changes proposed. Thanks! I'm not sure if this means that we can finally have valgrind packaged for Debian/kFreeBSD. The diffstat between "official" valgrind and this freebsd port is pretty big. Could something like a new "valgrind-bsd" package based on this valgrind-freebsd port be allowed on the archives? Regards! signature.asc Description: OpenPGP digital signature
Re: Porting valgrind to Debian/kFreeBSD
On 05/03/13 20:40, Petr Salinger wrote: >> The error message does not come from valgrind, but from the libc. >> >> According to [0], FreeBSD fully supports TLS only on i386 and ia64 >> which kinda >> explains why your amd64 build does not run (that page looks old >> though). Try >> disabling TLS explicitly on the valgrind build by passing >> --disable-tls to >> ./configure. > > The TLS is fully supported on both kfreebsd-amd64 and kfreebsd-i386. > > Moreover, I expect the [0] is simply outdated for plain FreeBSD. > >> [0] https://wiki.freebsd.org/ThreadLocalStorage > > In the valgrind_kfreebsd.patch patch, the path to elf_interp[] is wrong. > It should be "/lib/ld-kfreebsd-x86-64.so.1" for kfreebsd-amd64 > and "/lib/ld.so.1" for kfreebsd-i386. > > Petr > > > I fixed the paths and rebuilt it with --disable-tls Unfortunately nothing changed: the same error I tried it also with --enable-tls (just in case) and also same error :( signature.asc Description: OpenPGP digital signature
Porting valgrind to Debian/kFreeBSD
Hi!
From some time ago there is available a port of Valgrind to FreeBSD [1]
I tried to compile it on Debian/kFreeBSD, and after some patching
I was able to compile it on kfreebsd-amd64 sid (didn't tested i386).
Here are some quick instructions to replicate what I did
(valgrind_kfreebsd.patch is attached)
# wget
https://bitbucket.org/stass/valgrind-freebsd/downloads/valgrind-freebsd-3.8.0.tar.bz2
# tar xfav valgrind-freebsd-3.8.0.tar.bz2
# cd valgrind-freebsd-3.8.0
# wget
'http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD'
-qO config.guess
# wget
'http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD'
-qO config.sub
# cat /tmp/valgrind_kfreebsd.patch | patch -p1
# ./autogen.sh
# ./configure --prefix=/usr --enable-only64bit
# make
# make install [2]
However, it don't works as expected when you run it.
It fails with the error:
cannot set up thread-local storage: cannot set %fs base address for
thread-local storage
Which I have no real idea about how to fix.
See the example output when running it over this test.c
http://cs.ecs.baylor.edu/~donahoo/tools/valgrind/
debian-kfreebsd-64 test-valgrind # valgrind --tool=memcheck --leak-check=yes
--show-reachable=yes --num-callers=20 --track-fds=yes --main-stacksize=8 ./test
==21190== Memcheck, a memory error detector
==21190== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==21190== Using Valgrind-3.8.0 and LibVEX; rerun with -h for copyright info
==21190== Command: ./test
==21190==
cannot set up thread-local storage: cannot set %fs base address for
thread-local storage
==21190==
==21190== FILE DESCRIPTORS: 3 open at exit.
==21190== Open file descriptor 2: /dev/ttyp0
==21190==
==21190==
==21190== Open file descriptor 1: /dev/ttyp0
==21190==
==21190==
==21190== Open file descriptor 0: /dev/ttyp0
==21190==
==21190==
==21190==
==21190== HEAP SUMMARY:
==21190== in use at exit: 0 bytes in 0 blocks
==21190== total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==21190==
==21190== All heap blocks were freed -- no leaks are possible
==21190==
==21190== For counts of detected and suppressed errors, rerun with: -v
==21190== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
I'm sharing this with the hope that somebody can point some ideas about how to
continue.
Regards!
[1]
https://wiki.freebsd.org/Valgrind
http://valgrind.org/info/platforms.html
[2]
If want to use checkinstall check #702314
diff -ruN valgrind-freebsd-3.8.0/coregrind/launcher-freebsd.c valgrind-kfreebsd-3.8.0/coregrind/launcher-freebsd.c
--- valgrind-freebsd-3.8.0/coregrind/launcher-freebsd.c 2012-08-15 22:34:00.0 +0200
+++ valgrind-kfreebsd-3.8.0/coregrind/launcher-freebsd.c 2013-03-05 05:53:13.0 +0100
@@ -194,7 +194,7 @@
char launcher_name[PATH_MAX+1];
char* new_line;
char** new_env;
-#if __FreeBSD__ >= 7
+#if __FreeBSD__ >= 7 || defined(__FreeBSD_kernel__)
int oid[4];
vki_size_t len;
#endif
@@ -269,7 +269,7 @@
invocations of valgrind on child processes. */
memset(launcher_name, 0, PATH_MAX+1);
-#if __FreeBSD__ >= 7
+#if __FreeBSD__ >= 7 || defined(__FreeBSD_kernel__)
oid[0] = CTL_KERN;
oid[1] = KERN_PROC;
oid[2] = KERN_PROC_PATHNAME;
diff -ruN valgrind-freebsd-3.8.0/coregrind/m_debuginfo/readstabs.c valgrind-kfreebsd-3.8.0/coregrind/m_debuginfo/readstabs.c
--- valgrind-freebsd-3.8.0/coregrind/m_debuginfo/readstabs.c 2012-08-15 22:34:00.0 +0200
+++ valgrind-kfreebsd-3.8.0/coregrind/m_debuginfo/readstabs.c 2013-03-05 05:53:13.0 +0100
@@ -62,7 +62,9 @@
# endif
#elif defined(VGO_freebsd)
# include
+#if !defined(__FreeBSD_kernel__)
# include
+#endif
#else
# error "Unknown OS"
#endif
diff -ruN valgrind-freebsd-3.8.0/coregrind/m_libcsignal.c valgrind-kfreebsd-3.8.0/coregrind/m_libcsignal.c
--- valgrind-freebsd-3.8.0/coregrind/m_libcsignal.c 2012-08-15 22:34:00.0 +0200
+++ valgrind-kfreebsd-3.8.0/coregrind/m_libcsignal.c 2013-03-05 05:53:14.0 +0100
@@ -495,7 +495,7 @@
* there was no 32 bit syscall version until FreeBSD-7. So on older
* platforms we have to check.
*/
-# if __FreeBSD__ < 7
+# if defined(__FreeBSD__) && __FreeBSD__ < 7
static void sigtimedwait_zero_handler ( Int sig )
{
vg_assert(sig != VKI_SIGILL);
@@ -510,7 +510,7 @@
Int VG_(sigtimedwait_zero)( const vki_sigset_t *set,
vki_siginfo_t *info )
{
-# if __FreeBSD__ < 7
+# if defined(__FreeBSD__) && __FreeBSD__ < 7
Inti, ir;
SysRes sr;
vki_sigset_t pending, blocked, allbutone;
@@ -523,7 +523,7 @@
# endif
static const struct vki_timespec zero = { 0, 0 };
-# if __FreeBSD__ < 7
+# if defined(__FreeBSD__) && __FreeBSD__ < 7
osreldate = VG_(getosreldate)();
if (osreldate < 60)
have_sigtimedwait_zero = False;
@@ -538,7 +538,8 @@
SysRes res = VG_(do_syscall3)(__NR_sigtimedwait, (UWord)set, (UWord)info,
RUMP Kernels (was: Re: Debian GNU/kNetBSD Sources)
On 01/03/13 12:22, Martin wrote: > > I am talking about the RUMP anykernel if any of you know about it?? It > basically allows you to run the kernel in any configuration. Its basically > the IPC and hyper visor in one. So what i mean is RUMP allows one to run > kernelspace tools in userspace, meaning essentially you can create a > micro-kernel or the like from it. Didn't know about it. Looks very interesting. But it looks like you can run RUMP kernels on top of Linux or any other OS (no need for a NetBSD kernel). """ Q: On which hypervisors can a NetBSD rump kernel run? A rump kernel runs on a hypervisor called rumpuser which implements a set of high-level operations such as “create a thread” and “allocate a page of memory”. My crossbuild script hosted on GitHub can build NetBSD-based rump kernels on Linux, FreeBSD, DragonFly BSD and Solaris. To give an example, I’ve used a NetBSD-based rump kernel on a Raspberry Pi running Arch Linux. I’ve also run NetBSD-based rump kernels on Windows and a web browser’s JavaScript engine, so if you have a wild imagination, only the sky is the limit. """ https://fosdem.org/2013/interviews/2013-antii-kantee/ http://blog.netbsd.org/tnf/entry/netbsd_binary_kernel_modules_usable Here is the video from his fosdem 2013 track: https://fosdem.org/2013/schedule/event/operating_systems_anykernel/ signature.asc Description: OpenPGP digital signature
Gentoo NetBSD (was: Re: Debian GNU/kNetBSD Sources)
On 01/03/13 12:22, Martin wrote:
> Kool, how do u guys know where to look for this information lol. I spend
> hrs looking and never find a thing.
>
> I would love to see the GNU/NetBSD project rebooted at some stage but under
> GNU/kNetBSD. NetBSD has some really good tools that didn't exist in 2002
> which i think would make it easier and provide some unique qualities.
>
> I am talking about the RUMP anykernel if any of you know about it?? It
> basically allows you to run the kernel in any configuration. Its basically
> the IPC and hyper visor in one. So what i mean is RUMP allows one to run
> kernelspace tools in userspace, meaning essentially you can create a
> micro-kernel or the like from it.
>
> Obviously i am not familiar with glibc so it depends on how easy that is to
> move amongst other things.
A bit related with this:
I found there was a GSoC on 2009 that ported Gentoo's portage and its
tools to the NetBSD platform:
http://thread.gmane.org/gmane.linux.gentoo.summer-of-code/182
http://thread.gmane.org/gmane.linux.gentoo.bsd/137
https://code.google.com/p/google-summer-of-code-2009-gentoo/downloads/detail?name=Patrice_Clement.tar.gz&can=2&q=
https://dev.gentoo.org/~aballier/gnbsd/
I tried to boot the qemu image and worked without problems. The root
password is unset, you have to first select to boot in single user mode
on the boot menu, then remount rw / ("mount -o rw /"), change the
password for root and reboot in normal mode.
signature.asc
Description: OpenPGP digital signature
Re: Debian GNU/kNetBSD Sources
On 26/02/13 23:08, Axel Beckert wrote: > Hi, > > Steven Chamberlain wrote: >> On 26/02/13 08:26, Martin wrote: >>> I understand that the Debian GNU Netbsd project is inactive and has been >>> since 2002, but is the source of what was done still available? >> >> It would be nice to be able to dig this up, but it's actually kind of >> strange how few traces of it can still be found online. It seems like >> bandwidth, FTP space and source code repositories were still relatively >> scarce 10 years ago. > > IIRC the final death for Debian GNU/NetBSD and Debian GNU/FreeBSD > (without "k") was a fire in combination with no backup: > > http://web.archive.org/web/20110728174822/http://www.debian.org/ports/freebsd/bsd-libc-based > (Citing from there: "There were packages available but the public copy > is no longer available due to a server lost in fire. Due to this and > some other unfortunate circumstances, the progress on the port is > presently stalled.") > > See also http://www.debian.org/News/2002/20021122 > > Guessing from other sources on the net, it seems that the GNU/NetBSD > and Debian GNU/FreeBSD (without "k") projects had a lot in common, so > this probably also explains the lack of code and other traces on the > net for Debian GNU/NetBSD. > > Regards, Axel Here you have some more information: * Original thread: http://thread.gmane.org/gmane.comp.lib.glibc.alpha/6253 * Files linked in the thread recovered from the past: http://web.archive.org/web/20040407191631/http://people.debian.org/~rmh/knetbsd/pub/ signature.asc Description: OpenPGP digital signature
Re: Bug#595790: hostid: useless unless fixed
On 21/02/13 00:14, Michael Stone wrote: > Short version: > > My inclination is to simply better document that hostid is an interface > without clear semantics which exists for compatability with legacy > systems and should not be used in new applications. > > Longer version: > > What is the reason for wanting to use hostid? Historically this sort of > interface was most often used for software licensing and other such > applications which wanted to tie something to a particular piece of > hardware. On the proprietary hardware, the vendors would encode a serial > number which was nicely suited to that purpose. On linux, we don't have > such a hardware serial because we don't control the hardware, and we > don't have a real strong desire to facilitate software licensing > schemes. (If someone wants to cobble one together, fine, but we're not > going to do the work for it.) Intel tried to implement cpuids for this > purpose, and it flopped; we're unlikely to get further than they did. > Any scheme that relies on a cookie isn't going to provide that "tied to > the hardware" guarantee (a guarantee which is increasingly meaningless > in a VM world, anyway). And, dbus already went and reinvented that > wheel--does it make any sense to try to re-reinvent that wheel? You > still won't be able to rely on hostid having a useful value, because it > will be installation-dependent (unlike dbus, which got to define the > semantics from the ground up). You'll basically have an interface which > on some systems has a great semantic, and on others does not, so the > documentation will have to say exactly what it should say now: "use > something else". > > So should we just get rid of it? Doing so would probably break some > ancient stupid script somewhere, to save 40k. And it's within the realm > of possibility that someone, within a particular environment, is > actually managing the hostids to do something useful, so we shouldn't > break that. > Mike Stone > When you create a ZFS pool the value of the hostid is stored inside it. When the system is going to import a pool it checks if the hostid stored inside the pool matches the current host hostid. If it don't matches it refuses to import the pool. In this case you have to manually force an import, which in turns overwrites the old value for the hostid stored on the pool. This was meant (I'm guessing) when you have a big SAN/NAS cluster with many ZFS pools and many hosts accessing them at the same time. With this feature each one of the hosts can automatically import it's own pools just by looking at the hostid value without interfering on the others. http://distfiles.scode.org/tmp/zfs-handbook/zfs-hostid.html We are working on packaging ZoL (ZFS on Linux native with DKMS) [1] for Debian and we are checking how we can solve this issue. The solution that I have in mind is just create /etc/hostid on the postinst of the ZoL package [2]. But it would be much better if this could be done by coreutils (ideally at install time) Writing 4 random bytes on /etc/hostid is not perfect, but is much better than what we currently have now. I was looking closer to what FreeBSD does [3]: First they check if the file /etc/hostid exists. If it exists they set the hostid to that value. They have a sysctl variable in their kernel for setting the hostid. If this file is not available then they check if the system has smbios.system.uuid defined (in Linux this is /sys/class/dmi/id/product_uuid). If the system has this value then they just assign the hostid to that value (via sysctl). If this value is not available, then they just generate a random value and write it on /etc/hostid so it is preserved across reboots, and finally they set the value. Regards! [1] http://bugs.debian.org/686447 [2] http://lists.alioth.debian.org/pipermail/pkg-zfsonlinux-devel/2013-February/05.html [3] https://gitorious.org/freebsd/freebsd/blobs/HEAD/etc/rc.d/hostid signature.asc Description: OpenPGP digital signature
Re: About hostid and ZFS (was Re: [Pkg-zfsonlinux-devel] ZFS on Linux and native ZFS on BSD)
On 20/02/13 02:48, Carlos Alberto Lopez Perez wrote: > So this invalidates the original purpose of gethostid(). Which was meant > to be a value unique for each host, has turned into a value that is > shared by most of the hosts. > > http://bugs.debian.org/595790 > > Which then invalidates the original purpose of storing the hostid value > into the ZFS pool > http://distfiles.scode.org/tmp/zfs-handbook/zfs-hostid.htmle ^^ http://distfiles.scode.org/tmp/zfs-handbook/zfs-hostid.html signature.asc Description: OpenPGP digital signature
About hostid and ZFS (was Re: [Pkg-zfsonlinux-devel] ZFS on Linux and native ZFS on BSD)
On 17/02/13 15:19, Arno Töll wrote:
>> > There is also a question about /etc/hostid handling, do you know how
>> > is it handled in kBSD? Existing packaging work of Fedora ZoL makes
>> > hostid static, but I doubt it's desired.
> We do not define any hostid, in fact (and Debian/Linux neither ships
> /etc/hostid. In glibc, which kfreebsd uses as well, the hostid is
> determined by gethostid(2) which does some magic based on
> gethostbyname(2) when that file does not exist).
From spl/module/spl/spl-generic.c
/*
* Read the unique system identifier from the /etc/hostid file.
*
* The behavior of /usr/bin/hostid on Linux systems with the
* regular eglibc and coreutils is:
*
* 1. Generate the value if the /etc/hostid file does not exist
* or if the /etc/hostid file is less than four bytes in size.
*
* 2. If the /etc/hostid file is at least 4 bytes, then return
* the first four bytes [0..3] in native endian order.
*
* 3. Always ignore bytes [4..] if they exist in the file.
*
* Only the first four bytes are significant, even on systems that
* have a 64-bit word size.
*
* See:
*
* eglibc: sysdeps/unix/sysv/linux/gethostid.c
* coreutils: src/hostid.c
*
* Notes:
*
* The /etc/hostid file on Solaris is a text file that often reads:
*
* # DO NOT EDIT
* "0123456789"
*
* Directly copying this file to Linux results in a constant
* hostid of 4f442023 because the default comment constitutes
* the first four bytes of the file.
*
*/
The behavior of /usr/bin/hostid on Linux systems with the regular eglibc
and coreutils is:
# eglibc: sysdeps/unix/sysv/linux/gethostid.c
http://paste.debian.net/235758/
if ((exists /etc/hostid) && (sizeof(/etc/hostid) >= 4 bytes) {
hostid = read_4bytes(/etc/hostid);
else {
if ((defined(hostname) && defined(getip(hostname))) {
hostid = getip(hostname);
hostid = (hostid << 16 | hostid >> 16);
else
hostid = 0;
}
}
Therefore there is only one way to ensure the value of hostid preserves
across reboots and is to write it to /etc/hostid.
However, most of times the hostid of the machine will be 007f0101
because the IP address of $(hostname) on Debian systems is typically
127.0.1.1
## gethostip available on package syslinux
# gethostip $(hostname)
localhost.localdomain 127.0.1.1 7F000101
But gethostid() swaps the bytes
/* For the return value to be not exactly the IP address we do some
bit fiddling. */
return (int32_t) (in.s_addr << 16 | in.s_addr >> 16);
So 0x7F000101 turns into 0x007F0101
So, it turns out that most of times, hostid will return:
* 007f0101 (01017f00 in case of big-endian machines)
So this invalidates the original purpose of gethostid(). Which was meant
to be a value unique for each host, has turned into a value that is
shared by most of the hosts.
http://bugs.debian.org/595790
Which then invalidates the original purpose of storing the hostid value
into the ZFS pool
http://distfiles.scode.org/tmp/zfs-handbook/zfs-hostid.htmle
So I propose that we can do the following in the postinst of SPL:
* If /etc/hostid already exits do nothing
* If the value returned by hostid is other than 007f0101, 01017f00 or
then write it on /etc/hostid. Otherwise generate a random value
and write it on /etc/hostid.
Code: http://paste.debian.net/235773/
In FreeBSD systems, the hostid is unique for each machine. This value is
set to a random one on the first boot.
https://gitorious.org/freebsd/freebsd/blobs/HEAD/etc/rc.d/hostid
Another idea (more in line with FreeBSD) could be just generate always a
random value for /etc/hostid when this file don't exists on the system.
Perhaps this is the better approach.
if [ ! -f /etc/hostid ]; then
dd if=/dev/urandom bs=1 count=4 of=/etc/hostid 2>/dev/null
fi
Thoughts?
signature.asc
Description: OpenPGP digital signature
